-
Notifications
You must be signed in to change notification settings - Fork 100
PSA: UDP stream mpegts formatting #57
Description
In investigating some decoding issues encountered when using live preview on a HERO5, I came to realize an important point I haven't seen documented elsewhere, please forgive me if this is duplicate information. I suspect few issues posted to this repo are suffering from this behavior:
UDP payloads received from the device live preview feature are not well-formed mpegts
...at least without some manipulation of the payload. Depending on the demuxer / container parser implementation downstream, this may disrupt bitstream decoding.
For this analysis I used:
- GoPro HERO5
- 2.7K Res, 30FPS
- Triggering live preview with combination of
http://10.5.5.9/gp/gpControl/execute?p1=gpStream&a1=proto_v2&c1=restart
and UDP keepalive messages, as defined in this repo and https://github.com/KonradIT/goprowifihack (thanks for all this work, btw -- hats off to you, @KonradIT!)
Observe:
- mpegts defines a multiplexed bitstream segmented into (at most) 188-byte packets, each starting with the sync byte
0x47
- In my testing, I observe received UDP payload size of 1328 bytes per datagram/packet. I suspect this could change with different video configurations. This is not enough to indicate a problem, but in inspecting the bitstream, we see that
- The sync byte for the first TS packet(s) in every UDP payload is at a fixed offset from the first byte in the (UDP) packet.
It appears the device is prepending each UDP payload with a 12-byte header. I don't recognize the header format (doesn't seem to be RTP or SRT), but it appears there is at least a sequence number (for the UDP packet sequence, not the TS packets) and a seqno overflow counter packed in to the header. If you strip away the first 12 bytes of each received UDP packet and pass it to a TS parser, many the warnings previously emitted by ffmpeg/ffplay (eg ffplay udp://:8554
) and gstreamer about the input bitstream go away. Additionally, when decoding the AVC bitstream, there's a significant reduction in visible artifacts. I suspect those artifacts and errors are introduced when the fixed header seqno rolls over the TS sync byte magic number 0x47
; this would be enough to confuse most demuxers I have worked with.
I'll follow up if I figure out anything else that's interesting/relevant, but wanted to get this out there. I haven't figured out a good way to integrate a solution to this repository, but some form of packet preprocessing before passing to the TS demuxer might go a long way to improve the live stream experience. I would suggest to spawn a separate child to bind local UDP port 8554, strip the first 12 bytes of each received payload from the camera, and replay the result to a loopback port which your player/parser can bind to.