Inconsistent UDP output packet size

[Doychin Dokov]

Summary of the bug:
When outputting MPEG-TS to UDP and using the pkt_size option, ffmpeg does not produce output with constant packet size - instead, the option is used as a limit of maximum packet size. When looking at the sources, it's indeed a max packet size, but the docs just say "Set the size in bytes of UDP packets" - which is also the expected behavior according to me. Other software producing MPEG-TS over UDP/TCP (e.g. vlc, multicat, etc.) also behaves like that - output is constant packet-sized.
Happens not only when transcoding, but when copying a stream as well.
How to reproduce:

% ffmpeg -i input.ts -c:v copy -c:a copy -f mpegts udp://239.1.2.3:1234?pkt_size=1316
ffmpeg version N-54435-g4943eff
built on Jul  3 2013 23:19:37 with gcc 4.7 (Debian 4.7.2-5)

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Doychin Dokov]

So, this is considered normal and just a documentation issue?

[Carl Eugen Hoyos]

I thought that you were reporting a documentation issue.
If this is not correct, please provide your failing command line together with the complete, uncut console output and explain what is wrong about the output.

[Doychin Dokov]

The problem is that when adding the pkt_size=xxxx option, provided xxxx is a multiple of 188, I expect FFMPEG to produce output with packets which are all sized of xxxx bytes. But that is not the case, even if we are merely copying an input TS without any processing.

Example:

# ffmpeg -re -i test.ts -c:v copy -c:a copy -f mpegts udp://239.101.102.103:3001?pkt_size=1316
ffmpeg version N-54565-g970c8df Copyright (c) 2000-2013 the FFmpeg developers
  built on Jul  8 2013 23:13:17 with gcc 4.7 (Debian 4.7.2-5)
  configuration: --enable-libfdk-aac --prefix=/usr --enable-libx264 --enable-gpl --enable-nonfree --enable-version3
  libavutil      52. 38.100 / 52. 38.100
  libavcodec     55. 18.102 / 55. 18.102
  libavformat    55. 11.101 / 55. 11.101
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 79.100 /  3. 79.100
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
[mpeg2video @ 0x1d55900] Invalid frame dimensions 0x0.
    Last message repeated 10 times
[mpegts @ 0x1d51840] PES packet size mismatch
Input #0, mpegts, from 'test.ts':
  Duration: 00:01:00.56, start: 29903.192933, bitrate: 3145 kb/s
  Program 8
    Metadata:
      service_name    : ?CITY TV
      service_provider: ?VIVACOM
    Stream #0:0[0x3b]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x3a]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 256 kb/s
Output #0, mpegts, to 'udp://239.101.102.103:3001?pkt_size=1316':
  Metadata:
    encoder         : Lavf55.11.101
    Stream #0:0: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 25 fps, 90k tbn, 25 tbc
    Stream #0:1: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, 256 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
PES packet size mismatch.0 size=   23802kB time=00:01:00.00 bitrate=3249.7kbits/s
frame= 1491 fps= 25 q=-1.0 Lsize=   24127kB time=00:01:00.44 bitrate=3270.1kbits/s
video:20337kB audio:1874kB subtitle:0 global headers:0kB muxing overhead 8.623568%

Sample of the output - we see random packet sizes, n x 188:

16:02:35.598098 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598118 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598129 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 752   <<<
16:02:35.598155 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598163 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598169 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598175 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598181 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 752   <<<
16:02:35.598188 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598195 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598200 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 188   <<<
16:02:35.598215 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598221 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598227 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316
16:02:35.598232 IP 10.0.3.8.34947 > 239.101.102.103.3001: UDP, length 1316

Problem is much more severe when doing some processing inside ffmpeg.

[CaseyN]

I have verified this problem in ffmpeg version git-2013-08-15-60e9b85. This causes inconsistent playout issues with some hardware decoders and MPEG-TS analyzers (specifically Sencore MRD3187B)

[rycius]

I have verified this problem too, due to this bug a lot of older hw decoders doesn't play stream, even MainConcept sw decoder crashes after few minutes.

Please fix it :)

[Carl Eugen Hoyos]

Replying to rycius:

I have verified this problem too

Can you explain how I can reproduce the problem without a proprietary stream analyzer?

[CaseyN]

You may reproduce the problem using the command line supplied in the original description.

ffmpeg -re -i test.ts -c:v copy -c:a copy -f mpegts udp://239.101.102.103:3001?pkt_size=1316

test.ts should contain valid MPEG TS content.

The problem is observed using your favorite packet trace utility (I use tcpdump but wireshark may be easier to use).

All IP UDP packets should be 1316 bytes (7 MPEG TS packets at length of 188 bytes). As noted earlier ffmpeg occasionally outputs IP UDP packets containing fewer MPEG TS packets. The example above shows two packets with length 752 bytes ( 4 MPEG TS packets) and 1 packet of length 188 bytes (1 MPEG TS packet).

[Kieran Kunhya]

Strictly speaking this is a legal stream because FFmpeg outputs a VBR transport stream. SMPTE 2022-3/4 does allow this. However, a lot of equipment will expect a CBR transport stream.

[Doychin Dokov]

This has nothing to do with the bitrate of the stream, it's just that it's not packaged correctly. If you run it via multicat you'll get the correctly expected output will all packets sized the same (e.g. 1316).

[ksotik]

This patch adds a packet size check and after that ffmpeg produce output with constant packet size (if pkt_size url option is set)

[CaseyN]

Not fixed, ffmpeg continues to produce packets of inconsistent length.

Used the following command line:
ffmpeg -i 'udp://232.70.2.11:1234?source=172.31.22.135&fifo_size=1000000&overrun_nonfatal=1' -acodec copy -vcodec copy -f mpegts udp://232.70.33.1:1234?pkt_size=1316

Built from source today.
ffmpeg version git-2014-05-23-6a4832c Copyright (c) 2000-2014 the FFmpeg developers

built on May 23 2014 11:12:03 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
configuration: --prefix=/home/caseyn/src --extra-cflags=-I/home/caseyn/src/include --extra-ldflags=-L/home/caseyn/src/lib --bindir=/usr/bin --extra-libs=-ldl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
libavutil 52. 86.100 / 52. 86.100
libavcodec 55. 63.100 / 55. 63.100
libavformat 55. 40.100 / 55. 40.100
libavdevice 55. 13.101 / 55. 13.101
libavfilter 4. 5.100 / 4. 5.100
libswscale 2. 6.100 / 2. 6.100
libswresample 0. 19.100 / 0. 19.100
libpostproc 52. 3.100 / 52. 3.100

11:41:14.306157 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 1316
11:41:14.306162 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 376
11:41:14.309895 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 1316
11:41:14.309908 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 1316
11:41:14.309916 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 564
11:41:14.317978 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 1316
11:41:14.317991 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 1316
11:41:14.317999 IP 172.31.35.47.43299 > 232.70.33.1.1234: UDP, length 188

[smallishzulu]

aviobuf.c.patch​ added by ksotik works ok and accurate.
Last night I tested the output with tcpdump. All packets where 1316 bytes when I set pkt_size=1316.

My comment is this patch can be add to source when mpegts output is set and pkt_size is given.

[Carl Eugen Hoyos]

Replying to smallishzulu:

My comment is this patch can be add to source when mpegts output is set and pkt_size is given.

Then please suggest to ksotik that he sends his patch - made with git format-patch - to the ffmpeg-devel mailing list where it can be discussed.

[Andrey Utkin]

Not a bug. The produced stream is perfectly OK and shouldn't break anything. Please provide the usage scenario which is broken because of this.

[smallishzulu]

Dear Krieger,

To make ffmpeg stream workable with commonly used decoders, output needs to be multiple of 188 bytes. MTU is 1500 is standart. So, 7x118 = 1316 bytes. If a part of ts package added to left space on MTU, decoders loose sync or you face with pixels/drop frames. Thats why best practice is to set payload with 1316 bytes with padding null. Patch does it.

Dear Cehoyos,

I will be glad if someone or himself can do it. This is not a bug. It is like a feature. It can be good for broadcasting users.

[Carl Eugen Hoyos]

​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/180292

[Michael Niedermayer]

fixed in fee982048e338efd3b8c1b2dcaf826cb7944cd81 and 6cdf409884bd2eef95bb478dfb04c1149dade155
also note that to get constant packets you need "-flush_packets 0"

[smallishzulu]

.