mpegts malformed in specifc circumstances

[Christian Ebert]

This is tricky one, because it's very hard to reproduce and many criteria are involved.

It is a definitely a regression because it does not happen with git-2014-04-17-a8d01a7

Also I haven't found a way so far to generalize this more than below. Subtle changes to:

• filter combination (e.g. using -filter:v or omitting swf_flags)
• 1 pass encoding
• bitrate, maxrate, or bufsize setting (otherwise exactly the same commandline)

and all seems to go well. But does it?

Here's how far I got to abstracting and shortening it:

$ ffmpeg -y -v verbose -i BigBuckBunny_640x360.m4v -sn -f mpegts -filter:a aresample=osr=44100:ocl=stereo -filte
r_complex 'sws_flags=spline;scale=w=384:h=216,fps=24/1' -c:v libx264 -profile:v baseline -x264opts level=3.0:ref=1 -bufs
ize 1058348 -maxrate 1058348 -force_key_frames 'expr:gte(t,n_forced*10)' -b:v 529174 -c:a libfdk_aac -profile:a aac_he -
b:a 64000 -pass 1 test.ts
ffmpeg version N-65628-g6c36b3a Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 14 2014 15:21:41 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib
  libavutil      54.  3.100 / 54.  3.100
  libavcodec     56.  0.100 / 56.  0.100
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  0.100 /  1.  0.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'BigBuckBunny_640x360.m4v':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V M4A mp42isom
    creation_time   : 2008-06-03 05:23:28
    copyright       : © 2008 Blender Foundation | www.bigbuckbunny.org
    title           : Big Buck Bunny
    artist          : Peach Open Movie Team
    composer        : Sacha Goedegebure / Ton Roosendaal
    date            : 2008
  Duration: 00:09:56.46, start: 0.000000, bitrate: 1626 kb/s
    Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Sound Media Handler
    Stream #0:1(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 640x360, 1495 kb/s, 24 fps, 24 tbr, 2400 tbn, 4800 tbc (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Video Media Handler
[Parsed_scale_0 @ 0x7fb733d00140] w:384 h:216 flags:'spline' interl:0
[Parsed_fps_1 @ 0x7fb733d00580] fps=24/1
[graph 0 input from stream 0:1 @ 0x7fb733d008e0] w:640 h:360 pixfmt:yuv420p tb:1/2400 fr:24/1 sar:0/1 sws_param:flags=2
[Parsed_scale_0 @ 0x7fb733d00140] w:640 h:360 fmt:yuv420p sar:0/1 -> w:384 h:216 fmt:yuv420p sar:0/1 flags:0x400
[graph 1 input from stream 0:0 @ 0x7fb733d0a600] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[Parsed_aresample_0 @ 0x7fb733d0a260] ch:2 chl:stereo fmt:fltp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
[libx264 @ 0x7fb734804800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fb734804800] profile Constrained Baseline, level 3.0
[mpegts @ 0x7fb734803c00] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'test.ts':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V M4A mp42isom
    date            : 2008
    copyright       : © 2008 Blender Foundation | www.bigbuckbunny.org
    title           : Big Buck Bunny
    artist          : Peach Open Movie Team
    composer        : Sacha Goedegebure / Ton Roosendaal
    encoder         : Lavf56.1.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 384x216, q=-1--1, pass 1, 529 kb/s, 24 fps, 90k tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.0.100 libx264
    Stream #0:1(eng): Audio: aac (libfdk_aac) (HE-AAC), 44100 Hz, stereo, s16, 64 kb/s (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Sound Media Handler
      encoder         : Lavc56.0.100 libfdk_aac
Stream mapping:
  Stream #0:1 (h264) -> scale (graph 0)
  fps (graph 0) -> Stream #0:0 (libx264)
  Stream #0:0 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
No more output streams to write to, finishing.e=00:09:46.88 bitrate= 675.0kbits/s
frame=14316 fps=485 q=-1.0 Lsize=   48818kB time=00:09:56.50 bitrate= 670.4kbits/s
video:38799kB audio:4661kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 12.328111%
Input file #0 (BigBuckBunny_640x360.m4v):
  Input stream #0:0 (audio): 25688 packets read (9508842 bytes); 25688 frames decoded (26304512 samples);
  Input stream #0:1 (video): 14316 packets read (111513310 bytes); 14316 frames decoded;
  Total: 40004 packets (121022152 bytes) demuxed
Output file #0 (test.ts):
  Output stream #0:0 (video): 14316 frames encoded; 14316 packets muxed (39729896 bytes);
  Output stream #0:1 (audio): 12844 frames encoded (26304512 samples); 12847 packets muxed (4773112 bytes);
  Total: 27163 packets (44503008 bytes) muxed
[Parsed_fps_1 @ 0x7fb733d00580] 14316 frames in, 14316 frames out; 0 frames dropped, 0 frames duplicated.
[libx264 @ 0x7fb734804800] frame I:211   Avg QP:14.89  size: 22308
[libx264 @ 0x7fb734804800] frame P:14105 Avg QP:19.36  size:  2483
[libx264 @ 0x7fb734804800] mb I  I16..4: 18.5%  0.0% 81.5%
[libx264 @ 0x7fb734804800] mb P  I16..4:  2.8%  0.0%  0.0%  P16..4: 46.5%  0.0%  0.0%  0.0%  0.0%    skip:50.7%
[libx264 @ 0x7fb734804800] final ratefactor: 19.08
[libx264 @ 0x7fb734804800] coded y,uvDC,uvAC intra: 68.9% 71.3% 53.7% inter: 25.2% 18.6% 6.4%
[libx264 @ 0x7fb734804800] i16 v,h,dc,p: 33% 34% 21% 12%
[libx264 @ 0x7fb734804800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 21% 10%  7%  8%  9%  8%  8%  9%
[libx264 @ 0x7fb734804800] i8c dc,h,v,p: 50% 25% 15%  9%
[libx264 @ 0x7fb734804800] kb/s:532.84
~$ ffmpeg -y -v verbose -i BigBuckBunny_640x360.m4v -sn -f mpegts -filter:a aresample=osr=44100:ocl=stereo -filter_complex 'sws_flags=spline;scale=w=384:h=216,fps=24/1' -c:v libx264 -profile:v baseline -x264opts level=3.0:ref=1 -bufsize 1058348 -maxrate 1058348 -force_key_frames 'expr:gte(t,n_forced*10)' -b:v 529174 -c:a libfdk_aac -profile:a aac_he -b:a 64000 -pass 2 test.ts
ffmpeg version N-65628-g6c36b3a Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 14 2014 15:21:41 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib
  libavutil      54.  3.100 / 54.  3.100
  libavcodec     56.  0.100 / 56.  0.100
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  0.100 /  1.  0.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'BigBuckBunny_640x360.m4v':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V M4A mp42isom
    creation_time   : 2008-06-03 05:23:28
    copyright       : © 2008 Blender Foundation | www.bigbuckbunny.org
    title           : Big Buck Bunny
    artist          : Peach Open Movie Team
    composer        : Sacha Goedegebure / Ton Roosendaal
    date            : 2008
  Duration: 00:09:56.46, start: 0.000000, bitrate: 1626 kb/s
    Stream #0:0(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Sound Media Handler
    Stream #0:1(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 640x360, 1495 kb/s, 24 fps, 24 tbr, 2400 tbn, 4800 tbc (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Video Media Handler
[Parsed_scale_0 @ 0x7fe209600000] w:384 h:216 flags:'spline' interl:0
[Parsed_fps_1 @ 0x7fe209600440] fps=24/1
[graph 0 input from stream 0:1 @ 0x7fe2096007a0] w:640 h:360 pixfmt:yuv420p tb:1/2400 fr:24/1 sar:0/1 sws_param:flags=2
[Parsed_scale_0 @ 0x7fe209600000] w:640 h:360 fmt:yuv420p sar:0/1 -> w:384 h:216 fmt:yuv420p sar:0/1 flags:0x400
[graph 1 input from stream 0:0 @ 0x7fe20940e1c0] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[Parsed_aresample_0 @ 0x7fe20940a7c0] ch:2 chl:stereo fmt:fltp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
[libx264 @ 0x7fe20a800c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fe20a800c00] profile Constrained Baseline, level 3.0
[mpegts @ 0x7fe20a800000] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'test.ts':
  Metadata:
    major_brand     : M4V
    minor_version   : 1
    compatible_brands: M4V M4A mp42isom
    date            : 2008
    copyright       : © 2008 Blender Foundation | www.bigbuckbunny.org
    title           : Big Buck Bunny
    artist          : Peach Open Movie Team
    composer        : Sacha Goedegebure / Ton Roosendaal
    encoder         : Lavf56.1.100
    Stream #0:0: Video: h264 (libx264), yuv420p, 384x216, q=-1--1, pass 2, 529 kb/s, 24 fps, 90k tbn, 24 tbc (default)
    Metadata:
      encoder         : Lavc56.0.100 libx264
    Stream #0:1(eng): Audio: aac (libfdk_aac) (HE-AAC), 44100 Hz, stereo, s16, 64 kb/s (default)
    Metadata:
      creation_time   : 2008-06-03 05:23:28
      handler_name    : Apple Sound Media Handler
      encoder         : Lavc56.0.100 libfdk_aac
Stream mapping:
  Stream #0:1 (h264) -> scale (graph 0)
  fps (graph 0) -> Stream #0:0 (libx264)
  Stream #0:0 -> #0:1 (aac (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
No more output streams to write to, finishing.e=00:09:56.35 bitrate= 666.1kbits/s
frame=14316 fps=433 q=-1.0 Lsize=   48523kB time=00:09:56.50 bitrate= 666.4kbits/s
video:38508kB audio:4661kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 12.401180%
Input file #0 (BigBuckBunny_640x360.m4v):
  Input stream #0:0 (audio): 25688 packets read (9508842 bytes); 25688 frames decoded (26304512 samples);
  Input stream #0:1 (video): 14316 packets read (111513310 bytes); 14316 frames decoded;
  Total: 40004 packets (121022152 bytes) demuxed
Output file #0 (test.ts):
  Output stream #0:0 (video): 14316 frames encoded; 14316 packets muxed (39432182 bytes);
  Output stream #0:1 (audio): 12844 frames encoded (26304512 samples); 12847 packets muxed (4773112 bytes);
  Total: 27163 packets (44205294 bytes) muxed
[Parsed_fps_1 @ 0x7fe209600440] 14316 frames in, 14316 frames out; 0 frames dropped, 0 frames duplicated.
[libx264 @ 0x7fe20a800c00] frame I:211   Avg QP:15.41  size: 22122
[libx264 @ 0x7fe20a800c00] frame P:14105 Avg QP:19.60  size:  2465
[libx264 @ 0x7fe20a800c00] mb I  I16..4: 19.9%  0.0% 80.1%
[libx264 @ 0x7fe20a800c00] mb P  I16..4:  0.8%  0.0%  2.2%  P16..4: 29.6% 12.1%  5.6%  0.0%  0.0%    skip:49.7%
[libx264 @ 0x7fe20a800c00] coded y,uvDC,uvAC intra: 73.3% 80.8% 65.7% inter: 24.1% 27.7% 8.8%
[libx264 @ 0x7fe20a800c00] i16 v,h,dc,p: 35% 32% 11% 22%
[libx264 @ 0x7fe20a800c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 20% 12%  6%  8%  8%  8%  7%  8%
[libx264 @ 0x7fe20a800c00] i8c dc,h,v,p: 49% 24% 16% 11%
[libx264 @ 0x7fe20a800c00] kb/s:528.85
$ ffmpeg -i test.ts -f segment -c copy -map 0 -segment_time 10 -segment_list test.m3u8 test%02d.ts
ffmpeg version N-65628-g6c36b3a Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 14 2014 15:21:41 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-shared --enable-libfdk-aac --enable-libx265 --enable-libx264 --enable-libxvid --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libvidstab --extra-cflags=-I/sw/include --extra-libs=-L/sw/lib
  libavutil      54.  3.100 / 54.  3.100
  libavcodec     56.  0.100 / 56.  0.100
  libavformat    56.  1.100 / 56.  1.100
  libavdevice    56.  0.100 / 56.  0.100
  libavfilter     5.  0.100 /  5.  0.100
  libswscale      3.  0.100 /  3.  0.100
  libswresample   1.  0.100 /  1.  0.100
  libpostproc    53.  0.100 / 53.  0.100
Input #0, mpegts, from 'test.ts':
  Duration: 00:09:56.61, start: 1.400000, bitrate: 666 kb/s
  Program 1
    Metadata:
      service_name    : Big Buck Bunny
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 384x216, 24 fps, 24 tbr, 90k tbn, 48 tbc
    Stream #0:1[0x101](eng): Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 58 kb/s
[mpegts @ 0x7faa99826800] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
    Last message repeated 1 times
Output #0, segment, to 'test%02d.ts':
  Metadata:
    encoder         : Lavf56.1.100
    Stream #0:0: Video: h264 ([27][0][0][0] / 0x001B), yuv420p, 384x216, q=2-31, 24 fps, 24 tbn, 24 tbc
    Stream #0:1(eng): Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, 58 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpegts @ 0x7faa99826800] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
    Last message repeated 37 times
[mpegts @ 0x7faa99826800] H.264 bitstream malformed, no startcode found, use the h264_mp4toannexb bitstream filter (-bsf h264_mp4toannexb)
av_interleaved_write_frame(): Invalid data found when processing input
frame= 4565 fps=0.0 q=-1.0 Lsize=N/A time=00:03:10.33 bitrate=N/A
video:11145kB audio:1486kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!

Sometimes I also get:

[mpegts @ 0x7fecbd800000] H.264 bitstream error, startcode missing

while the segmentation seems to succeed, but does it?

[Christian Ebert]

Might be related to #3562 - at least as far as -filter_complex is concerned.

[Christian Ebert]

Replying to blacktrash:

Might be related to #3562 - at least as far as -filter_complex is concerned.

Nope, I can repro with -filter:v too. Don't have the time to extract the cases from a more involved script because failure or success(?) depend also on various other parameters.