Crash in h264_mp4toannexb on x86

[Carl Eugen Hoyos]

​http://thread.gmane.org/gmane.comp.video.ffmpeg.user/58404/focus=58412
The bitstream filter h264_mp4toannexb crashes on invalid data on 32bit Intel because memcpy() is called with a non-aligned pointer iiuc.

(gdb) r -i 3350_cut.mp4 -vcodec copy -vbsf h264_mp4toannexb -an -f null -
Starting program: ffmpeg_g -i 3350_cut.mp4 -vcodec copy -vbsf h264_mp4toannexb -an -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-74456-g84170d4 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (SUSE Linux)
  configuration: --cc='gcc -m32' --enable-debug=3
  libavutil      54. 30.100 / 54. 30.100
  libavcodec     56. 57.100 / 56. 57.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 33.100 /  5. 33.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
[aac @ 0x962d020] channel element 0.0 is not allocated
[h264 @ 0x962c360] AVC: nal size 1905361577
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size 1086319262
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -1286842782
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -1940703501
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -1523323908
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -2522996
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -2140930318
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size 1835705131
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -791953323
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -866066423
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size 173903557
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -1098099925
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size 805266031
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size -480804333
[h264 @ 0x962c360] no frame!
[h264 @ 0x962c360] AVC: nal size 87368954
[h264 @ 0x962c360] no frame!
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x962b260] decoding for stream 0 failed
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x962b260] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 1920x1080, 19958 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '3350_cut.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2015-08-08 22:22:54
  Duration: 00:01:00.78, start: 0.000000, bitrate: 336 kb/s
    Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), none, 1920x1080, 19958 kb/s, SAR 1:1 DAR 16:9, 24.22 fps, 24.25 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2015-08-08 22:22:54
      handler_name    : VideoHandle
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      creation_time   : 2015-08-08 22:22:54
      handler_name    : SoundHandle
Output #0, null, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf56.40.101
    Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), none, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 19958 kb/s, 24.22 fps, 24.25 tbr, 90k tbn, 90k tbc (default)
    Metadata:
      creation_time   : 2015-08-08 22:22:54
      handler_name    : VideoHandle
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
poutbuf: 0xffffc84c, sps_pps_size: 0, nal_header_size: 4, offset: 0, in: 0xffffc90c, in_size: 83886080

Program received signal SIGSEGV, Segmentation fault.
0xf7bba6ec in __memcpy_ssse3_rep () from /lib/libc.so.6
(gdb) bt
#0  0xf7bba6ec in __memcpy_ssse3_rep () from /lib/libc.so.6
#1  0x08408b9d in alloc_and_copy (in_size=83886080,
    in=0xffffc90c "\264\b\bp\212", <incomplete sequence \367>, sps_pps_size=0,
    sps_pps=0x0, poutbuf_size=0xffffc850, poutbuf=0xffffc84c)
    at libavcodec/h264_mp4toannexb_bsf.c:66
#2  h264_mp4toannexb_filter (bsfc=0x962bc80, avctx=0x962e2e0, args=0x0,
    poutbuf=0xffffc84c, poutbuf_size=0xffffc850,
    buf=0xffffc90c "\264\b\bp\212", <incomplete sequence \367>, buf_size=64,
    keyframe=1) at libavcodec/h264_mp4toannexb_bsf.c:252
#3  0x080d8f2d in write_frame (s=0x962da40, pkt=pkt@entry=0xffffc948,
    ost=ost@entry=0x962e6c0) at ffmpeg.c:691
#4  0x080e030d in do_streamcopy (ist=ist@entry=0x967f740, ost=0x962e6c0,
    pkt=pkt@entry=0xffffccd8) at ffmpeg.c:1891
#5  0x080e23dd in process_input_packet (pkt=0xffffccc8, ist=0x967f740)
    at ffmpeg.c:2407
#6  process_input (file_index=0) at ffmpeg.c:3816
#7  transcode_step () at ffmpeg.c:3904
#8  transcode () at ffmpeg.c:3957
#9  0x080c1746 in main (argc=<optimized out>, argv=<optimized out>) at ffmpeg.c:4140

This issue can be bisected like a regression but the crash actually depends on the alignment so both different versions and different compile options can make the crash disappear.

[Michael Niedermayer]

Fixed in 2bb54b82b5094fd906aa28c0443be08c95662a31