ouf-of-boundry memory access in cabac(H264)

[rakexue]

The specific h264 file decoding causes some random SIGBUS on arm Android.

CallStack shows the crash in get_cabac_inline(ARM).

Turn off UNCHECKED_BITSTREAM_READER, then find that
get_cabac_inline(plain C ) -> refill2 occasionally accesses the last two bytes just after the boundry(CABACContext.bytestream_end[0] and CABACContext.bytestream_end[1]).

[Carl Eugen Hoyos]

I tested on iOS both with and without --disable-safe-bitstream-reader but I don't see a crash. Please provide the backtrace and other information as explained on https://ffmpeg.org/bugreports.html

$ ffmpeg -i 70.ts -f null -
ffmpeg version N-72656-g7131aba Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --enable-gpl --arch=arm --target-os=darwin --enable-cross-compile --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk --cc='clang -arch armv7 -miphoneos-version-min=8.0' --cpu=cortex-a8
  libavutil      54. 26.101 / 54. 26.101
  libavcodec     56. 41.101 / 56. 41.101
  libavformat    56. 34.100 / 56. 34.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 16.101 /  5. 16.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mpegts, from '70.ts':
  Duration: 00:00:09.49, start: 507.013000, bitrate: 728 kb/s
  Program 1 
    Metadata:
      service_name    : 2.0.2Wang
      service_provider: QiYi
    Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 848x352, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x101]: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 65 kb/s
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf56.34.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 848x352, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.41.101 rawvideo
    Stream #0:1: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      encoder         : Lavc56.41.101 pcm_s16le
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
  Stream #0:1 -> #0:1 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
[null @ 0x2002200] Encoder did not produce proper pts, making some up.
[NULL @ 0x1802800] missing picture in access unit with size 8/A    
[h264 @ 0x2001400] no frame!
frame=  237 fps=111 q=0.0 Lsize=N/A time=00:00:09.49 bitrate=N/A    
video:15kB audio:1632kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

[rakexue]

Address: 00000000
Retrace CallStack: 
SIGBUS(UNKNOWN_SIGBUS): SIGBUS
         #00  pc 00076f6c  /data/data/xxxx/libffmpeg_neon.so
         #01  pc 00078428  /data/data/xxxx/libffmpeg_neon.so
         #02  pc 0007aca4  /data/data/xxxx/libffmpeg_neon.so (ff_h264_decode_mb_cabac +1d47)
         #03  pc 000d38f6  /data/data/xxxx/libffmpeg_neon.so
         #04  pc 000d6a04  /data/data/xxxx/libffmpeg_neon.so (ff_h264_execute_decode_slices +47)
         #05  pc 000a5bd6  /data/data/xxxx/libffmpeg_neon.so
         #06  pc 000a6634  /data/data/xxxx/libffmpeg_neon.so
         #07  pc 001080d2  /data/data/xxxx/libffmpeg_neon.so
         #08  pc 00011e00  /system/lib/libc.so (__thread_entry +30)
         #09  pc 000119cc  /system/lib/libc.so (pthread_create +b4)

[rakexue]

Then close

#define get_cabac_inline get_cabac_inline_arm

to use plain C

static void refill2(CABACContext *c){
    int i, x;

    x= c->low ^ (c->low-1);
    i= 7 - ff_h264_norm_shift[x>>(CABAC_BITS-1)];

    x= -CABAC_MASK;

#if CABAC_BITS == 16
        x+= (c->bytestream[0]<<9) + (c->bytestream[1]<<1);
#else
        x+= c->bytestream[0]<<1;
#endif

    c->low += x<<i;
    if (c->bytestream >= c->bytestream_end)
       av_log(NULL, AV_LOG_ERROR, "c->bytestream = %x, c->bytestream_end = %x \n", c->bytestream, c->bytestream_end);
#if !UNCHECKED_BITSTREAM_READER
    if (c->bytestream < c->bytestream_end)
#endif
        c->bytestream += CABAC_BITS/8;
}

static av_always_inline int get_cabac_inline(CABACContext *c, uint8_t * const state){
    int s = *state;
    int RangeLPS= ff_h264_lps_range[2*(c->range&0xC0) + s];
    int bit, lps_mask;

    c->range -= RangeLPS;
    lps_mask= ((c->range<<(CABAC_BITS+1)) - c->low)>>31;

    c->low -= (c->range<<(CABAC_BITS+1)) & lps_mask;
    c->range += (RangeLPS - c->range) & lps_mask;

    s^=lps_mask;
    *state= (ff_h264_mlps_state+128)[s];
    bit= s&1;

    lps_mask= ff_h264_norm_shift[c->range];
    c->range<<= lps_mask;
    c->low  <<= lps_mask;
    if(!(c->low & CABAC_MASK))
        refill2(c);
    return bit;
}
#endif

and some logs:

 if (c->bytestream >= c->bytestream_end)
       av_log(NULL, AV_LOG_ERROR, "c->bytestream = %x, c->bytestream_end = %x \n", c->bytestream, c->bytestream_end);

get the following output:

06-05 11:30:46.490: I/fromffmpeg(7229): c->bytestream = 5c0ad51f, c->bytestream_end = 5c0ad51f 
06-05 11:30:47.004: I/fromffmpeg(7229): c->bytestream = 5cdb754a, c->bytestream_end = 5cdb754a 
06-05 11:30:47.830: I/fromffmpeg(7229): c->bytestream = 5c3be7cb, c->bytestream_end = 5c3be7cb 
06-05 11:30:48.170: I/fromffmpeg(7229): c->bytestream = 5cdb7ab5, c->bytestream_end = 5cdb7ab5 
06-05 11:30:48.460: I/fromffmpeg(7229): c->bytestream = 5c0ad02d, c->bytestream_end = 5c0ad02d 
06-05 11:30:48.674: I/fromffmpeg(7229): c->bytestream = 5c420337, c->bytestream_end = 5c420337 
06-05 11:30:49.404: I/fromffmpeg(7229): c->bytestream = 5c0ad104, c->bytestream_end = 5c0ad104 
06-05 11:30:49.567: I/fromffmpeg(7229): c->bytestream = 5c3be106, c->bytestream_end = 5c3be106 
06-05 11:30:50.537: I/fromffmpeg(7229): c->bytestream = 5c420482, c->bytestream_end = 5c420482 
06-05 11:30:50.647: I/fromffmpeg(7229): c->bytestream = 5c3be35e, c->bytestream_end = 5c3be35e 
06-05 11:30:50.710: I/fromffmpeg(7229): c->bytestream = 5cdb7481, c->bytestream_end = 5cdb7481 
06-05 11:30:51.217: I/fromffmpeg(7229): c->bytestream = 5c4202a3, c->bytestream_end = 5c4202a3 
06-05 11:30:51.804: I/fromffmpeg(7229): c->bytestream = 5c0ad331, c->bytestream_end = 5c0ad331 
06-05 11:30:52.517: I/fromffmpeg(7229): c->bytestream = 5c3be272, c->bytestream_end = 5c3be272 
06-05 11:30:52.610: I/fromffmpeg(7229): c->bytestream = 5c0ad2b8, c->bytestream_end = 5c0ad2b8 
06-05 11:30:52.807: I/fromffmpeg(7229): c->bytestream = 5c420052, c->bytestream_end = 5c420052 
06-05 11:30:52.884: I/fromffmpeg(7229): c->bytestream = 5c0ad7ae, c->bytestream_end = 5c0ad7ae 
06-05 11:30:52.974: I/fromffmpeg(7229): c->bytestream = 5cdb7048, c->bytestream_end = 5cdb7048 
06-05 11:30:53.180: I/fromffmpeg(7229): c->bytestream = 5c3be3f8, c->bytestream_end = 5c3be3f8 
06-05 11:30:53.944: I/fromffmpeg(7229): c->bytestream = 5c0ad088, c->bytestream_end = 5c0ad088 
06-05 11:30:54.087: I/fromffmpeg(7229): c->bytestream = 5c0ad192, c->bytestream_end = 5c0ad192 
06-05 11:30:54.277: I/fromffmpeg(7229): c->bytestream = 5c4200b0, c->bytestream_end = 5c4200b0 
06-05 11:30:54.650: I/fromffmpeg(7229): c->bytestream = 5c3be8cf, c->bytestream_end = 5c3be8cf 
06-05 11:30:54.687: I/fromffmpeg(7229): c->bytestream = 5c42034e, c->bytestream_end = 5c42034e 
06-05 11:30:59.490: I/fromffmpeg(7229): c->bytestream = 5c42004d, c->bytestream_end = 5c42004d 
06-05 11:31:00.397: I/fromffmpeg(7229): c->bytestream = 5c3be068, c->bytestream_end = 5c3be068 
06-05 11:31:00.674: I/fromffmpeg(7229): c->bytestream = 5c3be705, c->bytestream_end = 5c3be705 
06-05 11:31:01.301: I/fromffmpeg(7229): c->bytestream = 5c0ad2fe, c->bytestream_end = 5c0ad2fe 
06-05 11:31:01.331: I/fromffmpeg(7229): c->bytestream = 5c3be76b, c->bytestream_end = 5c3be76b 
06-05 11:31:01.607: I/fromffmpeg(7229): c->bytestream = 5c3be8c2, c->bytestream_end = 5c3be8c2 

It shows that refill2 accesses c->bytestream_end[0] and c->bytestream_end[1].

[Carl Eugen Hoyos]

I am closing this ticket since the FFmpeg project never provided a libffmpeg shared library.
Please reopen if you can explain how to reproduce the issue with current unchanged sources.