Decoding mpegts h264 i-frame into jpeg occasionally gives gray or green images

[werner]

Summary of the bug: decoding mpegts h264 i-frame into jpeg occasionally gives gray or green images.
FFmpeg version:

ffmpeg version 2.2
built on Apr  1 2014 15:14:10 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
configuration: --shlibdir=/usr/lib64 --prefix=/usr --mandir=/usr/share/man --libdir=/usr/lib64 --enable-shared --disable-static --enable-debug --disable-stripping --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC -I/usr/include/gsm' --enable-gpl --enable-x11grab --enable-version3 --enable-pthreads --enable-libfaac --enable-nonfree --enable-avfilter --enable-libvpx --enable-libass --enable-libmp3lame --enable-libvorbis --enable-libxvid --enable-libx264 --enable-libgsm --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-postproc --enable-libdc1394 --enable-libfreetype --enable-avresample --enable-libtwolame --enable-librtmp
libavutil      52. 66.100 / 52. 66.100
libavcodec     55. 52.102 / 55. 52.102
libavformat    55. 33.100 / 55. 33.100
libavdevice    55. 10.100 / 55. 10.100
libavfilter     4.  2.100 /  4.  2.100
libavresample   1.  2.  0 /  1.  2.  0
libswscale      2.  5.102 /  2.  5.102
libswresample   0. 18.100 /  0. 18.100
libpostproc    52.  3.100 / 52.  3.100

How to reproduce: I am using the c libraries to create my own functions, and unfortunately I can't post the entire code here (I can show how the decoder/encoder are set up, see below). I know the code works as >90% of my videos work entirely and the rest of the videos work for most of the frames. I understand this is most likely due to corrupt i-frames or something with data not being there (the videos can be bad quality), and I even try to help it out by passing more frames to it after the i-frame. I don't necessarily expect valid images back (although if there was a way to get the decoder to do that by passing more data or something, that would be awesome if someone could help me down that path), but I do expect FFmpeg to throw some sort of error my way that I can detect it had a bad frame. I have a video clip I can upload that exhibits the behavior with this code, but it's 118MB - I don't know if I can upload that here. I will try after opening the bug or I can upload it somewhere else if necessary.

decoder:

AVCodec* decoder = avcodec_find_decoder(CODEC_ID_H264);
AVCodecContext* decoder_cc = avcodec_alloc_context3(decoder);
avcodec_open2(decoder_cc, decoder, NULL);
decoder_cc->flags2 |= CODEC_FLAG2_SHOW_ALL;
decoder_cc->skip_frame = AVDISCARD_BIDIR | AVDISCARD_NONREF;
//decoder_cc->err_recognition = AV_EF_CRCCHECK | AV_EF_BITSTREAM | AV_EF_BUFFER | AV_EF_CAREFUL; // these don't seem to help
...
// Initialized packet variables, etc
// At this point, we have parsed the mpegts stream to find the video pid and have found the sps header and it's slice data, which we then pass into decode
ret = avcodec_decode_video2(decoder_cc, picture, &got_packet, &decoder_pkt);
// all is valid, got_packet is 1, ret > 0, and all the flags on picture look good (no error flags set, key frame is 1, etc)

encoder:

AVFormatContext* output_stream = NULL;
avformat_alloc_output_context2(&output_stream, NULL, "image2", NULL);
AVCodec* encoder = avcodec_find_encoder(CODEC_ID_MJPEG);
AVStream* video_stream;
video_stream = avformat_new_stream(output_stream, encoder);
video_stream->codec->codec = encoder;
video_stream->codec->qcompress = 0.8f;
video_stream->codec->width = 200; // Note: scaling happens with SwsContext not shown here
video_stream->codec->height = 150;
video_stream->codec->time_base.num = 1;
video_stream->codec->pix_fmt = AV_PIX_FMT_YUVJ420P;
avcodec_open2(video_stream->codec, NULL, NULL);
//output_stream->pb initialized using simplified ffio_fdopen
...
// decoded picture is passed in
ret = avcodec_encode_video2(output_stream->streams[0]->codec, &encoder_pkt, picture, &got_packet);
// ret is 0 and got_packet is 1, but the image is all gray

[Carl Eugen Hoyos]

Is the problem reproducible with ffmpeg (the application)? Please test current git head before reporting problems on this bug tracker and please tell us if you feel this isn't explained on https://ffmpeg.org/bugreports.html
Please upload the sample, there is no filesize limit (but it may make sense if you cut the sample as long as your problem is reproducible).

[werner]

• No, the problem is not reproducible with ffmpeg itself (ffmpeg -skip_frame nokey -i udp://<ip:port> -f image2 -vf "select='eq(pict_type,PICT_TYPE_I)',fps=fps=1/3" -vsync vfr ~/images/thumb%04d.jpg), but I need to be able to support different output than the filesystem (storing in a memory cache with metadata) and processing via ffmpeg takes up more memory and cpu (again, I believe it's because it parses all the frames as opposed to just specific iframes).
• I thought I should upload the file here as an attachment (which the limit is marked as 2.5MB); I didn't realize there was an ftp server (which suggests a 10MB limit, which I can try to stay under). I haven't pinpointed a small chunk of the video yet that will reproduce the problem as it is streamed on a loop in our lab. I will parse the video and upload as small of a chunk as possible.

I did manage to fix my problem by saving old successful iframes and passing them along to the next iframe decode. Takes a bit more processing power, which I assume essentially emulates what ffmpeg does behind the scenes.

I still think ffmpeg should report back to the user in some manner if it can't process a frame and instead produces a gray image.

[Carl Eugen Hoyos]

Replying to werner:

I still think ffmpeg should report back to the user in some manner if it can't process a frame and instead produces a gray image.

I still wonder how FFmpeg can produce a gray frame (it rarely does here / it is a bug that we would like to fix if it does).

Note that valid H.264 streams do not necessarily contain I-frames.