frame duplication doesnt work with raw h264 input (missing timestamp generation code)

[hardiksharma]

I am trying to decode from .264 to yuv format.
I dropped slices (one by one) in h.264 file format and then decoded the .264 to yuv format. During decoding, decoder is dropping the respective frame if I drop the slice which contain whole frame. This type of scenario only comes with B frame till now where my frame size is lesser then slice size.

Command line to decode- ffmpeg -b:v 512k -r 30 -i error_$sliceno.264 -b:v 512k -x264opts sar=352:288 -r 30 -y decode_$sliceno.yuv

ffmpeg version N-34140-g246c8da, Copyright (c) 2000-2011 the FFmpeg developers
  built on Oct 28 2011 12:04:45 with gcc 4.4.5
  configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab
  libavutil    51. 22. 0 / 51. 22. 0
  libavcodec   53. 23. 0 / 53. 23. 0
  libavformat  53. 17. 0 / 53. 17. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 45. 1 /  2. 45. 1
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[h264 @ 0xa845c20] Format h264 probed with size=2048 and score=51
[h264 @ 0xa84c0a0] err{or,}_recognition separate: 1; 1
[h264 @ 0xa84c0a0] err{or,}_recognition combined: 1; 1
[h264 @ 0xa84c0a0] no picture
[h264 @ 0xa84c0a0] Possibly too many slices (16 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (17 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (18 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (19 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (20 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (24 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (33 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (36 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (40 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (44 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (75 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa845c20] max_analyze_duration 5000000 reached at 5000000
[h264 @ 0xa845c20] Estimating duration from bitrate, this may be inaccurate

Seems stream 0 codec frame rate differs from container frame rate: 60.00 (60/1) -> 30.00 (60/2)
Input #0, h264, from 'error_4942.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 152, 1/1200000: Video: h264 (High), yuv420p, 352x288, 1/60, 30 fps, 30 tbr, 1200k tbn, 60 tbc
[buffer @ 0xa9c9140] w:352 h:288 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[rawvideo @ 0xa84d880] err{or,}_recognition separate: 1; 1
[rawvideo @ 0xa84d880] err{or,}_recognition combined: 1; 1
[h264 @ 0xa84c0a0] err{or,}_recognition separate: 1; 1
[h264 @ 0xa84c0a0] err{or,}_recognition combined: 1; 1
Output #0, rawvideo, to 'out_4942.yuv':
  Metadata:
    encoder         : Lavf53.17.0
    Stream #0:0, 0, 1/90000: Video: rawvideo (I420 / 0x30323449), yuv420p, 352x288, 1/30, q=2-31, 512 kb/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0.0 -> #0.0 (h264 -> rawvideo)
Press [q] to stop, [?] for help
[h264 @ 0xa84c0a0] no picture
[h264 @ 0xa84c0a0] Possibly too many slices (16 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (17 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (18 >= 16), increase MAX_SLICES and recompile if there are artifacts

...

[h264 @ 0xa84c0a0] Possibly too many slices (203 >= 16), increase MAX_SLICES and recompile if there are artifacts
[h264 @ 0xa84c0a0] Possibly too many slices (204 >= 16), increase MAX_SLICES and recompile if there are artifacts
frame=  299 fps=  0 q=0.0 Lsize=   44402kB time=00:00:09.96 bitrate=36495.4kbits/s    

video:44402kB audio:0kB global headers:0kB muxing overhead 0.000000%

[hardiksharma]

Dropped 4942nd slice which is a B slice and only slice for that frame

[Carl Eugen Hoyos]

Please note that both "-b:v 512k -r 30" and "-b:v 512k -x264opts sar=352:288" are useless options: You cannot define a bitrate for the input sample, you cannot define a frame rate for a h264 input file, you cannot define a bitrate for yuv files and x264 options will not work for raw video output.
(-r 30 for output yuv files is ok, but redundant for your test case afaict.)
And please post command lines that actually work in this issue tracker (your command line uses variables that are not defined, please do not define them, use the file names instead).

The reference decoder only decodes 220 frames (bit-identical with FFmpeg), does your problem occur before or after? Is it possible to reproduce your problem with a smaller sample (for example three GOPs, with the error in the second)?
Is there a reason why you are intentionally producing h264 samples with high slice number without increasing MAX_SLICES in the FFmpeg source?

[hardiksharma]

Replying to cehoyos: My problem occurs after 220 frames because B frame with size lesser then my slice size (with only one slice) is coming in last gops only. It is reproducible anywhere if our frame size is lesser then slice size or one slice means one frame and if user dropped that slice to introduce error.
I tried to increase the MAX_SLICES size and compile it but it is behaving in the same way.

Please note that both "-b:v 512k -r 30" and "-b:v 512k -x264opts sar=352:288" are useless options: You cannot define a bitrate for the input sample, you cannot define a frame rate for a h264 input file, you cannot define a bitrate for yuv files and x264 options will not work for raw video output.
(-r 30 for output yuv files is ok, but redundant for your test case afaict.)
And please post command lines that actually work in this issue tracker (your command line uses variables that are not defined, please do not define them, use the file names instead).

The reference decoder only decodes 220 frames (bit-identical with FFmpeg), does your problem occur before or after? Is it possible to reproduce your problem with a smaller sample (for example three GOPs, with the error in the second)?
Is there a reason why you are intentionally producing h264 samples with high slice number without increasing MAX_SLICES in the FFmpeg source?

[Carl Eugen Hoyos]

Replying to hardiksharma:

Replying to cehoyos: My problem occurs after 220 frames because B frame with size lesser then my slice size (with only one slice) is coming in last gops only. It is reproducible anywhere if our frame size is lesser then slice size or one slice means one frame and if user dropped that slice to introduce error.

Is there a reason you cannot add more frames (with even more slices if that is really necessary) and then cut away GOPs in the beginning of the streams so you have (for example) three (or five) GOPs with the error occurring in the middle of the sample?

I tried to increase the MAX_SLICES size and compile it but it is behaving in the same way.

But don't you agree that the output looks significantly nicer?
(Although you should mention if you changed the source.)

[hardiksharma]

Replying to cehoyos:The video streams which I am getting for my experiments are raw yuv 300 frame streams. The scheme I am trying to implement in post-processing need same number of frames as input. I don't think I can get more frames for same video sequence. In my other experiment with everything same, I am not facing this error and that is even for 150 frames because there is no frame which is included in one slice. Don't you think it's a failure of frame copy?

I also agree with you that as this defect is only coming in probably last two gops only so if I can increase the number of gops by two and then delete them in the end probably I can recover from this error but I only have exact number of frame sequence for the implementation.

I am totally agree that the output will look much nicer but in this particular version, it didn't compiled properly. I don't know the exact reason but I will try as soon as we can sort out this problem.

Replying to hardiksharma:

Replying to cehoyos: My problem occurs after 220 frames because B frame with size lesser then my slice size (with only one slice) is coming in last gops only. It is reproducible anywhere if our frame size is lesser then slice size or one slice means one frame and if user dropped that slice to introduce error.

Is there a reason you cannot add more frames (with even more slices if that is really necessary) and then cut away GOPs in the beginning of the streams so you have (for example) three (or five) GOPs with the error occurring in the middle of the sample?

I tried to increase the MAX_SLICES size and compile it but it is behaving in the same way.

But don't you agree that the output looks significantly nicer?
(Although you should mention if you changed the source.)

[Michael Niedermayer]

such frame duplication would be the job of the fps filter or the application converting to fixed fps.
Both these also should work fine already for that if the input has timestamps.
For raw h264 timestamps arent always available, this is the same issue as #502

[Balling]

6fd91fa11909f27902498648680dbb3d13f1f175 increased MAX_SLICES to 32. I will just point out what a joke that macro is. Lavfilters increased it even further. ​https://git.1f0.de/gitweb/?p=ffmpeg.git;a=commit;h=550cf548b546d386a6c634351ad0c250a3e47f3b;js=1

[Balling]

Deleted

[Balling]

Okay, I checked ​​https://git.1f0.de/gitweb/?p=ffmpeg.git;a=commit;h=550cf548b546d386a6c634351ad0c250a3e47f3b;js=1 commit. Output is bitperfect with our ffmpeg with only 32 slices, while all warnings are gone. Hahah, what a joke. Please provide a sample where there is an actual difference. It is not that hard. Someone need to check hashes with reference decoder.

[Balling]

I actually found the only place where it makes a difference: apparently this file md5 of both [NV12] output of -hwaccel nvdec and -c:v h264_cuvid are bitperfect and both do not care about the patch above increasing stuff to 256. While both -hwaccel d3d11va and -hwaccel dxva2 care about this, they need 64 MAX_SLICES to be bitperfect NV12 with nvdec and cuvid. That can be also checked on this file: ​https://github.com/wang-bin/QtAV/issues/923

Very cool. But I doubt that this is somehow issue in h264, I think this is just dxva bug.

[Balling]

For this dumb file I had to recomile JM 19.0 with Visual Studio with

#define MAX_NUM_SLICES 500
in defines.h.

And... It is bitperfect, md5 E47FDCC046210054BA90953B95A62ADF!! Closing. It was fixed somewhere before 4a11a6f4ccc7c56ccc82adf0c3ab4054d4c22d1e.