h264_nvenc (and hevc_nvenc) with b_ref_mode middle creates invalid video while streaming

[smallishzulu]

Hello,

If b_ref_mode set to middle and b frame number is set to any number, FFmpeg gives out Invalid DTS error for mpegts.

(If bf is 0, than there is no problem. However, it is useless.)

/opt/ffmpeg/bin/ffmpeg -loglevel verbose -re -i /home/admintv/big-buck-bunny_294_1280x720.mp4  -aspect 16:9 -s 720x576 -map 0:0  -c:v:0 h264_nvenc -temporal-aq 1 -spatial-aq 1 -aq-strength 15 -2pass 1 -preset slow  -cbr 1 -rc cbr  -profile:v main   -vb 1550k -minrate 1550k -maxrate 1550k -bufsize 1550k -muxrate 1790k    -flags +ilme+ildct -top 1  -r 25 -pix_fmt yuv420p -map 0:1 -c:a:0 aac  -b:a:0 128k -g 50 -f mpegts -bf 4 -b_ref_mode middle 'udp://192.168.2.100:5000?pkt_size=188'
ffmpeg version N-91062-gf995aa8 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --prefix=/opt/ffmpeg --enable-nonfree --enable-gpl --extra-cflags='-I/opt/ffmpeg/include -I/usr/local/include -I/usr/local/cuda/include -I/opt/ndi/sdk/include' --extra-ldflags='-L/opt/ffmpeg/lib -L/usr/local/cuda/lib64 -L/opt/ndi/sdk/lib' --bindir=/opt/ffmpeg/bin --extra-libs=-ldl --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-nvenc --enable-libzvbi --enable-libfdk-aac --enable-libzimg --enable-libzmq --enable-libfreetype --enable-static --enable-shared --enable-hardcoded-tables --enable-vdpau --enable-cuda --enable-cuvid --enable-libvpx --enable-libmp3lame --enable-libndi_newtek --enable-openssl --enable-ffnvcodec --enable-libfontconfig --enable-libfribidi --enable-cuda-sdk --enable-libnpp
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 19.101 / 58. 19.101
  libavformat    58. 13.102 / 58. 13.102
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 22.100 /  7. 22.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
[h264 @ 0x10b9740] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/admintv/big-buck-bunny_294_1280x720.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    title           : Big Buck Bunny
    encoder         : Lavf52.13.0
  Duration: 00:09:56.46, start: 0.000000, bitrate: 2353 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 2220 kb/s, 24 fps, 24 tbr, 24 tbn, 48 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[h264 @ 0x1127740] Reinit context to 1280x720, pix_fmt: yuv420p
[graph_1_in_0_1 @ 0x141d8c0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
[graph 0 input from stream 0:0 @ 0x1a3cb80] w:1280 h:720 pixfmt:yuv420p tb:1/24 fr:24/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x1a3de80] w:720 h:576 flags:'bicubic' interl:0
[scaler_out_0_0 @ 0x1a3de80] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:yuv420p sar:64/45 flags:0x4
[h264_nvenc @ 0x10e0800] Loaded Nvenc version 8.1
[h264_nvenc @ 0x10e0800] Nvenc initialized successfully
[h264_nvenc @ 0x10e0800] 1 CUDA capable devices found
[h264_nvenc @ 0x10e0800] [ GPU #0 - < GeForce GTX 1080 > has Compute SM 6.1 ]
[h264_nvenc @ 0x10e0800] supports NVENC
[h264_nvenc @ 0x10e0800] AQ enabled.
[h264_nvenc @ 0x10e0800] Temporal AQ enabled.
[mpegts @ 0x10bc9c0] muxrate 1790000, pcr every 23 pkts, sdt every 595, pat/pmt every 119 pkts
Output #0, mpegts, to 'udp://192.168.2.100:5000?pkt_size=188':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: mp41
    title           : Big Buck Bunny
    encoder         : Lavf58.13.102
    Stream #0:0(und): Video: h264 (h264_nvenc) (Main), 1 reference frame, yuv420p(top first, left), 720x576 [SAR 64:45 DAR 16:9], q=-1--1, 1550 kb/s, 25 fps, 90k tbn, 25 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
      encoder         : Lavc58.19.101 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 1550000/0/1550000 buffer size: 1550000 vbv_delay: -1
    Stream #0:1(und): Audio: aac (LC), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
      encoder         : Lavc58.19.101 aac
[mpegts @ 0x10bc9c0] Invalid DTS: 7200 PTS: 3600 in output stream 0:0, replacing by guess457x
[mpegts @ 0x10bc9c0] Invalid DTS: 10800 PTS: 7200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 25200 PTS: 21600 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 28800 PTS: 25200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 46800 PTS: 39600 in output stream 0:0, replacing by guess64x
[mpegts @ 0x10bc9c0] Invalid DTS: 50400 PTS: 46800 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 64800 PTS: 61200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 68400 PTS: 64800 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 82800 PTS: 79200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 86400 PTS: 82800 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 100800 PTS: 97200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 104400 PTS: 100800 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 118800 PTS: 115200 in output stream 0:0, replacing by guess
[mpegts @ 0x10bc9c0] Invalid DTS: 122400 PTS: 118800 in output stream 0:0, replacing by guess
frame=   36 fps= 17 q=27.0 Lsize=     444kB time=00:00:02.09 bitrate=1739.5kbits/s speed=0.979x
video:342kB audio:33kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 18.366146%
Input file #0 (/home/admintv/big-buck-bunny_294_1280x720.mp4):
  Input stream #0:0 (video): 51 packets read (539726 bytes); 37 frames decoded;
  Input stream #0:1 (audio): 98 packets read (33347 bytes); 97 frames decoded (99328 samples);
  Total: 149 packets (573073 bytes) demuxed
Output file #0 (udp://192.168.2.100:5000?pkt_size=188):
  Output stream #0:0 (video): 36 frames encoded; 36 packets muxed (349930 bytes);
  Output stream #0:1 (audio): 97 frames encoded (99328 samples); 98 packets muxed (34119 bytes);
  Total: 134 packets (384049 bytes) muxed
[h264_nvenc @ 0x10e0800] Nvenc unloaded
[aac @ 0x10e3200] Qavg: 164.648
Exiting normally, received signal 2.

[smallishzulu]

Broken Video (Invalid DTS)

[haisk]

I 've got this problem as well .

b_ref_mode does increase the compression ratio so I hope ffmpeg team can fix this problem.

[Balling]

#7746, #7302, #7561 merged in this issue. There were actually a few open issues ​https://trac.ffmpeg.org/search?q=b_ref_mode (#7301 is about "-b_ref_mode each" not currently implemented for h264_nvenc in NVIDIA's Video SDK ​https://github.com/NVIDIA/video-sdk-samples/blob/aa3544dcea2fe63122e4feb83bf805ea40e58dbe/Samples/NvCodec/NvEncoder/nvEncodeAPI.h#L614)

So, the message is generated by this ​https://github.com/FFmpeg/FFmpeg/blob/6f2a3958cfac135c60b509a61a4fd39432d8f9a9/fftools/ffmpeg.c#L768
implemented here ​https://github.com/FFmpeg/FFmpeg/commit/22844132069ebd2c0b2ac4e7b41c93c33890bfb9
As we can see the DTS - PTS is 3600. But well this is not always the case. On my machine DTS - PTS is always 3754.

[mpegts @ 000001b30fe12f00] Invalid DTS: 7508 PTS: 3754 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 11261 PTS: 7508 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 26276 PTS: 22523 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 30030 PTS: 26276 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 45045 PTS: 41291 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 48799 PTS: 45045 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 63814 PTS: 60060 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 67568 PTS: 63814 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 82583 PTS: 78829 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 86336 PTS: 82583 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 101351 PTS: 97598 in output stream 0:0, replacing by guess
[mpegts @ 000001b30fe12f00] Invalid DTS: 105105 PTS: 101351 in output stream 0:0, replacing by guess

And in issue #7561 it is always 1800. Also it is said there "nvenc ffmpeg code or nvenc library from nvidia should either not produce bad timestamps that make ffmpeg report them".
In issue #7746 it is off by 1, so...

In #7302 Nvidia tells us that "Further, using B-frames as reference requires a DPB size of at least 3 (2 slots for the reference frames on which the B-frame would depend and one slot for the B-frame itself), so in this specific case (-dpb_size 2), the DPB is not large enough for use with the B-frames as reference feature." BTW, -dpb_size was -refs before but was renamed in commit ​https://github.com/FFmpeg/FFmpeg/commit/e929b2f248a9c49761475e07d1a089e05912d1c6 not so long ago.
Also it now says "InitializeEncoder failed: invalid param (8): DPB size should be greater than or equal to 3 for B as reference." generated by ​https://github.com/FFmpeg/FFmpeg/blob/ab0ef1abdf53e257f7628f2d264adc80038f3bcb/libavcodec/nvenc.c#L1262

This issue is the worst because it (if valid) produces broken output. Which is unacceptable. But everybody else says that everything is working for them. And it is working for me.

[Zhong,Li]

Replying to smallishzulu:

Hello,

If b_ref_mode set to middle and b frame number is set to any number, FFmpeg gives out Invalid DTS error for mpegts.

Could you please upload the broken output video?

[cs9k]

Hello. Since I began streaming and recording my gameplay footage a year ago, I've eagerly watched this topic to see when it may be resolved. While it hasn't yet been resolved, I do see that you need examples of output with the error. Good news: I can provide that!

This issue exists with ffmpeg 4.2.2, the 4.2.2. nightly from last week, and the ffmpeg implementation within OBS Studio's official releases as well. Currently in OBS Studio 25.0.4, you can not record in either 264 nor 265 using any b_ref_mode.

Strangely, b_ref_mode does indeed work when streaming from OBS Studio to both twitch and youtube. I am unsure why, and can provide a copy of footage from twitch if you would like.

Thank you all so much for the work you do! Footage can be found at the OneDrive link below. Spaces added to prevent web crawlers; remove spaces from around ".ms"

​https://1drv .ms /u/s!AtkTVhsX2nFFm0P0-aPBPe0bQ06E?e=c60q4I

[Balling]

need examples of output with the error. Good news: I can provide that

We need not output with the error (as it is 100% reproducible), but the file THAT IS BROKEN after this, which is not reproducible.

[cs9k]

Ah, I see. Unfortunately, all ffmpeg-encoded files that have the DTS/PTS error still encode, and are still playable.

The only broken output that I have (encoder stall) is from OBS, using its built in ffmpeg implementation (4.2.2 if the dll version is to be believed). I keep mentioning OBS as I believe the errors there are related somehow to the errors described here.

I have added the aforementioned files to my onedrive. I admit, I am no programmer, and I thank yall again for what you've done so far! Please let me know if there's anything else I can do; I have a pc, ffmpeg, and nothing but time right now.

[Balling]

I keep mentioning OBS as I believe

There were crazy amount of bugs fixed after 4.2.2 in nvenc in particular. Just saying. Sigh. But i will look. Looked, all files play correctly. Dunno...

[Balling]

BTW. If I understand this issue, here we have a thing that if we encode and then immidiatelly stream that to udp, be have broken picture (because replacing by guess does not work (?))... Maybe I will check this with VLC some day. OBS issue. ​https://github.com/obsproject/obs-studio/issues/2374

[cs9k]

"BTW. If I understand this issue, here we have a thing that if we encode and then immidiatelly stream that to udp, be have broken picture (because replacing by guess does not work (?))"

Aye, with b_ref_mode on middle, Twitch and YT properly ingest the output from OBS, and ffmpeg's command-line output and resulting files work just fine despite throwing the dts/pts error.

I suppose, thinking out loud about it, OBS Studio inability to handle the dts/pts error and how ffmpeg resolves it, is an OBS Studio issue. If you can confirm that that is the case (assuming nothing needs to be 'fixed' with ffmpeg), I can pursue that avenue for a resolution.

[Carl Eugen Hoyos]

Is there a bug that likely can be fixed in FFmpeg or is there a bug in an external library for which FFmpeg contains a work-around but OBS is missing such a work-around?

[cs9k]

Had a lengthy discussion with OBS Studio's developers and with the S.O. about OBS Studio and code.

Balling, I appreciate the work you've done so far to modify ffmpeg's code. However, the dts/pts mismatch still exists when using b_ref_mode=middle with NVENC HEVC/265. With Nvidia 445.xx and newer windows drivers, ffmpeg loads NVENC version 10.

Is there any new information available to you that might aid in an ffmpeg change that will prevent the dts/pts error from occurring at all? Thank you so much for looking into this again!

*edit: To clarify for cheoyos, OBS Studio developers are not going to change their code based on the workaround done thus far. They will modify code accordingly once the cause of the dts/pts error is resolved.

[Carl Eugen Hoyos]

Is there a bug that likely can be fixed in FFmpeg or is there a bug in an external library for which FFmpeg contains a work-around but OBS is missing such a work-around?

[Balling]

I renamed the issue, BTW, I think it looks like that codepath is not working with udp?
I also wanna say that when streaming there is another error besids "Invalid DTS: " that is not cool.

[cs9k]

My apologies if my shotgun-delivery of information was unclear.

I have not encountered the issue streaming to the internet. Twitch happily ingests and transcodes h264_nvenc output using the "-b_ref_mode middle" argument.

Local recording and transcoding are the only places I've run into the DTS/PTS error.

The error is reproducible using Zeranoe ffmpeg 4.2.2 and the 5/1/20 nightly on Windows 10, using both the mkv and mp4 containers, regardless of whether I use software or nvdec to decode the input.

I tested the following just now with the Zeranoe 5/1/20 ffmpeg nightly, and both threw the DTS/PTS error using "-b_ref_mode middle".

ffmpeg -loglevel verbose -i input.mkv -c:a copy -c:v h264_nvenc -pix_fmt yuv420p -preset:v slow -profile:v high -rc:v cbr -b:v 9M -bf:v 3 -b_ref_mode:v middle quicktest02.mp4

ffmpeg -loglevel verbose -i input.mkv -c:a copy -c:v hevc_nvenc -pix_fmt yuv420p -preset:v slow -profile:v main -tier:v high -rc:v cbr -b:v 9M -bf:v 3 -b_ref_mode:v middle quicktest03.mp4

I can provide debug logs, or any other logs, or run more tests for you, if you'd like. Please let me know.

[Timo R.]

Fixed in aaadf0dce8fa7b3b5073089498a84e758ceb975a