-c:v mpeg2video ignoring -qmin+qmax

[Warren Young]

I'm trying to get around a problem where setting a too-low -q:v with -f mpeg2video results in "rc buffer underflow" errors when setting -maxrate.

I realize that -q is a constant-quality mode and that this allows bit rate to go arbitrarily high, potentially hitting my -maxrate ceiling. The problem then is, what to do about it?

1. Increase -q, reducing quality for all files, even though they mostly don't hit the ceiling, just to account for the fact that occasionally I do get a file that does hit the ceiling, resulting in an encoding failure with these settings. This works, but results in bad quality across the board, just to cover the worst-case conditions.

2. Give up on constant-quality mode, letting Q vary between 2 and 31 per the encoder's defaults. This works, but unless I use 2-pass encoding, it gives worse quality than an optimally-chosen -q value.

3. Try to use -qmin and -qmax together to achieve both desires: -qmin set to my prior fixed -q value, plus -qmax set to the largest value I ever see a need for in testing, which is well under the default, 31.

The problem when I take route 3 is that ffmpeg appears to ignore it. This despite the fact that I'm just doing what ffmpeg itself suggests: "max bitrate possibly too small or try trellis with large lmax or increase qmax". Currently, giving -q + -qmax + -maxrate doesn't change anything.

Reproduction command line:

ffmpeg -i in.mp4 -f mpegts -c:v mpeg2video -maxrate 18M -bufsize 9M -qmin:v 8 -qmax:v 20 ... -y out.ts

The 18M maxrate value is from the ATSC/QAM limitations for broadcast video, which is why I'm using MPEG-TS as a container here. I have to get in under this limit or I'll overrun my hardware MPEG-2 decoders' buffers.

(The hard limit from the spec is more like 19.4 Mbit/sec, but I also need space for audio + mux overhead. Pathological cases aside, 18 should be plenty, even for 1080p@29.97 MPEG-2 video.)

You don't need any particular input file to show the problem. Although only certain inputs will give the "rc buffer overflow" symptom, you can see with any input that the output isn't constrained as the command line specifies. With ffmpeg 4.4 (current Homebrew build), I see output lines like this:

frame=  360 fps=132 q=31.0 Lsize=    4332kB time=00:00:11.97 bitrate=2962.5kbits/s speed= 4.4x    

It mostly shows q=31 with occasional flickers to q=24.8. Why those two values, and how can it get to them anyway when they're both above qmax?

Why doesn't it ever go down to 8, even though in the line above, it shows that it's achieved 3 Mbps, well under my 18 Mbps max?

I realize that you can't combine a constant-quality mode in ffmpeg with 2-pass encoding, but with the above combination of 3 bitrate control options (qmin+qmax+maxrate) I'd expect it to try qmin, then if that blew the maxrate budget, try qmax, and error out if that also fails. If encoding at qmax instead succeeds, try a binary search to find the highest q value that does fit inside the maxrate limit.

I'm expecting it to repeat this algorithm over each "chunk" of video, presumably set based on my -bufsize value, though possibly based on other limits, such as GOP size.

There would have to be a maximum number of iterations, but it should be able to find a near-optimal Q value for each chunk of video with a small number of tries:

• 2 steps: just try qmax if qmin fails, oscillating between the two as necessary
• 3 steps: try qmin+(qmax-qmin)/2 if qmin fails but qmax succeeds
• 4-5 steps: also try the quarter steps; only 4 steps if qmin+(qmax-qmin)/4 succeeds, but 5 if that fails, since you then need to try the ¾ step: qmin+(3*((qmax-qmin)/4))
• 9 steps: same algorithm as above, but for eighth-interval steps

For easy-to-encode video, Q should ride along at qmin, giving the effect of a constant-quality mode (-q=qmin) but by giving -qmax, I'm allowing the encoder to retry at lower qualities to get the video under my -maxrate limit.

I've seen this behavior on versions from 4.2.1 to 4.4. I've chosen the lowest value available in your Version drop-down.

[Carl Eugen Hoyos]

I don't think this is a valid ticket (you can either set a bitrate or use a constant quantizer) but feel free to reopen this ticket if you can test current FFmpeg git head, the only version supported here, and if you can provide the command line you tested together with the complete, uncut console output and an input sample if the issue is not reproducible with testsrc2.

[Warren Young]

you can either set a bitrate or use a constant quantizer

I've got bitrate graphs that say different. Not only does combining -q/qmin/qmax with -maxrate clip the bit rate as requested — to a point — ffmpeg doesn't refuse to encode when you combine these options.

(These graphs were produced by ​an R script I wrote that wraps ffprobe.)

complete, uncut console output

Here it is, boiled down to the smallest command I can come up with that shows the symptom:

$ ffmpeg -i 5233t.mp4 -c:v mpeg2video -c:a copy -qmax 10 -maxrate 18M -y 5233t.ts
ffmpeg version N-102053-g2a623bacc8 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 8 (GCC)
  configuration: --enable-libvpx --enable-libx264 --enable-gpl
  libavutil      56. 74.100 / 56. 74.100
  libavcodec     58.137.100 / 58.137.100
  libavformat    58. 79.100 / 58. 79.100
  libavdevice    58. 14.100 / 58. 14.100
  libavfilter     7.111.100 /  7.111.100
  libswscale      5. 10.100 /  5. 10.100
  libswresample   3. 10.100 /  3. 10.100
  libpostproc    55. 10.100 / 55. 10.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '5233t.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2021-04-27T01:07:37.000000Z
  Duration: 00:00:15.30, start: 0.000000, bitrate: 19516 kb/s
  Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 17072 kb/s, 28.38 fps, 29.97 tbr, 30k tbn, 60k tbc (default)
    Metadata:
      creation_time   : 2021-04-27T01:07:37.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      creation_time   : 2021-04-27T01:07:37.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[mpeg2video @ 0x2efbdc0] Automatically choosing VBV buffer size of 268 kbyte
Output #0, mpegts, to '5233t.ts':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    encoder         : Lavf58.79.100
  Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 29.97 fps, 90k tbn (default)
    Metadata:
      creation_time   : 2021-04-27T01:07:37.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.137.100 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 18000000/0/200000 buffer size: 2195456 vbv_delay: N/A
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      creation_time   : 2021-04-27T01:07:37.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
[mpeg2video @ 0x2efbdc0] rc buffer underflowime=00:00:05.14 bitrate=13052.9kbits/s speed=4.58x    
[mpeg2video @ 0x2efbdc0] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x2efbdc0] rc buffer underflowime=00:00:06.99 bitrate=14685.7kbits/s speed=4.28x    
[mpeg2video @ 0x2efbdc0] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x2efbdc0] rc buffer underflow
[mpeg2video @ 0x2efbdc0] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x2efbdc0] rc buffer underflow
[mpeg2video @ 0x2efbdc0] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x2efbdc0] rc buffer underflowime=00:00:08.66 bitrate=15496.2kbits/s speed=4.05x    
[mpeg2video @ 0x2efbdc0] max bitrate possibly too small or try trellis with large lmax or increase qmax
frame=  459 fps=127 q=31.0 Lsize=   23842kB time=00:00:15.31 bitrate=12751.1kbits/s speed=4.24x    
video:22621kB audio:593kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.702119%

Points of interest:

1. It is showing q=31.0 despite -qmax 10. If you add -vstats to the command, you will find that the q= value ramps quickly up to 31.0 and stays there most of the time, flickering down to 24.8, as reported above.

2. It is not the case that -qmax is being ignored, however. This may simply be a display bug, because changing -qmax to 11 in the above command allows it to succeed.

an input sample if the issue is not reproducible with testsrc2.

That's a bad test source for this sort of problem, since testsrc2 generates a nearly-CBR stream. You need something with some bitrate variability to give -maxrate something to clip.

Since I couldn't find an official ffmpeg test source with such variability, I've snipped a 15-second section from the particular video I ran into this with, ​here. (This is the 5233t.mp4 file referenced in the quoted command output above.)

I still believe you could have experienced the same problem with a large number of other videos. All you need is something sufficiently VBR, then set the bitrate control parameters tight enough that the symptom occurs.

If you point me at a suitably-VBR file you trust for testing such things, I'm sure I can make ffmpeg squeal with it, too.

[IvanDSM]

I'm having this issue with both FFmpeg 7.1.1-5 from the Arch Linux package repository and the latest BtbN auto build as of writing (2025-09-04 14:22, ffmpeg-N-120970-g9ee7796c54-linux64-gpl.tar.xz from ​https://github.com/BtbN/FFmpeg-Builds/releases/tag/autobuild-2025-09-04-14-22).

Here's my command line:

ffmpeg -threads 4 -i "Original/Petscop 11 [nXHUw2nw6aA].webm" -i "Original/Petscop 11 [nXHUw2nw6aA].en.idx" -vf scale=w=720:h=480:flags=lanczos+accurate_rnd+full_chroma_int,setsar=1,setdar=4/3,colorspace=iall=bt709:all=smpte170m:range=tv:format=yuv420p:dither=fsb -c:v mpeg2video -dc 10 -g 18 -trellis 2 -cmp rd -subcmp rd -mbcmp rd -precmp rd -mbd rd -b_strategy 2 -bidir_refine 4 -b:v 1024k -maxrate:v 1575k -bufsize:v 921.6k -qsquish 1 -qmin 4 -dct faan -dia_size 4096 -mpv_flags +mv0 -c:a ac3 -dsur_mode 0 -original 1 -dsurex_mode 0 -dheadphone_mode 0 -b:a 160k -c:s dvdsub -pass 1 Encoded/pet11_1stpass.mkv

And the relevant output line:

  Stream #0:0: Video: mpeg2video (Main) (mpg2 / 0x3267706D), yuv420p(tv, smpte170m, progressive), 720x480 [SAR 8:9 DAR 4:3], q=2-31, 1024 kb/s, 30 fps, 1k tbn (default)

Changing -qmin 4's position has no effect. Different -qmin values have no effect. -qmax also has no effect.

Please let me know of anything I can do to help with this issue. I'm willing to test anything, and can do so relatively quickly.

[IvanDSM]

To aid in debugging this, I've come up with a minimal reproducible example using testsrc2, with a much simpler set of command line arguments than the command line in my last comment.

I ran the following command with the latest auto build (2025-09-04 14:22, ffmpeg-N-120970-g9ee7796c54-linux64-gpl.tar.xz from ​​https://github.com/BtbN/FFmpeg-Builds/releases/tag/autobuild-2025-09-04-14-22):

ffmpeg -f lavfi -i testsrc2=duration=120:size=720x480 -c:v mpeg2video -b:v 1024k -qmin 4 -qmax 5 -an test.mkv

Which resulted in the following output:

ffmpeg version N-120970-g9ee7796c54-20250904 Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 15.1.0 (crosstool-NG 1.27.0.79_8f49ec5)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-zlib --enable-libxml2 --enable-libsoxr --enable-openssl --enable-libvmaf --enable-fontconfig --enable-libharfbuzz --enable-libfreetype --enable-libfribidi --enable-vulkan --enable-libshaderc --enable-libvorbis --enable-libxcb --enable-xlib --enable-libpulse --enable-opencl --enable-gmp --enable-lzma --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --enable-libdvdread --enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-libplacebo --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-libzmq --enable-lv2 --enable-libvpl --enable-openal --enable-liboapv --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-libvvenc --enable-whisper --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-libs='-lgomp -ldl' --extra-ldflags=-pthread --extra-ldexeflags=-pie --cc=x86_64-ffbuild-linux-gnu-gcc --cxx=x86_64-ffbuild-linux-gnu-g++ --ar=x86_64-ffbuild-linux-gnu-gcc-ar --ranlib=x86_64-ffbuild-linux-gnu-gcc-ranlib --nm=x86_64-ffbuild-linux-gnu-gcc-nm --extra-version=20250904
  libavutil      60. 12.100 / 60. 12.100
  libavcodec     62. 15.100 / 62. 15.100
  libavformat    62.  4.102 / 62.  4.102
  libavdevice    62.  2.100 / 62.  2.100
  libavfilter    11.  8.100 / 11.  8.100
  libswscale      9.  3.100 /  9.  3.100
  libswresample   6.  2.100 /  6.  2.100
Input #0, lavfi, from 'testsrc2=duration=120:size=720x480':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, yuv420p, 720x480 [SAR 1:1 DAR 3:2], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
Output #0, matroska, to 'test.mkv':
  Metadata:
    encoder         : Lavf62.4.102
  Stream #0:0: Video: mpeg2video (Main) (mpg2 / 0x3267706D), yuv420p(tv, progressive), 720x480 [SAR 1:1 DAR 3:2], q=2-31, 1024 kb/s, 25 fps, 1k tbn
    Metadata:
      encoder         : Lavc62.15.100 mpeg2video
    Side data:
      CPB properties: bitrate max/min/avg: 0/0/1024000 buffer size: 0 vbv_delay: N/A
[out#0/matroska @ 0x556795e64c00] video:41508KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.073374%
frame= 3000 fps=1128 q=31.0 Lsize=   41538KiB time=00:01:59.96 bitrate=2836.6kbits/s speed=45.1x elapsed=0:00:02.65    

Do note the use of q=31.0 in the last line as well as the q=2-31 range in the stream info line.

Adding -loglevel debug shows that the options are supposedly being correctly parsed:

Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-f' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'lavfi'.
Reading option '-i' ... matched as input url with argument 'testsrc2=duration=120:size=720x480'.
Reading option '-c:v' ... matched as option 'c' (select encoder/decoder ('copy' to copy stream without reencoding)) with argument 'mpeg2video'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '1024k'.
Reading option '-qmin' ... matched as AVOption 'qmin' with argument '4'.
Reading option '-qmax' ... matched as AVOption 'qmax' with argument '5'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option 'test.mkv' ... matched as output url.
Finished splitting the commandline.

Using either -qmin and -qmax -qmin:v and -qmax:v makes no difference. Changing the order of the arguments makes no difference either.

EDIT: Further simplified the example by removing the unnecessary -maxrate and -bufsize arguments, whose presence does not impact the bug.

EDIT 2: Discarded the public domain Alice in Wonderland example in favor of a testsrc2 example as requested earlier in the thread. My apologies.