NTSC IMX encoding in MXF not possible

[Thomas Mundt]

Summary of the bug:
The NTSC IMX 50 encoding in MXF seems to be broken in current git master. I get: frame size does not match index unit size, 208542 != 208541
It works with version 4.0.2

How to reproduce:
ffmpeg -f lavfi -i testsrc=size=720x486:rate=60000/1001 -f lavfi -i anullsrc=r=48000:n=1920:cl=mono -vf pad=720:512:0:oh-ih,interlace=tff -r 30000/1001 -c:v mpeg2video -pix_fmt yuv422p -top 1 -flags +ildct+ilme+low_delay -dc 10 -intra_vlc 1 -non_linear_quant 1 -ps 1 -g 1 -qscale:v 1 -qmin 1 -qmax 12 -rc_min_vbv_use 1 -rc_max_vbv_use 1 -b:v 50000000 -minrate 50000000 -maxrate 50000000 -bufsize 1668334 -rc_init_occupancy 1668334 -ac 8 -acodec pcm_s16le -ar 48000 -map 0:v:0 -map 1:a:0 -t 1 -f mxf_D10 out.mxf

ffmpeg version N-94528-gfaa9cd312f Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.1.1 (GCC) 20190807
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 33.100 / 56. 33.100
  libavcodec     58. 55.100 / 58. 55.100
  libavformat    58. 30.100 / 58. 30.100
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.100 /  7. 58.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'lavfi'.
Reading option '-i' ... matched as input url with argument 'testsrc=size=720x486:rate=60000/1001'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'lavfi'.
Reading option '-i' ... matched as input url with argument 'anullsrc=r=48000:n=1920:cl=mono'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'pad=720:512:0:oh-ih,interlace=tff'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '30000/1001'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'mpeg2video'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv422p'.
Reading option '-top' ... matched as option 'top' (top=1/bottom=0/auto=-1 field first) with argument '1'.
Reading option '-flags' ... matched as AVOption 'flags' with argument '+ildct+ilme+low_delay'.
Reading option '-dc' ... matched as AVOption 'dc' with argument '10'.
Reading option '-intra_vlc' ... matched as AVOption 'intra_vlc' with argument '1'.
Reading option '-non_linear_quant' ... matched as AVOption 'non_linear_quant' with argument '1'.
Reading option '-ps' ... matched as AVOption 'ps' with argument '1'.
Reading option '-g' ... matched as AVOption 'g' with argument '1'.
Reading option '-qscale:v' ... matched as option 'qscale' (use fixed quality scale (VBR)) with argument '1'.
Reading option '-qmin' ... matched as AVOption 'qmin' with argument '1'.
Reading option '-qmax' ... matched as AVOption 'qmax' with argument '12'.
Reading option '-rc_min_vbv_use' ... matched as AVOption 'rc_min_vbv_use' with argument '1'.
Reading option '-rc_max_vbv_use' ... matched as AVOption 'rc_max_vbv_use' with argument '1'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '50000000'.
Reading option '-minrate' ... matched as AVOption 'minrate' with argument '50000000'.
Reading option '-maxrate' ... matched as AVOption 'maxrate' with argument '50000000'.
Reading option '-bufsize' ... matched as AVOption 'bufsize' with argument '1668334'.
Reading option '-rc_init_occupancy' ... matched as AVOption 'rc_init_occupancy' with argument '1668334'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '8'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'pcm_s16le'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '48000'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:v:0'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '1:a:0'.
Reading option '-t' ... matched as option 't' (record or transcode "duration" seconds of audio/video) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mxf_D10'.
Reading option 'out.mxf' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option loglevel (set logging level) with argument 99.
Successfully parsed a group of options.
Parsing a group of options: input url testsrc=size=720x486:rate=60000/1001.
Applying option f (force format) with argument lavfi.
Successfully parsed a group of options.
Opening an input file: testsrc=size=720x486:rate=60000/1001.
detected 4 logical cores
[Parsed_testsrc_0 @ 000001f5baebeec0] Setting 'size' to value '720x486'
[Parsed_testsrc_0 @ 000001f5baebeec0] Setting 'rate' to value '60000/1001'
[Parsed_testsrc_0 @ 000001f5baebeec0] size:720x486 rate:60000/1001 duration:-1.000000 sar:1/1
[AVFilterGraph @ 000001f5baebdc80] query_formats: 2 queried, 1 merged, 0 already done, 0 delayed
[lavfi @ 000001f5baebd140] All info found
[lavfi @ 000001f5baebd140] stream 0: start_time: 0.000 duration: -153876590148193856.000
[lavfi @ 000001f5baebd140] format: start_time: 0.000 duration: -9223372036854.775 bitrate=0 kb/s
Input #0, lavfi, from 'testsrc=size=720x486:rate=60000/1001':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0, 1, 1001/60000: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24, 720x486 [SAR 1:1 DAR 40:27], 0/1, 59.94 tbr, 59.94 tbn, 59.94 tbc
Successfully opened the file.
Parsing a group of options: input url anullsrc=r=48000:n=1920:cl=mono.
Applying option f (force format) with argument lavfi.
Successfully parsed a group of options.
Opening an input file: anullsrc=r=48000:n=1920:cl=mono.
[Parsed_anullsrc_0 @ 000001f5baec1640] Setting 'r' to value '48000'
[Parsed_anullsrc_0 @ 000001f5baec1640] Setting 'n' to value '1920'
[Parsed_anullsrc_0 @ 000001f5baec1640] Setting 'cl' to value 'mono'
[AVFilterGraph @ 000001f5baec42c0] query_formats: 2 queried, 3 merged, 0 already done, 0 delayed
[Parsed_anullsrc_0 @ 000001f5baec1640] sample_rate:48000 channel_layout:'mono' nb_samples:1920
[lavfi @ 000001f5baec7340] All info found
[lavfi @ 000001f5baec7340] stream 0: start_time: 0.000 duration: -192153584101141.156
[lavfi @ 000001f5baec7340] format: start_time: 0.000 duration: -9223372036854.775 bitrate=384 kb/s
Input #1, lavfi, from 'anullsrc=r=48000:n=1920:cl=mono':
  Duration: N/A, start: 0.000000, bitrate: 384 kb/s
    Stream #1:0, 1, 1/48000: Audio: pcm_u8, 48000 Hz, mono, u8, 384 kb/s
Successfully opened the file.
Parsing a group of options: output url out.mxf.
Applying option vf (set video filters) with argument pad=720:512:0:oh-ih,interlace=tff.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 30000/1001.
Applying option c:v (codec name) with argument mpeg2video.
Applying option pix_fmt (set pixel format) with argument yuv422p.
Applying option top (top=1/bottom=0/auto=-1 field first) with argument 1.
Applying option qscale:v (use fixed quality scale (VBR)) with argument 1.
Applying option b:v (video bitrate (please use -b:v)) with argument 50000000.
Applying option ac (set number of audio channels) with argument 8.
Applying option acodec (force audio codec ('copy' to copy stream)) with argument pcm_s16le.
Applying option ar (set audio sampling rate (in Hz)) with argument 48000.
Applying option map (set input stream mapping) with argument 0:v:0.
Applying option map (set input stream mapping) with argument 1:a:0.
Applying option t (record or transcode "duration" seconds of audio/video) with argument 1.
Applying option f (force format) with argument mxf_D10.
Successfully parsed a group of options.
Opening an output file: out.mxf.
[file @ 000001f5baecd1c0] Setting default whitelist 'file,crypto'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))
  Stream #1:0 -> #0:1 (pcm_u8 (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[rawvideo @ 000001f5baec3480] PACKET SIZE: 1049760, STRIDE: 2160
[Parsed_pad_0 @ 000001f5baec1540] Setting 'width' to value '720'
[Parsed_pad_0 @ 000001f5baec1540] Setting 'height' to value '512'
[Parsed_pad_0 @ 000001f5baec1540] Setting 'x' to value '0'
[Parsed_pad_0 @ 000001f5baec1540] Setting 'y' to value 'oh-ih'
[Parsed_interlace_1 @ 000001f5baec0b40] Setting 'scan' to value 'tff'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'video_size' to value '720x486'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'pix_fmt' to value '2'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'time_base' to value '1001/60000'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 000001f5baec1740] Setting 'frame_rate' to value '60000/1001'
[graph 0 input from stream 0:0 @ 000001f5baec1740] w:720 h:486 pixfmt:rgb24 tb:1001/60000 fr:60000/1001 sar:1/1 sws_param:flags=2
[format @ 000001f5baec0d40] Setting 'pix_fmts' to value 'yuv422p'
[auto_scaler_0 @ 000001f5baec0e40] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 000001f5baec0e40] w:iw h:ih flags:'bicubic' interl:0
[Parsed_interlace_1 @ 000001f5baec0b40] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_pad_0' and the filter 'Parsed_interlace_1'
[AVFilterGraph @ 000001f5baec4400] query_formats: 6 queried, 4 merged, 1 already done, 0 delayed
[Parsed_pad_0 @ 000001f5baec1540] w:720 h:486 -> w:720 h:512 x:0 y:26 color:0x000000FF
[auto_scaler_0 @ 000001f5baec0e40] w:720 h:512 fmt:rgb24 sar:1/1 -> w:720 h:512 fmt:yuv422p sar:1/1 flags:0x4
[Parsed_interlace_1 @ 000001f5baec0b40] mode:4 filter:linear h:512 -> h:512
[Parsed_pad_0 @ 000001f5baec1540] Direct padding impossible allocating new frame
[mpeg2video @ 000001f5baed5780] intra_quant_bias = 96 inter_quant_bias = 0
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
cur_dts is invalid st:1 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[graph_1_in_1_0 @ 000001f5baec0f40] Setting 'time_base' to value '1/48000'
[graph_1_in_1_0 @ 000001f5baec0f40] Setting 'sample_rate' to value '48000'
[graph_1_in_1_0 @ 000001f5baec0f40] Setting 'sample_fmt' to value 'u8'
[graph_1_in_1_0 @ 000001f5baec0f40] Setting 'channel_layout' to value '0x4'
[graph_1_in_1_0 @ 000001f5baec0f40] tb:1/48000 samplefmt:u8 samplerate:48000 chlayout:0x4
[format_out_0_1 @ 000001f5baec1040] Setting 'sample_fmts' to value 's16'
[format_out_0_1 @ 000001f5baec1040] Setting 'sample_rates' to value '48000'
[format_out_0_1 @ 000001f5baec1040] Setting 'channel_layouts' to value '0x63f'
[format_out_0_1 @ 000001f5baec1040] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[AVFilterGraph @ 000001f5baec44c0] query_formats: 5 queried, 9 merged, 3 already done, 0 delayed
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] Using s16p internally between filters
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] Matrix coefficients:
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] FL: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] FR: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] FC: FC:1.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] LFE: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] BL: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] BR: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] SL: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] [SWR @ 000001f5bba12580] SR: FC:0.000000 
[auto_resampler_0 @ 000001f5baebfc40] ch:1 chl:mono fmt:u8 r:48000Hz -> ch:8 chl:7.1 fmt:s16 r:48000Hz
Output #0, mxf_d10, to 'out.mxf':
  Metadata:
    encoder         : Lavf58.30.100
    Stream #0:0, 0, 1001/30000: Video: mpeg2video (4:2:2), 1 reference frame, yuv422p(top first), 720x512 [SAR 1:1 DAR 45:32], 0/1, q=1-12, 50000 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc58.55.100 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 50000000/50000000/50000000 buffer size: 1668334 vbv_delay: 18446744073709551615
    Stream #0:1, 0, 1/48000: Audio: pcm_s16le, 48000 Hz, 7.1, s16, 6144 kb/s
    Metadata:
      encoder         : Lavc58.55.100 pcm_s16le
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[rawvideo @ 000001f5baec3480] PACKET SIZE: 1049760, STRIDE: 2160
[Parsed_pad_0 @ 000001f5baec1540] Direct padding impossible allocating new frame
Clipping frame in rate conversion by 0.000008
[mxf_d10 @ 000001f5baed2600] out st:0 dts:0
[mxf_d10 @ 000001f5baed2600] track 0: frame size does not match index unit size, 208542 != 208541
av_interleaved_write_frame(): Operation not permitted
No more output streams to write to, finishing.
[mxf_d10 @ 000001f5baed2600] out st:1 dts:0
[mxf_d10 @ 000001f5baed2600] essence container count:1
    Last message repeated 1 times
[mxf_d10 @ 000001f5baed2600] package type:1
[mxf_d10 @ 000001f5baed2600] package type:2
[mxf_d10 @ 000001f5baed2600] edit units count 0
[mxf_d10 @ 000001f5baed2600] essence container count:1
    Last message repeated 2 times
[mxf_d10 @ 000001f5baed2600] package type:1
[mxf_d10 @ 000001f5baed2600] package type:2
[mxf_d10 @ 000001f5baed2600] edit units count 0
frame=    1 fps=0.0 q=1.0 Lsize=      58kB time=00:00:00.04 bitrate=11785.0kbits/s speed=1.54x    
video:204kB audio:30kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (testsrc=size=720x486:rate=60000/1001):
  Input stream #0:0 (video): 2 packets read (2099520 bytes); 2 frames decoded; 
  Total: 2 packets (2099520 bytes) demuxed
Input file #1 (anullsrc=r=48000:n=1920:cl=mono):
  Input stream #1:0 (audio): 1 packets read (1920 bytes); 1 frames decoded (1920 samples); 
  Total: 1 packets (1920 bytes) demuxed
Output file #0 (out.mxf):
  Output stream #0:0 (video): 1 frames encoded; 1 packets muxed (208542 bytes); 
  Output stream #0:1 (audio): 1 frames encoded (1920 samples); 1 packets muxed (30720 bytes); 
  Total: 2 packets (239262 bytes) muxed
3 frames successfully decoded, 0 decoding errors
[AVIOContext @ 000001f5baf1e240] Statistics: 1 seeks, 6 writeouts
Conversion failed!

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Carl Eugen Hoyos]

Looks like a regression since 3d38e45eb85c7a2420cb48a9cd45625c28644b2e

[Tomas Härdin]

208542 is not a legal frame size for NTSC in D-10. See table 1 in S356m. In other words, your encoding parameters are wrong, or FFmpeg's rate control needs tweaking, or needs to complain that it can't fulfill exactly 50 Mbps for NTSC. However, 3d38e45eb85c7a2420cb48a9cd45625c28644b2e is also wrong because S356m does not require the MPEG-2 stream to be CBR, only that it is no more than 50 Mbps. A fix would be something like this:

Either revert or hide what 3d38e45eb85c7a2420cb48a9cd45625c28644b2e does behind a sony vegas pro 11 specific AVOption.
After that, implement the actual constraints specific in S356m.
Optionally, have the rate control system refuse impossible parameter combinations.

[Tomas Härdin]

For bonus points, send angry emails to Sony that they're not following S356m if they require strict CBR for D-10.

[Tomas Härdin]

Another part of the problem here is that libavcodec can't produce NTSC MPEG-2 that is always 208541 bytes per frame, since that implies a bitrate of 49999840.16 and libavcodec can't do fractional bitrates. A flag to make the encoder always produce packets of some given size would also solve this, but I don't know enough about the ratecontrol system to implement that. I tried experimenting with -rc_eq with no success.

[Thomas Mundt]

The mxfenc also allows 49999840 as bitrate for IMX50. However this results in: frame size does not match index unit size, 208541 != 208540
Then I patched mxfenc so it additionally allows 49999841 as bitrate. Now it works. I have tested a clip with more than 1 hour in length. But I´m not sure if ratecontrol keeps the frame size constant over a longer period of time.

[Thomas Mundt]

After further testing this seems to work stable.
Will send patch to ML.