bsf 264_metadata removes stuffing bytes from NAL units - destroys original bitrate

[emcodem]

Summary of the bug:
When using the h264_metadata bsf, "trailing zero bytes" of all NAL's are removed. This leads to the output having a different bitrate than the input.
This seems to be by design so i guess we talk about feature request not a bug. I tried to check and workaround in a custom branch but had no luck, it is just too complex. From feeling it starts in cbs_h2645_fragment_add_nals (cbs_
h2645.c) but when i just comment the section "Remove trailing zeroes", output is garbage.

How to reproduce:

% ffmpeg -i INPUT.h264 -codec copy -bsf:v h264_metadata=colour_primaries=9,h264_metadata=delete_filler=0 OUTPUT.h264

I created a simplified source stream for testing that only contains a single h264 frame in raw format, for playing with this it does not matter if we have one or multiple source frames or a container or elementary stream.
The example source file has 2 NAL's with "trailing_zero_bytes":

PPS: Original length 9472 bytes, length after bsf: 5 bytes
Last IDR slice: Original length ~1 MB, length after bsf: ~80kB

Just for explaination: the reason why the last IDR slice contains that much zeros is that the pic content is a colorbar but the output bitrate is constant 500Mbit/s (XAVC Class 300).

Full, uncut console output:
ffmpeg.exe -i C:\temp\ffmpeg_xavc\1xavc.h264 -codec copy -bsf:v h264_metadata=colour_primaries=9,h264_metadata=delete_filler=0 C:\temp\ffmpeg_xavc\out.h264 -y
ffmpeg version N-109725-g2d202985b7 Copyright (c) 2000-2023 the FFmpeg developers

built with gcc 12.1.0 (Rev1, Built by MSYS2 project)
configuration: --enable-gpl --enable-static
libavutil 57. 44.100 / 57. 44.100
libavcodec 59. 59.100 / 59. 59.100
libavformat 59. 36.100 / 59. 36.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 56.100 / 8. 56.100
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100

Input #0, h264, from 'C:\temp\ffmpeg_xavc\1xavc.h264':

Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (High 4:2:2 Intra), yuv422p10le(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 50 tbr, 1200k tbn

Output #0, h264, to 'C:\temp\ffmpeg_xavc\out.h264':

Metadata:

encoder : Lavf59.36.100

Stream #0:0: Video: h264 (High 4:2:2 Intra), yuv422p10le(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 50 tbr, 50 tbn

Stream mapping:

Stream #0:0 -> #0:0 (copy)

Press [q] to stop, ? for help
frame= 1 fps=0.0 q=-1.0 Lsize= 254kB time=00:00:00.00 bitrate=N/A speed= 0x
video:254kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

[emcodem]

Single h264 frame with trailing zero bytes in PPS and last NAL

[emcodem]

Just for info, i had a talk to mkver about this. He proposed to just append zeros to the pkt instead of remembering the trailing bytes on a per NALu basis. I played with this about 2 weeks but it was really too hard for me to find a way add the trailing bytes per nalu. I was not able to store any size modification of the nalu or the packet in h264_metadata_update_fragment method (h264_metadata_bsf.c). I guess mostly because of my missing understanding about how the ref stuff works.

Storing and restoring the trailing bytes on a per nalu base would probably be what everyone expects from the bsf (not actually modify anything else than it's instructed to modify) but for my usecase, it should be fine when i just append zeros to the packet which hopefully leads to the last IDR SLice NALu containing all the trailing bytes that have been removed by cbs and it's unit de and re-composition.

What i do for testing (if it works of course in production :D) is a simple hack in cbs.c ff_cbs_write_packet method; remember the original pkt size before updating it and right before return 0:

if (original_size > pkt->size)

av_grow_packet(pkt,original_size - pkt->size);