mxfenc mxf_opatom with only vbi_vanc_smpte_436M data stream assertion error

[Jacob Lifshay]

Summary of the bug:
muxing only the vbi_vanc_smpte_436M data stream from the provided .mxf file into a mxf_opatom file causes an assertion error:
How to reproduce:

% gdb -q --batch -ex run -ex bt --args ./ffmpeg-git_g -v 9 -loglevel 99 -nostats -nostdin -y -i mxf_vanc_causes_opatom_bug.mxf -map 0:1 -f mxf_opatom /tmp/out.mxf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
ffmpeg version N-120374-ge29016a9de Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14+deb12u1)
  configuration: --optflags=-O0 --assert-level=2 --disable-shared --enable-static --progs-suffix=-git --enable-sdl2 --enable-libx264 --enable-gpl --enable-libklvanc
  libavutil      60.  6.100 / 60.  6.100
  libavcodec     62.  8.100 / 62.  8.100
  libavformat    62.  1.103 / 62.  1.103
  libavdevice    62.  0.100 / 62.  0.100
  libavfilter    11.  3.100 / 11.  3.100
  libswscale      9.  0.100 /  9.  0.100
  libswresample   6.  0.100 /  6.  0.100
Splitting the commandline.
<snip>
[in#0/mxf @ 0x555558de7680] EOF while reading input
[out#0/mxf_opatom @ 0x555558e2e280] All streams finished
[out#0/mxf_opatom @ 0x555558e2e280] Terminating thread with return code 0 (success)
[in#0/mxf @ 0x555558de7680] Terminating thread with return code 0 (success)
[mxf_opatom @ 0x555558e2e3c0] essence container count:1
[mxf_opatom @ 0x555558e2e3c0] edit units count 4369
Assertion b>=-128 && b<=255 failed at libavformat/aviobuf.c:181
[Thread 0x7ffff73be6c0 (LWP 52538) exited]
[Thread 0x7fffeffff6c0 (LWP 52539) exited]

Thread 1 "ffmpeg-git_g" received signal SIGABRT, Aborted.
__pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x00007ffff76a9f4f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x00007ffff765afb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff7645472 in __GI_abort () at ./stdlib/abort.c:79
#4  0x0000555555c172c8 in avio_w8 (s=0x555558e45300, b=256) at libavformat/aviobuf.c:181
#5  0x0000555555c18128 in avio_wb16 (s=0x555558e45300, val=65543) at libavformat/aviobuf.c:445
#6  0x0000555555d60d91 in mxf_write_local_tag (s=0x555558e2e3c0, size=65543, tag=16138) at libavformat/mxfenc.c:681
#7  0x0000555555d6531e in mxf_write_index_table_segment (s=0x555558e2e3c0) at libavformat/mxfenc.c:2075
#8  0x0000555555d6a9d8 in mxf_write_footer (s=0x555558e2e3c0) at libavformat/mxfenc.c:3458
#9  0x0000555555d4f75f in av_write_trailer (s=0x555558e2e3c0) at libavformat/mux.c:1262
#10 0x000055555569ce8b in of_write_trailer (of=0x555558e2e280) at fftools/ffmpeg_mux.c:770
#11 0x00005555556d1ef0 in transcode (sch=0x555558de7380) at fftools/ffmpeg.c:907
#12 0x00005555556d21b9 in main (argc=15, argv=0x7fffffffdca8) at fftools/ffmpeg.c:1009

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[Jacob Lifshay]

ffmpeg in gdb output

[Jacob Lifshay]

I uploaded mxf_vanc_causes_opatom_bug.mxf to ​https://streams.videolan.org/ffmpeg/incoming/11684/
It's licensed CC-by-sa-4.0