The tee and fifo muxers do not respect the -program parameter(s) with mpegts

[Dennis E. Mungai]

Summary of the bug:
As the subject matter infers.
FFmpeg does not respect the -program parameter combination(s) with the underlying mpegts muxer(s) when it's called up from either tee or the fifo muxer(s).

A trivial example where this works perfectly with the mpegts muxer to create multiple PMTs associated with an audio & video stream pair is shown below:

ffmpeg -init_hw_device cuda=cu:0 \
-filter_hw_device cu \
-thread_queue_size 5120 -f lavfi -i testsrc=duration=360:size=1920x1080:rate=60 \
-thread_queue_size 5120 -f lavfi -i "sine=frequency=2000:duration=360" \
-filter_complex \
"[0:v]hwupload,scale_cuda[v1]; \
[v1]split=2[v2][v3]; \
[1:a]asplit=2[a1][a2]" \
-c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \
-b:v:0 1250k -minrate:v:0 1250k -maxrate:v:0 1250k -bufsize:v:0 1250k -c:v:0 h264_nvenc \
-no-scenecut:v:0 1 -r:v:0 60 -g:v:0 120 -tune:v:0 hq -rc:v:0 cbr -preset:v:0 p4 -bf:v:0 0 \
-b:v:1 1000k -minrate:v:1 1000k -maxrate:v:1 1000k -bufsize:v:1 1000k -c:v:1 h264_nvenc \
-no-scenecut:v:1 1 -r:v:1 60 -g:v:1 120 -tune:v:1 hq -rc:v:1 cbr -preset:v:1 p4 -bf:v:1 0 \
-noautoscale -flags +global_header+cgop \
-map "[v2]" -map "[v3]" -map "[a1]" -map "[a2]" \
-program title="Stream 1":st=0:st=1 \
-program title="Stream 2":st=2:st=3 \
-max_muxing_queue_size 5120 \
-flags -global_header+cgop -f mpegts \
"dump_stream.ts"

ffprobe confirms the correct creation of the PMTs and the associated streams:

Input #0, mpegts, from 'dump_stream.ts':
  Duration: 00:06:00.00, start: 1.400000, bitrate: 3018 kb/s
  Program 1 
    Metadata:
      service_name    : Stream 1
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn
  Stream #0:1[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn
  Program 2 
    Metadata:
      service_name    : Stream 2
      service_provider: FFmpeg
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:3[0x103]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s

Now, attempting the same with the tee and/or fifo muxer(s), independently or combined, and with the -program parameter(s) defined globally as shown in the trivial example below:

ffmpeg -init_hw_device cuda=cu:0 \
-filter_hw_device cu \
-thread_queue_size 5120 -f lavfi -i testsrc=duration=360:size=1920x1080:rate=60 \
-thread_queue_size 5120 -f lavfi -i "sine=frequency=2000:duration=360" \
-filter_complex \
"[0:v]hwupload,scale_cuda[v1]; \
[v1]split=2[v2][v3]; \
[1:a]asplit=2[a1][a2]" \
-c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \
-b:v:0 1250k -minrate:v:0 1250k -maxrate:v:0 1250k -bufsize:v:0 1250k -c:v:0 h264_nvenc \
-no-scenecut:v:0 1 -r:v:0 60 -g:v:0 120 -tune:v:0 hq -rc:v:0 cbr -preset:v:0 p4 -bf:v:0 0 \
-b:v:1 1000k -minrate:v:1 1000k -maxrate:v:1 1000k -bufsize:v:1 1000k -c:v:1 h264_nvenc \
-no-scenecut:v:1 1 -r:v:1 60 -g:v:1 120 -tune:v:1 hq -rc:v:1 cbr -preset:v:1 p4 -bf:v:1 0 \
-noautoscale -flags +global_header+cgop \
-map "[v2]" -map "[v3]" -map "[a1]" -map "[a2]" \
-program title="Stream 1":st=0:st=1 \
-program title="Stream 2":st=2:st=3 \
-max_muxing_queue_size 5120 \
-flags -global_header+cgop \
-f tee "[f=mpegts]dump_stream.ts"

Results in this output from ffprobe:

ffmpeg Input #0, mpegts, from 'dump_stream.ts':
  Duration: 00:06:00.04, start: 0.000000, bitrate: 3020 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn
  Stream #0:1[0x101]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 90k tbn
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:3[0x103]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s

Demonstrating that only one PMT is created, with all the streams bundled therein.

Attempting to pass these -program parameters directly to the underlying mpegts muxer(s) called by either tee or fifo, as shown below:

ffmpeg -init_hw_device cuda=cu:0 \
-filter_hw_device cu \
-thread_queue_size 5120 -f lavfi -i testsrc=duration=360:size=1920x1080:rate=60 \
-thread_queue_size 5120 -f lavfi -i "sine=frequency=2000:duration=360" \
-filter_complex \
"[0:v]hwupload,scale_cuda[v1]; \
[v1]split=2[v2][v3]; \
[1:a]asplit=2[a1][a2]" \
-c:a libfdk_aac -b:a 128k -ar 48000 -ac 2 \
-b:v:0 1250k -minrate:v:0 1250k -maxrate:v:0 1250k -bufsize:v:0 1250k -c:v:0 h264_nvenc \
-no-scenecut:v:0 1 -r:v:0 60 -g:v:0 120 -tune:v:0 hq -rc:v:0 cbr -preset:v:0 p4 -bf:v:0 0 \
-b:v:1 1000k -minrate:v:1 1000k -maxrate:v:1 1000k -bufsize:v:1 1000k -c:v:1 h264_nvenc \
-no-scenecut:v:1 1 -r:v:1 60 -g:v:1 120 -tune:v:1 hq -rc:v:1 cbr -preset:v:1 p4 -bf:v:1 0 \
-noautoscale -flags +global_header+cgop \
-map "[v2]" -map "[v3]" -map "[a1]" -map "[a2]" \
-max_muxing_queue_size 5120 \
-flags -global_header+cgop \
-f tee "[f=mpegts:program=title=Stream 1:st=0:st=1:program=title=Stream 2:st=2:st=3]dump_stream.ts"

Now results in this failure:

Unknown option 'st'
[mpegts @ 0x798844c7b380] Unknown option 'program'
[tee @ 0x602719acf580] All tee outputs failed.
[out#0/tee @ 0x602719acf440] Could not write header (incorrect codec parameters ?): Option not found
[fc#0 @ 0x602719aac480] Error sending frames to consumers: Option not found
[fc#0 @ 0x602719aac480] Task finished with error code: -1414549496 (Option not found)
[fc#0 @ 0x602719aac480] Terminating thread with return code -1414549496 (Option not found)
[libfdk_aac @ 0x602719b0a9c0] Trying to remove 1024 samples, but the queue is empty
    Last message repeated 1 times
[out#0/tee @ 0x602719acf440] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=42.0 Lq=42.0 size=       0KiB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

Which implies that the tee and/or fifo muxers are not mapping that program option to the underlying mpegts muxer(s) , be it globally OR directly in the configuration string(s) within the pseudomuxers.

Impact:

This breaks flows that require the resilience of the tee and fifo pseudomuxers with multi-program mpegts generation.

[Gyan]

So, in general nested muxers do not automatically pick up programs from the top-level context.

I submitted patches in 2023[1] to extend the API to make this convenient, but it got bikeshed.
I will attempt to revive this next month when I have time.

[1]: ​https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=8397