confused pipes

[Nils Goroll]

Summary of the bug: using more than one output pipe leads to missing and corrupt output

How to reproduce:

#!/bin/bash

input=video.mkv

runtest () {
    echo
    echo case ${tc}
    echo ============================
    rm -f named_pipe ${tc}_*
    mknod named_pipe p
    cat named_pipe >${tc}_pipe &
    catpid=$!
    cat ${input} | ffmpeg -i - "${args[@]}" | cat >${tc}_fd1
    kill $catpid 2>/dev/null
    echo ----------------------------
    ls -als ${tc}_*
    for f in ${tc}_* ; do
	echo
	echo $f:
	ffprobe -hide_banner ${f}
    done
}

tc=fd1_np
args=(
    -f matroska -map 0:v -vcodec copy      pipe:1
    -f wav      -map 0:a -acodec pcm_s32le pipe:named_pipe
)
runtest

#

tc=fd1_file
args=(
    -y
    -f matroska -map 0:v -vcodec copy      pipe:1
    -f wav      -map 0:a -acodec pcm_s32le named_pipe
)
runtest

exit

## same as fd1_np
tc=np_fd1
args=(
    -f wav      -map 0:a -acodec pcm_s32le pipe:named_pipe
    -f matroska -map 0:v -vcodec copy      pipe:1
)

runtest

output:

case fd1_np
============================
ffmpeg version N-115007-g1f8e5b6d95-1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/local --extra-version=1 --toolchain=hardened --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --disable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --disable-shared --enable-libfdk-aac --enable-nonfree
  libavutil      59. 16.101 / 59. 16.101
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.102 / 61.  3.102
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Input #0, matroska,webm, from 'fd:':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:01:00.00, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
        DURATION        : 00:01:00.003000000
  Stream #0:1: Audio: vorbis, 48000 Hz, stereo, fltp
      Metadata:
        ENCODER         : Lavc59.37.100 libvorbis
        DURATION        : 00:01:00.003000000
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #1:0 (vorbis (native) -> pcm_s32le (native))
Output #0, matroska, to 'pipe:1':
  Metadata:
    encoder         : Lavf61.3.102
  Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
        DURATION        : 00:01:00.003000000
Output #1, wav, to 'pipe:named_pipe':
  Metadata:
    ISFT            : Lavf61.3.102
  Stream #1:0: Audio: pcm_s32le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32, 3072 kb/s
      Metadata:
        DURATION        : 00:01:00.003000000
        encoder         : Lavc61.5.103 pcm_s32le
[out#0/matroska @ 0x557d535f8940] video:26KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 39.822365%
[out#1/wav @ 0x557d535f9a40] video:0KiB audio:22500KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.000443%
size=      36KiB time=00:00:59.92 bitrate=   4.9kbits/s speed=1.04e+03x    
----------------------------
1 -rw-r--r-- 1 slink slink 23076625 Apr 25 18:54 fd1_np_fd1
1 -rw-r--r-- 1 slink slink        0 Apr 25 18:54 fd1_np_pipe

fd1_np_fd1:
[matroska,webm @ 0x55de843c8b40] 0x00 at pos 2160 (0x870) invalid as first byte of an EBML number
[matroska,webm @ 0x55de843c8b40] Duplicate element
[matroska,webm @ 0x55de843c8b40] 0x00 at pos 110 (0x6e) invalid as first byte of an EBML number
[matroska,webm @ 0x55de843c8b40] Duplicate element
[matroska,webm @ 0x55de843c8b40] 0x00 at pos 202 (0xca) invalid as first byte of an EBML number
[matroska,webm @ 0x55de843c8b40] Element at 0x67 ending at 0x83ec0100000070 exceeds containing master element ending at 0x1413
[matroska,webm @ 0x55de843c8b40] 0x00 at pos 704676 (0xac0a4) invalid as first byte of an EBML number
[matroska,webm @ 0x55de843c8b40] Length 5 indicated by an EBML number's first byte 0x09 at pos 2985599 (0x2d8e7f) exceeds max length 4.
[matroska,webm @ 0x55de843c8b40] 0x00 at pos 2193011 (0x217673) invalid as first byte of an EBML number
[matroska,webm @ 0x55de843c8b40] Could not find codec parameters for stream 1 (Video: h264, none(progressive), 320x240): unspecified pixel format
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, matroska,webm, from 'fd1_np_fd1':
  Metadata:
    ENCODER         : Lavf61.3.102
  Duration: 00:01:00.00, start: 0.003000, bitrate: 3076 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
  Stream #0:1: Video: h264, none(progressive), 320x240, SAR 1:1 DAR 4:3, 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264

fd1_np_pipe:
fd1_np_pipe: Invalid data found when processing input

case fd1_file
============================
ffmpeg version N-115007-g1f8e5b6d95-1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/local --extra-version=1 --toolchain=hardened --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --disable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --disable-shared --enable-libfdk-aac --enable-nonfree
  libavutil      59. 16.101 / 59. 16.101
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.102 / 61.  3.102
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Input #0, matroska,webm, from 'fd:':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:01:00.00, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
        DURATION        : 00:01:00.003000000
  Stream #0:1: Audio: vorbis, 48000 Hz, stereo, fltp
      Metadata:
        ENCODER         : Lavc59.37.100 libvorbis
        DURATION        : 00:01:00.003000000
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #1:0 (vorbis (native) -> pcm_s32le (native))
Output #0, matroska, to 'pipe:1':
  Metadata:
    encoder         : Lavf61.3.102
  Stream #0:0: Video: h264 (High) (H264 / 0x34363248), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264
        DURATION        : 00:01:00.003000000
Output #1, wav, to 'named_pipe':
  Metadata:
    ISFT            : Lavf61.3.102
  Stream #1:0: Audio: pcm_s32le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32, 3072 kb/s
      Metadata:
        DURATION        : 00:01:00.003000000
        encoder         : Lavc61.5.103 pcm_s32le
[out#0/matroska @ 0x560e5ff71b40] video:26KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 39.822365%
[out#1/wav @ 0x560e5ff73d00] video:0KiB audio:22500KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.000443%
size=      36KiB time=00:00:59.92 bitrate=   4.9kbits/s speed=1e+03x    
----------------------------
1 -rw-r--r-- 1 slink slink    36523 Apr 25 18:54 fd1_file_fd1
1 -rw-r--r-- 1 slink slink 23040102 Apr 25 18:54 fd1_file_pipe

fd1_file_fd1:
Input #0, matroska,webm, from 'fd1_file_fd1':
  Metadata:
    ENCODER         : Lavf61.3.102
  Duration: 00:01:00.00, start: 0.003000, bitrate: 4 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1k tbn
      Metadata:
        ENCODER         : Lavc59.37.100 libx264

fd1_file_pipe:
[wav @ 0x55a0af5fba40] Ignoring maximum wav data size, file may be invalid
[wav @ 0x55a0af5fba40] Estimating duration from bitrate, this may be inaccurate
Input #0, wav, from 'fd1_file_pipe':
  Metadata:
    encoder         : Lavf61.3.102
  Duration: 00:01:00.00, bitrate: 3072 kb/s
  Stream #0:0: Audio: pcm_s32le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s32, 3072 kb/s

So, to summarize:

If two pipe: outputs are used, one output does not receive any data, while the other gets corrupted. If, however, a pipe: output is replaced with a regular file name (a named pipe in this case), the written data looks fine.

THANK YOU to everyone contributing to ffmpeg for your GREAT work!

[Cigaes]

<https://ffmpeg.org//ffmpeg-all.html#pipe>:

The accepted syntax is:
pipe:[number]

In pipe:named_pipe, that named_pipe does not look like a number.

Also, when reporting a bug, simplify your use case as most as possible.

[Nils Goroll]

Thank you, I just noticed my mistake and was about to add a comment.