Opened 4 years ago

Closed 4 years ago

#8387 closed defect (invalid)

Anomalies with the tee muxer's transparent recovery with HLS and DASH streams

Reported by: Dennis E. Mungai Owned by:
Priority: minor Component: avformat
Version: git-master Keywords: hls dash
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

Reproduced with this version compiled a few hours ago:

ffmpeg 
ffmpeg version N-95809-g804fce8bc2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

The tee (and fifo) muxer(s) is(/are) not able to adequately recover both HLS and DASH streams, when explicitly configured to do so. Take a look at the example below.

How to reproduce:
Take a look at the command below, which demonstrates a scenario of generating multiple HLS and DASH streams from one command:

#! /bin/bash
HLSOPTS=':use_fifo=1:fifo_options=recovery_wait_time=1\\:attempt_recovery=1\\:restart_with_keyframe=1\\:recover_any_error=1:f=hls:hls_wrap=6:hls_time=6:hls_list_size=6:hls_segment_type=fmp4:ignore_io_errors=1'
DASHOPTS='use_fifo=1:fifo_options=recovery_wait_time=1\\:attempt_recovery=1\\:restart_with_keyframe=1\\:recover_any_error=1:f=dash:seg_duration=6:window_size=6:extra_window_size=0:ignore_io_errors=1'


ffmpeg -t 3600 -nostdin -y \
   -fflags +autobsf+genpts-fastseek+igndts -vsync 1 \
   -f mpegts -fix_teletext_pts 1 -scan_all_pmts 1 -merge_pmt_versions 1 \
   -re -i 'feeds.ts' -filter_complex \
  "[i:0x100]yadif,split=6[s0][s1][s2][s3][s4][s5]; \
   [s0]scale=w=1920:h=1080,split=2[v0][v1]; \
   [s1]scale=w=1280:h=720,split=2[v2][v3]; \
   [s2]scale=w=960:h=540,format=nv12[v4]; \
   [s3]scale=w=768:h=432,split=2[v5][v6]; \
   [s4]scale=w=640:h=360[v7]; \
   [s5]scale=w=416:h=234[v8]" \
  -c:a aac -ac 2 -ar 48000 -b:a 128k \
  -b:v:0 7800k -minrate:v:0 7800k -maxrate:v:0 7800k -bufsize:v:0 7800k -c:v:0 libx264 \
  -profile:v:0 high -preset:v:0 fast -tune:v:0 zerolatency -g:v:0 60 -bf:v:0 0 -x264opts "no-scenecut" \
  -b:v:1 6000k -minrate:v:1 6000k -maxrate:v:1 6000k -bufsize:v:1 6000k -c:v:1 libx264 \
  -profile:v:1 high -preset:v:1 fast -tune:v:1 zerolatency -g:v:1 60 -bf:v:1 0 -x264opts "no-scenecut" \
  -b:v:2 4500k -minrate:v:2 4500k -maxrate:v:2 4500k -bufsize:v:2 4500k -c:v:2 libx264 \
  -profile:v:2 high -preset:v:2 fast -tune:v:2 zerolatency -g:v:2 60 -bf:v:2 0 -x264opts "no-scenecut" \
  -b:v:3 3000k -minrate:v:3 3000k -maxrate:v:3 3000k -bufsize:v:3 3000k -c:v:3 libx264 \
  -profile:v:3 main -preset:v:3 fast -tune:v:3 zerolatency -g:v:3 60 -bf:v:3 0 -x264opts "no-scenecut" \
  -b:v:4 2000k -minrate:v:4 2000k -maxrate:v:4 2000k -bufsize:v:4 2000k -c:v:4 libx264 \
  -profile:v:4 main -preset:v:4 fast -tune:v:4 zerolatency -g:v:4 60 -bf:v:4 0 -x264opts "no-scenecut" \
  -b:v:5 1100k -minrate:v:5 1100k -maxrate:v:5 1100k -bufsize:v:5 1100k -c:v:5 libx264 \
  -profile:v:5 main -preset:v:5 fast -tune:v:5 zerolatency -g:v:5 60 -bf:v:5 0 -x264opts "no-scenecut" \
  -b:v:6 730k -minrate:v:6 730k -maxrate:v:6 730k -bufsize:v:6 730k -c:v:6 libx264 \
  -profile:v:6 main -preset:v:6 fast -tune:v:6 zerolatency -g:v:6 60 -bf:v:6 0 -x264opts "no-scenecut" \
  -b:v:7 365k -minrate:v:7 365k -maxrate:v:7 365k -bufsize:v:7 365k -c:v:7 libx264 \
  -profile:v:7 baseline -preset:v:7 fast -tune:v:7 zerolatency -g:v:7 60 -bf:v:7 0 -x264opts "no-scenecut" \
  -b:v:8 145k -minrate:v:8 145k -maxrate:v:8 145k -bufsize:v:8 145k -c:v:8 libx264 \
  -profile:v:8 baseline -preset:v:8 fast -tune:v:8 zerolatency -g:v:8 60 -bf:v:8 0 -x264opts "no-scenecut" \
  -map "[v0]" -map "[v1]" -map "[v2]" -map "[v3]" -map "[v4]" -map "[v5]" -map "[v6]" -map "[v7]" -map "[v8]" -map 'i:0x101' \
  -flags +global_header+cgop \
  -max_muxing_queue_size 90000 -f tee  \
  "[select=\'v:0,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-9/manifest.m3u8'| \
   [select=\'v:1,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-8/manifest.m3u8'| \
   [select=\'v:2,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-7/manifest.m3u8'| \
   [select=\'v:3,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-6/manifest.m3u8'| \
   [select=\'v:4,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-5/manifest.m3u8'| \
   [select=\'v:5,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-4/manifest.m3u8'| \
   [select=\'v:6,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-3/manifest.m3u8'| \
   [select=\'v:7,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-2/manifest.m3u8'| \
   [select=\'v:8,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-1/manifest.m3u8'| \
   [$DASHOPTS:adaptation_sets='id=0,streams=v id=1,streams=a']'/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd'"

To simulate failure, one can either rename or delete a sub-directory to which the tee muxer's fifo slaves are writing packets to.

Here are the observations:

(a). With HLS:

  1. Rename operations will recover successfully once the original target directory name is restored. This works flawlessly, and the content therein is playable.
  1. Target directory deletion and recreation (which simulates a failure scenario with network file systems such as NFS, etc) results in a case whereby the underlying HLS muxer recovers successfully, BUT does not restore the init.mp4 file when the segment format is set to mp4.

Here's a snapshot of such a scenario, where the target directory is deleted and recreated:

pwd
/home/brainiarc7/Desktop/src/test/H264/Gear-3
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls -al 
total 3796
drwxr-xr-x  2 brainiarc7 brainiarc7   4096 Nov 20 01:04 .
drwxr-xr-x 12 brainiarc7 brainiarc7   4096 Nov 19 22:56 ..
-rw-r--r--  1 brainiarc7 brainiarc7   1309 Nov 20 00:52 init.mp4
-rw-r--r--  1 brainiarc7 brainiarc7 627855 Nov 20 01:03 manifest0.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 624993 Nov 20 01:04 manifest1.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 685980 Nov 20 01:03 manifest2.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 645685 Nov 20 01:03 manifest3.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 583939 Nov 20 01:03 manifest4.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 690322 Nov 20 01:03 manifest5.m4s
-rw-r--r--  1 brainiarc7 brainiarc7    292 Nov 20 01:04 manifest.m3u8

Now we delete that directory, recreate it and run ls-al:

brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ cd ..
brainiarc7@lucifer:~/Desktop/src/test/H264$ rm -fr Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264$ mkdir -p Gear-3
brainiarc7@lucifer:~/Desktop/src/test/H264$ cd Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls -al
total 3700
drwxr-xr-x  2 brainiarc7 brainiarc7   4096 Nov 20 01:06 .
drwxr-xr-x 12 brainiarc7 brainiarc7   4096 Nov 20 01:05 ..
-rw-r--r--  1 brainiarc7 brainiarc7 692562 Nov 20 01:05 manifest0.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 619051 Nov 20 01:06 manifest1.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 667125 Nov 20 01:05 manifest2.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 645877 Nov 20 01:05 manifest3.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 516622 Nov 20 01:05 manifest4.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 619269 Nov 20 01:05 manifest5.m4s
-rw-r--r--  1 brainiarc7 brainiarc7    292 Nov 20 01:06 manifest.m3u8

Despite the underlying hls muxer recovering, see that the init.mp4 was not recreated.
The content therein is therefore rendered unusable.

From mpv, we get:

brainiarc7@lucifer:~/Desktop/src/test$ cd H264/Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls
manifest0.m4s  manifest1.m4s  manifest2.m4s  manifest3.m4s  manifest4.m4s  manifest5.m4s  manifest.m3u8
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ mpv manifest.m3u8 
[ffmpeg/demuxer] hls: Failed to open an initialization section in playlist 0
[ffmpeg/demuxer] hls: Error when loading first segment 'manifest3.m4s'
[lavf] avformat_open_input() failed
[lavf] Leaking 1 nested connections (FFmpeg bug).

Playing: manifest3.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest4.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest5.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest0.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest1.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest2.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.


Exiting... (Some errors happened)

And with ffplay:

ffplay manifest.m3u8 
ffplay version N-95809-g804fce8bc2 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[hls @ 0x7f3528000b80] Skip ('#EXT-X-VERSION:7')
[hls @ 0x7f3528000b80] Opening 'init.mp4' for reading
[hls @ 0x7f3528000b80] Failed to open an initialization section in playlist 0
[hls @ 0x7f3528000b80] Error when loading first segment 'manifest3.m4s'
manifest.m3u8: No such file or directory

(b). On to DASH:

If we rename OR delete the target directories to which the underlying DASH muxer is writing to with the aforementioned muxer options, the tee muxer slaves (fifo'd) claims that the recovery is successful, BUT with the following spurious error in the console:

[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20977892936 >= 20957937
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20978894937 >= 20958938
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
^C[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20979896938 >= 20959939
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[hls @ 0x55e25c912000] Opening '/home/brainiarc7/Desktop/src/test/H264/Gear-9/manifest2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20980898939 >= 20960940
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful

Which is then repeated non-stop: "Application provided invalid, non monotonically increasing dts to muxer in stream 0"

The content rendered there-in, even after the directory is renamed or recreated with the original name is definitely unplayable. Both ffprobe and mpv die on attempting playback:

ffplay manifest.mpd 
ffplay version N-95809-g804fce8bc2 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[dash @ 0x7fb1dc000b80] decoding for stream 0 failed=    0B f=0/0   
[dash @ 0x7fb1dc000b80] decoding for stream 1 failed
[dash @ 0x7fb1dc000b80] decoding for stream 2 failed
[dash @ 0x7fb1dc000b80] decoding for stream 3 failed
[dash @ 0x7fb1dc000b80] decoding for stream 4 failed
[dash @ 0x7fb1dc000b80] decoding for stream 5 failed
[dash @ 0x7fb1dc000b80] decoding for stream 6 failed
[dash @ 0x7fb1dc000b80] decoding for stream 7 failed
[dash @ 0x7fb1dc000b80] decoding for stream 8 failed
Input #0, dash, from 'manifest.mpd':
  Duration: 00:00:33.00, start: 1397.396000, bitrate: 1 kb/s
  Program 0 
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 7800000
      id              : 0
    Stream #0:1: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 6000000
      id              : 1
    Stream #0:2: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 4500000
      id              : 2
    Stream #0:3: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 3000000
      id              : 3
    Stream #0:4: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 960x540 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 2000000
      id              : 4
    Stream #0:5: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 768x432 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 1100000
      id              : 5
    Stream #0:6: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 768x432 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 730000
      id              : 6
    Stream #0:7: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 365000
      id              : 7
    Stream #0:8: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 416x234 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 145000
      id              : 8
    Stream #0:9: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 0 kb/s
    Metadata:
      variant_bitrate : 128000
      id              : 9
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 0
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 2
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 3
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 4
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 5
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 6
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 7
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 8

And with mpv:

mpv manifest.mpd 
 (+) Video --vid=1 'bitrate 7800000' (h264 1920x1080)
     Video --vid=2 'bitrate 6000000' (h264 1920x1080)
     Video --vid=3 'bitrate 4500000' (h264 1280x720)
     Video --vid=4 'bitrate 3000000' (h264 1280x720)
     Video --vid=5 'bitrate 2000000' (h264 960x540)
     Video --vid=6 'bitrate 1100000' (h264 768x432)
     Video --vid=7 'bitrate 730000' (h264 768x432)
     Video --vid=8 'bitrate 365000' (h264 640x360)
     Video --vid=9 'bitrate 145000' (h264 416x234)
 (+) Audio --aid=1 'bitrate 128000' (aac 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float


Exiting... (End of file)

In conclusion: The tee (and fifo muxer combination) cannot transparently recover HLS (with mp4 as segment format) and DASH streams, as documented above.

Change History (3)

comment:1 by Carl Eugen Hoyos, 4 years ago

Component: undeterminedavformat
Keywords: fmp4 tee fifo removed
Priority: normalminor
Version: unspecifiedgit-master

in reply to:  description comment:2 by Steven Liu, 4 years ago

Replying to Brainiarc7:

Summary of the bug:

Reproduced with this version compiled a few hours ago:

ffmpeg 
ffmpeg version N-95809-g804fce8bc2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

The tee (and fifo) muxer(s) is(/are) not able to adequately recover both HLS and DASH streams, when explicitly configured to do so. Take a look at the example below.

How to reproduce:
Take a look at the command below, which demonstrates a scenario of generating multiple HLS and DASH streams from one command:

#! /bin/bash
HLSOPTS=':use_fifo=1:fifo_options=recovery_wait_time=1\\:attempt_recovery=1\\:restart_with_keyframe=1\\:recover_any_error=1:f=hls:hls_wrap=6:hls_time=6:hls_list_size=6:hls_segment_type=fmp4:ignore_io_errors=1'
DASHOPTS='use_fifo=1:fifo_options=recovery_wait_time=1\\:attempt_recovery=1\\:restart_with_keyframe=1\\:recover_any_error=1:f=dash:seg_duration=6:window_size=6:extra_window_size=0:ignore_io_errors=1'


ffmpeg -t 3600 -nostdin -y \
   -fflags +autobsf+genpts-fastseek+igndts -vsync 1 \
   -f mpegts -fix_teletext_pts 1 -scan_all_pmts 1 -merge_pmt_versions 1 \
   -re -i 'feeds.ts' -filter_complex \
  "[i:0x100]yadif,split=6[s0][s1][s2][s3][s4][s5]; \
   [s0]scale=w=1920:h=1080,split=2[v0][v1]; \
   [s1]scale=w=1280:h=720,split=2[v2][v3]; \
   [s2]scale=w=960:h=540,format=nv12[v4]; \
   [s3]scale=w=768:h=432,split=2[v5][v6]; \
   [s4]scale=w=640:h=360[v7]; \
   [s5]scale=w=416:h=234[v8]" \
  -c:a aac -ac 2 -ar 48000 -b:a 128k \
  -b:v:0 7800k -minrate:v:0 7800k -maxrate:v:0 7800k -bufsize:v:0 7800k -c:v:0 libx264 \
  -profile:v:0 high -preset:v:0 fast -tune:v:0 zerolatency -g:v:0 60 -bf:v:0 0 -x264opts "no-scenecut" \
  -b:v:1 6000k -minrate:v:1 6000k -maxrate:v:1 6000k -bufsize:v:1 6000k -c:v:1 libx264 \
  -profile:v:1 high -preset:v:1 fast -tune:v:1 zerolatency -g:v:1 60 -bf:v:1 0 -x264opts "no-scenecut" \
  -b:v:2 4500k -minrate:v:2 4500k -maxrate:v:2 4500k -bufsize:v:2 4500k -c:v:2 libx264 \
  -profile:v:2 high -preset:v:2 fast -tune:v:2 zerolatency -g:v:2 60 -bf:v:2 0 -x264opts "no-scenecut" \
  -b:v:3 3000k -minrate:v:3 3000k -maxrate:v:3 3000k -bufsize:v:3 3000k -c:v:3 libx264 \
  -profile:v:3 main -preset:v:3 fast -tune:v:3 zerolatency -g:v:3 60 -bf:v:3 0 -x264opts "no-scenecut" \
  -b:v:4 2000k -minrate:v:4 2000k -maxrate:v:4 2000k -bufsize:v:4 2000k -c:v:4 libx264 \
  -profile:v:4 main -preset:v:4 fast -tune:v:4 zerolatency -g:v:4 60 -bf:v:4 0 -x264opts "no-scenecut" \
  -b:v:5 1100k -minrate:v:5 1100k -maxrate:v:5 1100k -bufsize:v:5 1100k -c:v:5 libx264 \
  -profile:v:5 main -preset:v:5 fast -tune:v:5 zerolatency -g:v:5 60 -bf:v:5 0 -x264opts "no-scenecut" \
  -b:v:6 730k -minrate:v:6 730k -maxrate:v:6 730k -bufsize:v:6 730k -c:v:6 libx264 \
  -profile:v:6 main -preset:v:6 fast -tune:v:6 zerolatency -g:v:6 60 -bf:v:6 0 -x264opts "no-scenecut" \
  -b:v:7 365k -minrate:v:7 365k -maxrate:v:7 365k -bufsize:v:7 365k -c:v:7 libx264 \
  -profile:v:7 baseline -preset:v:7 fast -tune:v:7 zerolatency -g:v:7 60 -bf:v:7 0 -x264opts "no-scenecut" \
  -b:v:8 145k -minrate:v:8 145k -maxrate:v:8 145k -bufsize:v:8 145k -c:v:8 libx264 \
  -profile:v:8 baseline -preset:v:8 fast -tune:v:8 zerolatency -g:v:8 60 -bf:v:8 0 -x264opts "no-scenecut" \
  -map "[v0]" -map "[v1]" -map "[v2]" -map "[v3]" -map "[v4]" -map "[v5]" -map "[v6]" -map "[v7]" -map "[v8]" -map 'i:0x101' \
  -flags +global_header+cgop \
  -max_muxing_queue_size 90000 -f tee  \
  "[select=\'v:0,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-9/manifest.m3u8'| \
   [select=\'v:1,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-8/manifest.m3u8'| \
   [select=\'v:2,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-7/manifest.m3u8'| \
   [select=\'v:3,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-6/manifest.m3u8'| \
   [select=\'v:4,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-5/manifest.m3u8'| \
   [select=\'v:5,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-4/manifest.m3u8'| \
   [select=\'v:6,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-3/manifest.m3u8'| \
   [select=\'v:7,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-2/manifest.m3u8'| \
   [select=\'v:8,a\'$HLSOPTS]'/home/brainiarc7/Desktop/src/test/H264/Gear-1/manifest.m3u8'| \
   [$DASHOPTS:adaptation_sets='id=0,streams=v id=1,streams=a']'/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd'"

To simulate failure, one can either rename or delete a sub-directory to which the tee muxer's fifo slaves are writing packets to.

Here are the observations:

(a). With HLS:

  1. Rename operations will recover successfully once the original target directory name is restored. This works flawlessly, and the content therein is playable.
  1. Target directory deletion and recreation (which simulates a failure scenario with network file systems such as NFS, etc) results in a case whereby the underlying HLS muxer recovers successfully, BUT does not restore the init.mp4 file when the segment format is set to mp4.

Here's a snapshot of such a scenario, where the target directory is deleted and recreated:

pwd
/home/brainiarc7/Desktop/src/test/H264/Gear-3
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls -al 
total 3796
drwxr-xr-x  2 brainiarc7 brainiarc7   4096 Nov 20 01:04 .
drwxr-xr-x 12 brainiarc7 brainiarc7   4096 Nov 19 22:56 ..
-rw-r--r--  1 brainiarc7 brainiarc7   1309 Nov 20 00:52 init.mp4
-rw-r--r--  1 brainiarc7 brainiarc7 627855 Nov 20 01:03 manifest0.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 624993 Nov 20 01:04 manifest1.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 685980 Nov 20 01:03 manifest2.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 645685 Nov 20 01:03 manifest3.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 583939 Nov 20 01:03 manifest4.m4s
-rw-r--r--  1 brainiarc7 brainiarc7 690322 Nov 20 01:03 manifest5.m4s
-rw-r--r--  1 brainiarc7 brainiarc7    292 Nov 20 01:04 manifest.m3u8

Now we delete that directory, recreate it and run ls-al:

brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ cd ..
brainiarc7@lucifer:~/Desktop/src/test/H264$ rm -fr Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264$ mkdir -p Gear-3

are this operations do in the other terminal when the ffmpeg is running status?

brainiarc7@lucifer:~/Desktop/src/test/H264$ cd Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls -al
total 3700
drwxr-xr-x 2 brainiarc7 brainiarc7 4096 Nov 20 01:06 .
drwxr-xr-x 12 brainiarc7 brainiarc7 4096 Nov 20 01:05 ..
-rw-r--r-- 1 brainiarc7 brainiarc7 692562 Nov 20 01:05 manifest0.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 619051 Nov 20 01:06 manifest1.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 667125 Nov 20 01:05 manifest2.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 645877 Nov 20 01:05 manifest3.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 516622 Nov 20 01:05 manifest4.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 619269 Nov 20 01:05 manifest5.m4s
-rw-r--r-- 1 brainiarc7 brainiarc7 292 Nov 20 01:06 manifest.m3u8

}}}

Despite the underlying hls muxer recovering, see that the init.mp4 was not recreated.
The content therein is therefore rendered unusable.

From mpv, we get:

brainiarc7@lucifer:~/Desktop/src/test$ cd H264/Gear-3/
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ ls
manifest0.m4s  manifest1.m4s  manifest2.m4s  manifest3.m4s  manifest4.m4s  manifest5.m4s  manifest.m3u8
brainiarc7@lucifer:~/Desktop/src/test/H264/Gear-3$ mpv manifest.m3u8 
[ffmpeg/demuxer] hls: Failed to open an initialization section in playlist 0
[ffmpeg/demuxer] hls: Error when loading first segment 'manifest3.m4s'
[lavf] avformat_open_input() failed
[lavf] Leaking 1 nested connections (FFmpeg bug).

Playing: manifest3.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest4.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest5.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest0.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest1.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.

Playing: manifest2.m4s
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 1
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 2
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding trex (id 1)
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: could not find corresponding track id 0
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: trun track id unknown, no tfhd was found
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: error reading header
[lavf] avformat_open_input() failed
Failed to recognize file format.


Exiting... (Some errors happened)

And with ffplay:

ffplay manifest.m3u8 
ffplay version N-95809-g804fce8bc2 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[hls @ 0x7f3528000b80] Skip ('#EXT-X-VERSION:7')
[hls @ 0x7f3528000b80] Opening 'init.mp4' for reading
[hls @ 0x7f3528000b80] Failed to open an initialization section in playlist 0
[hls @ 0x7f3528000b80] Error when loading first segment 'manifest3.m4s'
manifest.m3u8: No such file or directory

(b). On to DASH:

If we rename OR delete the target directories to which the underlying DASH muxer is writing to with the aforementioned muxer options, the tee muxer slaves (fifo'd) claims that the recovery is successful, BUT with the following spurious error in the console:

[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20977892936 >= 20957937
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20978894937 >= 20958938
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
^C[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20979896938 >= 20959939
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream1-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream2-00001.m4s.tmp' for writing
[hls @ 0x55e25c912000] Opening '/home/brainiarc7/Desktop/src/test/H264/Gear-9/manifest2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream3-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream4-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream5-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream6-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream7-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream8-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream9-00001.m4s.tmp' for writing
[dash @ 0x55e25c970280] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 20980898939 >= 20960940
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/manifest.mpd.tmp' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream0.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream1.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream2.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream3.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream4.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream5.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream6.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream7.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream8.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/init-stream9.m4s' for writing
[dash @ 0x55e25c970280] Opening '/home/brainiarc7/Desktop/src/test/DASH/chunk-stream0-00001.m4s.tmp' for writing
[fifo @ 0x55e25c9661c0] Recovery successful

Which is then repeated non-stop: "Application provided invalid, non monotonically increasing dts to muxer in stream 0"

The content rendered there-in, even after the directory is renamed or recreated with the original name is definitely unplayable. Both ffprobe and mpv die on attempting playback:

ffplay manifest.mpd 
ffplay version N-95809-g804fce8bc2 Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
  configuration: --pkg-config-flags=--static --enable-static --disable-shared --prefix=/home/brainiarc7 --bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/include --extra-ldflags=-L/home/brainiarc7/lib --enable-cuda-nvcc --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/ --extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc --enable-libass --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-opencl --enable-libxml2 --enable-gpl --cpu=native --enable-opengl --enable-libfdk-aac --enable-libx264 --enable-libx265 --enable-openssl --enable-librtmp --enable-libpulse --enable-libzvbi --enable-librav1e --enable-pic --nvccflags='-gencode arch=compute_75,code=sm_75 -O2' --extra-libs='-lpthread -lm -lz -ldl' --enable-nonfree
  libavutil      56. 36.100 / 56. 36.100
  libavcodec     58. 62.100 / 58. 62.100
  libavformat    58. 35.100 / 58. 35.100
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 66.100 /  7. 66.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[dash @ 0x7fb1dc000b80] decoding for stream 0 failed=    0B f=0/0   
[dash @ 0x7fb1dc000b80] decoding for stream 1 failed
[dash @ 0x7fb1dc000b80] decoding for stream 2 failed
[dash @ 0x7fb1dc000b80] decoding for stream 3 failed
[dash @ 0x7fb1dc000b80] decoding for stream 4 failed
[dash @ 0x7fb1dc000b80] decoding for stream 5 failed
[dash @ 0x7fb1dc000b80] decoding for stream 6 failed
[dash @ 0x7fb1dc000b80] decoding for stream 7 failed
[dash @ 0x7fb1dc000b80] decoding for stream 8 failed
Input #0, dash, from 'manifest.mpd':
  Duration: 00:00:33.00, start: 1397.396000, bitrate: 1 kb/s
  Program 0 
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 7800000
      id              : 0
    Stream #0:1: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 6000000
      id              : 1
    Stream #0:2: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 4500000
      id              : 2
    Stream #0:3: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 3000000
      id              : 3
    Stream #0:4: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 960x540 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 2000000
      id              : 4
    Stream #0:5: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 768x432 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 1100000
      id              : 5
    Stream #0:6: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 768x432 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 730000
      id              : 6
    Stream #0:7: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 640x360 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 365000
      id              : 7
    Stream #0:8: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 416x234 [SAR 1:1 DAR 16:9], Closed Captions, 0 kb/s, 29.97 tbr, 30k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 145000
      id              : 8
    Stream #0:9: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 0 kb/s
    Metadata:
      variant_bitrate : 128000
      id              : 9
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 0
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 2
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 3
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 4
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 5
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 6
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 7
[dash @ 0x7fb1dc000b80] No longer receiving stream_index 8

And with mpv:

mpv manifest.mpd 
 (+) Video --vid=1 'bitrate 7800000' (h264 1920x1080)
     Video --vid=2 'bitrate 6000000' (h264 1920x1080)
     Video --vid=3 'bitrate 4500000' (h264 1280x720)
     Video --vid=4 'bitrate 3000000' (h264 1280x720)
     Video --vid=5 'bitrate 2000000' (h264 960x540)
     Video --vid=6 'bitrate 1100000' (h264 768x432)
     Video --vid=7 'bitrate 730000' (h264 768x432)
     Video --vid=8 'bitrate 365000' (h264 640x360)
     Video --vid=9 'bitrate 145000' (h264 416x234)
 (+) Audio --aid=1 'bitrate 128000' (aac 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float


Exiting... (End of file)

In conclusion: The tee (and fifo muxer combination) cannot transparently recover HLS (with mp4 as segment format) and DASH streams, as documented above.

comment:3 by Dennis E. Mungai, 4 years ago

Resolution: invalid
Status: newclosed

Update:

This ticket should be marked as invalid.

My research on this topic indicates that to enable transparent recovery for HLS and DASH streams via tee (and fifo) muxer(s), one must pass the following options:

HLSOPTS=':use_fifo=1:fifo_options=recovery_wait_time=1\\:attempt_recovery=1\\:drop_pkts_on_overflow=1\\:recover_any_error=1:f=hls:hls_wrap=6:hls_time=6:hls_list_size=6:hls_segment_type=fmp4:ignore_io_errors=0'
DASHOPTS='use_fifo=1:fifo_options=recovery_wait_streamtime=1\\:attempt_recovery=1\\:drop_pkts_on_overflow=1\\:recover_any_error=1:f=dash:seg_duration=6:window_size=6:extra_window_size=0:ignore_io_errors=0'

With tee spawning fifo'd slaves, the underlying muxer *must* be allowed to fail so that fifos can recover. See the "ignore_io_errors=0" option passed to the underlying HLS and tee muxer(s) spawned by fifo'd tees.

Secondly, the recovery logic must be set to rely on the heuristics based on recovery wait time, which requires the drop packets on overflow option to be enabled.

Without these options enabled, both HLS and DASH muxers cannot recover transparently when slaved to tee fifos.

What I was observing was NOT a bug, as initially assumed, but the correct behavior based on the options set in the tee's fifo slaves.

Marking ticket as closed.

Apologies for the noise.

Note: See TracTickets for help on using tickets.