no #EXT-X-DISCONTINUITY is added to hls manifest is source is -f concat

[larez]

Summary of the bug:

transmuxing a "-f concat -i concat.lst" to hls
should add #EXT-X-DISCONTINUITY after each list item

How to reproduce:

1) create concat list of ​http://...m3u8 files :

list.txt :
file '​http://../1.m3u8'
file '​http://../2.m3u8'
..
file '​http://../n.m3u8'

2)create master m3u8
cat <<'IDXEOF' > index.m3u8
#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2400000
hd.m3u8
IDXEOF

3) transmux
h_flags='delete_segments+append_list+omit_endlist'
hls_opts="-f hls -hls_time 4 -hls_list_size 10 -hls_flags $h_flags -hls_allow_cache 0 -start_number 1"

ffmpeg -v verbose -y -f concat -safe 0 -protocol_whitelist "file,http,https,tcp,tls" -re -i list.txt -vstats_file stats.txt -map 0:v:1 -c:v copy -copyts -start_at_zero -metadata:s:0 language=eng -map 0:a:1 -c:a copy -copyts -start_at_zero -metadata:s:1 language=eng -flags -global_header -hls_segment_filename sd_%05d.ts $hls_opts 'hd.m3u8' -map 0:a:1 -c:a copy -copyts -start_at_zero -metadata:s:1 language=eng -flags -global_header -hls_segment_filename hd_%05d.ts $hls_opts 'hd.m3u8'

ffmpeg creates then a readable live (vlc, kaltura hls test player accept ​http://your_hls_live_server/master m3u8) if all m8u8 source items from concat list were transcoded in the same way, but it would be better to write
#EXT-X-DISCONTINUITY into hd.m3u8 (in this case) after each
new source read from the list in case if encoding (or resoution) change. ffmpeg signals each new list read in its log, so, the logic could be easily added, e.g.
if source is concat and output is hls => signal discontinuity after the end of each source from the list

[larez]

oups, messed up the command line, the right one:

ffmpeg -v verbose -y -f concat -safe 0 -protocol_whitelist "file,http,https,tcp,tls" -re -i list.txt -vstats_file stats.txt -map 0:v:1 -c:v copy -copyts -start_at_zero -metadata:s:0 language=eng -map 0:a:1 -c:a copy -copyts -start_at_zero -metadata:s:1 language=eng -flags -global_header -hls_segment_filename hd_%05d.ts $hls_opts 'hd.m3u8'

[Timo R.]

For all I'm aware, the concat demuxer can only be used if the files it's concatenating would produce a valid media file if concated naively with cat.
It will take it, but produce technically invalid output and the mileage with that may vary.
So a bunch of otherwise unrelated sources need the concat filter and in turn re-encoding.

[Carl Eugen Hoyos]

No, after all if you concatenate files that can be concatenated you don’t want the discontinuity flag.

[larez]

here concat filter is useful for very simple http vod to live playout, it works just in my example, without transcoding you can "concat" heterogenous mix of vod hls sources of any origine/resolution/bitrate to live hls. you can do the same in a shell loop, but -f concat would be "safer" than respawning ffmpeg 1000 times and you get easy track of time (ffmpeg does it for you) spend from the beginning of your playout, it helps if you need to recover playout in case of crash. the only thing missing for this "live playoyt hack" to be hls compliant - insert EXT-X-DISCONTINUITY into playlist on item change. well, sure, i can make a custom patch just for myself...

[Timo R.]

It's not a trivial problem. After the concat demuxer is done, there is no notion anymore of what each individual item was.
So "just putting a discontinuity tag there" is not a trivial task.
And like previously stated, the concat demuxer does not support concating otherwise unrelated input sources.

[Gyan]

*For all I'm aware, the concat demuxer can only be used if the files it's concatenating would produce a valid media file if concated naively with cat.*

Only the stream parameters have to match. Ideally, the timebase should as well, for correct playback. Input containers are irrelevant.

*It's not a trivial problem. After the concat demuxer is done, there is no notion anymore of what each individual item was.*

It's not that hopeless. The concat demuxer does have a directive to attach string metedata to each packet and that directive is specified per-file. I've used it to get drawtext to print the name of the source file for the current frame within a concat input stream. It's a matter of the muxer seeking and acting upon that info. The pipeline is available.