HLS multi bitrate streaming, timestamps differ when one of the video track is copied

[krs_vital]

command:

/usr/local/bin/ffmpeg -hwaccel cuvid -c:v h264_cuvid -deint adaptive -dts_delta_threshold 3600 -err_detect ignore_err -correct_ts_overflow 0 -overrun_nonfatal 1 -i udp://239.192.7.105:1234 -y -sn -dn -master_pl_name TEST.m3u8 -map v:0 -c:v:0 copy -map a:0 -c:a:0 aac -map a:1 -c:a:1 aac -map a:2 -c:a:2 copy -map v:0 -c:v:1 h264_nvenc -b:v:1 3M -map a:0 -c:a:3 aac -map a:1 -c:a:4 aac -map a:2 -c:a:5 copy -map v:0 -c:v:2 h264_nvenc -b:v:2 1M -map a:0 -c:a:6 aac -map a:1 -c:a:7 aac -map a:2 -c:a:8 copy -var_stream_map "v:0,a:0,a:1,a:2 v:1,a:3,a:4,a:5 v:2,a:6,a:7,a:8" -ignore_unknown -strict -2 -strftime 1 -strftime_mkdir 1 -r 50 -hls_time 10 -hls_flags omit_endlist -hls_segment_filename /store/records/TEST/%Y/%m/%d/%H/%s_%v.ts /hls/tmp/TEST_%v.m3u8

...

[h264 @ 0x33f6e00] Increasing reorder buffer to 2
WARNING: defaulting hwaccel_output_format to cuda for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format cuda".
Input #0, mpegts, from 'udp://239.192.7.105:1234':

Duration: N/A, start: 75462.044967, bitrate: N/A
Program 812

Metadata:

service_name : Amedia Premium HD
service_provider: Orion Express

Stream #0:1[0x714]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:2[0xafc](rus): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:3[0xee4](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 160 kb/s
Stream #0:4[0xf16](rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:5[0x12cc](rus): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)

No Program

Stream #0:0[0x12]: Data: epg

Stream mapping:

Stream #0:1 -> #0:0 (copy)
Stream #0:2 -> #0:1 (mp2 (native) -> aac (native))
Stream #0:3 -> #0:2 (mp2 (native) -> aac (native))
Stream #0:4 -> #0:3 (copy)
Stream #0:1 -> #0:4 (h264 (h264_cuvid) -> h264 (h264_nvenc))
Stream #0:2 -> #0:5 (mp2 (native) -> aac (native))
Stream #0:3 -> #0:6 (mp2 (native) -> aac (native))
Stream #0:4 -> #0:7 (copy)
Stream #0:1 -> #0:8 (h264 (h264_cuvid) -> h264 (h264_nvenc))
Stream #0:2 -> #0:9 (mp2 (native) -> aac (native))
Stream #0:3 -> #0:10 (mp2 (native) -> aac (native))
Stream #0:4 -> #0:11 (copy)

Press [q] to stop, ? for help
Output #0, hls, to '/hls/tmp/TEST_%v.m3u8':

Metadata:

encoder : Lavf58.65.101
Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1(rus): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:2(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:3(rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:4: Video: h264 (Main), cuda(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 4000 kb/s, 50 fps, 90k tbn
Metadata:

encoder : Lavc58.119.100 h264_nvenc

Side data:

cpb: bitrate max/min/avg: 0/0/4000000 buffer size: 8000000 vbv_delay: N/A

Stream #0:5(rus): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:6(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:7(rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:8: Video: h264 (Main), cuda(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 50 fps, 90k tbn
Metadata:

encoder : Lavc58.119.100 h264_nvenc

Side data:

cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A

Stream #0:9(rus): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:10(eng): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

encoder : Lavc58.119.100 aac

Stream #0:11(rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s

MASTER PLAYLIST: (cat /hls/tmp/TEST.m3u8)
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=422400,RESOLUTION=1920x1080
TEST_0.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=4822400,RESOLUTION=1920x1080,CODECS="avc1.4d402a,mp4a.40.2,ac-3"
TEST_1.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=2622400,RESOLUTION=1920x1080,CODECS="avc1.4d402a,mp4a.40.2,ac-3"
TEST_2.m3u8

cat /hls/tmp/TEST_0.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:9.600000,
/store/records/TEST/2021/02/26/08/1614328701_0.ts
#EXTINF:9.120000,
/store/records/TEST/2021/02/26/08/1614328706_0.ts
#EXTINF:10.440000,
/store/records/TEST/2021/02/26/08/1614328715_0.ts
#EXTINF:10.040000,
/store/records/TEST/2021/02/26/08/1614328725_0.ts

cat /hls/tmp/TEST_1.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328701_1.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328705_1.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328715_1.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328725_1.ts

cat /hls/tmp/TEST_2.m3u8
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328701_2.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328705_2.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328715_2.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328725_2.ts
#EXTINF:10.000000,
/store/records/TEST/2021/02/26/08/1614328735_2.ts

if video copy, timer in #EXTINF tag differ from tag in chunk which has been encoded, so smoothly switching between playlists is not possible.

ffprobe /store/records/TEST/2021/02/26/08/1614328735_0.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_0.ts':

Duration: 00:00:10.31, start: 41.698667, bitrate: 3225 kb/s
Program 1

Metadata:

service_name : Service01
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1[0x101](rus): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
Stream #0:2[0x102](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
Stream #0:3[0x103](rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_1.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_1.ts':

Duration: 00:00:10.02, start: 41.421333, bitrate: 5397 kb/s
Program 1

Metadata:

service_name : Service01
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](rus): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 132 kb/s
Stream #0:2[0x102](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
Stream #0:3[0x103](rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_2.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_2.ts':

Duration: 00:00:10.02, start: 41.421333, bitrate: 2226 kb/s
Program 1

Metadata:

service_name : Service01
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream #0:1[0x101](rus): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 132 kb/s
Stream #0:2[0x102](eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 131 kb/s
Stream #0:3[0x103](rus): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s

[krs_vital]

ffprobe /store/records/TEST/2021/02/26/08/1614328735_0.ts
Duration: 00:00:10.31, start: 41.698667, bitrate: 3225 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_1.ts
Duration: 00:00:10.02, start: 41.421333, bitrate: 5397 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_2.ts
Duration: 00:00:10.02, start: 41.421333, bitrate: 2226 kb/s

Start time in "0" playlist version differ from "1" and "2":

41.698667 [ 0 ] <-> 41.421333 [ 1, 2 ]

[Carl Eugen Hoyos]

Please test current FFmpeg git and provide your simplified command line together with the complete, uncut console output to make this a valid ticket.

[Steven Liu]

Replying to krs_vital:

ffprobe /store/records/TEST/2021/02/26/08/1614328735_0.ts
Duration: 00:00:10.31, start: 41.698667, bitrate: 3225 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_1.ts
Duration: 00:00:10.02, start: 41.421333, bitrate: 5397 kb/s

ffprobe /store/records/TEST/2021/02/26/08/1614328735_2.ts
Duration: 00:00:10.02, start: 41.421333, bitrate: 2226 kb/s

Start time in "0" playlist version differ from "1" and "2":

41.698667 [ 0 ] <-> 41.421333 [ 1, 2 ]

ffprobe /store/records/TEST/2021/02/26/08/1614328735_0.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_0.ts':
Duration: 00:00:10.31, start: 41.698667, bitrate: 3225 kb/s
Program 1
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc


ffprobe /store/records/TEST/2021/02/26/08/1614328735_1.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_1.ts':
Duration: 00:00:10.02, start: 41.421333, bitrate: 5397 kb/s
Program 1
Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc


ffprobe /store/records/TEST/2021/02/26/08/1614328735_2.ts
Input #0, mpegts, from '/store/records/TEST/2021/02/26/08/1614328735_2.ts':
Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc

look at first stream info, the fps tbc is different to another two streams, maybe ffprobe show_packets look all of three streams and analyze the different should more clear.

[krs_vital]

the first stream is copy of the main stream, but 1 and 2 is streams from encoder, i think therefore timestamps are differ, it can be seen in playlists by #EXTINF tag:

0:
#EXTINF:9.600000,
#EXTINF:9.120000,
#EXTINF:10.440000,
#EXTINF:10.040000,

1:
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,

2:
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,

but why hls segmenter cut main stream by different timestamps?

[Steven Liu]

Replying to krs_vital:

the first stream is copy of the main stream, but 1 and 2 is streams from encoder, i think therefore timestamps are differ, it can be seen in playlists by #EXTINF tag:

0:
#EXTINF:9.600000,
#EXTINF:9.120000,
#EXTINF:10.440000,
#EXTINF:10.040000,

1:
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,

2:
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,
#EXTINF:10.000000,

but why hls segmenter cut main stream by different timestamps?

Maybe not only hls segmenter, you can try use -f segment
or just use three task process single every stream, maybe have same result.

[krs_vital]

i cant use 3 tasks because they will not be synced and player reading master playlist cant swithing between playlists without lags or errors
-f segment dosnt generate master playlist which is needed for me