Opened 2 years ago

Last modified 22 months ago

#9446 new defect

When video pid is "suspended" on input transport stream, ffmpeg creates equally long audio/video delay on output

Reported by: Haris Zukanovic Owned by:
Priority: normal Component: undetermined
Version: 4.3.2 Keywords: mpegts input discontinuity audio/video delay
Cc: Haris Zukanovic Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description (last modified by Haris Zukanovic)

Summary of the bug:

Transport streams (satellite received in this particular case, live stream) seem to allow for "suspending" of a pid during the broadcast. It seems that the PID simply carries no packets for aperiod of time, during which I assume video would simply not be played in a sat receiver.

I discovered this by capturing the .ts and analyzing it with tsduck tools. tsduck refers to this as "suspended/resumed" PID

captured .ts file
https://drive.google.com/file/d/1b8K6Gbm1I6v6Ik6wYNRsBbCW1QZfWojF/view?usp=sharing

in this case PID 1001 is the video stream on input.
tsp -v -m --synchronous-log -I file cut.ts -P history -O drop 2>&1 | grep "PID 1001"

  • history: 3: PID 1001 (0x03E9) first packet, clear
  • history: 26: PID 1001 (0x03E9), PES stream_id is 0xE0 (Video 0)
  • history: 49345: PID 1001 (0x03E9) suspended, clear, service 0x0001
  • history: 247381: PID 1001 (0x03E9) restarted, clear, service 0x0001
  • history: 300000: PID 1001 (0x03E9) last packet, clear

When this happens on input, ffmpeg produces an audio delay with respect to video on output.

The delay can be verified with mediainfo
mediainfo out0.ts out1.ts out2.ts out3.ts out4.ts | grep -i delay
Delay relative to video : -21 ms
Delay relative to video : 6 ms
Delay relative to video : 11 ms
Delay relative to video : 1 min 15 s
Delay relative to video : 1 min 15 s

I suspect something similar would happen is audio was "suspended"...

The expected result might be that a black picture is inserted instead of missing video, and silence inserted for missing audio... while the interruption is ongoing..
Any way we can fix this?

How to reproduce:

% 
ffmpeg version 4.4
built on Linux Ubuntu 19


ffmpeg -hide_banner -loglevel info -nostats -i cut.ts -preset ultrafast -f hls out.m3u8
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] sps_id 0 out of range
[h264 @ 0x7f8bef01d000] non-existing PPS 0 referenced
[h264 @ 0x7f8bef01d000] decode_slice_header error
[h264 @ 0x7f8bef01d000] no frame!
[mpegts @ 0x7f8bef808200] PES packet size mismatch
[mpegts @ 0x7f8bef808200] Packet corrupt (stream = 1, dts = 6307498633).
Input #0, mpegts, from 'cut.ts':
  Duration: 00:02:09.89, start: 69954.438144, bitrate: 3473 kb/s
  Program 1 
    Metadata:
      service_name    : A: Al Jazeera_FS
      service_provider: MKT
  Stream #0:0[0x3e9]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, bottom first), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn, 50 tbc
  Stream #0:1[0x3ea](bos): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 192 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x7f8bed820400] using SAR=16/11
[libx264 @ 0x7f8bed820400] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f8bed820400] profile Constrained Baseline, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x7f8bed820400] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, hls, to 'out.m3u8':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264, yuv420p(tv, bt709, bottom coded first (swapped)), 720x576 [SAR 16:11 DAR 20:11], q=2-31, 25 fps, 90k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(bos): Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc58.134.100 aac
[hls @ 0x7f8bed82ee00] Opening 'out0.ts' for writing
[hls @ 0x7f8bed82ee00] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7f8bed82ee00] Opening 'out1.ts' for writing
[hls @ 0x7f8bed82ee00] Opening 'out.m3u8.tmp' for writing
[hls @ 0x7f8bed82ee00] Opening 'out2.ts' for writing
[hls @ 0x7f8bed82ee00] Opening 'out.m3u8.tmp' for writing
[h264 @ 0x7f8bed984c00] Reference 3 >= 2
[h264 @ 0x7f8bed984c00] error while decoding MB 6 35, bytestream 177
[hls @ 0x7f8bed82ee00] Opening 'out3.ts' for writing
[hls @ 0x7f8bed82ee00] Opening 'out.m3u8.tmp' for writing
[mpegts @ 0x7f8bef808200] PES packet size mismatch
[mpegts @ 0x7f8bef808200] Packet corrupt (stream = 1, dts = 6307498633).
[h264 @ 0x7f8bed965000] error while decoding MB 5 16, bytestream -8
cut.ts: corrupt decoded frame in stream 0
[hls @ 0x7f8bed82ee00] Opening 'out4.ts' for writing
[hls @ 0x7f8bed82ee00] Opening 'out.m3u8.tmp' for writing
frame= 1118 fps=690 q=-1.0 Lsize=N/A time=00:02:08.89 bitrate=N/A dup=42 drop=2 speed=79.5x    
video:11964kB audio:748kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x7f8bed820400] frame I:5     Avg QP:18.40  size: 40116
[libx264 @ 0x7f8bed820400] frame P:1113  Avg QP:21.19  size: 10827
[libx264 @ 0x7f8bed820400] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x7f8bed820400] mb P  I16..4:  6.4%  0.0%  0.0%  P16..4: 30.9%  0.0%  0.0%  0.0%  0.0%    skip:62.7%
[libx264 @ 0x7f8bed820400] coded y,uvDC,uvAC intra: 28.2% 31.4% 10.8% inter: 15.5% 6.5% 0.8%
[libx264 @ 0x7f8bed820400] i16 v,h,dc,p: 50% 28% 11% 11%
[libx264 @ 0x7f8bed820400] i8c dc,h,v,p: 51% 22% 22%  5%
[libx264 @ 0x7f8bed820400] kb/s:2191.58
[aac @ 0x7f8bed983a00] Qavg: 42202.176

Change History (2)

comment:1 by Haris Zukanovic, 2 years ago

Description: modified (diff)

comment:2 by Marton Balint, 22 months ago

Priority: importantnormal
Note: See TracTickets for help on using tickets.