Seeking in MPEG TS with copyts overflows for 13+ hours

[Radek Hladík]

Summary of the bug:
I have a 24hour long recording in MPEGTS. I need to cut small segments at arbitrary times. For some reason the keyframe accurate seeking is not working for me, so I am trying to find the exact keyframe by myself.
First I cut a few minutes back with ffmpeg -c copy -copyts. Then I use ffprobe to detect the last keyframe before the requested time. Then call ffmpeg with exact -ss to cut the file.
Everything works fine for times less than cca 13:15:22. With times larger, something probably overflows. The time is exactly 4294980000 with timebase 90000...

How to reproduce:
Step 1: Create long file (13+ hours), i.e.:

% ffmpeg -t 57600 -f lavfi -i color=color=red -f lavfi -i sine=frequency=1000 -filter_complex "[1:a][1:a]amerge=inputs=2[aout]" -map 0:v -map "[aout]" -shortest -c:v h264_nvenc -preset fast  -c:a aac -ac 2 -vbr 2 -r 25 -f mpegts -y 16hours.ts

ffmpeg version ffmpeg-2021-10-30-git-d92fdc7144-full_build from ​https://www.gyan.dev/ffmpeg/builds/

Step 2: Seek to 13:00:00 and "cut" 1 minute with -copyts:

% ffmpeg  -ss 13:00:00.0 -i 16hours.ts  -to 13:01:00  -c copy -copyts test_ok_ts.ts

ffmpeg version 2021-10-30-git-d92fdc7144-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mpegts, from '16hours.ts':
  Duration: 16:00:00.02, start: 1.400000, bitrate: 150 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 99 kb/s
Output #0, mpegts, to 'test_ok_ts.ts':
  Metadata:
    encoder         : Lavf59.8.100
  Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 99 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=-1.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
frame= 1465 fps=0.0 q=-1.0 Lsize=    1227kB time=00:00:58.79 bitrate= 170.9kbits/s speed=3.18e+03x    
video:49kB audio:722kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 59.008598%

Step 2a: verify key frames in the created video:

% ffprobe version 2021-10-30-git-d92fdc7144-full_build-www.gyan.dev Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mpegts @ 000000febcf21b40] decoding for stream 0 failed
Input #0, mpegts, from 'test_ok_ts.ts':
  Duration: 00:00:58.83, start: 46802.595378, bitrate: 170 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 101 kb/s
46802.823222
46812.823222
46822.823222
46832.823222
46842.823222
46852.823222

Step 3: do the same for 13:15:23

% ffmpeg  -ss 13:15:23 -i 16hours.ts  -to 13:16:23  -c copy -copyts test_bad_ts.ts >a3 2>&1
ffmpeg version 2021-10-30-git-d92fdc7144-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, mpegts, from '16hours.ts':
  Duration: 16:00:00.02, start: 1.400000, bitrate: 150 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 98 kb/s
Output #0, mpegts, to 'test_bad_ts.ts':
  Metadata:
    encoder         : Lavf59.8.100
  Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 25 fps, 25 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 98 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=-1.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
frame=34102 fps=0.0 q=-1.0 size=   28416kB time=-12:52:28.22 bitrate=N/A speed=N/A    
frame=68068 fps=68068 q=-1.0 size=   56832kB time=-12:29:49.61 bitrate=N/A speed=N/A    
frame=103829 fps=69219 q=-1.0 size=   86528kB time=-12:05:59.17 bitrate=N/A speed=N/A    
frame=139021 fps=69510 q=-1.0 size=  115968kB time=-11:42:31.43 bitrate=N/A speed=N/A    
frame=175232 fps=70092 q=-1.0 size=  146176kB time=-11:18:23.01 bitrate=N/A speed=N/A    
frame=211451 fps=70483 q=-1.0 size=  176384kB time=-10:54:14.29 bitrate=N/A speed=N/A    
frame=246750 fps=70831 q=-1.0 Lsize=  205863kB time=-10:30:42.29 bitrate=N/A speed=N/A    
video:8299kB audio:120721kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 59.559387%

Step 3b:

% ffprobe  -skip_frame nokey -select_streams v:0 -show_entries frame=pkt_dts_time -of csv=print_section=0 test_bad_ts.ts
ffprobe version 2021-10-30-git-d92fdc7144-full_build-www.gyan.dev Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 16.100 /  8. 16.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mpegts @ 0000007c24ad1b40] decoding for stream 0 failed
Input #0, mpegts, from 'test_bad_ts.ts':
  Duration: 02:44:37.23, start: 1.400000, bitrate: 170 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 97 kb/s
8.634644
18.634644
28.634644
....
9848.634644
9858.634644
9868.634644

Some more info:

• The progress timer shows negative times.
• The resulting bad file is also much larger, probably the whole rest of the input file.
• The timestamps starts from 0 in the bad file.
• Without -copyts it works fine
• This testing is from Windows, but I've discovered the problem on linux (with older ffmpeg and with freshly compiled ffmpeg from git).
• For some reason -to needs to be set to "endtime" with -copyts while it have to be duration without -copyts
• If -vframes 100 is specified then ffmpeg stops after 100 frames - but still showing negative times and timestamps are reset.
• I know that mpegts timestamp have the resolution of cca 26.5 hours, but this problem occurs exactly in the half.

[Balling]

I know that mpegts timestamp have the resolution of cca 26.5 hours, but this problem occurs exactly in the half.

Not that simple, 2^{33 / 90000 seconds = 26 hours 30 minutes 43.718 seconds. There is rollover support, but not perfect. #7876, #7449, #8689.}

[Radek Hladík]

Replying to Balling:

I know that mpegts timestamp have the resolution of cca 26.5 hours, but this problem occurs exactly in the half.

Not that simple, 2^{33 / 90000 seconds = 26 hours 30 minutes 43.718 seconds. There is rollover support, but not perfect. #7876, #7449, #8689.}

I did not want to clutter the original ticket message with the rollover too much as I think that it is an unrelated problem. I've read through a lot of tickets in this tracker before posting, I even tried the patch from #7876 and it did not change a thing.

As far as I understand the rollover, the pts is 33-bit long unsigned number with 90000 timebase. However my problem occurs almost exactly at 2³² /90000=4294967296/90000 = 47721.85884 seconds = 13.2560719012346 hours. So it looks like to me that the 33rd bit gets lost somewhere and/or is converted into sign but I do not want to speculate as I do not know the ffmpeg code....

Also if I just copy the whole file without seeking, then everything works fine:

% ffmpeg  -i 16hours.ts   -c copy -copyts test_copy_ts.ts >a6 2>&1

and even the progress time is fine:

frame=1440000 fps=76557 q=-1.0 Lsize= 1200748kB time=15:59:59.99 bitrate= 170.8kbits/s speed=3.06e+03x    

[Balling]

Actually #7449 is specifically said to be near rollover not after it. But it has PCR (i.e. additional 9 bits to 33 bits of which 0-299 can be used, not all 512 values).

[Balling]

Nice related issue ​https://github.com/mpv-player/mpv/issues/7816

Also see #7673