Captured HD DVR (HDHomeRun) streams (in the UK) - wrongly identified as 1080p when it should be 1080i

[minxster]

Summary of the bug: ffprobe is miss-identifying a DVR recording as Progressive (1080p) when it should be 1080i. In the UK, HD TV channels can broadcast a mix of 1080i and 1080p. E.g. the main event (sports match) can be in 1080i, but adverts may (not always) be 1080p. I'm assuming this mix of formats is what is causing the issue, especially not knowing how formats are identified? To verify, if I play the file back in VLC, and turn off de-interlace, the video of the sports match is clearly interlaced (you can see the horizontal lines); if I manually turn on de-interlace, the video plays perfectly.

How to reproduce: This is difficult unless you can capture UK Freeview (HD) TV. I am a noob at all of this, but I do have a 6gb (over 3hrs) file that is being miss-identified. I'm happy to try and chop some of the video down (with help from you?), but the resulting filesize may still be big. I did manage to isolate a section of the main sports match, and ffprobe then clearly identified it as interlaced/1080i.

---

Output from a full commands I ran (ffprobe + ffmpeg idet:

C:\dump>\ffmpeg\bin\ffprobe "Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts"
ffprobe version 6.0-full_build-www.gyan.dev Copyright (c) 2007-2023 the FFmpeg developers

built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100

[mpegts @ 0000027364e2e2c0] start time for stream 2 is not set in estimate_timings_from_pts
Input #0, mpegts, from 'Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts':

Duration: 02:44:57.62, start: 1.400000, bitrate: 5232 kb/s
Program 1

Metadata:

service_name : Service01
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn
Stream #0:1[0x101](eng): Audio: aac_latm (LC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:2[0x102](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

C:\dump>C:\ffmpeg\bin\ffmpeg -filter:v idet -f rawvideo -y NUL -i "Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts"
ffmpeg version 6.0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers

built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100

[mpegts @ 000002650e45d380] start time for stream 2 is not set in estimate_timings_from_pts
Input #0, mpegts, from 'Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts':

Duration: 02:44:57.62, start: 1.400000, bitrate: 5232 kb/s
Program 1

Metadata:

service_name : Service01
service_provider: FFmpeg

Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn
Stream #0:1[0x101](eng): Audio: aac_latm (LC) ([17][0][0][0] / 0x0011), 48000 Hz, stereo, fltp
Stream #0:2[0x102](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))

Press [q] to stop, ? for help
Output #0, rawvideo, to 'NUL':

Metadata:

encoder : Lavf60.3.100

Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 622080 kb/s, 25 fps, 25 tbn

Metadata:

encoder : Lavc60.3.100 rawvideo

Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts: corrupt decoded frame in stream 0

Last message repeated 1 times

Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts: corrupt decoded frame in stream 0
Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts: corrupt decoded frame in stream 0

Last message repeated 1 times

Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts: corrupt decoded frame in stream 0

Last message repeated 38 times

... repeats ...

frame=247402 fps=850 q=-0.0 Lsize=751483575kB time=02:44:56.04 bitrate=622082.5kbits/s speed= 34x
video:751483575kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
[Parsed_idet_0 @ 0000026511e4ef00] Repeated Fields: Neither:247249 Top: 100 Bottom: 53
[Parsed_idet_0 @ 0000026511e4ef00] Single frame detection: TFF:186225 BFF: 565 Progressive: 43578 Undetermined: 17034
[Parsed_idet_0 @ 0000026511e4ef00] Multi frame detection: TFF:196512 BFF: 479 Progressive: 50411 Undetermined: 0
C:\dump>

---

I don't know if it helps, but if I probe the (whole) file with MediaInfo, it correctly identifies the format (and so do for all the sports matches I recorded (in that the are all 1080i))

MediaInfo (v22.12) Output:
root@plex-ubuntu:/media/truenas_raidz/TV/HDHomeRun/Live Six Nations Rugby (2011)/Season 2023# mediainfo Live\ Six\ Nations\ Rugby\ \(2011\)\ -\ 2023-03-18\ 12\ 00\ 00\ -\ France\ v\ Wales.ts
General
ID : 1 (0x1)
Complete name : Live Six Nations Rugby (2011) - 2023-03-18 12 00 00 - France v Wales.ts
Format : MPEG-TS
File size : 6.03 GiB
Duration : 2 h 44 min
Overall bit rate mode : Variable
Overall bit rate : 5 233 kb/s

Video
ID : 256 (0x100)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : 27
Duration : 2 h 44 min
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Scan type, store method : Interleaved fields
Scan order : Top Field First
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Muxing mode : LATM
Codec ID : 17-2
Duration : 2 h 44 min
Bit rate mode : Variable
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Delay relative to video : -1 s 539 ms
Language : English

Text
ID : 258 (0x102)
Menu ID : 1 (0x1)
Format : DVB Subtitle
Codec ID : 6
Duration : 2 h 44 min
Delay relative to video : 4 s 480 ms
Language : English

Menu
ID : 4096 (0x1000)
Menu ID : 1 (0x1)
Format : AVC / AAC / DVB Subtitle
Duration : 2 h 44 min
List : 256 (0x100) (AVC) / 257 (0x101) (AAC, English) / 258 (0x102) (DVB Subtitle, English)
Language : / English / English
Service name : Service01
Service provider : FFmpeg
Service type : digital television

[Marton Balint]

I am not sure what is the expected outcome. You wrote it yourself, the stream contains both progressive and interlaced content.

What ffprobe reports is most likely based on a small section (in the beginning) of the file which it uses to probe the h264 parameters. A smart encoder can choose the frame encoding mode (interlaced or progressive) on a per-scene basis, so what ffprobe reports should depend on the encoding mode of the first scene of your capture.

Why is this an issue for you by the way? Can't you just handle the content as if it would be interlaced all along?

[minxster]

I'm working on a set of scripts that can detect the files correctly and then decide whether they need transcoding/re-encoding or not. Currently files passed to things like Plex or Jellyfin are hit or miss on whether they will play correctly. In the problem/case I posted about, both pieces of software incorrectly identified the file and playback was badly affected. And ironically, both are using a form (I know Plex has it's own custom version) of FFmpeg!

Having a better scanning/checking facility, I think, is needed. Ideally we'd want to know if things are: (a) interlaced (b) progressive or (c) mixed. Maybe this needs to be an extra option of a "deep scan"? But clearly there are problems if files are miss-identified, wither it be media playback system, or people (like me) wanting to process the files correctly.

By having FFmpeg/probe better identify file(s), I can at least have scripts to clear up the problem. Let alone the other dependency systems/apps being better off.

[Marton Balint]

You assume that the identified parameters are constant during the whole file. This is not guaranteed at all. Aspect ratio change is quite common, but it is even possible for the resolution to change mid-stream.

If you are interested in deep scanning the file, then you should use ffprobe -show_frames for that. That should correctly identify interlaced/progressive for every frame, and you can check that with your scripts and act accordingly.

If something does not work as expected with Plex/Jellyfin, I suggest opening tickets there. Although I kind of assumed that Plex/Jellyfin are media servers only, so unless they do some magic transcoding on-demand, the player should present the interlaced or non-interlaced content properly. So it is also possible that the player itself you use is broken, and not the media servers.

[Balling]

HD TV channels can broadcast a mix of 1080i and 1080p

Not supported: same issue in HEVC #4141 and same issue in old MPEG2 on DVDs (​https://trac.ffmpeg.org/ticket/3968#comment:7 and #2602).

[minxster]

@Marton Balint : you raise a good point with other fields possibly changing, but from what I've tested, aspect and resolution don't change, at least not in the UK (as far as I can tell).

That said, I hear you about using the -show_frames, I'll take a look at that and see what I can do with it.

The media server are clearly using ffmpeg to check the file(s) and then they proceed to transmit the files according. But they don't check the frames. Both of their respective clients seem dumb and just play what they're sent, but they take a que from the media server on what the format is. I've raised this with Plex, was actually told to come here and log my issue. And don't get me wrong, there are ways to make media servers and their clients better without needing ffmpeg changes, but alas everything is out of my hands. I'm stuck between a rock and hard place, just trying to record and playback DVB-T2 and nothing seems to work right.

All of that said, it's only certain files (DVR files) that mess-up during playback and it all comes back to how ffmpeg decided it was progressive, then the media platforms continues this "problems" without better checks. If ffmpeg has come back and said either "mixed" or "interlaced", then it would have played find.

Left hand not talking to right hand, here I am stuck in the middle (with you) LOL

@Balling - thinks for the references, the years that people have been posting question/problems shows that people are struggling. For me, it just would have made sense for DVB-T2 just just be progressive and nothing else. But then again, it's an old format now! I wonder what the next mess-of-a-format that'll come along; that said HEVC interlaced looks like a PITA!