PMT descriptor overflow, results in incorrect streams

[Nicolás Jorge Dato]

Summary of the bug:
I have a mpegts sample that a PMT's descriptor overflows resulting in incorrect streams. I wasn't able to connect to the FTP upload.ffmpeg.org server to send this sample. (What should I do?) I have attached the sample to this ticket.

In that sample, the PMT has a program_info_length of 12 bytes. First, there is a descriptor ID 5, 4 bytes long. Then, it appears like there is an ID 1 descriptor, 15 bytes long. This overflows the program_info_length of 12 bytes.

I have found and fixed the issue in libavformat/mpegts.c, I'll send the patch to ffmpeg-devel mailing list: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285355.html

How to reproduce:

% ffprobe pmt_descriptor_overflow.ts
ffprobe version N-103620-gbbc24363f1 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11.2.0 (GCC)
  configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/doc/ffmpeg-4.4/html --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-gpl --enable-version3 --arch=x86_64 --enable-nonfree --disable-encoder=aac --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gnutls --enable-libbluray --enable-libcaca --enable-libcdio --enable-frei0r --enable-openal --enable-libopus --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libpulse --enable-libsmbclient --enable-libxml2 --enable-librsvg --enable-libdrm --enable-libsrt --enable-nvenc --enable-nvdec
  libavutil      57.  5.100 / 57.  5.100
  libavcodec     59.  7.103 / 59.  7.103
  libavformat    59.  5.100 / 59.  5.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  9.100 /  8.  9.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
[mpegts @ 0xca9d00] PES packet size mismatch
[mpegts @ 0xca9d00] Packet corrupt (stream = 1, dts = 3697332704).
[mpegts @ 0xca9d00] PES packet size mismatch
[mpegts @ 0xca9d00] Packet corrupt (stream = 1, dts = 3697332704).
[mpegts @ 0xca9d00] Could not find codec parameters for stream 0 (Unknown: none ([17][0][0][0] / 0x0011)): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mpegts, from 'pmt_descriptor_overflow.ts':
  Duration: 00:00:06.67, start: 41075.799822, bitrate: 2307 kb/s
  Program 1 
  Stream #0:0[0x1014]: Unknown: none ([17][0][0][0] / 0x0011)
  No Program
  Stream #0:1[0x10f]: Audio: aac (LC), 48000 Hz, stereo, fltp, 95 kb/s
  Stream #0:2[0x1011]: Video: h264 (Main), yuv420p(tv, smpte170m/smpte170m/bt709, progressive), 720x576 [SAR 12:11 DAR 15:11], 50 fps, 50 tbr, 90k tbn
Unsupported codec with id 0 for input stream 0

The stream #0:0 actually doesn't exists. Streams #0:1 and #0:2 are correct, however they should be inside Program 1, as those are registered in the PMT.

[Nicolás Jorge Dato]

I have sent the patch here:
https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285355.html

[Balling]

As you can check with nmap there is no ftp port open on upload.ffmpeg.org server (both on ipv4 and ipv6). Usually this is used ​https://streams.videolan.org/upload/

But better to attach here.

[Nicolás Jorge Dato]

A descriptor in the PMT overflows, ffmpeg shows incorect streams

[Nicolás Jorge Dato]

Replying to Balling:

But better to attach here.

Thanks, I didn't notice the button.

I think this is my first time here. I have a simple question, patchworks shows a warning asking me to wrap lines: ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210914184632.12637-1-nicolas.dato@gmail.com/

So, should I send a new version with the comment line wrapped?

[Balling]

Just remove work "handlign " there and resend. Also "an" should be "a".

[Balling]

Yeah, that 15 bytes ID 1 is this (except that it is supposed to be ID ):

​https://github.com/FFmpeg/FFmpeg/blob/f38458089f28df73a7badf459117d668ce988ca6/libavformat/mpegtsenc.c#L503

What about after those 15 bytes, they end in HDMV? What is supposed to be there? Use latest Wireshark to check. What about ff 1b 44 3f 28 04 64 00 00 30 28 3f 2a 02 7e ff 0f e1 0f f0 00 3d 92 37 27 FF FF.... FF? Is that ID ff size 1b? Because that does not look right.

0000   47 41 10 17 00 02 b0 37 00 01 c1 00 00 e1 00 f0   GA.....7........
0010   0c 05 04 48 44 00 0f 01 0f 0f ff fc fc 1b f0 11   ...HD...........
0020   f0 14 05 08 48 44 4d 56 ff 1b 44 3f 28 04 64 00   ....HDMV..D?(.d.
0030   28 3f 2a 02 7e ff 0f e1 0f f0 00 3d 92 37 27 ff   (?*.~......=.7'.
0040   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
0050   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
0060   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
0070   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
0080   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
0090   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
00a0   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff   ................
00b0   ff ff ff ff ff ff ff ff ff ff ff ff               ............

[Nicolás Jorge Dato]

Patch v3: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-September/285355.html

Here is more context about that sample. There is a TV station that provides a Live stream over HLS. The sample I have attached is a dump from that HLS stream.
I was trying to play it using Omxplayer (in a Raspberry Pi). Omxplayer didn't play anything, I figured out that Omxplayer reproduces "Program 1", and with this stream the Program and Streams are incorrect, thus Omxplayer was trying to reproduce a stream that doesn't exist.

I have no idea what is the intention of the muxer to add that ID 1 descriptor, and neither do I know why program_info_length is not set to the correct value. I don't have control over the muxer that generates that stream. I think that problem would be for another ticket if you like.

I made this ticket because when mpegts.c recognizes the descriptor overflows the program_info_length field, mpegts.c "jumps" to the end of the descriptors according to the program_info_length field.
It looks like this is the original idea. However, mpegts.c is actually jumping to program_info_length+2 because it doesn't subtract 2 bytes that mpegts.c has already read.

[Marton Balint]

Fixed in 5a0a9f78252825dfe1824eedbc373aea443e5e77.

[Balling]

It is now recognized as

  Stream #0:0[0x1011], 255, 1/90000: Video: h264 (Main), 1 reference frame (HDMV / 0x564D4448), yuv420p(tv, smpte170m/smpte170m/bt709, progressive, left), 720x576 [SAR 12:11 DAR 15:11], 0/1, 50 fps, 50 tbr, 90k tbn
  Stream #0:1[0x10f], 237, 1/90000: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 95 kb/s

compared to:

  Stream #0:0[0x1014], 0, 1/90000: Unknown: none ([17][0][0][0] / 0x0011)
  No Program
  Stream #0:1[0x10f], 267, 1/90000: Audio: aac (LC), 48000 Hz, stereo, fltp, 95 kb/s
  Stream #0:2[0x1011], 288, 1/90000: Video: h264 (Main), 1 reference frame, yuv420p(tv, smpte170m/smpte170m/bt709, progressive, left), 720x576 [SAR 12:11 DAR 15:11], 0/1, 50 fps, 50 tbr, 90k tbn
Unsupported codec with id 0 for input stream 0