ffprobe: Wrong aspect ratio is reported

[Peter Krefting]

Summary of the bug:

At $DAYJOB, we are using ffmpeg/ffprobe to detect the aspect ratio of a program in a transport stream. After our customer upgraded the encoder software, ffprobe now reports 16:9 aspect ratio instead of 4:3. The streams are otherwise encoded in the same way.

For the sample files, I have kept only the PIDs relevant to the service that reports the incorrect aspect ratio from the sample provided by the customer. They have agreed to sharing the sample for the purpose of this bug report. I am including a sample encoded with both the old and the new version for reference. Please note that the actual image is 16:9, but it should be signalled as 4:3.

I tried hooking into libavcodec/mpeg12dec.c:mpeg_decode_postinit() and mpeg_deocde_sequence_display_extension() to see if I can understand why it is detecting the clips differently, but so far I have not been able to understand what is going on.

How to reproduce:

Incorrect output "DAR 16:9":

$ ./ffprobe PGM05.ts 
ffprobe version N-96094-gaad0e26f93 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: 
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 69.101 /  7. 69.101
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
[mpeg2video @ 0x55b53eed8cc0] Invalid frame dimensions 0x0.
    Last message repeated 22 times
[mpegts @ 0x55b53eeafd80] start time for stream 3 is not set in estimate_timings_from_pts
[mpegts @ 0x55b53eeafd80] PES packet size mismatch
    Last message repeated 1 times
Input #0, mpegts, from 'PGM05.ts':
  Duration: 00:00:17.68, start: 225.307844, bitrate: 4513 kb/s
  Program 1 
    Metadata:
      service_name    : T1
      service_provider: H
  Program 2 
    Metadata:
      service_name    : T2
      service_provider: H
  Program 3 
    Metadata:
      service_name    : T3
      service_provider: H
  Program 4 
    Metadata:
      service_name    : T4
      service_provider: H
  Program 5 
    Metadata:
      service_name    : T5
      service_provider: H
    Stream #0:0[0x1001]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 720x480 [SAR 32:27 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1041]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 138 kb/s
    Stream #0:2[0x1051]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 155 kb/s
    Stream #0:3[0x1201]: Subtitle: arib_caption (Profile A) ([6][0][0][0] / 0x0006)
  Program 6 
    Metadata:
      service_name    : T6
      service_provider: H
  Program 7 
    Metadata:
      service_name    : T7
      service_provider: H
  Program 8 
    Metadata:
      service_name    : T8
      service_provider: H
Unsupported codec with id 96272 for input stream 3

Correct output "DAR 4:3":

$ ./ffprobe  PGM05.ts 
ffprobe version N-96094-gaad0e26f93 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: 
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 69.101 /  7. 69.101
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
[mpeg2video @ 0x5633746f1cc0] Invalid frame dimensions 0x0.
    Last message repeated 28 times
[mpegts @ 0x5633746c8d80] start time for stream 3 is not set in estimate_timings_from_pts
[mpegts @ 0x5633746c8d80] PES packet size mismatch
    Last message repeated 1 times
Input #0, mpegts, from 'PGM05.ts':
  Duration: 00:00:13.42, start: 46.268967, bitrate: 4539 kb/s
  Program 1 
    Metadata:
      service_name    : T1
      service_provider: H
  Program 2 
    Metadata:
      service_name    : T2
      service_provider: H
  Program 3 
    Metadata:
      service_name    : T3
      service_provider: H
  Program 4 
    Metadata:
      service_name    : T4
      service_provider: H
  Program 5 
    Metadata:
      service_name    : T5
      service_provider: H
    Stream #0:0[0x1001]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 720x480 [SAR 8:9 DAR 4:3], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x1041]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 127 kb/s
    Stream #0:2[0x1051]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 178 kb/s
    Stream #0:3[0x1201]: Subtitle: arib_caption (Profile A) ([6][0][0][0] / 0x0006)
  Program 6 
    Metadata:
      service_name    : T6
      service_provider: H
  Program 7 
    Metadata:
      service_name    : T7
      service_provider: H
  Program 8 
    Metadata:
      service_name    : T8
      service_provider: H
Unsupported codec with id 96272 for input stream 3

[Carl Eugen Hoyos]

If there is an issue, it is a regression since 628724b61d744738e21ada8341a282d6802625a3
​http://samples.ffmpeg.org/ffmpeg-bugs/roundup/issue562/widescreen-issue562.mpg

[Marton Balint]

I dont think this is an issue in ffmpeg.

The old firmware writes 720x480 to sequence_display_extension, the new writes 540x480. aspect_ratio_information is 2 for both files which means 4:3.

ISO/IEC 13818-2 states that if sequence_display_extension() is present then the sample aspect ratio may be calculated as follows: SAR = DAR × display_horizontal_size / display_vertical_size

So you should report this to Harmonic instead. Reopen if they can cite some specs which proves that ffmpeg is wrong in deciding the aspect ratio.

[Marton Balint]

Also, the encoder might not be wrong, if the 4:3 content is pillarboxed in the 16:9 video.
In this case the full 720x480 raster has a display aspect ratio of 16:9, but the to-be-displayed 540x480 raster indeed has a display aspect ratio of 4:3. However, ffmpeg/ffprobe always reports the display aspect ratio of the full raster, not the area of the active content.