Opened 5 years ago

Closed 5 years ago

#377 closed defect (wontfix)

ffmpeg reports inaccurate length of some gxf files

Reported by: acloutier Owned by:
Priority: important Component: undetermined
Version: git Keywords: gxf ffmpeg length bug
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

ffmpeg inaccurately reports the length of a good number of our gxf files. The last known version that reported the length correctly was 0.5.2 released versions 0.6.1, 0.6.3, 0.7.1, 0.7.1-rc1, and 0.8 also have this bug.

Example command line and output from the latest git version of ffmpeg:

ffmpeg -v 9 -loglevel 99 -i 20010920195800001.gxf 
ffmpeg version N-31720-g03cbe6c, Copyright (c) 2000-2011 the FFmpeg developers
  built on Aug  1 2011 15:34:44 with gcc 4.5.2
  configuration: --prefix=/usr --enable-shared --enable-libmp3lame --enable-gpl --enable-libfaac --enable-libvorbis --enable-pthreads --enable-libfaac --enable-libxvid --enable-x11grab --enable-libx264 --enable-libtheora --enable-nonfree --disable-stripping --enable-avfilter --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-version3
  WARNING: library configuration mismatch
  postproc    configuration: --extra-version=4:0.6.2-1ubuntu1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  libavutil    51. 11. 1 / 51. 11. 1
  libavcodec   53.  9. 0 / 53.  9. 0
  libavformat  53.  6. 0 / 53.  6. 0
  libavdevice  53.  2. 0 / 53.  2. 0
  libavfilter   2. 27. 5 /  2. 27. 5
  libswscale    2.  0. 0 /  2.  0. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[gxf @ 0x9f253a0] Format gxf probed with size=2048 and score=100
[mpeg2video @ 0x9f271e0] Unsupported bit depth: 0
[gxf @ 0x9f253a0] Probe buffer size limit 5000000 reached

Seems stream 0 codec frame rate differs from container frame rate: 59.94 (60000/1001) -> 59.94 (60000/1001)
Input #0, gxf, from '20010920195800001.gxf':
  Duration: 01:00:55.06, start: 0.000000, bitrate: 17599 kb/s
    Stream #0.0, 101, 1001/120000: Video: mpeg2video (4:2:2), yuv422p, 720x512 [SAR 128:135 DAR 4:3], 1001/60000, 10000 kb/s, 119.88 fps, 59.94 tbr, 119.88 tbn, 59.94 tbc
    Stream #0.1, 5, 1001/120000: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
    Stream #0.2, 5, 1001/120000: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
    Stream #0.3, 1, 1001/120000: Data: [0][0][0][0] / 0x0000
At least one output file must be specified

This is the output from 0.5.2 with the correct length:

/opt/ffmpeg/0.5.2/bin/ffmpeg -v 9 -i 20010920195800001.gxf
FFmpeg version 0.5.2, Copyright (c) 2000-2009 Fabrice Bellard, et al.
  configuration: --prefix=/opt/ffmpeg/0.5.2 --extra-cflags=-O3 -fPIC -ffast-math --extra-ldflags=-O3 -fPIC -ffast-math --extra-libs=-L/usr/local/lib -R/usr/local/lib --disable-mmx --disable-mmx2 --disable-ssse3 --enable-gpl --enable-nonfree --disable-ffserver --disable-ffplay --disable-stripping --disable-debug --disable-ipv6 --disable-protocol=udp --disable-encoder=nellymoser --disable-network --enable-swscale --enable-libmp3lame --enable-pthreads --enable-libfaac --enable-libfaad --enable-libx264 --disable-vhook
  libavutil     49.15. 0 / 49.15. 0
  libavcodec    52.20. 1 / 52.20. 1
  libavformat   52.31. 0 / 52.31. 0
  libavdevice   52. 1. 0 / 52. 1. 0
  libswscale     0. 7. 1 /  0. 7. 1
  built on Dec 22 2010 16:30:13, gcc: 4.5.1

Seems stream 0 codec frame rate differs from container frame rate: 59.94 (60000/1001) -> 29.97 (30000/1001)
Input #0, gxf, from '/data/rawVideo/2001/09/20/20010920195800001.gxf':
  Duration: 02:01:50.13, start: 0.000000, bitrate: 8799 kb/s
    Stream #0.0, 1001/60000: Video: mpeg2video, yuv422p, 720x512 [PAR 128:135 DAR 4:3], 1001/60000, 10000 kb/s, 29.97 tbr, 59.94 tbn, 59.94 tbc
    Stream #0.1, 1001/60000: Audio: pcm_s16le, 48000 Hz, mono, s16, 768 kb/s
    Stream #0.2, 1001/60000: Audio: pcm_s16le, 48000 Hz, mono, s16, 768 kb/s
    Stream #0.3, 1001/60000: Data: 0x0000
At least one output file must be specified

Attachments (1)

bisectLog (2.5 KB) - added by acloutier 5 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 5 years ago by cehoyos

Please make the sample available and if this is a regression, please find the version introducing it with git bisect.

comment:2 Changed 5 years ago by acloutier

A sample file cut down to 10MB is available here.

I'm working on the git-bisecting now.

Changed 5 years ago by acloutier

comment:3 Changed 5 years ago by acloutier

I've attached the git bisect log

Here is the result of the git bisect:

192c14fa5510e8b5e13ed5796ae87aabbfad84d6 is the first bad commit
commit 192c14fa5510e8b5e13ed5796ae87aabbfad84d6
Author: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Date: Sat Mar 6 22:20:33 2010 +0000

GXF time base is always based on "fields" per second even for
non-interlaced video.
Should fix issue 1766.


Originally committed as revision 22256 to svn://svn.ffmpeg.org/ffmpeg/trunk

:040000 040000 b10b0f20604611ac43e3ea4748322e576a450c05 ba3fe3fb8142cbc7c32881241f4d634aeb790402 M libavformat

comment:4 Changed 5 years ago by reimar

By my understanding of the format, the best guess it does right now is the best that is possible reliably.
The problem with this file is that it does not contain FPS in the track tags.
So FFmpeg is forced to fall back to the UMF tag (which is rather unreliable).
That one specifies 60 FPS, i.e. 120 fields per second (which is obviously not correct).
The file also claims to contain 438170 fields.
And that gives the calculated result.
Obviously it would be possible to just guess that 120 fields per second is not correct, but that does not seem like a good solution to me.
So unless you can give me some other info my conclusion would be that the file is broken and will only work correctly with programs/equipment that will break if you ever want to use it with 60 fps / 120 fields per second input.

comment:5 Changed 5 years ago by acloutier

Hmm.. it's possible that our in-house tool used to 'trim' these files smaller ( by replacing un-needed video with black ) didn't set the field correctly.

If it helps, here's the output for a file that doesn't have the issue ( it wasn't 'trimmed', so it's straight from our video encoder ):

ffmpeg -i /archive/rawVideo/2009/05/06/20090506215701002.gxf
ffmpeg version 0.7-rc1, Copyright (c) 2000-2011 the FFmpeg developers
  built on May 24 2011 15:20:15 with gcc 4.5.3
  configuration: --prefix=/opt/ffmpeg/0.7-rc1 --extra-cflags='-O3 -fPIC -ffast-math' --extra-ldflags='-O3 -fPIC -ffast-math' --extra-libs='-L/usr/local/lib -R/usr/local/lib' --disable-asm --disable-ssse3 --disable-amd3dnow --disable-amd3dnowext --enable-hardcoded-tables --enable-gpl --enable-nonfree --disable-ffserver --disable-ffplay --disable-debug --disable-protocol=udp --disable-encoder=nellymoser --disable-network --enable-swscale --enable-libmp3lame --enable-pthreads --enable-libfaac --enable-libx264
  libavutil    50. 40. 1 / 50. 40. 1
  libavcodec   52.120. 0 / 52.120. 0
  libavformat  52.108. 0 / 52.108. 0
  libavdevice  52.  4. 0 / 52.  4. 0
  libavfilter   1. 77. 0 /  1. 77. 0
  libswscale    0. 13. 0 /  0. 13. 0
Input #0, gxf, from '/archive/rawVideo/2009/05/06/20090506215701002.gxf':
  Duration: 02:03:59.13, start: 0.000000, bitrate: 11551 kb/s
    Stream #0.0: Video: mpeg2video (4:2:2), yuv422p, 720x512 [PAR 128:135 DAR 4:3], 10000 kb/s, 59.94 fps, 29.97 tbr, 59.94 tbn, 59.94 tbc
    Stream #0.1: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
    Stream #0.2: Audio: pcm_s16le, 48000 Hz, 1 channels, s16, 768 kb/s
    Stream #0.3: Data: [0][0][0][0] / 0x0000
At least one output file must be specified

Is there some flag we can pass ffmpeg to bypass this bad fps guess? I see some reference to putting -r before the -i to set the input frame rate, but that doesn't work.

comment:6 Changed 5 years ago by reimar

No, -r will only change the "playback" fps, but it will not change the time base the demuxer uses internally.
However you can at least easily detect the two cases:
119.88 tbn, 59.94 tbc
The tbn value is the time base the demuxer detects. tbc is the one that is encoded in the MPEG-2 stream within the container. For GXF and ordinary use cases it's probably reasonable to assume that if they mismatch there's something wrong.
For the "good" file these match btw:
59.94 tbn, 59.94 tbc

comment:7 Changed 5 years ago by acloutier

Thanks for the help. I think this patch will work for us (it just reverts the change/bugfix made in commit 192c14fa5510e8b5e13ed5796ae87aabbfad84d6), but it does sound like our files may have a unique issue.

diff --git a/libavformat/gxf.c b/libavformat/gxf.c
index 15893a4..42b3699 100644
--- a/libavformat/gxf.c
+++ b/libavformat/gxf.c
@@ -317,7 +317,7 @@ static int gxf_header(AVFormatContext *s, AVFormatParameters *ap) {
         st = s->streams[idx];
         if (!main_timebase.num || !main_timebase.den) {
             main_timebase.num = si->frames_per_second.den;
-            main_timebase.den = si->frames_per_second.num * 2;
+            main_timebase.den = si->frames_per_second.num * si->fields_per_frame;
         }
         st->start_time = si->first_field;
         if (si->first_field != AV_NOPTS_VALUE && si->last_field != AV_NOPTS_VALUE)
@@ -348,7 +348,7 @@ static int gxf_header(AVFormatContext *s, AVFormatParameters *ap) {
             if (!main_timebase.num || !main_timebase.den) {
                 // this may not always be correct, but simply the best we can get
                 main_timebase.num = fps.den;
-                main_timebase.den = fps.num * 2;
+                main_timebase.den = fps.num;
             }
         } else
             av_log(s, AV_LOG_INFO, "UMF packet too short\n");

comment:8 Changed 5 years ago by reimar

You most likely _will_ encounter issues with that approach.
My suggestion would be to _only_ change the second assignment. And even there I'd consider doing it only for fps > 60.
The reason is that to my knowledge there is no real right or wrong in the second case, whereas the first case definitely is correct as it is now.

comment:9 Changed 5 years ago by acloutier

Got it, thanks.

diff --git a/libavformat/gxf.c b/libavformat/gxf.c
index 15893a4..6c12048 100644

--- a/libavformat/gxf.c
+++ b/libavformat/gxf.c
@@ -348,6 +348,9 @@ static int gxf_header(AVFormatContext *s, AVFormatParameters *ap) {

             if (!main_timebase.num || !main_timebase.den) {
                 // this may not always be correct, but simply the best we can get
                 main_timebase.num = fps.den;
+                if ( ( (double) fps.num ) / fps.den > 59 )
+                    main_timebase.den = fps.num;
+                else
                     main_timebase.den = fps.num * 2;
             }
         } else

comment:10 Changed 5 years ago by reimar

  • Resolution set to wontfix
  • Status changed from new to closed

Yes, I think that should be a reasonable workaround for you.
You might want to look if you can improve that internal tool anyway though, because if you ever happen to have 60 fps progressive material you will probably need a much, much more complicated workaround to make everything still work.
I don't think it's really suitable for inclusion into main FFmpeg, so I'll set this to wontfix.

Note: See TracTickets for help on using tickets.