Opened 3 weeks ago

Last modified 12 days ago

#6810 new defect

Wrong DTS guess when PTS is reordered for the input without DTS (wrong 'Non-monotonous DTS' fixup)

Reported by: perexg Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: h264 dts
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

A conversion (copy) from matroska to matroska causes sometimes wrong PTS output. The input is H264 with re-ordered video frames (from live broadcast).

The error is directly related to "Non-monotonous DTS in output stream" fixup. The DTS is wrongly guessed - select_from_pts_buffer() in libavformat/utils.c returns wrong DTS.

Input PTS example (with wrong DTS):

demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22459 pkt_pts_time:22.459 pkt_dts:22279 pkt_dts_time:22.279
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22359 pkt_pts_time:22.359 pkt_dts:22359 pkt_dts_time:22.359
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22379 pkt_pts_time:22.379 pkt_dts:22379 pkt_dts_time:22.379
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22319 pkt_pts_time:22.319 pkt_dts:22319 pkt_dts_time:22.319
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22339 pkt_pts_time:22.339 pkt_dts:22339 pkt_dts_time:22.339
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22399 pkt_pts_time:22.399 pkt_dts:22399 pkt_dts_time:22.399
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:22419 pkt_pts_time:22.419 pkt_dts:22419 pkt_dts_time:22.419

For example, the line with pts == dts == 22359 is wrong, dts should be 22319 here (+40).

As a consequence, the monotonous DTS check will correct PTS for the last DTS, so the value 22319 will be changed to 22379, because of the invalid DTS source.

If you look to trac, I believe that several other bugs are similar to this one.

Attachments (3)

a.mkv (2.2 MB) - added by perexg 3 weeks ago.
Example mkv - wrong output PTS value 2980 (three frames with identical PTS)
a.2.mkv (2.2 MB) - added by perexg 3 weeks ago.
Example mkv - wrong output PTS value 2980 (three frames with identical PTS)
original.ts (1.8 MB) - added by perexg 12 days ago.
Original MPEG-TS which shows the problem (cutted using dd)

Change History (13)

comment:1 in reply to: ↑ description Changed 3 weeks ago by cehoyos

  • Keywords mkv dts added; matroska timestamp removed
  • Priority changed from critical to normal

Replying to perexg:

If you look to trac, I believe that several other bugs are similar to this one.

You mean this is a duplicate of another ticket?

Please provide the command line you tested together with the complete, uncut console output and an input sample that allows to reproduce.

Changed 3 weeks ago by perexg

Example mkv - wrong output PTS value 2980 (three frames with identical PTS)

Changed 3 weeks ago by perexg

Example mkv - wrong output PTS value 2980 (three frames with identical PTS)

comment:2 Changed 3 weeks ago by perexg

You mean this is a duplicate of another ticket?

It may be similar to #3572, #3621 and probably others - it's difficult to find all.

Please provide the command line you tested together with the complete, uncut console output and an input sample that allows to reproduce.

Command line: ffmpeg -y -loglevel trace -debug_ts -fdebug ts -i a.mkv -c:v copy -c:a copy /tmp/a.mkv

demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:2980 pkt_pts_time:2.98 pkt_dts:2980
muxer <- type:video pkt_pts:2980 pkt_pts_time:2.98 pkt_dts:2980 pkt_dts_time:2.98
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:2920 pkt_pts_time:2.92 pkt_dts:2920
muxer <- type:video pkt_pts:2980 pkt_pts_time:2.98 pkt_dts:2980 pkt_dts_time:2.98
demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:2940 pkt_pts_time:2.94 pkt_dts:2940
muxer <- type:video pkt_pts:2980 pkt_pts_time:2.98 pkt_dts:2980 pkt_dts_time:2.98

comment:3 Changed 3 weeks ago by perexg

Further analysis: It appears that delay (avctx->has_b_frames) has an insufficient value (2) to do full DTS recovery. If I force this value to 8 in the code, DTS is OK.

comment:4 Changed 3 weeks ago by perexg

Value 4 (has_b_frames) is sufficient. It looks like H264 decoder issue.

The H264 SPS has not set bitstream_restriction_flag . The sps->num_reorder_frames is correctly set in ff_h264_decode_seq_parameter_set() - comment: derive it based on the level . The problem is that h264_select_output_frame() checks for strict settings so code 'h->avctx->has_b_frames = FFMAX(h->avctx->has_b_frames, sps->num_reorder_frames);' is not executed by default. Adding "-strict 1" option fixes the issue completely:

ffmpeg -strict 1 -i a.mkv -c:v copy -c:a copy /tmp/a.mkv

So the "bug" is in the H264 decoder.

The primary source of the strict check is in the commit:

commit fb19e1443bc45e192545d3485ddb9c76e7d77855
Author: Michael Niedermayer <michaelni@gmx.at>
Date:   Sat Jul 19 16:16:00 2008 +0000

    Take the brain amputated spec literally if the user asks for it (-strict 1).
    That is, add 16 frames delay, cache trashing and av desync.
    fixes at least the following reference bitstreams:
    CABA3_Sony_C.jsv
    CACQP3_Sony_D.jsv
    CAMANL1_TOSHIBA_B.264
    CANL3_Sony_C.jsv
    CVBS3_Sony_C.jsv
    CVMANL1_TOSHIBA_B.264
    
    Originally committed as revision 14308 to svn://svn.ffmpeg.org/ffmpeg/trunk

I believe that the original purpose of the check was obsoleted with the new code in ff_h264_decode_seq_parameter_set() - comment: derive it based on the level , so I would propose to remove the condition in h264_select_output_frame() like :

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 35dcabd611..3b50d12fa4 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -1286,10 +1286,7 @@ static int h264_select_output_frame(H264Context *h)
     cur->mmco_reset = h->mmco_reset;
     h->mmco_reset = 0;
 
-    if (sps->bitstream_restriction_flag ||
-        h->avctx->strict_std_compliance >= FF_COMPLIANCE_STRICT) {
-        h->avctx->has_b_frames = FFMAX(h->avctx->has_b_frames, sps->num_reorder_frames);
-    }
+    h->avctx->has_b_frames = FFMAX(h->avctx->has_b_frames, sps->num_reorder_frames);
 
     for (i = 0; 1; i++) {
         if(i == MAX_DELAYED_PIC_COUNT || cur->poc < h->last_pocs[i]){

The mkv source is really the valid live H264 broadcast and the processing should not require additional (hard to determine / find) option for ffmpeg.

comment:5 Changed 2 weeks ago by perexg

A better patch is:

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 35dcabd611..04b10656e0 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -1287,6 +1287,7 @@ static int h264_select_output_frame(H264Context *h)
     h->mmco_reset = 0;
 
     if (sps->bitstream_restriction_flag ||
+        sps->ref_frame_count ||
         h->avctx->strict_std_compliance >= FF_COMPLIANCE_STRICT) {
         h->avctx->has_b_frames = FFMAX(h->avctx->has_b_frames, sps->num_reorder_frames);
     }

I sent this code to the ffmpeg-devel mailing list.

comment:6 Changed 12 days ago by cehoyos

  • Keywords h264 added; mkv removed

How was the input sample created?

comment:7 follow-up: Changed 12 days ago by perexg

Tvheadend (http://tvheadend.org), but it's bit-copy of the input stream from the live broadcasting (H.264 NALs), the cutted using mkvmerge. Note that only some channels show this behaviour.

comment:8 in reply to: ↑ 7 Changed 12 days ago by perexg

Replying to perexg:

Tvheadend (http://tvheadend.org), but it's bit-copy of the input stream from the live broadcasting (H.264 NALs), the cutted using mkvmerge. Note that only some channels show this behaviour.

then cutted using mkvmerge

comment:9 Changed 12 days ago by cehoyos

Please provide (a short cut of) the original transport stream.

Changed 12 days ago by perexg

Original MPEG-TS which shows the problem (cutted using dd)

comment:10 Changed 12 days ago by perexg

To reproduce DTS issue:

$ ffmpeg -y -i original.ts -c:v copy -c:a copy x.mkv
$ ffmpeg -y -i x.mkv -c:v copy -c:a copy y.mkv
[matroska @ 0x562a68e6eda0] Non-monotonous DTS in output stream 0:0; previous: 2667, current: 2627; changing to 2667. This may result in incorrect timestamps in the output file.

With strict set, there are only few errors at start (which can be tolerated):

$ ffmpeg -y -strict strict -i x.mkv -c:v copy -c:a copy y.mkv
[matroska @ 0x55e1e75f7060] Non-monotonous DTS in output stream 0:0; previous: 1707, current: 1687; changing to 1707. This may result in incorrect timestamps in the output file.

Last edited 12 days ago by perexg (previous) (diff)
Note: See TracTickets for help on using tickets.