Opened 5 years ago

Last modified 4 months ago

#5514 open defect

Interlaced HEVC Steam not Decoded Properly

Reported by: Jose Santiago Owned by:
Priority: normal Component: avcodec
Version: git-master Keywords: hevc
Cc: dhelsley@haivision.com, ffmpeg@tmm1.net Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

I have a 480i interlaced HEVC elementary stream. The HM reference decoder decodes the stream and generates 720x480 interlaced frames. FFMPEG/FFPLAY decodes the stream and 720x240 progressive frames and FFPLAY plays them at half the frame rate with the first/second field/frames bouncing up and down a line because these are actually 2 different fields of the same frame.

How to reproduce:

ffmpeg -i src13_interlaced.265 src13_interaced.yuv
ffplay src13_interlaced.265


$ ffmpeg -i src13_interlaced.265 src13_interaced.yuv
ffmpeg version git-2016-04-27-7bccbee-VF Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-16)
  configuration: --extra-version=VF --prefix=/mnt/kermit/work/git/ort/build/linux64/stage/ffmpeg --ld=/usr/bin/cc --target-os=linux --arch=x86_64 --cpu=x86_64 --enable-pic --enable-static --enable-shared --enable-rpath --enable-avfilter --enable-pthreads --enable-zlib --enable-bzlib --enable-runtime-cpudetect --enable-hardcoded-tables --disable-stripping --disable-doc --disable-dxva2 --disable-vaapi --disable-vda --disable-vdpau --yasmexe=/mnt/kermit/work/git/ort/build/linux64/tools/yasm/bin/yasm --sdl-config=/mnt/kermit/work/git/ort/build/linux64/stage/sdl/lib/../bin/sdl-config --extra-cflags='-I/mnt/kermit/work/git/ort/build/linux64/stage/zlib/include -I/mnt/kermit/work/git/ort/build/linux64/stage/bzip2/include -I/mnt/kermit/work/git/ort/build/linux64/stage/lzma/include -I/mnt/kermit/work/git/ort/build/linux64/stage/sdl/include -DVFBUILD_DISABLE_INTMATH_OPTIMIZATIONS=1' --extra-ldflags='-L/mnt/kermit/work/git/ort/build/linux64/stage/zlib/lib -L/mnt/kermit/work/git/ort/build/linux64/stage/bzip2/lib -L/mnt/kermit/work/git/ort/build/linux64/stage/lzma/lib -L/mnt/kermit/work/git/ort/build/linux64/stage/sdl/lib'
  libavutil      55. 19.100 / 55. 19.100
  libavcodec     57. 28.203 / 57. 28.203
  libavformat    57. 29.200 / 57. 29.200
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 39.202 /  6. 39.202
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
VFAVUtil_MetaData_RunUnitTests() Completed Successfully.
Input #0, hevc, from 'src13_interlaced.265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 720x240, 30 fps, 30 tbr, 1200k tbn, 30 tbc
Output #0, rawvideo, to 'src13_interaced.yuv':
  Metadata:
    encoder         : Lavf57.29.200
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 720x240, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
    Metadata:
      encoder         : Lavc57.28.203 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> rawvideo (native))
Press [q] to stop, [?] for help
frame=  336 fps=0.0 q=-0.0 size=   85050kB time=00:00:11.20 bitrate=62208.0kbitsframe=  337 fps=263 q=-0.0 size=   85303kB time=00:00:11.23 bitrate=62208.0kbitsframe=  522 fps=337 q=-0.0 Lsize=  132131kB time=00:00:17.40 bitrate=62208.0kbits/s speed=11.2x    
video:132131kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

Attachments (2)

hm_reference_decoder_test.txt (43.6 KB ) - added by Jose Santiago 5 years ago.
HM Reference Decoder Test Run
src13_interlaced_cut.265 (2.4 MB ) - added by Carl Eugen Hoyos 5 years ago.

Change History (16)

comment:1 by James, 5 years ago

Can you attach or upload the sample somewhere? Not much we can do otherwise

comment:2 by Jose Santiago, 5 years ago

The files are too large to upload to the ticket. 2.5MB limit. I have provided the following files for external download:

HEVC Elementary Stream:
http://162.97.176.4/src13_interlaced.265

Does not Play correctly with:
   ffplay src13_interlaced.265

FFMPEG Decoder Output:
http://162.97.176.4/src13_ffmpeg.yuv

Does not Play correctly with:
   ffplay -f rawvideo -pix_fmt yuv420p -framerate 30 -s 720x480 -i src13_ffmpeg.yuv
Better but half framerate and field bounce and incorrect resolution with:
   ffplay -f rawvideo -pix_fmt yuv420p -framerate 30 -s 720x240 -i src13_ffmpeg.yuv

HM Reference Decoder Output:
http://162.97.176.4/src13_hm.yuv

Plays correctly with:
   ffplay -f rawvideo -pix_fmt yuv420p -framerate 30 -s 720x480 -i src13_hm.yuv
Last edited 5 years ago by Jose Santiago (previous) (diff)

by Jose Santiago, 5 years ago

HM Reference Decoder Test Run

comment:3 by Hendrik, 5 years ago

Duplicate of ticket #4141

comment:4 by Dave Helsley, 5 years ago

There are differences. In #4141, according to the comments the stream in question has field_seq_flag=1 but pic_struct=3, which is improper syntax, and the reference decoder does the same thing as FFMPEG (output half-height frames).

With this particular elementary stream, field_seq_flag=1 and pic_struct=1 or 2 depending on which field it is, which is valid interlaced syntax. The reference decoder produces proper interlaced output frames with this test vector.

Last edited 5 years ago by Dave Helsley (previous) (diff)

comment:5 by Carl Eugen Hoyos, 5 years ago

Keywords: hevc added
Reproduced by developer: set
Status: newopen

by Carl Eugen Hoyos, 5 years ago

Attachment: src13_interlaced_cut.265 added

in reply to:  4 ; comment:6 by Balling, 10 months ago

Replying to dhelsley:

There are differences. In #4141, according to the comments the stream in question has field_seq_flag=1 but pic_struct=3, which is improper syntax, and the reference decoder does the same thing as FFMPEG (output half-height frames).

With this particular elementary stream, field_seq_flag=1 and pic_struct=1 or 2 depending on which field it is, which is valid interlaced syntax. The reference decoder produces proper interlaced output frames with this test vector.

Please can you look into this example https://yadi.sk/d/bQkXgt900uEdkg (DVB-T2 ts stream from Ostankino, USSR channel 58, Russia)? Is it the second as well? BTW, about reference decoder, is that this one? https://sourceforge.net/p/mjpeg/mailman/message/9927377/ or what?

in reply to:  6 ; comment:7 by Dave Helsley, 10 months ago

Replying to Balling:

Replying to dhelsley:

There are differences. In #4141, according to the comments the stream in question has field_seq_flag=1 but pic_struct=3, which is improper syntax, and the reference decoder does the same thing as FFMPEG (output half-height frames).

With this particular elementary stream, field_seq_flag=1 and pic_struct=1 or 2 depending on which field it is, which is valid interlaced syntax. The reference decoder produces proper interlaced output frames with this test vector.

Please can you look into this example https://yadi.sk/d/bQkXgt900uEdkg (DVB-T2 ts stream from Ostankino, USSR channel 58, Russia)? Is it the second as well? BTW, about reference decoder, is that this one? https://sourceforge.net/p/mjpeg/mailman/message/9927377/ or what?

Wow, old bug.

Your stream has field_seq_flag=1 and pic_struct=10,11 which is similar to our case (pic_struct=1,2 is top/bottom field, pic_struct=10,11 is top with next bottom/bottom with previous top or Top Field First).

Reference software can be found here https://hevc.hhi.fraunhofer.de/

in reply to:  7 comment:8 by Balling, 10 months ago

Replying to dhelsley:

pic_struct=10,11 is top with next bottom/bottom

I am afraid it is more difficult than that? If you play in Potplayer it will oscillate between i and ib (at least there are no other crazy types https://sourceforge.net/p/mjpeg/mailman/message/9927377/)

Fraunhofer, of course, LOL. He is everywhere, even VoLTE EVS (enhanced voice service) or VVC/H.266.

Last edited 10 months ago by Balling (previous) (diff)

comment:9 by Balling, 10 months ago

Well, it was partially implemented in 876dada0b58b5ecc80b2a25eb5c33974a71c8eb2 and
7c8b65f688ea75496e278b7c042f2eda746f3eac

So when you do ffprobe src13_interlaced_cut.265 it says

Input #0, hevc, from 'C:\Users\ZAQU\Downloads\src13_interlaced_cut.265':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv, top first), 720x240, 30 fps, 30 tbr, 1200k tbn, 30 t

But with https://yadi.sk/d/bQkXgt900uEdkg it is not like that, it says it is progressive, that is incorrect. I think I should try -filter:v idet. (P.S. Idet does not work even on src13_interlaced_cut.265)

Also after 876dada0b58b5ecc80b2a25eb5c33974a71c8eb2 it is showing itself in ffprobe -show_frames but even there there are not enough parameters as it is not just

interlaced_frame=1
top_field_first=1
Last edited 10 months ago by Balling (previous) (diff)

comment:10 by Aman, 6 months ago

Cc: ffmpeg@tmm1.net added

comment:11 by Aman, 6 months ago

I have a simple patch to make ffprobe report these hevc files as interlaced. Not sure if it is completely correct though, as the spec was hard for me to parse. See page 326 of Rec. ITU-T H.265 v7 (11/2019)

diff --git a/libavcodec/hevc_parser.c b/libavcodec/hevc_parser.c
index b444b99955..da9b9019e2 100644
--- a/libavcodec/hevc_parser.c
+++ b/libavcodec/hevc_parser.c
@@ -65,7 +65,24 @@ static int hevc_parse_slice_header(AVCodecParserContext *s, H2645NAL *nal,

     sh->first_slice_in_pic_flag = get_bits1(gb);
     s->picture_structure = sei->picture_timing.picture_struct;
-    s->field_order = sei->picture_timing.picture_struct;
+    switch (s->picture_structure) {
+        case 0:
+            s->field_order = AV_FIELD_PROGRESSIVE;
+            break;
+        case 1:
+        case 9:
+        case 11:
+            s->field_order = AV_FIELD_TT;
+            break;
+        case 2:
+        case 10:
+        case 12:
+            s->field_order = AV_FIELD_BB;
+            break;
+        default:
+            s->field_order = AV_FIELD_UNKNOWN;
+            break;
+    }

     if (IS_IRAP_NAL(nal)) {
         s->key_frame = 1;

comment:12 by Aman, 6 months ago

This makes ffprobe report the correct height, but does not help with playback at all.

diff --git a/libavcodec/hevc_parser.c b/libavcodec/hevc_parser.c
index b444b99955..d00093d592 100644
--- a/libavcodec/hevc_parser.c
+++ b/libavcodec/hevc_parser.c
@@ -105,6 +105,10 @@ static int hevc_parse_slice_header(AVCodecParserContext *s, H2645NAL *nal,
         den = ps->sps->vui.vui_time_scale;
     }

+    if (ps->sps->ptl.general_ptl.interlaced_source_flag) {
+        avctx->height = s->height * 2;
+    }
+
     if (num != 0 && den != 0)
         av_reduce(&avctx->framerate.den, &avctx->framerate.num,
                   num, den, 1 << 30);

comment:13 by Balling, 6 months ago

I think that interlaced_source_flag is not intended for check here, better check field_seq_flag for 1 as per spec ("field_seq_flag equal to 1 indicates that the CVS conveys pictures that represent fields" and further) and that is how it is done in https://gitlab.com/mbunkus/mkvtoolnix/-/commit/47058a6da2672944aa4544f8ecb7b9a79ba3f752#0dd3dcd183ebb1497b9a9e3522e4d11a174384a2_282_288

That will not fix #4141 of course, but that file there is not spec compliant. It can further be fixed by checking for picture_struct and interlaced_source_flag.

Indeed as I read the spec (and as mkver reads it), the fields (not frames! ) can be progressive in all cases, not only for frame doubling and trippling.

comment:14 by Balling, 4 months ago

Workaround, works for pic_struct=10,11:

ffplay "test" -vcodec hevc_cuvid -vf weave,hwupload_cuda,yadif_cuda=mode=0:parity=0:deint=0,hwdownload,format=nv12

You may want to -bsf:v hevc_mp4toannexb it.

Note: See TracTickets for help on using tickets.