Encoding with h264_amf, muxing in MP4, missing reference frame when seeking within 2nd and 3rd GOP

[lveilleux]

Summary of the bug:
I created a directshow filter that calls FFMPEG APIs to encode using h264_nvenc, h264_amf, h264_qsv and Xvid, then mux in MP4. It tries opening the encoders in that order, if it fails it moves on to the next one. I also tested with libx264, but did not include it in release build since I have not procured a commercial license for x264.

When running on a computer with compatible AMD card, it encodes using H264_AMF, but the resulting MP4 has problems seeking. If I seek within the 2nd or 3rd GOP, a reference frame is missing on some media players. Latest VLC and WMP work fine, some older versions of WMP shows the problem, and all directshow splitters I tried (LAV Splitter, GDCL MPEG4 Demuxer, Haali Media Splitter) have this problem. The same code shows no seeking problem when using H264_NVENC, H264_QSV or LIBX264 codecs.

This is the code that initializes the codec (for h264_amf, with some hard-coded parameters). Although I initialize at 30fps, it uses variable frame rate with timestmps coming from directshow IMediaSample.

AVCodec *videoCodec = avcodec_find_encoder_by_name("h264_amf");

m_c = avcodec_alloc_context3(videoCodec);
if (!(m_c)) return -1;

m_c->width = 1920;
m_c->height = 1080;
m_c->gop_size = 100;
m_c->bit_rate = 8M;
m_c->time_base.den = 90000;
m_c->time_base.num = 3000;
m_c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
m_c->pix_fmt = AV_PIX_FMT_YUV420P;

av_opt_set(m_c->priv_data, "rc", "cqp", 0);
av_opt_set_int(m_c->priv_data, "qp_i", 30, 0);
av_opt_set_int(m_c->priv_data, "qp_p", 30, 0);
av_opt_set_int(m_c->priv_data, "qp_b", 30, 0);
ret = avcodec_open2(m_c, videoCodec, NULL);

ffmpeg version: Windows x64, nightly build from 12 March 2019

I tried different settings/options, and the only way I am able to avoid this seeking problem is to force only 1 reference frame (m_c->refs = 1). But that results in bitrate/file size 3-4 times larger than if I leave m_c->refs to default. This is a little overkill to avoid seeking problem that only appear in 2nd and 3rd GOPs.

I attached a short desktop capture video that shows this (remember, some latest media players read it fine but none of the "popular" directshow MP4 demuxers). Average framerate is ~20.6fps and GOP is 100, so the 2nd and 3rd GOPs are ~5 to 14 seconds into the video.

[lveilleux]

The video I had exceeds the file limit. I'll create another one and attach it tomorrow.

[AlexisMay]

When using AMD hardware the additional option "-gops_per_idr 1" should be used.

[lveilleux]

Replying to AlexisMay:

When using AMD hardware the additional option "-gops_per_idr 1" should be used.

I saw this in a post I think on stack overflow, and it only applies to HEVC. Also looking at amfenc_h264.c and amfenc_hevc.c shows this option is only coded in HEVC.

[AlexisMay]

Can you share a little bit more about this? Can you share the problematic file? Can you reproduce the problem with latest FFmpeg from Git? Are you sure that in your own implementation your pipeline didn't skip encoded buffers? What is your GPU? Which version of Adrenalin software is used?

[lveilleux]

Problem MP4, seeking between 7 and 19 seconds

[lveilleux]

Replying to AlexisMay:

Can you share a little bit more about this? Can you share the problematic file? Can you reproduce the problem with latest FFmpeg from Git? Are you sure that in your own implementation your pipeline didn't skip encoded buffers? What is your GPU? Which version of Adrenalin software is used?

1- Hi, I attached a sample problem file. It's 15FPS with GOP=100, so the 2nd and 3rd GOPs are between ~7 and 19 seconds.

2- I used 24-May-2019 nightly build from zeranoe.

3- I'm not sure what you mean by skipping encoded buffers, what I can say is that the same MP4 muxing code does not cause problems when using NVENC (Quadro P2000), QSV (i7-8650U) or libx264. Also, when I set max_b_frames to something > 3, and private option bf_ref true, I don't see the problem but file size/bitrate more than doubles. If you want, I can dump my directshow filter code somewhere.

4- Here's the complete info I dumped from AMD configuration, I could not find any Catalyst or Adrenaline software info:

Graphics Card Manufacturer - Powered by AMD
Graphics Chipset - AMD FirePro W5100 Graphics Adapter
Device ID - 6649
Vendor ID - 1002
SubSystem ID - 230C
SubSystem Vendor ID - 1028
Revision ID - 00
Bus Type - PCI Express 3.0
Current Bus Settings - PCI Express 3.0 x16
BIOS Version - 015.049.000.011
BIOS Part Number - 113-C5870401-101
BIOS Date - 2016/01/14 04:07
Memory Size - 4096 MB
Memory Type - GDDR5
Memory Clock - 1500 MHz
Core Clock - 930 MHz
Total Memory Bandwidth - 96 GByte/s
2D Driver File Path - /REGISTRY/MACHINE/SYSTEM/ControlSet001/Control/Class/{4d36e968-e325-11ce-bfc1-08002be10318}/0000

Radeon Pro and AMD FirePro Settings Version - 2016.1004.2047.35575
Driver Packaging Version - 16.40.2802-161004a-307960C-Dell
Provider - Advanced Micro Devices, Inc.
2D Driver Version - 8.1.1.1571
Direct3D® Version - 9.14.10.01221
OpenGL® Version - 6.14.10.13456
OpenCL™ Version - 21.19.128.15
AMD Mantle Version - 9.1.10.0146
AMD Mantle API Version - Not Available
AMD Audio Driver Version - 10.0.1.0
Vulkan™ Driver Version - Not Available
Vulkan™ API Version - Not Available

Thanks for your help

[Mikhail]

The attached file has valid H264 elementary stream with IDRs each 100th frame as expected. I verified picture referencing, looks correct. I used Vega Analyzer for verification. I don’t see seeking problem with WMP, Movies&TV and VLC.
Usually, seeking problems are related to the lack of IDR or incorrect report of IDR frame type to the container writer.
I’ve noticed that you use very old graphical driver: 16.40.xxxx is from 2016. There is possibility that IDR frame types are reported incorrectly and it could lead to the seek problem. Could you please update the driver from here and try: ​https://www.amd.com/en/support/professional-graphics/firepro/firepro-wx000-series/firepro-w5000

[lveilleux]

My bad regarding the driver.

When I pasted the AMD info in this ticket yesterday I noticed that too. I updated to the latest release, and guess what... it fixed it.

Sorry for this "amateurish" oversight.