Opened 4 years ago
Last modified 3 months ago
#8916 open defect
Incorrect field order indication when encoding interlace top field first using h264_nvenc
Reported by: | Ptaah | Owned by: | |
---|---|---|---|
Priority: | important | Component: | avcodec |
Version: | git-master | Keywords: | nvenc regression |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
When encoding top field first interlaced video (or making one using tinterlace filter), the resulting h264 bitstream is marked as bottom field first. This behaviour was not present prior version 4.0.
How to reproduce:
% ffmpeg -i interlaced.mp4 -codec:v h264_nvenc -flags +ildct+ilme -an output.mp4 % ffprobe -i output.mp4 -show_entries "frame=interlaced_frame,top_field_first" -pretty [FRAME] interlaced_frame=1 top_field_first=0 [/FRAME]
Change History (29)
comment:1 by , 4 years ago
Keywords: | regression added; avcodec field tff bff removed |
---|
comment:2 by , 4 years ago
I don't know, which commit introduced it, I would've tested it more thoroughly, but Zeranoe is apparently out of business and I don't have many ffmpeg windows build. What I can say is, it worked correctly with ffmpeg 3.3.2 and it doesn't work with ffmpeg 4.1.1 all the way up to current build.
.\ffmpeg.exe -i interlaced.mp4 -codec:v h264_nvenc -flags +ildct+ilme -an output.mp4 ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 8.2.1 (GCC) 20190212 configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth libavutil 56. 22.100 / 56. 22.100 libavcodec 58. 35.100 / 58. 35.100 libavformat 58. 20.100 / 58. 20.100 libavdevice 58. 5.100 / 58. 5.100 libavfilter 7. 40.101 / 7. 40.101 libswscale 5. 3.100 / 5. 3.100 libswresample 3. 3.100 / 3. 3.100 libpostproc 55. 3.100 / 55. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'interlaced.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf57.73.100 Duration: 00:00:20.00, start: 0.000000, bitrate: 1857 kb/s Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 1856 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default) Metadata: handler_name : VideoHandler Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc)) Press [q] to stop, [?] for help Output #0, mp4, to 'output.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.20.100 Stream #0:0(und): Video: h264 (h264_nvenc) (Main) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 25 fps, 12800 tbn, 25 tbc (default) Metadata: handler_name : VideoHandler encoder : Lavc58.35.100 h264_nvenc Side data: cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: -1 frame= 500 fps=354 q=35.0 Lsize= 4532kB time=00:00:19.96 bitrate=1860.2kbits/s speed=14.1x video:4530kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.062222%
comment:4 by , 4 years ago
perhaps roll out own windows ffmpeg.exe build using MABS under Windows: https://github.com/m-ab-s/media-autobuild_suite
or, create a vmware ubuntu 20.04 VM and cross-compile ffmpeg.exe for windows using rdp's cross-compilation script: https://github.com/rdp/ffmpeg-windows-build-helpers
only if you are really really terribly desperate, create a vmware ubuntu 20.04 VM and cross-compile ffmpeg.exe for windows using the buggy often-not-working thing: https://github.com/hydra3333/h3333_python_cross_compile_script_v100
comment:6 by , 4 years ago
Strange thing is if you add a B frame with -bf 1 you will have a correct flag.
% ffmpeg -i interlaced.mp4 -codec:v h264_nvenc -flags +ildct+ilme -bf 1 -an output.mp4
% ffprobe -i output.mp4 -show_entries "frame=interlaced_frame,top_field_first" -pretty
gives you:
[FRAME]
interlaced_frame=1
top_field_first=1
FRAME
Tested with latest nighlty build and nvidia drivers.
comment:7 by , 4 years ago
ffmpeg -i interlaced.mp4 -codec:v h264_nvenc -flags +ildct+ilme -pix_fmt yuv420p -an output.mp4
up to 3.3.4 looks ok:
[FRAME]
interlaced_frame=1
top_field_first=1
3.3.5 Introduces this problem
[FRAME]
interlaced_frame=1
top_field_first=0
FRAME
follow-ups: 12 13 comment:8 by , 4 years ago
Status: | new → open |
---|
You should use https://github.com/BtbN/FFmpeg-Builds/ now, hydra3333
comment:9 by , 4 years ago
Great work, thanks. I will check what has changed in h264 nvenc implementation between those versions.
follow-up: 14 comment:10 by , 4 years ago
I didn't find any relevant changes between 3.3.4 and 3.3.5 version. Where did you get 3.3.4 and 3.3.5 binaries? I would like to check what git commit are they based on.
comment:11 by , 4 years ago
I just realized. Might have been 3.4 but labeled as 3.3.5. I googled old release to avoid the hassle of compiling.
Double checked and 3.4.0 gives you the error so its definitely introduced between 3.3.4 and 3.4
Here you can find a lot of the older versions complied for win
https://www.videohelp.com/software/ffmpeg/old-versions
comment:12 by , 4 years ago
Replying to Balling:
You should use https://github.com/BtbN/FFmpeg-Builds/ now, hydra3333
Thank you Balling.
(I prefer ffmpeg built with OpenCL, fdk-aac, h264-8/10/12bit, h265-multibit, av1, nvenc, so I'll have a look at that link !)
follow-up: 15 comment:13 by , 4 years ago
Replying to Balling:
You should use https://github.com/BtbN/FFmpeg-Builds/ now, hydra3333
Thank you Balling.
(I prefer ffmpeg built with OpenCL, fdk-aac, h264-8/10/12bit, h265-multibit, av1, nvenc, so I'll have a look at that link !)
edit: ah, I preferred not to install docker. Oh well.
follow-up: 16 comment:14 by , 4 years ago
Replying to Ptaah:
I didn't find any relevant changes between 3.3.4 and 3.3.5 version. Where did you get 3.3.4 and 3.3.5 binaries? I would like to check what git commit are they based on.
Found anything useful between the versions?
comment:15 by , 4 years ago
Replying to hydra3333:
Replying to Balling:
You should use https://github.com/BtbN/FFmpeg-Builds/ now, hydra3333
Thank you Balling.
(I prefer ffmpeg built with OpenCL, fdk-aac, h264-8/10/12bit, h265-multibit, av1, nvenc, so I'll have a look at that link !)
edit: ah, I preferred not to install docker. Oh well.
You do not need to use docker. https://github.com/BtbN/FFmpeg-Builds/releases those do have OpenCL, h264 is 12 bit by default, there is no such stuff anymore, there is no such library as av1 and nvenc is obviously on, as it is nvidia dev's repo.
comment:16 by , 4 years ago
Replying to mablip:
Replying to Ptaah:
I didn't find any relevant changes between 3.3.4 and 3.3.5 version. Where did you get 3.3.4 and 3.3.5 binaries? I would like to check what git commit are they based on.
Found anything useful between the versions?
Unfortunately I've been otherwise occupied for the past couple of weeks, but I have it in my todo list.
comment:17 by , 4 years ago
I've made some progress today. First I identified the range of versions in between which it got broken:
It works with avcodec 57.102.100 from 1.7.2017
It doesn't work with avcodec 57.105.100 from 1.9.2017
However I didn't find any significant changes in nvenc source files between these dates.
Then I went all in and began to parse the bitstream myself. What I noticed is, the good version generates bitstream with no SEI messages (simply SPS, PPS, IDR, P, ...), BUT the bad version generates two SEI messages after SPS and PPS. Second SEI has within itself pic_timing payload and it has pic_struct with value of 1 (H264_SEI_PIC_STRUCT_TOP_FIELD). After keyframe (which itself is top field) there is another SEI message with pic struct with value of 2 (H264_SEI_PIC_STRUCT_BOTTOM_FIELD).
Now let's look at the avcodec h264 decoder, specifically how it's determining field order in h264 bitstream.
h264_slice.c, line:1198
1198 if (cur->field_poc[0] != cur->field_poc[1]) { 1199 /* Derive top_field_first from field pocs. */ 1200 out->top_field_first = cur->field_poc[0] < cur->field_poc[1]; 1201 } else { 1202 if (sps->pic_struct_present_flag && h->sei.picture_timing.present) { 1203 /* Use picture timing SEI information. Even if it is a 1204 * information of a past frame, better than nothing. */ 1205 if (h->sei.picture_timing.pic_struct == H264_SEI_PIC_STRUCT_TOP_BOTTOM || 1206 h->sei.picture_timing.pic_struct == H264_SEI_PIC_STRUCT_TOP_BOTTOM_TOP) 1207 out->top_field_first = 1; 1208 else 1209 out->top_field_first = 0; 1210 } else if (out->interlaced_frame) { 1211 /* Default to top field first when pic_struct_present_flag 1212 * is not set but interlaced frame detected */ 1213 out->top_field_first = 1; 1214 } else { 1215 /* Most likely progressive */ 1216 out->top_field_first = 0; 1217 } 1218 }
When parsing bitstream generated by older version (without SEI) it either goes to line 1200, or 1213 (I'm not sure). However, when parsing bitstream generated by newer version (with SEI), it DEFINITELY goes to line 1209. Now it becomes clear. It's not a bug in nvenc implementation, it's seems to be a bug in avcodec h264 decoder, because it is not properly handling h264 interlaced with separated fields and SEI messages present. The reason might be simply because MBAFF is wildly more used than separated fields.
This hypothesis is supported by the fact, that MediaInfo correctly detects top field first with those specific files.
So the quick, even if not optimal, solution, is to strip output bitstream of all SEI. It will still work and the field order will be correctly detected by libavcodec.
One question however remains. Why nvenc started to output SEI nalu in h264 bitstream? I have no answer.
comment:18 by , 4 years ago
follow-up: 20 comment:19 by , 4 years ago
comment:20 by , 4 years ago
I don't think so, the bug is present in avcodec h264 decoder.
Replying to Balling:
Is this fixing this? https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210721172916.40739-1-jeebjp@gmail.com/
follow-up: 22 comment:21 by , 4 years ago
Does setting -vf setfield=tff fixes the issue? Also what about -bf -1 which is the default now?
follow-up: 23 comment:22 by , 4 years ago
comment:23 by , 4 years ago
Replying to Ptaah:
Replying to Balling:
Does setting -vf setfield=tff fixes the issue? Also what about -bf -1 which is the default now?
Just checked FFmpeg 4.4 and the issue seems to be resolved.
Is that because of 4c694093be68d401c60819e5171817c62afef8b2? Please test with -bf 0.
comment:24 by , 3 years ago
Hi, again. Can you test with -bf 0? I only have 2080 Ti, which does not support interlaced already, so cannot test.
follow-up: 27 comment:26 by , 21 months ago
Replying to Balling:
Can you retest?
Seems like it got worse somehow:
[FRAME] interlaced_frame=0 top_field_first=1 [/FRAME]
ffmpeg version is 2023-05-25-git-944243477b, nvidia driver 525.
comment:27 by , 16 months ago
Replying to Ptaah:
Replying to Balling:
Can you retest?
Seems like it got worse somehow:
[FRAME] interlaced_frame=0 top_field_first=1 [/FRAME]ffmpeg version is 2023-05-25-git-944243477b, nvidia driver 525.
Seems impossible after Nvidia driver ->520 with cuda 12 and forward to actually get an interlaced_frame. Everything fails for me with drivers above 520, impossible to get the correct flag.
comment:28 by , 3 months ago
When checking the encoded file with MediaInfo, it was found to be interlaced (TFF), but when played back, it was not deinterlaced.
After installing the driver (NVIDIA-Linux-x86_64-550.127.05.run), the video is now deinterlaced without any problems during playback.
The environment is as follows
ffmpeg: version N-114115-g9124d80
GPU: Quadro P400(Pascal)
Version with problem
Driver Version: 545.29.02
CUDA Version: 12.3
Version with fixed issue
Driver Version: 550.127.05
CUDA Version: 12.4
comment:29 by , 3 months ago
But we know 550 drive is first one without some big issues... https://forums.developer.nvidia.com/t/nvdec-quality-degradation-with-adaptive-deinterlace-since-535-drivers/273284/5
Which commit introduced the regression?
Please provide the command line you tested together with the complete, uncut console output to make this a valid ticket.