h264_qsv decoder does not work with RTSP
|Reported by:||Steve Sampson||Owned by:|
|Blocking:||Reproduced by developer:||no|
|Analyzed by developer:||no|
Description (last modified by )
Summary of the bug:
How to reproduce:
using ffmpeg version: ffmpeg version 5.0.1-full_build from (https://www.gyan.dev/ffmpeg/builds/) built on 2022-04-07. (Windows version) ffmpeg.exe -init_hw_device qsv:hw,child_device_type=d3d11va -hwaccel qsv -c:v h264_qsv -hwaccel_output_format qsv -i rtsp://user:firstname.lastname@example.org/axis-media/media.amp -f null -
Running on Windows 10 with an Intel GPU (Intel UHD Graphics 620, but I suspect this problem isn't related to the exact hardware version - it also reproduced on another machine). The RTSP stream comes from a camera. It looks like it has a pixel format of 720x576, pix_fmt: yuvj420p. If there is a way to capture this stream such that someone else can reproduce the error, please let me know. I suspect the reproduction of this problem requires a genuine RTSP server.
When running, it initializes the connection to the RTSP server and then logs an error
Error initializing the MFX video decoder: invalid video parameters (-15)
The same command line successfully decodes using an mp4 file instead of an RTSP input (captured from the camera using ffmpeg -i rtsp://user:email@example.com/axis-media/media.amp -acodec copy -vcodec copy c:/abc.mp4)
The RTSP stream decodes successfully without any hardware acceleration. That is, with a command line of "ffmpeg.exe -i rtsp://user:firstname.lastname@example.org/axis-media/media.amp -f null -". It also is correctly displayed in VLC. I tried many different combinations of command line parameters (for example just with -hwaccel qsv -c:v h264_qsv) and none were successful.
Note that I am not at all an expert on FFMPEG. I was tasked to GPU-enable our existing code and this is my first serious look at FFMPEG.
The use case is to display the video from a set of cameras (all RTSP, all H264). Our code uses the FFMPEG API and doesn't use the FFMPEG program but my tests have been using FFMPEG.
The actual build I wish(ed) to use is an older build of ffmpeg (branched in 2019 and slightly modified by my team - but the decoding wasn't touched). It gives a different error string but it may be a similar root cause. I tried to debug it to figure out what was going wrong. The error in that version was "non-existing PPS 0 referenced" (from parse_nal_units) and then "Decoding pixel format '(null)' is not supported" (from ff_qsv_process_data).
I can't easily make my own build of FFMPEG (the process of building with Windows is too complex for me to quickly do) or I would attempt to debug using the latest code.
The SPD returned by the camera is the following
SDP: v=0 o=- 1651685682795324 1651685682795324 IN IP4 172.16.1.105 s=Media Presentation e=NONE b=AS:50000 t=0 0 a=control:* a=range:npt=0.000000- m=video 0 RTP/AVP 96 c=IN IP4 0.0.0.0 b=AS:50000 a=framerate:25.0 a=transform:0.937500,0.000000,0.000000;0.000000,1.000000,0.000000;0.000000,0.000000,1.000000 a=control:trackID=1 a=rtpmap:96 H264/90000 a=fmtp:96 packetization-mode=1; profile-level-id=420029; sprop-parameter-sets=Z0IAKeKQFoJNgScFAQXh4kRU,aM48gA==