Added "-extension_picky" breaks various applications [Please open a separate ticket for each case so they can be tracked individually]

[QFox]

Summary of the bug:

mpv is using FFmpeg as key component to play audio/video.

Due to commit: ​https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/91d96dc8ddaebe0b6cb393f672085e6bfaf15a31

mpv/lavf is reporting Leaking connections bug.

[ffmpeg/demuxer] hls: detected format extension mov,mp4,m4a,3gp,3g2,mj2,psp,m4b,ism,ismv,isma,f4v,avif,heic,heif mismatches allowed extensions in url ​https://rr3---sn-cxaaj5o5q5-tt1r.googlevideo.com/videoplayback/id/d6935484ffd60804/itag/616/source/youtube/expire/1738051801/ei/eTyYZ6yPGpa20_wPiJezwAw/ip/174.89.123.164/requiressl/yes/ratebypass/yes/pfa/1/wft/1/sgovp/clen%3D90375315%3Bdur%3D269.520%3Bgir%3Dyes%3Bitag%3D356%3Blmt%3D1733983639269433/rqh/1/hls_chunk_host/rr3---sn-cxaaj5o5q5-tt1r.googlevideo.com/xpc/EgVo2aDSNQ%3D%3D/met/1738030201,/mh/hh/mm/31,26/mn/sn-cxaaj5o5q5-tt1r,sn-vgqsrnzd/ms/au,onr/mv/m/mvi/3/pl/24/rms/au,au/initcwndbps/4113750/bui/AY2Et-PZvOvbBluKfu1Qx327HBtdLOn9ruSh2ztKRfZsC-sDAcUOg6_xvfHXz-3NkGX_Sy8ccFG3RU-W/spc/9kzgDUrgWmNzFeQ1y3C4BL_QCGCWYV77wJBk3qS9_65oDx9d7OvxwuE/vprv/1/playlist_type/DVR/txp/4532434/mt/1738029886/fvip/2/keepalive/yes/fexp/51326932,51353498,51371294/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,pfa,wft,sgovp,rqh,xpc,bui,spc,vprv,playlist_type/sig/AJfQdSswRAIgSWCTm0dbCijtvY35m9FgNvMdISFOiqQnC44eouqkNM8CIEoYVqvp31W2K4UdqRRKuXGcmsigIbsFzzrB-9PI_0bc/lsparams/hls_chunk_host,met,mh,mm,mn,ms,mv,mvi,pl,rms,initcwndbps/lsig/AGluJ3MwRQIgGEtjXwe1GSIJQTTPfibgm2t0Ud9DYKWqu_dka23hxlgCIQCmItoENltdCZUQ3GY2Q6UnHKm72vvjh68nnoU4RfGv5A%3D%3D/playlist/index.m3u8/govp/slices%3D0-681101/gosq/0/file/seg.ts
[ffmpeg/demuxer] hls: Error when loading first segment '​https://rr3---sn-cxaaj5o5q5-tt1r.googlevideo.com/videoplayback/id/d6935484ffd60804/itag/616/source/youtube/expire/1738051801/ei/eTyYZ6yPGpa20_wPiJezwAw/ip/174.89.123.164/requiressl/yes/ratebypass/yes/pfa/1/wft/1/sgovp/clen%3D90375315%3Bdur%3D269.520%3Bgir%3Dyes%3Bitag%3D356%3Blmt%3D1733983639269433/rqh/1/hls_chunk_host/rr3---sn-cxaaj5o5q5-tt1r.googlevideo.com/xpc/EgVo2aDSNQ%3D%3D/met/1738030201,/mh/hh/mm/31,26/mn/sn-cxaaj5o5q5-tt1r,sn-vgqsrnzd/ms/au,onr/mv/m/mvi/3/pl/24/rms/au,au/initcwndbps/4113750/bui/AY2Et-PZvOvbBluKfu1Qx327HBtdLOn9ruSh2ztKRfZsC-sDAcUOg6_xvfHXz-3NkGX_Sy8ccFG3RU-W/spc/9kzgDUrgWmNzFeQ1y3C4BL_QCGCWYV77wJBk3qS9_65oDx9d7OvxwuE/vprv/1/playlist_type/DVR/txp/4532434/mt/1738029886/fvip/2/keepalive/yes/fexp/51326932,51353498,51371294/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,pfa,wft,sgovp,rqh,xpc,bui,spc,vprv,playlist_type/sig/AJfQdSswRAIgSWCTm0dbCijtvY35m9FgNvMdISFOiqQnC44eouqkNM8CIEoYVqvp31W2K4UdqRRKuXGcmsigIbsFzzrB-9PI_0bc/lsparams/hls_chunk_host,met,mh,mm,mn,ms,mv,mvi,pl,rms,initcwndbps/lsig/AGluJ3MwRQIgGEtjXwe1GSIJQTTPfibgm2t0Ud9DYKWqu_dka23hxlgCIQCmItoENltdCZUQ3GY2Q6UnHKm72vvjh68nnoU4RfGv5A%3D%3D/playlist/index.m3u8/govp/slices%3D0-681101/gosq/0/file/seg.ts'
[lavf] avformat_open_input() failed
[lavf] Leaking 1 nested connections (FFmpeg bug).
EDL: source file '​https://manifest.googlevideo.com/api/manifest/hls_playlist/expire/1738051801/ei/eTyYZ6yPGpa20_wPiJezwAw/ip/174.89.123.164/id/d6935484ffd60804/itag/616/source/youtube/requiressl/yes/ratebypass/yes/pfa/1/wft/1/sgovp/clen%3D90375315%3Bdur%3D269.520%3Bgir%3Dyes%3Bitag%3D356%3Blmt%3D1733983639269433/rqh/1/hls_chunk_host/rr3---sn-cxaaj5o5q5-tt1r.googlevideo.com/xpc/EgVo2aDSNQ%3D%3D/met/1738030201,/mh/hh/mm/31,26/mn/sn-cxaaj5o5q5-tt1r,sn-vgqsrnzd/ms/au,onr/mv/m/mvi/3/pl/24/rms/au,au/initcwndbps/4113750/bui/AY2Et-PZvOvbBluKfu1Qx327HBtdLOn9ruSh2ztKRfZsC-sDAcUOg6_xvfHXz-3NkGX_Sy8ccFG3RU-W/spc/9kzgDUrgWmNzFeQ1y3C4BL_QCGCWYV77wJBk3qS9_65oDx9d7OvxwuE/vprv/1/playlist_type/DVR/dover/13/txp/4532434/mt/1738029886/fvip/2/short_key/1/keepalive/yes/fexp/51326932,51353498,51371294/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,pfa,wft,sgovp,rqh,xpc,bui,spc,vprv,playlist_type/sig/AJfQdSswRAIgSWCTm0dbCijtvY35m9FgNvMdISFOiqQnC44eouqkNM8CIEoYVqvp31W2K4UdqRRKuXGcmsigIbsFzzrB-9PI_0bc/lsparams/hls_chunk_host,met,mh,mm,mn,ms,mv,mvi,pl,rms,initcwndbps/lsig/AGluJ3MwRQIgGEtjXwe1GSIJQTTPfibgm2t0Ud9DYKWqu_dka23hxlgCIQCmItoENltdCZUQ3GY2Q6UnHKm72vvjh68nnoU4RfGv5A%3D%3D/playlist/index.m3u8' has unknown duration.

How to reproduce:

download mpv here: https://github.com/shinchiro/mpv-winbuild-cmake/releases, using 20250128 version as example, play youtube video

mpv https://www.youtube.com/watch?v=1pNUhP_WCAQ


FFmpeg version: N-118369-g959b799c8

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

[MasterQuestionable]

͏    "[lavf] Leaking 1 nested connections (FFmpeg bug)."
͏    ; is MPV message.
͏    [ More details: ​https://github.com/mpv-player/mpv/pull/15947#issue-2873340938 ]

͏    "hls: detected format extension ... mismatches allowed extensions in url"
͏    "hls: Error when loading first segment"
͏    "[lavf] avformat_open_input() failed"
͏    .
͏    Is it really actual bug? Or misleading display..?
͏    ​https://github.com/FFmpeg/FFmpeg/commit/91d96dc8ddaebe0b6cb393f672085e6bfaf15a31#diff-42c0585a46cff2eadc3030e3d4b53b8413f1ecbacd6eeba68593c4e4343ac07aR761

͏    Seems like MPV bug?
͏    The change seems to cause back-compat incompatibility indeed.

͏    Explain somewhat the actual usage? (so may better estimate the rationale)

͏    ----

͏    I fear it would probably have the potential of breaking every downstream application...
͏    (via the default behavior change)

[Michael Niedermayer]

potenial patch is here: ​https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-January/338934.html

(this fixes the extension issue maybe)

please tell me if it fixes it, otherwise provide a testcase that works with ffmpeg alone or provide more details how to setup a mpv testcase on a ubuntu lts (not windows)

any leaked connections are unrelated to the quoted commit as no new exit pathes are introduced.

[MasterQuestionable]

͏    Maybe not making the new option default enabled?
͏    (so reduce potential breakage)

[QFox]

The patch fixed the problem.

But I don't know if it really fix the root cause.

[QFox]

I have no knowledge on running mpv on Ubuntu.
What I did on Windows, was configuring the mpv to call yt-dlp to access the YouTube stream.

mpv ​https://www.youtube.com/watch?v=1pNUhP_WCAQ --script-opts=ytdl_hook-ytdl_path=D:\Tools\yt-dlp\yt-dlp.exe -v

[MasterQuestionable]

͏    Does `yt-dlp` use FFmpeg directly for download?
͏    (it seems to use ​͏"curl-impersonate")

͏    See also [ ​https://github.com/yt-dlp/yt-dlp/issues/9710#issue-2249649551 ] for `yt-dlp` deployment.

[QFox]

As far as I know, yt-dlp is using curl_cffi and using FFmpeg to fix the downloaded file.

[MasterQuestionable]

͏    This should only affect that directly use FFmpeg to access network streams.
͏    Probably not `yt-dlp`. But unsure how it handles HLS formats?
͏    (I scarcely use)

[Michael Niedermayer]

Patch applied, i hope this fixes it, 9e12572933dc1c49e9b35d772ddcae896c2ba8a8

[MasterQuestionable]

͏    Hope the various services properly use filename extension...
͏    What would happen if it has no extension at all?

[kasper93]

The same issue on Twitter.

python -m yt_dlp https://x.com/<any_hls_video> --get-url
mpv -v <m3u8 playlist from the above>

[  0.694143]    ffmpeg/demuxer: hls: detected format mov,mp4,m4a,3gp,3g2,mj2 extension mov,mp4,m4a,3gp,3g2,mj2,psp,m4b,ism,ismv,isma,f4v,avif,heic,heif mismatches allowed extensions in url https://video.twimg.com/ext_tw_video/<...>.m4s
[  0.694239]    ffmpeg/demuxer: hls: Error when loading first segment 'https://video.twimg.com/ext_tw_video/<...>.m4s'
[  0.695505]              lavf: avformat_open_input() failed
[  0.695553]              lavf: Leaking 1 nested connections (FFmpeg bug).

[MasterQuestionable]

͏    Very likely that the services won't properly use filename extension...
͏    This option probably cannot be made default.

͏    Another breakage:
͏    ​https://trac.ffmpeg.org/ticket/11477
͏    ​https://trac.ffmpeg.org/ticket/11526
͏    ​https://trac.ffmpeg.org/ticket/11535

[gamer191]

Another breakage, this time with the .fmp4 extension: ​https://github.com/yt-dlp/yt-dlp/issues/12700

[MasterQuestionable]

͏    That really matters appears to be the arbitrary "file://" support in HLS..?
͏    ​https://bugzilla.redhat.com/show_bug.cgi?id=2334338

͏    But is it possible for the data exfiltration to remote?
͏    (not familiar enough with the protocol and potential interfaces)

[gamer191]

That really matters appears to be the arbitrary "​file://" support in HLS..?

I'm sure this has been brought up before. My suspicion (which could be wrong) is that ​file:// urls are only supported if the input is a local file.

To test that, I ran ffmpeg -i https://gist.githubusercontent.com/gamer191/f670714b0775ca5ef80f42e23c640da9/raw/4dca5244b3507ebaddd919701f5ed11edd23c1a2/test.m3u8 flag.mp4 which resulted in Protocol 'file' not on whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'!

[MasterQuestionable]

͏    Even if the remote source may arbitrarily read local files:
͏    As long as there's no interface to send the data out: shall be alright.

͏    The CVE description is quite unclear.
͏    ​https://github.com/advisories/GHSA-398c-f7w9-crc8
͏    (very much just the link in comment:14)

[Qotscha]

I use FFmpeg to download recordings from Elisa Viihde (Finnish online recording service) using HLS protocol. Updating FFmpeg broke downloading because .ec3 extension is used for EAC3 audio tracks.

[hls @ 000001d3298c99c0] URL https://viihde-amfcache29-pa.cdn.elisaviihde.fi/cdn/v5/qxqLMM3qqtQg2DbKy2EOUB-123-netpvr-37-20230401T060510000~20230401T070000000/qxqLMM3qqtQg2DbKy2EOUB-123-netpvr-37-20230401T060510000~20230401T070000000-audio_dut_1=256000-1.ec3 is not in allowed_extensions

I can get around this problem by using allowed_extensions option, but I wonder if .ec3 could be included in the allowed extensions by default.

[MasterQuestionable]

͏    I believe a thorough review on what the security problem really is:
͏    Is necessary to really address the fundamental problems underlying.

[uncoder]

Interesting that nowhere in RFC8216 ​https://www.rfc-editor.org/rfc/rfc8216 does it enforce file extensions.

Yeah, Apple's extended HLS vendor spec enforces a MIME type of vnd.apple.mpegURL, video/MP2T or video/mp4, above and the beyond RFC and webVTT spec asks for a MIME type of text/vtt (ref ​https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API/Web_Video_Text_Tracks_Format) but none of those enforce fixed file name extensions.

Yeah, there's good reason to enforce protocol/schema for http/https:// vs file:, since HTTP(S) is implicit in HLS, but file extensions were never defined.

From a real-world vendor perspective, there are several manifest-manipulator HLS services (eg AWS MediaTailor etc) that perform per-segment content replacement that do not provide extensions for their segments.

The CVS that triggered this change requests constraints above and beyond the HLS specification, requests constraints beyond Apple's vendor implementation. Yes, there are players that use the file extension to determine whether to follow a TS or MP4 path in the code (when they should be using MIME/content-type), but there is nowhere in the HLS specification that requires file extension.

I question the validity of the original CVS against spec.

[Michael Niedermayer]

Hello, everyone, a few important points:

1. please try to provide a replicable testcase. a m3u8 file and a whatever funny extension it has file thats correctly linked from that m3u8 file is an option. A working http(s) link is an option too.
2. If i cannot replicate it, i cannot replicate that a change fixes your reported issue. So iam then fixing things blindly and the fixes maybe wont completely fix it.
3. Also replicateble testcases are insanely usefull for future regression testing. Its very hard to do regression testing with 3rd party tools and remote urls that will not be the same format in a year

[Michael Niedermayer]

Please check the patches in this thread: ​https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2025-April/341929.html
They basically add the extensions to allowed_extensions. That may be enough or it may be not enough. I need feedback (and replicable testcases if it fails) for that

[kasper93]

please try to provide a replicable testcase. a m3u8 file and a whatever funny extension it has file thats correctly linked from that m3u8 file is an option. A working http(s) link is an option too.

​https://github.com/mpv-player/mpv/issues/15973

[Michael Niedermayer]

Replying to kasper93:

please try to provide a replicable testcase. a m3u8 file and a whatever funny extension it has file thats correctly linked from that m3u8 file is an option. A working http(s) link is an option too.

​https://github.com/mpv-player/mpv/issues/15973

which of these is it:
./ffmpeg -i ​https://github.com/mpv-player/mpv/issues/15973
./ffmpeg -i ​https://www.youtube.com/live/l8PMl7tUDIE?si=b3ytG9iBbq3v6rTb
??? (btw the 2nd is not a joke, that could be done and would be welcome)
Note this IS the reason this issue is not, and can not be fixed (by me). I have no testcase.
if you mean this:
mpv ​https://www.youtube.com/live/l8PMl7tUDIE?si=b3ytG9iBbq3v6rTb
that doesnt work with the mpv i have installed, nor does that use the ffmpeg i just applied a bugfix to. So if thats your testcase then please also provide full instructions how to build a working mpv linked to a locally modified ffmpeg

[kasper93]

??? (btw the 2nd is not a joke, that could be done and would be welcome)

​https://github.com/yt-dlp/yt-dlp/wiki/Installation

yt-dlp -f 234+270 https://www.youtube.com/live/l8PMl7tUDIE

[Michael Niedermayer]

Replying to kasper93:

??? (btw the 2nd is not a joke, that could be done and would be welcome)

​https://github.com/yt-dlp/yt-dlp/wiki/Installation

yt-dlp -f 234+270 https://www.youtube.com/live/l8PMl7tUDIE

Fix for this is on the ML, review/test welcome

[MasterQuestionable]

͏    I think the first thing would be, to assure the validity of the so called CVE itself.
͏    Else it would simply be:
͏    Invalid CVE caused invalid fix that caused valid breakage..?

[cgbug]

This should be fixed with a whitelist of allowed demuxers rather than an error prone list of file extensions.

[pal1000]

Reproducer:

ffmpeg -referer "https://flash1.bogulus.cfd/000/1reality.html" -i "https://flash1.bogulus.cfd/realy/usergenrx0j3qt.m3u8" -c copy -f mpegts - >nul

Legal note: Realitatea Plus, a Romanian news TV channel, is free to air. It also provides an open for all ​official live stream, but I reproduced this issue with the unofficial cool-etv.net stream.

[MasterQuestionable]

͏    Off-Topic:
͏    Does simply using "https://flash1.bogulus.cfd/" as the Referer work, in your instance?
͏    Refer: ​https://github.com/MasterInQuestion/talk/discussions/11#discussioncomment-8591289
͏    .
͏    I'm researching the relevant things, and seeking feedback.
͏    Thanks in advance.

[pal1000]

Replying to MasterQuestionable:

͏    Off-Topic:
͏    Does simply using "https://flash1.bogulus.cfd/" as the Referer work, in your instance?

Yes, it does. The trailing '/' is required.

[MasterQuestionable]

͏    Would:
͏    "https://flash1.bogulus.cfd/realy/usergenrx0j3qt.m3u8"
͏    ; work too..?
͏    (out of curiosity)

͏    And?
͏    "https://flash1.bogulus.cfd/realy/"
͏    "https://flash1.bogulus.cfd/000/"

͏    Many thanks.

[pal1000]

Replying to MasterQuestionable:

͏    Would:
͏    "https://flash1.bogulus.cfd/realy/usergenrx0j3qt.m3u8"
͏    ; work too..?
͏    (out of curiosity)

͏    And?
͏    "https://flash1.bogulus.cfd/realy/"
͏    "https://flash1.bogulus.cfd/000/"

͏    Many thanks.

All of these work.

[pal1000]

My testing indicate that working around this with -extension_picky 0 is more reliable than -allowed_extensions ALL. There are random streams for which the latter doesn't work for any good reason.

ffmpeg -hide_banner -allowed_extensions ALL -referer "https://onyx1.mofta.cfd/" -i "https://onyx1.mofta.cfd/nike/usergenm24qrnd.m3u8" -c copy -f mpegts - > nul
[hls @ 000001fe1ceef5c0] Skip ('#EXT-X-VERSION:3')
[hls @ 000001fe1ceef5c0] Opening 'https://onyx1.mofta.cfd/nike/usergenm24qrnd-got.htm' for reading
[hls @ 000001fe1ceef5c0] Skip ('#EXT-X-VERSION:3')
[hls @ 000001fe1ceef5c0] Opening 'https://onyx1.mofta.cfd/nike/tokenized22193431userxqh2312lgx34v1gk.html' for reading
[hls @ 000001fe1ceef5c0] Opening 'https://onyx1.mofta.cfd/nike/tokenized22193440userxqh2402lgx34v1gk.html' for reading
[hls @ 000001fe1ceef5c0] detected format mpegts extension none mismatches allowed extensions in url https://onyx1.mofta.cfd/nike/tokenized22193216userxqh2162lgx32v1gk.html
[hls @ 000001fe1ceef5c0] Error when loading first segment 'https://onyx1.mofta.cfd/nike/tokenized22193216userxqh2162lgx32v1gk.html'
[in#0 @ 000001fe1ceef200] Error opening input: Invalid data found when processing input
Error opening input file https://onyx1.mofta.cfd/nike/usergenm24qrnd.m3u8.
Error opening input files: Invalid data found when processing input

ffmpeg -hide_banner -extension_picky 0 -referer "https://onyx1.mofta.cfd/" -i "https://onyx1.mofta.cfd/nike/usergenm24qrnd.m3u8" -c copy -f mpegts - > nul
[hls @ 000001b1f384f580] Skip ('#EXT-X-VERSION:3')
[hls @ 000001b1f384f580] Opening 'https://onyx1.mofta.cfd/nike/usergenm24qrnd-got.htm' for reading
[hls @ 000001b1f384f580] Skip ('#EXT-X-VERSION:3')
[hls @ 000001b1f384f580] Opening 'https://onyx1.mofta.cfd/nike/tokenized22193525userxqh2252lgx35v1gk.html' for reading
[hls @ 000001b1f384f580] Opening 'https://onyx1.mofta.cfd/nike/tokenized22193534userxqh2342lgx35v1gk.html' for reading