FFmpeg doesn't work when using an encoded URL

[boris-petrov]

When using an encoded URL (at least with the FTP protocol, it's probably the same with HTTP and others), FFmpeg complains.

How to reproduce:

% ffprobe -of json -show_entries format=stream=codec_type "ftp://ffmpeg:ffmpeg%2B%2F2018@ftp.pbteu.com/ExportHD.mpg"
ffprobe version 4.1.git Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20181127
  configuration: --extra-libs='-lm -lpthread' --prefix=/home/boris/ffmpeg_build --pkg-config-flags=--static --extra-cflags='-I/home/boris/ffmpeg_build/include -I/opt/cuda/include/' --extra-ldflags='-L/home/boris/ffmpeg_build/lib -L/opt/cuda/lib64/' --bindir=/home/boris/bin --enable-version3 --enable-pic --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-openssl --enable-libsmbclient
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 46.100 / 58. 46.100
  libavformat    58. 26.100 / 58. 26.100
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
{
[ftp @ 0x555a056c5240] FTP authentication failed
[ftp @ 0x555a056c5240] FTP open failed
ftp://ffmpeg:ffmpeg%2B%2F2018@ftp.pbteu.com/ExportHD.mpg: Permission denied

}

You can see I've passed a username and a password (which is URL encoded as it contains a "/" character which is otherwise not possible to be sent). The password is "ffmpeg+/2018". A curl command works just fine with the same URL:

% curl "ftp://ffmpeg:ffmpeg%2B%2F2018@ftp.pbteu.com/ExportHD.mpg"
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

[Carl Eugen Hoyos]

The given command line cannot work, there is no url encoding for ftp.
It may be a security feature that / in username or password is not supported, could be a bug.

[Carl Eugen Hoyos]

Should be fixed in dd06f022b07438d650c82255dff16908ba04244a

[boris-petrov]

@cehoyos - thank you for the fix. Does this also work now for encoded "+" signs in the command (as in my example in the original post - "%2B")?

[boris-petrov]

@cehoyos - this still doesn't work for other encoded symbols like the "+" sign in the original post. And also for no other encoded symbol than the ones that are hardcoded. Please try out the command I've given. It still doesn't work even with a "master" build.

[boris-petrov]

@cehoyos - this still doesn't work for other encoded symbols like the "+" sign in the original post. And also for no other encoded symbol than the ones that are hardcoded. Please try out the command I've given. It still doesn't work even with a "master" build.

[Carl Eugen Hoyos]

$ ffmpeg -i ftp://ffmpeg:ffmpeg+/2018@ftp.pbteu.com/ExportHD.mpg
ffmpeg version N-93664-g1ae5a64 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.4.0 (GCC)
  configuration: --enable-opengl --enable-gpl --enable-libxml2 --enable-gnutls --enable-gmp --enable-version3 --enable-libx264
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 52.100 / 58. 52.100
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 49.100 /  7. 49.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, mpeg, from 'ftp://ffmpeg:ffmpeg+/2018@ftp.pbteu.com/ExportHD.mpg':
  Duration: 00:00:11.69, start: 0.479989, bitrate: 26433 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (High), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 256 kb/s

[boris-petrov]

I know it works this way but what I'm saying is that the command should work with an *encoded* URL.

For example, if there is a @ sign in the password, you *have* to encode it before passing it to FFmpeg (or any other program) because the @ sign has special meaning in URLs - it marks the end of the "username:password" part and the start of the hostname. Right now there is *no* way to make FFmpeg work with @ sign in the username or password because it expects a decoded URL. The same goes for : in the username. The right way to fix this is to make FFmpeg work with encoded URLs in all its protocols.

[Carl Eugen Hoyos]

Are you sure a colon is allowed as part of the user name?

[boris-petrov]

Well, I guess it depends on the protocol, but I haven't read about any protocol that has restrictions on what characters can be used in the username and password. I'll try to change the password in the example above to also contain a @ as an example. This will be next week. But in any case the correct behavior would be for FFmpeg to work with encoded URLs - then no such problems will exist.

[Carl Eugen Hoyos]

Sorry for being unclear: The rfc forbids colons as part of the username.

[Carl Eugen Hoyos]

It also states that ftp does not know any encoding (which explains why it didn’t work for you).

[boris-petrov]

Which RFC are you referring to? Also, I'm not sure what exactly you mean by "ftp does not know any encoding". I believe any character is allowed in any place in a URL if it is properly URL encoded. In my first post you saw the same URL being used in FFmpeg and in cURL. cURL correctly handles URL encoded characters and does the correct thing/request. FFmpeg expects URL decoded characters which is a problem when special characters like @, :, /, etc are used in specific places.

[Hendrik]

FFmpeg does quite simply not support any URL encoding, and that seems like it would be out of scope for us as well.

[boris-petrov]

Well, that's what this issue is all about. :) I've written it in the title.

You could reopen the issue if you feel that this is something that has to be fixed/implemented at some point in FFmpeg. I believe it should be.

[Marton Balint]

Reopened after c61d16cf3b815388c076c2a35b5c91e84b45d7be.

[Marton Balint]

a8ec0685ac1cfbeb0e87f47b86d4f0b5cf75d745 provides a workaround for ftp.