[FFProbe] invalid byte range requests

[Tom]

Summary of the bug:
When seeking isn't disabled (-seekable 0), with some input files ffprobe attempts to make byte range requests past the end of the file.

The response for the file (from s3) that is being requested has a

Content-Length: 393740

header.

You can see from the output below that ffprobe is requesting from byte 393740 and then from byte 8590008746, both of which are invalid.

ffprobe works correctly with the -seekable 0 flag on the same file.

I am not sure what the correct behaviour should be here.

Maybe:

• Exit with an error immediately before attempting a request with an invalid byte range
• If the seekable option is -1, automatically retry with seeking disabled
• ??

How to reproduce:
Unfortunately I am unable to provide a file with this problem here. But this shows the output.

% ffprobe "https://example.com/somefile.caf" -loglevel debug
ffprobe version N-90650-g3e9d676192 Copyright (c) 2007-2018 the FFmpeg developers
  built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
  configuration: --disable-x86asm
  libavutil      56. 13.100 / 56. 13.100
  libavcodec     58. 17.100 / 58. 17.100
  libavformat    58. 11.101 / 58. 11.101
  libavdevice    58.  2.100 / 58.  2.100
  libavfilter     7. 14.100 /  7. 14.100
  libswscale      5.  0.102 /  5.  0.102
  libswresample   3.  0.101 /  3.  0.101
[NULL @ 0x7f7f2f002e00] Opening 'https://<<redacted>>' for reading
[https @ 0x7f7f2ef00140] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[https @ 0x7f7f2ef00140] request: GET /<<redacted>> HTTP/1.1
User-Agent: Lavf/58.11.101
Accept: */*
Range: bytes=0-
Connection: close
Host: <<redacted>>.s3.amazonaws.com
Icy-MetaData: 1


[caf @ 0x7f7f2f002e00] Format caf probed with size=2048 and score=100
[https @ 0x7f7f2ef00140] request: GET /<<redacted>> HTTP/1.1
User-Agent: Lavf/58.11.101
Accept: */*
Range: bytes=393740-
Connection: close
Host: <<redacted>>.s3.amazonaws.com
Icy-MetaData: 1


[https @ 0x7f7f2ef00140] HTTP error 416 Requested Range Not Satisfiable
[https @ 0x7f7f2ef00140] request: GET /<<redacted>> HTTP/1.1
User-Agent: Lavf/58.11.101
Accept: */*
Range: bytes=393740-
Connection: close
Host: <<redacted>>.s3.amazonaws.com
Icy-MetaData: 1


[https @ 0x7f7f2ef00140] HTTP error 416 Requested Range Not Satisfiable
[caf @ 0x7f7f2f002e00] skipping CAF chunk: 00000000 ([0][0][0][0]), size 0
    Last message repeated 375 times
[caf @ 0x7f7f2f002e00] skipping CAF chunk: 00000000 ([0][0][0][0]), size 8590000126
[https @ 0x7f7f2ef00140] request: GET /<<redacted>> HTTP/1.1
User-Agent: Lavf/58.11.101
Accept: */*
Range: bytes=8590008746-
Connection: close
Host: <<redacted>.s3.amazonaws.com
Icy-MetaData: 1


[https @ 0x7f7f2ef00140] HTTP error 416 Requested Range Not Satisfiable
[caf @ 0x7f7f2f002e00] skipping CAF chunk: 00000000 ([0][0][0][0]), size 281457796579324
[https @ 0x7f7f2ef00140] request: GET /<<redacted>> HTTP/1.1
User-Agent: Lavf/58.11.101
Accept: */*
Range: bytes=281457796587956-
Connection: close
Host: <<redacted>.s3.amazonaws.com
Icy-MetaData: 1


[https @ 0x7f7f2ef00140] HTTP error 416 Requested Range Not Satisfiable
[caf @ 0x7f7f2f002e00] skipping CAF chunk: FFFBFFF9 ([255][251][255][249]), size -562971428651010
[AVIOContext @ 0x7f7f2ec12140] Statistics: 16347 bytes read, 0 seeks
https://<<redacted>>: Invalid data found when processing input

[Carl Eugen Hoyos]

I believe what you describe is the expected behaviour to evaluate the server capabilities but feel free to provide an actual command line including complete, uncut console output that allows to reproduce the issue..

[bubbleguuum]

I have observed the same thing on *some* (but not all) mp4 http streams hosted on Google Drive.
ffprobe will hammer the server like mad with repeated bogus Range requests resulting in 416 until eventually stopping after a minute or two. This would not be so bad if it didn't trigger Google Drive max download request limit for the file and flagging it *permanently* as not downloadable due to reaching max quota allowed (all subsequent access result in a 403).
ffprobe should not be doing these random Range requests (with -seekable 1), this is bogus and unnecessary and has a nasty side effect on Google Drive and other quota limiting servers due to the extreme hammering.
Providing a reproducible example is difficult as the Google Drive http download links are short lived.

[bubbleguuum]

Ah well... Apparently it is hard to understand the ffprobe should not hammer a web server with bogus (out of content-length) Range request... At least, I will have tried but this is a waste of time.

[Cigaes]

Apparently it is hard to understand that with no way of reproducing the issue it is not possible to fix it.