Opened 23 months ago

Last modified 23 months ago

#5696 open defect

ffmpeg broken for AES-encrypted HLS stream with Accept-Encoding header

Reported by: yan12125 Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: http
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
How to reproduce:

$ ffmpeg -loglevel debug -headers "Accept-Encoding: gzip, deflate" -i https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8 -c copy -f mp4 -bsf:a aac_adtstoasc output.mp4
ffmpeg version 3.1.1 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 6.1.1 (GCC) 20160602
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-netcdf --enable-shared --enable-version3 --enable-x11grab
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 47.100 /  6. 47.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-headers' ... matched as AVOption 'headers' with argument 'Accept-Encoding: gzip, deflate'.
Reading option '-i' ... matched as input file with argument 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mp4'.
Reading option '-bsf:a' ... matched as option 'bsf' (A comma-separated list of bitstream filters) with argument 'aac_adtstoasc'.
Reading option 'output.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8.
Successfully parsed a group of options.
Opening an input file: https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8.
[https @ 0x56373daa0320] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[https @ 0x56373daa0320] No trailing CRLF found in HTTP header.
[https @ 0x56373daa0320] request: GET /episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8 HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Range: bytes=0-
Connection: close
Host: d2sazdeahkz1yk.cloudfront.net
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[hls,applehttp @ 0x56373da9fb20] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x56373da9fb20] HLS request for url 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts', offset 0, playlist 0
[https @ 0x56373e0c8700] request: GET /episodes/S01E03-loading-view-controllers/hls_key HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: talk.objc.io
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[https @ 0x56373e0c8700] inflate return value: -3, invalid code lengths set
    Last message repeated 1 times
Unable to read key file https://talk.objc.io/episodes/S01E03-loading-view-controllers/hls_key
[AVIOContext @ 0x56373dc811a0] Statistics: 0 bytes read, 0 seeks
[https @ 0x56373deedc20] request: GET /episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: d2sazdeahkz1yk.cloudfront.net
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[hls,applehttp @ 0x56373da9fb20] Error when loading first segment 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts'
[AVIOContext @ 0x56373dbcc380] Statistics: 1050928 bytes read, 0 seeks
[AVIOContext @ 0x56373de18ce0] Statistics: 4997 bytes read, 0 seeks
https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8: Invalid data found when processing input

The key file is correctly gzipped:

$ curl "https://talk.objc.io/episodes/S01E03-loading-view-controllers/hls_key" | xxd
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    16    0    16    0     0     16      0 --:--:-- --:--:-- --:--:--    16
00000000: 9016 6c2e 9f21 f436 9478 2438 464e 917a  ..l..!.6.x$8FN.z

$ curl -H 'Accept-Encoding: gzip, deflate' "https://talk.objc.io/episodes/S01E03-loading-view-controllers/hls_key" | gunzip -c - | xxd
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    43    0    43    0     0     48      0 --:--:-- --:--:-- --:--:--    48
00000000: 9016 6c2e 9f21 f436 9478 2438 464e 917a  ..l..!.6.x$8FN.z

$ curl -H 'Accept-Encoding: gzip, deflate' "https://talk.objc.io/episodes/S01E03-loading-view-controllers/hls_key" | xxd
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    43    0    43    0     0     45      0 --:--:-- --:--:-- --:--:--    45
00000000: 1f8b 0800 2d89 7f57 0003 9a20 96a3 375f  ....-..W... ..7_
00000010: f18b d994 0a15 0b37 bf89 5500 0000 00ff  .......7..U.....
00000020: ff03 0070 464c a710 0000 00              ...pFL.....

Change History (6)

comment:1 Changed 23 months ago by cehoyos

Is this issue reproducible with current FFmpeg git head?

comment:2 Changed 23 months ago by yan12125

Let me try to build it first.

comment:3 Changed 23 months ago by yan12125

Same on git-master N-80924-gb450b82:

$ ffmpeg -loglevel debug -headers "Accept-Encoding: gzip, deflate" -i https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8 -c copy -f mp4 -bsf:a aac_adtstoasc output.mp4
ffmpeg version N-80924-gb450b82 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 6.1.1 (GCC) 20160602
  configuration: --prefix=/usr --disable-debug --disable-static --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libfdk-aac --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-nonfree --enable-shared --enable-version3 --enable-x11grab
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.103 / 57. 48.103
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 47.100 /  6. 47.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-headers' ... matched as AVOption 'headers' with argument 'Accept-Encoding: gzip, deflate'.
Reading option '-i' ... matched as input file with argument 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mp4'.
Reading option '-bsf:a' ... matched as option 'bsf' (A comma-separated list of bitstream filters) with argument 'aac_adtstoasc'.
Reading option 'output.mp4' ... matched as output file.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8.
Successfully parsed a group of options.
Opening an input file: https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8.
[https @ 0x55b97035e480] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[https @ 0x55b97035e480] No trailing CRLF found in HTTP header.
[https @ 0x55b97035e480] request: GET /episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8 HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Range: bytes=0-
Connection: close
Host: d2sazdeahkz1yk.cloudfront.net
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[hls,applehttp @ 0x55b97035db20] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x55b97035db20] HLS request for url 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts', offset 0, playlist 0
[https @ 0x55b97068f820] request: GET /episodes/S01E03-loading-view-controllers/hls_key HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: talk.objc.io
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[https @ 0x55b97068f820] inflate return value: -3, invalid code lengths set
    Last message repeated 1 times
Unable to read key file https://talk.objc.io/episodes/S01E03-loading-view-controllers/hls_key
[AVIOContext @ 0x55b970562a80] Statistics: 0 bytes read, 0 seeks
[https @ 0x55b970916980] request: GET /episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: d2sazdeahkz1yk.cloudfront.net
Icy-MetaData: 1
Accept-Encoding: gzip, deflate


[hls,applehttp @ 0x55b97035db20] Error when loading first segment 'https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p00000.ts'
[AVIOContext @ 0x55b97074f240] Statistics: 1050944 bytes read, 0 seeks
[AVIOContext @ 0x55b9704777a0] Statistics: 4997 bytes read, 0 seeks
https://d2sazdeahkz1yk.cloudfront.net/episodes/1b888bc4-d261-481d-a8ba-28c02b31eaab/1/1080p.m3u8: Invalid data found when processing input

comment:4 Changed 23 months ago by yan12125

  • Version changed from 3.0.2 to git-master

comment:5 Changed 23 months ago by cehoyos

  • Keywords http added

comment:6 Changed 23 months ago by cehoyos

  • Reproduced by developer set
  • Status changed from new to open
Note: See TracTickets for help on using tickets.