Encrypted HLS - wrong ranges in .m3u8 file

[d3im]

Summary of the bug:
How to reproduce:

% ffmpeg -i input.mp4 -c:v libx264 -crf 28 -profile high -level 41 -keyint_min 25 -c:a libfdk_aac -vbr 1 -hls_time 6 -hls_list_size 0 '''-hls_flags single_file''' -hls_playlist_type vod -hls_key_info_file file.keyinfo -f hls fftest/cle.m3u8

ffmpeg version 3.3.2 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 5.4.0 (Gentoo 5.4.0-r3 p1.3, pie-0.6.5)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.3.2/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags='-march=haswell -mavx2 -msse4.2 -msse4.1 -O2 -pipe -mmmx -msse -msse2 -mssse3 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=8192 -mtune=haswell' --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-version3 --disable-indev=oss --disable-outdev=oss --enable-nonfree --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --disable-gmp --enable-gpl --enable-hardcoded-tables --enable-iconv --enable-lzma --enable-network --disable-openssl --enable-postproc --enable-libsmbclient --enable-ffplay --enable-sdl2 --enable-vaapi --disable-vdpau --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --enable-libpulse --disable-libopencore-amrwb --disable-libopencore-amrnb --enable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disable-mmal --disable-libmodplug --enable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libschroedinger --enable-libspeex --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-libbs2b --disable-chromaprint --disable-libflite --enable-frei0r --disable-libfribidi --disable-fontconfig --enable-ladspa --disable-libass --enable-libfreetype --disable-librubberband --disable-netcdf --disable-libzmq --disable-libzimg --enable-libvidstab --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --disable-nvenc --disable-libopenh264 --disable-libsnappy --enable-libtheora --disable-libtwolame --disable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=haswell --disable-doc --disable-htmlpages --enable-manpages
libavutil      55. 58.100 / 55. 58.100
libavcodec     57. 89.100 / 57. 89.100
libavformat    57. 71.100 / 57. 71.100
libavdevice    57.  6.100 / 57.  6.100
libavfilter     6. 82.100 /  6. 82.100
libavresample   3.  5.  0 /  3.  5.  0
libswscale      4.  6.100 /  4.  6.100
libswresample   2.  7.100 /  2.  7.100
libpostproc    54.  5.100 / 54.  5.100

$ cat file.keyinfo
file.key
fftest/file.key

file.key is 16-byte file genereted by
openssl rand 16 > fftest/file.key

----
$ cat cle.m3u8
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:12
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="file.key"
#EXTINF:12.000000,
#EXT-X-BYTERANGE:8383484@0
cle.ts
#EXTINF:2.208333,
#EXT-X-BYTERANGE:597088@8383484
cle.ts
#EXTINF:10.416667,
#EXT-X-BYTERANGE:1874548@8980572
cle.ts
#EXTINF:3.166667,
#EXT-X-BYTERANGE:497448@10855120
cle.ts
#EXTINF:8.458333,
#EXT-X-BYTERANGE:4049332@11352568
cle.ts
#EXTINF:10.416667,
#EXT-X-BYTERANGE:1993364@15401900
cle.ts
#EXTINF:3.333333,
#EXT-X-BYTERANGE:641456@17395264
cle.ts
#EXT-X-ENDLIST

As workaround I used this:

ffmpeg -i input.mp4 -c:v libx264 -crf 28 -profile high -level 41 -keyint_min 25 -c:a libfdk_aac -vbr 1 -hls_time 6 -hls_list_size 0 -hls_playlist_type vod -hls_key_info_file file.keyinfo -f hls fftest/cle_s.m3u8

cat cle_s*.ts > clex.ts
and computed ranges manually. And it works now.

----
#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:12
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-KEY:METHOD=AES-128,URI="file.key"
#EXTINF:12.000000,
#EXT-X-BYTERANGE:8383488@0
clex.ts
#EXTINF:2.208333,
#EXT-X-BYTERANGE:597280@8383488
clex.ts
#EXTINF:10.416667,
#EXT-X-BYTERANGE:1874752@8980768
clex.ts
#EXTINF:3.166667,
#EXT-X-BYTERANGE:497648@10855520
clex.ts
#EXTINF:8.458333,
#EXT-X-BYTERANGE:4049536@11353168
clex.ts
#EXTINF:10.416667,
#EXT-X-BYTERANGE:1993568@15402704
clex.ts
#EXTINF:3.333333,
#EXT-X-BYTERANGE:641648@17396272
clex.ts
#EXT-X-ENDLIST

It seems The ranges are counted right before encrypting since non-encrypted HLS encoding have same ranges.
Fix seems to be made easy - count size of encrypted chunk instead of non-encrypted. But i really didn't search source, so I only guessing. Thanks.

[Steven Liu]

mhmm, hlsenc has avio_tell after write frame, but the offset get from avio_tell is incorrect
@Carl, do you have any advice?

[d3im]

Any progress on this? I'd like to use this functionality soon... Thanks

[Carl Eugen Hoyos]

Is this still reproducible?