HLS single file playback broken when encryption enabled

[Bryan Murphy]

HLS files generated with encryption enabled and the single segment (byte range) option result in videos that are unplayable in QuickTime and Safari on OSX. The same videos play without issue when encryption is not utilized.

The specific behavior is that video playback starts, but there are periods where video playback gets stuck and then resumes, get stuck, and then resumes. It seems to be related to the different segment offsets where some segments play without issue while other segments get frozen on a single frame.

I can reproduce this with origin/master checked out on 9/22/16.

Using the following as input:

​https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/input.ts

I created a "good" version here which does not utilize encryption and plays back without issue:

​https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/good/good.m3u8

I also created a "bad" version here which utilizes encryption and exhibits the undesired behavior:

​https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8

Here is the console log for generating the "bad" version:

bash-4.3# openssl rand 16 > output/bad/bad.key
bash-4.3# cat <<EOF > input.conf
> bad.key
> output/bad/bad.key
> EOF
bash-4.3# ffmpeg -y -i input.ts -c copy -map 0 -hls_flags single_file -hls_list_size 0 -hls_playlist_type vod -hls_key_info_file input.conf output/bad/bad.m3u8
ffmpeg version N-81723-g6d9a46e Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.0 (Alpine 5.3.0)
  configuration: --prefix=/usr --enable-avresample --enable-avfilter --enable-gnutls --enable-gpl --enable-libmp3lame --enable-librtmp --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-x11grab --disable-stripping --disable-static --enable-vaapi --enable-libopus --enable-nonfree --enable-libfdk-aac
  libavutil      55. 30.100 / 55. 30.100
  libavcodec     57. 57.101 / 57. 57.101
  libavformat    57. 50.100 / 57. 50.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 62.100 /  6. 62.100
  libavresample   3.  0.  0 /  3.  0.  0
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[h264 @ 0x7fb1ee66fae0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fb1ee66fae0] SPS unavailable in decode_picture_timing
[h264 @ 0x7fb1ee66fae0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fb1ee66fae0] SPS unavailable in decode_picture_timing
Input #0, mpegts, from 'input.ts':
  Duration: 00:00:30.06, start: 1.415333, bitrate: 3982 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Stream #0:1[0x101](und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 120 kb/s
Output #0, hls, to 'output/bad/bad.m3u8':
  Metadata:
    encoder         : Lavf57.50.100
    Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 23.98 tbc
    Stream #0:1(und): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, 120 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  720 fps=0.0 q=-1.0 Lsize=N/A time=00:00:29.99 bitrate=N/A speed=49.3x
video:13001kB audio:471kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

[Bryan Murphy]

For anybody else who may be running into this issue, we've had better success using ​Shaka Packager instead of FFmpeg to mux the video.

[Daniel Tiesling]

I'm experiencing this issue as well. Are there any workarounds?

@bmurphy1976 could you point me toward a Shaka example?

[Bryan Murphy]

Here's an example, hopefully this is enough to get you started:

packager \
	input=input.mp4,stream=video,playlist_name=video.m3u8,segment_template=video.m3u8.$Number$.ts \
	input=input.mp4,stream=audio,playlist_name=audio.m3u8,segment_template=audio.m3u8.$Number$.ts,hls_group_id=audio,hls_name=ENGLISH \
	--hls_master_playlist_output=output.m3u8

You probably want to look at these options as well:

--aes_signing_key $AES_KEY
--aes_signing_iv  $AES_IV

I'm not currently using the built-in AES encryption, I'm using older code I wrote that adds the encryption to the HLS files post-mux, but I think those are the correct options.

Also, keep in mind that Shaka spits out the audio and video streams as separate sets of segmented files. I have not found a way to create a combined audio/video set using Shaka.

That said, I've also found that mp42hls ​https://github.com/axiomatic-systems/Bento4/tree/master/Source/C%2B%2B/Apps may be another viable option. I'm still experimenting with it, but so far it at least works on my test videos (unlike ffmpeg). This one will spit out a combined audio/video stream so it may work better for you depending on your needs and licensing restrictions.