HLS streaming crashes, when "http_persistent" option enabled

[Serhii Marchuk]

FFmpeg crashes when streaming HLS to remote server with "http_persistent" option enabled.
After ~10 minute streaming it crashes with SIGSEGV.

Remote server is nginx with dav module. Configuration example:

location / {
    root  www;

    client_body_temp_path www/client_temp;

    dav_methods PUT DELETE;

    create_full_put_path  on;
    min_delete_depth 1;
    autoindex on;
    autoindex_format json;
    autoindex_exact_size off;
    autoindex_localtime off;

    add_header Accept-Ranges bytes;
}

How to reproduce:

% ffmpeg -v 9 -loglevel 99 -report -re -i ~/_garbage/samples/Starcraft_2_Replay.mkv -vcodec copy -hls_time 5 -http_persistent 1 -method 'PUT' -f HLS http://localhost/hls/test.m3u8

ffmpeg version N-48357-g2dfa362093 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.2.0-8ubuntu3.2)
  configuration: --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-hwaccels --enable-libfdk-aac --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libvpx --enable-openssl --enable-libzvbi --enable-libspeex

Core dump:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  ffurl_write (h=0x0, buf=buf@entry=0x7fff5a7538e2 "0\r\n\r\n", size=size@entry=5) at libavformat/avio.c:423
423	    if (!(h->flags & AVIO_FLAG_WRITE))
[Current thread is 1 (Thread 0x7feabc7268c0 (LWP 17245))]
(gdb) bt full
#0  ffurl_write (h=0x0, buf=buf@entry=0x7fff5a7538e2 "0\r\n\r\n", size=size@entry=5) at libavformat/avio.c:423
No locals.
#1  0x00007feabbaf4326 in http_shutdown (h=<optimized out>, flags=<optimized out>) at libavformat/http.c:1623
        ret = 0
        footer = "0\r\n\r\n"
        s = 0x5574239a6000
#2  0x00007feabbaee688 in hlsenc_io_close (filename=0x7fff5a7539b0 "http://localhost/hls/test.m3u8", pb=0x5574238d7670, s=0x5574238a8d80) at libavformat/hlsenc.c:288
        http_url_context = 0x5574239d6700
        hls = <optimized out>
        http_base_proto = <optimized out>
#3  hls_window (s=s@entry=0x5574238a8d80, last=last@entry=0, vs=vs@entry=0x557423918280) at libavformat/hlsenc.c:1434
        hls = 0x5574238d6d00
        en = <optimized out>
        target_duration = <optimized out>
        ret = -32
        temp_filename = "http://localhost/hls/test.m3u8\000\002\020:uZ\377\177\000\000\032\000\000\000\377\177\000\000\030\000\000\000\000\000\000\000\220\001\000\000\000\000\000\000\240\001\000\000\000\000\000\000\000\026\314\300%\350\362\002@:uZ\377\177\000\000@:uZ\377\177\000\000\300\000\000\000\000\000\000\000\300\001\000\000\000\000\000\000\006\000\000\000\000\000\000\000\032\000\000\000\066", '\000' <repeats 17 times>, "\061\060\067\063\067\064\061\070\062\062\000\026\314\300%\350\362\002\220:uZ\377\177\000\000\220:uZ\377\177\000\000p=uZ\377\177\000\000\252\355û\352\177\000\000\020<uZ\377\177\000\000\001"...
        sequence = 97
        proto = <optimized out>
        use_rename = 0
        warned_non_file = 102
        key_uri = <optimized out>
        iv_string = <optimized out>
        options = 0x0
        prog_date_time = 486.40000000000038
        prog_date_time_p = <optimized out>
        byterange_mode = <optimized out>
#4  0x00007feabbaf25b6 in hls_write_packet (s=0x5574238a8d80, pkt=<optimized out>) at libavformat/hlsenc.c:2264
        old_filename = 0x5574238a5320 "\200.\222#tU"
        byterange_mode = <optimized out>
        hls = 0x5574238d6d00
        oc = 0x557423918840
        st = <optimized out>
        end_pts = <optimized out>
        is_ref_pkt = <optimized out>
        ret = <optimized out>
        can_split = <optimized out>
        i = <optimized out>
        j = <optimized out>
        stream_index = 0
        range_length = <optimized out>
        buffer = 0x0
        vs = 0x557423918280
#5  0x00007feabbb65558 in write_packet (pkt=0x7fff5a753f00, s=0x5574238a8d80) at libavformat/mux.c:747
        ret = <optimized out>
        pts_backup = 46080000
        dts_backup = 46076400
#6  av_interleaved_write_frame (s=0x5574238a8d80, pkt=0x0) at libavformat/mux.c:1231
        opkt = {buf = 0x55742392b400, pts = 46083600, dts = 46080000, data = 0x557423b08ff0 "", size = 221256, stream_index = 0, flags = 1, side_data = 0x0, side_data_elems = 0, duration = 3600, pos = -1, 
          convergence_duration = 0}

[Serhii Marchuk]

Report

[Serhii Marchuk]

Core dump

[Carl Eugen Hoyos]

ffmpeg version N-48357-g2dfa362093

Could you confirm that the issue is reproducible with current unpatched FFmpeg?

[Serhii Marchuk]

Yes. ffmpeg version N-48365-g40102a2137

Nginx log before crash

2018/02/28 14:39:51 [debug] 25573#0: *1 recv: eof:1, avail:1
2018/02/28 14:39:51 [debug] 25573#0: *1 recv: fd:3 -1 of 4096
2018/02/28 14:39:51 [info] 25573#0: *1 recv() failed (104: Connection reset by peer), client: 127.0.0.1, server: localhost, request: "PUT /hls/test99.ts HTTP/1.1", host: "localhost"

[Carl Eugen Hoyos]

This version does not exist, please test current vanilla FFmpeg.

[Serhii Marchuk]

The "http_persistent" is not available in ffmpeg version 3.4.2.

[Carl Eugen Hoyos]

Please test and confirm that the crash you see is reproducible with vanilla current FFmpeg git head, not only with your custom version of FFmpeg with its local changes.

[Serhii Marchuk]

Yes, it reproducible with master branch. Without any patches. (​https://git.ffmpeg.org/ffmpeg.git) (commit 40102a21374096ce0ba05c67c6e7474f176af2d0 Tue Feb 27 17:51:02 2018 -0300)

ffmpeg version N-90182-g40102a2137 Copyright (c) 2000-2018 the FFmpeg developers

built with gcc 7 (Ubuntu 7.2.0-8ubuntu3.2)
configuration: --enable-shared --enable-nonfree --enable-gpl --enable-version3 --enable-hwaccels --enable-libfdk-aac --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libvpx --enable-openssl -
libavutil 56. 7.101 / 56. 7.101
libavcodec 58. 13.100 / 58. 13.100
libavformat 58. 10.100 / 58. 10.100
libavdevice 58. 2.100 / 58. 2.100
libavfilter 7. 12.100 / 7. 12.100
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

[Karthick Jeyapal]

There is a patch submitted to fix this issue. ​http://ffmpeg.org/pipermail/ffmpeg-devel/2018-February/225036.html
But some concerns were raised over that fix. We didn't get time so far to look into alternate fixes.

Anyways we feel that the concerns raised were hypocritical, as a similar fix in avio was recently pushed. ​https://github.com/FFmpeg/FFmpeg/commit/be4dfbf7b71e44a53ca8da882a081e35ea134c83 .