Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#8652 closed defect (fixed)

Segmentation fault with HLS var_stream_map

Reported by: udyojiar Owned by:
Priority: important Component: avformat
Version: git-master Keywords: hls crash SIGSEGV
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
Hello,
I have one video I'd like to convert it to hls in two profiles with var_stream_map.
At the end of the conversation FFmpeg dies with segmentation fault.
The result is the same with ffmpeg-4.2.2 and ffmpeg-snapshot.

How to reproduce:

% ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi \
-i FPS_test_1080p25_L4.1.mkv \
-filter_complex 'split[cq][cb]' \
-map '[cq]' -c:v h264_vaapi -qp 18 -map \
'[cb]' -c:v h264_vaapi -b:v 5M -maxrate 5M    \
-f hls -var_stream_map "v:0_v:1" \
-master_pl_name playlist.m3u8 -master_pl_publish_rate 5 \
-use_localtime 1 -hls_segment_type mpegts -hls_time 10 \
-hls_list_size 6 -hls_flags delete_segments \
-hls_segment_filename "/dev/shm/abr/503/%v/%s.bin" \
      "/dev/shm/abr/503/%v/playlist.m3u8"

ffmpeg version N-97605-g422f1e3 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (GCC)
  configuration: --prefix=/home/ico/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ico/ffmpeg_build/include --extra-ldflags=-L/home/ico/ffmpeg_build/lib --extra-libs=-lpthread --extra-libs=-lm --bindir=/home/ico/bin --enable-gpl --enable-libfdk_aac --enable-libx264 --enable-vaapi --enable-nonfree
  libavutil      56. 43.100 / 56. 43.100
  libavcodec     58. 82.100 / 58. 82.100
  libavformat    58. 42.102 / 58. 42.102
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from 'FPS_test_1080p25_L4.1.mkv':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.1
    creation_time   : 2014-09-18T20:01:03.000000Z
  Duration: 00:10:05.04, start: 0.000000, bitrate: 218 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Metadata:
      BPS             : 216733
      BPS-eng         : 216733
      DURATION        : 00:10:05.040000000
      DURATION-eng    : 00:10:05.040000000
      NUMBER_OF_FRAMES: 15126
      NUMBER_OF_FRAMES-eng: 15126
      NUMBER_OF_BYTES : 16391549
      NUMBER_OF_BYTES-eng: 16391549
      _STATISTICS_WRITING_APP: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_DATE_UTC: 2014-09-18 20:01:03
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-09-18 20:01:03
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, fltp, 44 kb/s (default)
    Metadata:
      BPS             : 34181
      BPS-eng         : 34181
      DURATION        : 00:00:01.056000000
      DURATION-eng    : 00:00:01.056000000
      NUMBER_OF_FRAMES: 44
      NUMBER_OF_FRAMES-eng: 44
      NUMBER_OF_BYTES : 4512
      NUMBER_OF_BYTES-eng: 4512
      _STATISTICS_WRITING_APP: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_DATE_UTC: 2014-09-18 20:01:03
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-09-18 20:01:03
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Multiple -c, -codec, -acodec, -vcodec, -scodec or -dcodec options specified for stream 0, only the last option '-c:v h264_vaapi' will be used.
Multiple -c, -codec, -acodec, -vcodec, -scodec or -dcodec options specified for stream 1, only the last option '-c:v h264_vaapi' will be used.
failed to open /usr/lib64/dri/hybrid_drv_video.so
Not using hybrid_drv_video.so
Stream mapping:
  Stream #0:0 (h264) -> split
  split:output0 -> Stream #0:0 (h264_vaapi)
  split:output1 -> Stream #0:1 (h264_vaapi)
Press [q] to stop, [?] for help
Segmentation fault (core dumped)

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (8)

comment:1 by udyojiar, 4 years ago

Here is the output of "gdb path/to/ffmpeg(_g)" ... "bt full"

Reading symbols from ffmpeg_sources/ffmpeg/ffmpeg_g...done.
(gdb) run -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i FPS_test_1080p25_L4.1.mkv -filter_complex 'split[cq][cb]' -map '[cq]' -c:v h264_vaapi -qp 18 -map '[cb]' -c:v h264_vaapi -b:v 5M -maxrate 5M    -f hls -var_stream_map "v:0_v:1" -master_pl_name playlist.m3u8 -master_pl_publish_rate 5 \
      -use_localtime 1 -hls_segment_type mpegts -hls_time 10 -hls_list_size 6 \
      -hls_flags delete_segments -hls_segment_filename "/dev/shm/abr/503/%v/%s.bin" \
      "/dev/shm/abr/503/%v/playlist.m3u8"
Starting program: /home/ico/ffmpeg_sources/ffmpeg/ffmpeg_g -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i FPS_test_1080p25_L4.1.mkv -filter_complex 'split[cq][cb]' -map '[cq]' -c:v h264_vaapi -qp 18 -map '[cb]' -c:v h264_vaapi -b:v 5M -maxrate 5M    -f hls -var_stream_map "v:0_v:1" -master_pl_name playlist.m3u8 -master_pl_publish_rate 5       -use_localtime 1 -hls_segment_type mpegts -hls_time 10 -hls_list_size 6       -hls_flags delete_segments -hls_segment_filename "/dev/shm/abr/503/%v/%s.bin"       "/dev/shm/abr/503/%v/playlist.m3u8"
Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-72.el8_1.1.x86_64
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
ffmpeg version N-97605-g422f1e3 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (GCC)
  configuration: --prefix=/home/ico/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ico/ffmpeg_build/include --extra-ldflags=-L/home/ico/ffmpeg_build/lib --extra-libs=-lpthread --extra-libs=-lm --bindir=/home/ico/bin --enable-gpl --enable-libfdk_aac --enable-libx264 --enable-vaapi --enable-nonfree
  libavutil      56. 43.100 / 56. 43.100
  libavcodec     58. 82.100 / 58. 82.100
  libavformat    58. 42.102 / 58. 42.102
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, matroska,webm, from 'FPS_test_1080p25_L4.1.mkv':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.1
    creation_time   : 2014-09-18T20:01:03.000000Z
  Duration: 00:10:05.04, start: 0.000000, bitrate: 218 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Metadata:
      BPS             : 216733
      BPS-eng         : 216733
      DURATION        : 00:10:05.040000000
      DURATION-eng    : 00:10:05.040000000
      NUMBER_OF_FRAMES: 15126
      NUMBER_OF_FRAMES-eng: 15126
      NUMBER_OF_BYTES : 16391549
      NUMBER_OF_BYTES-eng: 16391549
      _STATISTICS_WRITING_APP: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_DATE_UTC: 2014-09-18 20:01:03
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-09-18 20:01:03
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, fltp, 44 kb/s (default)
    Metadata:
      BPS             : 34181
      BPS-eng         : 34181
      DURATION        : 00:00:01.056000000
      DURATION-eng    : 00:00:01.056000000
      NUMBER_OF_FRAMES: 44
      NUMBER_OF_FRAMES-eng: 44
      NUMBER_OF_BYTES : 4512
      NUMBER_OF_BYTES-eng: 4512
      _STATISTICS_WRITING_APP: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_APP-eng: mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
      _STATISTICS_WRITING_DATE_UTC: 2014-09-18 20:01:03
      _STATISTICS_WRITING_DATE_UTC-eng: 2014-09-18 20:01:03
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Multiple -c, -codec, -acodec, -vcodec, -scodec or -dcodec options specified for stream 0, only the last option '-c:v h264_vaapi' will be used.
Multiple -c, -codec, -acodec, -vcodec, -scodec or -dcodec options specified for stream 1, only the last option '-c:v h264_vaapi' will be used.
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
warning: Loadable section ".note.gnu.property" outside of ELF segments
failed to open /usr/lib64/dri/hybrid_drv_video.so
Not using hybrid_drv_video.so
[New Thread 0x7ffff5a1e700 (LWP 25211)]
[New Thread 0x7ffff521d700 (LWP 25212)]
[New Thread 0x7ffff4a1c700 (LWP 25213)]
[New Thread 0x7ffff421b700 (LWP 25214)]
[New Thread 0x7ffff3a1a700 (LWP 25215)]
Stream mapping:
  Stream #0:0 (h264) -> split
  split:output0 -> Stream #0:0 (h264_vaapi)
  split:output1 -> Stream #0:1 (h264_vaapi)
Press [q] to stop, [?] for help
[New Thread 0x7ffff2f19700 (LWP 25216)]
[New Thread 0x7ffff2718700 (LWP 25217)]
[New Thread 0x7ffff1f17700 (LWP 25218)]
[New Thread 0x7ffff1716700 (LWP 25219)]

Thread 1 "ffmpeg_g" received signal SIGSEGV, Segmentation fault.
0x0000000000709e16 in hls_init (s=<optimized out>) at libavformat/hlsenc.c:2841
2841	            vs->has_video += vs->streams[j]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO;
Missing separate debuginfos, use: yum debuginfo-install bzip2-libs-1.0.6-26.el8.x86_64 libdrm-2.4.98-2.el8.x86_64 libpciaccess-0.14-1.el8.x86_64 libva-2.4.1-1.el8.x86_64 libva-intel-driver-2.4.0-1.el8.x86_64 zlib-1.2.11-10.el8.x86_64
(gdb) bt full
#0  0x0000000000709e16 in hls_init (s=<optimized out>) at libavformat/hlsenc.c:2841
        ret = 0
        i = 0
        j = <optimized out>
        hls = <optimized out>
        pattern = 0x156fafe "%d.ts"
        vs = 0x2dbac00
        basename_size = <optimized out>
        pattern_localtime_fmt = 0x156faf7 "-%s.ts"
        vtt_pattern = 0x156fb0b "%d.vtt"
        p = <optimized out>
        vtt_basename_size = <optimized out>
        http_base_proto = <optimized out>
        fmp4_init_filename_len = <optimized out>
#1  0x0000000000771d91 in init_muxer (s=s@entry=0x27a9e00, options=0x2899b88) at libavformat/mux.c:405
        ret = <optimized out>
        i = <optimized out>
        st = <optimized out>
        tmp = 0x0
        par = <optimized out>
        of = 0x1c4e920 <ff_hls_muxer>
        desc = <optimized out>
        e = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--c
#2  0x0000000000773d65 in avformat_init_output (options=0x8, s=0x27a9e00) at libavformat/mux.c:510
        ret = 0
        ret = <optimized out>
#3  avformat_write_header (s=0x27a9e00, options=options@entry=0x2899b88) at libavformat/mux.c:510
        ret = 0
        already_initialized = <optimized out>
        streams_already_initialized = 0
#4  0x00000000004a5b1b in check_init_output_file (of=0x2899b80, file_index=0) at fftools/ffmpeg.c:2955
        ret = <optimized out>
        i = <optimized out>
#5  0x00000000004a5d68 in init_output_stream (ost=ost@entry=0x27c8e40, error=error@entry=0x7fffffffd350 "", error_len=1024) at fftools/ffmpeg.c:3597
        ret = <optimized out>
#6  0x00000000004a9cae in reap_filters (flush=flush@entry=0) at fftools/ffmpeg.c:1427
        error = '\000' <repeats 1023 times>
        filter = 0x2bac280
        ret = 0
        ost = 0x27c8e40
        of = <optimized out>
        enc = 0x27c9080
        i = 1
#7  0x00000000004ac8b4 in transcode_step () at fftools/ffmpeg.c:4616
        ost = <optimized out>
        ist = <optimized out>
        ret = <optimized out>
        ost = <optimized out>
        ist = <optimized out>
        ret = <optimized out>
        error = <optimized out>
        i = <optimized out>
        ifilter = <optimized out>
#8  transcode () at fftools/ffmpeg.c:4660
        cur_time = 120222696971
        ret = <optimized out>
        i = <optimized out>
        os = <optimized out>
        ost = <optimized out>
        ist = <optimized out>
        timer_start = 120222687938
        total_packets_written = 0
#9  0x000000000048a3ee in main (argc=46, argv=0x7fffffffe058) at fftools/ffmpeg.c:4861
        i = <optimized out>
        ret = <optimized out>
        ti = {real_usec = 120222659506, user_usec = 11365, sys_usec = 3790}
(gdb)

comment:2 by Carl Eugen Hoyos, 4 years ago

Priority: normal → important

Is a specific input file needed to reproduce?

comment:3 by udyojiar, 4 years ago

I think that any input file will do. The file in example is from kodi samples "4. H.264 1080p/25.000 (MKV)" ​https://drive.google.com/file/d/0BwxFVkl63-lEWDUzUVUtZEw4cDA/view. At first I've tried with mpeg-ts input stream with the same result.

comment:4 by mkver, 4 years ago

Please test this patch:

From 1ec665efd699a5915d5b2028c5971176e3b6c1bd Mon Sep 17 00:00:00 2001
From: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Date: Tue, 5 May 2020 00:40:44 +0200
Subject: [PATCH] avformat/hlsenc: Improve checks for invalid stream mappings

The mapping of streams to the various variant streams to be created by
the HLS muxer is roughly as follows: Space and tab separate variant
stream group maps while the entries in each variant stream group map are
separated by ','.

The parsing process of each variant stream group proceeded as follows:
At first the number of occurences of "a:", "v:" and "s:" in each variant
stream group is calculated so that one can can allocate an array of
streams with this number of entries. Then each entry is checked and the
check for stream numbers was deficient: It did check that there is a
number beginning after the ":", but it did not check that the number
extends until the next "," (or until the end).

This means that an invalid variant stream group like v:0_v:1 will not be
rejected; the problem is that the variant stream in this example is
supposed to have two streams associated with it (because it contains two
"v:"), yet only one stream is actually associated with it (because there
is no ',' to start a second stream specifier). This discrepancy led to
segfaults (null pointer dereferencing) in the rest of the code (when the
nonexistent second stream associated to the variant stream was involved.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
---
 libavformat/hlsenc.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/libavformat/hlsenc.c b/libavformat/hlsenc.c
index 008a3f3947..843b906203 100644
--- a/libavformat/hlsenc.c
+++ b/libavformat/hlsenc.c
@@ -1867,7 +1867,7 @@ fail:
 
 static int get_nth_codec_stream_index(AVFormatContext *s,
                                       enum AVMediaType codec_type,
-                                      int stream_id)
+                                      int64_t stream_id)
 {
     unsigned int stream_index, cnt;
     if (stream_id < 0 || stream_id > s->nb_streams - 1)
@@ -1950,6 +1950,8 @@ static int parse_variant_stream_mapstring(AVFormatContext *s)
 
         nb_streams = 0;
         while (keyval = av_strtok(varstr, ",", &saveptr2)) {
+            int64_t num;
+            char *end;
             varstr = NULL;
             if (av_strstart(keyval, "language:", &val)) {
                 av_free(vs->language);
@@ -1998,10 +2000,12 @@ static int parse_variant_stream_mapstring(AVFormatContext *s)
                 return AVERROR(EINVAL);
             }
 
-            stream_index = -1;
-            if (av_isdigit(*val))
-                stream_index = get_nth_codec_stream_index (s, codec_type,
-                                                           atoi(val));
+            num = strtoll(val, &end, 10);
+            if (!av_isdigit(*val) || *end != '\0' || val == end) {
+                av_log(s, AV_LOG_ERROR, "Invalid stream number: '%s'\n", val);
+                return AVERROR(EINVAL);
+            }
+            stream_index = get_nth_codec_stream_index(s, codec_type, num);
 
             if (stream_index >= 0 && nb_streams < vs->nb_streams) {
                 for (i = 0; nb_streams > 0 && i < nb_streams; i++) {
-- 
2.20.1

comment:5 by udyojiar, 4 years ago

The suggested patch is improvement. No Segmentation fault with patch applied.

The result now is:

Press [q] to stop, [?] for help
[hls @ 0x3cc7600] Invalid stream number: '0_v:1'
[hls @ 0x3cc7600] Variant stream info update failed with status ffffffea
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:1 -- 
Conversion failed!

comment:6 by mkver, 4 years ago

And that's what you deserve as you use an invalid format (the entries of a variant stream group are separated by ',', not '_'.). You probably wanted to use "v:0,v:1". What makes you believe that you used a valid format?

comment:7 by mkver, 4 years ago

Resolution: → fixed
Status: new → closed

Fixed in c801ab43c36e8c4f88121aa09af26c77bcbd671b.

comment:8 by udyojiar, 4 years ago

thanks

Note: See TracTickets for help on using tickets.