Opened 5 years ago

Closed 3 years ago

#7919 closed defect (needs_more_info)

Live extracting subtitles from the stream

Reported by: Mikhail Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords: webvtt
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no


Good day
Experiencing problems when creating streaming playback.
I get the input file in parts (from webtorrent), in the process of downloading it I will convert it using ffmpeg and give it to the user in the live mode. I can switch audio and video streams. There is a problem with the subtitles.

I use the following options:

ffmpeg -i '/home/mishkapetran/Загрузки/Rick.mkv' \
-map 0:v:0 -c:v libx264 -profile:v baseline -preset:v superfast -b:v 1000k -level 3.0 -strict -2 -s 480x360 -f hls -hls_time 10 -hls_list_size 0 -segment_list rick_360p -hls_segment_filename /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_360p_video_%d.ts /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_360p_video.m3u8 \
-map 0:v:0 -c:v libx264 -profile:v baseline -preset:v superfast -b:v 1000k -level 3.0 -strict -2 -s 858x480 -f hls -hls_time 10 -hls_list_size 0 -segment_list rick_480p -hls_segment_filename /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_480p_video_%d.ts /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_480p_video.m3u8 \
-map 0:v:0 -c:v libx264 -profile:v baseline -preset:v superfast -b:v 1000k -level 3.0 -strict -2 -s 1280x720 -f hls -hls_time 10 -hls_list_size 0 -segment_list rick_720p -hls_segment_filename /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_720p_video_%d.ts /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_720p_video.m3u8 \
-map 0:1 -c:a aac -b:a 128k -ar 44100 -ac 2 -f hls -hls_time 10 -hls_list_size 0 -segment_list rick_audio_rus -hls_segment_filename /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_rus_audio_%d.aac /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_rus_audio.m3u8 \
-map 0:2 -c:a aac -b:a 128k -ar 44100 -ac 2 -f hls -hls_time 10 -hls_list_size 0 -segment_list rick_audio_eng -hls_segment_filename /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_eng_audio_%d.aac /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_eng_audio.m3u8

The average conversion speed on my local PC is about 2.5x (without subtitles).
As soon as I add subtitles to the options, the speed drops to 0.5x, and this is not acceptable for streaming.

My options for subtitles:

-map 0:3 -f segment -segment_time 10 -segment_format webvtt /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_sub_eng-%d.vtt /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_sub_eng.m3u8 \
-map 0:4 -f segment -segment_time 10 -segment_format webvtt /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_sub_rus-%d.vtt /home/mishkapetran/m3u8_test/hls_test/video_stream_source/rick_sub_rus.m3u8

I noticed:
1) If you convert to .webvtt, then .ts video elements are created along with the subtitle files.
2) If I use just .vtt, without fragments, then writing to the file occurs only after the file is completely processed. But for the .m3u8 playlist, the .vtt format is not suitable, so it will also stick to processing in parallel.

Tell me how you can implement the extraction of subtitles in my case.

My ffmpeg:

ffmpeg version 4.1.3-0york1~18.04 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version='0york1~18.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-nonfree --enable-libfdk-aac --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Change History (2)

comment:1 by Carl Eugen Hoyos, 5 years ago

Component: ffmpegundetermined
Keywords: ffmpeg subtitle live stream m3u8 vtt removed
Version: 4.1unspecified

To make this a valid ticket, please test current FFmpeg git head, provide the (simplified) command line that you tested and that allows to reproduce the issue together with the complete, uncut console output and provide an input sample.

comment:2 by Carl Eugen Hoyos, 3 years ago

Resolution: needs_more_info
Status: newclosed
Note: See TracTickets for help on using tickets.