Opened 2 months ago

Closed 6 weeks ago

#6109 closed defect (worksforme)

Video freeze when overlaying 2 RTSP streams in PIP, if network error occurs

Reported by: pero Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Hi All,

Summary of the bug:

I have two IP cameras. The internet connection of the cameras is not so stable, so there are network errors. I produce HLS stream with FFmpeg.

If I connect only to one camera with ffmpeg and if a network error occurs then ffmpeg exits. I think this is the expected behavior, I can workaround this with an infinite loop in a bash script.

But if I connect to both of the cameras and create "picture in picture" effect with overlay filter and a network error occurs, then a lot of times one of the camera's picture is freezing, but the other camera is streamed later on. FFmpeg does not exit, so I have no idea how to workaround this. I think if the streaming of one of the cameras stops for any reason, then FFmpeg should exit.

How to reproduce:
Here is the full ffmpeg command I use:


ffmpeg -nostdin -nostats 
-rtsp_transport tcp -thread_queue_size 512 -i rtsp://$IP:555 
-rtsp_transport tcp -thread_queue_size 512 -i rtsp://$IP:556 
-i /root/streams/vp.feszek/watermark_HD.png 
-filter_complex
"
[1]fps=25[belul];
[0]fps=25,scale=736:552,crop=w=222:h=552:x=288:y=0[kivul];
[belul[kivul]overlay=5:5[pip];[pip][2]overlay=0:0,
drawtext=fontfile=/root/fonts/courbd.ttf:textfile=/root/streams/vp.feszek/meteo/TEMP:fontsize=26:fontcolor=white:x=690:y=18:reload=1,
drawtext=fontfile=/root/fonts/courbd.ttf:textfile=/root/streams/vp.feszek/meteo/HUM:fontsize=26:fontcolor=white:x=930-tw:y=18:reload=1,
drawtext=fontfile=/root/fonts/courbd.ttf:text='%{localtime\:%Y.%m.%d.%T}':fontsize=30:fontcolor=white:x=w-390:y=h-100:shadowcolor=black:shadowx=1:shadowy=1,
split=5 [720p] [l1] [l2] [l3] [c];[c] fps=fps=1 [cur];[l1]scale=848:480 [480p];[l2]scale=640:360 [360p];[l3]scale=424:240 [240p]
"
-map [720p] -map 1:a -c:a aac -ab 64k -c:v h264_nvenc -b:v 1800k -r 25 -g 100 -bf 2 -refs 4 -no-scenecut 1 -strict_gop 1 -profile:v high -pixel_format yuv420p -preset default -metadata title="InfoCAM.hu Madarles B odu 720p"
-f tee "[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/720.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/$datum/720p/playlist.m3u8" 
-map [480p] -map 1:a -c:a aac -ab 64k -c:v h264_nvenc -b:v 1000k -r 25 -g 100 -bf 2 -refs 4 -no-scenecut 1 -strict_gop 1 -profile:v main -pixel_format yuv420p -preset default -metadata title="InfoCAM.hu Madarles B odu 480p" 
-f tee "[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/480.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/$datum/480p/playlist.m3u8" 
-map [360p] -map 1:a -c:a aac -ab 64k -c:v h264_nvenc -b:v 500k -r 25 -g 100 -bf 2 -refs 4 -no-scenecut 1 -strict_gop 1 -profile:v main -pixel_format yuv420p -preset default -metadata title="InfoCAM.hu Madarles B odu 360p" 
-f tee "[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/360.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/$datum/360p/playlist.m3u8" 
-map [240p] -map 1:a -c:a aac -ab 64k -c:v h264_nvenc -b:v 300k -r 25 -g 100 -bf 2 -refs 4 -no-scenecut 1 -strict_gop 1 -profile:v main -pixel_format yuv420p -preset default -metadata title="InfoCAM.hu Madarles B odu 240p" 
-f tee "[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/240.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/$datum/240p/playlist.m3u8" 
-map [cur] -f image2 -y -update 1 -r 1 -qscale:v 3 /var/www/html/nv/madarles-bodu/current.jpg

Output of FFmpeg:

ffmpeg version N-83033-g890320b Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --enable-filter=drawtext --enable-nonfree --enable-libfreetype
  libavutil      55. 43.100 / 55. 43.100
  libavcodec     57. 71.100 / 57. 71.100
  libavformat    57. 62.100 / 57. 62.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 68.100 /  6. 68.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
Input #0, rtsp, from 'rtsp://x.x.x.x:555':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.240000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1280x960, 25 fps, 24.92 tbr, 90k tbn, 50 tbc
Input #1, rtsp, from 'rtsp://x.x.x.x:556':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #1:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #1:1: Audio: mp2, 16000 Hz, mono, s16p, 128 kb/s
Input #2, png_pipe, from '/root/streams/vp.feszek/watermark_HD.png':
  Duration: N/A, bitrate: N/A
    Stream #2:0: Video: png, rgba(pc), 1280x720 [SAR 2834:2834 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
[swscaler @ 0x2901000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x2925b60] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x294d060] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 0x2b384c0] deprecated pixel format used, make sure you did set range correctly
Output #0, tee, to '[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/720.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/20170129120108/720p/playlist.m3u8':
  Metadata:
    title           : InfoCAM.hu Madarles B odu 720p
    encoder         : Lavf57.62.100
    Stream #0:0: Video: h264 (h264_nvenc) (High), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 1800 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.71.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/1800000 buffer size: 3600000 vbv_delay: -1
    Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc57.71.100 aac
Output #1, tee, to '[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/480.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/20170129120108/480p/playlist.m3u8':
  Metadata:
    title           : InfoCAM.hu Madarles B odu 480p
    encoder         : Lavf57.62.100
    Stream #1:0: Video: h264 (h264_nvenc) (Main), yuv420p, 848x480 [SAR 160:159 DAR 16:9], q=-1--1, 1000 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.71.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 2000000 vbv_delay: -1
    Stream #1:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc57.71.100 aac
Output #2, tee, to '[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/360.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/20170129120108/360p/playlist.m3u8':
  Metadata:
    title           : InfoCAM.hu Madarles B odu 360p
    encoder         : Lavf57.62.100
    Stream #2:0: Video: h264 (h264_nvenc) (Main), yuv420p, 640x360 [SAR 1:1 DAR 16:9], q=-1--1, 500 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.71.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/500000 buffer size: 1000000 vbv_delay: -1
    Stream #2:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc57.71.100 aac
Output #3, tee, to '[f=hls:hls_time=4:hls_list_size=5400:hls_flags=+delete_segments+append_list+omit_endlist]/var/www/html/nv/madarles-bodu/hls/240.m3u8|[f=hls:hls_playlist_type=2]/var/www/html/storage/nv/madarles-bodu/archivum/20170129120108/240p/playlist.m3u8':
  Metadata:
    title           : InfoCAM.hu Madarles B odu 240p
    encoder         : Lavf57.62.100
    Stream #3:0: Video: h264 (h264_nvenc) (Main), yuv420p, 424x240 [SAR 160:159 DAR 16:9], q=-1--1, 300 kb/s, 25 fps, 25 tbn, 25 tbc (default)
    Metadata:
      encoder         : Lavc57.71.100 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 0/0/300000 buffer size: 600000 vbv_delay: -1
    Stream #3:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
    Metadata:
      encoder         : Lavc57.71.100 aac
Output #4, image2, to '/var/www/html/nv/madarles-bodu/current.jpg':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf57.62.100
    Stream #4:0: Video: mjpeg, yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 1 fps, 1 tbn, 1 tbc (default)
    Metadata:
      encoder         : Lavc57.71.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 (h264) -> fps (graph 0)
  Stream #1:0 (h264) -> fps (graph 0)
  Stream #2:0 (png) -> overlay:overlay (graph 0)
  split:output0 (graph 0) -> Stream #0:0 (h264_nvenc)
  Stream #1:1 -> #0:1 (mp2 (native) -> aac (native))
  scale (graph 0) -> Stream #1:0 (h264_nvenc)
  Stream #1:1 -> #1:1 (mp2 (native) -> aac (native))
  scale (graph 0) -> Stream #2:0 (h264_nvenc)
  Stream #1:1 -> #2:1 (mp2 (native) -> aac (native))
  scale (graph 0) -> Stream #3:0 (h264_nvenc)
  Stream #1:1 -> #3:1 (mp2 (native) -> aac (native))
  fps (graph 0) -> Stream #4:0 (mjpeg)

If network error occurs, ffmpeg outputs this kind of errors:

[Parsed_overlay_4 @ 0x27d7860] [framesync @ 0x27d7988] Buffer queue overflow, dropping.
    Last message repeated 100 times
[h264 @ 0x2c85100] error while decoding MB 21 22, bytestream -7
[h264 @ 0x2c85100] concealing 1868 DC, 1868 AC, 1868 MV errors in P frame
[Parsed_overlay_4 @ 0x27d7860] [framesync @ 0x27d7988] Buffer queue overflow, dropping.
    Last message repeated 26 times
[h264 @ 0x2b4f460] concealing 3792 DC, 3792 AC, 3792 MV errors in P frame
[h264 @ 0x2b77b60] Increasing reorder buffer to 12
[h264 @ 0x2bdad60] error while decoding MB 42 49, bytestream -5
[h264 @ 0x2bdad60] concealing 887 DC, 887 AC, 887 MV errors in P frame
[h264 @ 0x2c21f00] error while decoding MB 58 18, bytestream -11
[h264 @ 0x2c21f00] concealing 2151 DC, 2151 AC, 2151 MV errors in I frame


In action: http://madarles.hu/madarles-b-odu

Can anybody help?

Thank you!

Change History (2)

comment:1 Changed 8 weeks ago by pero

Any suggestion, please?
Is it a bug or maybe I use wrong FFmpeg options?
I don't know which FFmpeg component may cause this issue.

One thing I experienced, that when I used libx264 to encode the stream, it was much more error tolerant. I mean that if a network error occured, FFmpeg did not exited, there were errors in the picture, but ffmpeg did not stop. Only serious network errors caused FFmpeg to exit.
Since I use nvenc, every little network error cause FFmpeg exit and in PIP mode it causes the described picture freeze. Is this error tolerance configurable? If yes, this could be a workaround.

Thank you

comment:2 Changed 6 weeks ago by pero

  • Resolution set to worksforme
  • Status changed from new to closed

I found out, that there is an option for overlay filter called "shortest".

In the docs:
"If set to 1, force the output to terminate when the shortest input terminates. Default value is 0."

I set it to 1 and it is working fine for 3 days now. When one of the stream fails, then overlay filter terminates the output and ffmpeg exits.

Note: See TracTickets for help on using tickets.