Audio is not in sync with video, (In: rtmp audio + rtmp video, Out: rtmp mix)

[alessandro marin]

Hi, we are having a strange behaviour while muxing two live rtmp strem audo/video using the following command:

avconv -thread_queue_size 512 -rtmp_live live -i "rtmp://streaming/live/Input1" -thread_queue_size 512 -rtmp_live live  -i "rtmp://streaming/live/Input2" -c:v copy -c:a copy -map 0:v -map 1:a  -flags +global_header -async -1 -rtmp_live live -f flv  "rtmp://streaming/live/Output"

produces a stream where the VIDEO is late about 5 sec.(audio comes before video in correct time) in this situation the audio and video are not sync.
Without -async -1 the audio is mixed in the correct way while the video is stretched respect to its source, only with -async -1 parameter I can have a correct behavior of the video but it is late respect to audio.

Also, I've tried with -vsync option (0,1,2,drop,-1) but they didn't have any effect.

[Carl Eugen Hoyos]

Please test current FFmpeg git head and provide the tested command line together with the complete, uncut console output to make this a valid ticket.

[alessandro marin]

Hi

This is the command used:
avconv -thread_queue_size 512 -rtmp_live live -i "rtmp://streaming/live/Input1" -thread_queue_size 512 -rtmp_live live -i "rtmp://streaming/live/Input2" -c:v copy -c:a copy -map 0:v -map 1:a -flags +global_header -async -1 -rtmp_live live -f flv "rtmp://streaming/live/Output"

And below the output of the tested command:

ffmpeg version 2.8.8-0ubuntu0.16.04.1 Copyright (c) 2000-2016 the FFmpeg developers

built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.2) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/i386-linux-gnu --incdir=/usr/include/i386-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv --disable-i686
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

Input #0, flv, from 'rtmp://streaming/live/Input1':

Duration: N/A, start: 0.000000, bitrate: N/A

Stream #0:0: Video: h264 (Baseline), yuv420p(tv), 320x240 [SAR 1:1 DAR 4:3], 10 fps, 10 tbr, 1k tbn, 20 tbc

Metadata:

duration 0.00
audiodatarate 54.69
audiosamplerate 22050.00
audiosamplesize 16.00
stereo FALSE
audiocodecid 10.00
encoder Lavf54.2.100
filesize 0.00

Input #1, flv, from 'rtmp://streaming/live/Input2':

Metadata:

encoder : Lavf54.2.100

Duration: 00:00:00.00, start: 0.000000, bitrate: N/A

Stream #1:0: Audio: aac (LC), 22050 Hz, mono, fltp, 56 kb/s

Output #0, flv, to 'rtmp://streaming/live/Output':

Metadata:

encoder : Lavf56.40.101
Stream #0:0: Video: h264 ([7][0][0][0] / 0x0007), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=2-31, 10 fps, 10 tbr, 1k tbn, 1k tbc
Stream #0:1: Audio: aac ([10][0][0][0] / 0x000A), 22050 Hz, mono, 56 kb/s

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (copy)

Press [q] to stop, ? for help
[flv @ 0x85e7680] Encoder did not produce proper pts, making some up.
frame= 4657 fps=9.3 q=-1.0 size= 14344kB time=00:08:28.51 bitrate= 231.1kbits/s

[Carl Eugen Hoyos]

Please reopen this ticket if the issue is reproducible with current FFmpeg git head.

[alessandro marin]

I updated ffmpeg with the last version 3.2-1.
I send the same command used for the version 2.8.8, but the behavior is different.
With the version 2.8.8. i can view via RTMP player the RTMP output and the video is late about 5 sec on the audio.
With the version 3.2-1 i can not view via RTMP player the RTMP output (is always in initialization state) but in the ffmpeg output log i can see new warnings not showed by old ffmpeg version, but no error.

What is the problem?

Output log:

ffmpeg version 3.2-1~16.04.york1 Copyright (c) 2000-2016 the FFmpeg developers

built with gcc 5.4.1 (Ubuntu 5.4.1-3ubuntu1~ubuntu16.04.1york0) 20161019
configuration: --prefix=/usr --extra-version='1~16.04.york1' --toolchain=hardened --libdir=/usr/lib/i386-linux-gnu --incdir=/usr/include/i386-linux-gnu --enable-gpl --disable-libtesseract --disable-stripping --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-opengl --enable-sdl2 --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-openal --enable-frei0r --enable-libopencv --enable-libx264 --enable-chromaprint --enable-shared
libavutil 55. 34.100 / 55. 34.100
libavcodec 57. 64.100 / 57. 64.100
libavformat 57. 56.100 / 57. 56.100
libavdevice 57. 1.100 / 57. 1.100
libavfilter 6. 65.100 / 6. 65.100
libavresample 3. 1. 0 / 3. 1. 0
libswscale 4. 2.100 / 4. 2.100
libswresample 2. 3.100 / 2. 3.100
libpostproc 54. 1.100 / 54. 1.100

[flv @ 0x819726c0] video stream discovered after head already parsed
Input #0, flv, from 'rtmp://streaming.com/live/in1':

Duration: N/A, start: 0.000000, bitrate: N/A

Stream #0:0: Video: h264 (Baseline), yuv420p(tv, progressive), 320x240 [SAR 1:1 DAR 4:3], 10 fps, 10 tbr, 1k tbn, 20 tbc

[flv @ 0x81980360] audio stream discovered after head already parsed
Input #1, flv, from 'rtmp://streaming.com/live/in2':

Metadata:

encoder : Lavf54.2.100

Duration: 00:00:00.00, start: 0.000000, bitrate: N/A

Stream #1:0: Audio: aac (LC), 22050 Hz, mono, fltp

Output #0, flv, to 'rtmp://streaming.com/live/output':

Metadata:

encoder : Lavf57.56.100
Stream #0:0: Video: h264 (Baseline) ([7][0][0][0] / 0x0007), yuv420p(tv, progressive), 320x240 [SAR 1:1 DAR 4:3], q=2-31, 10 fps, 10 tbr, 1k tbn, 1k tbc
Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 22050 Hz, mono

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (copy)

Press [q] to stop, ? for help
[flv @ 0x819a8d40] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[flv @ 0x819a8d40] Encoder did not produce proper pts, making some up.
frame= 3 fps=0.0 q=-1.0 size= 2590kB time=00:00:00.19 bitrate=109921.5kbits/s speed=0.000552x

[Carl Eugen Hoyos]

Replying to alessandro.marin:

I updated ffmpeg with the last version 3.2-1.

FFmpeg 3.2 is a few weeks old, please test current FFmpeg git head as explained on https://ffmpeg.org/bugreports.html if you want to report a (not security related) issue on this bug tracker.

If you believe this is a regression, please find out which change introduced the bug or provide a command line that allows me to reproduce the issue you see.