Opened 5 years ago

Closed 2 days ago

Last modified 2 days ago

#2873 closed enhancement (fixed)

ffmpeg concat demuxer fails: mpeg files with different audio and video stream order

Reported by: kadmandux Owned by:
Priority: wish Component: avformat
Version: git-master Keywords: concat demuxer
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
I've 2 MPEG files with different order in audio and video stream, which I want to join:

ffprobe MU2000134b.mpg
Input #0, mpeg, from 'MU2000134b.mpg':
  Duration: 00:00:40.14, start: 0.500000, bitrate: 7713 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p, 720x576 [SAR 64:45 DAR 16:9], max. 9100 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s

ffprobe MU2000135b.mpg
Input #0, mpeg, from 'MU2000135b.mpg':
  Duration: 00:02:09.82, start: 0.500000, bitrate: 7583 kb/s
    Stream #0:0[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
    Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p, 720x576 [SAR 64:45 DAR 16:9], max. 9100 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc

How to reproduce:

% ffmpeg -f concat -i lista.txt -c:a copy -c:v copy video.mpg
ffmpeg version N-55393-g3b2e99f Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug  8 2013 21:32:57 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-
amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --
enable-libxvid --enable-zlib
  libavutil      52. 41.100 / 52. 41.100
  libavcodec     55. 24.100 / 55. 24.100
  libavformat    55. 13.102 / 55. 13.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 82.100 /  3. 82.100
  libswscale      2.  4.100 /  2.  4.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
[concat @ 026eb160] Estimating duration from bitrate, this may be inaccurate
Input #0, concat, from 'lista.txt':
  Duration: 00:00:00.00, start: 0.000000, bitrate: 256 kb/s
    Stream #0:0: Video: mpeg2video (Main), yuv420p, 720x576 [SAR 64:45 DAR 16:9]
, max. 9100 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
Output #0, mpeg, to 'video.mpg':
  Metadata:
    encoder         : Lavf55.13.102
    Stream #0:0: Video: mpeg2video, yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-3
1, max. 9100 kb/s, 25 fps, 90k tbn, 25 tbc
    Stream #0:1: Audio: ac3, 48000 Hz, stereo, 256 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  368 fps=0.0 q=-1.0 size=   14306kB time=00:00:14.90 bitrate=7863.3kbits/
frame=  718 fps=717 q=-1.0 size=   27334kB time=00:00:28.90 bitrate=7747.0kbits/
frame=  915 fps=608 q=-1.0 size=   34646kB time=00:00:36.78 bitrate=7715.9kbits/
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=3763 size=41205
[mpeg @ 03a9ba40] packet too large, ignoring buffer limits to mux it
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=3763 size=41205
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=5788 size=41205
[mpeg @ 03a9ba40] packet too large, ignoring buffer limits to mux it
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=5788 size=41205
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=7825 size=41205
[mpeg @ 03a9ba40] packet too large, ignoring buffer limits to mux it
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=7825 size=41205
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=9862 size=41205
[mpeg @ 03a9ba40] packet too large, ignoring buffer limits to mux it
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=9862 size=41205
[mpeg @ 03a9ba40] buffer underflow i=1 bufi=11899 size=41205

The problem doesn't appears if, before the join operation, you reorder the streams (with "-map" option) to have the same order in both files.

Change History (9)

comment:1 Changed 5 years ago by Cigaes

  • Component changed from FFmpeg to avformat
  • Status changed from new to open
  • Type changed from defect to enhancement
  • Version changed from unspecified to git-master

This is currently the expected behaviour (“All files must have the same streams” says the doc; similar streams in different order are not “the same”).
Patches to implement per-file stream mapping are welcome.

comment:2 Changed 5 years ago by kadmandux

It seems that there is no stream order concept in mpeg-ps:

http://www.ffmpeg.org/trac/ffmpeg/ticket/2876

So, I think that there should be at least a little automatic guessing when you use the concat demuxer with mpeg files, so that you can join files with same content and simple stream schemes, like the files I use (only 2 streams: 1 video + 1 audio).

comment:3 Changed 5 years ago by kadmandux

What do you think about this approach?

1.- Add two simple functions to utils.c for to sort the streams based on id:

int av_compare_streams(const void *st1, const void *st2)
{
	return ((((AVStream *)st1)->id > ((AVStream *)st2)->id) ? 1 : 
			(((AVStream *)st1)->id < ((AVStream *)st2)->id) ? -1 : 0);
}

void av_sort_streams_by_id(AVFormatContext *s)
	qsort((void *)s->streams, s->nb_streams, sizeof(s->streams[0]), av_compare_streams);
	/*Adjust the indexes */
	for (int i=0;i<s->nb_streams;i++) {
		s->streams[i]->index = i;
	}
}


2.- Add the prototypes to "avformat.h"

void av_sort_streams_by_id(AVFormatContext *s);
int av_compare_streams(const void *st1, const void *st2);

3.- Modify ffprobe.c, for to use the new function, from inside "open_input_file" function:

	/* Reorder streams if the format requires it (added by kadmandux)
	 * Is the flag AVFMTCTX_NOHEADER a good candidate to use for this purpose?
	 * Or should I define a new specific flag in "avformat.h"  */
	if (fmt_ctx->ctx_flags & AVFMTCTX_NOHEADER)
		av_sort_streams_by_id(fmt_ctx);

This code should be executed after the call to "avformat_find_stream_info".

4.- Do the equivalent change in ffmpeg_opt.c, also inside "open_input_file" function.

Last edited 5 years ago by kadmandux (previous) (diff)

comment:4 Changed 5 years ago by Cigaes

Changing the order of the streams would probably not be as simple as changing the order in the array, but it can probably be done.

Something making the order of streams from MPEG-PS less random would probably be welcome (although I may be missing some drawbacks). Please post patches on the mailing list.

You can make it simpler than you suggested, though: just change avformat_find_stream_info(): if format is MPEG-PS and there was no streams already detected, then reorder at the end.

Unfortunately, for your use, this is still too fragile, IMHO. Unless you have a constraint you did not talk about, you would be better off just acknowledging that MPEG-PS is a very bad format and using another, especially for intermediate files.

comment:5 Changed 5 years ago by kadmandux

I did some testing and the stream_id doesn't seem a good candidate to sort the streams ... EDITING: There were two problems with my first tests. Now it works!

In the short time, I'm going to try your suggestion: use a different container format, like MKV, to use as input to the join step. Thanks!

Last edited 5 years ago by kadmandux (previous) (diff)

comment:6 Changed 5 years ago by kadmandux

It works now!

There were two problems:

1) The function av_compare_streams had an error. It was expecting pointers to streams and it gets passed pointers to pointers to streams (we are sorting an array with pointers, not an array with streams). So, this is the correct code:

int av_compare_streams(const void *st1, const void *st2)
{
	return (((*(AVStream **)st1)->id > (*(AVStream **)st2)->id) ? 1 : 
			((*(AVStream **)st1)->id < (*(AVStream **)st2)->id) ? -1 : 0);

2) The funcion avformat_find_stream_info was invoked from more places than I fixed, so I've made the changes at the end of function avformat_find_stream_info

And now it works great!

Last edited 5 years ago by kadmandux (previous) (diff)

comment:7 Changed 4 years ago by cehoyos

  • Priority changed from normal to wish
  • Resolution set to fixed
  • Status changed from open to closed

I believe this was fixed by Nicolas in 26dea773

comment:8 Changed 3 days ago by kadmandux

  • Resolution fixed deleted
  • Status changed from closed to reopened

Sorry, but I've just tested this now and the problem persists today.

Have a look at stream order shown for this two files:

D:\>ffprobe -i M2U00826.MPG
ffprobe version N-91536-geb94ec3257 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 7.3.1 (GCC) 20180722
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 22.100 / 58. 22.100
  libavformat    58. 17.101 / 58. 17.101
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 26.100 /  7. 26.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, mpeg, from 'M2U00826.MPG':
  Duration: 00:00:12.96, start: 0.226767, bitrate: 6674 kb/s
    Stream #0:0[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s
    Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc

D:\>ffprobe -i M2U00827.MPG
ffprobe version N-91536-geb94ec3257 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 7.3.1 (GCC) 20180722
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
  libavutil      56. 18.102 / 56. 18.102
  libavcodec     58. 22.100 / 58. 22.100
  libavformat    58. 17.101 / 58. 17.101
  libavdevice    58.  4.101 / 58.  4.101
  libavfilter     7. 26.100 /  7. 26.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, mpeg, from 'M2U00827.MPG':
  Duration: 00:02:42.24, start: 0.098722, bitrate: 6319 kb/s
    Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s

Both files are comming from the same VideoCam?, both are unedited and when I try to concat them with ffmpeg, I get the same errors reported some years ago.

What was the fix applied by Nicolas in 26dea773?

Thanks

Last edited 2 days ago by kadmandux (previous) (diff)

comment:9 Changed 2 days ago by kadmandux

  • Resolution set to fixed
  • Status changed from reopened to closed

Ok, seen and tested. I leave here some references for anothers interested:

https://gitlab.websupport.sk/peter.kovar/ffmpeg-mvc/commit/26dea7731e00cac3d9b47c68f26af36ce57b24d1?view=parallel

https://ffmpeg.org/ffmpeg-formats.html#Syntax

The muxing of my test files work ok with this "lista.txt":

file M2U00826.MPG
stream
exact_stream_id 0x1E0
stream
exact_stream_id 0x80

file M2U00826.MPG
stream
exact_stream_id 0x1E0
stream
exact_stream_id 0x80

Thanks a lot!

Last edited 2 days ago by kadmandux (previous) (diff)
Note: See TracTickets for help on using tickets.