Remuxing mp4/dash file takes time that is exponential to the duration

[fireattack]

Summary of the bug:

The input file is created by concatenating dash/mpd fragments downloaded from YouTube livestream.

A sample file is provided at ​https://1drv.ms/u/s!Akq11jtCTJYwg7YXUeLvKe3OGoODEQ [414M, audio only]

Keep in mind due to the nature of this issue, a long duration (this one is about 5h42m) file is needed to demonstrate it.

How to reproduce:

% ffmpeg -i audio.ts -c copy output.m4a
ffmpeg version 2022-08-29-git-f99d15cca0-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      57. 34.100 / 57. 34.100
  libavcodec     59. 42.103 / 59. 42.103
  libavformat    59. 30.100 / 59. 30.100
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 46.103 /  8. 46.103
  libswscale      6.  8.104 /  6.  8.104
  libswresample   4.  9.100 /  4.  9.100
  libpostproc    56.  7.100 / 56.  7.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000005d7600] Found duplicated MOOV Atom. Skipped it
    Last message repeated 10263 times
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\test\1\audio.ts':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6mp41
  Duration: 05:42:09.99, start: 0.000000, bitrate: 169 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
Output #0, ipod, to 'D:\copy-only.m4a':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6mp41
    encoder         : Lavf59.30.100
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=  324238kB time=05:42:09.96 bitrate= 129.4kbits/s speed=21.4x
video:0kB audio:320781kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.077679%

The above command took aboud 993s (16.3min) to finish, where about 34s was used on reading the input.
So about 959s on remuxing the whole 5h42m.

If adding -t 2:00:00, it only took 117s in total, or 83s for remuxing.
If adding -t 1:00:00, it only took 49s in total, or 15s for remuxing.

This looks like a bug to me, as I assume the time consumed should be roughly linear to the duration of the video.

[Carl Eugen Hoyos]

Can you reproduce with a 5h audio file that you created with FFmpeg?
The console output indicates that the input file is broken.

[fireattack]

Replying to Carl Eugen Hoyos:

Can you reproduce with a 5h audio file that you created with FFmpeg?
The console output indicates that the input file is broken.

No I can't, this is specific to this kind of video.

I agree the input file isn't the most standard due to duplicate MOOV atom, but FFMPEG can still read it (albeit takes longer) and generate proper output. It just that for some reason it takes exponentially long time for FFMPEG to process it.

[mkver]

Issue also happens with -c copy -f null -, i.e. it is a demuxer issue.

[mkver]

It is the second loop in get_frag_stream_info_from_pkt(). This should be fixable by using a binary search (frag_index->item[i].moof_offset is ascending with i).