Speedup init times of DASH streams

[lukasf]

Summary of the bug:

Initialization of the AVFormatContext is painfully slow when opening a DASH stream. On my 100MBit/s connection, opening DASH streams usually takes ~7-8 seconds, and that is before I even start reading packets. On slower connections the times are even higher, often more than 20 seconds. When opening the same stream with VLC, it only takes half the time (3-4 seconds on my connection, ~10 seconds on slow connections). And when opening in a browser player, it takes below one second even on slow connections. When using FFmpeg as a decoder library in a video player scenario, these slow connection times are unacceptable.

This is a good free sample stream to work with:
​http://www.bok.net/dash/tears_of_steel/cleartext/stream.mpd

There are three reasons for slow init times:

1. The DASH MPD header contains pretty detailed information about each stream. But that information is not used. Instead, every stream is opened and probed.

2. Each DASH stream usually has one init section, followed by the actual data sections. The init section is supposed to be enough to prepare the decoder. But currently, a probe buffer of a fixed size is filled and probing is done on that buffer. That buffer includes the small init section, plus some part of the actual data section. That means, two tcp connections are created for every stream, sequentially. Theoretically, it would be enough to do the probing on only the init section, saving the time to open and read parts of the first data section.

3. All streams are opened and probed sequentially. For DASH, we often have about 7-10 streams, sometimes even more. For every stream, usually two tcp connections are opened and read. This gets very very slow.

I am not sure if point 1 could be solved, because of how libavformat works internally. But I have created experimental patches for point 2 and 3. Solving point 2 will roughly cut init times in half. It requires changes to both "dashdec.c" and "mov.c". Point 3 will even more drastically reduce connection times, usually by factor "numberOfStreams". Solving point 2 and 3 together would lead to init times very well below 1 second on most connections.

I will submit patches to the mailing list when I am ready. But I already wanted to create awareness of the problem here and maybe start a discussion. Any feedback is appreciated.

Approach for point 2:

Use av_probe_input_format2() directly on the init buffer for probing. Only if that fails, use the normal probing with fixed/increasing buffer size.

Additionally, this needs a change in mov.c: In mov_read_header(), the mov decoder searches for both moov and mdat atoms. But for the initialization, only moov is really required. The mdat is the following atom that holds the actual data packets. I have changed this so that duing mov_read_header, only moov is searched and evaluated. Then later in mov_read_packet, the mdat is searched.

These two changes together will prevent opening of the DASH stream data segments.

Approach for point 3:

Create threads for each stream to do the initialization in parallel. It is not a big issue to open multiple threads here, since they will only wait for tcp connection to open and data to arrive. Still, I would make this a configurable parameter. Not sure if this should be on or off by default.

How to reproduce:

ffprobe started on 2020-01-29 at 20:37:38
Report written to "ffprobe-20200129-203738.log"
Command line:
"C:\\Program Files\\ffmpeg\\ffprobe.exe" http://www.bok.net/dash/tears_of_steel/cleartext/stream.mpd -report
ffprobe version 4.2.1 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 9.1.1 (GCC) 20190807
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[NULL @ 000001243588d340] Opening 'http://www.bok.net/dash/tears_of_steel/cleartext/stream.mpd' for reading
[http @ 000001243588dcc0] Setting default whitelist 'http,https,tls,rtp,tcp,udp,crypto,httpproxy'
[tcp @ 0000012435890380] Original list of addresses:
[tcp @ 0000012435890380] Address 208.97.177.147 port 80
[tcp @ 0000012435890380] Interleaved list of addresses:
[tcp @ 0000012435890380] Address 208.97.177.147 port 80
[tcp @ 0000012435890380] Starting connection attempt to 208.97.177.147 port 80
[tcp @ 0000012435890380] Successfully connected to 208.97.177.147 port 80
[http @ 000001243588dcc0] request: GET /dash/tears_of_steel/cleartext/stream.mpd HTTP/1.1

...