DASH has duplicate AAC timestamp

[philipn]

Summary of the bug:
Creating a DASH output from AAC with priming results in a duplicate timestamp in the second segment. The earliest presentation time of the second segment does not equal 0 + the duration of the first segment; it is off by 1 frame.

The issue results from the code block at ​https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/movenc.c#L5022 that sets the presentation time to 0. The workaround is to store that initial offset and add it back to the following presentation times at ​https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/movenc.c#L5016

How to reproduce:
Create a DASH output containing AAC, dump the packets and show there is a duplicate pts at the first sample in the second segment (pts = 238592).

% ffmpeg -f lavfi -i "sine=sample_rate=48000:frequency=1000:duration=10" -codec:a aac -f dash test.mpd
Input #0, lavfi, from 'sine=sample_rate=48000:frequency=1000:duration=10':
  Duration: N/A, start: 0.000000, bitrate: 768 kb/s
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, mono, s16, 768 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
[dash @ 0x7fa93a7e7900] Opening 'init-stream0.m4s' for writing
Output #0, dash, to 'test.mpd':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Audio: aac, 48000 Hz, mono, fltp, 69 kb/s
    Metadata:
      encoder         : Lavc59.18.100 aac
[dash @ 0x7fa93a7e7900] Opening 'chunk-stream0-00001.m4s.tmp' for writing
[dash @ 0x7fa93a7e7900] Opening 'test.mpd.tmp' for writing
[dash @ 0x7fa93a7e7900] Opening 'chunk-stream0-00002.m4s.tmp' for writing
[dash @ 0x7fa93a7e7900] Opening 'test.mpd.tmp' for writing
size=N/A time=00:00:10.00 bitrate=N/A speed=62.2x    
video:0kB audio:85kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

% ffprobe -show_packets test.mpd | grep pts=238592
Input #0, dash, from 'test.mpd':
  Duration: 00:00:10.00, start: -0.021333, bitrate: 1 kb/s
  Program 0 
    Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s
    Metadata:
      variant_bitrate : 69000
      id              : 0
pts=238592
pts=238592

% ffmpeg -version
ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 10.2.1 (Alpine 10.2.1_pre1) 20201203
configuration: --disable-debug --disable-doc --disable-ffplay --enable-fontconfig --enable-gpl --enable-libaom --enable-libaribb24 --enable-libass --enable-libbluray --enable-libfdk_aac --enable-libfreetype --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libsrt --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-libzmq --enable-nonfree --enable-openssl --enable-postproc --enable-shared --enable-small --enable-version3 --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib --extra-libs=-ldl --extra-libs=-lpthread --prefix=/opt/ffmpeg
libavutil      57. 17.100 / 57. 17.100
libavcodec     59. 18.100 / 59. 18.100
libavformat    59. 16.100 / 59. 16.100
libavdevice    59.  4.100 / 59.  4.100
libavfilter     8. 24.100 /  8. 24.100
libswscale      6.  4.100 /  6.  4.100
libswresample   4.  3.100 /  4.  3.100
libpostproc    56.  3.100 / 56.  3.100

[Balling]

Does is matter? Does it mean one of those valid frames will be dropped?

[philipn]

I think my analysis and potential fix was wrong. The timing in the DASH is correct but the packet timestamps read are wrong.

The first segment file contains 235 frames. The init segment edts determines that the segment starts at the second frame (time 1024), i.e. the segment presentation duration is 234 frames. The earliest presentation time property of the second segment should therefore be 234 * 1024 = 239616 and that is what it is set to.

The packet timing issue when reading however can be seen when re-encoding using "ffmpeg -i test.mpd test.mp4". This results in a "[aac @ 0x55bee3dd4280] Queue input is backward in time" warning.

Another example is opus, "ffmpeg -i test.mpd -codec:a libopus test.ogg" which results in
"[libopus @ 0x560e352de280] Queue input is backward in time
[ogg @ 0x560e352cc0c0] Non-monotonous DTS in output stream 0:0; previous: 240648, current: 240584; changing to 240648. This may result in incorrect timestamps in the output file."

The incorrect timing matters if the thing that is processing the media doesn't correct or ignore the issue in the right way.

[philipn]

I had a look at the latest code it appears that this issue has been fixed: c42d513ede6bc58b2cc7b0e47f90be27700843f8
The timing is taken from the base media decode time property in the tfdt rather than the earliest presentation time in the sidx.

I tried ffprobe using a later version (version 5.1.1) and confirmed it no longer has duplicate timestamps. I'd assumed before that the issue was with the DASH and so hadn't made sure to use the latest version of ffprobe (>= 5.0.0) like I'd done to create the DASH.

Sorry for the noise.