[Regression] Data stream in output may glitch "-stats" display since 7.0

[Denis Filippov]

Summary of the bug:
How to reproduce:
The stream from the TV-channel contains scte-35 tags and subtitles as a teletext. If you use the command for such a stream:

% ffmpeg -hide_banner -loglevel fatal -stats -i "srt://94.140.220.80:12108?streamid=571570#e05214d" -map 0:a -map 0:v -map "0:d?" -c:v h264 -c:a aac -f null -
ffmpeg version 7.1
built on Ubuntu 22.04

The result of statistic:

% frame=  393 fps= 34 q=13.0 size=N/A time=N/A bitrate=N/A speed=N/A

Stream data:

% Input #0, mpegts, from 'srt://94.140.220.80:12108?streamid=571570#e05214d':
  Duration: N/A, start: 30269.132444, bitrate: N/A
  Program 778
    Metadata:
      service_name    : HD_H264
      service_provider: 78
  Stream #0:0[0x2a6]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0xd2]: Audio: mp2 (mp3float) ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:2[0x23](rus,rus): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
  Stream #0:3[0x20]: Data: scte_35
Stream mapping:
  Stream #0:1 -> #0:0 (mp2 (native) -> aac (native))
  Stream #0:0 -> #0:1 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help

Without -map "0:d?" everything all right, result:

% frame=  361 fps= 34 q=20.0 size=N/A time=00:00:15.48 bitrate=N/A speed=1.46x

The problem is exclusively on versions starting from 7.0
Similarly, there are no problems with ffmpeg versions 5.x and 6.x.
I have tried all available Dockers as well as the local build on linux and macos - the result is similar: versions starting from 7 do not provide statistics with datastreams.

I can run the verification stream on demand. The stream is limited to 1 connection.

[Gyan]

Check with git master.

[Denis Filippov]

As expected, the result is the same for the master branch.
It seems to me that the problem is as follows: the scte-35 data stream does not have the data itself, but only the presence identifier. The data appears only when the mark itself is encoded, i.e. when it is present. In this case, the speed up, and then creeps down to 0.

% ffmpeg version git-2025-01-25-a0a89ef Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/opt/ffmpeg --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-libopencore-amrnb --enable-libopencore-amrwb --extra-libs=-ldl --enable-nonfree --enable-gpl --enable-libass --enable-openssl --enable-pthreads --enable-libfreetype --enable-libfontconfig --enable-libharfbuzz --enable-libfribidi --enable-libvidstab --enable-libwebp --enable-libxcb --enable-postproc --enable-small --enable-version3 --enable-libbluray --enable-libopenjpeg --enable-libkvazaar --enable-libpulse --enable-libzvbi --extra-libs=-lpthread --enable-libfdk-aac --enable-libmp3lame --enable-libtwolame --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-libfdk_aac --enable-libopus --enable-libvorbis --disable-libaom --enable-libsvtav1 --enable-libdav1d --enable-demuxer=dash --enable-libxml2 --enable-pic --enable-shared --enable-rpath --enable-libvvenc --enable-libzimg --enable-libsrt --enable-demuxer=dash --enable-librabbitmq --extra-cflags=-I/opt/ffmpeg/include --extra-cflags=-I/usr/include/x86_64-linux-gnu --extra-ldflags=-L/opt/ffmpeg/lib --extra-ldflags=-L/usr/lib/x86_64-linux-gnu --extra-ldflags=-L/usr/lib --extra-ldflags=-L/opt/ffmpeg/lib/x86_64-linux-gnu
  libavutil      59. 55.100 / 59. 55.100
  libavcodec     61. 31.101 / 61. 31.101
  libavformat    61.  9.106 / 61.  9.106
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  6.101 / 10.  6.101
  libswscale      8. 13.100 /  8. 13.100
  libswresample   5.  4.100 /  5.  4.100
  libpostproc    58.  4.100 / 58.  4.100
Input #0, mpegts, from 'srt://94.140.220.80:12108?streamid=571570#e05214d':
  Duration: N/A, start: 8491.654756, bitrate: N/A
  Program 778 
    Metadata:
      service_name    : SD_H264
      service_provider: 78
  Stream #0:0[0x2a6]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(tv, bt470bg/bt470bg/smpte170m, top first), 720x576 [SAR 16:11 DAR 20:11], 25 fps, 25 tbr, 90k tbn
  Stream #0:1[0xd2]: Audio: mp2 (mp3float) ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 128 kb/s
  Stream #0:2[0x23](rus,rus): Subtitle: dvb_teletext (libzvbi_teletextdec) ([6][0][0][0] / 0x0006), 492x250
  Stream #0:3[0x20]: Data: scte_35
Stream mapping:
  Stream #0:1 -> #0:0 (mp2 (native) -> aac (native))
  Stream #0:0 -> #0:1 (h264 (native) -> h264 (libx264))
  Stream #0:3 -> #0:2 (copy)
Press [q] to stop, [?] for help
[libx264 @ 0x5603d6590f00] using SAR=16/11
[libx264 @ 0x5603d6590f00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x5603d6590f00] profile High, level 3.0, 4:2:0, 8-bit
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf61.9.106
  Stream #0:0: Audio: aac, 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc61.31.101 aac
  Stream #0:1: Video: h264, yuv420p(tv, bt470bg/bt470bg/smpte170m, top coded first (swapped)), 720x576 [SAR 16:11 DAR 20:11], q=2-31, 25 fps, 25 tbn
    Metadata:
      encoder         : Lavc61.31.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:2: Data: scte_35
frame=   42 fps=0.0 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=   55 fps= 55 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=   68 fps= 45 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=   80 fps= 40 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=   93 fps= 37 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  105 fps= 35 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  118 fps= 34 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  130 fps= 32 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  144 fps= 32 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  155 fps= 31 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  168 fps= 31 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  180 fps= 30 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  193 fps= 30 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  205 fps= 29 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  218 fps= 29 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  230 fps= 29 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  243 fps= 29 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  255 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  269 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  280 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  296 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  304 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  317 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  330 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  343 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  358 fps= 28 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  368 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  380 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  393 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  405 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  418 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  430 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  443 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  455 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  473 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A    
frame=  486 fps= 27 q=28.0 size=N/A time=N/A bitrate=N/A speed=N/A

[Denis Filippov]

I added the recorded file from this stream. Here he is:
​https://drive.google.com/file/d/1Hucoq8uBb_sq6Qt464WKm7qxIKYtkWe_/view?usp=sharing
Command for checking:

% ffmpeg -re -i data_streams.ts -map 0 -sn -c:v h264 -c:a aac -f null -

If remove -map 0 the speed and time appears available
If use ffmpeg v5.x or v6.x the speed and time appears available

[MasterQuestionable]

͏    Direct link:
͏    ​https://drive.usercontent.google.com/download?confirm=t&export=download&id=1Hucoq8uBb_sq6Qt464WKm7qxIKYtkWe_
͏    (~ 35.86 MiB; "data_streams.ts")

͏    Test the 2?
͏    ffmpeg -i "data_streams.ts" -f null -
͏    ffmpeg -i "data_streams.ts" -map 0 -dn -f null -

[Denis Filippov]

I tested 2 commands:

The first:

% ffmpeg -i "data_streams.ts" -f null - 2> out1.txt

results in out1.txt

I change the second, by my request, because the data stream SHOULD be in result: instead -dn (is is the same as first command) I use -sn (disable teletext, and leave scte-35)

% ffmpeg -i "data_streams.ts" -map 0 -sn -f null - 2> out2.txt

results in out2.txt

[MasterQuestionable]

͏    Did the "-dn" version give that basically identical as "out1.txt"?

͏    Notice the Output info:
͏    ​https://trac.ffmpeg.org/attachment/ticket/11430/out1.txt#L30
͏    ​https://trac.ffmpeg.org/attachment/ticket/11430/out2.txt#L31
͏    .
͏    Could you confirm that old FFmpeg also include the data stream intended?

͏    ----

͏    Another interesting thing (though off-topic) is the generated texts are of mixed line-break styles:
͏    Mixed usage of "\r" ("CR", Mac), "\n" ("LF", Unix).
͏    [ ^ Bad for Plain Text interoperability; for general glitches. ]

͏    The texts are primarily of Unix-style "\n".
͏    But the "-stats" messages are unusually terminated with "\r". (another bug?)

͏    Also the "-stats" messages have notable trailing whitespace unnecessary.

[Denis Filippov]

Yes, "-dn" version is the same as out1.txt. Because without "-map 0" potions we used only audio and video streams. Option "-dn" disappeared data stream, and "-sn" disappeared teletext. And we have full the same as out1.txt.

[MasterQuestionable]

͏    What about the behavior in old FFmpeg?
͏    "Could you confirm that old FFmpeg also include the data stream intended?"

[Denis Filippov]

As I wrote in the first message, the problem is that we cannot switch from version 6 to version 7, because the speed output behavior has been changed. I am attaching the version conclusions in the files. I used the previous command:

% ffmpeg -i "data_streams.ts" -map 0 -sn -f null -

As a result, both versions have a speed value.

% frame= 1873 fps=1219 q=-0.0 size=N/A time=00:01:16.41 bitrate=N/A speed=49.7x

Only version 7.x have not speed/time stats

% frame= 1904 fps=1269 q=-0.0 size=N/A time=N/A bitrate=N/A speed=N/A

[Marth64]

I think this is related to: ​https://trac.ffmpeg.org/ticket/10792

[MasterQuestionable]

͏    Likely related with threaded architecture change.

͏    There seems to be several threading related options in:
͏    https://ffmpeg.org/ffmpeg.html
͏    ; would that workaround?

[Denis Filippov]

Yes, it looks like this is the same problem with streams other than audio and video. As I see this situation, PTS are taken from all elementary streams to synchronize between each other. But what kind of data should be used if the PTS are different? In my case, there is no PTS in the data stream at all. That is, there is no time increment. The PTS in the data stream will appear only when the label itself arrives. It will fully match the required PTS video and audio for synchronization. And speed and time will be appeared for little time. In ffmpeg version 7, there is a feeling that video or audio streams are no longer the main ones in time formation, but there is some kind of mathematics that multiplies time data by 0 if at least one stream does not have it at the moment. It's easy to fix. If at least 1 stream contains sound time data, then use it.
Unfortunately, our system is entirely built on speed and time data. Automation is tied to this data. We can't run the system without speed data. I have to work with the previous version, or think about writing a patch.

[Denis Filippov]

I could add the same case, but in other conditions. It more simplest for detect problem in code.
Just generate file, that has 1 hour video and 10 seconds audio. I was show PTS on video, that prove that PTS are present:

ffmpeg -f lavfi -i smptehdbars=size=1280x720:rate=25:duration=3600 -f lavfi \
-i sine=frequency=1000:sample_rate=48000:duration=10 \
-vf "drawtext=fontsize=40:text='PTS\:%{pts\:hms\:3}':x=(w-tw)/2:y=(h-lh)/2:fontcolor=white:box=1:boxcolor=0x00000000@1" \
-pix_fmt yuv420p -c:v libx264 -preset superfast -c:a aac -f mp4 -y dropped_audio.mp4

Next step, try to process this file by command:

ffmpeg -re -i dropped_audio.mp4 -c copy -f null -

As a result, the time stops at 10 seconds, and the speed starts to drop, because it cannot be calculated because the PTS is missing in one of the streams. But we can see that they are present in the video!
There is simply incorrect logic in the code when the data that is available is not taken, but rather overwritten.

[MasterQuestionable]

͏= Streams of unequal length may glitch "-stats" display since 7.0 =

͏    Seems like certain unintended change meanwhile...
͏    Can you try find the more accurate regression window?

[Denis Filippov]

The regression window is the absence of speed and time calculations in the absence of PTS in at least one stream. And that's the problem.

[MasterQuestionable]

͏    I mean try old FFmpeg versions trying to locate, the commit that caused the regression.
͏    (ascertaining the last working version so to help)