"select" filter behavior regression (4.4.2 ~)

[John]

Summary of the bug:
On ffmpeg 4.4.2-0ubuntu0.22.04.1 the following command executes correctly and generates a thumbnail atlas matching the given video
On ffmpeg 7.0 the exact same command misses the first 40 seconds of the video entirely

ffmpeg -y -i ​http://xxx.xxx.xxx.xxx/mega/95026a66a4bc129643911e37d4c9e485.mp4 -vf select='eq(n\,0)+not(mod(t\,10))',scale=160:-1,tile=5x88 -frames:v 1 -q:v 2 ./thumbnails_test/95026a66a4bc129643911e37d4c9e485_atlas.jpg

How to reproduce:
Run the command on different versions of ffmpeg

% ffmpeg -y -i http:/xxx.xxx.xxx.xxx/mega/95026a66a4bc129643911e37d4c9e485.mp4 -vf select='eq(n\,0)+not(mod(t\,10))',scale=160:-1,tile=5x88 -frames:v 1 -q:v 2 ./thumbnails_test/95026a66a4bc129643911e37d4c9e485_atlas.jpg
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)

ffmpeg version 7.0 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)

[John]

This is from the older ffmpeg version and it's correct

[John]

This is from the newer version and it's wrong

[MasterQuestionable]

͏    Effective:
͏    "select='eq( n, 0 ) + not( mod( t, 10 ) )'"

͏    See also:
͏    https://ffmpeg.org/ffmpeg-filters.html#select

͏    Documentation too vague: unable to tell exact meaning.
͏    But likely the example is misuse: "expr" seems to expect something like boolean, mostly.
͏    (but with strange implementation to handle sort of eerie stream mapping...)

[MasterQuestionable]

͏    Try input with ͏"testsrc2":
͏    ffmpeg -y -v debug -hide_banner -nostdin -nostats -f lavfi -i "testsrc2=r=1:d=30" -vf "select='eq( n, 0 ) + not( mod( t, 10 ) )'" -compression_level 1 -pred none -pix_fmt rgb24 "%d.png"

͏    The "eq( n, 0 ) + " part appears unnecessary.
͏    Removing only caused the difference for Frame #1:
-[Parsed_select_0] n:0.000000 pts:0.000000 t:0.000000 key:1 interlace_type:P pict_type:I scene:nan -> select:1.000000 select_out:0
+[Parsed_select_0] n:0.000000 pts:0.000000 t:0.000000 key:1 interlace_type:P pict_type:I scene:nan -> select:2.000000 select_out:0
͏    .
͏    And such difference doesn't seem to matter.

[MasterQuestionable]

͏    I guess it's the PTS somehow disappeared.
͏    As observed in my debugging of various medias:
͏    "best_effort_timestamp" variant should be used as "pts".
͏    "pts" maybe absent for certain medias. (or "dts", vice versa)

[John]

output of testsrc=d=100:s=hd1080:r=25 seems correct

[John]

Apologies, but I do not understand how to proceed from here.
I know it's an issue as well with the media file but as it worked on the old ffmpeg version, what is the workaround to make it work on 7.0 as well?

[Marton Balint]

I'd start with checking the frame pts values in the old and the new version. You can use the showinfo filter to do that.

[John]

older ffmpeg version result of : ffmpeg -i 95026a66a4bc129643911e37d4c9e485.mp4 -vf showinfo -f null - 2> frame_info.txt

[John]

newer ffmpeg version result of : ffmpeg -i 95026a66a4bc129643911e37d4c9e485.mp4 -vf showinfo -f null - 2> frame_info.txt

[MasterQuestionable]

͏    Try the description here:
͏    ​https://trac.ffmpeg.org/attachment/ticket/11052/sample.mp4.xml

͏    Name the output:
͏    "4.4.2.xml"
͏    "7.0.xml"

͏    Supposedly the first 60 s of each shall be of interest.

͏    ----

͏    Using "-v debug" may also cause the command to output debug information of "select".
͏    (displayed in comment:2)

[MasterQuestionable]

͏    "What is the workaround to make it work on 7.0 as well?"
͏    Afraid... no. Besides fixing the filter.
͏    ("select" seems to use undesirable interpretation of "pts", "t" alike)

[John]

I have attached the two xml files from running the following command:

ffprobe -v warning -hide_banner -threads 0 -show_entries frame -select_streams v:0 -of xml -read_intervals 0%+60 "95026a66a4bc129643911e37d4c9e485.mp4" > "output.xml"

[MasterQuestionable]

͏    The diff indicated nothing interesting.
͏    Which means the decoding shall be much similar between the versions.

͏    Try "-v debug" on your original command and see what "select" reports.
͏    Also try removing the "eq( n, 0 ) + " part.