decoder sometimes reporting pts values in incorrect order

[csab6597]

Decoding video data through C api I found that in some situations the frames coming out of the avcodec_receive_frame() function have their pts values not in the correct order, whilst the picture data itself seems to be in correct - that is display - order.
Check the video file provided. Have a look at the pts values by

ffprobe 02.mp4 -show_frames -select_streams 0

excerpt

...
[FRAME]
pts=478478
pkt_dts=480480
pict_type=P
coded_picture_number=478
[/FRAME]
[FRAME]
pts=479479
pkt_dts=481481
pict_type=P
coded_picture_number=479
[/FRAME]
[FRAME]
pts=481481
pkt_dts=482482
pict_type=B
coded_picture_number=481
[/FRAME]
[FRAME]
pts=480480
pkt_dts=483483
pict_type=P
coded_picture_number=480
[/FRAME]
[FRAME]
pts=482482
pkt_dts=484484
pict_type=P
coded_picture_number=482
[/FRAME]
...

You find that the pts values are in proper order up to codec_picture_number=479

Probably because suddenly a B frame is coming up, the next frame in the list is reported to be 481, but when looking at the raw image data in the AVFrame of that frame, it is indeed frame number 480 to be displayed. So the remedy for me right now is to swap the pts values between 480 and 481.

In the file provided this mixup will repeat again several times.

[csab6597]

uploaded sample media through ​http://streams.videolan.org/upload/

[Balling]

Sample is here: ​https://streams.videolan.org/ffmpeg/incoming/02.mp4

it is now hidden further in commit that removed (already deprecated before) coded_picture_number: 651f3aa7f911d897270ee687128d464bddb7883a

[FRAME]
media_type=video
stream_index=0
key_frame=0
pts=481481
pts_time=20.061708
pkt_dts=482482
pkt_dts_time=20.103417
best_effort_timestamp=481481
best_effort_timestamp_time=20.061708
duration=1001
duration_time=0.041708
pkt_pos=24060425
pkt_size=30016
width=1920
height=1080
crop_top=0
crop_bottom=0
crop_left=0
crop_right=0
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=B
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=unknown
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=left
[/FRAME]
[FRAME]
media_type=video
stream_index=0
key_frame=0
pts=480480
pts_time=20.020000
pkt_dts=483483
pkt_dts_time=20.145125
best_effort_timestamp=483483
best_effort_timestamp_time=20.145125
duration=1001
duration_time=0.041708
pkt_pos=23980133
pkt_size=80292
width=1920
height=1080
crop_top=0
crop_bottom=0
crop_left=0
crop_right=0
pix_fmt=yuv420p
sample_aspect_ratio=1:1
pict_type=P
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=unknown
color_space=unknown
color_primaries=unknown
color_transfer=unknown
chroma_location=left
[/FRAME]

Also see #11136

[Balling]

actually:

ffmpeg -i https://streams.videolan.org/ffmpeg/incoming/02.mp4 -vf vfrdet -f null
 NUL

prints

[null @ 000001bf2b551380] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 654 >= 653
[Parsed_vfrdet_0 @ 000001bf2d47da40] VFR:0.006116 (4/650) min: -1001 max: 2002 avg: 750
[null @ 000001bf2b551380] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 654 >= 654

I can prove this is incorrect file cause you can demux the file into Annex B (ffmpeg -i ​https://streams.videolan.org/ffmpeg/incoming/02.mp4 -c copy adfaq.h264) and then use mp4box from GPAC:
mp4box.exe -add adfaq.h264 cfr.mp4

Still ffmpeg has a bug when it remuxes this annex B. So... Still a bug