PNG decoder fails processing more than a certain number of frames

[panthalassa]

Summary of the bug: trying to encode a video from .png sequence with more than a certain number of frames in a given directory results in corrupted output and an out-of-memory scenario

How to reproduce:
Sorry but due to the absurd bandwith requirements I likely won't be able to upload the offending files. This can likely be reproduced by converting a long, losslessly compressed video into a sufficient number of PNG's (my stats: 11 minute 1080p video at 60fps = ~39,000 frames, each frame 1.9mb making the uncompressed video around 80gb) I acquired these frames from rendering in Blender

Present on all versions I've tested from 3.4.8 to 4.4.1. Any png-based input fails with too many files in the directory - symptoms include the encoded video having scrambled frame order after about 1000 frames, as well as the [png @ 0x??? chunk too big] error after 11000 to 14750 processed frames depending on settings.

Sending a kill signal for a soft quit producing a video of just over 1000 frames displays the same issue.

Taking a generous swathe of images around the error points and processing them separately produces no errors implying that a corrupted PNG is not to blame.

Putting a smaller (1-2000) number of frames in a directory and processing them separately produces a successful result longer than 1000 frames, past the previous point of onset for corruption. Unknown exactly how many frames are needed to cause the error.

Any format encoding that receiving raw PNG input appears to be bugged - libx264rgb, libx265, ffv1 tested

Latest tested version: clean install of N-104671-g2cddb2f7a8 on a live USB of Linux Mint 19.1 MATE following the instructions on the Compile FFMPEG page

% ffmpeg -framerate 60 -f image2 -pattern_type glob -i $1'/*.png' -vcodec libx265 -pix_fmt yuv420p10le -vf scale=out_range=pc $2

% ffmpeg -framerate 60 -f image2 -pattern_type glob -i $1'/*.png' -vcodec ffv1 -coder 1 -level 3 -threads 4 -slices 4 -pix_fmt bgra $2

ffmpeg version: N-104671-g2cddb2f7a8
built on ... Linux Mint 19.1 MATE live USB

[panthalassa]

I have uploaded a corrupted example file to the VideoLAN uploader what do I need do to move it here

Also there could be a confounding factor - I am performing this operation on a FUSE based NTFS filesystem, will attempt to reproduce on ext4

[panthalassa]

Nope, issue persists on an ext4 filesystem

[mkver]

I presume you uploaded ​this (at least ​the description says it belongs to this ticket). You could have used a more descriptive title than "test.mkv".

[panthalassa]

Replying to mkver:

that's the one sorry yes, clicked through before I checked everything properly. what can't be gleaned from the video is that around the 1000th frame, the sudden frames flashing are from very far into the rest of the series. Like it seems to go from 1000, to 10,000, to 1001, to 10,001, to 1002, to 10,002, and so forth

[panthalassa]

Ok after some more testing I've narrowed it down to the -pattern_type glob option. With this option enabled and using *.png for input causes the aforementioned symptoms. Removing this option and using %04d.png as input works successfully with no glitches

[panthalassa]

The odd part is that I've been using the unmodified script for years successfully. I have a variety of videos on Youtube made with glob input. However I did take a solid year off making videos, when I came back and updated stuff is when I noticed this issue. There's a chance this issue came about within the timeframe between now and December 2019 which is the date of the last video I posted, however I don't know what version I was running then so it's likely I had a build somewhat older than that

[Eric Reed]

Reproduced issue on FFmpeg 6.1.1-3ubuntu5.

ffmpeg -i 3046.png fails to decode PNG correctly.

File is too big to attach (~10MB), link to file on Google Drive: ​https://drive.google.com/file/d/1Se35Mrn6rwI2bl887geuY-HtWEFXCQ7R/view?usp=sharing

[Eric Reed]

Reproduced on master:

❯ ./ffmpeg -i /mnt/e/waifu2x/frames_out/3046.png
ffmpeg version N-117838-g4d3e96c90c Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-23ubuntu4)
  configuration: --prefix=/mnt/e/ffmpeg/build --ld=g++ --bindir=/mnt/e/ffmpeg/bin
  libavutil      59. 47.100 / 59. 47.100
  libavcodec     61. 25.100 / 61. 25.100
  libavformat    61.  9.100 / 61.  9.100
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  6.101 / 10.  6.101
  libswscale      8.  9.101 /  8.  9.101
  libswresample   5.  4.100 /  5.  4.100
[png @ 0x555668ffab40] chunk too big
[png_pipe @ 0x555668ff97c0] Could not find codec parameters for stream 0 (Video: png, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, png_pipe, from '/mnt/e/waifu2x/frames_out/3046.png':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: png, none, 25 fps, 25 tbr, 25 tbn
At least one output file must be specified

[Eric Reed]

Whoops, it seems my PNG file actually is corrupt. Curiously, neither MS Photos or GIMP reject it, although ImageMagick correctly does as well.

I can't explain the behavior from OP's reporting, but maybe the original command picked up a file that the subsequent one didn't, and the file happened to be corrupt?

[Eric Reed]

In the initial process of finding the corrupted PNG from a large batch, I was successful with a manual binary search. It seems like OP has failed at this step. Blender is mentioned, maybe the render had crashed partway through and was resumed, without OP realizing that the very last file to be rendered had not been fully written out.

The confounding variable here was probably that the timing of the error occurring cannot be reliably used to determine which PNG is corrupt (i.e. by watching the output frame tally). I don't fully understand why, but it may be due to buffering done by the video encoder or similar.