Regression in pnm parsing at commit 9fc1031ac2e8691e0140854d727b58cb62431b2b

[symphorien]

Summary of the bug:
when using the image2pipe input mode to transform a stream of pnm images to mp4, ffmpeg 4.2 and master complains about invalid input format. The same file works fine with ffmpeg 3.
I bisected the regression to commit:

commit 9fc1031ac2e8691e0140854d727b58cb62431b2b (HEAD, tag: bug, refs/bisect/bad)
Author: Michael Niedermayer <michael@niedermayer.cc>
Date:   Wed Apr 24 21:14:19 2019 +0200

    avcodec/pnm_parser: Remember the size of the image and do not reparse the header

which seems related.
How to reproduce:

$  target/bin/ffmpeg -f image2pipe -vcodec ppm -i pipe: -an -r 25 -y -vcodec mpeg4 -s 1920x1080 -aspect 16/9 /tmp/fodsfd.mp4 < small.data
ffmpeg version N-93685-g9fc1031ac2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.3.0 (GCC)
  configuration: --prefix=/home/symphorien/src/ffmpeg/target --enable-libx264 --enable-gpl
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 52.100 / 58. 52.100
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 50.100 /  7. 50.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[image2pipe @ 0x3c72840] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, image2pipe, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 1920x1080, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm (native) -> mpeg4 (native))
Output #0, mp4, to '/tmp/fodsfd.mp4':
  Metadata:
    encoder         : Lavf58.27.103
    Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.52.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Error while decoding stream #0:0: Invalid data found when processing inputkbits/s speed=0.276x    
    Last message repeated 1 times
frame=  240 fps=6.9 q=31.0 Lsize=    1694kB time=00:00:09.56 bitrate=1451.6kbits/s speed=0.274x    
video:1692kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.111327%

Unfortunately, the smallest reproducer file I could create is 1.4 GB. (~140 MB compressed). I tried to upload it to upload.ffmpeg.org as explained on https://ffmpeg.org/bugreports.html but I could not connect on the server.

To check the integrity of the input file, I split it into the individual frames with the unix split command, and examined them briefly with an image viewer and they seem valid.

[Carl Eugen Hoyos]

$ ffmpeg -f lavfi -i testsrc2=s=hd1080 -f rawvideo -vframes 241 -vcodec ppm out

$ cat out | ffmpeg -f image2pipe -vcodec ppm -i - -f null -
ffmpeg version N-96058-g8b5ef2dcff Copyright (c) 2000-2019 the FFmpeg developers
  built with clang version 8.0.1 (tags/RELEASE_801/final 366581)
  configuration: --cc=clang --enable-gpl --enable-gnutls --enable-libxml2 --enable-libx264 --enable-libx265 --enable-libvpx --enable-libaom
  libavutil      56. 36.101 / 56. 36.101
  libavcodec     58. 65.100 / 58. 65.100
  libavformat    58. 35.101 / 58. 35.101
  libavdevice    58.  9.101 / 58.  9.101
  libavfilter     7. 68.100 /  7. 68.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[image2pipe @ 0x3460f40] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, image2pipe, from 'pipe:':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 1920x1080, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (ppm (native) -> wrapped_avframe (native))
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.35.101
    Stream #0:0: Video: wrapped_avframe, rgb24, 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.65.100 wrapped_avframe
Error while decoding stream #0:0: Invalid data found when processing input    
    Last message repeated 1 times
frame=  240 fps= 62 q=-0.0 Lsize=N/A time=00:00:09.60 bitrate=N/A speed=2.49x    
video:126kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

[Carl Eugen Hoyos]

Should be fixed by Michael in c94cb8d9b21baeeecef962c72965dbedc4e0b0e1, thank you for the report!

[symphorien]

I confirm that I cannot reproduce the bug any more, thanks!