demux_decode example fails to decode audio that ffmpeg can decode

[Andrew Wason]

Summary of the bug:
I used ffmpeg with libflite to generate a FLAC in NUT audio file. The demux_decode example gets an error processing this file.

How to reproduce:
Generate the FLAC/NUT file using flite:

> ./ffmpeg_g -f lavfi -t 1 -i flite=text=hello:voice=rms -f nut -codec:a flac -sample_fmt s16 -ac 2 -ar 44100 -y /tmp/audio.nut
ffmpeg version N-113564-g3372876888 Copyright (c) 2000-2024 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --enable-libflite --prefix=/Users/aw/Projects/rectalogic/mediafx/experiments/vendor/ffmpeg6/installed --enable-gpl --extra-cflags=-I/opt/local/include --extra-ldflags=-L/opt/local/lib
  libavutil      58. 36.101 / 58. 36.101
  libavcodec     60. 39.100 / 60. 39.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 17.100 /  9. 17.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
Input #0, lavfi, from 'flite=text=hello:voice=rms':
  Duration: N/A, start: 0.000000, bitrate: 256 kb/s
  Stream #0:0: Audio: pcm_s16le, 16000 Hz, mono, s16, 256 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (pcm_s16le (native) -> flac (native))
Press [q] to stop, [?] for help
Output #0, nut, to '/tmp/audio.nut':
  Metadata:
    encoder         : Lavf60.20.100
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16, 128 kb/s
      Metadata:
        encoder         : Lavc60.39.100 flac
[out#0/nut @ 0x600002a58240] video:0KiB audio:16KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 2.148750%
size=      16KiB time=00:00:00.81 bitrate= 163.9kbits/s speed= 365x

ffmpeg can successfully demux/transcode this file:

> ./ffmpeg_g -i /tmp/audio.nut -c:a flac -f nut -y /tmp/audio2.nut
ffmpeg version N-113564-g3372876888 Copyright (c) 2000-2024 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --enable-libflite --prefix=/Users/aw/Projects/rectalogic/mediafx/experiments/vendor/ffmpeg6/installed --enable-gpl --extra-cflags=-I/opt/local/include --extra-ldflags=-L/opt/local/lib
  libavutil      58. 36.101 / 58. 36.101
  libavcodec     60. 39.100 / 60. 39.100
  libavformat    60. 20.100 / 60. 20.100
  libavdevice    60.  4.100 / 60.  4.100
  libavfilter     9. 17.100 /  9. 17.100
  libswscale      7.  6.100 /  7.  6.100
  libswresample   4. 13.100 /  4. 13.100
  libpostproc    57.  4.100 / 57.  4.100
Input #0, nut, from '/tmp/audio.nut':
  Metadata:
    encoder         : Lavf60.20.100
  Duration: 00:00:00.81, start: 0.000000, bitrate: 163 kb/s
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16
      Metadata:
        encoder         : Lavc60.39.100 flac
Stream mapping:
  Stream #0:0 -> #0:0 (flac (native) -> flac (native))
Press [q] to stop, [?] for help
Output #0, nut, to '/tmp/audio2.nut':
  Metadata:
    encoder         : Lavf60.20.100
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16, 128 kb/s
      Metadata:
        encoder         : Lavc60.39.100 flac
[out#0/nut @ 0x600003b400c0] video:0KiB audio:16KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 2.148750%
size=      16KiB time=00:00:00.81 bitrate= 163.9kbits/s speed= 266x

But the doc/examples/demux_decode.c sample gets an error:
Error submitting a packet for decoding (Invalid argument)

> cd doc/examples/
> ./demux_decode_g /tmp/audio.nut /tmp/video.raw /tmp/audio.raw
Could not find video stream in input file '/tmp/audio.nut'
Input #0, nut, from '/tmp/audio.nut':
  Metadata:
    encoder         : Lavf60.20.100
  Duration: 00:00:00.81, start: 0.000000, bitrate: 163 kb/s
  Stream #0:0: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16
      Metadata:
        encoder         : Lavc60.39.100 flac
Demuxing audio from file '/tmp/audio.nut' into '/tmp/audio.raw'
audio_frame n:0 nb_samples:4608 pts:0
audio_frame n:1 nb_samples:4608 pts:0.10449
audio_frame n:2 nb_samples:4608 pts:0.20898
audio_frame n:3 nb_samples:4608 pts:0.313469
audio_frame n:4 nb_samples:4608 pts:0.417959
audio_frame n:5 nb_samples:4608 pts:0.522449
audio_frame n:6 nb_samples:4608 pts:0.626939
audio_frame n:7 nb_samples:3465 pts:0.731429
Error submitting a packet for decoding (Invalid argument)
Demuxing succeeded.
Play the output audio file with the command:
ffplay -f s16le -ac 2 -ar 44100 /tmp/audio.raw

ffmpeg n4.4.4 also gets a similar error:
Error while decoding stream #0:0: Invalid argument

> ffmpeg -v 9 -loglevel 99 -i /tmp/audio.nut -c:a flac -f nut -y /tmp/audio2.nut
ffmpeg version 4.4.4 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --prefix=/opt/local --cc=/usr/bin/clang --mandir=/opt/local/share/man --enable-audiotoolbox --disable-indev=jack --disable-libjack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --enable-opencl --disable-outdev=xv --enable-sdl2 --disable-securetransport --enable-videotoolbox --enable-avfilter --enable-avresample --enable-fontconfig --enable-gnutls --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libfribidi --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libzimg --enable-libzvbi --enable-lzma --enable-pthreads --enable-shared --enable-swscale --enable-zlib --enable-libaom --enable-libsvtav1 --arch=arm64 --enable-gpl --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxvid --enable-postproc --enable-libflite
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument '/tmp/audio.nut'.
Reading option '-c:a' ... matched as option 'c' (codec name) with argument 'flac'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'nut'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '/tmp/audio2.nut' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url /tmp/audio.nut.
Successfully parsed a group of options.
Opening an input file: /tmp/audio.nut.
[NULL @ 0x12c009a00] Opening '/tmp/audio.nut' for reading
[file @ 0x6000034a97a0] Setting default whitelist 'file,crypto,data'
Probing mp3 score:1 size:2048
Probing nut score:100 size:2048
[nut @ 0x12c009a00] Format nut probed with size=2048 and score=100
[nut @ 0x12c009a00] Before avformat_find_stream_info() pos: 272 bytes read:16591 seeks:0 nb_streams:1
[nut @ 0x12c009a00] All info found
[nut @ 0x12c009a00] stream 0: start_time: 0 duration: NOPTS
[nut @ 0x12c009a00] format: start_time: 0 duration: 0.81 (estimate from pts) bitrate=163 kb/s
[nut @ 0x12c009a00] After avformat_find_stream_info() pos: 1482 bytes read:16591 seeks:0 frames:1
Input #0, nut, from '/tmp/audio.nut':
  Metadata:
    encoder         : Lavf60.20.100
  Duration: 00:00:00.81, start: 0.000000, bitrate: 163 kb/s
  Stream #0:0, 1, 1/44100: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16
    Metadata:
      encoder         : Lavc60.39.100 flac
Successfully opened the file.
Parsing a group of options: output url /tmp/audio2.nut.
Applying option c:a (codec name) with argument flac.
Applying option f (force format) with argument nut.
Successfully parsed a group of options.
Opening an output file: /tmp/audio2.nut.
[file @ 0x6000034a9b20] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
detected 12 logical cores
Stream mapping:
  Stream #0:0 -> #0:0 (flac (native) -> flac (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
    Last message repeated 8 times
Error while decoding stream #0:0: Invalid argument
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[graph_0_in_0_0 @ 0x6000028ac0b0] Setting 'time_base' to value '1/44100'
[graph_0_in_0_0 @ 0x6000028ac0b0] Setting 'sample_rate' to value '44100'
[graph_0_in_0_0 @ 0x6000028ac0b0] Setting 'sample_fmt' to value 's16'
[graph_0_in_0_0 @ 0x6000028ac0b0] Setting 'channel_layout' to value '0x3'
[graph_0_in_0_0 @ 0x6000028ac0b0] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:0x3
[format_out_0_0 @ 0x6000028ac210] Setting 'sample_fmts' to value 's16|s32'
[AVFilterGraph @ 0x600003ba0000] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
[flac @ 0x12c018400]  compression: 5
[flac @ 0x12c018400]  lpc type: Levinson-Durbin recursion with Welch window
[flac @ 0x12c018400]  prediction order: 1, 8
[flac @ 0x12c018400]  order method: estimate
[flac @ 0x12c018400]  partition order: 0, 8
[flac @ 0x12c018400]  block size: 4608
[flac @ 0x12c018400]  lpc precision: 15
Output #0, nut, to '/tmp/audio2.nut':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0, 0, 1/44100: Audio: flac ([172][241][0][0] / 0xF1AC), 44100 Hz, stereo, s16, 128 kb/s
    Metadata:
      encoder         : Lavc58.134.100 flac
[out_0_0 @ 0x6000028ac160] EOF on sink link out_0_0:default.
No more output streams to write to, finishing.
size=      16kB time=00:00:00.81 bitrate= 163.9kbits/s speed= 380x
video:0kB audio:16kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.154907%
Input file #0 (/tmp/audio.nut):
  Input stream #0:0 (audio): 9 packets read (16242 bytes); 8 frames decoded (35721 samples);
  Total: 9 packets (16242 bytes) demuxed
Output file #0 (/tmp/audio2.nut):
  Output stream #0:0 (audio): 8 frames encoded (35721 samples); 9 packets muxed (16242 bytes);
  Total: 9 packets (16242 bytes) muxed
8 frames successfully decoded, 1 decoding errors
[AVIOContext @ 0x12be06e90] Statistics: 0 seeks, 1 writeouts
[AVIOContext @ 0x12be05d60] Statistics: 16591 bytes read, 0 seeks

This is causing FFMS2 to fail to decode, see ​https://github.com/FFMS/ffms2/issues/431

[Andrew Wason]

The decode_filter_audio example also has this issue:

> cd doc/examples/
> ./decode_filter_audio_g /tmp/audio.nut >/dev/null
Output: srate:8000Hz fmt:s16 chlayout:mono
Error while sending a packet to the decoder
Error occurred: Invalid argument

I git bisected ffmpeg, and this issue was fixed in ffmpeg commit

fftools/ffmpeg_dec: restructure audio/video decoding loop

​https://github.com/FFmpeg/FFmpeg/commit/748f2999eee82fd507da327fa57e47730f5effc1

So the main ffmpeg decoding loop was changed in a way that it now handles this file, but the decoding loops in the samples were not and so they still fail.

I think these lines were the fix:
​https://github.com/FFmpeg/FFmpeg/blob/748f2999eee82fd507da327fa57e47730f5effc1/fftools/ffmpeg_dec.c#L426-L430

[Andrew Wason]

So the underlying issue is that av_read_frame can return an AVPacket with 0 size and non NULL data before EOF. Apparently this happens for fate-indeo3-2 and my ffmpeg generated NUT/FLAC sample. The main ffmpeg decoding loop has a workaround for this, but any other demuxer/decoders (like the ones in doc/examples) would need a similar workaround, or else this issue should be fixed in avformat or somewhere else itself.

[Andrew Wason]

It appears that libavformat/nutdec.c is returning the 0 size AVPacket

I added some logging

diff --git a/libavformat/nutdec.c b/libavformat/nutdec.c
index afa27b827c..69ccbb49cc 100644
--- a/libavformat/nutdec.c
+++ b/libavformat/nutdec.c
@@ -1146,6 +1146,9 @@ static int decode_frame(NUTContext *nut, AVPacket *pkt, int frame_code)
         pkt->flags |= AV_PKT_FLAG_KEY;
     pkt->pts = pts;

+    if (pkt->data && pkt->size == 0)
+        av_log(s, AV_LOG_WARNING, "Zero length packet in nut decode_frame\n");
+
     return 0;
 fail:
     av_packet_unref(pkt);

And when it returns the 0 size packet, nut decode_frame had allocated it earlier with:

ret = av_new_packet(pkt, size + nut->header_len[header_idx])

(lldb) p size + nut->header_len[header_idx]
(int) 0

So either this is a nut demuxer issue, or the nut muxer created a broken file. Or should av_new_packet not set data if size is 0?