Opened 5 years ago

Last modified 4 years ago

#7879 open defect

Header missing when decoding mp3 to wav / Lyrics3 tag problem

Reported by: Hendrik Schreiber Owned by:
Priority: minor Component: undetermined
Version: git-master Keywords: mp3
Cc: mhormann@gmx.de Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:

When converting an mp3 to wav, FFmpeg stops conversion upon encountering a missing header:

Header missing
Error while decoding stream #0:0: Invalid data found when processing input

sox and a custom-made decoder based on CoreAudio do not report any issues.

Here's the sox output:

sox -v99 test.mp3 out.wav
sox WARN sox: `out.wav' output clipped 9670914 samples; decrease volume?
sox WARN sox: `test.mp3' balancing clipped 19329677 samples; decrease volume?

Apart from the warnings, sox does not issue any messages, nor does it fail.

Based on the partial wav file created by FFmpeg and the debug output, I assume the failure occurs towards the end of the file in the last packet. Note that the file contains a Lyrics3 v2.00 block at the end (http://id3.org/Lyrics3v2).

I'm wondering what's the cause of the failure FFmpeg reports. Obvious options are:

  1. The header is not missing
  2. The header is missing, FFmpeg is strict and simply fails, while others are lenient, recover and march on
  3. Lyrics3 is simply not supported by FFmpeg

If 1., it's a bug
If 2., it might be nice to be able to ignore the error and ask FFmpeg to be more lenient. Is this perhaps already possible?
If 3., perhaps the ability to ignore Lyrics3 blocks should be added.

Thanks for looking into this!

How to reproduce:

ffmpeg -v 9 -loglevel 99 -i test.mp3 out.wav
ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.3_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/adoptopenjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input url with argument 'test.mp3'.
Reading option 'out.wav' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input url test.mp3.
Successfully parsed a group of options.
Opening an input file: test.mp3.
[NULL @ 0x7f8e37808800] Opening 'test.mp3' for reading
[file @ 0x7f8e37528500] Setting default whitelist 'file,crypto'
Probing mp3 score:12 size:2048
Probing mp3 score:12 size:4096
Probing mp3 score:12 size:8192
Probing mp3 score:12 size:16384
Probing mp3 score:51 size:7635
[mp3 @ 0x7f8e37808800] Format mp3 probed with size=32768 and score=51
id3v2 ver:3 flags:00 len:25123
[mp3 @ 0x7f8e37808800] pad 576 724
[mp3 @ 0x7f8e37808800] Skipping 0 bytes of junk at 26177.
[mp3 @ 0x7f8e37808800] Before avformat_find_stream_info() pos: 26177 bytes read:32768 seeks:0 nb_streams:1
[mp3 @ 0x7f8e37808800] demuxer injecting skip 1105 / discard 0
[mp3float @ 0x7f8e37816e00] skip 1105 / discard 0 samples due to side data
[mp3float @ 0x7f8e37816e00] skip 1105/1152 samples
[mp3 @ 0x7f8e37808800] All info found
[mp3 @ 0x7f8e37808800] stream 0: start_time: 0.025 duration: 229.642
[mp3 @ 0x7f8e37808800] format: start_time: 0.025 duration: 229.642 bitrate=320 kb/s
[mp3 @ 0x7f8e37808800] After avformat_find_stream_info() pos: 79425 bytes read:98304 seeks:0 frames:50
Input #0, mp3, from 'test.mp3':
  Metadata:
    title           : So Much Better
    artist          : BEAVE
    album           : Black Hole House Music 04-19 WEB
    genre           : House
    track           : 7/15
    TKEY            : 5A
    TBPM            : 126
    EnergyLevel     : 6
    TuningAlgorithm : ForestBasedKey;v=5.1.12
    Tuning          : 1
    Similarities    : 60557:75789:75604;43467:150093:62415;75789:167183:62043;51269:180744:33065;51269:89350:33065;0:121858:32693;150836:181301:32507;58700:81548:32507;44210:89907:32507;89907:150836:32507;58700:172942:32507;97709:143406:26377;36780:158638:25820;36780:189103:25
    SimilaritiesAlgorithm: SimpleDiagonalPaths;v=5.1.12
    Segments        : 0:14675:B:UNKOWN;14675:16346:B:UNKOWN;31021:13374:C:UNKOWN;44210:7058:A-:UNKOWN;51269:38080:A:VERSE;89350:33065:A:VERSE;122415:14303:D:UNKOWN;136533:14303:B:UNKOWN;150836:29907:A:VERSE;180744:33065:A:VERSE;213809:16346:E:OUTRO;
    SegmentsAlgorithm: SimpleDiagonalPaths;v=5.1.12
    AnalysisDate    : 2019-04-28T14:50:53
    publisher       : Black Hole Recordings
    comment         : Energy 6 - .: AFO (C) 2019 :.
    date            : 2019
  Duration: 00:03:49.64, start: 0.025057, bitrate: 320 kb/s
    Stream #0:0, 50, 1/14112000: Audio: mp3, 44100 Hz, stereo, fltp, 320 kb/s
    Metadata:
      encoder         : Lavf
Successfully opened the file.
Parsing a group of options: output url out.wav.
Successfully parsed a group of options.
Opening an output file: out.wav.
[file @ 0x7f8e3752e1c0] Setting default whitelist 'file,crypto'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 (mp3float) -> pcm_s16le (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[mp3float @ 0x7f8e37817a00] skip 1105 / discard 0 samples due to side data
[mp3float @ 0x7f8e37817a00] skip 1105/1152 samples
detected 8 logical cores
[graph_0_in_0_0 @ 0x7f8e3752fc40] Setting 'time_base' to value '1/44100'
[graph_0_in_0_0 @ 0x7f8e3752fc40] Setting 'sample_rate' to value '44100'
[graph_0_in_0_0 @ 0x7f8e3752fc40] Setting 'sample_fmt' to value 'fltp'
[graph_0_in_0_0 @ 0x7f8e3752fc40] Setting 'channel_layout' to value '0x3'
[graph_0_in_0_0 @ 0x7f8e3752fc40] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3
[format_out_0_0 @ 0x7f8e37530340] Setting 'sample_fmts' to value 's16'
[format_out_0_0 @ 0x7f8e37530340] auto-inserting filter 'auto_resampler_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_0'
[AVFilterGraph @ 0x7f8e37528300] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
[auto_resampler_0 @ 0x7f8e37530bc0] [SWR @ 0x112e7c000] Using fltp internally between filters
[auto_resampler_0 @ 0x7f8e37530bc0] ch:2 chl:stereo fmt:fltp r:44100Hz -> ch:2 chl:stereo fmt:s16 r:44100Hz
Output #0, wav, to 'out.wav':
  Metadata:
    INAM            : So Much Better
    IART            : BEAVE
    IPRD            : Black Hole House Music 04-19 WEB
    IGNR            : House
    IPRT            : 7/15
    TKEY            : 5A
    TBPM            : 126
    EnergyLevel     : 6
    TuningAlgorithm : ForestBasedKey;v=5.1.12
    Tuning          : 1
    Similarities    : 60557:75789:75604;43467:150093:62415;75789:167183:62043;51269:180744:33065;51269:89350:33065;0:121858:32693;150836:181301:32507;58700:81548:32507;44210:89907:32507;89907:150836:32507;58700:172942:32507;97709:143406:26377;36780:158638:25820;36780:189103:25
    SimilaritiesAlgorithm: SimpleDiagonalPaths;v=5.1.12
    Segments        : 0:14675:B:UNKOWN;14675:16346:B:UNKOWN;31021:13374:C:UNKOWN;44210:7058:A-:UNKOWN;51269:38080:A:VERSE;89350:33065:A:VERSE;122415:14303:D:UNKOWN;136533:14303:B:UNKOWN;150836:29907:A:VERSE;180744:33065:A:VERSE;213809:16346:E:OUTRO;
    SegmentsAlgorithm: SimpleDiagonalPaths;v=5.1.12
    AnalysisDate    : 2019-04-28T14:50:53
    ILNG            : English
    publisher       : Black Hole Recordings
    ICMT            : Energy 6 - .: AFO (C) 2019 :.
    ICRD            : 2019
    ISFT            : Lavf58.20.100
    Stream #0:0, 0, 1/44100: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
    Metadata:
      encoder         : Lavc58.35.100 pcm_s16le
[mp3 @ 0x7f8e37808800] demuxer injecting skip 0 / discard 195
[mp3float @ 0x7f8e37817a00] skip 0 / discard 195 samples due to side data
[mp3float @ 0x7f8e37817a00] discard 195/1152 samples
[mp3float @ 0x7f8e37817a00] Header missing
Error while decoding stream #0:0: Invalid data found when processing input
[out_0_0 @ 0x7f8e3752ffc0] EOF on sink link out_0_0:default.
No more output streams to write to, finishing.
size=   39555kB time=00:03:49.61 bitrate=1411.2kbits/s speed= 533x
video:0kB audio:39554kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000627%
Input file #0 (test.mp3):
  Input stream #0:0 (audio): 8792 packets read (9185902 bytes); 8791 frames decoded (10125932 samples);
  Total: 8792 packets (9185902 bytes) demuxed
Output file #0 (out.wav):
  Output stream #0:0 (audio): 8791 frames encoded (10125932 samples); 8791 packets muxed (40503728 bytes);
  Total: 8791 packets (40503728 bytes) muxed
8791 frames successfully decoded, 1 decoding errors
[AVIOContext @ 0x7f8e3752e240] Statistics: 4 seeks, 158 writeouts
[AVIOContext @ 0x7f8e37528680] Statistics: 9212079 bytes read, 0 seeks

Change History (3)

comment:1 by Hendrik Schreiber, 5 years ago

The test file can be downloaded from https://bit.ly/2Y3qwLj (was rejected in original post, because of file size limit).

comment:2 by Matthias C. Hormann, 4 years ago

Cc: mhormann@gmx.de added
Status: newopen
Summary: Header missing when decoding mp3 to wavHeader missing when decoding mp3 to wav / Lyrics3 tag problem

I have the same problem here, using either ffmpeg or loudgain (https://github.com/Moonbase59/loudgain/issues/11) which uses ffmpeg libraries like libavformat.

All my testing shows that ffmpeg apparently doesn’t understand Lyrics3 tags, assumes these are still audio data, and fails. In MP3 files, Lyrics3 tags are always just behind the audio data, and before a possible ID3v1/ID3v1.1 tag.

When leaving the audio and other tags intact and just removing the Lyrics3 tag on the same file, it can be decoded flawlessly and without error.

I know Lyrics3 tags are quite seldom used but still think ffmpeg should detect them and not try to decode as audio samples.

comment:3 by Carl Eugen Hoyos, 4 years ago

Keywords: Lyrics3 removed
Priority: normalminor
Reproduced by developer: set
Version: 4.1git-master
Note: See TracTickets for help on using tickets.