mxf format pcm_s24le decoding is broken.

[dem0n]

Summary of the bug:
I am trying to extract audio stream from mxf file and transcode it from pcm_s24le to pcm_s16le audio, but ffmpeg returns broken file instead. ffmpeg versions used: 1.0 and N-46710-g4facddd (from git). In ffmpeg 0.7.13 this is working well with the same sample-file.

How to reproduce:
To reproduce this behavior please use pcm_s24le_to_pcm_s16le.mxf sample-file, which was uploaded to ffmpeg public ftp and run ffmpeg with following params:

~/ffmpeg-git-20121113/bin/ffmpeg -y -i ./pcm_s24le_to_pcm_s16le.mxf -vn -map 0:0 -acodec pcm_s16le ./a0_0:0.wav

----
Detailed ffmpeg output:

$ ~/ffmpeg-git-20121113/bin/ffmpeg -v 9 -loglevel 99 -y -i ./pcm_s24le_to_pcm_s16le.mxf -vn -map 0:0 -acodec pcm_s16le ./a0_0:0.wav
ffmpeg version N-46710-g4facddd Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 13 2012 18:33:45 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-libx264 --prefix=/home/mkproxy-test/ffmpeg-git-20121113 --enable-gpl --enable-shared
  libavutil      52.  6.100 / 52.  6.100
  libavcodec     54. 71.100 / 54. 71.100
  libavformat    54. 36.100 / 54. 36.100
  libavdevice    54.  3.100 / 54.  3.100
  libavfilter     3. 23.100 /  3. 23.100
  libswscale      2.  1.102 /  2.  1.102
  libswresample   0. 16.100 /  0. 16.100
  libpostproc    52.  1.100 / 52.  1.100
[mxf @ 0x1dd0280] Format mxf probed with size=2048 and score=100
[mxf @ 0x1dd0280] File position before avformat_find_stream_info() is 11741
[mxf @ 0x1dd0280] KLV for edit unit 0 extends into next edit unit - OPAtom misinterpreted as OP1a?
[mxf @ 0x1dd0280] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[mxf @ 0x1dd0280] File position after avformat_find_stream_info() is 11765
Guessed Channel Layout for  Input Stream #0.0 : mono
Input #0, mxf, from './pcm_s24le_to_pcm_s16le.mxf':
  Metadata:
    timecode        : 00:58:00:00
  Duration: 01:55:06.90, start: 0.000000, bitrate: 10 kb/s
    Stream #0:0, 0, 1001/24000: Audio: pcm_s24le, 48000 Hz, mono, s32, 1152 kb/s
[abuffer @ 0x1dd3360] Setting entry with key 'time_base' to value '1/48000'
[abuffer @ 0x1dd3360] Setting entry with key 'sample_rate' to value '48000'
[abuffer @ 0x1dd3360] Setting entry with key 'sample_fmt' to value 's32'
[abuffer @ 0x1dd3360] Setting entry with key 'channel_layout' to value '0x4'
[graph 0 input from stream 0:0 @ 0x1dd37e0] tb:1/48000 samplefmt:s32 samplerate:48000 chlayout:0x4
[aformat @ 0x1dd3640] Setting entry with key 'sample_fmts' to value 's16'
[audio format for output stream 0:0 @ 0x1dd3dc0] auto-inserting filter 'auto-inserted resampler 0' between the filter 'Parsed_anull_0' and the filter 'audio format for output stream 0:0'
[auto-inserted resampler 0 @ 0x1dd4f40] chl:mono fmt:s32 r:48000Hz -> chl:mono fmt:s16 r:48000Hz
Output #0, wav, to './a0_0:0.wav':
  Metadata:
    ISMP            : 00:58:00:00
    ISFT            : Lavf54.36.100
    Stream #0:0, 0, 1/48000: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, mono, s16, 768 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (pcm_s24le -> pcm_s16le)
Press [q] to stop, [?] for help
./pcm_s24le_to_pcm_s16le.mxf: Operation not permitted
[output stream 0:0 @ 0x1dd3d60] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
size=       0kB time=00:00:00.00 bitrate=   0.0kbits/s   
video:0kB audio:0kB subtitle:0 global headers:0kB muxing overhead inf%
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Statistics: 9458516 bytes read, 2 seeks

Thanks in advance!

[Carl Eugen Hoyos]

Regression since bf2290a8.

Thank you for the useful report, for future tickets, please consider not to use silent samples;-)

[Tomas Härdin]

The sample file is broken. Assuming you didn't mess with the file with dd (or similar) then Amberfin needs to fix their muxer. Here's an analysis:

The essence is contained in one huge, 994593600 byte (@ $2DDD) KLV. The file is only 9437489 bytes. This is a problem.

The reason this file worked before is because av_get_packet() refuses to read past EOF. This is evident from the line "Truncating packet of size 994593600 to 9425725". This also means that almost the entire file was read into RAM, which is what bf2290a8 was meant to fix. Another problem is that reading the entire rest of the file also includes what looks like FooterPartition data at the end of the file.

The reason for this is that the demuxer assumes OP1a == frame-wrapped. It *might* be legal to use clip-wrapping instead. Indeed, the EssenceContainer UL in the file (06 0E 2B 34 04 01 01 01 0D 01 03 01 02 06 02 00) corresponds to "MXF-GC Clip-wrapped Broadcast Wave audio data (SMPTE 382M)" (per RP224v10). If I force the demuxer to treat the file as OPAtom (always clip-wrapped) then it works a little better. Sadly, because MXF is MXF, it is not easy to detect the wrapping kind used in general. I have a tool that figures out what a UL means based on the RP224v10 Excel sheet (yay perl), so it's conceivable to parse the wrapping kind information out of the .xls. First I need to know the file wasn't messed with though.

Speaking of FooterPartition, it's not located where the header claims it is:

ClosedCompleteHeader
  MajorVersion = 1
  MinorVersion = 2
  KAGSize = 1
  ThisPartition = 0
  PreviousPartition = 0
  FooterPartition = 994605365
                    ^^^^^^^^^-- way past EOF
  HeaderByteCount = 11489
  IndexByteCount = 128
  IndexSID = 129
  BodyOffset = 0
  BodySID = 1
  OperationalPattern = MXF OP1a SingleItem SinglePackage
  EssenceContainers
    EssenceContainer = MXF-GC AES-BWF Audio

By the way:

        Identification
          InstanceUID = {c380b4ee-5b6e-11e1-a3ce-00265533fdc2}
          ThisGenerationUID = {c480b4ee-5b6e-11e1-a3ce-00265533fdc2}
          CompanyName = AmberFin Limited.
          ProductName = iCR
          ProductVersion = Major="7", Minor="2", Patch="0", Build="44848", Release="VersionReleased"
          VersionString = 7.2.0.44848
          ProductUID = {84ae5ffc-4710-11dd-a6fe-0010c629ec73}
          ModificationDate = 2012-03-15 5:01:35.080
          ToolkitVersion = Major="1", Minor="3", Patch="0", Build="5", Release="VersionReleased"
          Platform = MXF Desktop

So: assuming you didn't mess with the file then you need to tell Amberfin to fix their MXF muxer(s).

In the meantime, using mxfsplit is a possible workaround if you don't mind the FooterPartition crap near the end:

$ mxfsplit pcm_s24le_to_pcm_s16le.mxf && ffmpeg -f s24le -ar 48000 -ac 1 -i _0001-G16010201.Stream foo.wav

[Carl Eugen Hoyos]

Fixed by Marton Balint in c6fff3d32f4e36e6ee4b2cf9e371a884d86a510b