audiotoolbox ac3 decoder will hang on some inputs

[Aman]

To reproduce, use ffplay to play this sample: ​http://s3.amazonaws.com/tmm1/ac3_at_deadlock.ts

One ffplay starts, hit the left/right keys in rapid succession 3-4 times to seek forward and back. Audio will stop playing, and when you try to quit with the Q key the player will deadlock.

$ ./ffplay_g -threads 1 -codec:a ac3_at -i /tmp/ac3_at_deadlock.ts
ffplay version n3.1.3-359-g2a65f1e Copyright (c) 2003-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.38)
...

Using lldb I see the audio decoder is stuck with the following backtrace:

* thread #6: tid = 0x721540, 0x00007fffbce57bc9 libsystem_platform.dylib`_platform_bzero$VARIANT$Haswell + 41
  * frame #0: 0x00007fffbce57bc9 libsystem_platform.dylib`_platform_bzero$VARIANT$Haswell + 41
    frame #1: 0x00000001150072c2 AudioCodecs`ACUDCDecoder::ProduceOutputBufferList(AudioBufferList*, unsigned int&, AudioStreamPacketDescription*) + 392
    frame #2: 0x000000011500712d AudioCodecs`ACUDCDecoder::ProduceOutputPackets(void*, unsigned int&, unsigned int&, AudioStreamPacketDescription*) + 53
    frame #3: 0x0000000115149384 AudioCodecs`ProduceOutputPackets(void*, void*, unsigned int*, unsigned int*, AudioStreamPacketDescription*, unsigned int*) + 23
    frame #4: 0x00007fffa6a9daa3 AudioToolbox`CodecConverter::DecoderFillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 1199
    frame #5: 0x00007fffa6aa9c93 AudioToolbox`BufferedAudioConverter::GetInputBytes(unsigned int, unsigned int&, CABufferList const*&) + 135
    frame #6: 0x00007fffa6aa9b82 AudioToolbox`CBRConverter::RenderOutput(CABufferList*, unsigned int, unsigned int&, AudioStreamPacketDescription*) + 106
    frame #7: 0x00007fffa6a9cc9d AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 281
    frame #8: 0x00007fffa6a9cf7f AudioToolbox`AudioConverterChain::RenderOutput(CABufferList*, unsigned int, unsigned int&, AudioStreamPacketDescription*) + 99
    frame #9: 0x00007fffa6a9cc9d AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 281
    frame #10: 0x00007fffa6a9c76f AudioToolbox`AudioConverterFillComplexBuffer + 282
    frame #11: 0x000000010d7316ab libavcodec.57.dylib`___lldb_unnamed_symbol344$$libavcodec.57.dylib + 715
    frame #12: 0x000000010dc753db libavcodec.57.dylib`avcodec_decode_audio4 + 251
    frame #13: 0x000000010d2fe7eb ffplay`___lldb_unnamed_symbol45$$ffplay + 747
    frame #14: 0x000000010d2fc0a1 ffplay`___lldb_unnamed_symbol39$$ffplay + 1153
    frame #15: 0x000000010eb2cf8d libSDL-1.2.0.dylib`SDL_RunThread + 53
    frame #16: 0x000000010eb4f764 libSDL-1.2.0.dylib`RunThread + 9
    frame #17: 0x00007fffbce5dabb libsystem_pthread.dylib`_pthread_body + 180
    frame #18: 0x00007fffbce5da07 libsystem_pthread.dylib`_pthread_start + 286
    frame #19: 0x00007fffbce5d231 libsystem_pthread.dylib`thread_start + 13

This happens reproducibly on both macOS and iOS.

[Carl Eugen Hoyos]

Is this issue reproducible with current FFmpeg git head?

[Aman]

Yes it is. I last tested with 47ee6f1 from a couple days ago.

[Carl Eugen Hoyos]

To make this a valid ticket, please:
Test current FFmpeg git head, compile without any options to provide a useful backtrace, provide ffmpeg -i console output for your sample and the backtrace.

[Aman]

$ ./ffmpeg -i /tmp/ac3_at_deadlock.ts
ffmpeg version N-82119-g1342339 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 8.0.0 (clang-800.0.38)
  configuration: --prefix=/tmp/ffmpeg-master --enable-shared --disable-static --disable-stripping
  libavutil      55. 33.100 / 55. 33.100
  libavcodec     57. 63.103 / 57. 63.103
  libavformat    57. 55.100 / 57. 55.100
  libavdevice    57.  0.103 / 57.  0.103
  libavfilter     6. 64.100 /  6. 64.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  2.100 /  2.  2.100
[mpeg2video @ 0x7ff5ea000600] Invalid frame dimensions 0x0.
    Last message repeated 13 times
[mpegts @ 0x7ff5ea002600] Could not find codec parameters for stream 2 (Unknown: none ([5][0][0][0] / 0x0005)): unknown codec
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from '/tmp/ac3_at_deadlock.ts':
  Duration: 00:00:31.75, start: 58675.240989, bitrate: 11415 kb/s
  Program 602
    Stream #0:0[0x1785]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bottom first), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
    Stream #0:1[0x1786](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s
    Stream #0:2[0x1788]: Unknown: none ([5][0][0][0] / 0x0005)
At least one output file must be specified

* thread #6: tid = 0x96c216, 0x00007fffa6bbbefd AudioToolbox`AudioCodecProduceOutputPackets + 131, name = 'decoder'
  * frame #0: 0x00007fffa6bbbefd AudioToolbox`AudioCodecProduceOutputPackets + 131
    frame #1: 0x00007fffa6a9daa3 AudioToolbox`CodecConverter::DecoderFillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 1199
    frame #2: 0x00007fffa6aa9c93 AudioToolbox`BufferedAudioConverter::GetInputBytes(unsigned int, unsigned int&, CABufferList const*&) + 135
    frame #3: 0x00007fffa6aa9b82 AudioToolbox`CBRConverter::RenderOutput(CABufferList*, unsigned int, unsigned int&, AudioStreamPacketDescription*) + 106
    frame #4: 0x00007fffa6a9cc9d AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 281
    frame #5: 0x00007fffa6a9cf7f AudioToolbox`AudioConverterChain::RenderOutput(CABufferList*, unsigned int, unsigned int&, AudioStreamPacketDescription*) + 99
    frame #6: 0x00007fffa6a9cc9d AudioToolbox`BufferedAudioConverter::FillBuffer(unsigned int&, AudioBufferList&, AudioStreamPacketDescription*) + 281
    frame #7: 0x00007fffa6a9c76f AudioToolbox`AudioConverterFillComplexBuffer + 282
    frame #8: 0x00000001073c1fe8 libavcodec.57.dylib`ffat_decode(avctx=0x00007fb0b701c200, data=0x00007fb0b6d0a580, got_frame_ptr=<unavailable>, avpkt=0x00007000098fd4c8) + 680 at audiotoolboxdec.c:549 [opt]
    frame #9: 0x0000000107914560 libavcodec.57.dylib`avcodec_decode_audio4(avctx=0x00007fb0b701c200, frame=<unavailable>, got_frame_ptr=0x00007000098fd5d4, avpkt=0x000000010e4486d0) + 256 at utils.c:2362 [opt]
    frame #10: 0x00000001068416bb ffplay_g`decoder_decode_frame(d=<unavailable>, frame=<unavailable>, sub=<unavailable>) + 747 at ffplay.c:597 [opt]
    frame #11: 0x000000010683f401 ffplay_g`audio_thread(arg=<unavailable>) + 1153 at ffplay.c:2025 [opt]
    frame #12: 0x0000000106e75442 libSDL2-2.0.0.dylib`SDL_RunThread + 60
    frame #13: 0x0000000106eb22e5 libSDL2-2.0.0.dylib`RunThread + 9
    frame #14: 0x00007fffbce5dabb libsystem_pthread.dylib`_pthread_body + 180
    frame #15: 0x00007fffbce5da07 libsystem_pthread.dylib`_pthread_start + 286
    frame #16: 0x00007fffbce5d231 libsystem_pthread.dylib`thread_start + 13

[Elon Musk]

Not our bug.