DTS decoder produces invalid sample format for last frame

[Marton Balint]

Summary of the bug:
How to reproduce:

% ffprobe invalid-sample-fmt-in-last-frame.dts -show_frames -of compact

Output

ffprobe version N-82844-g265d451 Copyright (c) 2007-2016 the FFmpeg developers
  built with gcc 4.8 (SUSE Linux)
  configuration: --disable-vaapi --enable-vdpau --enable-gpl --enable-libfreetype --enable-libzvbi --enable-libass --enable-frei0r --enable-openssl --enable-nonfree --enable-libx264 --enable-libfdk-aac --enable-nonfree --enable-libvorbis --enable-decklink --enable-libmp3lame --enable-openal
  libavutil      55. 43.100 / 55. 43.100
  libavcodec     57. 67.100 / 57. 67.100
  libavformat    57. 59.100 / 57. 59.100
  libavdevice    57.  2.100 / 57.  2.100
  libavfilter     6. 68.100 /  6. 68.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
  libpostproc    54.  2.100 / 54.  2.100
[dts @ 0x24ee260] Estimating duration from bitrate, this may be inaccurate
Input #0, dts, from 'invalid-sample-fmt-in-last-frame.dts':
  Duration: 00:00:00.07, start: 0.000000, bitrate: 1535 kb/s
    Stream #0:0: Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s
frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=0|pkt_pts_time=0.000000|pkt_dts=0|pkt_dts_time=0.000000|best_effort_timestamp=0|best_effort_timestamp_time=0.000000|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=0|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=960|pkt_pts_time=0.010667|pkt_dts=960|pkt_dts_time=0.010667|best_effort_timestamp=960|best_effort_timestamp_time=0.010667|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=2012|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=1920|pkt_pts_time=0.021333|pkt_dts=1920|pkt_dts_time=0.021333|best_effort_timestamp=1920|best_effort_timestamp_time=0.021333|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=4024|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=2880|pkt_pts_time=0.032000|pkt_dts=2880|pkt_dts_time=0.032000|best_effort_timestamp=2880|best_effort_timestamp_time=0.032000|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=6036|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=3840|pkt_pts_time=0.042667|pkt_dts=3840|pkt_dts_time=0.042667|best_effort_timestamp=3840|best_effort_timestamp_time=0.042667|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=8048|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=4800|pkt_pts_time=0.053333|pkt_dts=4800|pkt_dts_time=0.053333|best_effort_timestamp=4800|best_effort_timestamp_time=0.053333|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=10060|pkt_size=2012|sample_fmt=fltp|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

frame|media_type=audio|stream_index=0|key_frame=1|pkt_pts=5760|pkt_pts_time=0.064000|pkt_dts=5760|pkt_dts_time=0.064000|best_effort_timestamp=5760|best_effort_timestamp_time=0.064000|pkt_duration=960|pkt_duration_time=0.010667|pkt_pos=12072|pkt_size=2080|sample_fmt=s16p|nb_samples=512|channels=6|channel_layout=5.1(side)side_data|side_data_type=AVMatrixEncoding|side_data_size=4

The old DTS decoder worked correctly. The new DTS decoder also works if -core_only 1 is speicified.

[James]

I'm not sure if this is a bug. The last frame has an extension substream (XLL it seems) and the new decoder is processing it.

It's also why the previous decoder and the current one with -core_only 1 don't display this behavior: In both, exss is simply ignored.

[Hendrik]

What makes you even think its an "invalid" sample format? s16p is perfectly valid as a sample format. And as jamrial explained, the decoder picks up the lossless extension and switches to lossless output, as it should.

[Marton Balint]

I was actually hunting down an assertion failure in af_atempo, and I thought that an audio decoder should not change its sample format during decoding, it seemed very unusual.

But if this is not a bug, then I guess it is OK, and I will take a second look at af_atempo to fix the original assertion failure there. Thanks.