SACD DST (dff extension) sometimes produces errors

[MelkorLord]

I have a bunch of DSD files from my SACD collection ripped a long time ago with sacd-ripper ​https://github.com/sacd-ripper/sacd-ripper using the command line sacd_extract -i sacd.iso -m -p which produces Philips DSDIFF according to the usage help.

sacd_extract -i sacd.iso -m -s produces Sony DSF files from the very same SACD ISO and it's faster but resulting files are way bigger.

When I try to turn them into RF64 WAV files, some DFF (dst) files produce errors but DSF files, identified Stream #0:0: Audio: dsd_lsbf_planar, 352800 Hz, 5.1, fltp, 16934 kb/s by FFMPEG are absolutely clean and error free.

To (try to) validate that all files are clean, I used odio-sacd ​https://tari.in/www/software/odio-sacd/ which is a DST/DFF converter and it produces WAV files (non RF64) out of DFF files free of any error.

The command line is odio-sacd -i "music_file.dff" -r 176400

On the lower volume parts of the tracks, the FFMPEG WAV files have cracks and pops around the timelines where the decoding errors occurred. And the WAV files are always a tiny bit shorter in size and thus in time by 10's of milliseconds on those 5.1 files.

So I wonder if something is wrong with FFMPEG's DST decoder or if odio-sacd silently ignores them and mutes them, I have no way of knowing for sure.

From all the SACD ISOs I have, there's a pattern : For any given ISO, I either get errors on a lot of DFF or none of them. I have a lot of DFF files which decode cleanly with FFMPEG. But, if at least one DFF file in a given ISO produces decoding errors, I can be sure some of the next tracks will also produce errors. I couldn't find a pattern among SACD sources though (different artists, producers, etc)

FFMPEG's test command line is:
./ffmpeg-git-20200617 -report -i "music_file.dff" -codec:a pcm_s24le -ar 176400 -rf64 always -f null -

The report is attached.

If needed, I can send DFF files (300~500 MB each) to the FTP server, both working and producing decoding errors, just ask :-)

[MelkorLord]

FFMPEG report of DST (dst) file decoding

[Carl Eugen Hoyos]

Please only upload files that allow to reproduce an issue.

[MelkorLord]

I can not upload a DFF file for now as the FTP upload.ffmpeg.org does not seem to respond.

Trying to access that with a browser produces a TLS cert error and past that, the page lands on the Jenkins dashboard and I bet it's not the right place to upload a sample :-)

[Carl Eugen Hoyos]

​https://streams.videolan.org/upload/

[MelkorLord]

The DFF file was uploaded => dst_decoding_errors.dff and the correct Track ticket was mentioned.

[MelkorLord]

The uploaded DFF file very interesting on many levels.

It produces 1630 decoding errors with ffmpeg-git-20200616 but none with another FFMPEG version I have on another computer with a very old OS. Here is the info about this other FFMPEG version which came from ​https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4 but it's not available anymore due to deprecation :

# ffmpeg -version
ffmpeg version 4.1.3-0york1~14.04 Copyright (c) 2000-2019 the FFmpeg developers                                    
built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.4)
configuration: --prefix=/usr --extra-version='0york1~14.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu
--incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray
--enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libmodplug --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 22.100 / 56. 22.100
libavcodec     58. 35.100 / 58. 35.100
libavformat    58. 20.100 / 58. 20.100
libavdevice    58.  5.100 / 58.  5.100
libavfilter     7. 40.101 /  7. 40.101
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  3.100 /  5.  3.100
libswresample   3.  3.100 /  3.  3.100
libpostproc    55.  3.100 / 55.  3.100

The resulting WAV files are very different :

ffmpeg-git-20200617 => 872629760 bytes => 4:34.826 (1630 decoding errors)
ffmpeg 4.1.3 York => 941679776 bytes => 4:56.576 (no decoding error)

On the working file (4:56.576), there's silence at the 3:58.125 mark which lasts ~20 seconds.

On the bugged file (4:34.826), the silence part was stripped and the next song part is directly glued at the end of the first part.

So, maybe the decoding errors have something to do with near zero samples. This of course a wild guess based on empiric observation.

[MelkorLord]

This picked my curiosity so I made a small test. I 'null' decoded (like described in the bug report) all my DFF files (44) in a shell loop with -loglevel error to see the difference between ffmpeg-git-20200617 and ffmpeg-4.1.3 (York) and the results are surprising :

ffmpeg-git-20200617 : Only 3 files decoded with NO error (93.4% faulty files)
ffmpeg-4.1.3 (York) : 6 files decoded with errors (13.7% faulty files)

So, somewhere 4.1.x and 4.2.x something radically changed in FFMPEG's behavior.

[Carl Eugen Hoyos]

For the attached file, this is a regression since f6df99dba1ae64b05d08fba8160d13eb9795042f

[MelkorLord]

For the sake of fairness and to make sure you'd be able to reproduce the tests in the same conditions, I've downloaded ​https://www.johnvansickle.com/ffmpeg/old-releases/ffmpeg-4.1.4-amd64-static.tar.xz, which is the closest version to the custom build I have on my old OS, and tried again the 'null' decoding procedure on all my DFF files.

The results are exactly the same as with the custom FFMPEG ffmpeg-4.1.3 (York) on comment:6

But I see you already pinpointed a culprit. That was fast! Impressive to say the least.

[Carl Eugen Hoyos]

Couldn't you instead of testing a ticket that I have already set to reproduced provide a sample?

[MelkorLord]

Besides the already uploaded dst_decoding_errors.dff mentioned at comment:4, I've uploaded a new DST file to ​https://streams.videolan.org/upload/ named dst_decoding_errors_ffmpeg-4.1.4-amd64-static.dff which produces errors on both ffmpeg-git-20200617 and ffmpeg-4.1.4-amd64-static.

[Carl Eugen Hoyos]

No such samples showed up on streams.videolan.org

Feel free to test this patch:
https://ffmpeg.org/pipermail/ffmpeg-devel/2020-July/265524.html

[MelkorLord]

Yet no error was reported upon upload, only the small popup stating that the upload went fine and I may close the window or perform another upload.

Both DST files were around 200~300 MB which is within the accepted range of streams.videolan.org

[MelkorLord]

I've applied your patch to https://ffmpeg.org/releases/ffmpeg-snapshot-git.tar.bz2 and compiled a local version of FFMPEG.

Upon testing, no DST file produced any error. The file mentioned on comment:5 (which was really uploaded despite what you may think) is properly converted to the right file size and play time. A quick listening test did not show up any artifact.

This seems to be fixed AFAICS.

[Elon Musk]

Fixed in 1679f23beb3cfc3639352b3cbe7c08c00189c6b0.