Opened 20 months ago

Last modified 3 weeks ago

#8349 open enhancement

Dolby AC-4 Support

Reported by: Nomis101 Owned by:
Priority: wish Component: undetermined
Version: git-master Keywords:
Cc: wrb2c@riede.org, talkenrain@gmail.com, ffmpeg@tmm1.net Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

It would be nice to support the detection, muxing and demuxing of Dolby AC-4 audio streams (in MP4 and MKV).

Information: https://dolbyac4.com/
Specs: https://www.dolby.com/us/en/technologies/ac-4/Next-Generation-Entertainment-Services.pdf
Sample files: http://testassets.dashif.org/#testcase/details/586fb3879ae9045678eab64f

MediaInfo 19.09 added support for AC-4, so it would be nice to have support in FFmpeg as well.
https://mediaarea.net/MediaInfo/ChangeLog

Attachments (2)

frame-20-789726256.mpa (24.9 KB ) - added by Santiago Otero 6 months ago.
AC-4 sample from ATSC3.0
atsc3.ts (1.6 MB ) - added by csdesigns 6 months ago.
ATSC 3.0 OTA 5.1

Download all attachments as: .zip

Change History (52)

comment:1 by richardpl, 20 months ago

Resolution: wontfix
Status: newclosed

comment:2 by richardpl, 20 months ago

Component: avcodecundetermined
Priority: normalwish
Resolution: wontfix
Status: closedreopened

comment:4 by Willem Riede, 8 months ago

Cc: wrb2c@riede.org added

ATSC3.0 Broadcast video uses AC-4 sound. Support for AC-4 in ffmpeg appears to be required for Plex TV to be used to play such video streams. I would like to see renewed urgency for this feature. Unfortunately I don't have enough knowledge on the subject to program something like this but I would be happy to help testing if that's of use.

comment:5 by MikeV, 8 months ago

I'd be willing to contribute a bit on money if it would help get AC-4 supported. Is there a preferred mechanism for funding such feature requests?

comment:6 by talkenrain, 7 months ago

Cc: talkenrain@gmail.com added

There is a growing need for Dolby AC4 support within ffmpeg. As ATSC 3.0 broadcast becomes mainstream across the US many applications (like PLEX) depend on ffmpegs support of AC4 to be compatible with this growing format. Hardware that requires support has already hit the market. Please add me to the list of willing beta testers as the lack of AC4 support is now the only obstacle to full support with PLEX.

comment:7 by Bill Mackintosh, 7 months ago

Priority: wishimportant

It is very important to get AC-4 audio processing in ffmpeg for Plex users (such as myself) as ATSC 3.0 expands and hardware such as the HDHomeRun 4K tuner make use of this. Please add me to the list of potential beta testers as I have this setup (Plex & HDHomeRun 4K)

comment:8 by Carl Eugen Hoyos, 7 months ago

Priority: importantwish

comment:9 by netsjtr08, 7 months ago

Priority: wishimportant

Adding to the request for this development as well. no sound on plex with new ATSC 3.0 tuner.

comment:10 by Carl Eugen Hoyos, 7 months ago

Priority: importantwish

comment:11 by Chris_in_Texas, 7 months ago

With more ATSC3.0 channels coming nationwide we really need support for AC3. This will become critical for other systems that use ffmpeg in short time. Please add AC3 support in so we can continue to view 4K ATSC TV.

comment:12 by richardpl, 7 months ago

Post short samples of various configurations.

comment:13 by Kyle, 6 months ago

If you want a starting point, check out this series of patches. Someone submitted them back in March to patchwork, but they failed some tests:
https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=451

comment:14 by richardpl, 6 months ago

Looks like nobody cares about AC-4, because if they care they would actually provide AC-4 samples here.

by Santiago Otero, 6 months ago

Attachment: frame-20-789726256.mpa added

AC-4 sample from ATSC3.0

in reply to:  14 comment:15 by Kyle, 6 months ago

Replying to richardpl:

Looks like nobody cares about AC-4, because if they care they would actually provide AC-4 samples here.

siryurian just posted a sample for ATSC 3.0, but links to multiple AC-4 samples were already provided via the "DASH IF" link in the initial bug report. Just click "Accept" on their TOS popup and refresh the page to get a dozen different AC-4 samples for stereo, 5.1 and multitrack audio.

Also it's not that "nobody cares", it's that ATSC 3.0 is brand new and only active in select markets in the USA, the one device that allows the average user to view it on their PC only started shipping in October, and most users affected by this have no idea how to extract the AC-4 track to provide a usable sample. (Heck most of them had no idea that VLC, Kodi and Plex all rely on FFmpeg for audio and video decoding) I would have done it myself, but the 3 markets I have access to over the air, including the biggest market NYC, have not launched ATSC 3.0 services.

Last edited 6 months ago by Kyle (previous) (diff)

in reply to:  14 comment:16 by Curtisb, 6 months ago

Replying to richardpl:
Looks like nobody cares about AC-4, because if they care they would actually provide AC-4 samples here.

That is false and absurd conclusion for a number of reasons. Here are 100 comments and rising of people starting to realize this show stopping problem. ​https://forum.silicondust.com/forum/viewtopic.php?f=133&t=74323
To any interested devs, please let us non devs know what is needed. How many clips? Does the video content matter (HD, standard def, progressive/interlaced). Do you want it where it transitions into commercials? etc.
Here is two longer HD clips from a progressive and borked interlaced stream (due to bug 4141) for anyone who can use them.
https://1drv.ms/u/s!Ap22nCAxKBS7j-Jr9Q8eX45gp2Uh7w?e=3miW5h

Last edited 6 months ago by Curtisb (previous) (diff)

in reply to:  14 comment:17 by Curtisb, 6 months ago

double post error

Last edited 6 months ago by Curtisb (previous) (diff)

comment:18 by Nomis101, 6 months ago

FYI, with the patches applied from comment 13, the AC-4 streams from the samples of comment 14 are properly detected as AC4. But the files do play with no sound, because the channels are not detected (says "0 channels"). MediaInfo does detect the channels as "5.1 Main" and "Stereo Main".

% ffplay /Users/Nomis101/Downloads/ABC\ Stream\ -\ FFmpeg\ bug\ \#8349.ts
ffplay version N-100421-g1eb751955e Copyright (c) 2003-2020 the FFmpeg developers

built with Apple clang version 12.0.0 (clang-1200.0.32.28)
configuration: --pkg-config-flags=--static --enable-swscale --enable-avfilter --enable-fontconfig --enable-gpl --enable-version3 --enable-libass --enable-libbluray --enable-libfreetype --enable-libsoxr --enable-libaom --enable-libmp3lame --enable-librav1e --enable-encoder=librav1e --enable-libtheora --enable-libvorbis --enable-libvpx --disable-debug --enable-libmysofa --enable-libx264 --enable-libx265 --enable-libxvid --enable-nonfree --enable-libfdk-aac --enable-muxer=mov --enable-muxer=mp4 --enable-muxer=ipod --enable-encoder=eac3 --enable-encoder=ac3 --enable-encoder=flac --enable-ffplay --enable-librtmp --enable-libsmbclient --enable-opencl --enable-libssh --enable-openssl --enable-encoder=truehd --enable-bsf=aac_adtstoasc --enable-encoder=h264_videotoolbox --enable-encoder=hevc_videotoolbox --enable-sdl --extra-ldflags=-L/opt/local/lib
libavutil 56. 62.100 / 56. 62.100
libavcodec 58.115.102 / 58.115.102
libavformat 58. 65.100 / 58. 65.100
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7. 93.100 / 7. 93.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100

[NULL @ 0x7f81b782ce00] Decoder requires channel count but channels not set
[NULL @ 0x7f81b7817000] Decoder requires channel count but channels not set
[NULL @ 0x7f81b782da00] Decoder requires channel count but channels not set
[mpegts @ 0x7f81b7808a00] stream 3 : no PTS found at end of file, duration not set
[mpegts @ 0x7f81b7808a00] Could not find codec parameters for stream 1 (Audio: ac4 (AC-4 / 0x342D4341), 0 channels): unspecified sample rate
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[mpegts @ 0x7f81b7808a00] Could not find codec parameters for stream 2 (Audio: ac4 (AC-4 / 0x342D4341), 0 channels): unspecified sample rate
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[mpegts @ 0x7f81b7808a00] Could not find codec parameters for stream 3 (Audio: ac4 (STPP / 0x50505453), 0 channels): unspecified sample rate
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mpegts, from '/Users/polysom/Downloads/ABC Stream - FFmpeg bug #8349.ts':

Duration: 00:01:15.34, start: 12951.873511, bitrate: 3690 kb/s
Program 3

Stream #0:0[0x31]: Video: hevc (Main 10) ([36][0][0][0] / 0x0024), yuv420p10le(tv), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc
Stream #0:1[0x32](eng): Audio: ac4 (AC-4 / 0x342D4341), 0 channels
Stream #0:2[0x33](spa): Audio: ac4 (AC-4 / 0x342D4341), 0 channels
Stream #0:3[0x3a](ee`): Audio: ac4 (STPP / 0x50505453), 0 channels

[hevc @ 0x7f81b886d200] Could not find ref with POC 56
[hevc @ 0x7f81b8814e00] Could not find ref with POC 32 0B f=0/0
[hevc @ 0x7f81b8814e00] Could not find ref with POC 24
[hevc @ 0x7f81b8814e00] Could not find ref with POC 16
13006.05 M-V: -0.025 fd= 305 aq= 0KB vq= 370KB sq= 0B f=0/0

Last edited 6 months ago by Nomis101 (previous) (diff)

comment:19 by Nomis101, 6 months ago

Ah, sorry, the "0 channel" thing was just because of AV_CODEC_CAP_CHANNEL_CONF. If I add that to ac4dec.c I have sound with the sample files from above. So, it seems with this patch we are on a good way...

ffplay /Users/Nomis101/Downloads/ABC\ Stream\ -\ FFmpeg\ bug\ \#8349.ts
ffplay version N-100421-g1eb751955e Copyright (c) 2003-2020 the FFmpeg developers

built with Apple clang version 12.0.0 (clang-1200.0.32.28)
configuration: --pkg-config-flags=--static --enable-swscale --enable-avfilter --enable-fontconfig --enable-gpl --enable-version3 --enable-libass --enable-libbluray --enable-libfreetype --enable-libsoxr --enable-libaom --enable-libmp3lame --enable-librav1e --enable-encoder=librav1e --enable-libtheora --enable-libvorbis --enable-libvpx --disable-debug --enable-libmysofa --enable-libx264 --enable-libx265 --enable-libxvid --enable-nonfree --enable-libfdk-aac --enable-muxer=mov --enable-muxer=mp4 --enable-muxer=ipod --enable-encoder=eac3 --enable-encoder=ac3 --enable-encoder=flac --enable-ffplay --enable-librtmp --enable-libsmbclient --enable-opencl --enable-libssh --enable-openssl --enable-encoder=truehd --enable-bsf=aac_adtstoasc --enable-encoder=h264_videotoolbox --enable-encoder=hevc_videotoolbox --enable-sdl --extra-ldflags=-L/opt/local/lib
libavutil 56. 62.100 / 56. 62.100
libavcodec 58.115.102 / 58.115.102
libavformat 58. 65.100 / 58. 65.100
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7. 93.100 / 7. 93.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100

[mpegts @ 0x7fd2f082ac00] stream 3 : no PTS found at end of file, duration not set
Input #0, mpegts, from '/Users/Nomis101/Downloads/ABC Stream - FFmpeg bug #8349.ts':

Duration: 00:01:15.34, start: 12951.873511, bitrate: 3690 kb/s
Program 3

Stream #0:0[0x31]: Video: hevc (Main 10) ([36][0][0][0] / 0x0024), yuv420p10le(tv), 1280x720 [SAR 1:1 DAR 16:9], 59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc
Stream #0:1[0x32](eng): Audio: ac4 (AC-4 / 0x342D4341), 48000 Hz, 5.1(side), fltp
Stream #0:2[0x33](spa): Audio: ac4 (AC-4 / 0x342D4341), 48000 Hz, stereo, fltp
Stream #0:3[0x3a](ee`): Audio: ac4 (STPP / 0x50505453), 48000 Hz, 3.0, fltp

[hevc @ 0x7fd308885c00] Could not find ref with POC 56
13030.02 A-V: 0.019 fd= 174 aq= 0KB vq= 0KB sq= 0B f=0/0

comment:20 by Aman, 6 months ago

Cc: ffmpeg@tmm1.net added

I have a patch that fixed some issues with a few stations: https://github.com/FFmpeg/FFmpeg/commit/1b9ae54caf9d1b038583ee9d1e94f130b42b5f7d

comment:21 by richardpl, 6 months ago

Do you have samples to reproduce this, as that patch contradicts with specifications.

comment:22 by richardpl, 6 months ago

comment:23 by Yourgos Papanikolaou, 6 months ago

All i know is all the main stations in seattle have switched to ATSC 3.0 so I upgraded to a new HDHomeRun 4k for my Plex DVR. All I can see is really clear video with no audio stream found. So no football for me or anyone else in the same situation. Happy to help however I can and willing to chip in if needed.

in reply to:  23 ; comment:24 by Carl Eugen Hoyos, 6 months ago

Replying to georgeP:

Happy to help however I can

Did you see comment:14 and comment:21?

in reply to:  24 comment:26 by Yourgos Papanikolaou, 6 months ago

Replying to cehoyos:

Replying to georgeP:

Happy to help however I can

Did you see comment:14 and comment:21?

I have no idea how to do that. If you can provide instructions I will try.

comment:27 by MikeV, 6 months ago

I've uploaded a TS file from an OKC OTA station with AC-4 audio here:
https://drive.google.com/file/d/1xpEyd59cD4NgLtfYsWuBOrMjovzCO7xj/view?usp=sharing

Unfortunately I can't figure out how to demux the TS file and just upload the audio portion. If anyone has instructions for that, please let me know. I could record multiple samples.

comment:28 by phroenips, 6 months ago

Two more samples. These are 30 second extracts from a HDHomerun Quatro 4K. I'm also not sure how to demux just the audio, so these are video and audio

https://1drv.ms/v/s!ArfvDX-zLpfP9XamL2kwsu2JnpKR?e=TBE0XE
https://1drv.ms/v/s!ArfvDX-zLpfP9XVdkW4CMgI5EIKd?e=XvsCgN

in reply to:  22 comment:29 by mooninite, 6 months ago

Replying to richardpl:

This is latest AC4 WIP code: https://github.com/richardpl/FFmpeg/tree/ac4

I tested these patches against a 4.3 branch (Fedora/RPMFusion) and could successfully play live streaming data in the Dallas, TX market from a SiliconDust CONNECT 4K and Kodi. Audio is working. Thanks!

in reply to:  22 comment:30 by Nomis101, 6 months ago

Replying to richardpl:

This is latest AC4 WIP code: https://github.com/richardpl/FFmpeg/tree/ac4

Playback does also work nicely for me. But if I try to convert the audio stream in e.g. AAC, I see different errors with the provided samples.

Sample from comment 24 gives me:
[ac4 @ 0x7fef1480a200] substream audio data overread: 1153
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] substream audio data overread: 598
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] substream audio data underread: 8
[ac4 @ 0x7fef1480a200] substream audio data underread: 10eed= 229x
[ac4 @ 0x7fef1480a200] sect_cb[0][0] > 11
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] substream audio data underread: 257
[ac4 @ 0x7fef1480a200] substream audio data underread: 131
[ac4 @ 0x7fef1480a200] substream audio data overread: 1217
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] substream audio data underread: 104
[ac4 @ 0x7fef1480a200] substream audio data underread: 119
[ac4 @ 0x7fef1480a200] sect_cb[0][0] > 11
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] sect_cb[0][3] > 11
Error while decoding stream #0:0: Invalid data found when processing input
[ac4 @ 0x7fef1480a200] substream audio data underread: 106
[ac4 @ 0x7fef1480a200] substream audio data underread: 122
[ac4 @ 0x7fef1480a200] sect_cb[0][0] > 11
Error while decoding stream #0:0: Invalid data found when processing input

Sample from comment 27 gives me:
[ac4 @ 0x7fe01f008c00] substream audio data overread: 2
Error while decoding stream #0:1: Invalid data found when processing input
[mpegts @ 0x7fe020809a00] Packet corrupt (stream = 0, dts = 1229347550).
[ac4 @ 0x7fe01f008c00] invalid aspx num env: 7 (class 3)
Error while decoding stream #0:1: Invalid data found when processing input

Sample (first one) from comment 28 gives me:
[ac4 @ 0x7ff98980f600] substream audio data overread: 6
Error while decoding stream #0:2: Invalid data found when processing input

by csdesigns, 6 months ago

Attachment: atsc3.ts added

ATSC 3.0 OTA 5.1

in reply to:  22 comment:31 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:32 by LostDog, 5 months ago

I compiled the ac4 branch at https://github.com/richardpl/FFmpeg/tree/ac4 using Linux Mint 20.1. It built successfully and I was able to play the atsc3.ts samples with AC-4 audio using ffplay.

I've not been able to compile VLC or Kodi against this branch but I'm suspecting it's due to my own lack of understanding rather than the branch code.

in reply to:  32 ; comment:33 by Stephen Ferrell, 5 months ago

Status: reopenedopen
Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

in reply to:  33 ; comment:34 by LostDog, 5 months ago

Replying to ferrellsl:

Let us know if you're successful. I'd really like to have a PVR that records and plays ATSC3.0 streams without using a bunch of scripts and hacks. Over at the NextPVR forums there's a guy who has written some python scripts that use the binaries from this tree to transcode the ATSC3.0/AC4 audio stream into AC3 on-the-fly but I'm hoping for a cleaner solution.

For now, I can still record my ATSC3.0 channels using NextPVR and use the binaries from this tree for playback of those ATSC3.0 recordings without transcoding, and of course all the 4K channels in the Phoenix area are mirrored in 1080p/ATSC1.0 channels. I'm quite pleased with richardpl's progress. I was expecting things to move much slower than they have, or not at all. At least now I have a poor-man's PVR using scripts for playback. Having AC4 integrated natively into VLC, Kodi, or NextPVR would be very welcome from my standpoint.

I have successfully built Kodi v18.9 against the AC-4 branch of ffmpeg. ATSC3.0 captures work seemingly flawlessly with audio. I cannot verify if it's 5.1 and the channels are correct but on my computer with stereo headphones the audio sounds great.

Tuning live ATSC3.0 is really buggy using the HDHomerun plugins but it will occasionally play and when it does there is audio. I don't think this has anything to do with ffmpeg but rather the plugins themselves.

in reply to:  34 comment:35 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:36 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:37 by Moe Sparkman, 5 months ago

Priority: wishimportant

So I'm new to this. I'm on Plex installed on Ubuntu. I cloned this: https://github.com/richardpl/FFmpeg/tree/ac4. Configured it, but I'm unsure how to get plex use it. I appreciate any advice.

in reply to:  37 comment:38 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:39 by Carl Eugen Hoyos, 5 months ago

Priority: importantwish

in reply to:  39 ; comment:40 by Moe Sparkman, 5 months ago

Replying to cehoyos:

I did as you said. Unfortunately, no luck. I even tried both copying the binary to the PMS folder and creating a symbolic link, with no success. I'm a noob, sorry.

Thank you for your time.

in reply to:  40 comment:41 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

in reply to:  40 comment:42 by Carl Eugen Hoyos, 5 months ago

Replying to Moe247:

Replying to cehoyos:

I'm a noob, sorry.

I don't know your exact task, but in this case, you should probably wait for a package made by somebody else.

comment:43 by Nomis101, 5 months ago

Instead of using an incomplete WIP patch in various customized builds, would somebody be able to finish this patch and submit it to the list?

in reply to:  21 comment:44 by Aman, 5 months ago

Replying to richardpl:

Do you have samples to reproduce this, as that patch contradicts with specifications.

The sample in comment:27 reproduces, as does this one: https://tmm1.s3.amazonaws.com/atsc3-aspx-num-env.ts

Where in the spec does it state that aspx_num_env is capped at 5?

comment:45 by Stephen Ferrell, 5 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

in reply to:  45 comment:46 by Stephen Ferrell, 4 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:47 by richardpl, 4 months ago

Is this open source or what kind of endorsement is this at all?

in reply to:  47 comment:48 by Stephen Ferrell, 4 months ago

Last edited 7 weeks ago by Stephen Ferrell (previous) (diff)

comment:49 by prldm, 7 weeks ago

I compiled the ac4 branch of richardpl's fork and it looks like it can not decode AC4 which is inside an mp4 container.
https://www107.zippyshare.com/v/zsCHrWop/file.html

https://i.kek.sh/5APayCpR5HN.png

comment:50 by dotjoel, 3 weeks ago

I just wanted to point out that there is a bounty on this feature.
https://www.bountysource.com/issues/83125729-dolby-ac-4-support

Note: See TracTickets for help on using tickets.