Support writing album cover art image embedded in ogg / opus metadata

[llogan]

$ ./ffmpeg -i ../fate-suite/cover_art/cover_art.flac ../out.ogg
ffmpeg version N-71319-g43b4342 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.2 (GCC) 20150304 (prerelease)
  configuration: --enable-gpl --disable-doc
  libavutil      54. 22.100 / 54. 22.100
  libavcodec     56. 34.100 / 56. 34.100
  libavformat    56. 29.100 / 56. 29.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 13.101 /  5. 13.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, flac, from '../fate-suite/cover_art/cover_art.flac':
  Metadata:
    RATING          : 0
    ALBUM           : Ангелофрения
    ARTIST          : Мельница
    TITLE           : Дороги
    TRACKTOTAL      : 11
    TOTALTRACKS     : 11
    GENRE           : Folk
    DATE            : 2012
    track           : 2
  Duration: 00:03:09.05, start: 0.000000, bitrate: 1 kb/s
    Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
    Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 350x350 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn, 90k tbc
    Metadata:
      comment         : Other
Output #0, ogg, to '../out.ogg':
  Metadata:
    RATING          : 0
    ALBUM           : Ангелофрения
    ARTIST          : Мельница
    TITLE           : Дороги
    TRACKTOTAL      : 11
    TOTALTRACKS     : 11
    GENRE           : Folk
    DATE            : 2012
    track           : 2
    Stream #0:0: Video: theora, none, q=2-31, 128 kb/s
    Metadata:
      comment         : Other
    Stream #0:1: Audio: flac, 0 channels, 128 kb/s
Stream mapping:
  Stream #0:1 -> #0:0 (mjpeg (native) -> ? (?))
  Stream #0:0 -> #0:1 (flac (native) -> flac (native))
Encoder (codec theora) not found for output stream #0:0

Of course if Theora is supported by the build then a Theora video stream will be outputted (90k frame rate in this case).

Possibly related to #4442 since I assume the cover art mechanics would be similar:
​https://wiki.xiph.org/VorbisComment#Cover_art

[James]

I wrote a patch a couple years ago implementing FLAC's METADATA_BLOCK_PICTURE structure: https://ffmpeg.org/pipermail/ffmpeg-devel/2013-August/146869.html

As discussed in that thread, it's been on hold until a proper AVPacketList API is implemented in order to avoid duplicating code, but it could nonetheless be committed as is.
Feel free to resurrect that thread to see what's people opinion in the matter.

After that it would be a matter of sharing that code so the Ogg muxer can also make use of it.

[James]

Related to ticket #4442

[mitar]

It would be really useful if somebody could revive this patch.

[Justin Case]

Hello,

I am also interested in this request resolution.

Here is a patch someone pointed to me:

​https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=7314

It allows ffmpeg to embed an image into an ogg container as a metadata field containing binary data, just ​like opusenc --picture mycover.jpg does. This follows the ​FLAC specification.

Please can a developer merge this to mainstream if the code is OK?

Cheers,
Justin

PS: here is a ​question on SE describing the need

[yiltiyekni]

+1 for this one. It would be really nice for my project. Looking forward for the fix.

[yiltiyekni]

+1 for this one. It would be really nice for my project. Looking forward for the fix.

[iconoclasthero]

+1

It really sucks this has been an issue for a decade and there's been a solution for better than a year and it hasn't been implemented.

[mitar]

Which solution has there been?

[microchip]

How long is this going to take? Instead of trolling on IRC and arguing endlessly on the ML, get into the code and implement it!

[Aaron Liu]

Add me to the frustration over the unreviewed patch.

[neitsab]

Just got caught by this, trying to transcode a Flac file with an embedded cover to an Opus file, only to find that the cover had been stripped and that there were no options to embed it into a matching stream:

ffmpeg -i cover.flac -c:a libopus -map 0:a -map 0:v -c:v copy test.opus -y
ffmpeg version n7.1 Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 14.2.1 (GCC) 20240910
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libdvdnav --enable-libdvdread --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgsm --enable-libharfbuzz --enable-libiec61883 --enable-libjack --enable-libjxl --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libplacebo --enable-libpulse --enable-librav1e --enable-librsvg --enable-librubberband --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpl --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-vapoursynth --enable-version3 --enable-vulkan
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100

Input #0, flac, from cover.flac:
...
  Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
  Stream #0:1: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1400x1400 [SAR 72:72 DAR 1:1], 90k tbr, 90k tbn (attached pic)
      Metadata:
        comment         : Cover (front)
Stream mapping:
  Stream #0:0 -> #0:0 (flac (native) -> opus (libopus))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[libopus @ 0x623a17473700] No bit rate set. Defaulting to 96000 bps.
[opus @ 0x623a174755c0] Unsupported codec id in stream 1
[out#0/opus @ 0x623a174754c0] Could not write header (incorrect codec parameters ?): Invalid argument
[af#0:0 @ 0x623a17475f40] Error sending frames to consumers: Invalid argument
[af#0:0 @ 0x623a17475f40] Task finished with error code: -22 (Invalid argument)
[af#0:0 @ 0x623a17475f40] Terminating thread with return code -22 (Invalid argument)
[out#0/opus @ 0x623a174754c0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=-1.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

As a workaround, opusenc works out-of-the-box without any extra option, but darn it it forces us to use another tool! ;-)

Dear devs please consider reviewing this 🙏🏻

[iconoclasthero]

Replying to neitsab:

Just got caught by this, trying to transcode a Flac file with an embedded cover to an Opus file, only to find that the cover had been stripped and that there were no options to embed it into a matching stream:

⋮

As a workaround, opusenc works out-of-the-box without any extra option, but darn it it forces us to use another tool! ;-)

Dear devs please consider reviewing this 🙏🏻

As another workaround to modify existing ogg/opus files (i.e., rather than opusen which cannot and only accepts wav/flac input) opustags as it can modify the ogg/opus file directly/in-place.