Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#9180 closed enhancement (fixed)

AV1 extract keyframes via libaom

Reported by: svnpenn Owned by:
Priority: wish Component: avcodec
Version: git-master Keywords: libaom
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Using this file [1], I can run this command with no problem:

ffmpeg -i crushed-between-two-portals.mp4 %d.jpg

but if I run this command [2]:

ffmpeg -i crushed-between-two-portals.mp4 -vf "select='eq(pict_type,I)'" %d.jpg

I get this result:

Output file is empty, nothing was encoded (check -ss / -t / -frames parameters
if used)

I think it is because of the video stream:

Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, bt709),
1280x720 [SAR 1:1 DAR 16:9], 19 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 60k tbc
(default)

If I understand the code right, it seems the solution might be using
AV_PICTURE_TYPE_I instead [3], but that token is not currently supported [4].

  1. ​https://f002.backblazeb2.com/file/ql8mlh/crushed-between-two-portals.mp4
  2. [Create a thumbnail image every X seconds of the video](trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video)
  3. ​https://github.com/FFmpeg/FFmpeg/blob/309e3cc1/libavcodec/av1_parser.c#L124
  4. ​https://github.com/FFmpeg/FFmpeg/blob/309e3cc1/libavfilter/f_select.c#L53-L67

Change History (10)

comment:1 by Carl Eugen Hoyos, 3 years ago

Please post the command line that does not produce the expected output file together with the complete, uncut console output here to make this a valid ticket.

comment:2 by svnpenn, 3 years ago

I posted the command already above. Here is the output:

ffmpeg version 4.4-essentials_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[libaom-av1 @ 000002e5ac60f740] 3.0.0-241-g8c1aa3050
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'crushed-between-two-portals.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6av01mp41
    creation_time   : 2020-09-26T15:03:11.000000Z
  Duration: 00:02:53.77, start: 0.000000, bitrate: 459 kb/s
  Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, bt709), 854x480 [SAR 1:1 DAR 427:240], 5 kb/s, 29.97 fps, 29.97 tbr, 11988 tbn, 11988 tbc (default)
    Metadata:
      creation_time   : 2020-09-26T15:03:11.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
[libaom-av1 @ 000002e5ac625a80] 3.0.0-241-g8c1aa3050
Stream mapping:
  Stream #0:0 -> #0:0 (av1 (libaom-av1) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 000002e5adbd9280] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '%d.jpg':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6av01mp41
    encoder         : Lavf58.76.100
  Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 854x480 [SAR 1:1 DAR 427:240], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn (default)
    Metadata:
      creation_time   : 2020-09-26T15:03:11.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    0 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
Last edited 3 years ago by Carl Eugen Hoyos (previous) (diff)

comment:3 by Carl Eugen Hoyos, 3 years ago

Sorry, I am not a native speaker, in case you are: Is there a better word than together?

comment:4 by svnpenn, 3 years ago

Here you go:

ffmpeg -i crushed-between-two-portals.mp4 -vf "select='eq(pict_type,I)'" %d.jpg
ffmpeg version 4.4-essentials_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[libaom-av1 @ 000002e5ac60f740] 3.0.0-241-g8c1aa3050
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'crushed-between-two-portals.mp4':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6av01mp41
    creation_time   : 2020-09-26T15:03:11.000000Z
  Duration: 00:02:53.77, start: 0.000000, bitrate: 459 kb/s
  Stream #0:0(und): Video: av1 (Main) (av01 / 0x31307661), yuv420p(tv, bt709), 854x480 [SAR 1:1 DAR 427:240], 5 kb/s, 29.97 fps, 29.97 tbr, 11988 tbn, 11988 tbc (default)
    Metadata:
      creation_time   : 2020-09-26T15:03:11.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
[libaom-av1 @ 000002e5ac625a80] 3.0.0-241-g8c1aa3050
Stream mapping:
  Stream #0:0 -> #0:0 (av1 (libaom-av1) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 000002e5adbd9280] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '%d.jpg':
  Metadata:
    major_brand     : dash
    minor_version   : 0
    compatible_brands: iso6av01mp41
    encoder         : Lavf58.76.100
  Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 854x480 [SAR 1:1 DAR 427:240], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn (default)
    Metadata:
      creation_time   : 2020-09-26T15:03:11.000000Z
      handler_name    : ISO Media file produced by Google Inc.
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    0 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
frame=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

comment:5 by Carl Eugen Hoyos, 3 years ago

Component: undetermined → avcodec
Keywords: libaom added
Priority: normal → wish
Status: new → open
Summary: AV1 extract keyframes → AV1 extract keyframes via libaom
Type: defect → enhancement
Version: unspecified → git-master

(Fun fact: Your command line works fine on my system binary.)
I suspect that this will not be fixed.

comment:6 by svnpenn, 3 years ago

Looks like it is a build issue [1].

As a workaround, you can use the full build [2], or personally I am switching to
another builder [3], as BtbN builds are actually have a smaller extracted size.

  1. ​https://github.com/GyanD/codexffmpeg/issues/19
  2. ​https://github.com/GyanD/codexffmpeg/releases/download/4.4/ffmpeg-4.4-full_build.7z
  3. ​https://github.com/BtbN/FFmpeg-Builds/releases

comment:7 by svnpenn, 3 years ago

Here is addition information:

The input has an AV1 stream. The essentials build only has libaom linked and apparently libaom's decoder doesn't flag output frames correctly. The full build has dav1d also linked which is autoselected as it has higher priority and it does mark frames correctly. So this is a limitation of the decoder.

So it seems it is an issue with LibAOM

comment:8 by James, 3 years ago

Resolution: → fixed
Status: open → closed

Should be fixed in 8b954a727a74c5adaea9d8a1fc490c12cd4e0db8, for supported versions of libaom.

in reply to:  5 comment:9 by Carl Eugen Hoyos, 3 years ago

Replying to cehoyos:

(Fun fact: Your command line works fine on my system binary.)

I suspect that this will not be fixed.

Sorry for this comment: The libaom version installed on my system was too old.

comment:10 by svnpenn, 3 years ago

Last edited 3 years ago by svnpenn (previous) (diff)
Note: See TracTickets for help on using tickets.