Opened 2 years ago

Last modified 2 years ago

#10164 new defect

Segfault dual pass transcoding to AV1 with libaom

Reported by: Ferdi Scholten Owned by:
Priority: normal Component: undetermined
Version: git-master Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug: Transcoding video to AV1 with libaom in dual pass mode often ends with segfault. Transcoding the same video in single pass mode with the same parameters always works fine.

Might happen with several input formats such as AVC, h264 or MP4 ASP. The segfault does not always happen on all files of these types. The segfault does always occur at the end of the transcode, the resulting output video is missing everything after the last keyframe of the input.

This might already be going on for some time as I mainly use scripts (without screen output) to transcode in bulk.

ffmpeg[10706]: segfault at 28 ip 000055e092c976e1 sp 00007fff3e1e6f08 error 4 in ffmpeg[55e091270000+280a000] likely on CPU 5 (core 2, socket 0)

How to reproduce:
ffmpeg -i test.mkv -c:v libaom-av1 -pass 1 -cpu-used 4 -row-mt true -threads 8 -lag-in-frames 48 -crf 35 -tiles 1x2 -aq-mode 3 -an -f null /dev/null
ffmpeg -i test.mkv -c:v libaom-av1 -pass 2 -cpu-used 4 -row-mt true -threads 8 -lag-in-frames 48 -crf 35 -tiles 1x2 -aq-mode 3 -c:a libopus -vbr on -c:s copy A/test.mkv

ffmpeg version N-109745-g7d49fef8b4 Copyright (c) 2000-2023 the FFmpeg developers

built with gcc 12 (Ubuntu 12.2.0-3ubuntu1)
configuration: --prefix=/home/ferdi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ferdi/ffmpeg_build/include --extra-ldflags=-L/home/ferdi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/ferdi/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-libvmaf --enable-lto
libavutil 57. 44.100 / 57. 44.100
libavcodec 59. 61.100 / 59. 61.100
libavformat 59. 37.100 / 59. 37.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 56.100 / 8. 56.100
libswscale 6. 8.112 / 6. 8.112
libswresample 4. 9.100 / 4. 9.100
libpostproc 56. 7.100 / 56. 7.100

Attachments (2)

commandline.txt​ (318 bytes ) - added by Ferdi Scholten 2 years ago.
fileinfo.log​ (35.0 KB ) - added by Ferdi Scholten 2 years ago.

Download all attachments as: .zip

Change History (6)

by Ferdi Scholten, 2 years ago

Attachment: commandline.txt​ added

by Ferdi Scholten, 2 years ago

Attachment: fileinfo.log​ added

comment:1 by Ferdi Scholten, 2 years ago

Summary: Segfault dual pass transcoding AVC to AV1 → Segfault dual pass transcoding to AV1 with libaom

Will test if segfault also occurs using last release of ffmpeg instead of git master. Only recently started using 2 pass for libaom.

comment:2 by Ferdi Scholten, 2 years ago

Just built ffmpeg n5.1.2 this version also segfaults at the end of the transcode.

Output from first pass:

ffmpeg -i test.mkv -c:v libaom-av1 -pass 1 -cpu-used 4 -row-mt true -threads 8 -lag-in-frames 48 -crf 35 -tiles 1x2 -aq-mode 3 -an -f null /dev/null
ffmpeg version n5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Ubuntu 12.2.0-3ubuntu1)
  configuration: --prefix=/home/ferdi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ferdi/ffmpeg_build/include --extra-ldflags=-L/home/ferdi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/ferdi/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-libvmaf --enable-lto
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from 'test.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    ENCODER         : Lavf59.37.100
  Duration: 00:01:00.00, start: 0.000000, bitrate: 3810 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30.08 fps, 30.08 tbr, 1k tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:00:59.897000000
  Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : SoundHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:01:00.000000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> av1 (libaom-av1))
Press [q] to stop, [?] for help
[libaom-av1 @ 0x55f4346c53c0] 3.5.0
Output #0, null, to '/dev/null':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    encoder         : Lavf59.27.100
  Stream #0:0: Video: av1, yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30.08 fps, 30.08 tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:00:59.897000000
      encoder         : Lavc59.37.100 libaom-av1
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 1800 fps= 44 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded

Will try to get output from second pass as well. After the segfault, the terminal does not behave well and no longer has a cursor or visible output.

comment:3 by Ferdi Scholten, 2 years ago

Output from second pass:

 ffmpeg -i test.mkv -c:v libaom-av1 -pass 2 -cpu-used 4 -row-mt true -threads 8 -lag-in-frames 48 -crf 35 -tiles 1x2 -aq-mode 3 -c:a libopus -vbr on -c:s copy A/test.mkv
ffmpeg version n5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Ubuntu 12.2.0-3ubuntu1)
  configuration: --prefix=/home/ferdi/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/ferdi/ffmpeg_build/include --extra-ldflags=-L/home/ferdi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --ld=g++ --bindir=/home/ferdi/bin --enable-gpl --enable-gnutls --enable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-libvmaf --enable-lto
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from 'test.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    ENCODER         : Lavf59.37.100
  Duration: 00:01:00.00, start: 0.000000, bitrate: 3810 kb/s
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30.08 fps, 30.08 tbr, 1k tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:00:59.897000000
  Stream #0:1: Audio: aac (LC), 44100 Hz, stereo, fltp (default)
    Metadata:
      HANDLER_NAME    : SoundHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:01:00.000000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> av1 (libaom-av1))
  Stream #0:1 -> #0:1 (aac (native) -> opus (libopus))
Press [q] to stop, [?] for help
[libopus @ 0x55d4cba39080] No bit rate set. Defaulting to 96000 bps.
[libaom-av1 @ 0x55d4cba2d1c0] 3.5.0
Output #0, matroska, to 'A/test.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    encoder         : Lavf59.27.100
  Stream #0:0: Video: av1 (AV01 / 0x31305641), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30.08 fps, 1k tbn (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:00:59.897000000
      encoder         : Lavc59.37.100 libaom-av1
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000 Hz, stereo, flt, 96 kb/s (default)
    Metadata:
      HANDLER_NAME    : SoundHandler
      VENDOR_ID       : [0][0][0][0]
      DURATION        : 00:01:00.000000000
      encoder         : Lavc59.37.100 libopus
Segmentation fault4 q=0.0 size=    8448kB time=00:01:00.00 bitrate=1153.4kbits/s dup=0 drop=41 speed=0.0471x

comment:4 by Ferdi Scholten, 2 years ago

Done some more testing in 2pass transcoding:

Converting the input video to I420 (test2.y4m) rawvideo. Transcoding to av1 with libaom to these containers:
webm -> transcodes fine
mkv -> transcodes fine
mp4 -> with every frame: [vost#0:0/libaom-av1 @ 0x55bf6e589540] Overriding packet duration by frame rate, this should not happen Output video is fine

Also used aomenc to transcode test2.y4m to webm with same parameters as libaom, no problem and fast! (speed of this encode is about 150% of that of FFmpeg).

Transcoding the original test2.mkv (h264 video) directly to av1 in these containers:
mkv -> segfault at the end of the transcode
mp4 -> with every frame [vost#0:0/libaom-av1 @ 0x55bf6e589540] Overriding packet duration by frame rate, this should not happen followed by segfault at the end of the transcode. Resulting file cannot be played. [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5579892b3840] moov atom not found

Note: See TracTickets for help on using tickets.