Add lossless option to libaom-av1

[Ewout]

libaom supports lossless encoding, an option could be added to the CLI to enable this mode.

in libvpx-vp9:
-lossless <int> E..V..... Lossless mode (from -1 to 1) (default -1)

in aomenc:
--lossless=<arg> Lossless mode (0: false (default), 1: true)

[Carl Eugen Hoyos]

Please test this patch:

diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
index 09ef423..2b44bda 100644
--- a/libavcodec/libaomenc.c
+++ b/libavcodec/libaomenc.c
@@ -79,6 +79,7 @@ typedef struct AOMEncoderContext {
     aom_superblock_size_t superblock_size;
     int uniform_tiles;
     int row_mt;
+    int lossless;
 } AOMContext;

 static const char *const ctlidstr[] = {
@@ -96,6 +97,9 @@ static const char *const ctlidstr[] = {
 #ifdef AOM_CTRL_AV1E_SET_ROW_MT
     [AV1E_SET_ROW_MT]           = "AV1E_SET_ROW_MT",
 #endif
+#ifdef AOM_CTRL_AV1E_SET_LOSSLESS
+    [AV1E_SET_LOSSLESS]         = "AV1E_SET_LOSSLESS",
+#endif
 };

 static av_cold void log_encoder_error(AVCodecContext *avctx, const char *desc)
@@ -657,6 +661,9 @@ static av_cold int aom_init(AVCodecContext *avctx,
 #ifdef AOM_CTRL_AV1E_SET_ROW_MT
     codecctl_int(avctx, AV1E_SET_ROW_MT, ctx->row_mt);
 #endif
+#ifdef AOM_CTRL_AV1E_SET_LOSSLESS
+    codecctl_int(avctx, AV1E_SET_LOSSLESS, ctx->lossless);
+#endif

     // provide dummy value to initialize wrapper, values will be updated each _encode()
     aom_img_wrap(&ctx->rawimg, img_fmt, avctx->width, avctx->height, 1,
@@ -992,6 +999,7 @@ static const AVOption options[] = {
     { "tile-columns",     "Log2 of number of tile columns to use", OFFSET(tile_cols_log2), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 6, VE},
     { "tile-rows",        "Log2 of number of tile rows to use",    OFFSET(tile_rows_log2), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 6, VE},
     { "row-mt",           "Enable row based multi-threading",      OFFSET(row_mt),         AV_OPT_TYPE_BOOL, {.i64 = 0},  0, 1, VE},
+    { "lossless",         "Lossless mode",                         OFFSET(lossless),       AV_OPT_TYPE_BOOL, {.i64 = 0},  0, 1, VE}
     { NULL }
 };

[James]

AOM_CTRL_AV1E_SET_LOSSLESS exists in libaom 1.0.0, so no need for preprocessor guards.

[Carl Eugen Hoyos]

Like this?

diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
index 09ef423..9ec5c19 100644
--- a/libavcodec/libaomenc.c
+++ b/libavcodec/libaomenc.c
@@ -79,6 +79,7 @@ typedef struct AOMEncoderContext {
     aom_superblock_size_t superblock_size;
     int uniform_tiles;
     int row_mt;
+    int lossless;
 } AOMContext;

 static const char *const ctlidstr[] = {
@@ -96,6 +97,7 @@ static const char *const ctlidstr[] = {
 #ifdef AOM_CTRL_AV1E_SET_ROW_MT
     [AV1E_SET_ROW_MT]           = "AV1E_SET_ROW_MT",
 #endif
+    [AV1E_SET_LOSSLESS]         = "AV1E_SET_LOSSLESS",
 };

 static av_cold void log_encoder_error(AVCodecContext *avctx, const char *desc)
@@ -657,6 +659,7 @@ static av_cold int aom_init(AVCodecContext *avctx,
 #ifdef AOM_CTRL_AV1E_SET_ROW_MT
     codecctl_int(avctx, AV1E_SET_ROW_MT, ctx->row_mt);
 #endif
+    codecctl_int(avctx, AV1E_SET_LOSSLESS, ctx->lossless);

     // provide dummy value to initialize wrapper, values will be updated each _encode()
     aom_img_wrap(&ctx->rawimg, img_fmt, avctx->width, avctx->height, 1,
@@ -992,6 +995,7 @@ static const AVOption options[] = {
     { "tile-columns",     "Log2 of number of tile columns to use", OFFSET(tile_cols_log2), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 6, VE},
     { "tile-rows",        "Log2 of number of tile rows to use",    OFFSET(tile_rows_log2), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 6, VE},
     { "row-mt",           "Enable row based multi-threading",      OFFSET(row_mt),         AV_OPT_TYPE_BOOL, {.i64 = 0},  0, 1, VE},
+    { "lossless",         "Lossless mode",                         OFFSET(lossless),       AV_OPT_TYPE_BOOL, {.i64 = 0},  0, 1, VE}
     { NULL }
 };

[Ewout]

Thanks! Would love to test it, how can I build FFmpeg with this patch?

[Carl Eugen Hoyos]

Just apply it and rebuild as usual, nothing special should be needed.

[Ewout]

Sorry, I'm not experienced enough with compiling FFmpeg. If someone could provide me a Windows build with this patch (or if someone wants to talk me through it) I would be happy to test if the results are comparable to ​aomenc.

Otherwise, if someone else is able to test it please do. I think compressing a lossless PNG and checking if there's no PSNR loss should suffice. Comparing the output with aomenc with the --lossless options should also work.

cehoyos thanks for the patch anyway!

[Carl Eugen Hoyos]

Just out of curiosity: How did you compile HandBrake?
What’s wrong with ./configure --enable-libaom && make ffmpeg?

[Ewout]

GitHub can use CI (AppVeyor in this case) which did the trick. In general, I don't make things, I break them and then report where I did :).

[Ewout]

I did the following test:

1. Encoded a .png screenshot to .png with FFmpeg
2. Encoded this screenshot to a AV1 with -c:v libaom-av1 -cpu-used 0, once with -lossless 1 and once without.
3. Encoded these .mp4's back to .png's
4. Repeated step 2 and 3 once more.

This resulted in 5 .png's, the input and two output files from both the lossy and lossless.

• Input: 185.760 bytes
• Lossless Round 1: 171.312 bytes
• Lossless Round 2: 171.312 bytes
• Lossy Round 1: 263.532 bytes
• Lossy Round 2: 249.829 bytes

There were also 4 MP4's generated in the process:

• Lossless Round 1: 59.061 bytes
• Lossless Round 2: 59.061 bytes
• Lossy Round 1: 11.260 bytes
• Lossy Round 2: 11.064 bytes

Where the lossy files are losing data, the lossless files aren't. I also used ​diffchecker to check for difference between the input and output png's, the lossy showed a difference while the lossless didn't. Looks like it works great and can be merged.

All files and the commands used can be found here: ​https://mega.nz/#F!W24HjSjJ!Z266_XE4Ln4KTmP3ZOR4GA

Small note: Line 33 misses a comma at the end.

{ "lossless",         "Lossless mode",                         OFFSET(lossless),       AV_OPT_TYPE_BOOL, {.i64 = 0},  0, 1, VE}

Thanks to ​u/MrSmilingWolf for the Windows build and spotting the missing comma.

[Ewout]

cehoyos are you ready to submit the patch or do you want/need any more tests done?

[Carl Eugen Hoyos]

Please test with unpatched FFmpeg if -b:v 0 -crf 0 also produces lossless output.

[Carl Eugen Hoyos]

I can confirm that lossless encoding works fine with -crf 0.

[Balling]

Replying to cehoyos:

I can confirm that lossless encoding works fine with -crf 0.

You are wrong. It is not lossless.

While latest aoemenc from here is lossless even for 4:2:0: ​https://ci.appveyor.com/project/marcomsousa/build-aom/builds/38315239/artifacts did get inf SSIM and PSNR. I hope you can fix this ASAP.

[Parsed_ssim_0 @ 00000123da0a7280] SSIM Y:1.000000 (inf) U:1.000000 (inf) V:1.000000 (inf) All:1.000000 (inf)
[Parsed_psnr_1 @ 00000123da0a8140] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

Maybe you tested 4:4:4? Or something. Or, you know, SSIM was broken! See: #7825

I used aomenc.exe --lossless=1 -o output.webm source1.y4m

It will/can create VFR file, so be accurate and get back to y4m. It can also add some DTS warnings!

[null @ 000001a3eab6b7c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 89 >= 89
[null @ 000001a3eab6b7c0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 89 >= 89

And you MUST be very careful, because for high bitrate AV1 can be lossless in BY ITSELF!

​https://www.reddit.com/r/AV1/comments/aylb98/aomenc_near_lossless_encoding/

Also aomenc was not lossless some time ago, that also should be checked. ​https://www.texpion.com/2018/07/av1-vs-vp9-vs-avc-h264-vs-hevc-h265-1-lossless.html

[Carl Eugen Hoyos]

This change request was about an option that (completely incomprehensible) exists for libvpx but should not exist for libaom, it was therefore invalid. There may or may not be bugs related to libaom encoding but this ticket is not the right place to discuss them, and the fix for unset qmax is not to add an additional option.
Sorry to add that you have a tendency to add comments to tickets that need no additional comments.

[Balling]

Replying to cehoyos:

Sorry to add that you have a tendency to add comments to tickets that need no additional comments.

AV1 is not lossless in ffmpeg.

should not exist for libaom,

And why is that? It also exists in HEVC, because that is how x265 coder was implemented. It is the same in aomenc, it has this option. H264 does not have this option, because they used two different lossless options.

[Balling]

Oh, it looks like I found a workaround/ real approach! Nice.
-aom-params lossless=1 as said in help!

https://ffmpeg.org/ffmpeg-codecs.html

You could have just said that. No need to be so angry. I wanna add lossless encoding to our AV1 wiki. That is all.

[Carl Eugen Hoyos]

Replying to Balling:

AV1 is not lossless in ffmpeg.

Encoding with libaom via FFmpeg is not lossless by default and this is intended. To make a lossless AV1 encode with libaom using the ffmpeg binary, neither a new encoder option nor a private library option are needed.

[Balling]

Replying to cehoyos:

To make a lossless AV1 encode with libaom using the ffmpeg binary, neither a new encoder option nor a private library option are needed.

Well, -crf 0 or even -b:v 0 -crf 0 do not work.

private library option

Then why does it exist?

Encoding with libaom via FFmpeg is not lossless by default

Oh, do you think I did not know that? Seriously? Do you also think I do not know that by default hevc is not lossless? Or VP9? Or AVC? Wow, just wow.

[Balling]

So

-b:v 0 -crf 0 -aom-params lossless=1

does not work too. p.s. was wrong, it does work.

It should print AV1E_SET_LOSSLESS in aom_codec_control (with -v debug), but it does not.
See: ​https://aomedia.googlesource.com/aom/+/master/apps/aomenc.c#142

[Balling]

Seriously?? So I was right! ​https://patchwork.ffmpeg.org/project/ffmpeg/patch/CAB0OVGrR6T77qr2T1yeqrKqk6DU4Tp9ndxxxTwyp2e2y8dQmbA@mail.gmail.com/

Again, please apply normal patch, not these hacks "saw the earlier comments around using crf=0 as the trigger for this"! Internally, google API thinks there should be AV1E_SET_LOSSLESS!

​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200408001341.65204-2-ryo.hirafuji@gmail.com/#55201

He says that it is not new that lossless is broken!

https://ffmpeg.org/pipermail/ffmpeg-user/2020-June/048885.html

That is indeed blocking, VERY blocking for AVIF!

[Balling]

I still do not understand how AV1E_SET_LOSSLESS is being set, even with qmax fix...

[Balling]

Okay, so you can discard all above, my problem is only in the first frame! Just like here: https://ffmpeg.org/pipermail/ffmpeg-user/2020-June/048885.html

[pdr0]

That issue is fixed when using -c:v libaom-av1 -aom-params lossless=1

I participated in that thread and revisited it just now

I also tested a few different samples (patterns, and real), 8bit and 10bit, all more than 8frames, and all were lossless with the -aom-params lossless=1 switch. Confirmed "lossless" using ffmpeg psnr, and using methods outside of ffmpeg

Do you have a test source that is "not lossless" with -aom-params lossless=1 (1st frame or not) ?

[Balling]

Replying to pdr0:

That issue is fixed when using -c:v libaom-av1 -aom-params lossless=1

I participated in that thread and revisited it just now

I also tested a few different samples (patterns, and real), 8bit and 10bit, all more than 8frames, and all were lossless with the -aom-params lossless=1 switch. Confirmed "lossless" using ffmpeg psnr, and using methods outside of ffmpeg

Do you have a test source that is "not lossless" with -aom-params lossless=1 (1st frame or not) ?

Yes, -aom-params lossless=1 works for me after all. Thanks to

-framemd5 -

as it is much more effective here...

How can you be sure that lossless=1 actually GOT applied? -v debug does not show any difference, as there is no AV1E_SET_LOSSLESS! Also, it will force -cfr 32, the default value (but is still lossless)...

And of course no ENCODER_SETTINGS metadata. :(

[pdr0]

Replying to Balling:

Replying to pdr0:

That issue is fixed when using -c:v libaom-av1 -aom-params lossless=1

I participated in that thread and revisited it just now

I also tested a few different samples (patterns, and real), 8bit and 10bit, all more than 8frames, and all were lossless with the -aom-params lossless=1 switch. Confirmed "lossless" using ffmpeg psnr, and using methods outside of ffmpeg

Do you have a test source that is "not lossless" with -aom-params lossless=1 (1st frame or not) ?

Yes, -aom-params lossless=1 works for me after all. Thanks to

-framemd5 -

as it is much more effective here...

How can you be sure that lossless=1 actually GOT applied? -v debug does not show any difference, as there is no AV1E_SET_LOSSLESS! Also, it will force -cfr 32, the default value (but is still lossless)...

And of course no ENCODER_SETTINGS metadata. :(

ffmpeg report does not say lossless switch applied (only read)... but aomenc.exe does not display that info either with --verbose, nor does aomenc.exe output stream have metadata (yet). If/when it gets implemented in aomenc, then you should see it in ffmpeg libaom-av1

[Balling]

BTW, I checked 288ca1c66740640d5742f2921efbf502d83f8b6a, it indeed fixes the bug. Nice.

[Balling]

Will again verify that either -crf 0 and -aom-params lossless=1 give lossless for added AVIF!! MD5 for avif is the same.