#8753 closed defect (wontfix)

AV1 stream metadata missing level and tier

Reported by: gordon Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I've compiled ffmpeg/probe from snapshot.

The data output by ffprobe doesn't seem to indicate the level and tier for AV1 streams.

This information is needed to generate the codec param required for browsers.

https://aomediacodec.github.io/av1-isobmff/#codecsparam defines the codec representation as:
<sample entry 4CC>.<profile>.<level><tier>.<bitDepth>.<monochrome>.<chromaSubsampling>.<colorPrimaries>.<transferCharacteristics>.<matrixCoefficients>.<videoFullRangeFlag>

The values after bitDepth are optional. However, level and tier are required.

The sample entry is 'av01' constant.
The profile is provided in ffprobe output (i.e. Main == 0)
The bitdepth can be determined from 'pix_fmt' in ffprobe output

However, the level and tier do not seem to be computable from the information provided.

I get this metadata from ffprobe:

{
    "format": {
        "bit_rate": "151953",
        "duration": "29.833333",
        "filename": "/tmp/6485a47accb348c6a59faf2522f1b5e1/output/948900038b8746b9936e96ccf35e2f72.mp4",
        "format_long_name": "QuickTime / MOV",
        "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
        "nb_programs": 0,
        "nb_streams": 1,
        "probe_score": 100,
        "size": "566660",
        "start_time": "0.000000",
        "tags": {
            "compatible_brands": "iso5iso6mp41",
            "encoder": "Lavf58.46.101",
            "major_brand": "iso5",
            "minor_version": "512"
        }
    },
    "streams": [
        {
            "avg_frame_rate": "12/1",
            "bit_rate": "121914",
            "closed_captions": 0,
            "codec_long_name": "Alliance for Open Media AV1",
            "codec_name": "av1",
            "codec_tag": "0x31307661",
            "codec_tag_string": "av01",
            "codec_time_base": "1/12",
            "codec_type": "video",
            "coded_height": 120,
            "coded_width": 160,
            "color_range": "tv",
            "disposition": {
                "attached_pic": 0,
                "clean_effects": 0,
                "comment": 0,
                "default": 1,
                "dub": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "karaoke": 0,
                "lyrics": 0,
                "original": 0,
                "timed_thumbnails": 0,
                "visual_impaired": 0
            },
            "duration": "29.833333",
            "duration_ts": 366592,
            "has_b_frames": 0,
            "height": 120,
            "index": 0,
            "level": -99,
            "pix_fmt": "yuv420p",
            "profile": "Main",
            "r_frame_rate": "12/1",
            "refs": 1,
            "start_pts": 0,
            "start_time": "0.000000",
            "tags": {
                "handler_name": "VideoHandler",
                "language": "und"
            },
            "time_base": "1/12288",
            "width": 160
        }
    ]
}

Some additional references:
https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs
https://aomediacodec.github.io/av1-spec/#annex-a-profiles-and-levels

Attachments (1)

issue-8753.mp4 (414.2 KB ) - added by Carl Eugen Hoyos 15 months ago.

Download all attachments as: .zip

Change History (14)

comment:1 by James, 16 months ago

What version of ffprobe is this, and what options did you use to configure ffmpeg?
Does it include the external libdav1d decoder or not?

in reply to:  1 comment:2 by gordon, 16 months ago

Replying to jamrial:

What version of ffprobe is this, and what options did you use to configure ffmpeg?
Does it include the external libdav1d decoder or not?

It is build against libaom

vagrant@vagrant:~$ ffprobe -version
ffprobe version N-98246-g157feea Copyright (c) 2007-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/home/vagrant/ffmpeg/build --libdir=/home/vagrant/ffmpeg/build/lib --extra-cflags=-I/home/vagrant/ffmpeg/build/include --extra-ldflags=-L/home/vagrant/ffmpeg/build/lib --bindir=/home/vagrant/bin --disable-doc --disable-ffplay --disable-static --enable-shared --enable-openssl --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libaom --enable-nonfree
libavutil 56. 55.100 / 56. 55.100
libavcodec 58. 92.100 / 58. 92.100
libavformat 58. 46.101 / 58. 46.101
libavdevice 58. 11.100 / 58. 11.100
libavfilter 7. 86.100 / 7. 86.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100
vagrant@vagrant:~$

comment:3 by James, 16 months ago

Can't reproduce this. With libaom as the only decoder available i still get level to show the value stored in the AV1 bitstream instead of the default -99 (Which is how libavcodec signals undefined/unknown).

Can you try with the latest git head commit? And also upload a sample that reproduces this somewhere.

in reply to:  3 comment:4 by gordon, 16 months ago

Replying to jamrial:

Can't reproduce this. With libaom as the only decoder available i still get level to show the value stored in the AV1 bitstream instead of the default -99 (Which is how libavcodec signals undefined/unknown).

Can you try with the latest git head commit? And also upload a sample that reproduces this somewhere.

That is good news. Do you also see the "tier" listed?

comment:5 by James, 16 months ago

Tier is a concept in AV1 that has no direct translation in any of the generic fields libavcodec or libavformat can export, so it will not be show in the output of ffprobe.
If what you want is AV1 specific bitstream information, you could try the trace_headers bitstream filter. It will print the sequence header and frame header bitstream values in an easy to parse form.

And again, did you try with git head to see if level contains the right value?

in reply to:  5 comment:6 by gordon, 16 months ago

Replying to jamrial:

Tier is a concept in AV1 that has no direct translation in any of the generic fields libavcodec or libavformat can export, so it will not be show in the output of ffprobe.
If what you want is AV1 specific bitstream information, you could try the trace_headers bitstream filter. It will print the sequence header and frame header bitstream values in an easy to parse form.

And again, did you try with git head to see if level contains the right value?

Great thanks. I am planning to and will report back soon

comment:7 by gordon, 15 months ago

I am still observing the same issue with the level being reported as -99 using snapshot (ffmpeg version N-98541-g24911b9)

I've tried with a few different files. The files are generated using the same version of ffmpeg

comment:8 by gordon, 15 months ago

I am also not observing the tier info in the trace_headers output.

/usr/local/bin/ffmpeg -i /tmp/c6527c7674cc47cca697c1ea9d043676/output/bdec6d7e45b74078ae561a53f449d65c.mp4 -c:v copy -bsf:v trace_headers -f null /dev/null

Example of output - I've only output the first one because it is so verbose:

[trace_headers @ 0x563884b33a40] Extradata
[trace_headers @ 0x563884b33a40] OBU header
[trace_headers @ 0x563884b33a40] 0           obu_forbidden_bit                                           0 = 0
[trace_headers @ 0x563884b33a40] 1           obu_type                                                 0001 = 1
[trace_headers @ 0x563884b33a40] 5           obu_extension_flag                                          0 = 0
[trace_headers @ 0x563884b33a40] 6           obu_has_size_field                                          1 = 1
[trace_headers @ 0x563884b33a40] 7           obu_reserved_1bit                                           0 = 0
[trace_headers @ 0x563884b33a40] 8           leb128_byte[0]                                       00001011 = 11
[trace_headers @ 0x563884b33a40] 8           obu_size                                                      = 11
[trace_headers @ 0x563884b33a40] Sequence Header
[trace_headers @ 0x563884b33a40] 16          seq_profile                                               000 = 0
[trace_headers @ 0x563884b33a40] 19          still_picture                                               0 = 0
[trace_headers @ 0x563884b33a40] 20          reduced_still_picture_header                                0 = 0
[trace_headers @ 0x563884b33a40] 21          timing_info_present_flag                                    0 = 0
[trace_headers @ 0x563884b33a40] 22          initial_display_delay_present_flag                          0 = 0
[trace_headers @ 0x563884b33a40] 23          operating_points_cnt_minus_1                            00000 = 0
[trace_headers @ 0x563884b33a40] 28          operating_point_idc[0]                           000000000000 = 0
[trace_headers @ 0x563884b33a40] 40          seq_level_idx[0]                                        00001 = 1
[trace_headers @ 0x563884b33a40] 45          frame_width_bits_minus_1                                 1001 = 9
[trace_headers @ 0x563884b33a40] 49          frame_height_bits_minus_1                                1000 = 8
[trace_headers @ 0x563884b33a40] 53          max_frame_width_minus_1                            1001111111 = 639
[trace_headers @ 0x563884b33a40] 63          max_frame_height_minus_1                            101100111 = 359
[trace_headers @ 0x563884b33a40] 72          frame_id_numbers_present_flag                               0 = 0
[trace_headers @ 0x563884b33a40] 73          use_128x128_superblock                                      0 = 0
[trace_headers @ 0x563884b33a40] 74          enable_filter_intra                                         1 = 1
[trace_headers @ 0x563884b33a40] 75          enable_intra_edge_filter                                    1 = 1
[trace_headers @ 0x563884b33a40] 76          enable_interintra_compound                                  0 = 0
[trace_headers @ 0x563884b33a40] 77          enable_masked_compound                                      1 = 1
[trace_headers @ 0x563884b33a40] 78          enable_warped_motion                                        1 = 1
[trace_headers @ 0x563884b33a40] 79          enable_dual_filter                                          0 = 0
[trace_headers @ 0x563884b33a40] 80          enable_order_hint                                           1 = 1
[trace_headers @ 0x563884b33a40] 81          enable_jnt_comp                                             1 = 1
[trace_headers @ 0x563884b33a40] 82          enable_ref_frame_mvs                                        1 = 1
[trace_headers @ 0x563884b33a40] 83          seq_choose_screen_content_tools                             1 = 1
[trace_headers @ 0x563884b33a40] 84          seq_choose_integer_mv                                       1 = 1
[trace_headers @ 0x563884b33a40] 85          order_hint_bits_minus_1                                   110 = 6
[trace_headers @ 0x563884b33a40] 88          enable_superres                                             0 = 0
[trace_headers @ 0x563884b33a40] 89          enable_cdef                                                 1 = 1
[trace_headers @ 0x563884b33a40] 90          enable_restoration                                          0 = 0
[trace_headers @ 0x563884b33a40] 91          high_bitdepth                                               0 = 0
[trace_headers @ 0x563884b33a40] 92          mono_chrome                                                 0 = 0
[trace_headers @ 0x563884b33a40] 93          color_description_present_flag                              0 = 0
[trace_headers @ 0x563884b33a40] 94          color_range                                                 0 = 0
[trace_headers @ 0x563884b33a40] 95          chroma_sample_position                                     00 = 0
[trace_headers @ 0x563884b33a40] 97          separate_uv_delta_q                                         0 = 0
[trace_headers @ 0x563884b33a40] 98          film_grain_params_present                                   0 = 0
[trace_headers @ 0x563884b33a40] 99          trailing_one_bit                                            1 = 1
[trace_headers @ 0x563884b33a40] 100         trailing_zero_bit                                           0 = 0
[trace_headers @ 0x563884b33a40] 101         trailing_zero_bit                                           0 = 0
[trace_headers @ 0x563884b33a40] 102         trailing_zero_bit                                           0 = 0
[trace_headers @ 0x563884b33a40] 103         trailing_zero_bit                                           0 = 0

comment:9 by gordon, 15 months ago

I was expecting to see a field named seq_tier per https://aomediacodec.github.io/av1-isobmff/#codecsparam

The level parameter value SHALL equal the first level value indicated by seq_level_idx in the Sequence Header OBU.

The tier parameter value SHALL be equal to M when the first seq_tier value in the Sequence Header OBU is equal to 0, and H when it is equal to 1.

comment:10 by gordon, 15 months ago

I have uploaded a demonstration file at https://github.com/thenewguy/FFmpeg/files/4973372/issue-8753.zip

Note that the file is created by FFMPEG so one would expect it to be parsed correctly by FFMPEG as well

Also available here: https://github.com/thenewguy/FFmpeg/issues/1

Last edited 15 months ago by gordon (previous) (diff)

in reply to:  3 comment:11 by gordon, 15 months ago

Replying to jamrial:

Can't reproduce this. With libaom as the only decoder available i still get level to show the value stored in the AV1 bitstream instead of the default -99 (Which is how libavcodec signals undefined/unknown).

Can you try with the latest git head commit? And also upload a sample that reproduces this somewhere.

I have posted a sample file. Are you able to take a look?

by Carl Eugen Hoyos, 15 months ago

Attachment: issue-8753.mp4 added

comment:12 by gordon, 14 months ago

Is there any more information I can provide here?

comment:13 by gordon, 10 months ago

Resolution: wontfix
Status: newclosed

going to close this since ffprobe cannot include the expected data per https://trac.ffmpeg.org/ticket/8753#comment:5

I've still never seen the correct level reported by ffprobe data with latest release 4.3.1 or master, however av1 tier data can be omitted from the header in certain cases, so it would be expected not to show up in the trace_headers output all the time

Note: See TracTickets for help on using tickets.