AVSubtitle structure has rectangles outside screen

[Peter Krefting]

Summary of the bug:

With this packet captured off-air, avcodec_decode_subtitle2() generates an AVSubtitle structure with rectangles that are to be rendered outside the 720x576 screen.

How to reproduce:

• open the file using avformat_open_input() and read a frame using av_read_frame()
• Allocate an AVCodecContext for AV_CODEC_ID_DVB_SUBTITLE and pass the AVPacket to avcodec_decode_subtitle2()

Expected result: All AVSubtitleRect structures in the decoded AVSubtitle structure should render inside the 720x576 screen.

Actual result: One of the generated rectangles renders outside the screen.

ffmpeg version 5.0.1 (9687cae2b468e09e35df4cea92cc2e6a0e6c93b3).

[Peter Krefting]

Sample for ticket 9995

[Carl Eugen Hoyos]

Please test current FFmpeg git head, the only version supported on this bug tracker, and please either provide the ffmpeg command line you tested together with the complete, uncut console output or provide working C code that allows us to reproduce the issue.

[Peter Krefting]

Boiling down the basics and linking with the latest master, I get this output:

$ ./ffmpeg-ticket-9995 ffmpeg-ticket-9995.ts 
Using N-108886-g79508ee523
Received subtitle frame with 2 rectangles
Using screen 720×576
Subtitle rectangle 0 is 720×40@0,556 with 16 colours
 - vertical overflow
Subtitle rectangle 1 is 720×40@0,516 with 16 colours

Source:

#include <iostream>

extern "C"{
#include <libavformat/avformat.h>
#include <libavcodec/avcodec.h>
}

int main(int argc, char *argv[])
{
    if (argc < 2) {
        std::cerr << "Usage: " << argv[0] << " filename" << std::endl;
        return 1;
    }

    std::cout << "Using " << av_version_info() << std::endl;

    AVFormatContext* avf_ctx = avformat_alloc_context();
    std::string url = "file:";
    url += argv[1];
    if (avformat_open_input(&avf_ctx, url.c_str(), nullptr, nullptr) != 0) {
        std::cerr << "Unable to open " << url << std::endl;
        return 1;
    }

    AVPacket avpkt = { 0 };

    av_read_frame(avf_ctx, &avpkt);
    AVStream* stream = avf_ctx->streams[avpkt.stream_index];
    AVCodecParameters* par = stream->codecpar;
    const AVCodec* codec = avcodec_find_decoder(par->codec_id);
    AVCodecContext* decoder = avcodec_alloc_context3(codec);
    avcodec_parameters_to_context(decoder, par);
    avcodec_open2(decoder, codec, nullptr);

    AVSubtitle sub{};
    int got = 0;
    avcodec_decode_subtitle2(decoder, &sub, &got, &avpkt);

    std::cout << "Received subtitle frame with " << sub.num_rects << " rectangles" << std::endl;

    // Find full dimensions
    // Use values from Display Definition Segment, if any,
    // otherwise 720 by 576 (ETSI EN 300 743 V1.61 §5.1.3)
    int screen_height = decoder->coded_height ? decoder->coded_height : 576;
    int screen_width  = decoder->coded_width  ? decoder->coded_width  : 720;

    std::cout << "Using screen " << screen_width << "×" << screen_height << std::endl;
    for (unsigned int i = 0; i < sub.num_rects; ++ i) {
        AVSubtitleRect* rect = sub.rects[i];
        std::cout << "Subtitle rectangle " << i << " is "
                  << sub.rects[i]->w << "×" << sub.rects[i]->h << "@"
                  << sub.rects[i]->x << "," << sub.rects[i]->y << " with "
                  << sub.rects[i]->nb_colors << " colours" << std::endl;
        if (sub.rects[i]->x + sub.rects[i]->w > screen_width) {
            std::cout << " - horizontal overflow" << std::endl;
        }
        if (sub.rects[i]->y + sub.rects[i]->h > screen_height) {
            std::cout << " - vertical overflow" << std::endl;
        }
    }
    avsubtitle_free(&sub);
    av_packet_unref(&avpkt);
    avformat_close_input(&avf_ctx);
    return 0;
}

[Cigaes]

As far as I can see, you get what is encoded in the subtitles bitstream. If it does not match the corresponding video, complain to whoever authored the stream. FFmpeg will not alter the contents as there are good reasons applications might want the whole contents even if it overflows.

[Peter Krefting]

I have now tested the clip with a third-party (commercial) analysis tool, and it agrees that the subtitles renders outside the screen, so this is garbage in, which needs to be guarded against on the receiving side.