RTMPProto Overflow Interger (bytes_read) Bug

[Raul Barajas]

Summary of the bug:
There is a bug in rtmpproto.c inside get_packet. If the bytes_read warps because the value is greater than 4,294,967,295 then the ack will never be send again to the FMS server resulting in the FMS server closing the connection. If the rt->last_bytes_read = rt->bytes_read; was outside the if statement then it would address the bug.

How to reproduce:
In order to reproduce you need to bytes_read to exceed the value of 4,294,967,295 by listening to a stream. The higher the bandwidth the stream the faster the bug will occur.

I would have submitted a patch except I have never done that, this actually my first time submitting a bug ticket. The reason I'm even submitting something is because after working with an Adobe engineer, I agreed to submit a bug report.

[Carl Eugen Hoyos]

Please either test attached patch or explain how the issue can be reproduced.

[Raul Barajas]

We tested the attached patch and the issue still occurred. We contacted Adobe again and after a lot of testing and tracing we figured out that FMS requires the reset to happen at or close to 4,000,000,000. The fix we tried and confirmed it works as expected is the following.

if (rt->bytes_read - rt->last_bytes_read > rt->client_report_size) {
            av_log(s, AV_LOG_DEBUG, "Sending bytes read report\n");
            if ((ret = gen_bytes_read(s, rt, rpkt.timestamp + 1)) < 0)
                return ret;
            if (rt->bytes_read > 4000000000)
                rt->bytes_read = 0;
            rt->last_bytes_read = rt->bytes_read;
        }

[Carl Eugen Hoyos]

Please provide your tested patch as a unified diff (for example as made by git format-patch) to get this issue fixed.

[FCYannick]

There you are! ;-)

This also solved my issue of not being able to stream a RTMP input more than 4 GB in a row.