Opened 5 years ago

Closed 4 years ago

#7894 closed defect (fixed)

SChannel TLS implementation bug

Reported by: Adion Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: schannel tls
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no


There's a bug in tls_schannel.c that prevents rtmps to work correctly when using this tls implementation. (It hangs in an infinite loop in tcp_read)
It is present in ffmpeg v4.1.3 and at this time is also still present in master.

The problem is in tls_read, when only a subset of the data returned by the server is requested.
First data is read from the server into c->enc_buf.
Next the data is decoded, but data is only encoded until enough data is decoded to fulfill the read request.
In the case of SECBUFFER_EXTRA, there may still be undecoded data in enc_buf when the loop stops and the data is returned.

On the next call to tls_read, there is not enough data in the decoded buffer, so it requests more data from the tcp layer, but this never comes as the data is already there, just not decoded yet.

I fixed it by decoding all the data before returning. Another way would be to check if there is still undecoded data in the encode data on the next request and decode it at that time.

I changed

while (c->enc_buf_offset > 0 && sspi_ret == SEC_E_OK && c->dec_buf_offset < len) {


while (c->enc_buf_offset > 0 && sspi_ret == SEC_E_OK) {

Related to this, I found it odd that by default there is no timeout specified, is this normal?

Change History (2)

comment:1 by Carl Eugen Hoyos, 5 years ago

Please send your patch - made with git format-patch - to the FFmpeg development mailing list.

comment:2 by Carl Eugen Hoyos, 4 years ago

Resolution: fixed
Status: newclosed

Fixed by Jan Ekström in 39977fff20048f1798a95c593d6034a0e73ebbe5

Note: See TracTickets for help on using tickets.