FFPLAY: memory consumption grows while decoding h264 when loss of input packets present

[kiwi]

Summary of the bug:
The memory consumption grows while decoding h264,
especially strong when
loss of input packets / corrupted packet sequence present.

How to reproduce:
I stream video to my application that uses ffplay for video playback.
When I insert some "spoiling" of input stream (stream
with packet loss and corrupted packets)
memory consumption significantly increases from 270 MB
up to 2CG.

ffplay udp://239.255.124.71:3000

ffmpeg version: FFmpeg 3.4 "Cantor"

ffplay output:
ffplay version 3.4 Copyright (c) 2003-2017 the FFmpeg developers

built with gcc 7.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-lib

x264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx

libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100

[h264 @ 0000000000585600] non-existing PPS 0 referenced 0B f=0/0
[AVBSFContext @ 00000000005fff60] Invalid NAL unit 0, skipping.
[h264 @ 0000000000585600] non-existing PPS 0 referenced
[h264 @ 0000000000585600] decode_slice_header error
[h264 @ 0000000000585600] no frame!
[h264 @ 0000000000585600] non-existing PPS 0 referenced 0B f=0/0
[AVBSFContext @ 00000000005fff60] Invalid NAL unit 0, skipping.
[h264 @ 0000000000585600] non-existing PPS 0 referenced
[h264 @ 0000000000585600] decode_slice_header error
[h264 @ 0000000000585600] no frame!
[h264 @ 0000000000585600] non-existing PPS 0 referenced 0B f=0/0
[AVBSFContext @ 00000000005fff60] Invalid NAL unit 0, skipping.
[h264 @ 0000000000585600] non-existing PPS 0 referenced
[h264 @ 0000000000585600] decode_slice_header error
[h264 @ 0000000000585600] no frame!
[h264 @ 0000000000585600] non-existing PPS 0 referenced 0B f=0/0
[AVBSFContext @ 00000000005fff60] Invalid NAL unit 0, skipping.
[h264 @ 0000000000585600] non-existing PPS 0 referenced
[h264 @ 0000000000585600] decode_slice_header error
[h264 @ 0000000000585600] no frame!
[AVBSFContext @ 00000000005fff60] Invalid NAL unit 0, skipping.
Input #0, mpegts, from 'udp://239.255.124.71:3000':q= 0B f=0/0

Duration: N/A, start: 1293.373333, bitrate: N/A
Program 1

Stream #0:0[0x4d9]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc

[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0

Last message repeated 1 times

[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[h264 @ 0000000003fe2ae0] error while decoding MB 4 0, bytestream -32
[h264 @ 000000000583e140] [h264 @ 0000000003fe2ae0] error while decoding MB 21 0, bytestream -8
concealing 8160 DC, 8160 AC, 8160 MV errors in I frame
[h264 @ 000000000583e140] concealing 8160 DC, 8160 AC, 8160 MV errors in I frame
[mpegts @ 000000000056fe60] PES packet size mismatch
[h264 @ 00000000058e1d80] error while decoding MB 97 4, bytestream -9
[h264 @ 00000000058e1d80] concealing 7632 DC, 7632 AC, 7632 MV errors in P frame
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[h264 @ 0000000004490020] error while decoding MB 95 6, bytestream -6
[h264 @ 0000000004490020] concealing 7394 DC, 7394 AC, 7394 MV errors in P frame
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[h264 @ 0000000004189020] error while decoding MB 37 4, bytestream -8
[h264 @ 0000000004189020] concealing 7692 DC, 7692 AC, 7692 MV errors in P frame
[h264 @ 000000000588a520] [h264 @ 000000000588a060] Reference 2 >= 2
concealing 7126 DC, 7126 AC, 7126 MV errors in P frame
[h264 @ 000000000588a520] error while decoding MB 117 2, bytestream 401
[h264 @ 000000000588a520] concealing 7852 DC, 7852 AC, 7852 MV errors in P frame
[h264 @ 000000000588aa00] error while decoding MB 115 4, bytestream -6
[h264 @ 000000000588aa00] concealing 7614 DC, 7614 AC, 7614 MV errors in P frame
[h264 @ 0000000005828fe0] error while decoding MB 113 2, bytestream -18
[h264 @ 0000000005828fe0] concealing 7856 DC, 7856 AC, 7856 MV errors in P frame
[h264 @ 0000000003fe2ae0] [h264 @ 000000000583e140] cabac decode of qscale diff failed at 9 11
error while decoding MB 46 4, bytestream -8
[h264 @ 0000000003fe2ae0] [h264 @ 000000000583e140] error while decoding MB 9 10, bytestream 104
concealing 7683 DC, 7683 AC, 7683 MV errors in P frame
[h264 @ 0000000003fe2ae0] concealing 7000 DC, 7000 AC, 7000 MV errors in P frame
[h264 @ 00000000058e1d80] error while decoding MB 75 6, bytestream -6
[h264 @ 00000000058e1d80] concealing 7414 DC, 7414 AC, 7414 MV errors in P frame
[h264 @ 0000000004490020] left block unavailable for requested intra mode
[h264 @ 0000000004490020] error while decoding MB 0 4, bytestream 425
[h264 @ 0000000004490020] concealing 7729 DC, 7729 AC, 7729 MV errors in P frame
[h264 @ 0000000004189020] error while decoding MB 78 4, bytestream -7
[h264 @ 0000000004189020] concealing 7651 DC, 7651 AC, 7651 MV errors in P frame
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=0/0
[h264 @ 000000000588a060] concealing 7813 DC, 7813 AC, 7813 MV errors in P frame
[h264 @ 000000000588a520] concealing 7142 DC, 7142 AC, 7142 MV errors in P frame
[h264 @ 000000000588aa00] error while decoding MB 7 0, bytestream -16
[h264 @ 000000000588aa00] concealing 8160 DC, 8160 AC, 8160 MV errors in I frame
[mpegts @ 000000000056fe60] DTS 120516600 < 568803600 out of order
[mpegts @ 000000000056fe60] PES packet size mismatch= 0B f=1/1

[kiwi]

To reproduce memory consumption I have three PCAP files. Each file with size about 10 MB

[Carl Eugen Hoyos]

Please test current FFmpeg git head and please provide the pcap files if they can be used to reproduce the issue.

[Carl Eugen Hoyos]

Is the issue only reproducible with FFplay, not with FFmpeg?

[kiwi]

Replying to cehoyos:

Is the issue only reproducible with FFplay, not with FFmpeg?

This issue reproducible with FFplay and with FFmpeg too.

[kiwi]

Replying to cehoyos:

Please test current FFmpeg git head and please provide the pcap files if they can be used to reproduce the issue.

I have PCAP files bigger than 2.5 MB to reproduce the issue. How can I send these files to FFMPEG development team?

[Carl Eugen Hoyos]

You can use dropbox, datafilehost, ​https://streams.videolan.org/upload/ or another filehoster.

Please also explain what I have to do to reproduce.

[kiwi]

Here three PCAP files to reproduce the issue.
Links to files
File # 1 ​https://www.dropbox.com/s/c6fe69ggdjsu1ul/PacketStream1.pcap?dl=0
File # 2 ​https://www.dropbox.com/s/vgvl8014dxa94fs/BadPacketStream.pcap?dl=0
File # 3 ​https://www.dropbox.com/s/vq1emz9k17m5iqw/PacketStream2.pcap?dl=0

Steps to reproduce memory consumption issue

1. Open the packet player (for example Colasoft packet player),and add files in this order:

file #1 (PacketStream1.pcap),
file #2 (BadPacketStream.pcap), - packet stream with packets loss and corrupted packets
file #3 (PacketStream2pcap)

Note: the order of files very important. If you play PCAP files in other order the issue will not appear.

2. Run ffplay to play packets that received via UDP

for example ffplay udp://239.255.124.71:3000

3. Press on play button in the Colasoft packet player.

4 memory consumption will grow up to 2G during PacketStream2pcap

[Kieran Kunhya]

Thanks, I've seen this here too but have not been able to reproduce as easily as you. We are looking into this.

[Kieran Kunhya]

Reproduced locally, bisecting now.

[Kieran Kunhya]

382a68b0088b06b8df20d0133d767d53d8f161ef is the first bad commit

[Kieran Kunhya]

Actually 382a68 is only part of the problem, we think underlying issue is:
772ad7142dff590c7bb30370acf5d3c9a8fb512b

[Kieran Kunhya]

Fixed in 03b82b3ab9883cef017e513c7d0b3b986b3b3e7b