Opened 4 years ago
Last modified 18 months ago
#9243 new defect
rising memory consumption when -decryption_key is used
| Reported by: | Ammar | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | avformat |
| Version: | git-master | Keywords: | mov |
| Cc: | Ammar, fireattack | Blocked By: | |
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description (last modified by )
I am trying to translate a CENC-encrypted live DASH stream to an HLS one, by piping dash segments one by one into ffmpeg. The result is perfect, but memory usage of the long-running process rises steadily forever.
Command line:
% ffmpeg -f mp4 -decryption_key 9ab6fa563cab6ba9cb30f18fd57f1f3b -i /tmp/videoPipe -f mp4 -decryption_key 9ab6fa563cab6ba9cb30f18fd57f1f3b -i /tmp/audioPipe -y -c copy -f hls /var/www/html/index.m3u8
fmpeg version N-57204-gc7c138e411-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2021 the FFmpeg developers built with gcc 8 (Debian 8.3.0-6) configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg libavutil 57. 0.100 / 57. 0.100 libavcodec 59. 0.100 / 59. 0.100 libavformat 59. 0.100 / 59. 0.100 libavdevice 59. 0.100 / 59. 0.100 libavfilter 8. 0.101 / 8. 0.101 libswscale 6. 0.100 / 6. 0.100 libswresample 4. 0.100 / 4. 0.100 libpostproc 56. 0.100 / 56. 0.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/videoPipe': Metadata: major_brand : isom minor_version : 1 compatible_brands: isomavc1dash creation_time : 2021-05-14T23:49:00.000000Z Duration: 32405:58:20.16, start: 116661492.240000, bitrate: N/A Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt470bg), 640x360 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn (default) Metadata: creation_time : 2021-05-14T23:49:00.000000Z handler_name : ETI ISO Video Media Handler vendor_id : [0][0][0][0] encoder : Elemental H.264 Side data: unknown side data type 24 (67 bytes) Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/audioPipe': Metadata: major_brand : isom minor_version : 1 compatible_brands: isomavc1dash creation_time : 2021-05-14T21:51:25.000000Z Duration: 32405:58:20.21, start: 116661492.187021, bitrate: N/A Stream #1:0(und): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp (default) Metadata: creation_time : 2021-05-14T21:51:25.000000Z handler_name : ETI ISO Audio Media Handler vendor_id : [0][0][0][0] Side data: unknown side data type 24 (67 bytes) Output #0, hls, to '/var/www/html/index.m3u8': Metadata: major_brand : isom minor_version : 1 compatible_brands: isomavc1dash encoder : Lavf59.0.100 Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt470bg), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbr, 90k tbn (default) Metadata: creation_time : 2021-05-14T23:49:00.000000Z handler_name : ETI ISO Video Media Handler vendor_id : [0][0][0][0] encoder : Elemental H.264
Side data:
unknown side data type 24 (67 bytes)
Stream #0:1(und): Audio: aac (HE-AAC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp (default)
Metadata:
creation_time : 2021-05-14T21:51:25.000000Z
handler_name : ETI ISO Audio Media Handler
vendor_id : [0][0][0][0]
Side data:
unknown side data type 24 (67 bytes)
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #1:0 -> #0:1 (copy)
frame= 0 fps=0.0 q=-1.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 16 fps=0.0 q=-1.0 size=N/A time=00:00:00.56 bitrate=N/A speed= 1.1x
frame= 29 fps= 28 q=-1.0 size=N/A time=00:00:01.08 bitrate=N/A speed=1.06x
frame= 42 fps= 27 q=-1.0 size=N/A time=00:00:01.60 bitrate=N/A speed=1.05x
frame= 55 fps= 27 q=-1.0 size=N/A time=00:00:02.12 bitrate=N/A speed=1.04x
frame= 68 fps= 27 q=-1.0 size=N/A time=00:00:02.64 bitrate=N/A speed=1.04x
frame= 80 fps= 26 q=-1.0 size=N/A time=00:00:03.12 bitrate=N/A speed=1.02x
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x65976c0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 1024)
frame= 93 fps= 26 q=-1.0 size=N/A time=00:00:03.64 bitrate=N/A speed=1.02x
frame= 106 fps= 26 q=-1.0 size=N/A time=00:00:04.16 bitrate=N/A speed=1.02x
frame= 119 fps= 26 q=-1.0 size=N/A time=00:00:04.68 bitrate=N/A speed=1.02x
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x654a2c0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 1024)
frame= 132 fps= 26 q=-1.0 size=N/A time=00:00:05.20 bitrate=N/A speed=1.02x
frame= 144 fps= 26 q=-1.0 size=N/A time=00:00:05.68 bitrate=N/A speed=1.01x
[hls @ 0x67c4f00] Opening '/var/www/html/segment-1.ts.tmp' for writing
[hls @ 0x67c4f00] Opening '/var/www/html/index.m3u8.tmp' for writing
frame= 157 fps= 26 q=-1.0 size=N/A time=00:00:06.20 bitrate=N/A speed=1.01x
frame= 170 fps= 26 q=-1.0 size=N/A time=00:00:06.72 bitrate=N/A speed=1.02x
frame= 182 fps= 26 q=-1.0 size=N/A time=00:00:07.20 bitrate=N/A speed=1.01x
frame= 194 fps= 25 q=-1.0 size=N/A time=00:00:07.68 bitrate=N/A speed=1.01x
[hls @ 0x67c4f00] Non-monotonous DTS in output stream 0:0; previous: 709200, current: -7200; changing to 709201. This may result in incorrect timestamps in the output file.
[hls @ 0x67c4f00] Non-monotonous DTS in output stream 0:0; previous: 709201, current: -3600; changing to 709202. This may result in incorrect timestamps in the output file.
frame= 407 fps= 50 q=-1.0 size=N/A time=00:00:08.20 bitrate=N/A speed=1.01x
frame= 420 fps= 49 q=-1.0 size=N/A time=00:00:08.72 bitrate=N/A speed=1.01x
....
Input files are FIFO pipes into which I push the init.mp4 file first, and then sequentially DASH fMP4 segments as they become available, via an external script.
Restarting the process every once in a while works, but I'd rather actually fix the problem if possible.
Change History (8)
comment:1 by , 4 years ago
| Description: | modified (diff) |
|---|
comment:2 by , 4 years ago
| Component: | ffmpeg → undetermined |
|---|---|
| Keywords: | cenc Widevine HLS DASH removed |
comment:3 by , 4 years ago
| Description: | modified (diff) |
|---|
comment:4 by , 4 years ago
| Description: | modified (diff) |
|---|
The issue is in the demuxing of encrypted mp4. The fmp4 files are coming from a DASH stream, but I'm concating them and piping in, with -f mp4. The result is basically one very long mp4 file.
A minimal demonstration appears to be:
ffmpeg -decryption_key 012121249abc0cfd0 -re -i verylonginputfile_encrypted.mp4 -c copy output.ts
The -re is just to allow observation over time, but has no effect on the outcome.
I'll provide sample files asap
follow-up: 6 comment:5 by , 3 years ago
This is a duplicate of #7641. The cause of this is extremely wasteful buffer handling when it comes to some of the CENC-related atoms. Unfortunately I still didn't find the time to submit a patch.
comment:6 by , 3 years ago
| Component: | undetermined → avformat |
|---|---|
| Keywords: | mov added |
Replying to Starchild:
This is a duplicate of #7641. The cause of this is extremely wasteful buffer handling when it comes to some of the CENC-related atoms. Unfortunately I still didn't find the time to submit a patch.
Given that #7641 has no similarities with a valid ticket, this one should not be closed as a duplicate.
comment:7 by , 2 years ago
| Cc: | added |
|---|
Note:
See TracTickets
for help on using tickets.
Please test if the issue is due to the demuxing of dash or the muxing of his.