Opened 6 years ago

Closed 6 years ago

#6923 closed defect (duplicate)

transcode with ffmpeg may eat up a lot of memory

Reported by: JasonZhang Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
When I transcode this source file with ffmpeg,ffmpeg may eat up a lot of memory about dozens of GB.

the report here:
ffmpeg started on 2017-12-19 at 10:59:15
Report written to "ffmpeg-20171219-105915.log"
Command line:
ffmpeg -i memory_error_fifo.mp4 -s 854x480 -vcodec libx264 -x264opts "keyint=50" -b:v 238k -vf "movie=namei.jpg [watermark]; [vf0] fifo [vf1];[vf1][watermark] overlay=main_w-overlay_w-0:0[vf2]" -acodec libfdk_aac -ar 44100 -movflags faststart -f mp4 output.mp4 -y -v trace -report
ffmpeg version N-89095-gb3c1172 Copyright (c) 2000-2017 the FFmpeg developers

built with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-18)
configuration: --enable-gpl --enable-nonfree --enable-avresample --enable-libx264 --enable-libx265 --enable-libxvid --enable-libqy265 --enable-libfdk-aac --enable-openssl --enable-ffplay --enable-libfreetype --extra-cflags='-I/usr/local/include -Iext/deps/include' --extra-ldflags='-L/usr/local/lib -Lext/deps/lib_soft' --extra-libs='-lFastUdx -lqy265 -lx264 -lx265 -lfdk-aac -lcurl -lstdc++ -ldl -lpthread -lm' --prefix=/home/zhangzhixin/workdir/misc/ffmpeg/build
libavutil 56. 0.100 / 56. 0.100
libavcodec 58. 3.102 / 58. 3.102
libavformat 58. 2.100 / 58. 2.100
libavdevice 58. 0.100 / 58. 0.100
libavfilter 7. 0.101 / 7. 0.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 0.101 / 5. 0.101
libswresample 3. 0.101 / 3. 0.101
libpostproc 55. 0.100 / 55. 0.100

Splitting the commandline.
Reading option '-i' ... matched as input url with argument 'memory_error_fifo.mp4'.
Reading option '-s' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '854x480'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'libx264'.
Reading option '-x264opts' ... matched as AVOption 'x264opts' with argument 'keyint=50'.
Reading option '-b:v' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '238k'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'movie=namei.jpg [watermark]; [vf0] fifo [vf1];[vf1][watermark] overlay=main_w-overlay_w-0:0[vf2]'.
Reading option '-acodec' ... matched as option 'acodec' (force audio codec ('copy' to copy stream)) with argument 'libfdk_aac'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '44100'.
Reading option '-movflags' ... matched as AVOption 'movflags' with argument 'faststart'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mp4'.
Reading option 'output.mp4' ... matched as output url.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'trace'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option v (set logging level) with argument trace.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url memory_error_fifo.mp4.
Successfully parsed a group of options.
Opening an input file: memory_error_fifo.mp4.
[NULL @ 0x2d76ba0] Opening 'memory_error_fifo.mp4' for reading
[file @ 0x2d77420] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] Processing st: 0, edit list 0 - media time: 0, duration: 148409010
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] Before avformat_find_stream_info() pos: 181496607 bytes read:1197919 seeks:1 nb_streams:2
[h264 @ 0x2d78480] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x2d78480] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x2d78480] Reinit context to 1280x720, pix_fmt: yuv420p
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x2d78480] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d78480] nal_unit_type: 1, nal_ref_idc: 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] max_analyze_duration 5000000 reached at 5015270 microseconds st:0
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2d76ba0] After avformat_find_stream_info() pos: 70798 bytes read:1296223 seeks:2 frames:147
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'memory_error_fifo.mp4':

Metadata:

major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1970-01-01T00:00:00.000000Z

Duration: 01:22:26.97, start: 0.000000, bitrate: 293 kb/s

Stream #0:0(und), 147, 1/30000: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709, left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 238 kb/s, 29.31 fps, 30 tbr, 30k tbn, 60 tbc (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler

Stream #0:1(und), 0, 1/44100: Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 0 kb/s (default)
Metadata:

creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler

Successfully opened the file.
Parsing a group of options: output url output.mp4.
Applying option s (set frame size (WxH or abbreviation)) with argument 854x480.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument libx264.
Applying option b:v (video bitrate (please use -b:v)) with argument 238k.
Applying option vf (set video filters) with argument movie=namei.jpg [watermark]; [vf0] fifo [vf1];[vf1][watermark] overlay=main_w-overlay_w-0:0[vf2].
Applying option acodec (force audio codec ('copy' to copy stream)) with argument libfdk_aac.
Applying option ar (set audio sampling rate (in Hz)) with argument 44100.
Applying option f (force format) with argument mp4.
Successfully parsed a group of options.
Opening an output file: output.mp4.
[file @ 0x3883cc0] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 2 logical cores
[h264 @ 0x2d9ece0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x2d9ece0] nal_unit_type: 8, nal_ref_idc: 3
Stream mapping:

Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))

Press [q] to stop, ? for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x2d9ece0] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d9ece0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x2d9ece0] Reinit context to 1280x720, pix_fmt: yuv420p
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x32f8340] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x32f8340] nal_unit_type: 1, nal_ref_idc: 2
[h264 @ 0x3280cc0] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x3280cc0] nal_unit_type: 1, nal_ref_idc: 2
[Parsed_movie_0 @ 0x32b9aa0] Setting 'filename' to value 'namei.jpg'
[NULL @ 0x2dcc7c0] Opening 'namei.jpg' for reading
[file @ 0x3883fe0] Setting default whitelist 'file,crypto'
[image2 @ 0x2dcc7c0] Format image2 probed with size=2048 and score=50
[image2 @ 0x2dcc7c0] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[mjpeg @ 0x2dcd720] marker=d8 avail_size_in_buf=43205
[mjpeg @ 0x2dcd720] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x2dcd720] marker=e0 avail_size_in_buf=43203
[mjpeg @ 0x2dcd720] marker parser used 16 bytes (128 bits)
[mjpeg @ 0x2dcd720] marker=e1 avail_size_in_buf=43185
[mjpeg @ 0x2dcd720] marker parser used 47 bytes (376 bits)
[mjpeg @ 0x2dcd720] marker=db avail_size_in_buf=43135
[mjpeg @ 0x2dcd720] index=0
[mjpeg @ 0x2dcd720] qscale[0]: 0
[mjpeg @ 0x2dcd720] marker parser used 67 bytes (536 bits)
[mjpeg @ 0x2dcd720] marker=db avail_size_in_buf=43066
[mjpeg @ 0x2dcd720] index=1
[mjpeg @ 0x2dcd720] qscale[1]: 0
[mjpeg @ 0x2dcd720] marker parser used 67 bytes (536 bits)
[mjpeg @ 0x2dcd720] marker=c0 avail_size_in_buf=42997
[mjpeg @ 0x2dcd720] Changing bps from 0 to 8
[mjpeg @ 0x2dcd720] sof0: picture: 200x200
[mjpeg @ 0x2dcd720] component 0 1:1 id: 0 quant:0
[mjpeg @ 0x2dcd720] component 1 1:1 id: 1 quant:1
[mjpeg @ 0x2dcd720] component 2 1:1 id: 2 quant:1
[mjpeg @ 0x2dcd720] pix fmt id 11111100
[mjpeg @ 0x2dcd720] marker parser used 17 bytes (136 bits)
[mjpeg @ 0x2dcd720] marker=c4 avail_size_in_buf=42978
[mjpeg @ 0x2dcd720] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x2dcd720] marker=c4 avail_size_in_buf=42945
[mjpeg @ 0x2dcd720] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x2dcd720] marker=c4 avail_size_in_buf=42762
[mjpeg @ 0x2dcd720] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x2dcd720] marker=c4 avail_size_in_buf=42729
[mjpeg @ 0x2dcd720] marker parser used 0 bytes (0 bits)
[mjpeg @ 0x2dcd720] escaping removed 792 bytes
[mjpeg @ 0x2dcd720] marker=da avail_size_in_buf=42546
[mjpeg @ 0x2dcd720] marker parser used 41754 bytes (334032 bits)
[mjpeg @ 0x2dcd720] marker=d9 avail_size_in_buf=0
[mjpeg @ 0x2dcd720] decode frame unused 0 bytes
[image2 @ 0x2dcc7c0] After avformat_find_stream_info() pos: 43207 bytes read:43207 seeks:0 frames:1
[Parsed_movie_0 @ 0x32b9aa0] seek_point:0 format_name:(null) file_name:namei.jpg stream_index:-1
[Parsed_overlay_2 @ 0x2dcfb00] Setting 'x' to value 'main_w-overlay_w-0'
[Parsed_overlay_2 @ 0x2dcfb00] Setting 'y' to value '0'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'time_base' to value '1/30000'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x2dd5940] Setting 'frame_rate' to value '30/1'
[graph 0 input from stream 0:0 @ 0x2dd5940] w:1280 h:720 pixfmt:yuv420p tb:1/30000 fr:30/1 sar:1/1 sws_param:flags=2
[scaler_out_0_0 @ 0x2dd68a0] Setting 'w' to value '854'
[scaler_out_0_0 @ 0x2dd68a0] Setting 'h' to value '480'
[scaler_out_0_0 @ 0x2dd68a0] Setting 'flags' to value 'bicubic'
[scaler_out_0_0 @ 0x2dd68a0] w:854 h:480 flags:'bicubic' interl:0
[format @ 0x2dcf1e0] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|nv12|nv16|nv21'
[auto_scaler_0 @ 0x2f15c20] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 0x2f15c20] w:iw h:ih flags:'bicubic' interl:0
[Parsed_overlay_2 @ 0x2dcfb00] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_movie_0' and the filter 'Parsed_overlay_2'
[AVFilterGraph @ 0x2d76880] query_formats: 7 queried, 5 merged, 1 already done, 0 delayed
[swscaler @ 0x2def720] deprecated pixel format used, make sure you did set range correctly
[auto_scaler_0 @ 0x2f15c20] w:200 h:200 fmt:yuvj444p sar:1/1 -> w:200 h:200 fmt:yuva420p sar:1/1 flags:0x4
[Parsed_overlay_2 @ 0x2dcfb00] main w:1280 h:720 fmt:yuv420p overlay w:200 h:200 fmt:yuva420p
[Parsed_overlay_2 @ 0x2dcfb00] [framesync @ 0x2dcfc48] Selected 1/30000 time base
[Parsed_overlay_2 @ 0x2dcfb00] [framesync @ 0x2dcfc48] Sync level 2
[scaler_out_0_0 @ 0x2dd68a0] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:854 h:480 fmt:yuv420p sar:1280/1281 flags:0x4
[libx264 @ 0x2d7f2a0] using mv_range_thread = 72
[libx264 @ 0x2d7f2a0] using SAR=1280/1281
[libx264 @ 0x2d7f2a0] using cpu capabilities: none!
[libx264 @ 0x2d7f2a0] profile High, level 3.1
[libx264 @ 0x2d7f2a0] 264 - core 152 r2851 ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=238 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x2d9ece0] nal_unit_type: 9, nal_ref_idc: 0
[h264 @ 0x2d9ece0] nal_unit_type: 1, nal_ref_idc: 2

How to reproduce:
ffmpeg -i "memory_error_fifo.mp4" -s 854x480 -vcodec libx264 -x264opts keyint=50 -b:v 238k -vf "movie=namei.jpg [watermark]; [vf0] fifo [vf1];[vf1][watermark] overlay=main_w-overlay_w-0:0[vf2]" -acodec libfdk_aac -ar 44100 -movflags faststart -f mp4 output.mp4 -y -v trace -report
n3.5-dev-1292-gce001bb with master commit ce001bb8fc6677541c401a614e05e5058d58dde1
built on linux

attached source file: https://pan.baidu.com/s/1pLzbwbp

Change History (1)

comment:1 by llogan, 6 years ago

Cc: lq@chinaffmpeg.org removed
Resolution: duplicate
Status: newclosed

Duplicate of #6922.

Note: See TracTickets for help on using tickets.