Opened 10 years ago

Closed 10 months ago

#3020 closed enhancement (wontfix)

FFmpeg should do framerate convert before scaling if (and only if) that is expected to be faster

Reported by: kaifu Owned by:
Priority: normal Component: ffmpeg
Version: git-master Keywords: ffmpeg
Cc: Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: yes

Description

Summary of the bug:
How to reproduce:

%time ffmpeg -i test.mp4 -vcodec mjpeg -s 960x640 -r 0.2 -y out.mov

we may compare the real time with result of running scripts that transcoding to other video format, such as msmpeg4, x264,...

ffmpeg version 2.0.1
built on linux/MINGW32

Details see below:

$ time ./ffmpeg -i test.mp4 -vcodec mjpeg -s 960x640 -r 0.2 -an -y out.mov
ffmpeg version 2.0 Copyright (c) 2000-2013 the FFmpeg developers

built on Oct 1 2013 00:27:45 with gcc 4.3.2 (4.3.2-tdm-1 for MinGW)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-memalign-hack --enable-pthreads --disable-txtpages
libavutil 52. 46.100 / 52. 46.100
libavcodec 55. 33.101 / 55. 33.101
libavformat 55. 18.104 / 55. 18.104
libavdevice 55. 3.100 / 55. 3.100
libavfilter 3. 88.100 / 3. 88.100
libswscale 2. 5.100 / 2. 5.100
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2013-04-09 11:10:32
copyright :
copyright-eng :

Duration: 00:03:53.87, start: 0.000000, bitrate: 897 kb/s

Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x960, 846 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 47 kb/s (default)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

[swscaler @ 03783020] deprecated pixel format used, make sure you did set range correctly
Output #0, mov, to 'out.mov':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
copyright-eng :
copyright :
encoder : Lavf55.18.104
Stream #0:0(und): Video: mjpeg (jpeg / 0x6765706A), yuvj420p, 960x640, q=2-31, 200 kb/s, 16384 tbn, 0.20 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> mjpeg)

Press [q] to stop, ? for help
frame= 50 fps=0.7 q=0.0 Lsize= 4130kB time=00:04:10.00 bitrate= 135.3kbits/s dup=0 drop=5561
video:4129kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.023129%

real 1m9.687s
user 0m0.031s
sys 0m0.015s

$ time ./ffmpeg -i test.mp4 -vcodec msmpeg4 -s 960x640 -r 0.2 -an -y out.avi
ffmpeg version 2.0 Copyright (c) 2000-2013 the FFmpeg developers

built on Oct 1 2013 00:27:45 with gcc 4.3.2 (4.3.2-tdm-1 for MinGW)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-memalign-hack --enable-pthreads --disable-txtpages
libavutil 52. 46.100 / 52. 46.100
libavcodec 55. 33.101 / 55. 33.101
libavformat 55. 18.104 / 55. 18.104
libavdevice 55. 3.100 / 55. 3.100
libavfilter 3. 88.100 / 3. 88.100
libswscale 2. 5.100 / 2. 5.100
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2013-04-09 11:10:32
copyright :
copyright-eng :

Duration: 00:03:53.87, start: 0.000000, bitrate: 897 kb/s

Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x960, 846 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 47 kb/s (default)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

Output #0, avi, to 'out.avi':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
copyright-eng :
ICOP :
ISFT : Lavf55.18.104
Stream #0:0(und): Video: msmpeg4v3 (msmpeg4) (MP43 / 0x3334504D), yuv420p, 960x640, q=2-31, 200 kb/s, 0.20 tbn, 0.20 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> msmpeg4)

Press [q] to stop, ? for help
frame= 49 fps=1.7 q=1.6 Lsize= 3972kB time=00:04:05.00 bitrate= 132.8kbits/s dup=0 drop=5562
video:3965kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.169484%

real 0m29.644s
user 0m0.015s
sys 0m0.031s

I tested the benchmark of ffmpeg, and I can not find any issue in codec's performance, but I profiled the transcoder and find the performance issue is in the below call-chain, it is avfilter issue.
main()->transcode()->transcode_step()->process_input()->output_packet()->decode_video()->av_buffersrc_add_frame_flags()->av_buffersrc_add_frame_internal()->ctx->output_pads[0].request_frame(ctx->outputs[0])

Attachments (1)

test.mp4 (2.5 MB ) - added by kaifu 10 years ago.
test sample for this issue

Change History (9)

comment:1 by kaifu, 10 years ago

Analyzed by developer: unset
Reproduced by developer: unset

comment:2 by Elon Musk, 10 years ago

Why are you using different containers?
If you want to do real benchmark use '-f null -'

by kaifu, 10 years ago

Attachment: test.mp4 added

test sample for this issue

in reply to:  2 comment:3 by kaifu, 10 years ago

Replying to richardpl:

Why are you using different containers?
If you want to do real benchmark use '-f null -'

I tested it again following your advice with attached stream, but this issue reproduced yet.
$ time ./ffmpeg -i test.mp4 -vcodec mjpeg -r 0.2 -f null -
ffmpeg version 2.0 Copyright (c) 2000-2013 the FFmpeg developers

built on Oct 1 2013 00:27:45 with gcc 4.3.2 (4.3.2-tdm-1 for MinGW)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-memalign-hack --enable-pthreads -- disable-txtpages
libavutil 52. 46.100 / 52. 46.100
libavcodec 55. 33.101 / 55. 33.101
libavformat 55. 18.104 / 55. 18.104
libavdevice 55. 3.100 / 55. 3.100
libavfilter 3. 88.100 / 3. 88.100
libswscale 2. 5.100 / 2. 5.100
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2013-04-09 11:10:32
copyright :
copyright-eng :

Duration: 00:03:53.87, start: 0.000000, bitrate: 89 kb/s

Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709) , 1280x960, 846 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 47 kb/s (default)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

[swscaler @ 02f309e0] deprecated pixel format used, make sure you did set range correctly
Output #0, null, to 'pipe:':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
copyright-eng :
copyright :
encoder : Lavf55.18.104
Stream #0:0(und): Video: mjpeg, yuvj420p, 1280x960, q=2-31, 200 kb/s, 90k tb n, 0.20 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (defaul t)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> mjpeg)
Stream #0:1 -> #0:1 (aac -> pcm_s16le)

Press [q] to stop, ? for help
frame= 3 fps=0.0 q=0.0 size=N/A time=00:00:05.00 bitrate=N/A dup=0 drop=25 frame= 4 fps=3.9 q=0.0 size=N/A time=00:00:10.00 bitrate=N/A dup=0 drop=57 frame= 4 fps=2.6 q=0.0 size=N/A time=00:00:10.00 bitrate=N/A dup=0 drop=90 frame= 4 fps=2.0 q=0.0 size=N/A time=00:00:10.00 bitrate=N/A dup=0 drop=123 frame= 4 fps=1.6 q=0.0 size=N/A time=00:00:10.00 bitrate=N/A dup=0 drop=158 frame= 5 fps=1.6 q=0.0 size=N/A time=00:00:15.00 bitrate=N/A dup=0 drop=193 frame= 5 fps=1.4 q=0.0 size=N/A time=00:00:15.00 bitrate=N/A dup=0 drop=226 frame= 5 fps=1.2 q=0.0 size=N/A time=00:00:15.00 bitrate=N/A dup=0 drop=260 frame= 5 fps=1.1 q=0.0 size=N/A time=00:00:15.00 bitrate=N/A dup=0 drop=293 frame= 6 fps=1.2 q=0.0 size=N/A time=00:00:20.00 bitrate=N/A dup=0 drop=325 frame= 6 fps=1.1 q=0.0 size=N/A time=00:00:20.00 bitrate=N/A dup=0 drop=358 frame= 6 fps=1.0 q=0.0 size=N/A time=00:00:20.00 bitrate=N/A dup=0 drop=393 frame= 7 fps=1.1 q=0.0 size=N/A time=00:00:25.00 bitrate=N/A dup=0 drop=425 frame= 7 fps=1.0 q=0.0 size=N/A time=00:00:25.00 bitrate=N/A dup=0 drop=461 [h264 @ 034243c0] AVC: nal size 19894

Last message repeated 1 times

[h264 @ 034243c0] no frame!
frame= 7 fps=1.0 q=0.0 Lsize=N/A time=00:00:35.00 bitrate=N/A dup=0 drop=474
video:506kB audio:3368kB subtitle:0 global headers:0kB muxing overhead -100.0005 55%

real 0m7.620s
user 0m0.000s
sys 0m0.030s

$ time ./ffmpeg -i test.mp4 -vcodec msmpeg4 -r 0.2 -f null -
ffmpeg version 2.0 Copyright (c) 2000-2013 the FFmpeg developers

built on Oct 1 2013 00:27:45 with gcc 4.3.2 (4.3.2-tdm-1 for MinGW)
configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --enable-nonfree --disable-ffserver --enable-memalign-hack --enable-pthreads -- disable-txtpages
libavutil 52. 46.100 / 52. 46.100
libavcodec 55. 33.101 / 55. 33.101
libavformat 55. 18.104 / 55. 18.104
libavdevice 55. 3.100 / 55. 3.100
libavfilter 3. 88.100 / 3. 88.100
libswscale 2. 5.100 / 2. 5.100
libswresample 0. 17.103 / 0. 17.103
libpostproc 52. 3.100 / 52. 3.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2013-04-09 11:10:32
copyright :
copyright-eng :

Duration: 00:03:53.87, start: 0.000000, bitrate: 89 kb/s

Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709) , 1280x960, 846 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 47 kb/s (default)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

Output #0, null, to 'pipe:':

Metadata:

major_brand : mp42
minor_version : 0
compatible_brands: isommp42
copyright-eng :
copyright :
encoder : Lavf55.18.104
Stream #0:0(und): Video: msmpeg4v3 (msmpeg4), yuv420p, 1280x960, q=2-31, 200 kb/s, 90k tbn, 0.20 tbc (default)
Metadata:

creation_time : 2013-04-09 11:10:30
handler_name : GPAC ISO Video Handler

Stream #0:1(und): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (defaul t)
Metadata:

creation_time : 2013-04-09 11:10:32
handler_name : GPAC ISO Audio Handler

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> msmpeg4)
Stream #0:1 -> #0:1 (aac -> pcm_s16le)

Press [q] to stop, ? for help
frame= 4 fps=0.0 q=2.0 size=N/A time=00:00:20.00 bitrate=N/A dup=0 drop=149 frame= 5 fps=5.0 q=2.0 size=N/A time=00:00:25.00 bitrate=N/A dup=0 drop=288 frame= 7 fps=4.6 q=2.0 size=N/A time=00:00:35.00 bitrate=N/A dup=0 drop=420 [h264 @ 037540e0] AVC: nal size 19894

Last message repeated 1 times

[h264 @ 037540e0] no frame!
frame= 7 fps=4.2 q=2.0 Lsize=N/A time=00:00:35.00 bitrate=N/A dup=0 drop=474
video:458kB audio:3368kB subtitle:0 global headers:0kB muxing overhead -100.0005 61%

real 0m1.985s
user 0m0.000s
sys 0m0.016s

comment:4 by Carl Eugen Hoyos, 10 years ago

Component: avfilterundetermined
Keywords: transcoder avfilter performance removed

Please test (and report) all issues with current git head.

Could you elaborate a little on the problem you have? Iiuc, you are reporting that the msmpeg4 encoder is faster than the mjpeg encoder but I am not sure that this is a useful explanation / that they can be compared like this.
An enhancement request for a faster mjpeg encoder is probably valid but I wonder if this is everything this issue is about...

Note that you cannot avoid the scaling step (the scaler is also responsible for colour-space conversion), your h264 input sample has a different colour space than mjpeg (while msmpeg4 has the same as your h264 sample). So if this is your problem, I don't think there is anything that can be fixed about it.

comment:5 by Michael Niedermayer, 10 years ago

Analyzed by developer: set
Component: undeterminedFFmpeg
Keywords: ffmpeg added
Reproduced by developer: set
Version: 2.0.1git-master

jpeg does not support yuv420, it uses yuvj420 which needs colorspace convertion that msmpeg4 does not need.
and scaling is done before framerate handling currently.
You can force it to be done the other way around with -vf fps=0.2,scale

comment:6 by Michael Niedermayer, 10 years ago

Keywords: mjpeg removed
Status: newopen
Summary: FFMPEG performance need optimize when transcoding to mjpegFFmpeg should do framerate convert before scaling if (and only if) that is expected to be faster

this isnt really mjpeg related, except by chance

in reply to:  5 comment:7 by kaifu, 10 years ago

Replying to michael:

jpeg does not support yuv420, it uses yuvj420 which needs colorspace convertion that msmpeg4 does not need.
and scaling is done before framerate handling currently.
You can force it to be done the other way around with -vf fps=0.2,scale

Thank you! michael.
I try testing it with your advice. This issue is resolved when frame rate set be low. The colorspace conversion or scaling will be performance issue if frame rate is unchange. Is it right?

comment:8 by Elon Musk, 10 months ago

Resolution: wontfix
Status: openclosed
Note: See TracTickets for help on using tickets.