Opened 5 months ago

Closed 5 months ago

Last modified 5 months ago

#7202 closed defect (needs_more_info)

mjpeg compression creates laggy video

Reported by: jsteinkamp Owned by:
Priority: normal Component: undetermined
Version: unspecified Keywords:
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:
When I create mjpeg mov files with ffmpeg the mov file skips frames
to keep up with the framerate, it does not seem to be decoding correctly.
I included the same file created with Adobe After Effects for comparison.
I have been able to reproduce this multiple times on various players.
Quicktime 7 shows the problem the most. Larger pixel resolution files are worse.
I would love to be able to use ffmpeg because it is much faster to work with.
download 172MB example files from: http://jsteinkamp.com/mjp_test.zip
How to reproduce:

ffmpeg -i input -vcodec mjpeg -q:v 8 -f mov output.mov
ffmpeg version N-91024-g293a6e8332
  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --e
nable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libblur
ay --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enab
le-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-li
bvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --en
able-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-
libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enabl
e-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enabl
e-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enab
le-dxva2 --enable-avisynth
  libavutil      56. 18.100 / 56. 18.100
  libavcodec     58. 19.101 / 58. 19.101
  libavformat    58. 13.102 / 58. 13.102
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 21.100 /  7. 21.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, image2, from 'tiffs\hibiscus1.%04d.tif':
  Duration: 00:00:06.67, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: tiff, rgba, 1024x768 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr,
30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Incompatible pixel format 'rgba' for codec 'mjpeg', auto-selecting format 'yuvj4
44p'
[swscaler @ 000000000281a800] deprecated pixel format used, make sure you did se
t range correctly
Output #0, mov, to 'hibiscus1_ffmpeg_mjpeg.mov':
  Metadata:
    encoder         : Lavf58.13.102
    Stream #0:0: Video: mjpeg (jpeg / 0x6765706A), yuvj444p(pc), 1024x768 [SAR 1
:1 DAR 4:3], q=2-31, 200 kb/s, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.19.101 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame=   76 fps=0.0 q=8.0 size=    4096kB time=00:00:02.43 bitrate=13789.2kbits/
frame=  157 fps=156 q=8.0 size=    8704kB time=00:00:05.13 bitrate=13890.1kbits/
frame=  200 fps=157 q=8.0 Lsize=   11530kB time=00:00:06.63 bitrate=14238.8kbits
/s speed= 5.2x
video:11528kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing
 overhead: 0.014147%

Patches should be submitted to the ffmpeg-devel mailing list and not this bug tracker.

Change History (14)

comment:1 Changed 5 months ago by cehoyos

  • Keywords mjpeg removed
  • Priority changed from important to normal

Please provide your actual command line together with the complete, uncut console output to make this a valid ticket and please explain what ’’laggy’’ means in this context: Since there is no audio, how can video lag behind?

comment:2 Changed 5 months ago by mkver

Link doesn't work for me.

comment:3 Changed 5 months ago by jsteinkamp

sorry I should have checked the link:
http://jsteinkamp.com/mjpg_test.zip

comment:4 in reply to: ↑ description Changed 5 months ago by cehoyos

  • Resolution set to needs_more_info
  • Status changed from new to closed

Replying to jsteinkamp:

Incompatible pixel format 'rgba' for codec 'mjpeg', auto-selecting format 'yuvj444p'

I can only guess that you missed this line.

comment:5 follow-ups: Changed 5 months ago by jsteinkamp

  • Resolution needs_more_info deleted
  • Status changed from closed to reopened

Please don't close, changing the rgba to yuvj444p makes no difference.

set input=tiffs\hibiscus1.%%04d.tif -pix_fmt yuvj444p
set codec=-vcodec mjpeg -q:v 8 -f mov
ffmpeg -y -framerate 30 -i %input% -frames 200 -vf "setsar=1" %codec% hibiscus1_ffmpeg_mjpeg.mov



ffmpeg version N-91024-g293a6e8332 Copyright (c) 2000-2018 the FFmpeg developers

  built with gcc 7.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --e
nable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libblur
ay --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enab
le-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-li
bvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --en
able-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-
libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enabl
e-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enabl
e-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enab
le-dxva2 --enable-avisynth
  libavutil      56. 18.100 / 56. 18.100
  libavcodec     58. 19.101 / 58. 19.101
  libavformat    58. 13.102 / 58. 13.102
  libavdevice    58.  4.100 / 58.  4.100
  libavfilter     7. 21.100 /  7. 21.100
  libswscale      5.  2.100 /  5.  2.100
  libswresample   3.  2.100 /  3.  2.100
  libpostproc    55.  2.100 / 55.  2.100
Input #0, image2, from 'tiffs\hibiscus1.%04d.tif':
  Duration: 00:00:06.67, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: tiff, rgba, 1024x768 [SAR 1:1 DAR 4:3], 30 fps, 30 tbr,
30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (tiff (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 000000000291c000] deprecated pixel format used, make sure you did se
t range correctly
Output #0, mov, to 'hibiscus1_ffmpeg_mjpeg.mov':
  Metadata:
    encoder         : Lavf58.13.102
    Stream #0:0: Video: mjpeg (jpeg / 0x6765706A), yuvj444p(pc), 1024x768 [SAR 1
:1 DAR 4:3], q=2-31, 200 kb/s, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc58.19.101 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
frame=   75 fps=0.0 q=8.0 size=    4096kB time=00:00:02.40 bitrate=13980.8kbits/
frame=  159 fps=158 q=8.0 size=    8960kB time=00:00:05.20 bitrate=14115.3kbits/
frame=  200 fps=160 q=8.0 Lsize=   11530kB time=00:00:06.63 bitrate=14238.8kbits
/s speed=5.31x
video:11528kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing
 overhead: 0.014147%
Last edited 5 months ago by cehoyos (previous) (diff)

comment:6 in reply to: ↑ 5 Changed 5 months ago by cehoyos

  • Resolution set to needs_more_info
  • Status changed from reopened to closed

Replying to jsteinkamp:

Please don't close, changing the rgba to yuvj444p makes no difference.

Of course not (it's the chosen pix_fmt anyway), but that is not the pix_fmt of the sample video you said is ok.

Note that you still failed to explain the actual issue.

comment:7 Changed 5 months ago by jsteinkamp

HI The issue is this, when you make a mjpg movie it does not play very well, especially in quicktime. But I have also noticed this in other players. If you search the internet you will see many mentions of ffmpeg not working well with mjpg. Aftereffects mjpeg movies with the same compression work great. The issue becomes very obvious with larger resolution movies.

comment:8 Changed 5 months ago by cehoyos

Could you point me to one of those mentions?

Which players - except for Quicktime - have issues with the sample you provided?

comment:9 in reply to: ↑ 5 Changed 5 months ago by cehoyos

Replying to jsteinkamp:

set input=tiffs\hibiscus1.%%04d.tif -pix_fmt yuvj444p
set codec=-vcodec mjpeg -q:v 8 -f mov
ffmpeg -y -framerate 30 -i %input% -frames 200 -vf "setsar=1" %codec% hibiscus1_ffmpeg_mjpeg.mov

For future tickets: Please remember not to use variables in the command line, and remove unneeded parts of the command line like in this case setsar=1 for input with SAR 1:1-

comment:10 Changed 5 months ago by Gyan

Both video outputs - AE and FF - play identically for me using ffplay, Potplayer and VLC on Windows.

Comparing the two outputs via -lavfi blend=difference,blackframe doesn't seem to show any dropped or rather duplicated frames in the FF output.

comment:11 Changed 5 months ago by jsteinkamp

HI I need some time to make you a demonstration player, the files will be large. I am using Unity with the AVPro plugin. I can also see results with Director + quicktime players. The problem is subtle. It mostly causes noticeable issues when I run multiple videos in sync, or large pixel format videos. Like i said, the AfterEffects? movies play better, no lagging.

VLC, MPC-HC do play the videos better than quicktime. Quicktime accentuates the issue.

I would like to have this issue resolved because ffmpeg is so much faster to work with compared to After Effects. Mjpeg files have good colors, black blacks, and can be frame searched compared to other codecs.

comment:12 follow-up: Changed 5 months ago by jsteinkamp

HI I have figured out the issue: ffmpeg compresses Photo JPEG with yuvj444p 24 bit by default. This compression does not play well on any computers since it uses software to decode. yuvj420p plays much better since it is 16 bit. Older Adobe After Effects compresses yuvj420p when you select medium compression, of course they do not explain any of this. I realize Photo JPEG is an older file format, but i really need it because the blacks are blacker and it searches well for realtime looping. I still believe there is a bit of a lag, when I sync 3 projections using photoJPG files, the files created in After Effects run better than files made in ffmpeg. The main draw back for using After Effects is it takes much longer to make the movie and they stopped supporting this format.

comment:13 in reply to: ↑ 12 Changed 5 months ago by cehoyos

Replying to jsteinkamp:

HI I have figured out the issue: ffmpeg compresses Photo JPEG with yuvj444p 24 bit by default.

This is not correct (it depends on the input file).

This compression does not play well on any computers since it uses software to decode.

This sounds wrong.

But didn’t we already solve this mystery above?

comment:14 Changed 5 months ago by jsteinkamp

Hi To be clear, I am suggesting you change your default setting to yuv420p for people who do not know the difference between the yuvj444p and the yuvj420p options.
And I don't think the mystery is completely solved because the AfterEffects? compressed files do play better.
I might send you a player to prove this.

Note: See TracTickets for help on using tickets.