Opened 8 years ago

Closed 6 years ago

Last modified 13 months ago

#5654 closed defect (fixed)

avfoundation screen capture produces misaligned picture at 1366x768 resolution

Reported by: evanrinehart Owned by:
Priority: normal Component: avdevice
Version: git-master Keywords: avfoundation
Cc: thilo.borgmann@mail.de, mixelpix@mac.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Following the instructions here https://trac.ffmpeg.org/wiki/Capture/Desktop for OSX to try and get a .mov screen capture video will output a .mov file whose picture seems to have each row shifted a constant amount to the right with respect to the previous row. So the image appears skewed.

By specifying each of the available pixel_formats I am able to get different amounts of skew, or in the case of nv12, also some fun random green artifacts thrown in. 0rgb bgr0 gives almost visible pictures, these seem to be off by 1 pixel in each row, giving a 2:1 skew.

Using .avi instead of .mov gives essentially the same picture only with worse quality due to chosen compression. So it doesn't seem like the output format is the problem.

rkern in #ffmpeg reported that it works for them at resolutions 1280x720, 960x540, 1024x576
but I can't check this since I can't change the rez on my macbook air.

Attaching the .mov file and a raw YUV frame.

Specifics:
ffmpeg-3.0.2 or current git master (0ea58059)
OSX El Captaina
Macbook Air with screen resolution 1366x768

Command:
ffmpeg -f avfoundation -i "1:0" out.mov

Attachments (7)

out.mov (689.9 KB ) - added by evanrinehart 8 years ago.
MOV output
out.yuv (2.0 MB ) - added by evanrinehart 8 years ago.
YUV frame
log-mov.txt (5.6 KB ) - added by evanrinehart 8 years ago.
console output of command that generated mov file
log-yuv.txt (2.1 KB ) - added by evanrinehart 8 years ago.
console output of command that generated yuv frame
out-cropped.mov (1.1 MB ) - added by evanrinehart 8 years ago.
cropped version of problem output file
outyuv_fixed.jpg (140.8 KB ) - added by Rick Kern 8 years ago.
fix-5654.patch (3.8 KB ) - added by Rick Kern 8 years ago.

Change History (22)

by evanrinehart, 8 years ago

Attachment: out.mov added

MOV output

by evanrinehart, 8 years ago

Attachment: out.yuv added

YUV frame

by evanrinehart, 8 years ago

Attachment: log-mov.txt added

console output of command that generated mov file

by evanrinehart, 8 years ago

Attachment: log-yuv.txt added

console output of command that generated yuv frame

comment:1 by evanrinehart, 8 years ago

Command that produced the YUV frame, suggested by rkern in #ffmpeg:

ffmpeg -f avfoundation -i "1:0" -frames:v 1 -c:v rawvideo out.yuv

comment:2 by Carl Eugen Hoyos, 8 years ago

Component: avdeviceundetermined
Keywords: capture avfoundation skew alignment removed
Resolution: duplicate
Status: newclosed

I believe this is a duplicate if #4079, I added a workaround there.

comment:3 by evanrinehart, 8 years ago

I tried both the command that is the problem in #4079 and both suggested workarounds, rgb24 and yuv444p and they both produce the same issue I am having with a full screen capture (though the output is cropped). You can see the problem by looking at the attached .mov or I can upload the cropped version.

Command I used:
ffmpeg -f avfoundation -pix_fmt uyvy422 -video_device_index 1 -audio_device_index 0 -i "" -r 60 -c:v libx264 -crf 0 -vf format=yuv444p,crop=426:240:507:339 -c:a pcm_s16le -ar 48000 out.mov

Because others seem to have no trouble on "normal" resolutions I think my circumstances are caused by the 1366x768 resolution. I am still looking at the avfoundation.m source file to try and understand where the bad width is coming from.

I don't really consider this issue closed.

by evanrinehart, 8 years ago

Attachment: out-cropped.mov added

cropped version of problem output file

comment:4 by Thilo Borgmann, 8 years ago

Cc: thilo.borgmann@mail.de added
Resolution: duplicate
Status: closedreopened

Reopened because I don't think it really is a duplicate.

comment:5 by Carl Eugen Hoyos, 8 years ago

Component: undeterminedavdevice
Keywords: avfoundation added

Please remove audio from your command line and provide complete, uncut console output.

comment:6 by Thilo Borgmann, 8 years ago

Keywords: capture skew alignment added

comment:7 by evanrinehart, 8 years ago

> ffmpeg -f avfoundation -i "1:none" out.mov
ffmpeg version N-80421-g0ea5805 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-opengl
  libavutil      55. 24.100 / 55. 24.100
  libavcodec     57. 46.100 / 57. 46.100
  libavformat    57. 38.102 / 57. 38.102
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 46.102 /  6. 46.102
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[avfoundation @ 0x7fd0eb800000] Selected pixel format (yuv420p) is not supported by the input device.
[avfoundation @ 0x7fd0eb800000] Supported pixel formats:
[avfoundation @ 0x7fd0eb800000]   uyvy422
[avfoundation @ 0x7fd0eb800000]   yuyv422
[avfoundation @ 0x7fd0eb800000]   nv12
[avfoundation @ 0x7fd0eb800000]   0rgb
[avfoundation @ 0x7fd0eb800000]   bgr0
[avfoundation @ 0x7fd0eb800000] Overriding selected pixel format to use uyvy422 instead.
Input #0, avfoundation, from '1:none':
  Duration: N/A, start: 18501.637500, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1366x768, 30 tbr, 1000k tbn, 1000k tbc
File 'out.mov' already exists. Overwrite ? [y/N] y
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x7fd0eb866a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7fd0eb866a00] profile High 4:2:2, level 3.2, 4:2:2 8-bit
[libx264 @ 0x7fd0eb866a00] 264 - core 148 r2668 fd2c324 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[mov @ 0x7fd0eb024a00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf57.38.102
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv422p, 1366x768, q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc57.46.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=  734 fps= 48 q=-1.0 Lsize=    2849kB time=00:00:24.36 bitrate= 957.9kbits/s dup=412 drop=0 speed= 1.6x
video:2840kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.328457%
[libx264 @ 0x7fd0eb866a00] frame I:4     Avg QP:16.87  size:278497
[libx264 @ 0x7fd0eb866a00] frame P:209   Avg QP:23.56  size:  7342
[libx264 @ 0x7fd0eb866a00] frame B:521   Avg QP:32.17  size:   497
[libx264 @ 0x7fd0eb866a00] consecutive B-frames:  3.8%  4.4%  0.8% 91.0%
[libx264 @ 0x7fd0eb866a00] mb I  I16..4: 38.2% 11.2% 50.6%
[libx264 @ 0x7fd0eb866a00] mb P  I16..4:  0.7%  1.0%  1.8%  P16..4:  2.7%  1.1%  0.4%  0.0%  0.0%    skip:92.3%
[libx264 @ 0x7fd0eb866a00] mb B  I16..4:  0.1%  0.5%  0.0%  B16..8:  1.8%  0.2%  0.0%  direct: 0.1%  skip:97.3%  L0:51.5% L1:46.9% BI: 1.7%
[libx264 @ 0x7fd0eb866a00] 8x8 transform intra:35.5% inter:20.6%
[libx264 @ 0x7fd0eb866a00] coded y,uvDC,uvAC intra: 33.5% 27.7% 25.7% inter: 0.5% 0.4% 0.3%
[libx264 @ 0x7fd0eb866a00] i16 v,h,dc,p: 45% 50%  4%  1%
[libx264 @ 0x7fd0eb866a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 12% 71%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7fd0eb866a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 49% 16%  1%  2%  1%  9%  1%  6%
[libx264 @ 0x7fd0eb866a00] i8c dc,h,v,p: 79% 11%  9%  1%
[libx264 @ 0x7fd0eb866a00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fd0eb866a00] ref P L0: 53.3% 10.0% 24.0% 12.7%
[libx264 @ 0x7fd0eb866a00] ref B L0: 70.6% 28.1%  1.3%
[libx264 @ 0x7fd0eb866a00] ref B L1: 97.6%  2.4%
[libx264 @ 0x7fd0eb866a00] kb/s:950.66

comment:8 by Carl Eugen Hoyos, 8 years ago

Keywords: capture skew alignment removed
Version: unspecifiedgit-master

Please test the following and upload the output files, zip them if necessary:

$ ffmpeg -pixel_format uyvy422 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.uyvy
$ ffmpeg -pixel_format yuyv422 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.yuyv
$ ffmpeg -pixel_format nv12 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.nv12
$ ffmpeg -pixel_format 0rgb -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.0rgb
$ ffmpeg -pixel_format bgr0 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.bgr0

by Rick Kern, 8 years ago

Attachment: outyuv_fixed.jpg added

by Rick Kern, 8 years ago

Attachment: fix-5654.patch added

comment:9 by Rick Kern, 8 years ago

Can you test fix-5654.patch?

The linesize on the uyvy422 frame was 1376 (see outyuv_fixed.jpg), but the buffer size was for a linesize of 1366.

comment:10 by evanrinehart, 8 years ago

I did not see any reference to 1367 or 1366 in the image, and my local output only ever showed 1366 (in reference to screen size).

However that patch seems to have fixed it! Quicktime still refuses to load the resulting .mov file "The file may be damaged or may not be a movie file that is compatible with QuickTime Player." But it works fine in VLC.

Great!

I will continue my testing trying different encoders and containers with avfoundations capture.

in reply to:  10 comment:11 by Patrick Kennedy, 7 years ago

Replying to evanrinehart:

However that patch seems to have fixed it!

Hello, I have run into the same problem with my macOS (Sierra) Air Book's default display size (1366x768). Thank you for posting this ticket about AV Foundation's screen capture. Please excuse my ignorance, but how do I install the patch? I've read the patch man pages and searched on-line for a tutorial or description of the procedure, but I am not sure what I am supposed to do to install the patch.

If it matters, I have used Homebrew to install FFmpeg. The file which needs patching is here:
/Users/mixelpix/Library/Caches/Homebrew/ffmpeg--git/libavdevice/avfoundation.m
...but I don't understand the overall process - for example, will I also need to use make commands to re-compile the ffmpeg executable in /usr/local/Cellar/ffmpeg/HEAD-3405d6c/bin/ffmpeg? I apologize if this is not an appropriate place to pose this question. Thank you for your consideration.

comment:12 by Patrick Kennedy, 7 years ago

Cc: mixelpix@mac.com added

comment:14 by Carl Eugen Hoyos, 6 years ago

Resolution: fixed
Status: reopenedclosed

comment:15 by Balling, 13 months ago

This is still broken on 1656x932, see #10228.

Note: See TracTickets for help on using tickets.