Opened 13 months ago

Last modified 6 months ago

#5654 reopened defect

avfoundation screen capture produces misaligned picture at 1366x768 resolution

Reported by: evanrinehart Owned by:
Priority: normal Component: avdevice
Version: git-master Keywords: avfoundation
Cc: thilo.borgmann@mail.de, mixelpix@mac.com Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Following the instructions here https://trac.ffmpeg.org/wiki/Capture/Desktop for OSX to try and get a .mov screen capture video will output a .mov file whose picture seems to have each row shifted a constant amount to the right with respect to the previous row. So the image appears skewed.

By specifying each of the available pixel_formats I am able to get different amounts of skew, or in the case of nv12, also some fun random green artifacts thrown in. 0rgb bgr0 gives almost visible pictures, these seem to be off by 1 pixel in each row, giving a 2:1 skew.

Using .avi instead of .mov gives essentially the same picture only with worse quality due to chosen compression. So it doesn't seem like the output format is the problem.

rkern in #ffmpeg reported that it works for them at resolutions 1280x720, 960x540, 1024x576
but I can't check this since I can't change the rez on my macbook air.

Attaching the .mov file and a raw YUV frame.

Specifics:
ffmpeg-3.0.2 or current git master (0ea58059)
OSX El Captaina
Macbook Air with screen resolution 1366x768

Command:
ffmpeg -f avfoundation -i "1:0" out.mov

Attachments (7)

out.mov (689.9 KB) - added by evanrinehart 13 months ago.
MOV output
out.yuv (2.0 MB) - added by evanrinehart 13 months ago.
YUV frame
log-mov.txt (5.6 KB) - added by evanrinehart 13 months ago.
console output of command that generated mov file
log-yuv.txt (2.1 KB) - added by evanrinehart 13 months ago.
console output of command that generated yuv frame
out-cropped.mov (1.1 MB) - added by evanrinehart 13 months ago.
cropped version of problem output file
outyuv_fixed.jpg (140.8 KB) - added by kernrj 13 months ago.
fix-5654.patch (3.8 KB) - added by kernrj 13 months ago.

Change History (19)

Changed 13 months ago by evanrinehart

MOV output

Changed 13 months ago by evanrinehart

YUV frame

Changed 13 months ago by evanrinehart

console output of command that generated mov file

Changed 13 months ago by evanrinehart

console output of command that generated yuv frame

comment:1 Changed 13 months ago by evanrinehart

Command that produced the YUV frame, suggested by rkern in #ffmpeg:

ffmpeg -f avfoundation -i "1:0" -frames:v 1 -c:v rawvideo out.yuv

comment:2 Changed 13 months ago by cehoyos

  • Component changed from avdevice to undetermined
  • Keywords capture avfoundation skew alignment removed
  • Resolution set to duplicate
  • Status changed from new to closed

I believe this is a duplicate if #4079, I added a workaround there.

comment:3 Changed 13 months ago by evanrinehart

I tried both the command that is the problem in #4079 and both suggested workarounds, rgb24 and yuv444p and they both produce the same issue I am having with a full screen capture (though the output is cropped). You can see the problem by looking at the attached .mov or I can upload the cropped version.

Command I used:
ffmpeg -f avfoundation -pix_fmt uyvy422 -video_device_index 1 -audio_device_index 0 -i "" -r 60 -c:v libx264 -crf 0 -vf format=yuv444p,crop=426:240:507:339 -c:a pcm_s16le -ar 48000 out.mov

Because others seem to have no trouble on "normal" resolutions I think my circumstances are caused by the 1366x768 resolution. I am still looking at the avfoundation.m source file to try and understand where the bad width is coming from.

I don't really consider this issue closed.

Changed 13 months ago by evanrinehart

cropped version of problem output file

comment:4 Changed 13 months ago by thilo.borgmann

  • Cc thilo.borgmann@mail.de added
  • Resolution duplicate deleted
  • Status changed from closed to reopened

Reopened because I don't think it really is a duplicate.

comment:5 Changed 13 months ago by cehoyos

  • Component changed from undetermined to avdevice
  • Keywords avfoundation added

Please remove audio from your command line and provide complete, uncut console output.

comment:6 Changed 13 months ago by thilo.borgmann

  • Keywords capture skew alignment added

comment:7 Changed 13 months ago by evanrinehart

> ffmpeg -f avfoundation -i "1:none" out.mov
ffmpeg version N-80421-g0ea5805 Copyright (c) 2000-2016 the FFmpeg developers
  built with Apple LLVM version 7.0.2 (clang-700.1.81)
  configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-shared --enable-libass --enable-libfdk-aac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-opengl
  libavutil      55. 24.100 / 55. 24.100
  libavcodec     57. 46.100 / 57. 46.100
  libavformat    57. 38.102 / 57. 38.102
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 46.102 /  6. 46.102
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[avfoundation @ 0x7fd0eb800000] Selected pixel format (yuv420p) is not supported by the input device.
[avfoundation @ 0x7fd0eb800000] Supported pixel formats:
[avfoundation @ 0x7fd0eb800000]   uyvy422
[avfoundation @ 0x7fd0eb800000]   yuyv422
[avfoundation @ 0x7fd0eb800000]   nv12
[avfoundation @ 0x7fd0eb800000]   0rgb
[avfoundation @ 0x7fd0eb800000]   bgr0
[avfoundation @ 0x7fd0eb800000] Overriding selected pixel format to use uyvy422 instead.
Input #0, avfoundation, from '1:none':
  Duration: N/A, start: 18501.637500, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1366x768, 30 tbr, 1000k tbn, 1000k tbc
File 'out.mov' already exists. Overwrite ? [y/N] y
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x7fd0eb866a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 0x7fd0eb866a00] profile High 4:2:2, level 3.2, 4:2:2 8-bit
[libx264 @ 0x7fd0eb866a00] 264 - core 148 r2668 fd2c324 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[mov @ 0x7fd0eb024a00] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
Output #0, mov, to 'out.mov':
  Metadata:
    encoder         : Lavf57.38.102
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv422p, 1366x768, q=-1--1, 30 fps, 15360 tbn, 30 tbc
    Metadata:
      encoder         : Lavc57.46.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=  734 fps= 48 q=-1.0 Lsize=    2849kB time=00:00:24.36 bitrate= 957.9kbits/s dup=412 drop=0 speed= 1.6x
video:2840kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.328457%
[libx264 @ 0x7fd0eb866a00] frame I:4     Avg QP:16.87  size:278497
[libx264 @ 0x7fd0eb866a00] frame P:209   Avg QP:23.56  size:  7342
[libx264 @ 0x7fd0eb866a00] frame B:521   Avg QP:32.17  size:   497
[libx264 @ 0x7fd0eb866a00] consecutive B-frames:  3.8%  4.4%  0.8% 91.0%
[libx264 @ 0x7fd0eb866a00] mb I  I16..4: 38.2% 11.2% 50.6%
[libx264 @ 0x7fd0eb866a00] mb P  I16..4:  0.7%  1.0%  1.8%  P16..4:  2.7%  1.1%  0.4%  0.0%  0.0%    skip:92.3%
[libx264 @ 0x7fd0eb866a00] mb B  I16..4:  0.1%  0.5%  0.0%  B16..8:  1.8%  0.2%  0.0%  direct: 0.1%  skip:97.3%  L0:51.5% L1:46.9% BI: 1.7%
[libx264 @ 0x7fd0eb866a00] 8x8 transform intra:35.5% inter:20.6%
[libx264 @ 0x7fd0eb866a00] coded y,uvDC,uvAC intra: 33.5% 27.7% 25.7% inter: 0.5% 0.4% 0.3%
[libx264 @ 0x7fd0eb866a00] i16 v,h,dc,p: 45% 50%  4%  1%
[libx264 @ 0x7fd0eb866a00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 12% 71%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x7fd0eb866a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 15% 49% 16%  1%  2%  1%  9%  1%  6%
[libx264 @ 0x7fd0eb866a00] i8c dc,h,v,p: 79% 11%  9%  1%
[libx264 @ 0x7fd0eb866a00] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x7fd0eb866a00] ref P L0: 53.3% 10.0% 24.0% 12.7%
[libx264 @ 0x7fd0eb866a00] ref B L0: 70.6% 28.1%  1.3%
[libx264 @ 0x7fd0eb866a00] ref B L1: 97.6%  2.4%
[libx264 @ 0x7fd0eb866a00] kb/s:950.66

comment:8 Changed 13 months ago by cehoyos

  • Keywords capture skew alignment removed
  • Version changed from unspecified to git-master

Please test the following and upload the output files, zip them if necessary:

$ ffmpeg -pixel_format uyvy422 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.uyvy
$ ffmpeg -pixel_format yuyv422 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.yuyv
$ ffmpeg -pixel_format nv12 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.nv12
$ ffmpeg -pixel_format 0rgb -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.0rgb
$ ffmpeg -pixel_format bgr0 -f avfoundation -i "1:0" -frames:v 1 -f rawvideo -vcodec copy out.bgr0

Changed 13 months ago by kernrj

Changed 13 months ago by kernrj

comment:9 Changed 13 months ago by kernrj

Can you test fix-5654.patch?

The linesize on the uyvy422 frame was 1376 (see outyuv_fixed.jpg), but the buffer size was for a linesize of 1366.

comment:10 follow-up: Changed 13 months ago by evanrinehart

I did not see any reference to 1367 or 1366 in the image, and my local output only ever showed 1366 (in reference to screen size).

However that patch seems to have fixed it! Quicktime still refuses to load the resulting .mov file "The file may be damaged or may not be a movie file that is compatible with QuickTime? Player." But it works fine in VLC.

Great!

I will continue my testing trying different encoders and containers with avfoundations capture.

comment:11 in reply to: ↑ 10 Changed 6 months ago by mixelpix

Replying to evanrinehart:

However that patch seems to have fixed it!

Hello, I have run into the same problem with my macOS (Sierra) Air Book's default display size (1366x768). Thank you for posting this ticket about AV Foundation's screen capture. Please excuse my ignorance, but how do I install the patch? I've read the patch man pages and searched on-line for a tutorial or description of the procedure, but I am not sure what I am supposed to do to install the patch.

If it matters, I have used Homebrew to install FFmpeg. The file which needs patching is here:
/Users/mixelpix/Library/Caches/Homebrew/ffmpeg--git/libavdevice/avfoundation.m
...but I don't understand the overall process - for example, will I also need to use make commands to re-compile the ffmpeg executable in /usr/local/Cellar/ffmpeg/HEAD-3405d6c/bin/ffmpeg? I apologize if this is not an appropriate place to pose this question. Thank you for your consideration.

comment:12 Changed 6 months ago by mixelpix

  • Cc mixelpix@mac.com added
Note: See TracTickets for help on using tickets.