Opened 7 years ago

Closed 7 years ago

Last modified 3 months ago

#4805 closed defect (fixed)

swscale does not process pixels when only color-matrix differs between input and output

Reported by: rmk Owned by:
Priority: normal Component: swscale
Version: git-master Keywords:
Cc: mirh Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

swscale does not seem to process pixels when only color-matrix differs between input and output (i.e. pixel format and size remain the same).

It would be great to be able to use swscale also just for colorspace transformations like for bt601<->bt709 conversions.

Observed this as api user but verified it by converting two files as follows and comparing them via framemd5:

ffmpeg -i fcp7_xdcam_ex_1080_25p_1s.mov -vf scale=in_color_matrix=bt709:out_color_matrix=bt601 -c:v ffv1 -an scale_test_bt601.nut
ffmpeg -i xdcam_ex/fcp7_xdcam_ex_1080_25p_1s.mov -c:v ffv1 -an scale_test_unscaled.nut

The frames of the resulting files are identical.

Change History (9)

comment:1 by rmk, 7 years ago

Full command lines and output:

Roberts-MacBook-Pro:colortest krueger$ ffmpeg -i ~/lesspain/samples/software/fcp_7/xdcam_ex/fcp7_xdcam_ex_1080_25p_1s.mov -vf scale=in_color_matrix=bt709:out_color_matrix=bt601 -c:v ffv1 -an -y scale_test_bt601.nut
ffmpeg version N-74034-gce46627 Copyright (c) 2000-2015 the FFmpeg developers

built with Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
configuration: --enable-gpl --enable-libx264
libavutil 54. 29.100 / 54. 29.100
libavcodec 56. 55.100 / 56. 55.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 29.100 / 5. 29.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fb62380da00] nclc: pri 1 trc 1 matrix 1Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/krueger/lesspain/samples/software/fcp_7/xdcam_ex/fcp7_xdcam_ex_1080_25p_1s.mov':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2013-12-11 15:38:12

Duration: 00:00:01.00, start: 0.000000, bitrate: 25365 kb/s

Stream #0:0(eng): Video: mpeg2video (Main) (xdve / 0x65766478), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 23773 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
Metadata:

creation_time : 2013-12-11 15:38:12
handler_name : Apple Alias-Datensteuerung
encoder : XDCAM EX 1080p25 (35 Mb/s VBR)
timecode : 01:00:00:00

Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, 2 channels, s16, 1536 kb/s (default)
Metadata:

creation_time : 2013-12-11 15:38:12
handler_name : Apple Alias-Datensteuerung

Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:

creation_time : 2013-12-11 15:38:16
handler_name : Apple Alias-Datensteuerung
timecode : 01:00:00:00

Output #0, nut, to 'scale_test_bt601.nut':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
encoder : Lavf56.40.101
Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
Metadata:

creation_time : 2013-12-11 15:38:12
X-Language : eng
handler_name : Apple Alias-Datensteuerung
timecode : 01:00:00:00
encoder : Lavc56.55.100 ffv1

Stream mapping:

Stream #0:0 -> #0:0 (mpeg2video (native) -> ffv1 (native))

Press [q] to stop, ? for help
frame= 25 fps=0.0 q=-0.0 Lsize= 3715kB time=00:00:01.00 bitrate=30435.4kbits/s
video:3714kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.034393%
Roberts-MacBook-Pro:colortest krueger$ ffmpeg -i ~/lesspain/samples/software/fcp_7/xdcam_ex/fcp7_xdcam_ex_1080_25p_1s.mov -c:v ffv1 -an -y scale_test_unscaled.nut
ffmpeg version N-74034-gce46627 Copyright (c) 2000-2015 the FFmpeg developers

built with Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
configuration: --enable-gpl --enable-libx264
libavutil 54. 29.100 / 54. 29.100
libavcodec 56. 55.100 / 56. 55.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 29.100 / 5. 29.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fa1bc00da00] nclc: pri 1 trc 1 matrix 1Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/krueger/lesspain/samples/software/fcp_7/xdcam_ex/fcp7_xdcam_ex_1080_25p_1s.mov':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2013-12-11 15:38:12

Duration: 00:00:01.00, start: 0.000000, bitrate: 25365 kb/s

Stream #0:0(eng): Video: mpeg2video (Main) (xdve / 0x65766478), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 23773 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
Metadata:

creation_time : 2013-12-11 15:38:12
handler_name : Apple Alias-Datensteuerung
encoder : XDCAM EX 1080p25 (35 Mb/s VBR)
timecode : 01:00:00:00

Stream #0:1(eng): Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, 2 channels, s16, 1536 kb/s (default)
Metadata:

creation_time : 2013-12-11 15:38:12
handler_name : Apple Alias-Datensteuerung

Stream #0:2(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
Metadata:

creation_time : 2013-12-11 15:38:16
handler_name : Apple Alias-Datensteuerung
timecode : 01:00:00:00

Output #0, nut, to 'scale_test_unscaled.nut':

Metadata:

major_brand : qt
minor_version : 537199360
compatible_brands: qt
encoder : Lavf56.40.101
Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 51200 tbn, 25 tbc (default)
Metadata:

creation_time : 2013-12-11 15:38:12
X-Language : eng
handler_name : Apple Alias-Datensteuerung
timecode : 01:00:00:00
encoder : Lavc56.55.100 ffv1

Stream mapping:

Stream #0:0 -> #0:0 (mpeg2video (native) -> ffv1 (native))

Press [q] to stop, ? for help
frame= 25 fps=0.0 q=-0.0 Lsize= 3715kB time=00:00:01.00 bitrate=30435.4kbits/s
video:3714kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.034393%

comment:2 by Hendrik, 7 years ago

We have the colormatrix filter for this kind of conversions.

-vf colormatrix=src=bt709:dst=bt601

comment:3 by gjdfgh, 7 years ago

Hard to tell what's going on here without knowing the parameters set on libswscale, but I'd say this definitely should work. Without special filter.

It's quite possible that libswscale doesn't implement this at all, though.

in reply to:  2 comment:4 by rmk, 7 years ago

Replying to heleppkes:

We have the colormatrix filter for this kind of conversions.

-vf colormatrix=src=bt709:dst=bt601

I know but

  • it's GPL (in my case that means I cannot use it),
  • it's very limited as far as pixel format is concerned, compared to swscale
  • it's more limited as far as color space support is concerned, compared to swscale
  • it seems like a good idea to only do the maintenance for coefficients etc. in one place and since the swscale api contract (as well as the vf_scale options) seems well-suited for the task, it is probably surprising for users to find that the functionality is not in that place (although there might be reasons as far as code complexity is concerned, that I don't know, not really knowing the impl code)

I can only guess that the reason the colormatrix filter is there, was that someone else ran into this limitation, maybe knew the colormatrix support from ffmbc and decided to rather port than tackle the more complex task of making swscale support it (which is completely legitimate, of course).

comment:5 by Carl Eugen Hoyos, 7 years ago

Component: undeterminedswscale
Reproduced by developer: set
Status: newopen
Type: enhancementdefect
Version: unspecifiedgit-master

See also http://thread.gmane.org/gmane.comp.video.ffmpeg.user/57005/focus=57093 and my incomplete patch in http://ffmpeg.org/pipermail/ffmpeg-devel/2015-May/173223.html - the second hunk doesn't work for some reason, the first hunk now looks like this and should be necessary afaict:

diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index 9698bfb..be061e2 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -350,6 +350,7 @@ static int config_props(AVFilterLink *outlink)
     scale->isws[0] = scale->isws[1] = scale->sws = NULL;
     if (inlink0->w == outlink->w &&
         inlink0->h == outlink->h &&
+        scale->in_range == scale->out_range &&
         inlink0->format == outlink->format)
         ;
     else {

in reply to:  5 comment:6 by rmk, 7 years ago

Replying to cehoyos:

See also http://thread.gmane.org/gmane.comp.video.ffmpeg.user/57005/focus=57093

was this ever made into a ticket? because I think I just also ran into that.

comment:7 by rmk, 7 years ago

Created ticket 4812 for it.

comment:8 by Carl Eugen Hoyos, 7 years ago

Resolution: fixed
Status: openclosed

This was fixed by Michael in 8e05f9217ae567a26ea71dd29acd79b1631ae593 (and possibly a subsequent commit).

Thank you for the report, sorry about mixing up range and matrix here.

comment:9 by mirh, 3 months ago

Cc: mirh added

FWIW this only fixed the "manually specified" case though #9693

Note: See TracTickets for help on using tickets.