Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#4417 closed defect (invalid)

v210 is lossy with values below 0x00000001

Reported by: dericed Owned by:
Priority: important Component: avcodec
Version: git-master Keywords: v210 regression
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

The v210 encoder appears to be lossy for particular input sample values. I'm attaching a sample that demonstrates this. The file v210.mov represents a v210 frame as captured from a tape source via blackmagic software. The resulting 10 bit frame includes sample values such as 0x0000000000.

Since the input is v210 and the output is v210 I would hope the transcoding is lossless; however it isn't. Here's my encoding.

ffmpeg -i v210.mov -c:v v210 v210_2_v210.mov
ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-libfreetype --enable-libfaac --enable-libass --enable-ffplay --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.0/include ' --enable-nonfree --enable-vda
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'v210.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
  Duration: 00:00:00.03, start: 0.000000, bitrate: 219748 kb/s
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486, 223725 kb/s, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Blackmagic 10 Bit
Output #0, mov, to 'v210_2_v210.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486 [SAR 9:10 DAR 4:3], q=2-31, 200 kb/s, 29.97 fps, 30k tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.26.100 v210
Stream mapping:
  Stream #0:0 -> #0:0 (v210 (native) -> v210 (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=     912kB time=00:00:00.03 bitrate=223917.0kbits/s    
video:911kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.086806%

To compare md5 outputs, the original v210.mov file gives 8f0386913ae78d42b7639eb33d1a1e89.

ffmpeg -i v210.mov -f md5 -
ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-libfreetype --enable-libfaac --enable-libass --enable-ffplay --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.0/include ' --enable-nonfree --enable-vda
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'v210.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
  Duration: 00:00:00.03, start: 0.000000, bitrate: 219748 kb/s
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486, 223725 kb/s, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Blackmagic 10 Bit
Output #0, md5, to 'pipe:':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486 [SAR 9:10 DAR 4:3], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.26.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (v210 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
MD5=8f0386913ae78d42b7639eb33d1a1e89
frame=    1 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.03 bitrate=   8.9kbits/s    
video:1367kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

and the v210_2_v210.mov file gives a different output: 50c42834a0370266cb014e2e6e260c25

ffmpeg -i v210_2_v210.mov -f md5 -
ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.6.1 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-libfreetype --enable-libfaac --enable-libass --enable-ffplay --enable-libopenjpeg --disable-decoder=jpeg2000 --extra-cflags='-I/usr/local/Cellar/openjpeg/1.5.0/include ' --enable-nonfree --enable-vda
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'v210_2_v210.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
  Duration: 00:00:00.03, start: 0.000000, bitrate: 219748 kb/s
    Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le, 720x486, 223725 kb/s, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.26.100 v210
Output #0, md5, to 'pipe:':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.25.101
    Stream #0:0(eng): Video: rawvideo (Y3[10][10] / 0xA0A3359), yuv422p10le, 720x486 [SAR 9:10 DAR 4:3], q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.26.100 rawvideo
Stream mapping:
  Stream #0:0 -> #0:0 (v210 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
MD5=50c42834a0370266cb014e2e6e260c25
frame=    1 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.03 bitrate=   8.9kbits/s    
video:1367kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

An image the pronounces the difference between the two files can be made with:

ffmpeg -i v210.mov -i v210_2_v210.mov -filter_complex blend=all_mode=difference128,histeq difference.jpg

To investigate the difference, I made a histogram of the first 8 bits of the samples with this command:

ffmpeg -i v210.mov -pix_fmt yuv422p16be -f rawvideo - | xxd -b -c 2 | cut -c 10-17 | sort | uniq -c

For a histogram of values 0x00000000 to 0x00000111 on v210.mov I get:

5052 00000000
5720 00000001
1951 00000010
3777 00000011
11735 00000100
11730 00000101
8696 00000110
2900 00000111

and with v210_2_v210.mov, I get:

10772 00000001
1951 00000010
3777 00000011
11735 00000100
11730 00000101
8696 00000110
2900 00000111

From this I conclude that the v210 encoder is rounding up 0x00000000 to 0x00000001. I ran the same tests with other codecs that support yuv422p10le and did not have the same issue. For instance (if the input v210 contains a 0x00000000 values) then v210->ffv1 is lossless but v210->v210 isn't.

Note: By using v210 to v210 in the example above I'm hoping to simplify the ticket. I found this because I was transcoding v210 files to lossless codecs and then in some cases later needing to transcode the lossless back to v210 but was finding the differences noted here. Thanks to Brian Wheeler for helping identify this scenario.

Attachments (2)

v210.mov (912.0 KB) - added by dericed 4 years ago.
v210 frame captured from tape
v210_2_v210.mov (912.0 KB) - added by dericed 4 years ago.
v210.mov transcoded to v210 via ffmpeg

Download all attachments as: .zip

Change History (10)

Changed 4 years ago by dericed

v210 frame captured from tape

Changed 4 years ago by dericed

v210.mov transcoded to v210 via ffmpeg

comment:1 Changed 4 years ago by cehoyos

  • Priority changed from critical to normal

comment:2 Changed 4 years ago by kierank

Hi Dave,

This is described in the Apple spec:
https://developer.apple.com/library/mac/technotes/tn2162/_index.html#//apple_ref/doc/uid/DTS40013070-CH1-TNTAG8-V210__4_2_2_COMPRESSION_TYPE

"This compression type uses scheme B ("Video-Range" Mapping with Unsigned Y´, Offset Binary Cb, Cr) to get from EY´, ECb, and ECr to Y´, Cb, and Cr."

"Certain Y´, Cb, and Cr component values v are reserved as synchronization signals and must not appear in a buffer:

...

For n=8 bits, these are values 0 and 255. For n=10 bits, these are values 0, 1, 2, 3, 1020, 1021, 1022, and 1023. The writer of a QuickTime? image is responsible for omitting these values. The reader of a QuickTime? image may assume that they are not present."

Therefore the code doesn't allow these values.

Kieran

comment:3 Changed 4 years ago by cehoyos

  • Keywords regression added
  • Priority changed from normal to important

If there is an issue, it is a regression since 74bf9d62
Some values in your input file are apparently not compliant to "specifications" and have to be clipped to produce v210 output "according to specifications".

https://developer.apple.com/library/mac/technotes/tn2162/_index.html#//apple_ref/doc/uid/DTS40013070-CH1-TNTAG7-SCHEME_B___VIDEO_RANGE__MAPPING_WITH_UNSIGNED_Y____OFFSET_BINARY_CB__CR

comment:4 Changed 4 years ago by dericed

Argh, this is not the outcome I anticipated, but apparently ffmpeg is doing the "right thing" by rounding some values up or down. I do wonder though is Apple's documentation is THE official specification for v210 or if somehow v210 is standardized by multiple entities that vary in uses this video range rule or not. But it does look like 74bf9d62 is more of a bug fix than a regression.
Perhaps I should convert this ticket to an enhancement request to support an encoding option to create non-spec v210 that preserves the input samples rather than rounds them. Then users would default to "spec v210" but could opt for "lossless v210" depending on their needs.

comment:5 Changed 4 years ago by cehoyos

The problem with the option is that it will make encoding very significantly slower if used: The asm optimisation is doing the clipping, and we will not pass the (future) option to the asm code.

comment:6 Changed 4 years ago by dericed

Maybe a picky issue but the Apple documentation is documenting "Uncompressed Y´CbCr? Video in QuickTime? Files". So while the rules therein may refer to v210 in mov, perhaps v210 is avi is not under the same conditions. I've been digging though msdn but haven't found a comparable v210 spec there.

Though slow I would still be interested in an option to run a lossless v210, but this would be low priority in my opinion.

comment:7 Changed 4 years ago by dericed

  • Resolution set to needs_more_info
  • Status changed from new to closed

I'm going to close this ticket for now, as (at least for QuickTime?) v210 is supposed to be lossy. If I can find any documentation that v210 is not supposed to be lossy in other containers I'll re-open.

comment:8 Changed 4 years ago by cehoyos

  • Resolution changed from needs_more_info to invalid
Note: See TracTickets for help on using tickets.