10bits grayscale DPX not correctly transcoded

[yohann]

Summary of the bug:
I am trying to transcode a 10 bit grayscale DPX image sequence to FFV1. The picture's output is distorted, pixel rows are skewed/shifted even though no transcoding error is reported.

It might be an issue with how the DPX are interpreted as the issue also occurs if I try to output the picture to any other video codec/still image format.

The image sequence is made by a Lasergraphics Director film scanner.
Tried on macOS High Sierra and Arch Linux with v4.2.1 of ffmpeg. A coworker has also been able to reproduce the phenomenon on Windows 10.

Command line for FFV1 output:

% ffmpeg -v 9 -loglevel 99 -report \
-i "20731272-DPX (2560x1920)-0086711.dpx" \
-c:v ffv1 -level 3 -g 1 -slicecrc 1 -slices 16 output.mkv

Log for FFV1 output (attached file):

ffmpeg started on 2019-11-20 at 14:29:34
Report written to "ffmpeg-20191120-142934.log"
ffmpeg version n4.2.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-i' ... matched as input url with argument '20731272-DPX (2560x1920)-0086711.dpx'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'ffv1'.
Reading option '-level' ... matched as AVOption 'level' with argument '3'.
Reading option '-g' ... matched as AVOption 'g' with argument '1'.
Reading option '-slicecrc' ... matched as AVOption 'slicecrc' with argument '1'.
Reading option '-slices' ... matched as AVOption 'slices' with argument '16'.
Reading option 'output.mkv' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url 20731272-DPX (2560x1920)-0086711.dpx.
Successfully parsed a group of options.
Opening an input file: 20731272-DPX (2560x1920)-0086711.dpx.
[NULL @ 0x556ec08935c0] Opening '20731272-DPX (2560x1920)-0086711.dpx' for reading
[file @ 0x556ec0894040] Setting default whitelist 'file,crypto'
Probing image2 score:50 size:2048
Probing dpx_pipe score:51 size:2048
[dpx_pipe @ 0x556ec08935c0] Format dpx_pipe probed with size=2048 and score=51
[dpx_pipe @ 0x556ec08935c0] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[dpx_pipe @ 0x556ec08935c0] Probe buffer size limit of 5000000 bytes reached
[dpx_pipe @ 0x556ec08935c0] Stream #0: not enough frames to estimate rate; consider increasing probesize
[dpx_pipe @ 0x556ec08935c0] stream 0: start_time: -368934881474191040.000 duration: -368934881474191040.000
[dpx_pipe @ 0x556ec08935c0] format: start_time: -9223372036854.775 duration: -9223372036854.775 bitrate=0 kb/s
[dpx_pipe @ 0x556ec08935c0] After avformat_find_stream_info() pos: 6561792 bytes read:6561792 seeks:0 frames:1
Input #0, dpx_pipe, from '20731272-DPX (2560x1920)-0086711.dpx':
  Duration: N/A, bitrate: N/A
    Stream #0:0, 1, 1/25: Video: dpx, 1 reference frame, gray10le, 2560x1920, 0/1, 24 tbr, 25 tbn, 24 tbc
Successfully opened the file.
Parsing a group of options: output url output.mkv.
Applying option c:v (codec name) with argument ffv1.
Successfully parsed a group of options.
Opening an output file: output.mkv.
[file @ 0x556ec089b0c0] Setting default whitelist 'file,crypto'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (dpx (native) -> ffv1 (native))
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
detected 40 logical cores
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'video_size' to value '2560x1920'
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'pix_fmt' to value '171'
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'time_base' to value '1/25'
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x556ec1255880] Setting 'frame_rate' to value '24/1'
[graph 0 input from stream 0:0 @ 0x556ec1255880] w:2560 h:1920 pixfmt:gray10le tb:1/25 fr:24/1 sar:0/1 sws_param:flags=2
[format @ 0x556ec12560c0] Setting 'pix_fmts' to value 'yuv420p|yuva420p|yuva422p|yuv444p|yuva444p|yuv440p|yuv422p|yuv411p|yuv410p|bgr0|bgra|yuv420p16le|yuv422p16le|yuv444p16le|yuv444p9le|yuv422p9le|yuv420p9le|yuv420p10le|yuv422p10le|yuv444p10le|yuv420p12le|yuv422p12le|yuv444p12le|yuva444p16le|yuva422p16le|yuva420p16le|yuva444p10le|yuva422p10le|yuva420p10le|yuva444p9le|yuva422p9le|yuva420p9le|gray16le|gray|gbrp9le|gbrp10le|gbrp12le|gbrp14le|gbrap10le|gbrap12le|ya8|gray10le|gray12le|gbrp16le|rgb48le|gbrap16le|rgba64le|gray9le|yuv420p14le|yuv422p14le|yuv444p14le|yuv440p10le|yuv440p12le'
[AVFilterGraph @ 0x556ec0893340] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[ffv1 @ 0x556ec0899ac0] bits_per_raw_sample > 8, forcing range coder
[matroska @ 0x556ec0897fc0] get_metadata_duration returned: 0
Output #0, matroska, to 'output.mkv':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0, 0, 1/1000: Video: ffv1, 1 reference frame (FFV1 / 0x31564646), gray10le, 2560x1920, 0/1, q=2-31, 200 kb/s, 24 fps, 1k tbn, 24 tbc
    Metadata:
      encoder         : Lavc58.54.100 ffv1
Clipping frame in rate conversion by 0.000008
[matroska @ 0x556ec0897fc0] Writing block of size 3345085 with pts 0, dts 0, duration 42 at relative offset 9 in cluster at offset 895. TrackNumber 1, keyframe 1
[out_0_0 @ 0x556ec1256840] EOF on sink link out_0_0:default.
No more output streams to write to, finishing.
[matroska @ 0x556ec0897fc0] end duration = 42
[matroska @ 0x556ec0897fc0] stream 0 end duration = 42
frame=    1 fps=0.0 q=-0.0 Lsize=    3268kB time=00:00:00.00 bitrate=26768272.0kbits/s speed=0.00199x
video:3267kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.028370%
Input file #0 (20731272-DPX (2560x1920)-0086711.dpx):
  Input stream #0:0 (video): 1 packets read (6561792 bytes); 1 frames decoded;
  Total: 1 packets (6561792 bytes) demuxed
Output file #0 (output.mkv):
  Output stream #0:0 (video): 1 frames encoded; 1 packets muxed (3345085 bytes);
  Total: 1 packets (3345085 bytes) muxed
1 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x556ec089b440] Statistics: 8 seeks, 19 writeouts
[AVIOContext @ 0x556ec089c500] Statistics: 6561792 bytes read, 0 seeks

[yohann]

log output

[yohann]

jpg output

[yohann]

The original DPX is too large to be uploaded here, I made a link where you can download it and the ticket related files :
​https://cloud.cfav.fr/index.php/s/x8x27npz27W2L95

File list details (renamed so they can be more explicit) :

• Original DPX : 10bit_grayscale_20731272-DPX (2560x1920)-0086711.dpx
• FFV1 outputted file : FFV1_skewed_rows_output.mkv
• Log file of FFV1 encoding : ffmpeg-20191120-142934.log
• JPG outputted file for preview purposes : JPG_skewed_rows_output.jpg

[Carl Eugen Hoyos]

Two possible patches sent, the related tickets are #2392 and #4409.

For future tickets, please remember that only current FFmpeg git head is supported.

[yohann]

Thanks for the input, same issue occurs with the current snapshot build.

[Carl Eugen Hoyos]

Does the scanner also allow to produce a 10bit RGB file?
Please provide such a sample with the same resolution.

[yohann]

I just uploaded a sample which you can download on the ​same link I previously provided.
The file name is 10bit_rgb_2560x1920.dpx.

[Carl Eugen Hoyos]

Can you choose the horizontal resolution on the scanner? Is a horizontal resolution of 2048 as for the sample from ticket #2392 possible?

[Carl Eugen Hoyos]

Sorry, I don't think that would help either: Is an additional fourth channel (with infrared / transparency information) with the original resolution possible?

[yohann]

Replying to cehoyos:

Sorry, I don't think that would help either: Is an additional fourth channel (with infrared / transparency information) with the original resolution possible?

The scanner has very few options in that regard, it's either grayscale or RGB for the colorspace. No fourth channel option.

(By the way, the scanner model is Lasergraphics Scanstation and not Director as I first thought)

[Balling]

Fixed in 331858d0f68324edd17307eee2dcae1abfd5d302 (tested before & after).