Opened 3 years ago

Last modified 10 months ago

#2690 open enhancement

Red Cinema r3d files not supported

Reported by: MilosL Owned by:
Priority: wish Component: avcodec
Version: git-master Keywords: r3d j2k
Cc: elliottbalsley@gmail.com Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

Summary of the bug:
I have tested it with more files from different sources, on fresh compiled git version of ffmpeg, on ffmpeg compiled without "libopenjpeg", but it doesn't work - R3d files from RED Cinema cannot be decoded.

[me@localhost modules]# ffmpeg -v 9 -loglevel 99 -i /me/Desktop/A002_C006_0205V7_001.R3D
ffmpeg version 1.0.7 Copyright (c) 2000-2013 the FFmpeg developers

built on May 14 2013 21:59:35 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-frei0r --enable-gnutls --enable-libass --enable-libcdio --enable-libcelt --enable-libdc1394 --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 59.100 / 54. 59.100
libavformat 54. 29.104 / 54. 29.104
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100

[r3d @ 0x7c8780] Format r3d probed with size=2048 and score=100
[r3d @ 0x7c8780] File position before avformat_find_stream_info() is 324
[libopenjpeg @ 0x7d6c80] Error decoding codestream.

Last message repeated 3 times

[r3d @ 0x7c8780] Probe buffer size limit 5000000 reached
[r3d @ 0x7c8780] decoding for stream 0 failed
[r3d @ 0x7c8780] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[r3d @ 0x7c8780] File position after avformat_find_stream_info() is 6494364
/root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters
[AVIOContext @ 0x7d7e20] Statistics: 6529424 bytes read, 3 seeks

Change History (29)

comment:1 Changed 3 years ago by cehoyos

Please test (and provide console output) for current git head and please provide a sample.

comment:2 Changed 3 years ago by cehoyos

  • Component changed from FFmpeg to undetermined
  • Keywords red removed

comment:3 Changed 3 years ago by MilosL

Sample file uploaded via ftp: A002_C024_0205C9_001.R3D
RedCineX application says under metadata: Redcode: 9:1(RedCode36), noHDR). File was downloaded anywhere in the net. If needed, i can provide more samples.

Console output:

# /usr/local/bin/ffmpeg -v 9 -loglevel 99 -i /root/Desktop/A002_C006_0205V7_001.R3D
ffmpeg version N-54115-g37baecc Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun 20 2013 01:31:03 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8)
  configuration: 
  libavutil      52. 37.101 / 52. 37.101
  libavcodec     55. 16.100 / 55. 16.100
  libavformat    55.  9.100 / 55.  9.100
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 77.101 /  3. 77.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input file with argument '/root/Desktop/A002_C006_0205V7_001.R3D'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input file /root/Desktop/A002_C006_0205V7_001.R3D.
Successfully parsed a group of options.
Opening an input file: /root/Desktop/A002_C006_0205V7_001.R3D.
[r3d @ 0x25f5060] Format r3d probed with size=2048 and score=100
[r3d @ 0x25f5060] File position before avformat_find_stream_info() is 324
[jpeg2000 @ 0x25f5a80] SOC marker not present
    Last message repeated 3 times
[r3d @ 0x25f5060] Probe buffer size limit of 5000000 bytes reached
[r3d @ 0x25f5060] decoding for stream 0 failed
[r3d @ 0x25f5060] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[r3d @ 0x25f5060] File position after avformat_find_stream_info() is 6494364
/root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters
[AVIOContext @ 0x25fd780] Statistics: 6529424 bytes read, 3 seeks
Last edited 3 years ago by cehoyos (previous) (diff)

comment:4 follow-up: Changed 3 years ago by cehoyos

Last edited 3 years ago by cehoyos (previous) (diff)

comment:5 Changed 3 years ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords j2k added
  • Priority changed from normal to wish
  • Reproduced by developer set
  • Status changed from new to open
  • Type changed from defect to enhancement
  • Version changed from 1.0.7 to git-master

comment:6 in reply to: ↑ 4 Changed 3 years ago by cehoyos

Replying to cehoyos:

Your console output looks incomplete / severely edited, please provide the complete, uncut console output.

Sorry about this, I misread the console output.

comment:7 follow-up: Changed 3 years ago by MilosL

Sorry, is there a timetable for red support? R3d is listed as supported decoder - so for me it's look more like bug as enhancement or wish. Can i help in any way?

comment:8 in reply to: ↑ 7 Changed 3 years ago by cehoyos

Replying to MilosL:

Can i help in any way?

A patch fixing the ticket - sent to ffmpeg-devel - is very welcome!

comment:9 Changed 3 years ago by cehoyos

After reading these links:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78598
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78938
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/80922
Old version ("3") r3d files are "supported" insofar as they can be demuxed but since FFmpeg currently does not (yet) support Bayer colourspaces, the files cannot be decoded.
The file you uploaded is version "4", either this version needs a patch to the demuxer or to the decoder, in any case, that wouldn't fix the missing Bayer support.

Last edited 3 years ago by cehoyos (previous) (diff)

comment:10 follow-up: Changed 3 years ago by spookybathtub

Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf
I think this would be preferable to rewriting it from scratch, because the R3D format is under such heavy development by RED, and it would be too hard to keep up with the frequent updates to their color science, etc.

comment:11 in reply to: ↑ 10 Changed 3 years ago by cehoyos

Replying to spookybathtub:

Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf

Is it compatible with the GPL or the LGPL in your opinion?

comment:12 Changed 3 years ago by spookybathtub

I think it would be compatible with LGPL, so you can avoid distributing their source code. But this legal stuff is way over my head, so I'm not sure.

comment:13 follow-up: Changed 3 years ago by spookybathtub

This is the response I got from RED's legal team:

You are not allowed to distribute any of the R3D SDK documentation, header files, sample code or library files directly. This means you would have to create either a custom ffmpeg binary people can download, or provide some sort of binary plugin if ffmpeg has such a system.

Is this doable?

comment:14 in reply to: ↑ 13 Changed 3 years ago by cehoyos

Replying to spookybathtub:

Is this doable?

Don't you agree that it would be much simpler to get the (existing) Bayer patch(es) into FFmpeg and update the demuxer?

Note that if it would be legal to distribute such a binary of FFmpeg (I think this is unlikely but ianal) it would not be legal to have x264 support (which is what most people want).

comment:15 follow-up: Changed 3 years ago by spookybathtub

  • Cc elliottbalsley@gmail.com added

I'm not sure which would be simpler. I don't know how to code any of this myself, but I am very grateful for all the work done by the ffmpeg developers so far.
I do know that the SDK provides a huge amount of options for how to process R3D raw files, and many of these options are essential when dealing with RAW — for example you have to set the ISO, white balance, etc. before converting to a linear color space. It seems like it would be rather difficult to recreate all these settings.
I've read that about 3 years ago, RED started using some kind of encryption in the files. See http://ffmpeg.org/pipermail/ffmpeg-user/2011-September/002444.html

As an alternative, RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg. If anyone else wants to try, you can get it here: https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.

I agree it would not be practical to distribute crippled ffmpeg binaries, but is it possible to make some kind of "binary plugin" that doesn't reveal the source code, but can be used when compiling ffmpeg?

comment:16 in reply to: ↑ 15 Changed 3 years ago by cehoyos

Replying to spookybathtub:

RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg.

Why?
Consider discussing this on ffmpeg-user.

If anyone else wants to try, you can get it here: https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.

Why??

comment:17 follow-up: Changed 3 years ago by spookybathtub

R3D is an extremely complex codec, with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED. Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.

The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch. In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately. But maybe I'm wrong.

comment:18 in reply to: ↑ 17 ; follow-up: Changed 3 years ago by ubitux

Replying to spookybathtub:

R3D is an extremely complex codec

codec? Isn't it just a relatively simple format with jpeg2000 in it?

with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED.

Check out all the color filters in libavfilter, including lut3d.

Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.

I wonder what this has to do with the software implementation.

The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch.

Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)

In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately.

lol

But maybe I'm wrong.

Yes.

comment:19 in reply to: ↑ 18 Changed 3 years ago by cehoyos

Replying to ubitux:

Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)

The pixel format is Bayer, a patch was sent but is unfinished.

comment:20 Changed 3 years ago by spookybathtub

Yes, it's a bayer pattern, and it does use a modified JPEG2000 wavelet compression. But the debayered image is raw data with no specific color space until it is converted with RedColor3 or some other algorithm.

comment:21 follow-up: Changed 2 years ago by compn

bayer colorspace support was added a few months ago.
http://git.videolan.org/?p=ffmpeg.git&a=search&h=HEAD&st=commit&s=bayer
whats next to support .r3d ?

comment:22 in reply to: ↑ 21 Changed 11 months ago by cehoyos

Replying to compn:

bayer colorspace support was added a few months ago.

FFmpeg supports bayer formats with 8 bit per pixel and 16 bit per pixel, the r3d formats have 12 bit per component (48 bit per pixel).

comment:23 Changed 11 months ago by spookybathtub

That used to be true, but the current cameras are 16 bit.
Source: https://support.red.com/entries/27935746-Is-the-RED-DRAGON-sensor-16-bit-

comment:24 Changed 11 months ago by compn

do these 16bit bayer colorspaces have to be reverse engineered ?

can someone upload a 16bit r3d file for testing with? possibly with the original input so we can make sure its identical decoding.

comment:25 Changed 11 months ago by spookybathtub

I can upload a sample. What do you mean by "original input"?

comment:26 Changed 11 months ago by compn

something so we know what the input looks like. correct colors and stuff. screenshot etc

comment:27 Changed 11 months ago by cehoyos

Please find the requested samples in http://samples.ffmpeg.org/r3d/

comment:28 Changed 10 months ago by spookybathtub

Those samples are ancient, and not 16 bit. I don't know the best way to upload samples here, but here is a recent sample from RED Dragon, along with two JPEG references — one debayered as REDlogFilm, and the other as REDgamma4. Both use DRAGONcolor2. Note: The JPEG preview on Google Drive does not look right at all, probably because it can't handle this high resolution (6144x3160).

https://drive.google.com/file/d/0B52QuT8oHvtZa2pRUktwb0JUVlE/view?usp=sharing
https://drive.google.com/file/d/0B52QuT8oHvtZNnBVMGpJQlRzNUk/view?usp=sharing
https://drive.google.com/file/d/0B52QuT8oHvtZek03NUdLdHlZazQ/view?usp=sharing

comment:29 Changed 10 months ago by compn

thank you spooky, i have mirrored your new samples here
http://samples.ffmpeg.org/r3d/

please keep pinging devs until this r3d stuff gets supported. :)

Note: See TracTickets for help on using tickets.