Red Cinema r3d files not supported

[MilosL]

Summary of the bug:
I have tested it with more files from different sources, on fresh compiled git version of ffmpeg, on ffmpeg compiled without "libopenjpeg", but it doesn't work - R3d files from RED Cinema cannot be decoded.

[me@localhost modules]# ffmpeg -v 9 -loglevel 99 -i /me/Desktop/A002_C006_0205V7_001.R3D
ffmpeg version 1.0.7 Copyright (c) 2000-2013 the FFmpeg developers

built on May 14 2013 21:59:35 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-frei0r --enable-gnutls --enable-libass --enable-libcdio --enable-libcelt --enable-libdc1394 --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 59.100 / 54. 59.100
libavformat 54. 29.104 / 54. 29.104
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100

[r3d @ 0x7c8780] Format r3d probed with size=2048 and score=100
[r3d @ 0x7c8780] File position before avformat_find_stream_info() is 324
[libopenjpeg @ 0x7d6c80] Error decoding codestream.

Last message repeated 3 times

[r3d @ 0x7c8780] Probe buffer size limit 5000000 reached
[r3d @ 0x7c8780] decoding for stream 0 failed
[r3d @ 0x7c8780] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[r3d @ 0x7c8780] File position after avformat_find_stream_info() is 6494364
/root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters
[AVIOContext @ 0x7d7e20] Statistics: 6529424 bytes read, 3 seeks

[Carl Eugen Hoyos]

Please test (and provide console output) for current git head and please provide a sample.

[MilosL]

Sample file uploaded via ftp: A002_C024_0205C9_001.R3D
RedCineX application says under metadata: Redcode: 9:1(RedCode36), noHDR). File was downloaded anywhere in the net. If needed, i can provide more samples.

Console output:

# /usr/local/bin/ffmpeg -v 9 -loglevel 99 -i /root/Desktop/A002_C006_0205V7_001.R3D
ffmpeg version N-54115-g37baecc Copyright (c) 2000-2013 the FFmpeg developers
  built on Jun 20 2013 01:31:03 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8)
  configuration: 
  libavutil      52. 37.101 / 52. 37.101
  libavcodec     55. 16.100 / 55. 16.100
  libavformat    55.  9.100 / 55.  9.100
  libavdevice    55.  2.100 / 55.  2.100
  libavfilter     3. 77.101 /  3. 77.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'.
Reading option '-i' ... matched as input file with argument '/root/Desktop/A002_C006_0205V7_001.R3D'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument 9.
Successfully parsed a group of options.
Parsing a group of options: input file /root/Desktop/A002_C006_0205V7_001.R3D.
Successfully parsed a group of options.
Opening an input file: /root/Desktop/A002_C006_0205V7_001.R3D.
[r3d @ 0x25f5060] Format r3d probed with size=2048 and score=100
[r3d @ 0x25f5060] File position before avformat_find_stream_info() is 324
[jpeg2000 @ 0x25f5a80] SOC marker not present
    Last message repeated 3 times
[r3d @ 0x25f5060] Probe buffer size limit of 5000000 bytes reached
[r3d @ 0x25f5060] decoding for stream 0 failed
[r3d @ 0x25f5060] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[r3d @ 0x25f5060] File position after avformat_find_stream_info() is 6494364
/root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters
[AVIOContext @ 0x25fd780] Statistics: 6529424 bytes read, 3 seeks

[Carl Eugen Hoyos]

Replying to cehoyos:

Your console output looks incomplete / severely edited, please provide the complete, uncut console output.

Sorry about this, I misread the console output.

[MilosL]

Sorry, is there a timetable for red support? R3d is listed as supported decoder - so for me it's look more like bug as enhancement or wish. Can i help in any way?

[Carl Eugen Hoyos]

Replying to MilosL:

Can i help in any way?

A patch fixing the ticket - sent to ffmpeg-devel - is very welcome!

[Carl Eugen Hoyos]

After reading these links:
​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78598
​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78938
​http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/80922
Old version ("3") r3d files are "supported" insofar as the can be demuxed but since FFmpeg currently does not (yet) support Bayer colourspaces, the files cannot be decoded.
The file you uploaded is version "4", either this version needs a patch to the demuxer or to the decoder, in any case, that wouldn't fix the missing Bayer support.

[Elliott]

Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at ​http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf
I think this would be preferable to rewriting it from scratch, because the R3D format is under such heavy development by RED, and it would be too hard to keep up with the frequent updates to their color science, etc.

[Carl Eugen Hoyos]

Replying to spookybathtub:

Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at ​http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf

Is it compatible with the GPL or the LGPL in your opinion?

[Elliott]

I think it would be compatible with LGPL, so you can avoid distributing their source code. But this legal stuff is way over my head, so I'm not sure.

[Elliott]

This is the response I got from RED's legal team:

You are not allowed to distribute any of the R3D SDK documentation, header files, sample code or library files directly. This means you would have to create either a custom ffmpeg binary people can download, or provide some sort of binary plugin if ffmpeg has such a system.

Is this doable?

[Carl Eugen Hoyos]

Replying to spookybathtub:

Is this doable?

Don't you agree that it would be much simpler to get the (existing) Bayer patch(es) into FFmpeg and update the demuxer?

Note that if it would be legal to distribute such a binary of FFmpeg (I think this is unlikely but ianal) it would not be legal to have x264 support (which is what most people want).

[Elliott]

I'm not sure which would be simpler. I don't know how to code any of this myself, but I am very grateful for all the work done by the ffmpeg developers so far.
I do know that the SDK provides a huge amount of options for how to process R3D raw files, and many of these options are essential when dealing with RAW — for example you have to set the ISO, white balance, etc. before converting to a linear color space. It seems like it would be rather difficult to recreate all these settings.
I've read that about 3 years ago, RED started using some kind of encryption in the files. See ​http://ffmpeg.org/pipermail/ffmpeg-user/2011-September/002444.html

As an alternative, RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg. If anyone else wants to try, you can get it here: ​https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.

I agree it would not be practical to distribute crippled ffmpeg binaries, but is it possible to make some kind of "binary plugin" that doesn't reveal the source code, but can be used when compiling ffmpeg?

[Carl Eugen Hoyos]

Replying to spookybathtub:

RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg.

Why?
Consider discussing this on ffmpeg-user.

If anyone else wants to try, you can get it here: ​https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.

Why??

[Elliott]

R3D is an extremely complex codec, with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED. Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.

The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch. In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately. But maybe I'm wrong.

[Clément Bœsch]

Replying to spookybathtub:

R3D is an extremely complex codec

codec? Isn't it just a relatively simple format with jpeg2000 in it?

with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED.

Check out all the color filters in libavfilter, including lut3d.

Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.

I wonder what this has to do with the software implementation.

The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch.

Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)

In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately.

lol

But maybe I'm wrong.

Yes.

[Carl Eugen Hoyos]

Replying to ubitux:

Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)

The pixel format is Bayer, a patch was sent but is unfinished.

[Elliott]

Yes, it's a bayer pattern, and it does use a modified JPEG2000 wavelet compression. But the debayered image is raw data with no specific color space until it is converted with RedColor3 or some other algorithm.

[compn]

bayer colorspace support was added a few months ago.
​http://git.videolan.org/?p=ffmpeg.git&a=search&h=HEAD&st=commit&s=bayer
whats next to support .r3d ?

[Carl Eugen Hoyos]

Replying to compn:

bayer colorspace support was added a few months ago.

FFmpeg supports bayer formats with 8 bit per pixel and 16 bit per pixel, the r3d formats have 12 bit per component (48 bit per pixel).

[Elliott]

That used to be true, but the current cameras are 16 bit.
Source: ​https://support.red.com/entries/27935746-Is-the-RED-DRAGON-sensor-16-bit-

[compn]

do these 16bit bayer colorspaces have to be reverse engineered ?

can someone upload a 16bit r3d file for testing with? possibly with the original input so we can make sure its identical decoding.

[Elliott]

I can upload a sample. What do you mean by "original input"?

[compn]

something so we know what the input looks like. correct colors and stuff. screenshot etc

[Carl Eugen Hoyos]

Please find the requested samples in ​http://samples.ffmpeg.org/r3d/

[Elliott]

Those samples are ancient, and not 16 bit. I don't know the best way to upload samples here, but here is a recent sample from RED Dragon, along with two JPEG references — one debayered as REDlogFilm, and the other as REDgamma4. Both use DRAGONcolor2. Note: The JPEG preview on Google Drive does not look right at all, probably because it can't handle this high resolution (6144x3160).

​https://drive.google.com/file/d/0B52QuT8oHvtZa2pRUktwb0JUVlE/view?usp=sharing
​https://drive.google.com/file/d/0B52QuT8oHvtZNnBVMGpJQlRzNUk/view?usp=sharing
​https://drive.google.com/file/d/0B52QuT8oHvtZek03NUdLdHlZazQ/view?usp=sharing

[compn]

thank you spooky, i have mirrored your new samples here
​http://samples.ffmpeg.org/r3d/

please keep pinging devs until this r3d stuff gets supported. :)

[boxerab]

Are there plans to support new R3D formats ? I tried transcoding a file from here

​http://www.red.com/sample-r3d-files

but I get an "invalid data found" error

These are 8K reels.

[boxerab]

ffmpeg command line was

ffmpeg -i foo.r3d -vcodec libx264 -preset slow bar.mp4

with ffmpeg version 2.2.14

[boxerab]

woops, old version of ffmpeg!
Checked with latest version, but same error message

[Ewout]

Still not supported with build ffmpeg-20181028-bdfd2e3

C:\ffmpeg\bin\ffmpeg -i C:\R3D_8K.R3D -c:v libaom-av1 -crf 30 -b:v 0 -strict experimental av1_test_8K_R3D.mkv
ffmpeg version N-92288-gbdfd2e3c79 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20181017

C:\R3D_8K.R3D: Invalid data found when processing input

[ahmetcetin]

Is there any news/plans supporting r3d formats? Red cameras are already widely popular.

[Balling]

Just will add big list of videos (even with 120 Hz)

​https://www.red.com/sample-r3d-files

[twosocks]

Ten years have passed...

Still not supported with ffmpeg-6.0

[Peter Kovář]

I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4

[Peter Kovář]

Compressed files are available for evaluation at:

​https://video.reflexion.tv/HEVC/

​https://video.reflexion.tv/VVC/

Anyone willing to participate?

[Camille]

Replying to Peter Kovář:

I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4

Hi Peter,

Your PoC sounds promising!

I have a few queries:

• Would it be feasible to concatenate spanned R3D clips and incorporate an audio track from a WAV file?
• Is there a possibility of integrating your code directly into FFmpeg?

We are in the midst of developing a server encoding application tailored for raw camera files, including XDCAM and R3D. However, the absence of comprehensive R3D support in FFmpeg poses certain limitations. Given the importance of this feature for our project, we're considering sponsoring its development.

One of my primary concerns revolves around potential copyright challenges, especially when implementing the RED SDK in an open-source framework.

I'd appreciate your insights on these matters.

[Justin Taylor]

Only just realized that Red footage wasn't compatible with ffmpeg today, but sounds like it's been an ongoing issue for some time.

Any updates on this one?

[Elon Musk]

No, and using closed source SDK is not welcomed.

[Peter Kovář]

Replying to Camille:

Replying to Peter Kovář:

I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4

R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4

Hi Peter,

Your PoC sounds promising!

I have a few queries:

• Would it be feasible to concatenate spanned R3D clips and incorporate an audio track from a WAV file?
• Is there a possibility of integrating your code directly into FFmpeg?

We are in the midst of developing a server encoding application tailored for raw camera files, including XDCAM and R3D. However, the absence of comprehensive R3D support in FFmpeg poses certain limitations. Given the importance of this feature for our project, we're considering sponsoring its development.

One of my primary concerns revolves around potential copyright challenges, especially when implementing the RED SDK in an open-source framework.

I'd appreciate your insights on these matters.

Camille, could you contact me via ​peter.kovar@reflexion.tv?