Opened 11 years ago
Last modified 9 months ago
#2690 open enhancement
Red Cinema r3d files not supported
Reported by: | MilosL | Owned by: | |
---|---|---|---|
Priority: | wish | Component: | avcodec |
Version: | git-master | Keywords: | r3d j2k |
Cc: | elliottbalsley@gmail.com | Blocked By: | |
Blocking: | Reproduced by developer: | yes | |
Analyzed by developer: | no |
Description
Summary of the bug:
I have tested it with more files from different sources, on fresh compiled git version of ffmpeg, on ffmpeg compiled without "libopenjpeg", but it doesn't work - R3d files from RED Cinema cannot be decoded.
[me@localhost modules]# ffmpeg -v 9 -loglevel 99 -i /me/Desktop/A002_C006_0205V7_001.R3D
ffmpeg version 1.0.7 Copyright (c) 2000-2013 the FFmpeg developers
built on May 14 2013 21:59:35 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-frei0r --enable-gnutls --enable-libass --enable-libcdio --enable-libcelt --enable-libdc1394 --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 59.100 / 54. 59.100
libavformat 54. 29.104 / 54. 29.104
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100
[r3d @ 0x7c8780] Format r3d probed with size=2048 and score=100
[r3d @ 0x7c8780] File position before avformat_find_stream_info() is 324
[libopenjpeg @ 0x7d6c80] Error decoding codestream.
Last message repeated 3 times
[r3d @ 0x7c8780] Probe buffer size limit 5000000 reached
[r3d @ 0x7c8780] decoding for stream 0 failed
[r3d @ 0x7c8780] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[r3d @ 0x7c8780] File position after avformat_find_stream_info() is 6494364
/root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters
[AVIOContext @ 0x7d7e20] Statistics: 6529424 bytes read, 3 seeks
Change History (42)
comment:1 by , 11 years ago
comment:2 by , 11 years ago
Component: | FFmpeg → undetermined |
---|---|
Keywords: | red removed |
comment:3 by , 11 years ago
Sample file uploaded via ftp: A002_C024_0205C9_001.R3D
RedCineX application says under metadata: Redcode: 9:1(RedCode36), noHDR). File was downloaded anywhere in the net. If needed, i can provide more samples.
Console output:
# /usr/local/bin/ffmpeg -v 9 -loglevel 99 -i /root/Desktop/A002_C006_0205V7_001.R3D ffmpeg version N-54115-g37baecc Copyright (c) 2000-2013 the FFmpeg developers built on Jun 20 2013 01:31:03 with gcc 4.7.2 (GCC) 20121109 (Red Hat 4.7.2-8) configuration: libavutil 52. 37.101 / 52. 37.101 libavcodec 55. 16.100 / 55. 16.100 libavformat 55. 9.100 / 55. 9.100 libavdevice 55. 2.100 / 55. 2.100 libavfilter 3. 77.101 / 3. 77.101 libswscale 2. 3.100 / 2. 3.100 libswresample 0. 17.102 / 0. 17.102 Splitting the commandline. Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'. Reading option '-i' ... matched as input file with argument '/root/Desktop/A002_C006_0205V7_001.R3D'. Finished splitting the commandline. Parsing a group of options: global . Applying option v (set logging level) with argument 9. Successfully parsed a group of options. Parsing a group of options: input file /root/Desktop/A002_C006_0205V7_001.R3D. Successfully parsed a group of options. Opening an input file: /root/Desktop/A002_C006_0205V7_001.R3D. [r3d @ 0x25f5060] Format r3d probed with size=2048 and score=100 [r3d @ 0x25f5060] File position before avformat_find_stream_info() is 324 [jpeg2000 @ 0x25f5a80] SOC marker not present Last message repeated 3 times [r3d @ 0x25f5060] Probe buffer size limit of 5000000 bytes reached [r3d @ 0x25f5060] decoding for stream 0 failed [r3d @ 0x25f5060] Could not find codec parameters for stream 0 (Video: jpeg2000, 4096x2304, 1/240000): unspecified pixel format Consider increasing the value for the 'analyzeduration' and 'probesize' options [r3d @ 0x25f5060] File position after avformat_find_stream_info() is 6494364 /root/Desktop/A002_C006_0205V7_001.R3D: could not find codec parameters [AVIOContext @ 0x25fd780] Statistics: 6529424 bytes read, 3 seeks
comment:5 by , 11 years ago
Component: | undetermined → avcodec |
---|---|
Keywords: | j2k added |
Priority: | normal → wish |
Reproduced by developer: | set |
Status: | new → open |
Type: | defect → enhancement |
Version: | 1.0.7 → git-master |
comment:6 by , 11 years ago
Replying to cehoyos:
Your console output looks incomplete / severely edited, please provide the complete, uncut console output.
Sorry about this, I misread the console output.
follow-up: 8 comment:7 by , 11 years ago
Sorry, is there a timetable for red support? R3d is listed as supported decoder - so for me it's look more like bug as enhancement or wish. Can i help in any way?
comment:8 by , 11 years ago
Replying to MilosL:
Can i help in any way?
A patch fixing the ticket - sent to ffmpeg-devel - is very welcome!
comment:9 by , 11 years ago
After reading these links:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78598
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78938
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/80922
Old version ("3") r3d files are "supported" insofar as they can be demuxed but since FFmpeg currently does not (yet) support Bayer colourspaces, the files cannot be decoded.
The file you uploaded is version "4", either this version needs a patch to the demuxer or to the decoder, in any case, that wouldn't fix the missing Bayer support.
follow-up: 11 comment:10 by , 11 years ago
Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf
I think this would be preferable to rewriting it from scratch, because the R3D format is under such heavy development by RED, and it would be too hard to keep up with the frequent updates to their color science, etc.
comment:11 by , 11 years ago
Replying to spookybathtub:
Would it be possible to utilize the official RED SDK to decode these files in ffmpeg? It's free, but not open source. You can read their license agreement at http://www.red.com/docs/RED_R3D_SDK_Agreement.pdf
Is it compatible with the GPL or the LGPL in your opinion?
comment:12 by , 11 years ago
I think it would be compatible with LGPL, so you can avoid distributing their source code. But this legal stuff is way over my head, so I'm not sure.
follow-up: 14 comment:13 by , 11 years ago
This is the response I got from RED's legal team:
You are not allowed to distribute any of the R3D SDK documentation, header files, sample code or library files directly. This means you would have to create either a custom ffmpeg binary people can download, or provide some sort of binary plugin if ffmpeg has such a system.
Is this doable?
comment:14 by , 11 years ago
Replying to spookybathtub:
Is this doable?
Don't you agree that it would be much simpler to get the (existing) Bayer patch(es) into FFmpeg and update the demuxer?
Note that if it would be legal to distribute such a binary of FFmpeg (I think this is unlikely but ianal) it would not be legal to have x264 support (which is what most people want).
follow-up: 16 comment:15 by , 11 years ago
Cc: | added |
---|
I'm not sure which would be simpler. I don't know how to code any of this myself, but I am very grateful for all the work done by the ffmpeg developers so far.
I do know that the SDK provides a huge amount of options for how to process R3D raw files, and many of these options are essential when dealing with RAW — for example you have to set the ISO, white balance, etc. before converting to a linear color space. It seems like it would be rather difficult to recreate all these settings.
I've read that about 3 years ago, RED started using some kind of encryption in the files. See http://ffmpeg.org/pipermail/ffmpeg-user/2011-September/002444.html
As an alternative, RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg. If anyone else wants to try, you can get it here: https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.
I agree it would not be practical to distribute crippled ffmpeg binaries, but is it possible to make some kind of "binary plugin" that doesn't reveal the source code, but can be used when compiling ffmpeg?
comment:16 by , 11 years ago
Replying to spookybathtub:
RED provides a free command line conversion tool called REDline, but I can't figure out any way to pipe its output to ffmpeg.
Why?
Consider discussing this on ffmpeg-user.
If anyone else wants to try, you can get it here: https://drive.google.com/file/d/0B52QuT8oHvtZZE5TMmdfSVZybFU/edit?usp=sharing. If that can be made to work, then there is hardly any reason to develop native R3D support in ffmpeg.
Why??
follow-up: 18 comment:17 by , 11 years ago
R3D is an extremely complex codec, with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED. Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.
The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch. In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately. But maybe I'm wrong.
follow-up: 19 comment:18 by , 11 years ago
Replying to spookybathtub:
R3D is an extremely complex codec
codec? Isn't it just a relatively simple format with jpeg2000 in it?
with dozens of raw processing parameters that need to be specified, and color science that is under heavy development by RED.
Check out all the color filters in libavfilter, including lut3d.
Also, REDline takes advantage of the Red Rocket hardware accelerator card for a huge speed benefit over CPU.
I wonder what this has to do with the software implementation.
The R3D SDK is kept up to date, and supports Red Rocket. But if licensing prohibits that, then I suppose the alternative would be to write a decoder from scratch.
Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)
In that case, it seems unlikely that ffmpeg developers could keep up to date with the color science and perform the debayer and raw conversions accurately.
lol
But maybe I'm wrong.
Yes.
comment:19 by , 11 years ago
Replying to ubitux:
Isn't the issue just about supporting some random pixel format, just like with XYZ with D-Cinema? (recently added)
The pixel format is Bayer, a patch was sent but is unfinished.
comment:20 by , 11 years ago
Yes, it's a bayer pattern, and it does use a modified JPEG2000 wavelet compression. But the debayered image is raw data with no specific color space until it is converted with RedColor3 or some other algorithm.
follow-up: 22 comment:21 by , 10 years ago
bayer colorspace support was added a few months ago.
http://git.videolan.org/?p=ffmpeg.git&a=search&h=HEAD&st=commit&s=bayer
whats next to support .r3d ?
comment:22 by , 9 years ago
Replying to compn:
bayer colorspace support was added a few months ago.
FFmpeg supports bayer formats with 8 bit per pixel and 16 bit per pixel, the r3d formats have 12 bit per component (48 bit per pixel).
comment:23 by , 9 years ago
That used to be true, but the current cameras are 16 bit.
Source: https://support.red.com/entries/27935746-Is-the-RED-DRAGON-sensor-16-bit-
comment:24 by , 9 years ago
do these 16bit bayer colorspaces have to be reverse engineered ?
can someone upload a 16bit r3d file for testing with? possibly with the original input so we can make sure its identical decoding.
comment:26 by , 9 years ago
something so we know what the input looks like. correct colors and stuff. screenshot etc
comment:28 by , 9 years ago
Those samples are ancient, and not 16 bit. I don't know the best way to upload samples here, but here is a recent sample from RED Dragon, along with two JPEG references — one debayered as REDlogFilm, and the other as REDgamma4. Both use DRAGONcolor2. Note: The JPEG preview on Google Drive does not look right at all, probably because it can't handle this high resolution (6144x3160).
https://drive.google.com/file/d/0B52QuT8oHvtZa2pRUktwb0JUVlE/view?usp=sharing
https://drive.google.com/file/d/0B52QuT8oHvtZNnBVMGpJQlRzNUk/view?usp=sharing
https://drive.google.com/file/d/0B52QuT8oHvtZek03NUdLdHlZazQ/view?usp=sharing
comment:29 by , 9 years ago
thank you spooky, i have mirrored your new samples here
http://samples.ffmpeg.org/r3d/
please keep pinging devs until this r3d stuff gets supported. :)
comment:30 by , 8 years ago
Are there plans to support new R3D formats ? I tried transcoding a file from here
http://www.red.com/sample-r3d-files
but I get an "invalid data found" error
These are 8K reels.
comment:31 by , 8 years ago
ffmpeg command line was
ffmpeg -i foo.r3d -vcodec libx264 -preset slow bar.mp4
with ffmpeg version 2.2.14
comment:32 by , 8 years ago
woops, old version of ffmpeg!
Checked with latest version, but same error message
comment:33 by , 6 years ago
Still not supported with build ffmpeg-20181028-bdfd2e3
C:\ffmpeg\bin\ffmpeg -i C:\R3D_8K.R3D -c:v libaom-av1 -crf 30 -b:v 0 -strict experimental av1_test_8K_R3D.mkv ffmpeg version N-92288-gbdfd2e3c79 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 8.2.1 (GCC) 20181017 C:\R3D_8K.R3D: Invalid data found when processing input
comment:34 by , 5 years ago
Is there any news/plans supporting r3d formats? Red cameras are already widely popular.
comment:36 by , 17 months ago
Still not supported with ffmpeg-6.0
follow-up: 39 comment:37 by , 17 months ago
I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4
comment:38 by , 16 months ago
Compressed files are available for evaluation at:
https://video.reflexion.tv/HEVC/
https://video.reflexion.tv/VVC/
Anyone willing to participate?
follow-up: 42 comment:39 by , 15 months ago
Replying to Peter Kovář:
I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4
Hi Peter,
Your PoC sounds promising!
I have a few queries:
- Would it be feasible to concatenate spanned R3D clips and incorporate an audio track from a WAV file?
- Is there a possibility of integrating your code directly into FFmpeg?
We are in the midst of developing a server encoding application tailored for raw camera files, including XDCAM and R3D. However, the absence of comprehensive R3D support in FFmpeg poses certain limitations. Given the importance of this feature for our project, we're considering sponsoring its development.
One of my primary concerns revolves around potential copyright challenges, especially when implementing the RED SDK in an open-source framework.
I'd appreciate your insights on these matters.
comment:40 by , 13 months ago
Only just realized that Red footage wasn't compatible with ffmpeg today, but sounds like it's been an ongoing issue for some time.
Any updates on this one?
comment:42 by , 9 months ago
Replying to Camille:
Replying to Peter Kovář:
I wrote PoC GPU accelerated code based upon R3DSDKv8_4_0-Beta2. Still unfinished and undecided how to release that into the wild.
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -c:v libx265 -vf scale=4096:2160 v-raptor-xl-8k-vv-bird HEVC 2160p120.mp4
R3D /2TB/Video/R3D/v-raptor-xl-8k-vv-bird/A001_C003_12214C.RDC/A001_C003_12214C_001.R3D | ffmpeg -video_size 8192x4320 -framerate 120 -f rawvideo -pixel_format rgb48le -i - -strict -1 -pix_fmt yuv420p10 -color_primaries bt2020 -colorspace bt2020_ncl -color_trc bt2020-10 -r 120 -vcodec vvc -b:v 0 -qp 32 -vf scale=4096:2160 -y v-raptor-xl-8k-vv-bird VVC 2160p120.mp4
Hi Peter,
Your PoC sounds promising!
I have a few queries:
- Would it be feasible to concatenate spanned R3D clips and incorporate an audio track from a WAV file?
- Is there a possibility of integrating your code directly into FFmpeg?
We are in the midst of developing a server encoding application tailored for raw camera files, including XDCAM and R3D. However, the absence of comprehensive R3D support in FFmpeg poses certain limitations. Given the importance of this feature for our project, we're considering sponsoring its development.
One of my primary concerns revolves around potential copyright challenges, especially when implementing the RED SDK in an open-source framework.
I'd appreciate your insights on these matters.
Camille, could you contact me via peter.kovar@reflexion.tv?
Please test (and provide console output) for current git head and please provide a sample.