mjpeg enc: ljpeg output is incompatible with other programs

[ami_stuff]

I have tried to load the output file from FFmpeg to "My ViewPad" and "PicTools Apollo", but these programs rejected the file as broken.

C:\>ffmpeg -i 1.bmp -vcodec ljpeg ffmpeg.jpg
ffmpeg version N-56198-gbbcaf25 Copyright (c) 2000-2013 the FFmpeg developers
  built on Sep 11 2013 19:06:31 with gcc 4.7.3 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp
e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena
ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l
ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp
eex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aa
cenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264
 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 43.100 / 52. 43.100
  libavcodec     55. 31.101 / 55. 31.101
  libavformat    55. 16.102 / 55. 16.102
  libavdevice    55.  3.100 / 55.  3.100
  libavfilter     3. 84.100 /  3. 84.100
  libswscale      2.  5.100 /  2.  5.100
  libswresample   0. 17.103 /  0. 17.103
  libpostproc    52.  3.100 / 52.  3.100
Input #0, image2, from '1.bmp':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: bmp, bgr24, 320x240, 25 tbr, 25 tbn, 25 tbc
Output #0, image2, to 'ffmpeg.jpg':
  Metadata:
    encoder         : Lavf55.16.102
    Stream #0:0: Video: ljpeg, bgr24, 320x240, q=2-31, 200 kb/s, 90k tbn, 25 tbc

Stream mapping:
  Stream #0:0 -> #0:0 (bmp -> ljpeg)
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:126kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.017076%

jpegdump outputs this:

C:\>JPEGDUMP.EXE C:\apollo.jpg

C:\apollo.jpg:
  width 320, height 240  components 3
    id 1 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 0

C:\>JPEGDUMP.EXE C:\myvp.jpg

C:\myvp.jpg:
  width 320, height 240  components 3
    id 82 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 71 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 66 horizontal sampling 1, vertical sampling 1, quantization table 0

C:\>JPEGDUMP.EXE C:\ffmpeg.jpg

C:\ffmpeg.jpg:
  Approximate quality factor for qtable 0: 61 (scale 77.31, var 1710.91)
  width 320, height 240  components 3
    id 1 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 0

[Carl Eugen Hoyos]

Do they show the output of the following command (bgr vs rgb)?

$ ffmpeg -i tests/lena.pnm -vcodec ljpeg out.jpg

[Elon Musk]

How are you sure they actually support lossless jpeg?

You cannot give jpg extension to ljpg file.

[Carl Eugen Hoyos]

Replying to richardpl:

How are you sure they actually support lossless jpeg?

Did you test the attached files?

You cannot give jpg extension to ljpg file.

As in: You cannot give abc extension to an avi file?
(And I actually suspect that "ljpg" is to some degree FFmpeg-specific but this is of course irrelevant.)

Sorry, your comment is very difficult to understand...

[ami_stuff]

Replying to cehoyos:

Do they show the output of the following command (bgr vs rgb)?

$ ffmpeg -i tests/lena.pnm -vcodec ljpeg out.jpg

No, I still get from My ViewPad "Load error with code 'Bad image file'" and from Apollo 'ERR_BAD_DATA'"

[ami_stuff]

(​http://www.programmersheaven.com/download/17054/Download.aspx)
jpegdump have an -v option:

C:\>jpegdump -v apollo.jpg

apollo.jpg:
offset $0 SOI
offset $2 APP0 (length 15)
  LJIF\000\001\000\000\000\000\000\000\001
offset $13 DHT (length 80)
  table 0
  table 1
  table 2
offset $65 SOF3 (spatial lossless Huffman) (length 17)
  sample precision 8
  width 320, height 240  components 3
    id 1 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 0
offset $78 SOS (length 12)
  components 3
    id 1 dc table 0, ac table 0
    id 2 dc table 1, ac table 0
    id 3 dc table 2, ac table 0
  spectral selection 1 to 0, bit position high 0, low 0
offset $25bc0 EOI

C:\>jpegdump -v myvp.jpg

myvp.jpg:
offset $0 SOI
offset $2 APP14 (length 14)
  Adobed\000\000\000\000\000\000
offset $12 COM (length 38)
  Created by fCoder Graphics Processor
offset $3a SOF3 (spatial lossless Huffman) (length 17)
  sample precision 8
  width 320, height 240  components 3
    id 82 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 71 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 66 horizontal sampling 1, vertical sampling 1, quantization table 0
offset $4d DHT (length 28)
  table 0
offset $6b SOS (length 12)
  components 3
    id 82 dc table 0, ac table 0
    id 71 dc table 0, ac table 0
    id 66 dc table 0, ac table 0
  spectral selection 1 to 63, bit position high 0, low 0
offset $25f0f EOI

C:\>jpegdump -v ffmpeg.jpg

ffmpeg.jpg:
offset $0 SOI
offset $2 COM (length 16)
  Lavc55.31.101\000
offset $14 DQT (length 67)
  table 0 precision 8
  Approximate quality factor for qtable 0: 61 (scale 77.31, var 1710.91)
offset $59 DHT (length 418)
  table 0
  table 1
  table 16
  table 17
offset $1fd SOF3 (spatial lossless Huffman) (length 17)
  sample precision 9
  width 320, height 240  components 3
    id 1 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 0
offset $210 SOS (length 12)
  components 3
    id 1 dc table 0, ac table 0
    id 2 dc table 1, ac table 0
    id 3 dc table 1, ac table 0
  spectral selection 1 to 0, bit position high 0, low 0
offset $22ffb EOI

[Carl Eugen Hoyos]

Maybe not writing DQT?

[ami_stuff]

Replying to cehoyos:

Maybe not writing DQT?

still doesn't decode (with neither program) with commented out

put_marker(p, DQT);

C:\>jpegdump -v ffmpeg_2.jpg

ffmpeg_2.jpg:
offset $0 SOI
offset $2 COM (length 16)
  Lavc55.31.101\000
offset $57 DHT (length 418)
  table 0
  table 1
  table 16
  table 17
offset $1fb SOF3 (spatial lossless Huffman) (length 17)
  sample precision 9
  width 320, height 240  components 3
    id 1 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 2 horizontal sampling 1, vertical sampling 1, quantization table 0
    id 3 horizontal sampling 1, vertical sampling 1, quantization table 0
offset $20e SOS (length 12)
  components 3
    id 1 dc table 0, ac table 0
    id 2 dc table 1, ac table 0
    id 3 dc table 1, ac table 0
  spectral selection 1 to 0, bit position high 0, low 0
offset $1f741 EOI

so probably they want sample precision 8

[Carl Eugen Hoyos]

You were fast;-)
Patch attached anyway.

[maweber]

Sorry to warm this up.
Did this patch never go live?
I cannot convert a ljpeg (of ffmpeg) in any converters I found.
Seems you guys fixed this?

Thanks
M

[Carl Eugen Hoyos]

Which converters did you test?
How do you know they support lossless jpeg? (Support for lossless jpeg is very unusual.)

[maweber]

GDCM, xmedcon, PVRG JPEG (linux port), patched GraphicsMagick, combinations with netpbm tools...

You're absolutely right. It is so seldom I may have to drop it completely.
I was looking into it because of its speed compared to jpeg2000.
I searched a lot of descriptions, mainly on DICOM applications. but even with them its hard.
it is more common i think to have a ljpeg wrapped inside a DCM.
GDCM seemed the most reliable of those.

[Carl Eugen Hoyos]

What is a DCM?
Is it supported by FFmpeg?

[maweber]

no. the container DCM is not supported by ffmpeg.

.dcm is the container for a range of specs defined by the DICOM standard.

As far as I understood, it is an exchange format used by medical people.
it defines a lot of client/patient meta stuff inside the container.

But it seems that they early standardized ljpeg (among others) for their codec.
Thats why you find mostly DICOM related converters if you search for ljpeg, i guess.
My aim was not a DCM container, but to get directly from ljpeg or jpeg-LS to something like PPM, which GDCM claims.

So basically, for your understanding, I was producing ljpeg with ffmpeg, saved bandwidth, and wanted to unpack it on another machine to PPM, for easy processing. I'm processing 160k+ pictures, so I was looking to have something small, yet lossless.

have a good day

[Carl Eugen Hoyos]

I suspect that ffv1 compresses better than ljpeg.

[maweber]

thanks, thats a new trace.
especially if it's homebrew.

perfect!
cheers
manu

[ami_stuff]

I attached file saved with lead software and screenshot how lead software decodes "ffmpeg.jpg" file (see "screen.png" - the first file at the top on the screenshot is "ffmpeg.jpg")

[ami_stuff]

btw if someone wants dicom samples (with rle/jpeg2000/jpeg-ls/jpeg lossless compression) then I can create them

[Carl Eugen Hoyos]

Replying to ami_stuff:

btw if someone wants dicom samples (with rle/jpeg2000/jpeg-ls/jpeg lossless compression) then I can create them

Please open an enhancement request with such samples.

[ami_stuff]

What is reference decoder for jpeg lossless? Does it decode files created with ffmpeg correctly?

[ami_stuff]

there are 3 files in the archive:

lead_ljpeg.avi - -file encoded with lead's ljpeg encoder
ffmpeg_ljpeg.avi - file encoded with ffmpeg's ljpeg encoder
ffmpeg_ljpeg_decoded_by_lead_codec.avi - output from lead's ljpeg decoder

​http://www.datafilehost.com/d/b09e7868

[Carl Eugen Hoyos]

Reference software is this patch afaict: ​ftp://ftp.graphicsmagick.org/pub/GraphicsMagick/delegates/ljpeg-6b.tar.gz
It can be applied on the reference jpg software: ​http://www.ijg.org/files/jpegsrc.v6b.tar.gz
FFmpeg can decode the files produced with the resulting encoder, the decoder does decode FFmpeg samples visually ok but warns about their bitdepth (Must downscale data from 9 bits to 8) and the output is not lossless.

[ami_stuff]

so how one can modify ffmpeg's ljpeg encoder to output "sample presision 8"?