Support mjpeg rotation

[rrjp]

Summary of the bug:

Take a video with rotate: 90 metadata set (e.g. iPhone) and extract a frame to use as a thumbnail. Rotation / orientation metadata does not appear to be set in the resulting jpeg.

How to reproduce:

Extract the JPEG

ffmpeg -itsoffset 4 -i bz_12900.mov -y -vframes 1 bz.jpg

ffmpeg version N-68129-g928322c Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec  1 2014 02:39:44 with gcc 4.9.2 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-
libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-l
ibtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.101 / 56. 15.101
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bz_12900.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2014-09-16 06:15:50
    model           : iPad 2
    model-eng       : iPad 2
    encoder         : 7.1.2
    encoder-eng     : 7.1.2
    date            : 2014-09-16T11:45:50+0530
    date-eng        : 2014-09-16T11:45:50+0530
    location        : +12.9394+077.6255+897.588/
    location-eng    : +12.9394+077.6255+897.588/
    make            : Apple
    make-eng        : Apple
  Duration: 00:00:13.95, start: 0.000000, bitrate: 10840 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 10740 kb/s, 29.97 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-09-16 06:15:50
      handler_name    : Core Media Data Handler
      encoder         : H.264
    Side data:
      displaymatrix: rotation of -90.00 degrees
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 64 kb/s (default)
    Metadata:
      creation_time   : 2014-09-16 06:15:50
      handler_name    : Core Media Data Handler
[swscaler @ 0000000000489c20] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to 'bz.jpg':
  Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    make-eng        : Apple
    model           : iPad 2
    model-eng       : iPad 2
    make            : Apple
    encoder         : Lavf56.15.101
    date            : 2014-09-16T11:45:50+0530
    date-eng        : 2014-09-16T11:45:50+0530
    location        : +12.9394+077.6255+897.588/
    location-eng    : +12.9394+077.6255+897.588/
    Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 1280x720, q=2-31, 200 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2014-09-16 06:15:50
      handler_name    : Core Media Data Handler
      encoder         : Lavc56.13.100 mjpeg
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
frame=    1 fps=0.0 q=6.2 Lsize=N/A time=00:00:00.03 bitrate=N/A dup=1 drop=1
video:49kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

We see rotate:90 in the output section above (in the video section which is confusing since no video file was produced), but if I look at tags in resulting file using exiftool it's not there:

ExifTool Version Number         : 9.72
File Name                       : BZ_12900.jpeg
Directory                       : .
File Size                       : 54 kB
File Modification Date/Time     : 2014:12:01 11:06:44-05:00
File Access Date/Time           : 2014:12:01 11:07:06-05:00
File Inode Change Date/Time     : 2014:12:01 11:07:06-05:00
File Permissions                : rw-r-----
File Type                       : JPEG
MIME Type                       : image/jpeg
Comment                         : Lavc55.18.102
Image Width                     : 1280
Image Height                    : 720
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Image Size                      : 1280x720

Also don't see it in ffmpeg output.

ffmpeg -i bz.jpg -f ffmetadata bz.jpg.ff.txt

ffmpeg version N-68129-g928322c Copyright (c) 2000-2014 the FFmpeg developers
  built on Dec  1 2014 02:39:44 with gcc 4.9.2 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-
libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-l
ibtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-decklink --enable-zlib
  libavutil      54. 15.100 / 54. 15.100
  libavcodec     56. 13.100 / 56. 13.100
  libavformat    56. 15.101 / 56. 15.101
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  2.103 /  5.  2.103
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, image2, from 'bz.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 10088 kb/s
    Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1280x720, 25 tbr, 25 tbn, 25 tbc
File 'bz.jpg.ff.txt' already exists. Overwrite ? [y/N] y
Output #0, ffmetadata, to 'bz.jpg.ff.txt':
  Metadata:
    encoder         : Lavf56.15.101
Stream mapping:
Press [q] to stop, [?] for help
size=       0kB time=-577014:-32:-22.-77 bitrate=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Seems like if rotate: 90 is detected in the video that orientation: 8 should be set in the resulting JPEG or -vf transpose=1 should automatically happen to correctly orient the extracted frame.

[rrjp]

Was reported as a defect because issue ​https://trac.ffmpeg.org/ticket/515 claims that the functionality exists:

"By default all stream metadata, including rotation, are copied to the output."

That doesn't appear to happen in any meaningful way in this case.

[Carl Eugen Hoyos]

That's because the rotation metadata is currently only supported for mov.

[Roger Pack]

I think I just ran into this today:

original:

identify -verbose 20150630_165653.jpg | grep -i orient

Orientation: RightTop

exif:Orientation: 6
exif:thumbnail:Orientation: 6

then if I use ffmpeg to convert it from jpeg to jpeg:

ffmpeg -i 20150630_165653.jpg converted.jpg
ffmpeg version N-44261-g0083c16 Copyright (c) 2000-2015 the FFmpeg developers

built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
libavutil 54. 27.100 / 54. 27.100
libavcodec 56. 45.101 / 56. 45.101
libavformat 56. 40.100 / 56. 40.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 19.100 / 5. 19.100
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.100 / 1. 2.100
libpostproc 53. 3.100 / 53. 3.100

Input #0, image2, from '20150630_165653.jpg':

Duration: 00:00:00.04, start: 0.000000, bitrate: 1007788 kb/s

Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 5312x2988, 25 tbr, 25 tbn, 25 tbc

Output #0, image2, to 'converted.jpg':

Metadata:

encoder : Lavf56.40.100
Stream #0:0: Video: mjpeg, yuvj422p(pc), 5312x2988, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
Metadata:

encoder : Lavc56.45.101 mjpeg

Stream mapping:

Stream #0:0 -> #0:0 (mjpeg (native) -> mjpeg (native))

Press [q] to stop, ? for help
frame= 1 fps=0.0 q=9.8 Lsize=N/A time=00:00:00.04 bitrate=N/A
video:615kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

$ identify -verbose converted.jpg | grep -i orient

Orientation: Undefined

orientation has been lost...

My guess would be that maybe if it has a certain orientation, ffmpeg should "auto rotate" it so that it gets appropriate input, perhaps?