Opened 5 months ago

Last modified 5 months ago

#6945 reopened enhancement

ffmpeg fails at jpeg EXIF orientation (test included)

Reported by: JohnnyX Owned by:
Priority: wish Component: avcodec
Version: git-master Keywords: mjpeg rotate
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

Summary of the bug:

If you try viewing these test images in any other image viewer, they'll all display them correctly (all in the same orientation). Ffmpeg on the other hand fails to display them correctly.

This is a major issue when using ffmpeg-based players as an image viewer, because most images these days (from digital cameras and phones) use EXIF orientation to tell the viewer how to display the image.

Trying to use ffmpeg as an image viewer currently leads to a sad experience where images are rotated and flipped into crazy angles, rather than being displayed correctly. ;-)

How to reproduce:

  • Try viewing/converting the JPEG with ffmpeg:
ffmpeg -i Landscape_2.jpg test.png
  • The result will be badly flipped/rotated.
  • All images are supposed to look identical (as they will in a correct viewer).

What needs fixing:

  • So the fix would be: "If input format == jpeg/exif with orientation flag, apply the orientation filters to the output".

There are 8 JPEG EXIF orientations. They represent various combinations of rotate + vflip + hflip. And all of the code for rotate/flip filters is already available in ffmpeg. Just look at the video autorotate code above. It seems to be very easy to add this JPEG fix by just looking at that commit and the EXIF commit.

I don't know this codebase at all and would just screw it up if I attempt this fix. Hopefully someone here wants to fix ffmpeg's broken JPEG support. It's important these days, since most camera sensors capture pixels in a single orientation and then add an EXIF orientation tag to assign the display-orientation. By lacking that feature, ffmpeg currently fails badly at almost all JPEG input.

Change History (6)

comment:1 follow-ups: Changed 5 months ago by cehoyos

  • Keywords exif jpg jpeg orientation removed
  • Priority changed from important to normal
  • Resolution set to duplicate
  • Status changed from new to closed

comment:2 in reply to: ↑ 1 Changed 5 months ago by JohnnyX

Replying to cehoyos:

Hey cehoyos. Would you mind saying what this is a duplicate of? I did a lot of searching before opening this ticket.

comment:3 in reply to: ↑ 1 Changed 5 months ago by JohnnyX

Replying to cehoyos:

Hey cehoyos. Would you mind saying what this is a duplicate of? I did a lot of searching before opening this ticket.

comment:4 follow-up: Changed 5 months ago by cehoyos

I thought you mentioned this is a duplicate of ticket #4149, is it not?

comment:5 in reply to: ↑ 4 Changed 5 months ago by JohnnyX

Replying to cehoyos:

I thought you mentioned this is a duplicate of ticket #4149, is it not?

Hehe, no. Please re-open this ticket.

I apologize if I wasn't clear. I was only referring to those old tickets since they contained discussions about "ffmpeg's stance" on auto-rotation of input material.

Number 4149 is someone who asked about "passthrough" writing (output) of video rotation data into ffmpeg's generated mjpeg snapshots from videos. It's a very muddy ticket actually. You might even want to close that one or ask for more info in it, to see what they really need (and it's 3 years old, so it seems dead). I only linked to it because it contained some related statements.

This ticket on the other hand is about something that's close to "critical" level: ffmpeg's reading of (input) jpeg files is totally incorrect because it unfortunately doesn't insert vflip/hflip/rotate video transformation filters to the output graph as-required by the the jpeg format's EXIF orientation value. This means that ffmpeg is unable to understand large amounts of modern jpegs taken from digital cameras, which very frequently use EXIF orientation to assign the final display-orientation. You can try out ffmpeg with the test files in my original message above. The result is very sad. :-\ ffmpeg will decode those jpegs as their raw pixels without any of their assigned transformations, which means that ffmpeg's output will just be randomly rotated and flipped (mirrored) rather than their intended orientation.

It seems like the appropriate fix would be to combine the techniques from the two linked commits; the one that handles video autorotate and the one that handles jpeg EXIF reading.

comment:6 Changed 5 months ago by cehoyos

  • Component changed from undetermined to avcodec
  • Keywords mjpeg rotate added
  • Priority changed from normal to wish
  • Resolution duplicate deleted
  • Status changed from closed to reopened
  • Type changed from defect to enhancement
  • Version changed from unspecified to git-master
Note: See TracTickets for help on using tickets.