Opened 4 years ago

Closed 13 months ago

#1845 closed defect (fixed)

Encoded movies with mov_text subtitles do not play with QT Player

Reported by: Atarikid Owned by:
Priority: normal Component: avformat
Version: git-master Keywords: mov mov_text
Cc: philipl, eisa01@gmail.com, zagser168@yandex.ru Blocked By:
Blocking: Reproduced by developer: yes
Analyzed by developer: no

Description

When a .mkv contains subrip subtitle(s) and you encode to mp4 with -c:s mov_text the subtitles do not work when using the QuickTime? Player (OSX). It plays fine with VLC though.

When encoding with Handbrake (which uses mov_text for mp4) it works fine with QuickTime? Player.

FFmpeg commandline:
/Users/atarikid/Desktop/ffmpeg -i /Volumes/Data/Movies/Homeland??.mkv -c:a aac -c:v libx264 -strict -2 -c:s mov_text /Users/atarikid/Desktop/test.mp4

Attachments (1)

Subtitles issues examples.zip (1.4 MB) - added by Atarikid 4 years ago.

Download all attachments as: .zip

Change History (32)

comment:1 Changed 4 years ago by cehoyos

  • Component changed from FFmpeg to undetermined
  • Keywords mov added

Could you point us to a file with mov_text that is played correctly with QuickTime??

Changed 4 years ago by Atarikid

comment:2 Changed 4 years ago by Atarikid

I have uploaded two encoded movies.
One with FFmpeg and commandline:
/Users/atarikid/Desktop/ffmpeg -i /Volumes/Data/Movies/Homeland???.mkv -c:a aac -c:v libx264 -strict -2 -c:s mov_text /Users/atarikid/Desktop/test.mp4

The subtitles do not work with QuickTime? Player (main OSX player). Works with VLC

The second is encoded with Handbrake (with mov_text):
Subtitles work fine with QuickTime? Player and VLC

comment:3 Changed 4 years ago by Atarikid

It is note worthy that with QuickTime? Player you cannot even select the subtitles stream. So maybe the header created by FFmpeg is wrong?

comment:4 Changed 4 years ago by cehoyos

I tested "Handbrake-subtitles work with QTPlayer.mp4" with QuickTime? 7.7.2 and it shows no subtitles here. Which version did you test?

comment:5 follow-up: Changed 4 years ago by Atarikid

Tested with QuickTime? Player 10.2 (OS X 10.8 and OS X 10.7).
You first have to choose the subtitles in the menu. By default the subtitles are disabled in QT Players.

BTW QT 7.7.2 is a tremendous old version no-one uses with OSX. :)

comment:6 in reply to: ↑ 5 Changed 4 years ago by cehoyos

  • Component changed from undetermined to avformat
  • Reproduced by developer set
  • Status changed from new to open

Replying to Atarikid:

BTW QT 7.7.2 is a tremendous old version no-one uses with OSX. :)

Not everybody owns a Mac...

comment:7 Changed 4 years ago by Atarikid

Besides being the most irritating answer ever, what does this answer contributes to the query?

comment:8 Changed 4 years ago by cehoyos

I told you I tested QuickTime? 7.7.2 (this was current at the time of my writing on Windows, it's now 7.7.3 iirc) and was unable to reproduce the issue, you answered that my version of QuickTime? is outdated (which is true, I wasn't aware of it though). It took me some time to find a Mac that allows to reproduce the problem (on my Mac, 7.7.2 is also the latest version) and I wanted to explain why it took me so long to reproduce the issue.

Long-time experience tells me that tickets that cannot be easily reproduced are not easily fixed (in this case it would have been possible imo that somebody who does want to work on it would have been unable to reproduce the problem exactly as I was unable to).

comment:9 follow-up: Changed 4 years ago by Atarikid

OK, sorry for my reply. But only 'Not everybody owns a Mac' came out a bit rude imo :-)

Anyhow, glad you could replicate this issue. FFmpeg is quit common on Mac OS X (many converters use it). So making it work for QT Player is imo important.

comment:10 in reply to: ↑ 9 Changed 4 years ago by cehoyos

Replying to Atarikid:

OK, sorry for my reply. But only 'Not everybody owns a Mac' came out a bit rude imo :-)

That is exactly the feeling I had reading "QT 7.7.2 is a tremendous old version no-one uses"

comment:11 Changed 4 years ago by cehoyos

  • Keywords mov_text added

comment:12 Changed 4 years ago by cehoyos

There is another (older) sample - http://samples.ffmpeg.org/mov/subtitles-embedded/BCDisassembly1RightSideTxt2.mov - that plays fine with QT 7.7.3
It works fine with QT if the subtitle stream is copied (-scodec copy), shows weird (enlarged, red, misplaced) subtitles on QT with -scodec mov_text

comment:13 follow-up: Changed 4 years ago by philipl

It sounds like the player is, at least to some extent, respecting the text formatting and positioning flags in the header - which no other player I'm aware of does. That's why I was never able to test this.

We used a fixed style which should yield "12 point Serif white-on-transparent" with undefined positioning. This could easily lead to invisible subtitles due to the lack of positioning information, or misplaced subtitles (top left corner?).

The only thing that really surprises me is the red colour. I know I didn't mess that up.

comment:14 in reply to: ↑ 13 Changed 4 years ago by cehoyos

  • Cc philipl added

Replying to philipl:

It sounds like the player is, at least to some extent, respecting the text formatting and positioning flags in the header - which no other player I'm aware of does.

Excuse my ignorance: What other player than QuickTime? supports mov_text in mov?

comment:15 Changed 4 years ago by philipl

mplayer, vlc - all the usual suspects. Am I missing something?

And iOS devices, although also made by Apple use very different playback software than Quicktime on the desktop.

comment:16 Changed 4 years ago by cehoyos

I tried the following with the sample attached to this ticket (that plays subtitles on my iPhone):
$ ffmpeg -i Handbrake-subtitles\ work\ with\ QTPlayer.mp4 -strict -2 -qscale 5 -scodec mov_text outmovtext.mov
The resulting file shows no subtitles on my iPhone.

comment:17 follow-up: Changed 4 years ago by philipl

The situation is confusing, to say the least.

On my ipad: both the attached samples work correctly.

When I remux the Handbrake sample, I get a file that's identified as having subtitles, but the subtitle duration is way off, and the subtitles 'stick' on screen for too long - I think there's some bug in how the time base is being translated. But the subtitles, are nevertheless, being transcoded and shown in the ipad.

I'm surprised to see a significant difference between ipad and iphone.

comment:18 follow-up: Changed 4 years ago by philipl

Also note that my windows QT 7.7.3 reports the attached Handbrake sample as having no subtitles. So I really don't know what's going on right now.

comment:19 in reply to: ↑ 17 Changed 4 years ago by cehoyos

Replying to philipl:

When I remux the Handbrake sample

I did not test remuxing but re-encoding the subtitles with -scodec mov_text

comment:20 in reply to: ↑ 18 Changed 4 years ago by cehoyos

Replying to philipl:

Also note that my windows QT 7.7.3 reports the attached Handbrake sample as having no subtitles.

It works fine with newer QT, see above.

comment:21 Changed 4 years ago by philipl

So, there are multiple things going on here that affect the success, or otherwise, of this process.

1) File type subtleties

There are meaningful differences between mov/mp4/ipod output types as generated by the mov encoder. I always use 'ipod' and I've succeeded in producing files with subtitle tracks that are recognised by all tested players (QT, iPad, mplayer) this way. But not always. I've got input samples that don't seem to yield subtitles recognised by QT even with the same command line. Don't get it. As mentioned above, there may be some subtitle timebase translation failure going on when remuxing existing mp4 files.

2) The definite feature gaps

a) The width/height and positional offset of the text track need to be set to meaningful values by the mov encoder (this header is not handled by the mov_text encoder).

b) The text box (which exists within the text track) needs to be sized to meaningful values by the mov_text encoder.

In practice, these values are closely related and in turn closely related to the width/height of the video track. I have no idea how one would sanely set these values in a file with multiple video tracks and/or ones with varying sizes.

A rough heuristic that works for a single constant size video is that the text track has the same width as the video, and some 'reasonable' height in pixels - which we might be able to derive as a percentage of the video height. The vertical offset of the text track is then video height - text height - assuming we want the subtitle area to be aligned exactly with the bottom of the video area. (It is legal to offset outside the video area and the quicktime player will make this work)

The text box is then 'normally' sized to the dimensions of the text track (ie: fills the whole track). This means that the mov_text encoder cannot know ahead of time what a reasonable size for the text box is. We'd really have to make the mov encoder re-write these values when writing the track header.

This is all massively overengineered and very hard to get right, because - of course, subtitle size and display area really should be controlled by the playback device and not by the file encoder!

comment:22 Changed 4 years ago by philipl

Oh, and to state it explicitly: by hard-coding various sizes and offsets in the encoders, I was able to produce a file with subtitles that rendered correctly in QT 7.3.3 on windows and OSX Player 10.1 (what I was able to test with on a Mac I could fine).

So, the fundamental encoding, with respect to atom types and formatting is correct. But in the absence of accurate sizing information, it's not going to display correctly.

comment:23 Changed 4 years ago by cehoyos

Is the sizing information something the encoder or the muxer has to write? Does the muxer have enough information?

comment:24 Changed 4 years ago by philipl

When I say "mov encoder", I mean the muxer.

The muxer is the *only* thing that has enough information, because it's the only thing that knows about the video tracks that the subtitles will be used with.

comment:25 Changed 4 years ago by Nick

for additional comments and example files see also:
https://ffmpeg.org/trac/ffmpeg/ticket/2488


comment:26 Changed 3 years ago by eisa01

  • Cc eisa01@gmail.com added

comment:27 Changed 3 years ago by zagser168

  • Cc zagser168@yandex.ru added

comment:28 Changed 3 years ago by ubitux

Is this still true? It's potentially fixed by 30ce2890.

comment:29 Changed 3 years ago by eisa01

The bug is still there for me, assuming I compiled it correctly

(cloned the git repository and ran configure, make, make install and verified the new compilation date of ffmpeg)
ffmpeg version N-55736-gb329ff3 Copyright (c) 2000-2013 the FFmpeg developers

built on Aug 23 2013 23:40:45 with Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn)

Version 0, edited 3 years ago by eisa01 (next)

comment:30 Changed 3 years ago by philipl

This bug's not going to go away without either a bunch of work in the muxer to translate video track sizing information into subtitle track sizing, or adding a bunch of encode/mux options that expose all the silly box dimension settings. Based on feedback when I started discussing this many months ago, the second option is preferred but is trickier to execute than it looks.

comment:31 Changed 13 months ago by philipl

  • Resolution set to fixed
  • Status changed from open to closed

During GSoC this year, my intern implemented support for most of the text formatting capabilities in timed text, including applying a non-trivial default style when encoding. He didn't do anything with display area positioning and sizing, but nevertheless, we now observe that, at least on OSX, the QT Player is happily showing subtitles at a reasonable size in a reasonable position. I suspect this is due to the default style being more meaningful, and triggering some default positioning logic in the QT Player.

So, certainly this is fixed on OSX. My intern said he was not able to see any subtitles on windows with or without his changes. That still needs investigating, but it's a separate issue, so I'm going to mark this as closed.

Note: See TracTickets for help on using tickets.