Support Extended Language Tag Atom (elng) for MOV muxer

[jeanhackman]

Summary of the bug:
Attempting to use ffmpeg to set audio language to "frm" results in audio language of English on output file. Tried alternate language codes - "spa" and "por" and they work as expected where audio language on output file is Spanish or Portuguese respectively.

How to reproduce:

% ffmpeg -i test_pattern.mov \
-i french_5.1_audio/5.1_pcm_fra.aiff \
-i stereo_pcm_fra_dvs_audio.aiff \
-vcodec copy \
-map 0:0:v \
-acodec copy \
-map 1:0:a -metadata:s:a:0 language=fra \
-map 2:0:a -metadata:s:a:1 language=frm \
audio_map_test_files/test_pattern_fra51_fraDVS.mov

ffmpeg version N-93873-g6276b4db97
built on May 16 2019 built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)

[jeanhackman]

ffmpeg -v 9 -loglevel 99 -i output per each input of 3 input files used

[jeanhackman]

I used the following command to create "test_pattern.mov" input file

ffmpeg -f lavfi -i testsrc=duration=10:size=1280x720:rate=60 -vcodec test_pattern.mov

[Carl Eugen Hoyos]

Does it work if you use the file suffix .mp4?
Do you have a mov file with the intended language descriptor?

[jeanhackman]

Hi cehoyos,

Thank you for looking into this.
It appears ProRes is not a supported format in mp4 container:

ffmpeg -i test_pattern.mov \
-i french_5.1_audio/5.1_pcm_fra.aiff \
-i stereo_pcm_fra_dvs_audio.aiff \
-vcodec copy \
-map 0:v \
-acodec copy \
-map 1:a -metadata:s:a:0 language=fra \
-map 2:a -metadata:s:a:1 language=frm \
audio_map_test_files/test_pattern_fra51_fraDVS.mp4
ffmpeg version N-93873-g6276b4db97 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.4)
  configuration:
  libavutil      56. 28.100 / 56. 28.100
  libavcodec     58. 52.101 / 58. 52.101
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 53.101 /  7. 53.101
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test_pattern.mov':
  Metadata:
    major_brand     : qt
    minor_version   : 512
    compatible_brands: qt
    encoder         : Lavf58.12.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 37525 kb/s
    Stream #0:0(eng): Video: prores (Standard) (apcn / 0x6E637061), yuv422p10le(tv, smpte170m/unknown/unknown, progressive), 1280x720, 37522 kb/s, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.18.100 prores
Input #1, aiff, from 'french_5.1_audio/5.1_pcm_fra.aiff':
  Duration: 00:00:04.50, start: 0.000000, bitrate: 2116 kb/s
    Stream #1:0: Audio: pcm_s16be, 22050 Hz, 5.1(side), s16, 2116 kb/s
Guessed Channel Layout for Input Stream #2.0 : stereo
Input #2, aiff, from 'stereo_pcm_fra_dvs_audio.aiff':
  Duration: 00:00:02.08, start: 0.000000, bitrate: 705 kb/s
    Stream #2:0: Audio: pcm_s16be, 22050 Hz, stereo, s16, 705 kb/s
[mp4 @ 0x7fefc7046400] Could not find tag for codec prores in stream #0, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #0:1 (copy)
  Stream #2:0 -> #0:2 (copy)
    Last message repeated 1 times

I do not have a .mov offhand that has "French, Middle" audio language descriptor, but I'm working to see if I can get ahold of one.

[jeanhackman]

However, I was just successful in creating an MPEG-TS container with MPEG-2 video and AAC audio including the "frm" audio language descriptor. I think this will suffice for the testing I am performing. I'm assuming passing "frm" audio language descriptor is either a limitation of the MPEG-4 container ".mov" or a limitation within ffmpeg currently.

ffmpeg -i test_pattern.mov \
-i french_5.1_audio/5.1_pcm_fra.aiff \
-i stereo_pcm_fra_dvs_audio.aiff \
-vcodec mpeg2video \
-map 0:v \
-acodec aac \
-map 1:a -metadata:s:a:0 language=fra \
-map 2:a -metadata:s:a:1 language=frm \
audio_map_test_files/test_pattern_fra51_fraDVS.ts

mediainfo audio_map_test_files/test_pattern_fra51_fraDVS.ts
...
Audio #2
ID                                       : 258 (0x102)
Menu ID                                  : 1 (0x1)
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Format version                           : Version 4
Muxing mode                              : ADTS
Codec ID                                 : 15-2
Duration                                 : 1 s 857 ms
Bit rate mode                            : Variable
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 22.05 kHz
Frame rate                               : 21.533 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -46 ms
Language                                 : frm

[Carl Eugen Hoyos]

Replying to jeanhackman:

However, I was just successful in creating an MPEG-TS container with MPEG-2 video and AAC audio including the "frm" audio language descriptor.

If you remux this file to mp4, does the resulting file show the desired language descriptor?

[jeanhackman]

yes

ffmpeg -i audio_map_test_files/test_pattern_fra51_fraDVS.ts \
-vcodec copy \
-acodec copy \
-map 0 \
audio_map_test_files/test_pattern_fra51_fraDVS.mp4

mediainfo audio_map_test_files/test_pattern_fra51_fraDVS.mp4
...
Audio #2
...
Language                                 : frm

[jeanhackman]

though, full disclosure, I abandoned the first output file created per comment 3 as x264 and ac3 are more applicable for my testing needs. here is the command line used to created the new "test_pattern_fra51_fraDVS.ts" file mentioned in comment 6:

/usr/local/Cellar/ffmpeg/4.1.3_1/bin/ffmpeg -i test_pattern.mov \
-i french_5.1_audio/mono_pcm_fra_front_left_audio.aiff \
-i french_5.1_audio/mono_pcm_fra_front_right_audio.aiff \
-i french_5.1_audio/mono_pcm_fra_front_center_audio.aiff \
-i french_5.1_audio/mono_pcm_fra_lfe_audio.aiff \
-i french_5.1_audio/mono_pcm_fra_rear_left_audio.aiff \
-i french_5.1_audio/mono_pcm_fra_rear_right_audio.aiff \
-i stereo_pcm_fra_dvs_audio.aiff \
-vcodec libx264 \
-map 0:v \
-acodec ac3 \
-map 1:a -metadata:s:a:0 language=fra \
-map 2:a -metadata:s:a:1 language=fra \
-map 3:a -metadata:s:a:2 language=fra \
-map 4:a -metadata:s:a:3 language=fra \
-map 5:a -metadata:s:a:4 language=fra \
-map 6:a -metadata:s:a:5 language=fra \
-map 7:a -metadata:s:a:6 language=frm \
audio_map_test_files/test_pattern_fraPCM_51_fraDVS.ts

[Carl Eugen Hoyos]

What is the actual application that you use to see the language description?

[jeanhackman]

mediainfo --version
MediaInfo Command line,
MediaInfoLib - v18.12

[Carl Eugen Hoyos]

No, I doubt that mediainfo can be used to do anything useful with a media file, my question is what actual application are you using?

[jeanhackman]

Replying to cehoyos:

No, I doubt that mediainfo can be used to do anything useful with a media file, my question is what actual application are you using?

I typically use Manzanita MP2TSAE for MPEG-TS container compliance/analysis and sometimes a command line tool that I wrote myself.
Here is output from MP2TSAE while digging into the PMT to look at the French DVS stream:

  Stream #8
    stream_type:	0x81 (User Private)
    reserved:	7
    elementary_PID:	0x107 (263)
    reserved:	15
    ES_info_length:	6
    ISO_639_language_descriptor
      tag:	0x0A (10)
      length:	4
      ISO_639_language_code:	0x66726D (frm)
      audio_type:	0 (Undefined)

As you can see the ISO_639_language_code is present and has value "frm" as expected.
Not that this is the place for debate, but I wholeheartedly disagree with your jab at mediainfo and can vouch that it is actually used in fortune 50, enterprise-level production workflows. As you can see, mediainfo in this case accurately represents the language code.

[Marton Balint]

As far as I see QuickTime only supports Macintosh language codes:

​https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap4/qtff4.html

FRM is not among those. Is there an application which generates a quicktime file with frm language?

[galad]

MOV supports both ISO 639-2/T and extended languages code too: ​https://developer.apple.com/library/archive/documentation/QuickTime/QTFF/QTFFChap2/qtff2.html#//apple_ref/doc/uid/TP40000939-CH204-SW16

[Marton Balint]

There is a note however in the specs:

Note: ISO language codes cannot be used for all elements of a QuickTime file. Currently, ISO language codes can be used only for user data text. All other elements, including text tracks, must be specified using Macintosh language codes.

So using extended language tags is the only way to support this?

[jeanhackman]

Replying to cus:

There is a note however in the specs:

Note: ISO language codes cannot be used for all elements of a QuickTime file. Currently, ISO language codes can be used only for user data text. All other elements, including text tracks, must be specified using Macintosh language codes.

So using extended language tags is the only way to support this?

Looks to be so, yes. Thanks for adding links to the documentation.

[llogan]

Related abandoned patch:

avformat/isom: allow ISO 639 codes for mov
​https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200620001006.159305-1-yonglel@google.com/