#7203 closed defect (invalid)
Problem with encoding type "Cyrillic (DOS)" with metadata delivery.
| Reported by: | max79 | Owned by: | |
|---|---|---|---|
| Priority: | normal | Component: | undetermined |
| Version: | unspecified | Keywords: | id3v2 |
| Cc: | Blocked By: | ||
| Blocking: | Reproduced by developer: | no | |
| Analyzed by developer: | no |
Description
Summary of the bug:
TITLE metadata tag should looks like this- "Евангелие о следовании за Христом" for my example below.
To reproduce:
ffmpeg version N-91024-g293a6e8332 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
libavutil 56. 18.100 / 56. 18.100
libavcodec 58. 19.101 / 58. 19.101
libavformat 58. 13.102 / 58. 13.102
libavdevice 58. 4.100 / 58. 4.100
libavfilter 7. 21.100 / 7. 21.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libpostproc 55. 2.100 / 55. 2.100
[mp3 @ 0000000000498380] Skipping 313 bytes of junk at 4096.
[mp3 @ 0000000000498380] Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from 'https://audio3.azbyka.ru/Svjashhennoe_pisanie/nikola_serbski_tolk/01_Evangelie_o_sledovanii_za_Xristom.mp3':
Metadata:
title : ├Е├в├а├н├г├е├л├и├е ├о ├▒├л├е├д├о├в├а├н├и├и ├з├а ├Х├░├и├▒├▓├о├м
genre : Other
album : ├Б├е├▒├е├д├╗ ├н├а ├Е├в├а├н├г├е├л├и├┐ 1├╖.
track : 1
artist : ├С├в├┐├▓├и├▓├е├л├╝ ├Н├и├к├о├л├а├й ├С├е├░├б├▒├к├и├й
id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e
id3v2_priv.AverageLevel: {\x04\x00\x00
id3v2_priv.PeakValue: !\x00\x00\x00
Duration: 00:39:40.34, start: 0.000000, bitrate: 96 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp, 96 kb/s
At least one output file must be specified
Attachments (1)
Change History (13)
comment:1 by , 6 years ago
| Component: | ffmpeg → undetermined |
|---|---|
| Priority: | important → normal |
by , 6 years ago
| Attachment: | 01-vstuplenie.mp3 added |
|---|
Problem with encoding type "Cyrillic (DOS)" with metadata delivery.
comment:2 by , 6 years ago
Please see attachment. TITLE metadata tag from this example should looks like this:
"Дорожка 1"
but now it looks like:
title : ├Д├о├░├о├ж├к├а 1
Anyway you can download it from here: https://azbyka.ru/audio/audio1/Zhitija-i-tvorenija-svjatykh/Nikolaj-Serbskij/molytvy-na-ozere/01-vstuplenie.mp3
comment:3 by , 6 years ago
This file uses id3v2.3 tags. The TIT2-tag (the tag containing the title) is as follows in hex: 0x54 49 54 32 00 00 00 0A 00 00 00 C4 EE F0 EE E6 EA E0 20 31. According to the standard the 0x00 after the length field indicates that the tag uses ISO-8859-1 as encoding, an encoding that does not contain cyrillic characters. For such purposes Unicode could (and should) be used, but isn't. This is a bug in the tool that created said file, not in FFmpeg.
Btw: The last nine bytes are the actual titel; in Windows-1251 they would be read as "Дорожка 1"; in the Cyrillic DOS code page 866 that you are referring to it means "─юЁюцър 1". In ISO-8859-1 they mean "Äîðîæêà 1". FFmpeg's output to the console is encoded as UTF-8, but cmd.exe (that you seem to use) expects applications to use the native legacy codepage of the system (for Russian Windows versions, this is usually Code page 866; cmd.exe is by the way Unicode compatible). The UTF-8 that FFmpeg writes to the console is 0xC3 84 C3 AE C3 B0 C3 AE C3 A6 C3 AA C3 A0 20 31. In CP 866 0xC3 is "├" whereas 0x84 is "Д". That six of the seven characters of the word (seem to) have been preserved does not really have a deeper meaning. It is accidental.
comment:4 by , 6 years ago
Thank you for clarification. But if it's not ffmpeg bug why you didn't close this ticket?
follow-up: 7 comment:5 by , 6 years ago
I am not an FFmpeg developer, but just an interested visitor to this board. I don't know whether I am allowed to close tickets and therefore I leave it to the real guys in charge.
comment:6 by , 6 years ago
| Keywords: | id3v2 added |
|---|---|
| Resolution: | → invalid |
| Status: | new → closed |
comment:7 by , 6 years ago
Replying to mkver:
I am not an FFmpeg developer, but just an interested visitor to this board. I don't know whether I am allowed to close tickets and therefore I leave it to the real guys in charge.
Please go ahead and close tickets if you think they should be closed, and provide an explanation why it should be closed (like you did in comment:3). If someone disagrees they can always add a comment and/or re-open it.
comment:8 by , 5 years ago
| Resolution: | invalid |
|---|---|
| Status: | closed → reopened |
So, you say this is not bug in ffmpeg. I believe you. But if russian characters are in PATH in windows the problem it the same! This IS A BUG! The problem is only on windows))
PS C:\Users\xxxx> L:\5777\ffmpeg-20190911-944d76a-win64-static\bin\ffprobe.exe "Z:\Басков Николай (Flac)\2003 - Сила любви (избранное)\Николай Басков - Сила любви (избранное).flac"
ffprobe version N-94862-g944d76a3e0 Copyright (c) 2007-2019 the FFmpeg developers
built with gcc 9.1.1 (GCC) 20190807
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 35.100 / 56. 35.100
libavcodec 58. 56.101 / 58. 56.101
libavformat 58. 32.104 / 58. 32.104
libavdevice 58. 9.100 / 58. 9.100
libavfilter 7. 58.102 / 7. 58.102
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, flac, from 'Z:\╨С╨░╤Б╨║╨╛╨▓ ╨Э╨╕╨║╨╛╨╗╨░╨╣ (Flac)\2003 - ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡)\╨Э╨╕╨║╨╛╨╗╨░╨╣ ╨С╨░╤Б╨║╨╛╨▓ - ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡).flac':
Metadata:
ARTIST : ╨Э╨╕╨║╨╛╨╗╨░╨╣ ╨С╨░╤Б╨║╨╛╨▓
TITLE : O sole Mio
ALBUM : ╨б╨╕╨╗╨░ ╨╗╤О╨▒╨▓╨╕ (╨╕╨╖╨▒╤А╨░╨╜╨╜╨╛╨╡)
DATE : 2001
track : 02
GENRE : Vocal
COMMENT : Track 2
Duration: 01:05:28.69, start: 0.000000, bitrate: 844 kb/s
Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
comment:9 by , 5 years ago
Wow!!! It is your problem after all even in Metadata section! Here what it looks like in linux (compare to prev. comment):
ffprobe ./Николай\ Басков\ -\ Сила\ любви\ \(избранное\).flac
ffprobe version 4.1.4-1+b2 Copyright (c) 2007-2019 the FFmpeg developers
built with gcc 9 (Debian 9.2.1-1)
configuration: --prefix=/usr --extra-version=1+b2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, flac, from '/mnt/hgfs/Eurovision Song Contest 2019. Tel Aviv, Israel/Николай Басков - Сила любви (избранное).flac':
Metadata:
ARTIST : Николай Басков
TITLE : O sole Mio
ALBUM : Сила любви (избранное)
DATE : 2001
track : 02
GENRE : Vocal
COMMENT : Track 2
Duration: 01:05:28.69, start: 0.000000, bitrate: 844 kb/s
Stream #0:0: Audio: flac, 44100 Hz, stereo, s16
comment:10 by , 5 years ago
| Status: | reopened → open |
|---|
https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=7520&p=16316&hilit=cyrillic#top
As you can see this problem is common!!
comment:11 by , 5 years ago
| Resolution: | → invalid |
|---|---|
| Status: | open → closed |
This issue is about two things:
- A file that claims to use cp ISO-8859-1, but actually isn't (in the sense that when the bytes are interpreted via ISO-8859-1 the result is not what the user expected). As explained, the file in question shows the unexpected "Äîðîæêà 1" even when the shell expects UTF-8 input.
- cmd.exe showing an even weirder output than "Äîðîæêà 1" (and doing so even for valid files) because ffmpeg outputs UTF-8 whereas cmd.exe expects something different.
Your issue is lacking the first part and the second part is a duplicate of #5398. I therefore reclose this.
comment:12 by , 4 years ago
So, #5398 was fixed, https://azbyka.ru/audio/audio1/Zhitija-i-tvorenija-svjatykh/Nikolaj-Serbskij/molytvy-na-ozere/01-vstuplenie.mp3 now works correctly (except for
id3v2_priv.AverageLevel: {\x0f\x00\x00
id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e
id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
id3v2_priv.WM/UniqueFileIdentifier: ;\x00\x00\x00
id3v2_priv.PeakValue: !f\x00\x00
and the attached file has as mkver correctly said
ffprobe version git-2020-02-27-9b22254 Copyright (c) 2007-2020 the FFmpeg developers
built with gcc 9.2.1 (GCC) 20200122
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 42.100 / 56. 42.100
libavcodec 58. 73.102 / 58. 73.102
libavformat 58. 39.101 / 58. 39.101
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.100 / 7. 77.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[mp3 @ 00000208b1eec240] Estimating duration from bitrate, this may be inaccurate
Input #0, mp3, from 'C:\Users\ZAQU\Downloads\01-vstuplenie.mp3':
Metadata:
title : Äîðîæêà 1
album : Íåèçâåñòíûé äèñê (05.02.2009 10:58:36)
track : 1
TLEN : 67240
id3v2_priv.WM/UniqueFileIdentifier: ;\x00\x00\x00
id3v2_priv.WM/MediaClassSecondaryID: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
id3v2_priv.WM/MediaClassPrimaryID: \xbc}`\xd1#\xe3\xe2K\x86\xa1H\xa4*(D\x1e
id3v2_priv.AverageLevel: {\x0f\x00\x00
id3v2_priv.PeakValue: !f\x00\x00
Duration: 00:01:07.04, start: 0.000000, bitrate: 128 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s
Please provide an input sample.