#11056 closed defect (invalid)
Position of key frame incorrect
Reported by: | David Johansen | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | avformat |
Version: | 6.1.1 | Keywords: | ffprobe pos mp4 |
Cc: | David Johansen, MasterQuestionable | Blocked By: | |
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
How to reproduce:
% ffprobe -show_entries packet=pos,flags -of csv=p=0 -i kf_pos_test.mp4 | grep K ffprobe version 6.1.1 Copyright (c) 2007-2023 the FFmpeg developers built with gcc 12 (Debian 12.2.0-14) configuration: --prefix=/usr --enable-shared --disable-static --disable-debug --disable-doc --disable-ffplay --enable-gpl --enable-libfdk_aac --enable-libx264 --enable-libx265 --enable-librsvg --enable-openssl --enable-nonfree --enable-libaom --extra-libs=-lpthread libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'kf_pos_test.mp4': Metadata: major_brand : iso5 minor_version : 512 compatible_brands: iso5iso6mp41 encoder : Lavf60.16.100 Duration: 00:00:09.92, start: 0.224000, bitrate: 835 kb/s Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv), 1280x720, 748 kb/s, 15.15 fps, 15.15 tbr, 90k tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] 1158,K__ 106174,K__ 205264,K__ 309370,K__ 414778,K__ 516205,K__ 620919,K__ 725663,K__ 827272,K__ 931036,K__
That last entry should be 930976
so it's 60 bytes later than it should be. What can I do to help diagnose the cause of this?
Here's a file to demonstrate the issue:
https://drive.google.com/file/d/1kE9eNNYnBq_Lr-il2YWXGH4nOZLWs2Nx/view?usp=sharing
Change History (21)
comment:2 by , 6 months ago
Looks like this was flagged as similar because they both have a Google Drive link, because looking at the actual ticket, they are very different
comment:3 by , 6 months ago
͏ Sorry for the confusion.
͏ The Google download link is interpreted from yours. (optimized delivery)
comment:4 by , 6 months ago
Cc: | added |
---|---|
Component: | undetermined → avformat |
Keywords: | mp4 added |
͏ Remuxing ("-c copy") again to MP4 via FFmpeg, or to MKV:
͏ Would the problem still persist?
comment:5 by , 6 months ago
How would I verify the result? In this case, I know what the results should be because I catted the keyframes together into a single file, but if I remuxed then I wouldn't have that info
comment:6 by , 6 months ago
Use hex editor and find out what is the offset.
Ib sill do not get why does this matter. What is your need to know the perfect offset of last keyframe?
Does it happen on other keyframes not in the end (on other files)?
comment:7 by , 6 months ago
͏ No wonder then...
͏ The MP4 format may be a bit more (or much more...) complicated than that: mere concatenation may not properly present.
͏ In particular, it has the concepts similar to Matroska's Cluster.
comment:8 by , 6 months ago
͏ "why does this matter"
<^> Theoretical completeness.
͏ And potential hidden problems.
͏ Significant difference, tends to be caused by insignificant things.
follow-up: 10 comment:9 by , 6 months ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
There is 0x00015827 at offset 931036. This corresponds to a NAL unit with size 88103 which fits nicely with the packet size 88107 as reported by ffprobe. There is 0x0000047C at offset 930976. This is already invalid with a NALU length size of two. With a NALU length size of three, this would be a NALU with nal_unit_type 28 -- one of the unspecified ones. The next NALU would then have a size of 0x340000, which is larger than the whole file. With a nal unit length size of four, the next NALU has a length of more than 2GiB.
To sum it up, your offset 930976 (for which you provided no real explanation) is just wrong.
comment:10 by , 6 months ago
Replying to Balling:
Use hex editor and find out what is the offset.
Ib sill do not get why does this matter. What is your need to know the perfect offset of last keyframe?
Does it happen on other keyframes not in the end (on other files)?
Yes, I will find an example where it happens in the middle of the file
Replying to MasterQuestionable:
͏ No wonder then...
͏ The MP4 format may be a bit more (or much more...) complicated than that: mere concatenation may not properly present.
͏ In particular, it has the concepts similar to Matroska's Cluster.
I misrepresented what's being done slightly. These are fragmented mp4 files that are being made by the HLS muxer that I'm then combining into a single file so they're simpler to store/serve, but I need to know the location of the keyframes to be able to provide the necessary EXT-X-BYTERANGE
info
Replying to mkver:
There is 0x00015827 at offset 931036. This corresponds to a NAL unit with size 88103 which fits nicely with the packet size 88107 as reported by ffprobe. There is 0x0000047C at offset 930976. This is already invalid with a NALU length size of two. With a NALU length size of three, this would be a NALU with nal_unit_type 28 -- one of the unspecified ones. The next NALU would then have a size of 0x340000, which is larger than the whole file. With a nal unit length size of four, the next NALU has a length of more than 2GiB.
To sum it up, your offset 930976 (for which you provided no real explanation) is just wrong.
Using the values that I provided for EXT-X-BYTERANGE
it will playback in Video.js and Safari, but using the values that ffprobe
provides it stops, so that's my current criteria for deciding if it's "correct"
comment:11 by , 6 months ago
͏ Mere concatenation of bunch of MP4 cannot end in valid MP4...
͏ You have some misunderstanding on "concatenation".
͏ See also:
͏ https://stackoverflow.com/questions/35177797/what-is-fragmented-mp4-fmp4-and-how-is-it-different-than-normal-mp4#35180327
comment:12 by , 6 months ago
Mere concatenation of bunch of MP4 cannot end in valid
Yet segments of fmp4 do end up as perfect and valid mp4.
comment:13 by , 6 months ago
but using the values that ffprobe provides it stops
Then this is a bug, as many of them in Video.js
comment:14 by , 6 months ago
͏ Seriously..?
͏ Won't the container data clash and do garbage?
͏ ----
͏ Unsure if it's Video.js bug: as much is unknown about the actual file.
͏ See also: https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.2.2
͏ Probably didn't set EXT-X-BYTERANGE correctly..?
͏ Seemingly problem in the definition:
͏ "If o is not present, a previous Media Segment MUST appear in the Playlist file and MUST be a sub-range of the same media resource, or ..."
<^> What must be a sub-range of?
comment:15 by , 4 months ago
Here's an example video that I used to demonstrate what I'm doing and that the expected values don't match up:
https://drive.google.com/file/d/1WxOrSi-GNB45nLUUiR4PT7c4H2VurtKk/view?usp=sharing
Run this command to make a fragmented mp4 with each fragment having a single key frame in it:
ffmpeg -i input.mp4 -c:v libx264 -copyts -muxdelay 0 -force_key_frames source -hls_time 0.9 -hls_init_time 0.9 -hls_list_size 0 -hls_flags independent_segments -hls_segment_type fmp4 -f hls index.m3u8
Then combine the files into a single file by starting with the init:
cat init.mp4 > output.mp4
And then adding the rest of the files:
for f in index*.m4s; do echo $f; cat $f >> output.mp4; done
I would expect that the size and position of the keyframes would match up with the file sizes, but that's not the case:
ffprobe -show_entries packet=pos,flags -of csv=p=0 -i output.mp4 | grep K
Is there something I'm doing incorrectly or an assumption I'm making that's not correct?
follow-up: 17 comment:16 by , 4 months ago
͏ "extra_keyframes_example.mp4":
͏ https://drive.usercontent.google.com/download?export=download&id=1WxOrSi-GNB45nLUUiR4PT7c4H2VurtKk
͏ (~ 18.34 MiB)
͏ Simple concatenation via `cat` alike seems to do garbage.
͏ Your commands also look problematic. (haven't detailedly verified)
comment:17 by , 4 months ago
Replying to MasterQuestionable:
͏ Simple concatenation via `cat` alike seems to do garbage.
͏ Your commands also look problematic. (haven't detailedly verified)
The resulting video plays correctly in QuickTime and VLC and isn't HLS playback of a fragmented MP4 doing this as well?
comment:18 by , 4 months ago
͏ The structure of fMP4 shown in comment:11 linked clearly indicated something:
͏ That may be raw-concat'd is not the whole container, but only certain elements.
͏ FFmpeg currently may not handle non-perfectly-valid files in the most sensible way:
͏ https://trac.ffmpeg.org/ticket/10930#comment:5
͏ https://trac.ffmpeg.org/ticket/10008#comment:7
͏ https://trac.ffmpeg.org/ticket/11183#comment:1
͏ However, for your case: the application just seems to tolerate your error...
comment:19 by , 3 months ago
I'm definitely willing to adjust the way I process/create the files to do it the right way, so could you help me understand what needs to be adjusted to accomplish that?
From my understanding, Fragmented MPEG-4 is so the init section and the data are separated and can be combined like this for HLS playback, but is that incorrect?
comment:20 by , 3 months ago
͏ I don't use HLS. So my knowledge on creating which would be limited.
͏ As for fMP4, highlight from the previous linked:
͏ https://i.sstatic.net/jc4x3.png
comment:21 by , 3 months ago
Yes, the generated files using the commands I sent above follow that and conform to the description of Fragmented MPEG-4 in the HLS spec with it using moof
rather than moov
͏ Do alike:
͏ https://trac.ffmpeg.org/attachment/ticket/11052/sample.mp4.xml
͏ https://drive.usercontent.google.com/download?export=download&id=1kE9eNNYnBq_Lr-il2YWXGH4nOZLWs2Nx
͏ (~ 1,011.36 KiB; MP4)