concat regression with mp4

[compn]

posting this for user who does not want to make an account -compn

#This log and other links have expiration of 1 hour so copy and backup#
##I'll try to explain full story in stages in best possible way, bear with my english##
###I don't know if its a bug so decide yourself and if it comes out as a bug then submit it (I support FFMPEG :D)###

I was surfing WWW and suddenly I stumble upon a page which has a video embedded into it which I like and want to download and keep it for future reference. Its a Stem Cell Video claiming to provide cure for anything to everything concerned to humans. And the story starts :-

1st) a) original page link which has video embedded into it -> ​http://thelongevitystudy.com/160725A.php

b) page source and network monitor tool of browser tells that video link is -> ​http://video.limelight.com/player/fp10loader.swf?deepLink=true&playerForm=3408cce3252244e9b8db28df9f21f924&channelId=dd542dc018c44edc8bdcafe9cd63b1b7&85527519

c) video link mentioned in 1st)b) looks like its an swf player loading video from net, and not a direct link to video file. I narrow down the video link to ​http://video.limelight.com/player/fp10loader.swf?channelId=dd542dc018c44edc8bdcafe9cd63b1b7&85527519 i.e i)limelight player with ii)channel id - dd542dc018c44edc8bdcafe9cd63b1b7&85527519 *Its a video in 2 parts*

2nd) I use youtube-dl.exe to download video with command - youtube-dl.exe "limelight:channel:dd542dc018c44edc8bdcafe9cd63b1b7&85527519" It downloads both the parts in best quality. I now have 2 videos i.e 2 parts - p1 & p2 of a longer single video played online.

3rd) a) Now I wanna join and keep a single file. So I use ffmpeg to join both parts. I download ffmpeg latest version from site and join video files with it. ffmpeg version N-81664-g6f062eb

b) this new version of ffmpeg gives an output-file-joined.mp4 which has correct file size and time duration. This combined video file plays 1st part of video correctly but doesnt plays 2nd part video at all. However, audio is playing fine irrespective of video part.

4th) a) Now, I use an older version of ffmpeg which came automaically with "thekmplayer". its installed by defalt with and in the directory of "thekmplayer". ffmpeg version N-76347-gdd36749

b) now this older version of ffmpeg "thekmplayer version" joined both the parts (same video files as before) and produced a combined.mp4 file which is playing both the parts flawlessly. audio is also fine.

*Irrespective of files being joined, their type, make, codec, other details, etc and of ffmpeg older and newer version, thier build, code etc, one thing comes out in the end, that, files to be joined are not playing correctly when joined by new version of ffmpeg and are playing flawlessly when joined by old version of ffmpeg embedded into thekmplayer which concludes that there is something happening with newer version of ffmpeg itself. Other factors like files to be joined & system on which files are joined etc are constant. Audio is not being mentioned for the fat that it remains same in all scenarios & plays fine everytime.

Technical Details :-

5th) I used same command for joining on both the older and newer version of ffmpeg - "ffmpeg -f concat -i list.txt -c copy joined.mp4"

6th) cmd output - (first in cmd output is new version of ffmpeg and later output is of older ffmpeg version of kmplayer),(there is this message in new version cmd output inbetween if its relevant- [mp4 @ 00000000004be8c0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.) :-

new version ffmpeg cmd output :-
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
e:\toer>c:\Users\DELL\Desktop\ffmpeg-20160915-6f062eb-win64-static\bin\ffmpeg.exe -f concat -i lst.txt -c copy out1.mp4
ffmpeg version N-81664-g6f062eb Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.4.0 (GCC)

configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 29.100 / 55. 29.100
libavcodec 57. 55.101 / 57. 55.101
libavformat 57. 49.100 / 57. 49.100
libavdevice 57. 0.102 / 57. 0.102
libavfilter 6. 62.100 / 6. 62.100
libswscale 4. 1.100 / 4. 1.100
libswresample 2. 1.100 / 2. 1.100
libpostproc 54. 0.100 / 54. 0.100

[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000000492e40] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from 'lst.txt':

Duration: N/A, start: -0.042667, bitrate: 1191 kb/s

Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1024x576, 1063 kb/s, 23.98 fps, 23.98 tbr, 11988 tbn, 47.95 tbc
Metadata:

handler_name : VideoHandler

Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

handler_name : SoundHandler

[mp4 @ 00000000004be8c0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.

Last message repeated 1 times

Output #0, mp4, to 'out1.mp4':

Metadata:

encoder : Lavf57.49.100
Stream #0:0(eng): Video: h264 (High) ([33][0][0][0] / 0x0021), yuv420p, 1024x576, q=2-31, 1063 kb/s, 23.98 fps, 23.98 tbr, 11988 tbn, 11988 tbc
Metadata:

handler_name : VideoHandler

Stream #0:1(eng): Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo, 128 kb/s
Metadata:

handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)

Press [q] to stop, ? for help
frame= 5850 fps=0.0 q=-1.0 size= 40122kB time=00:04:03.96 bitrate=1347.2kbits/
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000000492e40] Auto-inserting h264_mp4toannexb bitstream filter
frame=12152 fps=12151 q=-1.0 size= 56177kB time=00:08:26.90 bitrate= 907.9kbit
frame=20249 fps=13499 q=-1.0 size= 70702kB time=00:14:04.56 bitrate= 685.8kbit
frame=28751 fps=14375 q=-1.0 size= 91217kB time=00:19:59.18 bitrate= 623.1kbit
frame=37600 fps=15040 q=-1.0 size= 103890kB time=00:26:08.27 bitrate= 542.7kbit
frame=46315 fps=15438 q=-1.0 size= 115903kB time=00:32:11.77 bitrate= 491.5kbit
frame=55117 fps=15747 q=-1.0 size= 128255kB time=00:38:18.88 bitrate= 457.0kbit
frame=64149 fps=16037 q=-1.0 size= 139814kB time=00:44:35.58 bitrate= 428.1kbit
frame=72858 fps=16190 q=-1.0 size= 149617kB time=00:50:38.84 bitrate= 403.3kbit
frame=77849 fps=16220 q=-1.0 Lsize= 157048kB time=00:54:07.12 bitrate= 396.2kbits/s speed= 677x
video:104520kB audio:50737kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.153978%

old version ffmpeg cmd output :-
e:\toer>"c:\Program Files (x86)\The KMPlayer\ffmpeg.exe" -f concat -i lst.txt -c copy out2.mp4
ffmpeg version N-76347-gdd36749 Copyright (c) 2000-2015 the FFmpeg developers built with gcc 5.2.0 (GCC)

configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enab

le-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib

libavutil 55. 5.100 / 55. 5.100
libavcodec 57. 12.100 / 57. 12.100
libavformat 57. 11.100 / 57. 11.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 14.101 / 6. 14.101
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.100 / 2. 0.100
libpostproc 54. 0.100 / 54. 0.100

[mov,mp4,m4a,3gp,3g2,mj2 @ 050086c0] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from 'lst.txt':

Duration: N/A, start: 0.000000, bitrate: 1191 kb/s

Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1024x576, 1063 kb/s, 23.98 fps, 23.98 tbr, 11988 tbn, 47.95 tbc
Metadata:

handler_name : VideoHandler

Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:

handler_name : SoundHandler

[mp4 @ 057e8840] Codec for stream 0 does not use global headers but container format requires global headers
[mp4 @ 057e8840] Codec for stream 1 does not use global headers but container format requires global headers
Output #0, mp4, to 'out2.mp4':

Metadata:

encoder : Lavf57.11.100
Stream #0:0(eng): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1024x576, q=2-31, 1063 kb/s, 23.98 fps, 23.98 tbr, 11988 tbn, 11988 tbc
Metadata:

handler_name : VideoHandler

Stream #0:1(eng): Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, 128 kb/s
Metadata:

handler_name : SoundHandler

Stream mapping:

Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)

Press [q] to stop, ? for help
frame= 3324 fps=0.0 q=-1.0 size= 21183kB time=00:02:18.60 bitrate=1252.0kbits/
frame= 6411 fps=6411 q=-1.0 size= 43814kB time=00:04:27.34 bitrate=1342.5kbits
[mov,mp4,m4a,3gp,3g2,mj2 @ 050ce100] Auto-inserting h264_mp4toannexb bitstream filter
frame= 9358 fps=6238 q=-1.0 size= 52066kB time=00:06:30.35 bitrate=1092.6kbits
frame=12705 fps=6352 q=-1.0 size= 56894kB time=00:08:49.96 bitrate= 879.4kbits
frame=16602 fps=6640 q=-1.0 size= 64181kB time=00:11:32.48 bitrate= 759.3kbits
frame=20320 fps=6773 q=-1.0 size= 70815kB time=00:14:07.55 bitrate= 684.5kbits
frame=24112 fps=6889 q=-1.0 size= 79628kB time=00:16:45.73 bitrate= 648.6kbits
frame=27776 fps=6944 q=-1.0 size= 89959kB time=00:19:18.50 bitrate= 636.1kbits
frame=31556 fps=7012 q=-1.0 size= 96231kB time=00:21:56.18 bitrate= 598.9kbits
frame=35498 fps=7099 q=-1.0 size= 101358kB time=00:24:40.57 bitrate= 560.8kbits
frame=39253 fps=7137 q=-1.0 size= 106051kB time=00:27:17.22 bitrate= 530.6kbits
frame=42915 fps=7152 q=-1.0 size= 111651kB time=00:29:49.97 bitrate= 511.0kbits
frame=46372 fps=7134 q=-1.0 size= 115960kB time=00:32:14.16 bitrate= 491.1kbits
frame=49801 fps=7114 q=-1.0 size= 120258kB time=00:34:37.18 bitrate= 474.3kbits
frame=52891 fps=7052 q=-1.0 size= 124666kB time=00:36:46.01 bitrate= 462.9kbits
frame=56257 fps=7032 q=-1.0 size= 129783kB time=00:39:06.41 bitrate= 453.1kbits
frame=59208 fps=6965 q=-1.0 size= 133683kB time=00:41:09.50 bitrate= 443.5kbits
frame=62633 fps=6959 q=-1.0 size= 138376kB time=00:43:32.33 bitrate= 433.9kbits
frame=65855 fps=6932 q=-1.0 size= 141599kB time=00:45:46.73 bitrate= 422.3kbits
frame=69470 fps=6947 q=-1.0 size= 145235kB time=00:48:17.51 bitrate= 410.6kbits
frame=72528 fps=6907 q=-1.0 size= 149298kB time=00:50:25.06 bitrate= 404.3kbits
frame=75807 fps=6891 q=-1.0 size= 152841kB time=00:52:41.83 bitrate= 396.0kbits
frame=77849 fps=6874 q=-1.0 Lsize= 157078kB time=00:54:07.12 bitrate= 396.3kbits/s
video:104550kB audio:50737kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.153785%

8th) cmd window youtube-dl output (youtube-dl downloads any link video with best quality by default or we can manually select it) :-

a) firstly, I get link and its files info with command - youtube-dl -F "limelight:channel:dd542dc018c44edc8bdcafe9cd63b1b7"

(capital -F option is used to get info and its different from small -f option which is for format selection). Here we can see that best quality is listed for both parts & have same details for both parts which are downloaded later on as p1 and p2.

cmd output for info :-
C:\Users\DELL>Desktop\youtube-dl.exe -F "limelight:channel:dd542dc018c44edc8bdcafe9cd63b1b7&85527519"
[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading PlaylistService getPlaylistByChannelId JSON
[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading API media JSON

[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading PlaylistService getMobilePlaylistWithNItemsByChannelId?begin=0&count=-1 JSON
[download] Downloading playlist: LCR_Vital_Stem_VSL_160725A
[limelight:channel] playlist LCR_Vital_Stem_VSL_160725A: Collected 2 video ids (downloading 2 of them)
[download] Downloading video 1 of 2
[info] Available formats for 33c9591cf925413c909f3932e91cbb54:
format code extension resolution note
Mobile3gp 3gp unknown
MobileH264 mp4 unknown
rtmp-128 flv 480x270 224k video@ 128k, 23.976fps, audio@ 96k
http-128 mp4 480x270 224k video@ 128k, 23.976fps, audio@ 96k
rtmp-322 flv 480x270 450k video@ 322k, 23.976fps, audio@128k
http-322 mp4 480x270 450k video@ 322k, 23.976fps, audio@128k
rtmp-472 flv 480x270 600k video@ 472k, 23.976fps, audio@128k
http-472 mp4 480x270 600k video@ 472k, 23.976fps, audio@128k
rtmp-772 flv 1024x576 900k video@ 772k, 23.976fps, audio@128k
http-772 mp4 1024x576 900k video@ 772k, 23.976fps, audio@128k
rtmp-1072 flv 1024x576 1200k video@1072k, 23.976fps, audio@128k
http-1072 mp4 1024x576 1200k video@1072k, 23.976fps, audio@128k (best)
[download] Downloading video 2 of 2
[info] Available formats for db80e21ebd6a4310bab041fe8f3a5904:
format code extension resolution note
Mobile3gp 3gp unknown
MobileH264 mp4 unknown
rtmp-128 flv 480x270 224k video@ 128k, 23.976fps, audio@ 96k
http-128 mp4 480x270 224k video@ 128k, 23.976fps, audio@ 96k
rtmp-322 flv 480x270 450k video@ 322k, 23.976fps, audio@128k
http-322 mp4 480x270 450k video@ 322k, 23.976fps, audio@128k
rtmp-472 flv 480x270 600k video@ 472k, 23.976fps, audio@128k
http-472 mp4 480x270 600k video@ 472k, 23.976fps, audio@128k
rtmp-772 flv 1024x576 900k video@ 772k, 23.976fps, audio@128k
http-772 mp4 1024x576 900k video@ 772k, 23.976fps, audio@128k
rtmp-1072 flv 1024x576 1200k video@1072k, 23.976fps, audio@128k
http-1072 mp4 1024x576 1200k video@1072k, 23.976fps, audio@128k (best)
[download] Finished downloading playlist: LCR_Vital_Stem_VSL_160725A

b) Now, files download at their best quality (default) with command - youtube-dl "limelight:channel:dd542dc018c44edc8bdcafe9cd63b1b7"

Here thing to be noted is that after download completes and 2parts p1 & p2 are obtained, an operating system (win7) file properties details dialog is checked for both files which gives details for p1 file as "resolution=1024x576 data.rate=1064kbps total.bitrate=1192kbps frame.rate=23fps". For 2nd part its a little bit strange as details show "resolution=1024x576 data.rate=168kbps total.bitrate=296kbps frame.rate=23fps", I re-downloaded 2nd part i.e p2 file with "-f https-1072" option for best quality but resulting file showed same details as before.

cmd output for download of 2parts p1 & p2 :-

e:\toer>c:\users\dell\Desktop\youtube-dl.exe "limelight:channel:dd542dc018c44edc8bdcafe9cd63b1b7&85527519"
[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading PlaylistService getPlaylistByChannelId JSON
[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading API media JSON

[limelight:channel] dd542dc018c44edc8bdcafe9cd63b1b7: Downloading PlaylistService getMobilePlaylistWithNItemsByChannelId?begin=0&count=-1 JSON
[download] Downloading playlist: LCR_Vital_Stem_VSL_160725A
[limelight:channel] playlist LCR_Vital_Stem_VSL_160725A: Collected 2 video ids (downloading 2 of them)
[download] Downloading video 1 of 2
[download] Destination: LCR_Vital_Stem_VSL_160725A_p1-33c9591cf925413c909f3932e91cbb54.mp4
[download] 100% of 49.40MiB in 06:24
[download] Downloading video 2 of 2
[download] Destination: LCR_Vital_Stem_VSL_160725A_p2-db80e21ebd6a4310bab041fe8f3a5904.mp4
[download] 100% of 103.97MiB in 06:58
[download] Finished downloading playlist: LCR_Vital_Stem_VSL_160725A

9th) a) win7 file property dialogue tells all the files - 2 separate video parts considered combined and both the older and newer version ffmpeg created joined files have the same file.size=153MB, duration=54:07, resolution=1024x576, frame rate=23fps, container=mp4. However, according to same file property dialogue, 1st part p1 data.rate=1064kbps and total.bitrate=1192kbps and for 2nd part p2 data.rate=168kbps total.bitrate=296kbps. Incase of both joined output files from newer & older version of ffmpeg, data.rate=263kbps, total.bitrate=391kbps which is different from original files p1 & p2.

b) VLC Media Codec Info tool details that every property of all the file are same, except frame rate which is 23.976000 for original 2parts p1 & p2 and 23.975358 for joined file created from newer version of ffmpeg and 23.975357 for joined file created by older version of ffmpeg respectively.

[Carl Eugen Hoyos]

Please reopen if you can provide part 1 and part 2 and the file lst.txt.