FFmpeg write flv to HttpFS(Hadoop) failed

[arlenliu]

Summary of the bug:
FFmpeg write flv to stdin,and send data to curl by pipe.
When press "q",ffmpeg report error:
[flv @ 029fab20] Failed to update header with correct duration.
[flv @ 029fab20] Failed to update header with correct filesize.

How to reproduce:
C:\Users\Administrator>"C:\ffmpeg-20150424\bin\ffmpeg.exe" -f dshow -i video="screen-capture-recorder" -f flv - | "C:\Program Files (x86)\curl\curl.exe" -X PUT -L "​http://myhttpfs.com:14000/webhdfs/v1/arlentest04151435.flv?op=CREATE&user.name=root&data=true" --header "Content-Type:application/octet-stream" --header "Transfer-Encoding:chunked" -T -
ffmpeg version N-71664-gcd69c0e Copyright (c) 2000-2015 the FFmpeg developers

built with gcc 4.9.2 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av

isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-l
ibilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enab
le-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --en
able-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --ena
ble-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc

--enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enabl

e-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --ena
ble-decklink --enable-zlib

libavutil 54. 23.101 / 54. 23.101
libavcodec 56. 35.101 / 56. 35.101
libavformat 56. 31.100 / 56. 31.100
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 14.100 / 5. 14.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100

Input #0, dshow, from 'video=screen-capture-recorder':

Duration: N/A, start: 369242.584000, bitrate: N/A

Stream #0:0: Video: rawvideo, bgr0, 1364x667, 30 tbr, 10000k tbn, 30 tbc

Output #0, flv, to 'pipe:':

Metadata:

encoder : Lavf56.31.100
Stream #0:0: Video: flv1 (flv) ([2][0][0][0] / 0x0002), yuv420p, 1364x667, q

=2-31, 200 kb/s, 30 fps, 1k tbn, 30 tbc

Metadata:

encoder : Lavc56.35.101 flv

Stream mapping:

Stream #0:0 -> #0:0 (rawvideo (native) -> flv1 (flv))

Press [q] to stop, ? for help
frame= 16 fps=0.0 q=31.0 size= 563kB time=00:00:00.53 bitrate=8659.6kbits/
frame= 31 fps= 31 q=31.0 size= 673kB time=00:00:01.03 bitrate=5337.5kbits/
frame= 47 fps= 31 q=31.0 size= 781kB time=00:00:01.56 bitrate=4081.8kbits/
frame= 62 fps= 31 q=31.0 size= 960kB time=00:00:02.06 bitrate=3806.0kbits/
frame= 77 fps= 30 q=31.0 size= 1068kB time=00:00:02.56 bitrate=3408.9kbits/
frame= 92 fps= 30 q=31.0 size= 1177kB time=00:00:03.06 bitrate=3144.9kbits/
[flv @ 029fab20] Failed to update header with correct duration.
[flv @ 029fab20] Failed to update header with correct filesize.
frame= 99 fps= 30 q=31.0 Lsize= 1285kB time=00:00:03.30 bitrate=3189.1kbits
/s
video:1283kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing
overhead: 0.136711%

best wished for you.

[Carl Eugen Hoyos]

Please test if one of the attached files work for you.

[arlenliu]

Thanks cehoyos.
I have test all the files attached,but the issue still exist.
I find the key problem is,if ffmpeg 's output is stdout(-) and filetype is flv,it will report these error:

Failed to update header with correct duration.
Failed to update header with correct filesize.

[Carl Eugen Hoyos]

How did you test the files I uploaded?

Are you just reporting that a warning is shown or is there an actual issue with the produced files?

[Carl Eugen Hoyos]

I uploaded a fourth file: Please test all four with HttpFS(Hadoop) and report if uploading succeeds for them or not.

[arlenliu]

The command i input by test is,
ffmpeg -i "c:\noduration.flv" -f flv pipe:1
ffmpeg -i "c:\nofilesize.flv" -f flv pipe:1
ffmpeg -i "c:\nothing.flv" -f flv pipe:1
The issue exist.

But,if i input:
ffmpeg -i "c:\noduration.flv" -f flv "c:\out.flv"
it is ok.

so i guess the problem is stdout,not HttpFS.

[arlenliu]

thanks

[Carl Eugen Hoyos]

What I meant was: What happens if you use the four files I uploaded instead of c:\out.flv? Does it work for some of the files but not for others? Which ones?

[arlenliu]

If only upload these files to Hadoop,it is ok

[Carl Eugen Hoyos]

All four are ok? Or are some ok and others not?

[Carl Eugen Hoyos]

I uploaded a fifth file: Please test uploading to Hadoop with all five files and report back: The issue is probably easy to fix but I cannot test myself. I will know how to fix the problem once you tested all five files.

[arlenliu]

Thanks.
you mean run this command?
"C:\Program Files (x86)\curl\curl.exe" -X PUT -L "​​http://myhttpfs.com:14000/webhdfs/v1/invalid.flv?op=CREATE&user.name=root&data=true" --header "Content-Type:application/octet-stream" --header "Transfer-Encoding:chunked" -T invalid.flv

[Carl Eugen Hoyos]

I think so, yes. Please test all five files I attached here.

[Carl Eugen Hoyos]

If the issue described here was only that FFmpeg shows an error for -f flv - then this is an invalid report: The errors should be shown for pipe output.
If there is something wrong with the uploaded files, please reopen this ticket: It may be possible to fix the issue.