Opened 13 months ago
Last modified 11 months ago
#10565 new defect
Error submitting a packet to the muxer: Broken pipe, Error muxing a packet
Reported by: | Rob | Owned by: | |
---|---|---|---|
Priority: | normal | Component: | undetermined |
Version: | unspecified | Keywords: | Error submitting a packet to the muxer: Broken pipe, Error muxing a packet |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
I am streaming a video (mp4) to a RTMP destination and hitting this error. The RTMP destination is receiving the data and creating HLS TS files from it - using nginx RTMP module.
Once this error it hit, FFMPEG keeps going, but the server stops creating TS files at that point.
[aost#0:1/copy @ 0x135ab40] Error submitting a packet to the muxer: Broken pipe/s speed=1.05x
[out#0/flv @ 0x1359700] Error muxing a packet
[flv @ 0x13c96c0] Failed to update header with correct duration.ate= 17.9kbits/s speed=1.04x
[flv @ 0x13c96c0] Failed to update header with correct filesize.
[out#0/flv @ 0x1359700] Error writing trailer: Broken pipe
[out#0/flv @ 0x1359700] Error closing file: Broken pipe
How to reproduce:
ffmpeg -re -i low.mp4 -map 0 -c copy -f flv rtmp://127.0.0.1:1935/<my stream key> ffmpeg version N-112061-g654e4b00e2 Copyright (c) 2000-2023 the FFmpeg developers
I have attached a ZIP of the file I am using to reproduce it.
Change History (13)
comment:1 by , 13 months ago
comment:2 by , 13 months ago
Here is the file I use to reproduce please let me know if you have trouble downloading it:
https://www.dropbox.com/scl/fi/5ybita2l41tr7tyhzgy25/low.mp4?rlkey=gzahw6wp6soytuyc2185zs4ef&dl=0
comment:3 by , 13 months ago
It looks like this is a similar issue when this command is run:
ffmpeg -stream_loop -1 -re -i low2.mp4 -map 0 -c copy -f flv rtmp://127.0.0.1:1935/<stream key>
This file plays fine once, but when it repeats (start over) it gives the broken pipe and causes a problem.
av_interleaved_write_frame(): Broken pipesize= 71202kB time=00:05:10.50 bitrate=1878.6kbits/s speed= 1x
[out#0/flv @ 0x7fecbe7124c0] Error muxing a packet
Here is the file to test with:
https://www.dropbox.com/scl/fi/0ark8tz3d9p0qjqgtuk8w/low2.mp4?rlkey=bvkirv59rtxut0933kkc0rueg&dl=0
comment:4 by , 13 months ago
Please provide the command line you tested together with the complete, uncut console output to make this a valid question.
comment:5 by , 13 months ago
(base) adeMacBook-Pro-2:ufbuild liuqi$ ./ffmpeg -i ~/kuaishou/Test/low.mp4 -c copy -t 10 out.flv ffmpeg(82080,0x7ff85bdb6640) malloc: nano zone abandoned due to inability to reserve vm space. ffmpeg version N-112061-g654e4b00e2 Copyright (c) 2000-2023 the FFmpeg developers built with Apple clang version 14.0.3 (clang-1403.0.22.14.1) configuration: --prefix=/usr/local/ --libdir=/usr/local/lib/ --cc=clang --enable-htmlpages --enable-libx264 --enable-gpl --extra-cflags='-I/usr/local/include/GraphicsMagick/ -I/usr/local/opt/openssl@3.1/include/ -O0 -g3 -fsanitize=address -Wno-error -fPIC -I/usr/local/include -fno-omit-frame-pointer' --extra-ldflags='-O0 -g3 -fsanitize=address -fno-omit-frame-pointer -Wno-error -fPIC -L/usr/local/lib -L/usr/local/opt/openssl@3.1/lib -lpng' --enable-libfreetype --enable-fontconfig --enable-libspeex --enable-libopus --enable-libzmq --enable-libx265 --enable-libass --enable-videotoolbox --disable-optimizations --enable-audiotoolbox --enable-opengl --disable-stripping --samples=../../fate-suite/ --enable-encoder=hevc_videotoolbox --enable-hwaccel=hevc_videotoolbox --enable-hwaccel=h264_videotoolbox --enable-openssl --enable-nonfree --disable-shared --enable-libfdk_aac --enable-libxml2 --enable-libmp3lame --enable-lcms2 --enable-libwebp --enable-libvpx --enable-libaom libavutil 58. 24.100 / 58. 24.100 libavcodec 60. 26.100 / 60. 26.100 libavformat 60. 12.100 / 60. 12.100 libavdevice 60. 2.101 / 60. 2.101 libavfilter 9. 11.100 / 9. 11.100 libswscale 7. 3.100 / 7. 3.100 libswresample 4. 11.100 / 4. 11.100 libpostproc 57. 2.100 / 57. 2.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/liuqi/kuaishou/Test/low.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf59.27.100 Duration: 00:08:44.33, start: 0.000000, bitrate: 677 kb/s Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 384x216 [SAR 1:1 DAR 16:9], 349 kb/s, 30 fps, 30 tbr, 15360 tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] encoder : Lavc59.37.100 h264_nvenc timecode : 01:00:00:00 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) Metadata: handler_name : VideoHandler timecode : 01:00:00:00 Output #0, flv, to 'out.flv': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.12.100 Stream #0:0(und): Video: h264 (Constrained Baseline) ([7][0][0][0] / 0x0007), yuv420p(tv, bt709, progressive), 384x216 [SAR 1:1 DAR 16:9], q=2-31, 349 kb/s, 30 fps, 30 tbr, 1k tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] encoder : Lavc59.37.100 h264_nvenc timecode : 01:00:00:00 Stream #0:1(und): Audio: aac (LC) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 319 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help [out#0/flv @ 0x611000003100] video:431kB audio:392kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.729315% frame= 303 fps=0.0 q=-1.0 Lsize= 837kB time=00:00:09.98 bitrate= 687.2kbits/s speed= 991x
write flv ok.
(base) adeMacBook-Pro-2:ufbuild liuqi$ ./ffmpeg -re -i ~/kuaishou/Test/low.mp4 -c copy -f flv rtmp://127.0.0.1/stream/live ffmpeg(82623,0x7ff85bdb6640) malloc: nano zone abandoned due to inability to reserve vm space. ffmpeg version N-112061-g654e4b00e2 Copyright (c) 2000-2023 the FFmpeg developers built with Apple clang version 14.0.3 (clang-1403.0.22.14.1) configuration: --prefix=/usr/local/ --libdir=/usr/local/lib/ --cc=clang --enable-htmlpages --enable-libx264 --enable-gpl --extra-cflags='-I/usr/local/include/GraphicsMagick/ -I/usr/local/opt/openssl@3.1/include/ -O0 -g3 -fsanitize=address -Wno-error -fPIC -I/usr/local/include -fno-omit-frame-pointer' --extra-ldflags='-O0 -g3 -fsanitize=address -fno-omit-frame-pointer -Wno-error -fPIC -L/usr/local/lib -L/usr/local/opt/openssl@3.1/lib -lpng' --enable-libfreetype --enable-fontconfig --enable-libspeex --enable-libopus --enable-libzmq --enable-libx265 --enable-libass --enable-videotoolbox --disable-optimizations --enable-audiotoolbox --enable-opengl --disable-stripping --samples=../../fate-suite/ --enable-encoder=hevc_videotoolbox --enable-hwaccel=hevc_videotoolbox --enable-hwaccel=h264_videotoolbox --enable-openssl --enable-nonfree --disable-shared --enable-libfdk_aac --enable-libxml2 --enable-libmp3lame --enable-lcms2 --enable-libwebp --enable-libvpx --enable-libaom libavutil 58. 24.100 / 58. 24.100 libavcodec 60. 26.100 / 60. 26.100 libavformat 60. 12.100 / 60. 12.100 libavdevice 60. 2.101 / 60. 2.101 libavfilter 9. 11.100 / 9. 11.100 libswscale 7. 3.100 / 7. 3.100 libswresample 4. 11.100 / 4. 11.100 libpostproc 57. 2.100 / 57. 2.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/liuqi/kuaishou/Test/low.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf59.27.100 Duration: 00:08:44.33, start: 0.000000, bitrate: 677 kb/s Stream #0:0[0x1](und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 384x216 [SAR 1:1 DAR 16:9], 349 kb/s, 30 fps, 30 tbr, 15360 tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] encoder : Lavc59.37.100 h264_nvenc timecode : 01:00:00:00 Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 319 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) Metadata: handler_name : VideoHandler timecode : 01:00:00:00 Output #0, flv, to 'rtmp://127.0.0.1/stream/live': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.12.100 Stream #0:0(und): Video: h264 (Constrained Baseline) ([7][0][0][0] / 0x0007), yuv420p(tv, bt709, progressive), 384x216 [SAR 1:1 DAR 16:9], q=2-31, 349 kb/s, 30 fps, 30 tbr, 1k tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] encoder : Lavc59.37.100 h264_nvenc timecode : 01:00:00:00 Stream #0:1(und): Audio: aac (LC) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 319 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help frame= 738 fps= 31 q=-1.0 size= 2008kB time=00:00:24.50 bitrate= 671.3kbits/s speed=1.02x
push rtmp ok
(base) adeMacBook-Pro-2:ufbuild liuqi$ ./ffmpeg -i rtmp://127.0.0.1/stream/live ffmpeg(82694,0x7ff85bdb6640) malloc: nano zone abandoned due to inability to reserve vm space. ffmpeg version N-112061-g654e4b00e2 Copyright (c) 2000-2023 the FFmpeg developers built with Apple clang version 14.0.3 (clang-1403.0.22.14.1) configuration: --prefix=/usr/local/ --libdir=/usr/local/lib/ --cc=clang --enable-htmlpages --enable-libx264 --enable-gpl --extra-cflags='-I/usr/local/include/GraphicsMagick/ -I/usr/local/opt/openssl@3.1/include/ -O0 -g3 -fsanitize=address -Wno-error -fPIC -I/usr/local/include -fno-omit-frame-pointer' --extra-ldflags='-O0 -g3 -fsanitize=address -fno-omit-frame-pointer -Wno-error -fPIC -L/usr/local/lib -L/usr/local/opt/openssl@3.1/lib -lpng' --enable-libfreetype --enable-fontconfig --enable-libspeex --enable-libopus --enable-libzmq --enable-libx265 --enable-libass --enable-videotoolbox --disable-optimizations --enable-audiotoolbox --enable-opengl --disable-stripping --samples=../../fate-suite/ --enable-encoder=hevc_videotoolbox --enable-hwaccel=hevc_videotoolbox --enable-hwaccel=h264_videotoolbox --enable-openssl --enable-nonfree --disable-shared --enable-libfdk_aac --enable-libxml2 --enable-libmp3lame --enable-lcms2 --enable-libwebp --enable-libvpx --enable-libaom libavutil 58. 24.100 / 58. 24.100 libavcodec 60. 26.100 / 60. 26.100 libavformat 60. 12.100 / 60. 12.100 libavdevice 60. 2.101 / 60. 2.101 libavfilter 9. 11.100 / 9. 11.100 libswscale 7. 3.100 / 7. 3.100 libswresample 4. 11.100 / 4. 11.100 libpostproc 57. 2.100 / 57. 2.100 Input #0, flv, from 'rtmp://127.0.0.1/stream/live': Metadata: |RtmpSampleAccess: true major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf60.12.100 server : SRS/6.0.66(Bee) server_version : 6.0.66 Duration: N/A, start: 0.000000, bitrate: 669 kb/s Stream #0:0: Data: none Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp, 319 kb/s Stream #0:2: Video: h264 (Constrained Baseline), yuv420p(tv, bt709, progressive), 384x216 [SAR 1:1 DAR 16:9], 349 kb/s, 30 fps, 30 tbr, 1k tbn At least one output file must be specified (base) adeMacBook-Pro-2:ufbuild liuqi$
get stream info from rtmp sever ok.
comment:6 by , 13 months ago
[2023-09-18 10:09:14.122][INFO][82422][5ht538q7] Process: cpu=0.00%,0MB, threads=2 [2023-09-18 10:09:17.907][INFO][82422][w829w4s4] Hybrid cpu=0.00%,0MB, cid=1,0, timer=62,0,0, clock=0,48,1,0,0,0,0,0,0 [2023-09-18 10:09:19.124][INFO][82422][5ht538q7] Process: cpu=0.00%,0MB, threads=2 [2023-09-18 10:09:21.239][INFO][82422][l8s714l8] RTMP client ip=127.0.0.1:55821, fd=14 [2023-09-18 10:09:21.243][INFO][82422][l8s714l8] complex handshake success [2023-09-18 10:09:21.243][INFO][82422][l8s714l8] connect app, tcUrl=rtmp://127.0.0.1:1935/stream, pageUrl=, swfUrl=, schema=rtmp, vhost=127.0.0.1, port=1935, app=stream, args=null [2023-09-18 10:09:21.244][INFO][82422][l8s714l8] protocol in.buffer=0, in.ack=0, out.ack=0, in.chunk=128, out.chunk=128 [2023-09-18 10:09:21.244][INFO][82422][l8s714l8] client identified, type=fmle-publish, vhost=127.0.0.1, app=stream, stream=live, param=, duration=0ms [2023-09-18 10:09:21.245][INFO][82422][l8s714l8] connected stream, tcUrl=rtmp://127.0.0.1:1935/stream, pageUrl=, swfUrl=, schema=rtmp, vhost=__defaultVhost__, port=1935, app=stream, stream=live, param=, args=null [2023-09-18 10:09:21.245][INFO][82422][l8s714l8] source url=/stream/live, ip=127.0.0.1, cache=1/2500, is_edge=0, source_id=/0519o9uu [2023-09-18 10:09:21.246][INFO][82422][l8s714l8] ignore disabled exec for vhost=__defaultVhost__ [2023-09-18 10:09:21.247][INFO][82422][l8s714l8] start publish mr=0/350, p1stpt=20000, pnt=5000, tcp_nodelay=0 [2023-09-18 10:09:21.247][INFO][82422][l8s714l8] got metadata, width=384, height=216, vcodec=7, acodec=10 [2023-09-18 10:09:21.250][INFO][82422][l8s714l8] 55B video sh, codec(7, profile=Baseline, level=1.3, 384x216, 0kbps, 0.0fps, 0.0s) [2023-09-18 10:09:21.250][INFO][82422][l8s714l8] 4B audio sh, codec(10, profile=LC, 2channels, 0kbps, 48000HZ), flv(16bits, 2channels, 44100HZ) [2023-09-18 10:09:21.251][INFO][82422][l8s714l8] cleanup when unpublish [2023-09-18 10:09:21.251][INFO][82422][l8s714l8] TCP: before dispose resource(RtmpConn)(0x61200005fbc0), conns=1, zombies=0, ign=0, inz=0, ind=0 [2023-09-18 10:09:21.253][ERROR][82422][l8s714l8][4] serve error code=2003(Amf0Decode)(Decode AMF0 message failed) : service cycle : rtmp: stream service : rtmp: receive thread : handle publish message : rtmp: consume message : rtmp: decode message : decode message : decode command name : String invalid marker=0 thread [82422][l8s714l8]: do_cycle() [./src/app/srs_app_rtmp_conn.cpp:263][errno=4] thread [82422][l8s714l8]: service_cycle() [./src/app/srs_app_rtmp_conn.cpp:457][errno=4] thread [82422][l8s714l8]: do_publishing() [./src/app/srs_app_rtmp_conn.cpp:1031][errno=35] thread [82422][l8s714l8]: consume() [./src/app/srs_app_recv_thread.cpp:380][errno=35] thread [82422][l8s714l8]: handle_publish_message() [./src/app/srs_app_rtmp_conn.cpp:1186][errno=35] thread [82422][l8s714l8]: process_publish_message() [./src/app/srs_app_rtmp_conn.cpp:1231][errno=35] thread [82422][l8s714l8]: decode_message() [./src/protocol/srs_protocol_rtmp_stack.cpp:384][errno=35] thread [82422][l8s714l8]: do_decode_message() [./src/protocol/srs_protocol_rtmp_stack.cpp:607][errno=35] thread [82422][l8s714l8]: srs_amf0_read_string() [./src/protocol/srs_protocol_amf0.cpp:1513][errno=35](Interrupted system call) [2023-09-18 10:09:21.255][INFO][82422][70a0507r] TCP: clear zombies=1 resources, conns=1, removing=0, unsubs=0 [2023-09-18 10:09:21.255][INFO][82422][l8s714l8] TCP: disposing #0 resource(RtmpConn)(0x61200005fbc0), conns=1, disposing=1, zombies=0
I look at the rtmp server log about the error mesage as your command
./ffmpeg -re -i ~/kuaishou/Test/low.mp4 -map 0 -c copy -f flv rtmp://127.0.0.1/stream/live
.
Maybe the problem is happened in rtmp server side, or you can try command
./ffmpeg -re -i ~/kuaishou/Test/low.mp4 -map 0:a -map 0:v -c copy -f flv rtmp://127.0.0.1/stream/live
comment:7 by , 13 months ago
Looks the problem is happened when it publish timecode stream:
Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74) Metadata: handler_name : VideoHandler timecode : 01:00:00:00
comment:8 by , 13 months ago
Hi Steven,
It's late here and I will follow up with your requested information tomorrow (if you need any from me). What I did find out earlier is when I send to the NGINX RTMP module it fails but when I sent to Wowza it does not seem to fail, as you suggest it might be on in the NGINX RTMP module.
This is the one we are using:
https://github.com/winshining/nginx-http-flv-module
When you say above that the problem happened were you able to reproduce the issue or are you not able to reproduce it to your RTMP server?
comment:9 by , 13 months ago
This is caused by timecode of FFmpeg, SRS RTMP server fixed by https://github.com/ossrs/srs/issues/3803
FFmpeg sends AMF0 data, but it doesn't begin with the AMF0 string:
Packet #72: 04 00 00 64 00 00 04 12 01 00 00 00 Packet #74: 00 01 a5 e0 (lldb) p *msg (SrsCommonMessage) $0 = { header = { timestamp_delta = 100 payload_length = 4 message_type = '\x12' 18 '\x12' stream_id = 1 timestamp = 100 perfer_cid = 4 } size = 4 payload = 0x0000602000003570 "" } (lldb) x/4xb msg->payload 0x602000003570: 0x00 0x01 0xa5 0xe0
RTMP server should be able to handle this message by simply ignoring it:
[2023-09-18 11:38:45.099][WARN][91322][148t53jz][35] Ignore FFmpeg timecode, data=[00 01 a5 e0]
Highly recommend to use SRS v5.0.179+ or v6.0.77+ to replace Nginx RTMP, or to forward stream to Nginx RTMP.
However, there is a workaround for this issue, to remove the -map 0
then it works well:
ffmpeg -stream_loop -1 -re -i low2.mp4 -c copy -f flv rtmp://127.0.0.1:1935/<stream key>
Or you can patch your NGINX RTMP server, refer to https://github.com/ossrs/srs/pull/3804
comment:10 by , 13 months ago
Hi,
Thank you for taking the time to look into this.
In my example, I gave you the most simplified command line to reproduce it and kept the -map 0 even though I did not need it as I had a single input. However, my actual command line (for our application) has N inputs with N outputs so I need to use the -map N command to properly tie the inputs to the output (so I will either need to patch RTMP NGINX or use SRS to forward).
Have a great day.
comment:11 by , 13 months ago
So this does appear to be a bug in ffmpeg, and the RMTP servers (ossrs and nginx rtmp) put in workarounds. From the developer of the nginx rtmp:
FFmpeg sends a buggy AMF0 structure when -map option is used: the AMF type is 0x00, it means it is a number and the following 8 bytes are used to represent a number, but there are only 3 bytes left.
comment:12 by , 12 months ago
Hi Steven,
I have follow up question for you and since you originally replied it appears that you have a great understanding of this (much better than I do).
I am experiencing another issue related to this and I am hoping you can shed some light on it.
Both nginx RTMP and SRS developers put in workarounds on the server to now blow up when this condition is hit:
(from the developer)
"FFmpeg sends a buggy AMF0 structure when -map option is used: the AMF type is 0x00, it means it is a number and the following 8 bytes are used to represent a number, but there are only 3 bytes left."
However, it seems like after this condition is hit ffmpeg stops sending some information that the server determines if the client is still alive.
Both nginx RTMP and SRS have a "drop idle publisher" option:
drop_idle_publisher 5s;
We set this so publishers that legitimately stop will be dropped.
However, in the case of FFMPEG is it still sending RTMP data but the server thinks it's dead and will drop it. This is what the nginx rtmp developer said:
"Debug logs in nginx's error.log showed that FFmpeg indeed didn't send any messages in 5s after malformed AMF was sent."
Do you have any insight as to why the "map" function stops sending the data that the server needs to determine that the client is still connected.
Even though the server developers put in workarounds for the bug in ffmpeg, it seems that there is an underlying issue that happens after that bug condition is met.
Thanks for your time.
-Rob
comment:13 by , 11 months ago
No, it's not ScriptData, It just DATA if you using map option, it's only timecode data. There have Data channel for Data in flv muxer.
libavformat/flvenc.c
685 case AVMEDIA_TYPE_DATA: 686 if (par->codec_id != AV_CODEC_ID_TEXT && par->codec_id != AV_CODEC_ID_NONE) 687 return unsupported_codec(s, "Data", par->codec_id); 688 flv->data_par = par; 689 break;
Hi,
I would like to attach the video file that I can reproduce this issue but the size limit is 2.5 MB and my video (lowest resolution) is 40MB. Is there any way I can attach a file bigger than 2.5MB?