#8958 closed defect (invalid)
ffmpeg git: yuv4mpeg pipe seems to be broken
Reported by: | BlueSwordM | Owned by: | |
---|---|---|---|
Priority: | minor | Component: | undetermined |
Version: | git-master | Keywords: | |
Cc: | Blocked By: | ||
Blocking: | Reproduced by developer: | no | |
Analyzed by developer: | no |
Description
Summary of the bug:
Piping from yuv4mpegpipe to another external encoder seems to be broken, at least on Ubuntu and Arch. Static and shared builds seem to encounter the same issue. The problem started rising with builds from October 29th to October 30th. Problem is still present using the git-master.
My snapshot from October 20th works fine.
The issue arises no matter the container(Matroska or MP4), the source codec(AVC/HEVC/VP9/MPEG2), and the encoder(rav1e/x264/aomenc/libvpx, all from their git-masters).
How to reproduce:
- Compile ffmpeg normally on Ubuntu 20.04/20.10.
(./configure --enable-gpl --enable-version3 --enable-nonfree)
- Run the command below
% ffmpeg -v 9 -loglevel 99 -i input.mkv -r 30 -s 1920x1080 -f yuv4mpegpipe -pix_fmt yuv420p - | aomenc --cpu-used=6 -o output.ivf -
- I get this error, and the piping stops.
% av_interleaved_write_frame(): Broken pipe Error writing trailer of pipe:: Broken pipe
Full log here:
% Splitting the commandline. Reading option '-v' ... matched as option 'v' (set logging level) with argument '9'. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument '99'. Reading option '-i' ... matched as input url with argument '/home/bluezakm/Downloads/1080p_30s.mkv'. Reading option '-f' ... matched as option 'f' (force format) with argument 'yuv4mpegpipe'. Reading option '-' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option v (set logging level) with argument 9. Successfully parsed a group of options. Parsing a group of options: input url /home/bluezakm/Downloads/1080p_30s.mkv. Successfully parsed a group of options. Opening an input file: /home/bluezakm/Downloads/1080p_30s.mkv. [NULL @ 0x56220b1ff800] Opening '/home/bluezakm/Downloads/1080p_30s.mkv' for reading [file @ 0x56220b200340] Setting default whitelist 'file,crypto,data' Probing matroska,webm score:100 size:2048 [matroska,webm @ 0x56220b1ff800] Format matroska,webm probed with size=2048 and score=100 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x22B59D at pos. 4375 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225410794 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225410840 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225410879 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225410923 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225411000 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225411067 [matroska,webm @ 0x56220b1ff800] Unknown entry 0x447B at pos. 225411148 st:0 removing common factor 1000000 from timebase [matroska,webm @ 0x56220b1ff800] Before avformat_find_stream_info() pos: 5484 bytes read:33189 seeks:2 nb_streams:1 [h264 @ 0x56220b203840] nal_unit_type: 7(SPS), nal_ref_idc: 3 [h264 @ 0x56220b203840] nal_unit_type: 8(PPS), nal_ref_idc: 3 [h264 @ 0x56220b203840] nal_unit_type: 7(SPS), nal_ref_idc: 3 [h264 @ 0x56220b203840] nal_unit_type: 8(PPS), nal_ref_idc: 3 [h264 @ 0x56220b203840] nal_unit_type: 5(IDR), nal_ref_idc: 3 [h264 @ 0x56220b203840] Format yuv420p chosen by get_format(). [h264 @ 0x56220b203840] Reinit context to 1920x1088, pix_fmt: yuv420p [h264 @ 0x56220b203840] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 [h264 @ 0x56220b203840] Increasing reorder buffer to 1 [h264 @ 0x56220b203840] no picture [h264 @ 0x56220b203840] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 Last message repeated 1 times [h264 @ 0x56220b203840] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 [h264 @ 0x56220b203840] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 Last message repeated 1 times [h264 @ 0x56220b203840] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 [matroska,webm @ 0x56220b1ff800] All info found [matroska,webm @ 0x56220b1ff800] stream 0: start_time: 0 duration: NOPTS [matroska,webm @ 0x56220b1ff800] format: start_time: 0 duration: 30.03 (estimate from stream) bitrate=60049 kb/s [matroska,webm @ 0x56220b1ff800] After avformat_find_stream_info() pos: 2026108 bytes read:2053813 seeks:2 frames:9 Input #0, matroska,webm, from '/home/bluezakm/Downloads/1080p_30s.mkv': Metadata: encoder : libebml v1.4.0 + libmatroska v1.6.2 creation_time : 2020-10-31T02:09:11.000000Z Duration: 00:00:30.03, start: 0.000000, bitrate: 60049 kb/s Stream #0:0(eng), 9, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(tv, bt709, progressive, left), 1920x1080 (1920x1088), 0/1, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default) Metadata: BPS-eng : 60045841 DURATION-eng : 00:00:30.030000000 NUMBER_OF_FRAMES-eng: 720 NUMBER_OF_BYTES-eng: 225397078 _STATISTICS_WRITING_APP-eng: mkvmerge v51.0.0 ('I Wish') 64-bit _STATISTICS_WRITING_DATE_UTC-eng: 2020-10-31 02:09:11 _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Successfully opened the file. Parsing a group of options: output url -. Applying option f (force format) with argument yuv4mpegpipe. Successfully parsed a group of options. Opening an output file: -. [pipe @ 0x56220b204840] Setting default whitelist 'crypto,data' Successfully opened the file. detected 16 logical cores [h264 @ 0x56220b7510c0] nal_unit_type: 7(SPS), nal_ref_idc: 3 [h264 @ 0x56220b7510c0] nal_unit_type: 8(PPS), nal_ref_idc: 3 Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native)) Press [q] to stop, [?] for help cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) Last message repeated 1 times [h264 @ 0x56220b7510c0] nal_unit_type: 5(IDR), nal_ref_idc: 3 [h264 @ 0x56220b7510c0] Format yuv420p chosen by get_format(). [h264 @ 0x56220b7510c0] Reinit context to 1920x1088, pix_fmt: yuv420p [h264 @ 0x56220b7510c0] no picture cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b2b3200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b2fc200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b302d40] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b262c00] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b27f800] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220ba29600] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220ba46340] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220ba630c0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220ba7fe40] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220ba9cbc0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220bab9940] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220bad66c0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220baf3440] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220bb101c0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 [h264 @ 0x56220bb2cf40] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 0 cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) [h264 @ 0x56220b7510c0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2 [graph 0 input from stream 0:0 @ 0x56220c27c840] Setting 'video_size' to value '1920x1080' [graph 0 input from stream 0:0 @ 0x56220c27c840] Setting 'pix_fmt' to value '0' [graph 0 input from stream 0:0 @ 0x56220c27c840] Setting 'time_base' to value '1/1000' [graph 0 input from stream 0:0 @ 0x56220c27c840] Setting 'pixel_aspect' to value '1/1' [graph 0 input from stream 0:0 @ 0x56220c27c840] Setting 'frame_rate' to value '24000/1001' [graph 0 input from stream 0:0 @ 0x56220c27c840] w:1920 h:1080 pixfmt:yuv420p tb:1/1000 fr:24000/1001 sar:1/1 [AVFilterGraph @ 0x56220c27ac40] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed Output #0, yuv4mpegpipe, to 'pipe:': Metadata: encoder : Lavf58.64.100 Stream #0:0(eng), 0, 1001/24000: Video: wrapped_avframe, 1 reference frame, yuv420p(tv, bt709, progressive, left), 1920x1080 [SAR 1:1 DAR 16:9], 0/1, q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc (default) Metadata: BPS-eng : 60045841 DURATION-eng : 00:00:30.030000000 NUMBER_OF_FRAMES-eng: 720 NUMBER_OF_BYTES-eng: 225397078 _STATISTICS_WRITING_APP-eng: mkvmerge v51.0.0 ('I Wish') 64-bit _STATISTICS_WRITING_DATE_UTC-eng: 2020-10-31 02:09:11 _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES encoder : Lavc58.112.100 wrapped_avframe Clipping frame in rate conversion by 0.000008 Error parsing header; not a YUV2MPEG2 file? Fatal: Unsupported Y4M stream. av_interleaved_write_frame(): Broken pipe No more output streams to write to, finishing. Error writing trailer of pipe:: Broken pipe frame= 1 fps=0.0 q=-0.0 Lsize= 32kB time=00:00:00.04 bitrate=6302.1kbits/s speed=0.853x video:1kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6029.850586% Input file #0 (/home/bluezakm/Downloads/1080p_30s.mkv): Input stream #0:0 (video): 17 packets read (3133913 bytes); 2 frames decoded; Total: 17 packets (3133913 bytes) demuxed Output file #0 (pipe:): Output stream #0:0 (video): 1 frames encoded; 1 packets muxed (536 bytes); Total: 1 packets (536 bytes) muxed 2 frames successfully decoded, 0 decoding errors [AVIOContext @ 0x56220b207a40] Statistics: 0 seeks, 96 writeouts [AVIOContext @ 0x56220b208740] Statistics: 3167254 bytes read, 2 seeks Conversion failed!
Change History (8)
comment:1 by , 4 years ago
comment:2 by , 4 years ago
commit 7369595c556516b90bf3acdf85043dc11add7d89 added a way to pass through color_range property from decoded frame to encoder.
This adds XCOLORRANGE=LIMITED in the header of the output file for me.. which is not handled by aomenc properly. Guess that the issue is on their side.
You could get around this for now by setting -color_range 0 on the command line.
comment:3 by , 4 years ago
Keywords: | regression added |
---|---|
Priority: | normal → important |
comment:4 by , 4 years ago
Keywords: | regression removed |
---|---|
Priority: | important → minor |
Resolution: | → invalid |
Status: | new → closed |
Not our bug.
comment:5 by , 4 years ago
For context, quoting the discussion I had with taliho on IRC last night.
02:11 < taliho4> JEEB: maybe interesting for you https://trac.ffmpeg.org/ticket/8958 02:13 <@JEEB> taliho4: yup, I plugged the pipes together, and the AVFrame received from the filter chain contained that 02:13 <@JEEB> if the AVFrame didn't have that info set, it would not be set in the muxer, either 02:15 <@JEEB> I'd say the current behavior of passing the AVFrame's flags by default makes sense. if that enables features in muxers etc which break other implementations it's a consideration whether that should be disabled by default (put under -strict experimental or so), or if the other implementations should be helped 02:15 <@JEEB> and yea looking at that log it specifically sets the range in the h.264 stream 02:15 <@JEEB> > Stream #0:0(eng), 9, 1/1000: Video: h264 (High), 1 reference frame, yuv420p(tv, bt709, progressive, left) 02:22 < taliho4> yeah makes sense
Long story short, the information is only specified if it is exactly specified in the AVFrame. And if passing that information - as would be expected - enables features in muxers etc that shouldn't be enabled by default, that is a separate problem, which was just not noticed before as often (as it would require setting the flag manually - or in some cases just utilizing the J pixel formats).
For the record, I have no idea which features of y4m are "standard" and which are not, but as far as I can tell we're writing these values out without any checks. The only points where there are checks are with regards to certain pixel formats :) .
So either:
- The flags are valid and considered standard enough - and aomenc should fix their Y4M parser (and in the mean time users unfortunately would have to utilize -color_range unknown in cases where their input actually defines that it is either full range or not). In this case the ticket should be closed like it is currently.
- The flags are not considered standard enough and we move them under some flag. Not exactly perfect either since then we by default lose the capability of saying that something is full range or limited range explicitly.
Note1:
This was most likely already causing issues with aomenc with YUVJ pixel formats previously, since those cause the range to always be written looking at the muxer code.
Note2:
https://linux.die.net/man/5/yuv4mpeg (which I think is the origin of y4m) notes that
X[string] - 'metadata' (optional; unparsed, but passed around)
Thus, I am heavily leaning towards option 1, which would be no change on our output by default as clearly aomenc is not handling the format well enough.
comment:6 by , 4 years ago
Yeah, I'll make sure to relay this issue to the aom developers and other AV1 encoder projects devs.
Thanks a lot for the insight, and a temporary way to fix this issue when piping. :D
comment:7 by , 4 years ago
Resolution: | invalid → fixed |
---|
So, the issue was fixed entirely on SVT-AV1's and aom's part.
The bug report can be closed.
comment:8 by , 4 years ago
Resolution: | fixed → invalid |
---|
Here's the entire log output. Clipped some stuff by accident.