Opened 10 years ago

Closed 10 years ago

#635 closed defect (invalid)

Incorrect Memory Layout when using avpicture_layout with PIX_FMT_NV12

Reported by: Joachim Owned by:
Priority: normal Component: avcodec
Version: unspecified Keywords: NV12
Cc: Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

when decoding (e.g. a h264 stream) with output format PIX_FMT_NV12, the AVFrame contains 3 data buffers with Luma and Cb and Cr planes.

when using avpicture_layout to copythe AVPicture to a previously allocated memory buffer, the bytes are layouted incorrectly.
NV12 is all Luma bytes first, and after that the Cb and Cr bytes interleaved (see http://msdn.microsoft.com/en-us/library/windows/desktop/dd206750(v=vs.85).aspx#nv12)

i´ve worked around this manually, but it would be great if this incorrect behavior could be fixed within libavcodec itself.

Change History (4)

in reply to:  description comment:1 by Carl Eugen Hoyos, 10 years ago

Keywords: PIX_FMT_NV12 layout removed

Replying to jkersch:

when decoding (e.g. a h264 stream) with output format PIX_FMT_NV12

I don't think this is possible with libavcodec.

The only decoder that supports NV12 is rawvideo.
A rawvideo with pix_fmt nv12 produced with FFmpeg (that can be decoded correctly with FFmpeg) plays also fine with mplayer -vo vdpau (which supports displaying NV12).

Last edited 10 years ago by Carl Eugen Hoyos (previous) (diff)

comment:2 by Joachim, 10 years ago

the Luma plane is correct, however, for Cb and Cr there are two different buffers instead of an interleaved one where the byte are merged in the form (Cb)(Cr)(Cb)(Cr).
linesize however correctly displays only 2 planes.
any chance to get this layouted directly within avpicture_layout?

i want to pack those buffers into a media sample to use avcodec as a Media Foundation Transform decoder. (which works great, except the hack to output NV12)

in reply to:  2 comment:3 by Carl Eugen Hoyos, 10 years ago

Replying to jkersch:

the Luma plane is correct, however, for Cb and Cr there are two different buffers instead of an interleaved one where the byte are merged in the form (Cb)(Cr)(Cb)(Cr).

That is because libavcodec's H.264 decoder can only output to yuv420p (if the original sample is 8bit yuv420 which I assume). It cannot output to nv12, no matter what the input is.

libswscale can convert yuv420p to nv12.

comment:4 by Carl Eugen Hoyos, 10 years ago

Resolution: invalid
Status: newclosed
Note: See TracTickets for help on using tickets.