#5610 closed defect (invalid)

FFmpeg Libraries Seek Wildly On Opening W64-Format Files Via VIO

Reported by: mocmaint Owned by:
Priority: important Component: avformat
Version: git-master Keywords: wav regression
Cc: michael Blocked By:
Blocking: Reproduced by developer: no
Analyzed by developer: no

Description

I am testing the next patchset for the MOC player which adds the ability to feed FFmpeg's libav* libraries input via the callbacks registered using avio_alloc_context(). Some of those tests involve a variety of formats and codecs.

When attempting to play a W64-format file, the application becomes unresponsive and GDB shows seeking patterns well outside the bounds of the files through repeated calls to the seek and read callbacks.

As this is a library problem, there is no command line invocation of an FFmpeg utility, and those utilities do not appear to use callbacks via avio_alloc_context().

There is no console output (obviously) and no messages are issued by the libraries (via the logging callback).

The sequence of library calls (omitting error checking) is:

AVFormatContext *ic = avformat_alloc_context();
ic->pb = avio_alloc_context(...);
avformat_open_input(ic, ...);

avformat_open_input() never returns.

The problem occurs on a W64-format file of any size, but a suitable test file can be generated with:

sox -b16 -c2 -r48000 -n -L test.wav synth 10 sine 440 vol 0.5

The problem was introduced in commit 14d50c19 in FFmpeg 1.1 and still exists in FFmpeg 3.0.2.

It is possible that the error return values of the seek and read callbacks are incorrect, but they appear undocumented and do work in all other formats tested and also with the W64 format prior to commit 14d50c19 and with LibAV 11.6. The return values used by similar code found vary.

Full FFmpeg configure originally used is:

CFLAGS="-O2 -fPIC" ./configure --prefix=/usr/local/stow/$(basename $PWD) --libdir='${prefix}/lib64' --shlibdir='${prefix}/lib64' --enable-shared --disable-static --disable-yasm --enable-libspeex --enable-libtheora --enable-libopus

'$PWD' was '/tmp/ffmpeg-14d50c19'.

MOC is available using:

svn co -r2872 svn://svn.daper.net/moc/trunk

Aggregated patch for the patchset being tested is available using:

wget ftp://ftp.daper.net/pub/soft/moc/unstable/moc-r2872+ffmpeg_memleak-1.patch.gz

Apply to the SVN trunk revision 2872 using:

zcat moc-r2872+ffmpeg_memleak-1.patch.gz | patch -p1

The MOC FFmpeg decoder will be found in:

decoder_plugins/ffmpeg

Build as per instructions in README. After installing, delete all installed decoder plugins in 'lib{,64}/moc/decoder_plugins' *except* 'libffmpeg_decoder.*'.

Then launch MOC as 'mocp' in the directory containing W64 files.

Change History (15)

comment:1 follow-up: Changed 18 months ago by cehoyos

Is this issue reproducible with current FFmpeg git head?

comment:2 in reply to: ↑ 1 Changed 18 months ago by mocmaint

Replying to cehoyos:

Is this issue reproducible with current FFmpeg git head?

Yes - e8a236ad.

comment:3 follow-up: Changed 18 months ago by michael

Which *seek calls in the ffmpeg source seek wildly?

comment:4 Changed 18 months ago by michael

  • Cc michael added

comment:5 in reply to: ↑ 3 Changed 18 months ago by mocmaint

If you use GDB to set a breakpoint on ffmpeg_io_seek_cb() in MOC's FFmpeg decoder you will be able to backtrace into the FFmpeg libraries to identify the seeks when 'offset' goes into the petabyte range.

comment:6 Changed 18 months ago by cehoyos

  • Component changed from undetermined to avformat
  • Keywords wav added; w64 removed
  • Priority changed from normal to important
  • Version changed from unspecified to git-master

comment:7 follow-up: Changed 18 months ago by michael

i get "FATAL_ERROR: Can't send() int to the server!"

without the callback being called
no doubt thats a mistake on my side, but i dont have the time to figure out atm what i did wrong

comment:8 in reply to: ↑ 7 ; follow-up: Changed 18 months ago by mocmaint

Replying to michael:

"FATAL_ERROR: Can't send() int to the server!"

That indicates that the server side of MOC has failed for some reason. The reason may appear in your system's log files, or you can run the server separately from the client using:

mocp -D -SF .

in one VT or xterm to start the server in non-daemon mode and then just:

mocp

in a second VT or xterm to start the client.

When that message appears you can check the server's output for more information.

Note that you will have to have configured MOC without '--disable-debug' (and '--enable-debug=gdb' is best when using GDB) for this to work.

comment:9 in reply to: ↑ 8 Changed 18 months ago by mocmaint

I have created a test rig for you which is available at:

ftp://ftp.daper.net/pub/soft/moc/unstable/seek-test.c.gz

This test rig emulates the essential behaviour of MOC and triggers the problem on *.w64 files (but not on others).

Please read the comments at the start of the file on compiling and running, and especially on the definition of ATEOF.

In creating this test rig, I uncovered a potential bug in MOC which causes it to return a non-zero value at EOF, but this does not trigger the problem on non-W64 files although forcing the return value to zero causes FFmpeg to handle W64 files correctly (or at least circumvents the seeking problem).

comment:10 in reply to: ↑ description Changed 18 months ago by mocmaint

I have now circumvented this problem within MOC, although it and the documentation issue still exist within FFmpeg. However, for my purposes you could now lower the priority of this problem if you wish.

comment:11 Changed 18 months ago by michael

I dont see any huge seek offsets:

sox -b16 -c2 -r48000 -n -L test.wav synth 10 sine 440 vol 0.5
md5sum test.wav
23429ea98c131c67ba8b160fa5fea002 test.wav
./seek-test test.wav
Calling ffmpeg_open():

Calling avformat_alloc_context()
Calling avio_alloc_context()
Calling avformat_open_input()

read_cb: 2048

read(): 2048

seek_cb called: 0, 65536

file_size(): 1920044

seek_cb called: 0, 65536

file_size(): 1920044

seek_cb called: 0, 65536

file_size(): 1920044

seek_cb called: 0, 65536

file_size(): 1920044

seek_cb called: 0, 65536

file_size(): 1920044

read_cb: 63532

read(): 63532

seek_cb called: 44, 0

lseek(): 44

Calling ffmpeg_close():

Calling avformat_close_input()

comment:12 Changed 18 months ago by michael

I see this:
==17514== Conditional jump or move depends on uninitialised value(s)
==17514== at 0x478278: main (seek-test.c:134)
but thats not related

comment:13 Changed 17 months ago by mocmaint

I have now commited the latest MOC patchset which includes a mitigation to this problem which limits FFmpeg to a single out-of-bounds seek (which MOC can tolerate). I will leave it in your hands to pursue it or close it as you wish.

Last edited 17 months ago by mocmaint (previous) (diff)

comment:14 Changed 14 months ago by richardpl

How is this related to w64 if you are creating wav files with sox?

comment:15 Changed 14 months ago by richardpl

  • Resolution set to invalid
  • Status changed from new to closed

w64 demuxer parses complete file if file is seekable to find metadata.
Looks like your program implementation of avio callbacks is fragile.
It should not keep seeking or reading if eof is reached.

Note: See TracTickets for help on using tickets.