"alsa" capturing with "-c:a copy" had audio stuttering in AVI

[chconnor]

(I an non-expert -- apologies in advance if this is my confusion or a bug in my capture card's driver!)

I am capturing an audio stream and a v4l2 stream from an S/Video capture card (Hauppauge USB-Live2) like this:

ffmpeg -report -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -f v4l2 -i /dev/video2 -c:a copy -c:v rawvideo broken.avi

It generates a broken file that has stuttering audio, even though the individual audio and video streams work fine if demuxed. If I remux the file like this the result also works:

ffmpeg -i broken.avi -c:a copy -c:v copy working-remuxed.avi

If I manually specify the audio sample type and rate at capture, it works:

ffmpeg -report -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -f v4l2 -i /dev/video2 -c:a pcm_s16le -ar 48000 -c:v rawvideo working.avi

This strikes me as odd since in the broken case ffmpeg correctly identifies the incoming audio as pcm_s16le and 48kHz.

If I analyze the broken file with ashowinfo, it shows audio pts that are increasing by 4x the nb_samples. I don't know if this is wrong or not, but the "working" version has audio pts that increase by 1x the nb_samples.

Broken version (see how first nb_samples is 1024 and pts goes from 0 to 4096):

$ ffmpeg -i test.avi -vf showinfo -af ashowinfo -fps_mode passthrough -copyts -f null /dev/null 2> pts.txt
...
[Parsed_ashowinfo_0 @ 0x70654c002f80] n:0 pts:0 pts_time:0 fmt:s16 channels:2 chlayout:stereo rate:48000 nb_samples:1024 checksum:73F22CAC plane_checksums: [ 73F22CAC ]
[Parsed_ashowinfo_0 @ 0x70654c002f80] n:1 pts:4096 pts_time:0.0853333 fmt:s16 channels:2 chlayout:stereo rate:48000 nb_samples:256 checksum:929680FC plane_checksums: [ 929680FC ]
[Parsed_ashowinfo_0 @ 0x70654c002f80] n:2 pts:5120 pts_time:0.106667 fmt:s16 channels:2 chlayout:stereo rate:48000 nb_samples:1024 checksum:7A55BBD5 plane_checksums: [ 7A55BBD5 ]

I note that I get non-monotonic DTS warnings when capturing (either broken or working version).

Attached is a log for working and broken capture, as well as ashowinfo results for working and broken.

[chconnor]

Log of capture (broken version)

[chconnor]

Log of capture (working version)

[chconnor]

ashowinfo results (broken version)

[chconnor]

ashowinfo results (working version)

[MasterQuestionable]

͏    Would manually specifying only audio sample rate work? (i.e. no "-c:a pcm_s16le")
͏    Or vice-versa? (no "-ar 48000")

͏    What if you capture only the audio?

[MasterQuestionable]

͏    Background:
͏    (Advanced Linux Sound Architecture)
͏    https://ffmpeg.org/ffmpeg-devices.html#alsa
͏    ​https://www.google.com/search?hl=en&gl=ca&num=10&q=%22alsa%22+capture

[chconnor]

Capturing with only -c:a pcms16le:

ffmpeg -report -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -f v4l2 -i /dev/video2 -c:a pcm_s16le -c:v rawvideo test-pcms16le.avi

...generates a working file (with same "non-monotonic" errors).

Capturing with only -ar:

ffmpeg -report -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -f v4l2 -i /dev/video2 -ar 48000 -c:v rawvideo test-ar.avi

...generates a working file (with same "non-monotonic" errors). But the resulting audio stream is mp3 (mp3 (mp3float) (U[0][0][0] / 0x0055), 48000 Hz, stereo, fltp, 128 kb/s), and during capture there are additional errors like this:

[libmp3lame @ 0x61fc116cd800] Queue input is backward in time

Capturing only audio:

ffmpeg -report -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a copy test.wav

...generates a working .wav file (with same "non-monotonic" errors).

Let me know if you'd like logs or ashowinfo from any of those tests.

[MasterQuestionable]

͏    Output not as "test.wav" but "test.avi"?
͏    Also try some other containers? (e.g. ".mkv")

͏    Unprocessed log files typically won't be of interest.
͏    I analyzed some: spam-fest like.
͏    ​https://trac.ffmpeg.org/ticket/11159#comment:3
͏    ​https://trac.ffmpeg.org/ticket/11231#comment:6
͏    ​https://trac.ffmpeg.org/ticket/10998

[chconnor]

Ah, sorry:

ffmpeg -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a copy test-audio-only.avi

...generates a broken file (with same "non-monotonic" errors). Stream is pcm wav.

ffmpeg -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a copy test-audio-only.mkv

...generates a working file with no errors reported (!) during capture. Stream is pcm wav.

ffmpeg -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a copy test-audio-only.mov

...generates a working file (with same "non-monotonic" errors). Stream is pcm wav.

[MasterQuestionable]

͏    Try "-avoid_negative_ts 2"?
͏    ​https://trac.ffmpeg.org/wiki/Seeking#avoid_negative_ts

[chconnor]

ffmpeg -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a copy -avoid_negative_ts -1 test-ants.avi

...I did -avoid_negative_ts with -1, 0, 1, and 2 -- all were broken in the same way, and all yielded "non-monotonic" errors during capture.

[MasterQuestionable]

͏    Would `-af "asetpts=PTS-STARTPTS"` work? Out of curiosity.
͏    I guess won't with "-c copy".
͏    https://ffmpeg.org/ffmpeg-filters.html#setpts
͏    ​https://trac.ffmpeg.org/ticket/11126#comment:2

[chconnor]

Yeah I can't try the asetpts with -c:a copy. When I specify -c:a pcm_s16le, then using asetpts will generate a working file (with non-monotonic errors and rational-seeming pts in the ashowinfo), but as tested above, specifying pcm_s16le alone also generates a working file so I'm not sure if that generated any useful info?:

ffmpeg -f alsa -ac 2 -i hw:CARD=Cx231xxAudio,DEV=0 -c:a pcm_s16le -af "asetpts=PTS-STARTPTS" test-asetpts.avi

(Let me know if there is a smarter way to try asetpts without specifying -c:a pcm_s16le or -ar 48000, etc.)

Thanks for working on this.

[MasterQuestionable]

͏    Enough info gained.
͏    I believe there's some bug in the AVI muxer:
͏    That fails to handle certain input timestamps.

͏    Unfamiliar with whose exact implementation.
͏    And have no recent plan of revising it.
͏    However the info gathered there shall help others analyze and probably fix.


͏    Regardless, 2 more possibly related options:
͏    "-fps_mode", "-copyts"
͏    https://ffmpeg.org/ffmpeg.html#Advanced-options