PulseAudio input performance regression

[GreatEmerald]

Summary of the bug:

There has been a regression that severely reduces the performance of recording input PulseAudio streams using the pulse input device. Before the regression, I could record one audio and one video stream at a constant 30 frames per second (capped), but after the regression the recording keeps dropping frames, resulting in an unacceptable 25 FPS on average recording performance. It doesn't matter what video or audio codecs are in use, the result is always the same.

In order to identify where the problem lies, I used git bisect to see what was the regressing commit. Unfortunately, the last bisected commits didn't build properly (configure doesn't give me all of the options that I use for building a complete FFmpeg, and even with some of the options disabled, it only builds the libraries, and not the binary, which doesn't allow me to test whether the regression is still there or not). Still, bisecting allowed me to narrow down the list to these commits:

9dced8542618c12e0496e7db3067a61ba9080a45
e77c86629fe803bc07ccfe323151486fe977b133
6c9eac19988fb85d9bf911e20e9675d814ab10dd
b7327887ea260d26e4fe98d34149cd57168f2ba3
7af99a01c4e77efa002a672d72f142257e87bf38
836ce90566d5c14f2f45303b9cc46b321ddca61f
2e215267901dfbe90d2c041d7e5944aa32b3e071
560f7774a4edde550e35f85e4b1c6cc74c85f48f
3b266da3d35f3f7a61258b78384dfe920d875d29
ea9367e9212076d30deee6d28907d3d61ef46f7e
4778783160fb8d7aa39dafdabc756f1691b830f7

More precisely, commit 4778783160fb8d7aa39dafdabc756f1691b830f7 is the first confirmed bad commit, while commit dcd207c4cbcd9dd72b73e5b544c055b29885be64 was the last known good commit.

There is a non-ideal workaround for this regression, which is to switch to ALSA input rather than use the native PulseAudio one. However, that creates popping sounds in the recording.

I'm using openSUSE 12.1 x86_64, currently with PulseAudio 2.0, although before that I used PA 1.1. There is no change in performance between the two versions.

From the hardware side, I have two sound cards - a dedicated Creative SoundBlaster X-Fi XtremeGamer card, and an integrated Intel Azalia chip. Using either soundcard, recording an output monitor (system sounds) doesn't cause this problem, but recording an actual input device, such as a microphone, does.

How to reproduce:

To test this, I run FFmpeg like this:

/usr/bin/ffmpeg -xerror \
 -f pulse -i alsa_input.pci-0000_00_14.2.analog-stereo \
 -f x11grab -i :0.0 \
 -codec:a pcm_s16le -threads 0 -y Video.mkv

The complete output of this command, using the last confirmed bad version, is as follows:

ffmpeg version N-39892-g4778783 Copyright (c) 2000-2012 the FFmpeg developers
  built on Jun 22 2012 18:44:00 with gcc 4.6.2
  configuration: --shlibdir=/usr/lib64 --prefix=/usr --mandir=/usr/share/man --libdir=/usr/lib64 --enable-shared --disable-static --enable-debug --disable-stripping --extra-cflags='-fmessage-length=0 -O2 -march=native -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -fPIC -I/usr/include/gsm' --enable-gpl --enable-x11grab --enable-version3 --enable-pthreads --enable-avfilter --enable-libpulse
  libavutil      51. 46.100 / 51. 46.100
  libavcodec     54. 14.101 / 54. 14.101
  libavformat    54.  3.100 / 54.  3.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 70.100 /  2. 70.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 11.100 /  0. 11.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, pulse, from 'alsa_input.pci-0000_00_14.2.analog-stereo':
  Duration: N/A, start: 0.634532, bitrate: N/A
    Stream #0:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
[x11grab @ 0x63a7c0] device: :0.0 -> display: :0.0 x: 0 y: 0 width: 640 height: 480
[x11grab @ 0x63a7c0] shared memory extension found
Input #1, x11grab, from ':0.0':
  Duration: N/A, start: 1340381175.492614, bitrate: N/A
    Stream #1:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 640x480, 294617 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
Incompatible pixel format 'bgr0' for codec 'mpeg4', auto-selecting format 'yuv420p'
[buffer @ 0x639680] w:640 h:480 pixfmt:bgr0 tb:1/1000000 sar:0/1 sws_param:flags=2
[buffersink @ 0x641620] auto-inserting filter 'auto-inserted scale 0' between the filter 'src' and the filter 'out'
[scale @ 0x641ec0] w:640 h:480 fmt:bgr0 sar:0/1 -> w:640 h:480 fmt:yuv420p sar:0/1 flags:0x4
Guessed Channel Layout for  Input Stream #0.0 : stereo
Output #0, matroska, to 'Video.mkv':
  Metadata:
    encoder         : Lavf54.3.100
    Stream #0:0: Video: mpeg4, yuv420p, 640x480, q=2-31, 200 kb/s, 1k tbn, 29.97 tbc
    Stream #0:1: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #1:0 -> #0:0 (rawvideo -> mpeg4)
  Stream #0:0 -> #0:1 (pcm_s16le -> pcm_s16le)
Press [q] to stop, [?] for help
frame=  264 fps= 24 q=25.8 Lsize=    2764kB time=00:00:10.98 bitrate=2061.9kbits/s dup=0 drop=27    
video:687kB audio:2059kB global headers:0kB muxing overhead 0.625024%

[GreatEmerald]

I just upgraded to openSUSE 12.2 RC2 and tested this further with FFmpeg 0.11.1. From the testing, it seems that this issue affects only one sound source, effectively the microphone input of the integrated Intel HD Audio chip. I tested this with two motherboards (both had the Intel HD Audio chip, but they were from different manufacturers), so it's not just a problem on my end, even though the scope of the problem seems to be a lot smaller that I initially thought.

[Carl Eugen Hoyos]

Please test / also add output for current git head.

[GreatEmerald]

All right, did some more testing. The latest git revision does indeed fix the issue on my machine. Since I wanted to know what fixed it, I did a "reverse bisect" of sorts, and I was pointed to the revision 3630a075132a8d059b16f92a291e523c1bc7ebc9 as the first one to compile and run correctly. Looking at its contents, it would appear that the particular commit that did the trick was 5db5169e46a5f1676aafb82ec8c3f5dc6fb6bb6d (avconv: multithreaded demuxing.) Though I'd imagine that it would be more of a workaround, as it could still be a problem on single-core machines or so. I think I'll run another test on another, slower machine with the same audio chip and see if it's also fixed there.

[GreatEmerald]

And now tested on another machine - it confirmed the bug fixed by the same commit.

[Carl Eugen Hoyos]

Thank you for testing again!