[PlanetCCRMA] The JACK Story

Sun Nov 22 16:37:25 PST 2009

On Sun, 2009-11-22 at 02:58 -0500, Orcan Ogetbil wrote:
> On Sun, Nov 22, 2009 at 12:37 AM, Fernando Lopez-Lezcano wrote:
> > On Sat, 2009-11-21 at 23:20 -0500, Orcan Ogetbil wrote:
> >> On Sat, Nov 21, 2009 at 10:55 PM, Fernando Lopez-Lezcano wrote:
> >> > On Sat, 2009-11-21 at 22:02 -0500, Orcan Ogetbil wrote:
> >
> >> >> Would you like to have them parallelly installable? We can achieve
> >> >> parallel installation via "alternatives".
> >> >
> >> > I'm not completely sure if that is possible. Keep in mind that all jack
> >> > server and client _libraries_ need to be switched in and out (ie: it is
> >> > not just a matter of switching the jackd binaries.
> >> >
> >> I am confident that we can handle it. Java, as a much bigger suite
> >> with many libraries, does it. We will have to reside the actual
> >> binaries+libraries in certain directories and alternatives will do its
> >> magic with the symlinks.
> >
> > Ok, it would be worth trying. Having them both would be good. And the
> > user can choose the default.
> >
> > What is the default default? :-), the last one installed? Sorry, I'm not
> > familiar with the alternatives system.
> 
> As far as I know, "alternatives" has a priority measure that you
> specify in the %post of your package. I don't know which one should be
> the default. I use both from time to time and I can't decide. I would
> say whichever is more stable.

For some time the clear choice was jack2 as jack1 was really really
unstable (the long 0.109.2 release, pretty unusable). Today (jack1
0.118.0) that is not so clear cut as jack1 is again very usable. But I
would choose jack2 as it is the future of jack, and can use multiple
cores - that is very desirable. 

I'll have to learn about "alternatives".... do you have a url handy?

> > The priority I'm talking about is the one jackd runs at so it is
> > internal to jack. I change the default in the source code to match
> > rtirq's tweaked priorities. Jackd does not look for it anywhere, you can
> > change it from the command line or qjackctl but it would be best to have
> > the best choice be automatic for the rt kernel, of course :-)
> 
> So, what would priority 60 do with Fedora's kernel? Sorry I'm rather
> ignorant with kernel related stuff. I run your jack2 on Fedora's
> kernel quite frequently. Although I haven't done anything other than
> simple 3-4 track recording recently, I didn't have trouble with the
> current setting.

Time to explain in more detail. There are two aspects that make it
difficult:

a) rt priorities of processes in the kernel and jack
b) default settings in limits.conf (or equivalent - now we have rtkit
for that) for what Fedora users can do

== Fedora kernel:

Fedora's kernel is not patched with the rt patch. So no processes run rt
in the kernel (AFAIK), see below for what happens in the rt kernel. The
allowed maximum priority for users in the jackuser group is 20:

----
## Automatically appended by jack-audio-connection-kit
@jackuser - rtprio 20
@jackuser - memlock 4194304
----

So, in that context the default rt priority for jackd (10) works fine as
it is less than 20, that is users in the jackuser group can run jackd
with the -R option. 

== Planet CCRMA rt kernel:

In an rt patched kernel processes associated with hardware interrupts,
timers and other internal kernel stuff run in the rt scheduler ring. So
they have more priority than any other SCHED_OTHER "normal" process. The
default priorities of rt kernel processes is 50 (all hardware interrupts
as far as I can remember) and 49 for all others. Which is more than the
user limit of 20 and a lot more than the jack default of 10. 

So things work. But they are not optimized for best performance. In our
context better performance means the most reliable audio performance at
low latencies (the shorter the latency the harder it is, I routinely run
at 128x2, it is not hard to do 64x2). 

The optimal setup which gives the best performance is to have the
hardware interrupt process which services the soundcard to have higher
priority than almost _all_ other hardware interrupt processes in the
kernel. Then the soundcard driver has priority at the kernel level and
can't be interrupted by, say disk i/o or the network interface or the
mouse. If that were allowed to happen we would potentially get clicks
(xruns in alsa parlance). 

But that is not enough. The rt threads inside jackd itself and its
clients should have higher priorities than the kernel rt threads that
deal with interrupts other than the soundcard. So that the threads that
actually feed samples to the soundcard can't be interrupted by disk i/o,
etc, etc. 

So... 

 priority of soundcard interrupt > priority of rt threads in jack
 priority of rt threads in jack > priority of all other rt threads

Rt priorities can be set or queried with chrt. So it is possible to tune
everything at boot time for best performance. I currently do that with
the rtirq script, written by Rui Nuno Capela. It is a boot time service
that reorders the priorities of all interrupts to give the best
performance. What it currently does is (more or less):

- puts the rtc priority at the top (80, I think)
- puts the soundcard priorities below that, around 70
- then i8042 keyboard/mouse
- leaves all others alone at 50 and 49

(I'd have to check the scripts again to make sure but that's the idea).

So a default priority for jack of 60 puts it in the right place in the
ladder of priorities. 

The unpatched default of 10 is not enough, as all kernel priorities are
above it. In that case rtirq is still better than nothing as the
soundcard is prioritized over other hardware interrupts. 

....

I have raised this issue in the past and the result was more or less
complete indifference if I remember correctly.  

There are two options for integrating Jack into Fedora and _not_ losing
any performence:

1) adapt to Fedora's default of max rt priority for users of 20. That
would involve changing rtirq and squeezing _all_ kernel and jack
priorities within the range of 20. No pretty but doable, I guess (I
don't see a reason of why 20 is a good choice). 

2) convince the maintainer of jack in Fedora _and_ all other parties
involved in Fedora that 20 is a bit to low and perhaps 50 or 70 would be
better. I don't see much hope in that (or even 90, if you want to run an
overlord process that monitors all other rt processes like in the case
of rtkit then limiting users to _anything_ would be fine, it should not
matter whether it is 20 or 90). 

I would be _very_ opposed to anything that involves an unnecessary
compromise in performance. Then I would still have to package a jack
that overrides the one in Fedora as I do now, and what's the point of
that?

Another problem is that Planet CCRMA assumes the user _will_ run jack
with rt and thus it gives rt access to all users. That is unacceptable
in Fedora but makes it not work as an out of the box solution for Planet
CCRMA. Not a _real_ problem as I can override that with a mini-package
in Planet CCRMA. 

Sorry for the length, hopefully I have covered everything relevant. 

> > Thanks for bringing this up! I'm all for jack2 moving to Fedora proper,
> > the trick would be to figure out the jackd rt priority stuff so that it
> > can run under both kernels with no tweaking on the part of the user. No
> > answer to that yet.... :-(
> 
> Yeah, it would be cool if we can figure out a best way. By the way,
> (maybe I should have said this in the beginning) I don't maintain
> Fedora's jack and I hope the current maintainer will cooperate.

Let's hope for the best...

It is too bad that I was very slow when Fedora got started and did not
get to maintain the jack package from the beginning. 

-- Fernando