[PlanetCCRMA] fc6/7: new rt kernel; fc6/7: updated libgig

[Fernando Lopez-Lezcano]

On Sun, 2007-07-08 at 15:54 -0700, Fernando Lopez-Lezcano wrote:
> On Sun, 2007-07-08 at 15:26 -0400, Hector Centeno wrote:
> > Ok, so just to confirm, that 2.6.21-0182.rt19.1.fc7.ccrmart works fine
> > on my desktop but on my laptop it makes Firefox and Tomboy to crash.
> > On the same laptop using 2.6.21-0182.rt17.1.fc7.ccrmart there is no
> > problem.
> 
> Maybe this is a processor related bug. What processors do you have in
> both machines?
> 
> I did not manage to hang it with gmail but I have privoxy enabled, maybe
> some flash add related stuff is being intercepted in my case and does
> not execute. 
> 
> Turns out that since rt11 the rt patch includes a new scheduler (the
> CFS, Completely Fair Scheduler, also by Ingo Molnar) and may be causing
> this problem. I posted to lkml and asked Ingo if it is possible to split
> out this added functionality... sigh...

After some traces I gathered were sent to the appropriate kernel gurus
Ingo Molnar found the problem and sent me a small patch. I just tested
it on 2.6.22.1-rt4 and it seems to have fixed the problem with flash9. 

I'll test a 2.6.21 kernel with this patch shortly. And probably release
a new kernel if that fixes the problem on that kernel as well. 

-- Fernando

-------- Forwarded Message --------
From: Ingo Molnar 
To: Fernando Lopez-Lezcano 
Cc: Gabriel C , Carsten Emde , jcaceres, Steven Rostedt , RT-Users ,
LKML , Thomas Gleixner , Rui Nuno Capela , Paul E. McKenney 
Subject: Re: v2.6.21.5-rt19 (sched_getaffinity?)
Date: Wed, 18 Jul 2007 09:18:52 +0200

* Fernando Lopez-Lezcano <nando at ccrma.Stanford.EDU> wrote:

> > does lockdep pinpoint anything?
> 
> Lots of stuff, and at the end the lock report for the problem. 
> Hopefully some of this will help... I have attached the whole bootup 
> sequence as logged in /var/log/messages.

yeah, it pinpointed the bug. It seems to be an interaction between 
RCU-preempt (Paul Cc:-ed) and sched_mc_power_savings_store(): 
detach_destroy_domains() uses synchronize_sched() which uses 
getaffinity, which takes sched_hotcpu_mutex, and 
arch_reinit_sched_domains does it too - see the lockdep report below. 
I've added a quick workaround below as well, which should keep your box 
from hanging.

	Ingo

=============================================
[ INFO: possible recursive locking detected ]
[ 2.6.22-0182.rt4.3.fc7.ccrmart #1
---------------------------------------------
sched-powersave/3251 is trying to acquire lock:
 (sched_hotcpu_mutex){--..}, at: [<c0424a37>] sched_getaffinity+0x14/0x94

but task is already holding lock:
 (sched_hotcpu_mutex){--..}, at: [<c04245a5>] arch_reinit_sched_domains+0xe/0x33

other info that might help us debug this:
1 lock held by sched-powersave/3251:
 #0:  (sched_hotcpu_mutex){--..}, at: [<c04245a5>] arch_reinit_sched_domains+0xe/0x33

stack backtrace:
 [<c040600c>] show_trace_log_lvl+0x1a/0x2f
 [<c0406ae8>] show_trace+0x12/0x14
 [<c0406b50>] dump_stack+0x16/0x18
 [<c0446f46>] __lock_acquire+0x172/0xb67
 [<c0447d03>] lock_acquire+0x56/0x6f
 [<c061d414>] _mutex_lock+0x2b/0x38
 [<c0424a37>] sched_getaffinity+0x14/0x94
 [<c0460841>] __synchronize_sched+0x11/0x5f
 [<c0423fa8>] detach_destroy_domains+0x2c/0x30
 [<c04245af>] arch_reinit_sched_domains+0x18/0x33
 [<c0424606>] sched_power_savings_store+0x3c/0x49
 [<c0424634>] sched_mc_power_savings_store+0xe/0x10
 [<c0561f11>] sysdev_class_store+0x20/0x25
 [<c04bbc6c>] sysfs_write_file+0xaf/0xd0
 [<c048183c>] vfs_write+0xaf/0x163
 [<c0481e8a>] sys_write+0x3d/0x61
 [<c040501a>] syscall_call+0x7/0xb
 =======================
thinkpad_acpi: ThinkPad ACPI Extras v0.14

--------------------->
Index: linux-rt.q/kernel/sched.c
===================================================================
--- linux-rt.q.orig/kernel/sched.c
+++ linux-rt.q/kernel/sched.c
@@ -6699,7 +6699,6 @@ static void detach_destroy_domains(const
 
 	for_each_cpu_mask(i, *cpu_map)
 		cpu_attach_domain(NULL, i);
-	synchronize_sched();
 	arch_destroy_sched_domains(cpu_map);
 }