Skip to content

Commit df55f46

Browse files
gaurav-jindalIngo Molnar
authored andcommitted
sched/idle: Optimize the generic idle loop
Currently, smp_processor_id() is used to fetch the current CPU in cpu_idle_loop(). Every time the idle thread runs, it fetches the current CPU using smp_processor_id(). Since the idle thread is per CPU, the current CPU is constant, so we can lift the load out of the loop, saving execution cycles/time in the loop. x86-64: Before patch (execution in loop): 148: 0f ae e8 lfence 14b: 65 8b 04 25 00 00 00 00 mov %gs:0x0,%eax 152: 00 153: 89 c0 mov %eax,%eax 155: 49 0f a3 04 24 bt %rax,(%r12) After patch (execution in loop): 150: 0f ae e8 lfence 153: 4d 0f a3 34 24 bt %r14,(%r12) ARM64: Before patch (execution in loop): 168: d5033d9f dsb ld 16c: b9405661 ldr w1,[x19,#84] 170: 1100fc20 add w0,w1,#0x3f 174: 6b1f003f cmp w1,wzr 178: 1a81b000 csel w0,w0,w1,lt 17c: 130c7000 asr w0,w0,#6 180: 937d7c00 sbfiz x0,x0,#3,#32 184: f8606aa0 ldr x0,[x21,x0] 188: 9ac12401 lsr x1,x0,x1 18c: 36000e61 tbz w1,#0,358 After patch (execution in loop): 1a8: d50339df dsb ld 1ac: f8776ac0 ldr x0,[x22,x23] ab0: ea18001f tst x0,x24 1b4: 54000ea0 b.eq 388 Further observance on ARM64 for 4 seconds shows that cpu_idle_loop is called 8672 times. Shifting the code will save instructions executed in loop and eventually time as well. Signed-off-by: Gaurav Jindal <[email protected]> Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Reviewed-by: Sanjeev Yadav <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Mike Galbraith <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Link: http://lkml.kernel.org/r/[email protected] Signed-off-by: Ingo Molnar <[email protected]>
1 parent 1a99ae3 commit df55f46

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

kernel/sched/idle.c

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,8 @@ static void cpuidle_idle_call(void)
201201
*/
202202
static void cpu_idle_loop(void)
203203
{
204+
int cpu = smp_processor_id();
205+
204206
while (1) {
205207
/*
206208
* If the arch has a polling bit, we maintain an invariant:
@@ -219,7 +221,7 @@ static void cpu_idle_loop(void)
219221
check_pgt_cache();
220222
rmb();
221223

222-
if (cpu_is_offline(smp_processor_id())) {
224+
if (cpu_is_offline(cpu)) {
223225
cpuhp_report_idle_dead();
224226
arch_cpu_idle_dead();
225227
}

0 commit comments

Comments
 (0)