From patchwork Thu May 15 04:11:30 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Chaugule X-Patchwork-Id: 30215 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f197.google.com (mail-ve0-f197.google.com [209.85.128.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 954AA20446 for ; Thu, 15 May 2014 04:11:49 +0000 (UTC) Received: by mail-ve0-f197.google.com with SMTP id db11sf2130593veb.8 for ; Wed, 14 May 2014 21:11:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=IU/Ya91BvJ3TMKxEDpowFD4YqrL0+2Q+MK8jVPm+TrE=; b=XjzrJcYe1VAU4ld4PNfQIi8UFuOXIgYhdlsmvptPJuiejyfcMZB0bZQyiokLPzgUPu tdAx0RPYSg2xXwm8+eVPu/MueCESiH233Ags9+GYyIPKWI7TOknha8T4QIPnlYKCaA3z 4K09cTT+g6Ukm36bz7akgK+6POhkLm/55n8Hu/wFcWP+PLmtPFsevofKjLmTe1i2bomg gOTm1nxmpsU8p08vmvpirHbPEyyXn/c6XAtYJGLw/AYTmag9RmscQVsaZKZ+6t25LHcu RDgxT9TPSPEOgUNs1SYQnilol310LQ2TyLYhGFQtHSjR2o6I/mGL5kGBwcU6Fupgy9fk Gr1Q== X-Gm-Message-State: ALoCoQnt+ptoASfADfnyEwiVRORs+efs1KlIKgLflyOB1fORyERvNMI/BDcqzlJXDOnc3qAyg2Le X-Received: by 10.236.149.193 with SMTP id x41mr3421286yhj.24.1400127109249; Wed, 14 May 2014 21:11:49 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.84.239 with SMTP id l102ls68972qgd.81.gmail; Wed, 14 May 2014 21:11:49 -0700 (PDT) X-Received: by 10.53.13.133 with SMTP id ey5mr3780713vdd.8.1400127109101; Wed, 14 May 2014 21:11:49 -0700 (PDT) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx.google.com with ESMTPS id wd2si697377veb.167.2014.05.14.21.11.49 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 14 May 2014 21:11:49 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) client-ip=209.85.220.181; Received: by mail-vc0-f181.google.com with SMTP id ld13so3640231vcb.26 for ; Wed, 14 May 2014 21:11:49 -0700 (PDT) X-Received: by 10.220.184.72 with SMTP id cj8mr6567936vcb.10.1400127109010; Wed, 14 May 2014 21:11:49 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp291015vcb; Wed, 14 May 2014 21:11:48 -0700 (PDT) X-Received: by 10.224.161.10 with SMTP id p10mr9667293qax.12.1400127108152; Wed, 14 May 2014 21:11:48 -0700 (PDT) Received: from mail-qg0-f46.google.com (mail-qg0-f46.google.com [209.85.192.46]) by mx.google.com with ESMTPS id o1si1946465qai.8.2014.05.14.21.11.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 14 May 2014 21:11:48 -0700 (PDT) Received-SPF: pass (google.com: domain of ashwin.chaugule@linaro.org designates 209.85.192.46 as permitted sender) client-ip=209.85.192.46; Received: by mail-qg0-f46.google.com with SMTP id q108so841828qgd.33 for ; Wed, 14 May 2014 21:11:48 -0700 (PDT) X-Received: by 10.140.44.72 with SMTP id f66mr11164202qga.78.1400127108003; Wed, 14 May 2014 21:11:48 -0700 (PDT) Received: from localhost.localdomain (cpe-098-027-049-158.nc.res.rr.com. [98.27.49.158]) by mx.google.com with ESMTPSA id k9sm6107147qaj.25.2014.05.14.21.11.46 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 14 May 2014 21:11:47 -0700 (PDT) From: Ashwin Chaugule To: Catalin.Marinas@arm.com Cc: linaro-acpi@lists.linaro.org, linux-arm-kernel@lists.infradead.org, rob.herring@linaro.org, christoffer.dall@linaro.org, Marc.Zyngier@arm.com, mark.rutland@arm.com, anup.patel@linaro.org, patches@linaro.org, Ashwin Chaugule Subject: [PATCH v10 3/3] ARM: Check if a CPU has gone offline Date: Thu, 15 May 2014 00:11:30 -0400 Message-Id: <1400127090-1320-4-git-send-email-ashwin.chaugule@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1400127090-1320-1-git-send-email-ashwin.chaugule@linaro.org> References: <1400127090-1320-1-git-send-email-ashwin.chaugule@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ashwin.chaugule@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , PSCIv0.2 adds a new function called AFFINITY_INFO, which can be used to query if a specified CPU has actually gone offline. Calling this function via cpu_kill ensures that a CPU has quiesced after a call to cpu_die. This helps prevent the CPU from doing arbitrary bad things when data or instructions are clobbered (as happens with kexec) in the window between a CPU announcing that it is dead and said CPU leaving the kernel. Signed-off-by: Ashwin Chaugule Signed-off-by: Mark Rutland Reviewed-by: Rob Herring Acked-by: Catalin Marinas --- arch/arm/kernel/psci_smp.c | 33 +++++++++++++++++++++++++++++++++ arch/arm64/include/asm/cpu_ops.h | 2 ++ arch/arm64/kernel/psci.c | 31 +++++++++++++++++++++++++++++++ arch/arm64/kernel/smp.c | 22 ++++++++++++++++++++++ 4 files changed, 88 insertions(+) diff --git a/arch/arm/kernel/psci_smp.c b/arch/arm/kernel/psci_smp.c index 570a48c..28a1db4 100644 --- a/arch/arm/kernel/psci_smp.c +++ b/arch/arm/kernel/psci_smp.c @@ -16,6 +16,8 @@ #include #include #include +#include +#include #include #include @@ -66,6 +68,36 @@ void __ref psci_cpu_die(unsigned int cpu) /* We should never return */ panic("psci: cpu %d failed to shutdown\n", cpu); } + +int __ref psci_cpu_kill(unsigned int cpu) +{ + int err, i; + + if (!psci_ops.affinity_info) + return 1; + /* + * cpu_kill could race with cpu_die and we can + * potentially end up declaring this cpu undead + * while it is dying. So, try again a few times. + */ + + for (i = 0; i < 10; i++) { + err = psci_ops.affinity_info(cpu_logical_map(cpu), 0); + if (err == PSCI_0_2_AFFINITY_LEVEL_OFF) { + pr_info("CPU%d killed.\n", cpu); + return 1; + } + + msleep(10); + pr_info("Retrying again to check for CPU kill\n"); + } + + pr_warn("CPU%d may not have shut down cleanly (AFFINITY_INFO reports %d)\n", + cpu, err); + /* Make platform_cpu_kill() fail. */ + return 0; +} + #endif bool __init psci_smp_available(void) @@ -78,5 +110,6 @@ struct smp_operations __initdata psci_smp_ops = { .smp_boot_secondary = psci_boot_secondary, #ifdef CONFIG_HOTPLUG_CPU .cpu_die = psci_cpu_die, + .cpu_kill = psci_cpu_kill, #endif }; diff --git a/arch/arm64/include/asm/cpu_ops.h b/arch/arm64/include/asm/cpu_ops.h index 1524130..d7b4b38 100644 --- a/arch/arm64/include/asm/cpu_ops.h +++ b/arch/arm64/include/asm/cpu_ops.h @@ -39,6 +39,7 @@ struct device_node; * from the cpu to be killed. * @cpu_die: Makes a cpu leave the kernel. Must not fail. Called from the * cpu being killed. + * @cpu_kill: Ensures a cpu has left the kernel. Called from another cpu. * @cpu_suspend: Suspends a cpu and saves the required context. May fail owing * to wrong parameters or error conditions. Called from the * CPU being suspended. Must be called with IRQs disabled. @@ -52,6 +53,7 @@ struct cpu_operations { #ifdef CONFIG_HOTPLUG_CPU int (*cpu_disable)(unsigned int cpu); void (*cpu_die)(unsigned int cpu); + int (*cpu_kill)(unsigned int cpu); #endif #ifdef CONFIG_ARM64_CPU_SUSPEND int (*cpu_suspend)(unsigned long); diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c index 90df6e6..9e9798f 100644 --- a/arch/arm64/kernel/psci.c +++ b/arch/arm64/kernel/psci.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -403,6 +404,35 @@ static void cpu_psci_cpu_die(unsigned int cpu) pr_crit("unable to power off CPU%u (%d)\n", cpu, ret); } + +static int cpu_psci_cpu_kill(unsigned int cpu) +{ + int err, i; + + if (!psci_ops.affinity_info) + return 1; + /* + * cpu_kill could race with cpu_die and we can + * potentially end up declaring this cpu undead + * while it is dying. So, try again a few times. + */ + + for (i = 0; i < 10; i++) { + err = psci_ops.affinity_info(cpu_logical_map(cpu), 0); + if (err == PSCI_0_2_AFFINITY_LEVEL_OFF) { + pr_info("CPU%d killed.\n", cpu); + return 1; + } + + msleep(10); + pr_info("Retrying again to check for CPU kill\n"); + } + + pr_warn("CPU%d may not have shut down cleanly (AFFINITY_INFO reports %d)\n", + cpu, err); + /* Make op_cpu_kill() fail. */ + return 0; +} #endif const struct cpu_operations cpu_psci_ops = { @@ -413,6 +443,7 @@ const struct cpu_operations cpu_psci_ops = { #ifdef CONFIG_HOTPLUG_CPU .cpu_disable = cpu_psci_cpu_disable, .cpu_die = cpu_psci_cpu_die, + .cpu_kill = cpu_psci_cpu_kill, #endif }; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index f0a141d..c3cb160 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -228,6 +228,19 @@ int __cpu_disable(void) return 0; } +static int op_cpu_kill(unsigned int cpu) +{ + /* + * If we have no means of synchronising with the dying CPU, then assume + * that it is really dead. We can only wait for an arbitrary length of + * time and hope that it's dead, so let's skip the wait and just hope. + */ + if (!cpu_ops[cpu]->cpu_kill) + return 1; + + return cpu_ops[cpu]->cpu_kill(cpu); +} + static DECLARE_COMPLETION(cpu_died); /* @@ -241,6 +254,15 @@ void __cpu_die(unsigned int cpu) return; } pr_notice("CPU%u: shutdown\n", cpu); + + /* + * Now that the dying CPU is beyond the point of no return w.r.t. + * in-kernel synchronisation, try to get the firwmare to help us to + * verify that it has really left the kernel before we consider + * clobbering anything it might still be using. + */ + if (!op_cpu_kill(cpu)) + pr_warn("CPU%d may not have shut down cleanly\n", cpu); } /*