From patchwork Thu Jan 10 17:51:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 155260 Delivered-To: patches@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp2176385jaa; Thu, 10 Jan 2019 09:52:15 -0800 (PST) X-Received: by 2002:ad4:42d1:: with SMTP id f17mr10772251qvr.59.1547142735576; Thu, 10 Jan 2019 09:52:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547142735; cv=none; d=google.com; s=arc-20160816; b=X3b2edDFfGOTXNxzm+MGq2xhP5p85PNamzui97+YggptSo9HXPJk8P3KNbTgucOYyK XSFJ8/3/XFCi7uThQL3/z+kugHX83QBb/5Hh/2FX5jeTGRbd+bxiuyI/Z/QJnmKsUL9T cK2JMdlBoei/U0CwgxcQVD/63wIqox4VbGi28o3tLlxwp7BkZQniuAa7oyddWoHbmxDY ws3Ns6i/vd6ACrlrRCLLXrxM7SMBkHy9zqJofHHLHVBwxHPJ9eZqNHGKRwEImRTg9ati ixyUftJC3UxbOg57DEG4NFz9zuNlq+Pr/MAiSA4CpQb7t3G7RFTaryuea7JrfXnsAPCM 1q9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=1hhaJXoC3+V1XgOExRQF6uRGWM6Y1bUSSamsFfUInZk=; b=VwxYamV+WQFpj6jM9UkMsTOdd2Zmr9pY/3yW2WdIr5dfCMw6QyQs7bU69FFaPq9ThU CWMPCQJ7ZzedunSdm2UFu5mYCGUT9ipGYL8+ztpu/WXMolPGbuNrSpmYf6/Y2jzCA7vV UWcdQOEghNSbrTbtrW3OHe+i9SHzViON0fg40lOpP0N4A2/nSXEaJsbtvpd1QTP2rAbS p+uTmqAD3rhOXP7rWVhFx0OS2/BHoZQ+m29TSePfpxWb3NO7iY5byVvBv1U6qHCqGMPu XiRkdEu92f8bOa4W5FWfaYr584NKxTvKBSLm5wi+o4H8sJ62CZYe2P8A4rxpRizGY0vY dcjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KTiuPXr4; spf=pass (google.com: domain of dave.long@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=dave.long@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r12sor72057237qtc.26.2019.01.10.09.52.15 for (Google Transport Security); Thu, 10 Jan 2019 09:52:15 -0800 (PST) Received-SPF: pass (google.com: domain of dave.long@linaro.org designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KTiuPXr4; spf=pass (google.com: domain of dave.long@linaro.org designates 209.85.220.65 as permitted sender) smtp.mailfrom=dave.long@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1hhaJXoC3+V1XgOExRQF6uRGWM6Y1bUSSamsFfUInZk=; b=KTiuPXr4rGss9E3k887ZZSC/YbQBWvPyD9iVEyveD4U6du30y67CK1ssIfi/fjUya7 oopEDUAIo4fEe+BT8wwzb41hCN6iFQUb9V8KW3stGKusHoCXdaJk8g9GAInIAQjXUl8v EXjDryv3EezetdF+0hE8PtVNkK2ePm/JsF2FA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1hhaJXoC3+V1XgOExRQF6uRGWM6Y1bUSSamsFfUInZk=; b=nkJfEx2uCPucXtOSlBeJADEwrjZJpTFSCoj05eY8zSGUSmB+xGqfwZFipnwnxGAX7k 4bluWvrlccWn8KVxvOet7cL7dfEUalaqE9xO1PnJ5gVrSKTXnhloZJC7VD29eL0GvH+2 u/yiW7ZSB7nsD0x97qtrDlB/ZRfHS5f8LcI7HEn9JEREr6wWOOK+luCPcf9hdITgTIE0 E8xX6XUUojnisLA1fbTLbDEn5JIug73j4Sz+bPTdDGASlYS7mj/5+iAKU7dDpmnqG0zZ 8huAHDMIXW5nmp2eiBl4bNsapXMmYcGHF+31ZWqzsfmJh+ZNfUy9phBSfhRktZlpLHt1 apTA== X-Gm-Message-State: AJcUukcKP9GheQwblfFP1LCSdNUxdfXTHsGoA8nONi5EiqQungtVQahe GMLw0Trg1wN7tyDKInY9GIr1TNgF X-Google-Smtp-Source: ALg8bN5E0p9ljd5BQiPaFxRlb9/mP9sWx1Y48+oD5LnBETzlvCeWzCN5cuZn35Aap1mJT1g4lJ1BYQ== X-Received: by 2002:ac8:75cc:: with SMTP id z12mr10420271qtq.249.1547142735265; Thu, 10 Jan 2019 09:52:15 -0800 (PST) Return-Path: Received: from localhost.localdomain (pool-72-71-243-63.cncdnh.fast00.myfairpoint.net. [72.71.243.63]) by smtp.googlemail.com with ESMTPSA id b8sm41815904qka.79.2019.01.10.09.52.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Jan 2019 09:52:14 -0800 (PST) From: David Long To: stable@vger.kernel.org, Russell King - ARM Linux , Florian Fainelli , Julien Thierry , Tony Lindgren , Marc Zyngier , Mark Rutland Cc: Greg KH , Mark Brown Subject: [PATCH 4.14 15/17] ARM: spectre-v2: per-CPU vtables to work around big.Little systems Date: Thu, 10 Jan 2019 12:51:48 -0500 Message-Id: <20190110175150.5239-16-dave.long@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190110175150.5239-1-dave.long@linaro.org> References: <20190110175150.5239-1-dave.long@linaro.org> From: Russell King Commit 383fb3ee8024d596f488d2dbaf45e572897acbdb upstream. In big.Little systems, some CPUs require the Spectre workarounds in paths such as the context switch, but other CPUs do not. In order to handle these differences, we need per-CPU vtables. We are unable to use the kernel's per-CPU variables to support this as per-CPU is not initialised at times when we need access to the vtables, so we have to use an array indexed by logical CPU number. We use an array-of-pointers to avoid having function pointers in the kernel's read/write .data section. Reviewed-by: Julien Thierry Signed-off-by: Russell King Signed-off-by: David A. Long --- arch/arm/include/asm/proc-fns.h | 23 +++++++++++++++++++++++ arch/arm/kernel/setup.c | 5 +++++ arch/arm/kernel/smp.c | 31 +++++++++++++++++++++++++++++++ arch/arm/mm/proc-v7-bugs.c | 17 ++--------------- 4 files changed, 61 insertions(+), 15 deletions(-) -- 2.17.1 diff --git a/arch/arm/include/asm/proc-fns.h b/arch/arm/include/asm/proc-fns.h index c259cc49c641..e1b6f280ab08 100644 --- a/arch/arm/include/asm/proc-fns.h +++ b/arch/arm/include/asm/proc-fns.h @@ -104,12 +104,35 @@ extern void cpu_do_resume(void *); #else extern struct processor processor; +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +#include +/* + * This can't be a per-cpu variable because we need to access it before + * per-cpu has been initialised. We have a couple of functions that are + * called in a pre-emptible context, and so can't use smp_processor_id() + * there, hence PROC_TABLE(). We insist in init_proc_vtable() that the + * function pointers for these are identical across all CPUs. + */ +extern struct processor *cpu_vtable[]; +#define PROC_VTABLE(f) cpu_vtable[smp_processor_id()]->f +#define PROC_TABLE(f) cpu_vtable[0]->f +static inline void init_proc_vtable(const struct processor *p) +{ + unsigned int cpu = smp_processor_id(); + *cpu_vtable[cpu] = *p; + WARN_ON_ONCE(cpu_vtable[cpu]->dcache_clean_area != + cpu_vtable[0]->dcache_clean_area); + WARN_ON_ONCE(cpu_vtable[cpu]->set_pte_ext != + cpu_vtable[0]->set_pte_ext); +} +#else #define PROC_VTABLE(f) processor.f #define PROC_TABLE(f) processor.f static inline void init_proc_vtable(const struct processor *p) { processor = *p; } +#endif #define cpu_proc_init PROC_VTABLE(_proc_init) #define cpu_check_bugs PROC_VTABLE(check_bugs) diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c index c1588e31aa83..a6d27284105a 100644 --- a/arch/arm/kernel/setup.c +++ b/arch/arm/kernel/setup.c @@ -115,6 +115,11 @@ EXPORT_SYMBOL(elf_hwcap2); #ifdef MULTI_CPU struct processor processor __ro_after_init; +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +struct processor *cpu_vtable[NR_CPUS] = { + [0] = &processor, +}; +#endif #endif #ifdef MULTI_TLB struct cpu_tlb_fns cpu_tlb __ro_after_init; diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index e61af0600133..f6b1c9d2e178 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include #include @@ -102,6 +103,30 @@ static unsigned long get_arch_pgd(pgd_t *pgd) #endif } +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +static int secondary_biglittle_prepare(unsigned int cpu) +{ + if (!cpu_vtable[cpu]) + cpu_vtable[cpu] = kzalloc(sizeof(*cpu_vtable[cpu]), GFP_KERNEL); + + return cpu_vtable[cpu] ? 0 : -ENOMEM; +} + +static void secondary_biglittle_init(void) +{ + init_proc_vtable(lookup_processor(read_cpuid_id())->proc); +} +#else +static int secondary_biglittle_prepare(unsigned int cpu) +{ + return 0; +} + +static void secondary_biglittle_init(void) +{ +} +#endif + int __cpu_up(unsigned int cpu, struct task_struct *idle) { int ret; @@ -109,6 +134,10 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) if (!smp_ops.smp_boot_secondary) return -ENOSYS; + ret = secondary_biglittle_prepare(cpu); + if (ret) + return ret; + /* * We need to tell the secondary core where to find * its stack and the page tables. @@ -360,6 +389,8 @@ asmlinkage void secondary_start_kernel(void) struct mm_struct *mm = &init_mm; unsigned int cpu; + secondary_biglittle_init(); + /* * The identity mapping is uncached (strongly ordered), so * switch away from it before attempting any exclusive accesses. diff --git a/arch/arm/mm/proc-v7-bugs.c b/arch/arm/mm/proc-v7-bugs.c index 5544b82a2e7a..9a07916af8dd 100644 --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -52,8 +52,6 @@ static void cpu_v7_spectre_init(void) case ARM_CPU_PART_CORTEX_A17: case ARM_CPU_PART_CORTEX_A73: case ARM_CPU_PART_CORTEX_A75: - if (processor.switch_mm != cpu_v7_bpiall_switch_mm) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = harden_branch_predictor_bpiall; spectre_v2_method = "BPIALL"; @@ -61,8 +59,6 @@ static void cpu_v7_spectre_init(void) case ARM_CPU_PART_CORTEX_A15: case ARM_CPU_PART_BRAHMA_B15: - if (processor.switch_mm != cpu_v7_iciallu_switch_mm) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = harden_branch_predictor_iciallu; spectre_v2_method = "ICIALLU"; @@ -88,11 +84,9 @@ static void cpu_v7_spectre_init(void) ARM_SMCCC_ARCH_WORKAROUND_1, &res); if ((int)res.a0 != 0) break; - if (processor.switch_mm != cpu_v7_hvc_switch_mm && cpu) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = call_hvc_arch_workaround_1; - processor.switch_mm = cpu_v7_hvc_switch_mm; + cpu_do_switch_mm = cpu_v7_hvc_switch_mm; spectre_v2_method = "hypervisor"; break; @@ -101,11 +95,9 @@ static void cpu_v7_spectre_init(void) ARM_SMCCC_ARCH_WORKAROUND_1, &res); if ((int)res.a0 != 0) break; - if (processor.switch_mm != cpu_v7_smc_switch_mm && cpu) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = call_smc_arch_workaround_1; - processor.switch_mm = cpu_v7_smc_switch_mm; + cpu_do_switch_mm = cpu_v7_smc_switch_mm; spectre_v2_method = "firmware"; break; @@ -119,11 +111,6 @@ static void cpu_v7_spectre_init(void) if (spectre_v2_method) pr_info("CPU%u: Spectre v2: using %s workaround\n", smp_processor_id(), spectre_v2_method); - return; - -bl_error: - pr_err("CPU%u: Spectre v2: incorrect context switching function, system vulnerable\n", - cpu); } #else static void cpu_v7_spectre_init(void)