From patchwork Fri Jun 1 17:12:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 137571 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1198587lji; Fri, 1 Jun 2018 10:12:21 -0700 (PDT) X-Google-Smtp-Source: ADUXVKJXxnaHL8t8MtNnncMFMiFGQw8dqOHLp3Kk9P3zJqf9/FRy8gZvicTXU7LmlG5ke1QpRWCl X-Received: by 2002:a65:43c9:: with SMTP id n9-v6mr9376295pgp.399.1527873141311; Fri, 01 Jun 2018 10:12:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527873141; cv=none; d=google.com; s=arc-20160816; b=r9jdhWZr8tbVAyWEWNfbU7rwHJKkErFuyvEuqfJ5SKY7ufgJimhlDOPHwgjbNY9jcE BnVqNajXR9LzhZMVH/swjpKkKwa5jke4vuC4vXpkGR+E41BBnws18h54eDycjQf9Zd7M 40ey9uFr9A3RlLtvfUn4DhtxsJQS5RRcRm/Y82YoDsA6TkACWMMO8hJ4R1WM0C2dXWQD nt1pH3JtOg5t2JUVTcKejsA85DTpwefpNptH0SvJ5PXNH1ddYDfcUVusWVlSDBtlzo0T e/70QGBhbWSah2+tcUEaDEFFbPwxy/DQxw2/L4QgDN35qm9F+mtr+Awo7XNFg5gUPP1K c9TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=8qwjRnOByH6vACvfgtkxNb9HxAPwMT43L6ogreFWJVE=; b=ALqRFvMVmt2o3Wx4pZpTuInmLmNmRtyjyubG5YY8rnZnce96GQIqPoanjrsLK1Uw54 BSWwUEk81V1eAYRFEWKIYvwY7r4d4v8URrpACrPxssR93eIo3/T2gHrKxUPEbXRNnWw8 2LJbuMeG821yyi51dk724YwYqk42cpUpHVvYthrKZKw1fpwpkQY/rHP2K8F6GJGrfGgY 3hKpTLjmFPZo7iHFLI73Z3YQn1qZxJuaek4wNznIhVV1Ix6xx5XFRHjRvh6UoYhJxBYA SjdfdpNN0PWm/zkeOW+otjm04m2gldXIUPCJfItdcIqWtmiWyQIXbCpSl/+qAmUjXW7g WK4Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 84-v6si4135003pfj.6.2018.06.01.10.12.21; Fri, 01 Jun 2018 10:12:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752203AbeFARMU (ORCPT + 13 others); Fri, 1 Jun 2018 13:12:20 -0400 Received: from foss.arm.com ([217.140.101.70]:56096 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751964AbeFARMT (ORCPT ); Fri, 1 Jun 2018 13:12:19 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7209515AB; Fri, 1 Jun 2018 10:12:19 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AB08E3F53D; Fri, 1 Jun 2018 10:12:18 -0700 (PDT) From: Mark Rutland To: stable@vger.kernel.org Cc: marc.zyngier@arm.com, mark.rutland@arm.com Subject: [PATCH v4.9.y 1/2] arm64: Add hypervisor safe helper for checking constant capabilities Date: Fri, 1 Jun 2018 18:12:09 +0100 Message-Id: <20180601171210.16870-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180601171210.16870-1-mark.rutland@arm.com> References: <20180601171210.16870-1-mark.rutland@arm.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Suzuki K Poulose commit a4023f682739439b434165b54af7cb3676a4766e upstream. The hypervisor may not have full access to the kernel data structures and hence cannot safely use cpus_have_cap() helper for checking the system capability. Add a safe helper for hypervisors to check a constant system capability, which *doesn't* fall back to checking the bitmap maintained by the kernel. With this, make the cpus_have_cap() only check the bitmask and force constant cap checks to use the new API for quicker checks. Cc: Robert Ritcher Cc: Tirumalesh Chalamarla Signed-off-by: Suzuki K Poulose Reviewed-by: Will Deacon Reviewed-by: Marc Zyngier Signed-off-by: Catalin Marinas [4.9: restore cpus_have_const_cap() to previously-backported code] Signed-off-by: Mark Rutland [v4.9 backport] --- arch/arm64/include/asm/cpufeature.h | 19 ++++++++++++------- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 2 +- arch/arm64/include/asm/mmu.h | 2 +- arch/arm64/kernel/cpufeature.c | 5 +++-- arch/arm64/kernel/process.c | 2 +- drivers/irqchip/irq-gic-v3.c | 13 +------------ 7 files changed, 20 insertions(+), 25 deletions(-) -- 2.11.0 diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 0bc0b1de90c4..9890d20f2356 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -9,8 +9,6 @@ #ifndef __ASM_CPUFEATURE_H #define __ASM_CPUFEATURE_H -#include - #include #include #include @@ -27,6 +25,8 @@ #ifndef __ASSEMBLY__ +#include +#include #include /* CPU feature register tracking */ @@ -104,14 +104,19 @@ static inline bool cpu_have_feature(unsigned int num) return elf_hwcap & (1UL << num); } +/* System capability check for constant caps */ +static inline bool cpus_have_const_cap(int num) +{ + if (num >= ARM64_NCAPS) + return false; + return static_branch_unlikely(&cpu_hwcap_keys[num]); +} + static inline bool cpus_have_cap(unsigned int num) { if (num >= ARM64_NCAPS) return false; - if (__builtin_constant_p(num)) - return static_branch_unlikely(&cpu_hwcap_keys[num]); - else - return test_bit(num, cpu_hwcaps); + return test_bit(num, cpu_hwcaps); } static inline void cpus_set_cap(unsigned int num) @@ -200,7 +205,7 @@ static inline bool cpu_supports_mixed_endian_el0(void) static inline bool system_supports_32bit_el0(void) { - return cpus_have_cap(ARM64_HAS_32BIT_EL0); + return cpus_have_const_cap(ARM64_HAS_32BIT_EL0); } static inline bool system_supports_mixed_endian_el0(void) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 0a33ea304e63..7f402f64c744 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -398,7 +398,7 @@ static inline void __cpu_init_stage2(void) static inline bool kvm_arm_harden_branch_predictor(void) { - return cpus_have_cap(ARM64_HARDEN_BRANCH_PREDICTOR); + return cpus_have_const_cap(ARM64_HARDEN_BRANCH_PREDICTOR); } #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index eac73a640ea7..824c83db9b47 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -341,7 +341,7 @@ static inline void *kvm_get_hyp_vector(void) vect = __bp_harden_hyp_vecs_start + data->hyp_vectors_slot * SZ_2K; - if (!cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + if (!cpus_have_const_cap(ARM64_HAS_VIRT_HOST_EXTN)) vect = lm_alias(vect); } diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index d51158a61892..6ac34c75f4e1 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -37,7 +37,7 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && - cpus_have_cap(ARM64_UNMAP_KERNEL_AT_EL0); + cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0); } typedef void (*bp_hardening_cb_t)(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index a0ee01202503..e28665411bd1 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -47,6 +47,7 @@ unsigned int compat_elf_hwcap2 __read_mostly; #endif DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS); +EXPORT_SYMBOL(cpu_hwcaps); DEFINE_STATIC_KEY_ARRAY_FALSE(cpu_hwcap_keys, ARM64_NCAPS); EXPORT_SYMBOL(cpu_hwcap_keys); @@ -762,7 +763,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, * ThunderX leads to apparent I-cache corruption of kernel text, which * ends as well as you might imagine. Don't even try. */ - if (cpus_have_cap(ARM64_WORKAROUND_CAVIUM_27456)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_27456)) { str = "ARM64_WORKAROUND_CAVIUM_27456"; __kpti_forced = -1; } @@ -1203,5 +1204,5 @@ void __init setup_cpu_features(void) static bool __maybe_unused cpufeature_pan_not_uao(const struct arm64_cpu_capabilities *entry, int __unused) { - return (cpus_have_cap(ARM64_HAS_PAN) && !cpus_have_cap(ARM64_HAS_UAO)); + return (cpus_have_const_cap(ARM64_HAS_PAN) && !cpus_have_const_cap(ARM64_HAS_UAO)); } diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 0972ce58316d..e917d119490c 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -291,7 +291,7 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, memset(childregs, 0, sizeof(struct pt_regs)); childregs->pstate = PSR_MODE_EL1h; if (IS_ENABLED(CONFIG_ARM64_UAO) && - cpus_have_cap(ARM64_HAS_UAO)) + cpus_have_const_cap(ARM64_HAS_UAO)) childregs->pstate |= PSR_UAO_BIT; p->thread.cpu_context.x19 = stack_start; p->thread.cpu_context.x20 = stk_sz; diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 0b1d5bdd0862..f7b8681aed3f 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -120,11 +120,10 @@ static void gic_redist_wait_for_rwp(void) } #ifdef CONFIG_ARM64 -static DEFINE_STATIC_KEY_FALSE(is_cavium_thunderx); static u64 __maybe_unused gic_read_iar(void) { - if (static_branch_unlikely(&is_cavium_thunderx)) + if (cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_23154)) return gic_read_iar_cavium_thunderx(); else return gic_read_iar_common(); @@ -908,14 +907,6 @@ static const struct irq_domain_ops partition_domain_ops = { .select = gic_irq_domain_select, }; -static void gicv3_enable_quirks(void) -{ -#ifdef CONFIG_ARM64 - if (cpus_have_cap(ARM64_WORKAROUND_CAVIUM_23154)) - static_branch_enable(&is_cavium_thunderx); -#endif -} - static int __init gic_init_bases(void __iomem *dist_base, struct redist_region *rdist_regs, u32 nr_redist_regions, @@ -938,8 +929,6 @@ static int __init gic_init_bases(void __iomem *dist_base, gic_data.nr_redist_regions = nr_redist_regions; gic_data.redist_stride = redist_stride; - gicv3_enable_quirks(); - /* * Find out how many interrupts are supported. * The GIC only supports up to 1020 interrupt sources (SGI+PPI+SPI)