From patchwork Thu Apr 12 11:11:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 133215 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp1571925ljb; Thu, 12 Apr 2018 04:12:57 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+2qvsK9HDn1fbnuF59ANOy50IuA19LCleSg7I0B4f8XO92FNVdybkaV2ECfV74RmPn0346 X-Received: by 2002:a17:902:3181:: with SMTP id x1-v6mr571635plb.2.1523531577394; Thu, 12 Apr 2018 04:12:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523531577; cv=none; d=google.com; s=arc-20160816; b=mUpKU66BJoVsilam3kH5N6o6r6oGdeCu3atoQvUNsc4bWtMl2Mb/2y5i7/7cKH8f9M 0bUkvxOiJDPsWep9y8lRHaBgQszvBi928CXf+I4bi7V+FXbqq0imyX/29nODUybToZ4u cTN0z8F1mZhULDZiKHgg2l6S24xH1qI8HC8pgq0O+C4OGcDFzMb71LdGU/boZNcTWaA6 6j28PZ7xoZdm+FG364zvZ79i+P7BQ07ircob/2Ac6BBXsh7F9k7JLih8def0Ry4nUapq 3uj6/jDdU+cnmfbtsFiHWid2dFkW3vg+O0EJ/mD+9BMYnvo8yqIxHkkbL02XyF1Q9MeZ 5EvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=16z6l1kI1rkVSnjXtvq4/qartus6ORDTd3pRRcKSVw8=; b=e+SooEp0tiyNoNV329HfvQGlgNreZWBYtsTVK+8f6I74nzV0NpTqt0yo+PIRVepI78 1aT/0IJY8C0eUDm18U0/c6v9DsPJLM6e3bMv9YHDr3NxsfGgmanA0lyPzCGqDs5nxabl vEBDWUt2fa16yqYPyEKfA5sbXjTccLxlndBZcDb1Oyc+kEFNvkypkjHbA1fn5h3fFu2A v8aJI6bttxE6zy0Fm663lCQqGK6NYpMy98ofF5jVWvlERb6DA9VYm+1ajKKs+yUoPHQ+ wvYU1K6AfxwPz1hA/yaseH9yYJPqxvj+m9KZsTh4FJRTcSq+YA41H2fFpRTyYpTYIGnJ M4Qg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id bg3-v6si3001807plb.118.2018.04.12.04.12.57; Thu, 12 Apr 2018 04:12:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752914AbeDLLM4 (ORCPT + 11 others); Thu, 12 Apr 2018 07:12:56 -0400 Received: from foss.arm.com ([217.140.101.70]:59486 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752854AbeDLLM4 (ORCPT ); Thu, 12 Apr 2018 07:12:56 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ECA4980D; Thu, 12 Apr 2018 04:12:55 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5206F3F24A; Thu, 12 Apr 2018 04:12:54 -0700 (PDT) From: Mark Rutland To: stable@vger.kernel.org Cc: mark.brown@linaro.org, ard.biesheuvel@linaro.org, marc.zyngier@arm.com, will.deacon@arm.com, catalin.marinas@arm.com, ghackmann@google.com, shankerd@codeaurora.org Subject: [PATCH v4.9.y 20/42] arm64: KVM: Use per-CPU vector when BP hardening is enabled Date: Thu, 12 Apr 2018 12:11:16 +0100 Message-Id: <20180412111138.40990-21-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180412111138.40990-1-mark.rutland@arm.com> References: <20180412111138.40990-1-mark.rutland@arm.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 6840bdd73d07216ab4bc46f5a8768c37ea519038 upstream. Now that we have per-CPU vectors, let's plug then in the KVM/arm64 code. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas [v4.9: account for files moved to virt/ upstream, use cpus_have_cap()] Signed-off-by: Mark Rutland [v4.9 backport] --- arch/arm/include/asm/kvm_mmu.h | 10 ++++++++++ arch/arm/kvm/arm.c | 9 ++++++++- arch/arm64/include/asm/kvm_mmu.h | 38 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/switch.c | 2 +- 4 files changed, 57 insertions(+), 2 deletions(-) -- 2.11.0 diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index a58bbaa3ec60..d10e36235438 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -223,6 +223,16 @@ static inline unsigned int kvm_get_vmid_bits(void) return 8; } +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ARM_KVM_MMU_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index c38bfbeec306..5e666b624dc1 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -1088,7 +1088,7 @@ static void cpu_init_hyp_mode(void *dummy) pgd_ptr = kvm_mmu_get_httbr(); stack_page = __this_cpu_read(kvm_arm_hyp_stack_page); hyp_stack_ptr = stack_page + PAGE_SIZE; - vector_ptr = (unsigned long)kvm_ksym_ref(__kvm_hyp_vector); + vector_ptr = (unsigned long)kvm_get_hyp_vector(); __cpu_init_hyp_mode(pgd_ptr, hyp_stack_ptr, vector_ptr); __cpu_init_stage2(); @@ -1345,6 +1345,13 @@ static int init_hyp_mode(void) goto out_err; } + + err = kvm_map_vectors(); + if (err) { + kvm_err("Cannot map vectors\n"); + goto out_err; + } + /* * Map the Hyp stack pages */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 6d22017ebbad..80bf33715ecb 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -313,5 +313,43 @@ static inline unsigned int kvm_get_vmid_bits(void) return (cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR1_VMIDBITS_SHIFT) == 2) ? 16 : 8; } +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +#include + +static inline void *kvm_get_hyp_vector(void) +{ + struct bp_hardening_data *data = arm64_get_bp_hardening_data(); + void *vect = kvm_ksym_ref(__kvm_hyp_vector); + + if (data->fn) { + vect = __bp_harden_hyp_vecs_start + + data->hyp_vectors_slot * SZ_2K; + + if (!cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + vect = lm_alias(vect); + } + + return vect; +} + +static inline int kvm_map_vectors(void) +{ + return create_hyp_mappings(kvm_ksym_ref(__bp_harden_hyp_vecs_start), + kvm_ksym_ref(__bp_harden_hyp_vecs_end), + PAGE_HYP_EXEC); +} + +#else +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} +#endif + #endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */ diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 0c848c18ca44..cf6d962f9639 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -50,7 +50,7 @@ static void __hyp_text __activate_traps_vhe(void) val &= ~CPACR_EL1_FPEN; write_sysreg(val, cpacr_el1); - write_sysreg(__kvm_hyp_vector, vbar_el1); + write_sysreg(kvm_get_hyp_vector(), vbar_el1); } static void __hyp_text __activate_traps_nvhe(void)