From patchwork Wed Feb 26 18:47:36 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 25407 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f197.google.com (mail-ve0-f197.google.com [209.85.128.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 9CFA6203C4 for ; Wed, 26 Feb 2014 18:48:39 +0000 (UTC) Received: by mail-ve0-f197.google.com with SMTP id oz11sf5832367veb.8 for ; Wed, 26 Feb 2014 10:48:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id:cc :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :content-type:content-transfer-encoding; bh=Vq4WBT/VKL5r7rIEYntxNdO7P+JzDIWNQtgBXj3tu4Y=; b=JiutIFvuriHwOP1H7eNQ1mQat94tuIiJBlI/Pkfs9nwunZtwxqvZlru/v41phW961B vHX76of5p6AWbi+4b39nz51YVdVQxkKpfsFfeNIBxkwTiPjI3BOOzvHwS+CKrA55UP9B G5CKJWs8PqneW5yrgtUftkrv3uvp/0saqmbkMAlXvG31zXQJlDPKobXbtSYSJql4FY24 qS1MynV1WrX8jlByVk5NSWxHoyU9T1Yt6qqABHSoafhKi4+Idw+OGVdSMo9aHAryPlRG 8xRjPmtwx2CWGDvTKrAPYeVVrEbV9LNQJV67oRx9urr+bfVOu77qEzpAGWiuKUjLSkv4 J9WQ== X-Gm-Message-State: ALoCoQnl+KVv2yj8y5ie73Qo1Po/v0PxKtD4Nlfd40nfXzWvMG8oPAHt8EGjse/H3Pm4B3y8l/mQ X-Received: by 10.58.179.101 with SMTP id df5mr4127890vec.38.1393440519416; Wed, 26 Feb 2014 10:48:39 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.27.117 with SMTP id 108ls334129qgw.77.gmail; Wed, 26 Feb 2014 10:48:39 -0800 (PST) X-Received: by 10.221.39.196 with SMTP id tn4mr109283vcb.51.1393440519273; Wed, 26 Feb 2014 10:48:39 -0800 (PST) Received: from mail-ve0-f173.google.com (mail-ve0-f173.google.com [209.85.128.173]) by mx.google.com with ESMTPS id ry8si229483vdc.32.2014.02.26.10.48.39 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Feb 2014 10:48:39 -0800 (PST) Received-SPF: neutral (google.com: 209.85.128.173 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.173; Received: by mail-ve0-f173.google.com with SMTP id jw12so2654920veb.32 for ; Wed, 26 Feb 2014 10:48:39 -0800 (PST) X-Received: by 10.53.9.107 with SMTP id dr11mr5795694vdd.1.1393440519174; Wed, 26 Feb 2014 10:48:39 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp47228vcz; Wed, 26 Feb 2014 10:48:38 -0800 (PST) X-Received: by 10.180.13.197 with SMTP id j5mr9327437wic.14.1393440518077; Wed, 26 Feb 2014 10:48:38 -0800 (PST) Received: from casper.infradead.org (casper.infradead.org. [2001:770:15f::2]) by mx.google.com with ESMTPS id f18si11028328wiw.2.2014.02.26.10.48.37 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Feb 2014 10:48:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:770:15f::2 as permitted sender) client-ip=2001:770:15f::2; Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WIjWc-0005sW-J6; Wed, 26 Feb 2014 18:48:10 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WIjWa-00032a-1L; Wed, 26 Feb 2014 18:48:08 +0000 Received: from fw-tnat.austin.arm.com ([217.140.110.23] helo=collaborate-mta1.arm.com) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WIjWW-00031s-SS for linux-arm-kernel@lists.infradead.org; Wed, 26 Feb 2014 18:48:06 +0000 Received: from e102391-lin.cambridge.arm.com (e102391-lin.cambridge.arm.com [10.1.209.166]) by collaborate-mta1.arm.com (Postfix) with ESMTP id C994813F803; Wed, 26 Feb 2014 12:47:36 -0600 (CST) From: Marc Zyngier To: Gleb Natapov , Paolo Bonzini Subject: [PATCH] arm/arm64: KVM: detect CPU reset on CPU_PM_EXIT Date: Wed, 26 Feb 2014 18:47:36 +0000 Message-Id: <1393440456-2177-1-git-send-email-marc.zyngier@arm.com> X-Mailer: git-send-email 1.8.3.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140226_134805_033536_B28C5EF8 X-CRM114-Status: GOOD ( 18.32 ) X-Spam-Score: -1.9 (-) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (-1.9 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: Rob Herring , Lorenzo Pieralisi , Andre Przywara , kvm@vger.kernel.org, Christoffer Dall , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: marc.zyngier@arm.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.173 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Commit 1fcf7ce0c602 (arm: kvm: implement CPU PM notifier) added support for CPU power-management, using a cpu_notifier to re-init KVM on a CPU that entered CPU idle. The code assumed that a CPU entering idle would actually be powered off, loosing its state entierely, and would then need to be reinitialized. It turns out that this is not always the case, and some HW performs CPU PM without actually killing the core. In this case, we try to reinitialize KVM while it is still live. It ends up badly, as reported by Andre Przywara (using a Calxeda Midway): [ 3.663897] Kernel panic - not syncing: unexpected prefetch abort in Hyp mode at: 0x685760 [ 3.663897] unexpected data abort in Hyp mode at: 0xc067d150 [ 3.663897] unexpected HVC/SVC trap in Hyp mode at: 0xc0901dd0 The trick here is to detect if we've been through a full re-init or not by looking at HVBAR (VBAR_EL2 on arm64). This involves implementing the backend for __hyp_get_vectors in the main KVM HYP code (rather small), and checking the return value against the default one when the CPU notifier is called on CPU_PM_EXIT. Reported-by: Andre Przywara Tested-by: Andre Przywara Cc: Lorenzo Pieralisi Cc: Rob Herring Acked-by: Christoffer Dall Signed-off-by: Marc Zyngier --- Paulo, Gleb, Can you please queue this as a fix for 3.14? It fixes an issue that has been introduced during the merge window, and it would be good to have it plugged quickly. Thanks, M. arch/arm/kvm/arm.c | 3 ++- arch/arm/kvm/interrupts.S | 11 ++++++++++- arch/arm64/kvm/hyp.S | 27 +++++++++++++++++++++++++-- 3 files changed, 37 insertions(+), 4 deletions(-) diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index 1d8248e..bd18bb8 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -878,7 +878,8 @@ static int hyp_init_cpu_pm_notifier(struct notifier_block *self, unsigned long cmd, void *v) { - if (cmd == CPU_PM_EXIT) { + if (cmd == CPU_PM_EXIT && + __hyp_get_vectors() == hyp_default_vectors) { cpu_init_hyp_mode(NULL); return NOTIFY_OK; } diff --git a/arch/arm/kvm/interrupts.S b/arch/arm/kvm/interrupts.S index ddc1553..0d68d40 100644 --- a/arch/arm/kvm/interrupts.S +++ b/arch/arm/kvm/interrupts.S @@ -220,6 +220,10 @@ after_vfp_restore: * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c). Return values are * passed in r0 and r1. * + * A function pointer with a value of 0xffffffff has a special meaning, + * and is used to implement __hyp_get_vectors in the same way as in + * arch/arm/kernel/hyp_stub.S. + * * The calling convention follows the standard AAPCS: * r0 - r3: caller save * r12: caller save @@ -363,6 +367,11 @@ hyp_hvc: host_switch_to_hyp: pop {r0, r1, r2} + /* Check for __hyp_get_vectors */ + cmp r0, #-1 + mrceq p15, 4, r0, c12, c0, 0 @ get HVBAR + beq 1f + push {lr} mrs lr, SPSR push {lr} @@ -378,7 +387,7 @@ THUMB( orr lr, #1) pop {lr} msr SPSR_csxf, lr pop {lr} - eret +1: eret guest_trap: load_vcpu @ Load VCPU pointer to r0 diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S index 3b47c36..2c56012 100644 --- a/arch/arm64/kvm/hyp.S +++ b/arch/arm64/kvm/hyp.S @@ -694,6 +694,24 @@ __hyp_panic_str: .align 2 +/* + * u64 kvm_call_hyp(void *hypfn, ...); + * + * This is not really a variadic function in the classic C-way and care must + * be taken when calling this to ensure parameters are passed in registers + * only, since the stack will change between the caller and the callee. + * + * Call the function with the first argument containing a pointer to the + * function you wish to call in Hyp mode, and subsequent arguments will be + * passed as x0, x1, and x2 (a maximum of 3 arguments in addition to the + * function pointer can be passed). The function being called must be mapped + * in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c). Return values are + * passed in r0 and r1. + * + * A function pointer with a value of 0 has a special meaning, and is + * used to implement __hyp_get_vectors in the same way as in + * arch/arm64/kernel/hyp_stub.S. + */ ENTRY(kvm_call_hyp) hvc #0 ret @@ -737,7 +755,12 @@ el1_sync: // Guest trapped into EL2 pop x2, x3 pop x0, x1 - push lr, xzr + /* Check for __hyp_get_vectors */ + cbnz x0, 1f + mrs x0, vbar_el2 + b 2f + +1: push lr, xzr /* * Compute the function address in EL2, and shuffle the parameters. @@ -750,7 +773,7 @@ el1_sync: // Guest trapped into EL2 blr lr pop lr, xzr - eret +2: eret el1_trap: /*