From patchwork Tue Oct 20 16:19:05 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 55317 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f198.google.com (mail-wi0-f198.google.com [209.85.212.198]) by patches.linaro.org (Postfix) with ESMTPS id D623523024 for ; Tue, 20 Oct 2015 16:21:01 +0000 (UTC) Received: by wibhd5 with SMTP id hd5sf1666852wib.0 for ; Tue, 20 Oct 2015 09:21:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:cc:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list; bh=T7rmY4kfRkPU4PPY8CXVCmx9Jj2FogNTv95sujkNVOQ=; b=D88p5sGIGhoYmuaL19B9xe9vV089mVqhZ6hWcmRCmmaVVDX8AKrSV09PgptYYDtSSp 3Y+JCn8mtxCUbw7Ci3OZ4qpxyrBjyJDo3IP8fEQJ3575pIMaSczMfbwNFi7KS6essGG2 dKjErH19Bgwb2PmXrpDSwTan2T6g+ukcwo2SRtFNbcerJX8VCCb9aDIkV0aIS/peIDx7 ojbjwQLB+gGXYTt7exxm/ZMC+Elw12DsaNQhi+/1jNXavgTVdGMCUq/HYlFAEaj+SHW7 pYhZQSLA7LS3xKmjKwyl0xG0KHHsRn5MZusuyAwEkrYNOabTSqDKe/VzAfaQVOVmosFL H9Xw== X-Gm-Message-State: ALoCoQneK3hl3eGYUykNJ2qFiXnybhIq+l4ojOoMpx8FrqKeG7D5Lx2oFyW7k7A8hqZTPILA4Vgb X-Received: by 10.112.144.105 with SMTP id sl9mr844509lbb.3.1445358061212; Tue, 20 Oct 2015 09:21:01 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.25.28.195 with SMTP id c186ls100136lfc.71.gmail; Tue, 20 Oct 2015 09:21:01 -0700 (PDT) X-Received: by 10.25.17.78 with SMTP id g75mr1627214lfi.12.1445358061078; Tue, 20 Oct 2015 09:21:01 -0700 (PDT) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com. [209.85.215.45]) by mx.google.com with ESMTPS id su6si2881140lbb.2.2015.10.20.09.21.01 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Oct 2015 09:21:01 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) client-ip=209.85.215.45; Received: by lffz202 with SMTP id z202so8495076lff.3 for ; Tue, 20 Oct 2015 09:21:01 -0700 (PDT) X-Received: by 10.25.207.82 with SMTP id f79mr1622558lfg.73.1445358060936; Tue, 20 Oct 2015 09:21:00 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp2179453lbq; Tue, 20 Oct 2015 09:20:59 -0700 (PDT) X-Received: by 10.107.168.4 with SMTP id r4mr5653899ioe.4.1445358059756; Tue, 20 Oct 2015 09:20:59 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id 3si3742322ioo.57.2015.10.20.09.20.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Oct 2015 09:20:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZoZdc-0001CQ-Kl; Tue, 20 Oct 2015 16:19:48 +0000 Received: from mail-lf0-f53.google.com ([209.85.215.53]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZoZct-0000Wy-LP for linux-arm-kernel@lists.infradead.org; Tue, 20 Oct 2015 16:19:06 +0000 Received: by lfaz124 with SMTP id z124so8435065lfa.1 for ; Tue, 20 Oct 2015 09:18:41 -0700 (PDT) X-Received: by 10.25.168.213 with SMTP id r204mr1584310lfe.54.1445357921506; Tue, 20 Oct 2015 09:18:41 -0700 (PDT) Received: from localhost.localdomain (0187900153.0.fullrate.dk. [2.110.55.193]) by smtp.gmail.com with ESMTPSA id t188sm644170lfe.33.2015.10.20.09.18.40 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 20 Oct 2015 09:18:40 -0700 (PDT) From: Christoffer Dall To: Paolo Bonzini , kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [GIT PULL 4/6] arm/arm64: KVM: Fix arch timer behavior for disabled interrupts Date: Tue, 20 Oct 2015 18:19:05 +0200 Message-Id: <1445357947-6022-5-git-send-email-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.1.2.330.g565301e.dirty In-Reply-To: <1445357947-6022-1-git-send-email-christoffer.dall@linaro.org> References: <1445357947-6022-1-git-send-email-christoffer.dall@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20151020_091904_326710_3A049DA3 X-CRM114-Status: GOOD ( 23.68 ) X-Spam-Score: -2.6 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.215.53 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.215.53 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Cc: Marc Zyngier , Christoffer Dall MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: christoffer.dall@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 We have an interesting issue when the guest disables the timer interrupt on the VGIC, which happens when turning VCPUs off using PSCI, for example. The problem is that because the guest disables the virtual interrupt at the VGIC level, we never inject interrupts to the guest and therefore never mark the interrupt as active on the physical distributor. The host also never takes the timer interrupt (we only use the timer device to trigger a guest exit and everything else is done in software), so the interrupt does not become active through normal means. The result is that we keep entering the guest with a programmed timer that will always fire as soon as we context switch the hardware timer state and run the guest, preventing forward progress for the VCPU. Since the active state on the physical distributor is really part of the timer logic, it is the job of our virtual arch timer driver to manage this state. The timer->map->active boolean field indicates whether we have signalled this interrupt to the vgic and if that interrupt is still pending or active. As long as that is the case, the hardware doesn't have to generate physical interrupts and therefore we mark the interrupt as active on the physical distributor. We also have to restore the pending state of an interrupt that was queued to an LR but was retired from the LR for some reason, while remaining pending in the LR. Cc: Marc Zyngier Reported-by: Lorenzo Pieralisi Signed-off-by: Christoffer Dall --- virt/kvm/arm/arch_timer.c | 19 +++++++++++++++++++ virt/kvm/arm/vgic.c | 43 +++++++++++-------------------------------- 2 files changed, 30 insertions(+), 32 deletions(-) diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c index 48c6e1a..b9d3a32 100644 --- a/virt/kvm/arm/arch_timer.c +++ b/virt/kvm/arm/arch_timer.c @@ -137,6 +137,8 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu) void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu) { struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu; + bool phys_active; + int ret; /* * We're about to run this vcpu again, so there is no need to @@ -151,6 +153,23 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu) */ if (kvm_timer_should_fire(vcpu)) kvm_timer_inject_irq(vcpu); + + /* + * We keep track of whether the edge-triggered interrupt has been + * signalled to the vgic/guest, and if so, we mask the interrupt and + * the physical distributor to prevent the timer from raising a + * physical interrupt whenever we run a guest, preventing forward + * VCPU progress. + */ + if (kvm_vgic_get_phys_irq_active(timer->map)) + phys_active = true; + else + phys_active = false; + + ret = irq_set_irqchip_state(timer->map->irq, + IRQCHIP_STATE_ACTIVE, + phys_active); + WARN_ON(ret); } /** diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c index 596455a..ea21bc2 100644 --- a/virt/kvm/arm/vgic.c +++ b/virt/kvm/arm/vgic.c @@ -1092,6 +1092,15 @@ static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu) struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; struct vgic_lr vlr = vgic_get_lr(vcpu, lr_nr); + /* + * We must transfer the pending state back to the distributor before + * retiring the LR, otherwise we may loose edge-triggered interrupts. + */ + if (vlr.state & LR_STATE_PENDING) { + vgic_dist_irq_set_pending(vcpu, irq); + vlr.hwirq = 0; + } + vlr.state = 0; vgic_set_lr(vcpu, lr_nr, vlr); clear_bit(lr_nr, vgic_cpu->lr_used); @@ -1241,7 +1250,7 @@ static void __kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu) struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu; struct vgic_dist *dist = &vcpu->kvm->arch.vgic; unsigned long *pa_percpu, *pa_shared; - int i, vcpu_id, lr, ret; + int i, vcpu_id; int overflow = 0; int nr_shared = vgic_nr_shared_irqs(dist); @@ -1296,31 +1305,6 @@ epilog: */ clear_bit(vcpu_id, dist->irq_pending_on_cpu); } - - for (lr = 0; lr < vgic->nr_lr; lr++) { - struct vgic_lr vlr; - - if (!test_bit(lr, vgic_cpu->lr_used)) - continue; - - vlr = vgic_get_lr(vcpu, lr); - - /* - * If we have a mapping, and the virtual interrupt is - * presented to the guest (as pending or active), then we must - * set the state to active in the physical world. See - * Documentation/virtual/kvm/arm/vgic-mapped-irqs.txt. - */ - if (vlr.state & LR_HW) { - struct irq_phys_map *map; - map = vgic_irq_map_search(vcpu, vlr.irq); - - ret = irq_set_irqchip_state(map->irq, - IRQCHIP_STATE_ACTIVE, - true); - WARN_ON(ret); - } - } } static bool vgic_process_maintenance(struct kvm_vcpu *vcpu) @@ -1430,13 +1414,8 @@ static int vgic_sync_hwirq(struct kvm_vcpu *vcpu, struct vgic_lr vlr) WARN_ON(ret); - if (map->active) { - ret = irq_set_irqchip_state(map->irq, - IRQCHIP_STATE_ACTIVE, - false); - WARN_ON(ret); + if (map->active) return 0; - } return 1; }