From patchwork Thu Jul 10 18:13:16 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefano Stabellini X-Patchwork-Id: 33451 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f71.google.com (mail-qa0-f71.google.com [209.85.216.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 5C6A4203C0 for ; Thu, 10 Jul 2014 18:16:07 +0000 (UTC) Received: by mail-qa0-f71.google.com with SMTP id m5sf27137895qaj.10 for ; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=weCpBqf1PAYaHj6laBYoa9bdOIPenOhm4omx6zESHU4=; b=M1Z3+6Yb1kaX6OzWstJmEXLpi3YV2/C8RnjsgbzNDoq6P+IIa2ngsvUBoMttbbhiy1 Vu6wMa5DDVn6GIw5xeGiyCSrsA4M34KWFvJQpfiwzmh3Xg66cC5TZMBYfbGtwfMRM5IF CZxnbbY+I3ngBpNuJofZ/n+uGNq0ODQ/46tCsGnsR0rCqrehZiZODuDWP5qKOu02gVnq TxqbLUCvKHYQZhBOIhfmX0pPZHE1llb5xf86sEBFY5L7UJIWqXEdGLyUxZ69fzVAQYXl H3/OVhEJpIOQ6fOYetj5tgBGywATxHW8PzdPFEGKeAhjNN9X9kVQeq/0t3X47/6Z6LZy kwLw== X-Gm-Message-State: ALoCoQlrHfD3p2sZBzos7JzFisHDa8jyNMSIivbV/xav3GcnOsJRmv+nVaM5/3/pCJIPv3bnFGGF X-Received: by 10.58.111.73 with SMTP id ig9mr4850926veb.40.1405016161922; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.41.231 with SMTP id z94ls151806qgz.63.gmail; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) X-Received: by 10.52.253.140 with SMTP id aa12mr17222106vdd.11.1405016161701; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx.google.com with ESMTPS id xt7si23371529veb.104.2014.07.10.11.16.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Jul 2014 11:16:01 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) client-ip=209.85.220.181; Received: by mail-vc0-f181.google.com with SMTP id hu12so107699vcb.12 for ; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) X-Received: by 10.220.92.135 with SMTP id r7mr46140763vcm.11.1405016161615; Thu, 10 Jul 2014 11:16:01 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp156638vcb; Thu, 10 Jul 2014 11:16:00 -0700 (PDT) X-Received: by 10.194.243.200 with SMTP id xa8mr17984500wjc.97.1405016160106; Thu, 10 Jul 2014 11:16:00 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id p16si13883004wie.89.2014.07.10.11.15.59 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 10 Jul 2014 11:16:00 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X5IrZ-0008R0-AA; Thu, 10 Jul 2014 18:14:33 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X5IrW-0008Oj-H3 for xen-devel@lists.xensource.com; Thu, 10 Jul 2014 18:14:30 +0000 Received: from [85.158.139.211:5380] by server-13.bemta-5.messagelabs.com id CC/1E-02995-508DEB35; Thu, 10 Jul 2014 18:14:29 +0000 X-Env-Sender: Stefano.Stabellini@citrix.com X-Msg-Ref: server-12.tower-206.messagelabs.com!1405016065!14834761!3 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 27621 invoked from network); 10 Jul 2014 18:14:28 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-12.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 10 Jul 2014 18:14:28 -0000 X-IronPort-AV: E=Sophos;i="5.01,639,1400025600"; d="scan'208";a="151781395" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 10 Jul 2014 18:14:25 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Thu, 10 Jul 2014 14:14:24 -0400 Received: from kaball.uk.xensource.com ([10.80.2.59]) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1X5IrK-00037s-R0; Thu, 10 Jul 2014 19:14:18 +0100 From: Stefano Stabellini To: Date: Thu, 10 Jul 2014 19:13:16 +0100 Message-ID: <1405016003-19131-3-git-send-email-stefano.stabellini@eu.citrix.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: References: MIME-Version: 1.0 X-DLP: MIA1 Cc: julien.grall@citrix.com, Ian.Campbell@citrix.com, Stefano Stabellini Subject: [Xen-devel] [PATCH v8 03/10] xen/arm: inflight irqs during migration X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: stefano.stabellini@eu.citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: We need to take special care when migrating irqs that are already inflight from one vcpu to another. See "The effect of changes to an GICD_ITARGETSR", part of chapter 4.3.12 of the ARM Generic Interrupt Controller Architecture Specification. The main issue from the Xen point of view is that the lr_pending and inflight lists are per-vcpu. The lock we take to protect them is also per-vcpu. In order to avoid issues, if the irq is still lr_pending, we can immediately move it to the new vcpu for injection. Otherwise if it is in a GICH_LR register, set a new flag GIC_IRQ_GUEST_MIGRATING, so that we can recognize when we receive an irq while the previous one is still inflight (given that we are only dealing with hardware interrupts here, it just means that its LR hasn't been cleared yet on the old vcpu). If GIC_IRQ_GUEST_MIGRATING is set, we only set GIC_IRQ_GUEST_QUEUED and interrupt the old vcpu. To know which one is the old vcpu, we introduce a new field to pending_irq, called vcpu_migrate_from. When clearing the LR on the old vcpu, we take special care of injecting the interrupt into the new vcpu. To do that we need to release the old vcpu lock before taking the new vcpu lock. Signed-off-by: Stefano Stabellini --- Changes in v8: - rebase on ab78724fc5628318b172b4344f7280621a151e1b; - rename target to new_target to avoid conflicts. Changes in v7: - move the _VPF_down check before setting GIC_IRQ_GUEST_QUEUED; - fix comments; - rename trl to target; - introduce vcpu_migrate_from; - do not kick new vcpu on MIGRATING, kick the old vcpu instead; - separate moving GIC_IRQ_GUEST_QUEUED earlier into a different patch. Changes in v6: - remove unnecessary casts to (const unsigned long *) to the arguments of find_first_bit and find_next_bit; - instroduce a corresponding smb_rmb call; - deal with migrating an irq that is inflight and still lr_pending; - replace the dsb with smb_wmb and smb_rmb, use them to ensure the order of accesses to GIC_IRQ_GUEST_QUEUED and GIC_IRQ_GUEST_MIGRATING; Changes in v5: - pass unsigned long to find_next_bit for alignment on aarch64; - call vgic_get_target_vcpu instead of gic_add_to_lr_pending to add the irq in the right inflight queue; - add barrier and bit tests to make sure that vgic_migrate_irq and gic_update_one_lr can run simultaneously on different cpus without issues; - rework the loop to identify the new vcpu when ITARGETSR is written; - use find_first_bit instead of find_next_bit. --- xen/arch/arm/gic.c | 28 ++++++++++++++++++++-- xen/arch/arm/vgic-v2.c | 43 +++++++++++++++++++++++++++------- xen/arch/arm/vgic.c | 56 ++++++++++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/vgic.h | 8 +++++++ 4 files changed, 124 insertions(+), 11 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 256d9cf..a0252f9 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -33,6 +33,7 @@ #include #include #include +#include static void gic_restore_pending_irqs(struct vcpu *v); @@ -375,10 +376,33 @@ static void gic_update_one_lr(struct vcpu *v, int i) clear_bit(GIC_IRQ_GUEST_ACTIVE, &p->status); p->lr = GIC_INVALID_LR; if ( test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) && - test_bit(GIC_IRQ_GUEST_QUEUED, &p->status) ) + test_bit(GIC_IRQ_GUEST_QUEUED, &p->status) && + !test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) gic_raise_guest_irq(v, irq, p->priority); - else + else { + int m, q; list_del_init(&p->inflight); + + m = test_and_clear_bit(GIC_IRQ_GUEST_MIGRATING, &p->status); + p->vcpu_migrate_from = NULL; + /* check MIGRATING before QUEUED */ + smp_rmb(); + q = test_bit(GIC_IRQ_GUEST_QUEUED, &p->status); + if ( m && q ) + { + struct vcpu *v_target; + + /* It is safe to temporarily drop the lock because we + * are finished dealing with this LR. We'll take the + * lock before reading the next. */ + spin_unlock(&v->arch.vgic.lock); + /* vgic_get_target_vcpu takes the rank lock, ensuring + * consistency with other itarget changes. */ + v_target = vgic_get_target_vcpu(v, irq); + vgic_vcpu_inject_irq(v_target, irq); + spin_lock(&v->arch.vgic.lock); + } + } } } diff --git a/xen/arch/arm/vgic-v2.c b/xen/arch/arm/vgic-v2.c index 9629cbe..5116a99 100644 --- a/xen/arch/arm/vgic-v2.c +++ b/xen/arch/arm/vgic-v2.c @@ -356,34 +356,59 @@ static int vgic_v2_distr_mmio_write(struct vcpu *v, mmio_info_t *info) goto write_ignore; case GICD_ITARGETSR + 8 ... GICD_ITARGETSRN: + { + /* unsigned long needed for find_next_bit */ + unsigned long target; + int i; if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width; rank = vgic_rank_offset(v, 8, gicd_reg - GICD_ITARGETSR, DABT_WORD); if ( rank == NULL) goto write_ignore; /* 8-bit vcpu mask for this domain */ BUG_ON(v->domain->max_vcpus > 8); - tr = (1 << v->domain->max_vcpus) - 1; + target = (1 << v->domain->max_vcpus) - 1; if ( dabt.size == 2 ) - tr = tr | (tr << 8) | (tr << 16) | (tr << 24); + target = target | (target << 8) | (target << 16) | (target << 24); else - tr = (tr << (8 * (gicd_reg & 0x3))); - tr &= *r; + target = (target << (8 * (gicd_reg & 0x3))); + target &= *r; /* ignore zero writes */ - if ( !tr ) + if ( !target ) goto write_ignore; /* For word reads ignore writes where any single byte is zero */ if ( dabt.size == 2 && - !((tr & 0xff) && (tr & (0xff << 8)) && - (tr & (0xff << 16)) && (tr & (0xff << 24)))) + !((target & 0xff) && (target & (0xff << 8)) && + (target & (0xff << 16)) && (target & (0xff << 24)))) goto write_ignore; vgic_lock_rank(v, rank); + i = 0; + while ( (i = find_next_bit(&target, 32, i)) < 32 ) + { + unsigned int irq, new_target, old_target; + unsigned long old_target_mask; + struct vcpu *v_target, *v_old; + + new_target = i % 8; + old_target_mask = vgic_byte_read(rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR, DABT_WORD)], 0, i/8); + old_target = find_first_bit(&old_target_mask, 8); + + if ( new_target != old_target ) + { + irq = gicd_reg + (i / 8); + v_target = v->domain->vcpu[new_target]; + v_old = v->domain->vcpu[old_target]; + vgic_migrate_irq(v_old, v_target, irq); + } + i += 8 - new_target; + } if ( dabt.size == DABT_WORD ) rank->itargets[REG_RANK_INDEX(8, gicd_reg - GICD_ITARGETSR, - DABT_WORD)] = tr; + DABT_WORD)] = target; else vgic_byte_write(&rank->itargets[REG_RANK_INDEX(8, - gicd_reg - GICD_ITARGETSR, DABT_WORD)], tr, gicd_reg); + gicd_reg - GICD_ITARGETSR, DABT_WORD)], target, gicd_reg); vgic_unlock_rank(v, rank); return 1; + } case GICD_IPRIORITYR ... GICD_IPRIORITYRN: if ( dabt.size != DABT_BYTE && dabt.size != DABT_WORD ) goto bad_width; diff --git a/xen/arch/arm/vgic.c b/xen/arch/arm/vgic.c index 538e7a1..fbe3293 100644 --- a/xen/arch/arm/vgic.c +++ b/xen/arch/arm/vgic.c @@ -182,6 +182,46 @@ struct vcpu *vgic_get_target_vcpu(struct vcpu *v, unsigned int irq) return v_target; } +void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq) +{ + unsigned long flags; + struct pending_irq *p = irq_to_pending(old, irq); + + /* nothing to do for virtual interrupts */ + if ( p->desc == NULL ) + return; + + /* migration already in progress, no need to do anything */ + if ( test_bit(GIC_IRQ_GUEST_MIGRATING, &p->status) ) + return; + + spin_lock_irqsave(&old->arch.vgic.lock, flags); + + if ( list_empty(&p->inflight) ) + { + spin_unlock_irqrestore(&old->arch.vgic.lock, flags); + return; + } + /* If the IRQ is still lr_pending, re-inject it to the new vcpu */ + if ( !list_empty(&p->lr_queue) ) + { + list_del_init(&p->lr_queue); + list_del_init(&p->inflight); + spin_unlock_irqrestore(&old->arch.vgic.lock, flags); + vgic_vcpu_inject_irq(new, irq); + return; + } + /* if the IRQ is in a GICH_LR register, set GIC_IRQ_GUEST_MIGRATING + * and wait for the EOI */ + if ( !list_empty(&p->inflight) ) + { + p->vcpu_migrate_from = old; + set_bit(GIC_IRQ_GUEST_MIGRATING, &p->status); + } + + spin_unlock_irqrestore(&old->arch.vgic.lock, flags); +} + void vgic_disable_irqs(struct vcpu *v, uint32_t r, int n) { const unsigned long mask = r; @@ -333,6 +373,7 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq) struct pending_irq *iter, *n = irq_to_pending(v, irq); unsigned long flags; bool_t running; + struct vcpu *vcpu_migrate_from; spin_lock_irqsave(&v->arch.vgic.lock, flags); @@ -344,6 +385,21 @@ void vgic_vcpu_inject_irq(struct vcpu *v, unsigned int irq) } set_bit(GIC_IRQ_GUEST_QUEUED, &n->status); + vcpu_migrate_from = n->vcpu_migrate_from; + /* update QUEUED before MIGRATING */ + smp_wmb(); + if ( test_bit(GIC_IRQ_GUEST_MIGRATING, &n->status) ) + { + spin_unlock_irqrestore(&v->arch.vgic.lock, flags); + + /* The old vcpu must have EOIed the SGI but not cleared the LR. + * Give it a kick. */ + running = n->vcpu_migrate_from->is_running; + vcpu_unblock(n->vcpu_migrate_from); + if ( running ) + smp_send_event_check_mask(cpumask_of(n->vcpu_migrate_from->processor)); + return; + } if ( !list_empty(&n->inflight) ) { diff --git a/xen/include/asm-arm/vgic.h b/xen/include/asm-arm/vgic.h index 19eed7e..c66578b 100644 --- a/xen/include/asm-arm/vgic.h +++ b/xen/include/asm-arm/vgic.h @@ -55,17 +55,24 @@ struct pending_irq * GIC_IRQ_GUEST_ENABLED: the guest IRQ is enabled at the VGICD * level (GICD_ICENABLER/GICD_ISENABLER). * + * GIC_IRQ_GUEST_MIGRATING: the irq is being migrated to a different + * vcpu while it is still inflight and on an GICH_LR register on the + * old vcpu. + * */ #define GIC_IRQ_GUEST_QUEUED 0 #define GIC_IRQ_GUEST_ACTIVE 1 #define GIC_IRQ_GUEST_VISIBLE 2 #define GIC_IRQ_GUEST_ENABLED 3 +#define GIC_IRQ_GUEST_MIGRATING 4 unsigned long status; struct irq_desc *desc; /* only set it the irq corresponds to a physical irq */ int irq; #define GIC_INVALID_LR ~(uint8_t)0 uint8_t lr; uint8_t priority; + /* keeps track of the vcpu this irq is currently migrating from */ + struct vcpu *vcpu_migrate_from; /* inflight is used to append instances of pending_irq to * vgic.inflight_irqs */ struct list_head inflight; @@ -164,6 +171,7 @@ extern int vcpu_vgic_free(struct vcpu *v); extern int vgic_to_sgi(struct vcpu *v, register_t sgir, enum gic_sgi_mode irqmode, int virq, unsigned long vcpu_mask); +extern void vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int irq); #endif /* __ASM_ARM_VGIC_H__ */ /*