From patchwork Mon Mar 17 11:31:02 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 26345 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f70.google.com (mail-qa0-f70.google.com [209.85.216.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 05322202FA for ; Mon, 17 Mar 2014 11:32:48 +0000 (UTC) Received: by mail-qa0-f70.google.com with SMTP id hw13sf11926484qab.1 for ; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id :mime-version:cc:subject:precedence:list-id:list-unsubscribe :list-post:list-help:list-subscribe:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :list-archive:content-type:content-transfer-encoding; bh=wyMSma/jg0YCG/a8IhFsj9YFVkcnsImf5622P5QI1ek=; b=PVkJ7EChVvPXOo/3+rObAQV9etMjL8BsE0aT+16l8PqO8ka1sVLFx6kTOQqspPL789 maVcTkKEuqKaab2TDqu+1nNGDJFb98dkCqpDQHO5sjNiAc3LMbBsx6dOC50DbzIsHVw6 q5jIAg13BZMYXHKklnme0IBMzGEnrdTkCMhyMHuv8ZCKzQguRtMMLGJP0Gh3RpemAKnI WcNUCRe6lzAHJhZChsuLuqWTSfxvdcyCDry5DiRYNWyoztp/42up5GO7ytay6ldbPlWA IJbkQZYBt3RAh7xt1CPAQAYgyaYFwQpi1qYxicFwb5k8s1WPoidPfuZRpArXF0UaGGdG 44XQ== X-Gm-Message-State: ALoCoQn6VphVk9o1NJFixVnCrHCplFfDOfSANVOieGgyPAf/pn2h4MZmboYTKv6r86s1xqkff4sF X-Received: by 10.58.39.202 with SMTP id r10mr9585934vek.12.1395055968490; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.48.38 with SMTP id n35ls1625053qga.68.gmail; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) X-Received: by 10.220.164.80 with SMTP id d16mr19068325vcy.15.1395055968404; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) Received: from mail-ve0-f180.google.com (mail-ve0-f180.google.com [209.85.128.180]) by mx.google.com with ESMTPS id b2si5141957vcy.85.2014.03.17.04.32.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Mar 2014 04:32:48 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.180 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.180; Received: by mail-ve0-f180.google.com with SMTP id jz11so5405858veb.39 for ; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) X-Received: by 10.52.30.230 with SMTP id v6mr16392932vdh.6.1395055968294; Mon, 17 Mar 2014 04:32:48 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp119973vck; Mon, 17 Mar 2014 04:32:47 -0700 (PDT) X-Received: by 10.50.137.100 with SMTP id qh4mr12608812igb.4.1395055967648; Mon, 17 Mar 2014 04:32:47 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id au7si14203295igc.48.2014.03.17.04.32.47 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 17 Mar 2014 04:32:47 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WPVl6-0007uT-By; Mon, 17 Mar 2014 11:31:08 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WPVl4-0007uK-Nu for xen-devel@lists.xen.org; Mon, 17 Mar 2014 11:31:07 +0000 Received: from [193.109.254.147:21008] by server-3.bemta-14.messagelabs.com id 2C/47-22179-9FCD6235; Mon, 17 Mar 2014 11:31:05 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-3.tower-27.messagelabs.com!1395055864!2163998!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 12561 invoked from network); 17 Mar 2014 11:31:05 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-3.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 17 Mar 2014 11:31:05 -0000 X-IronPort-AV: E=Sophos;i="4.97,669,1389744000"; d="scan'208";a="112073056" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 17 Mar 2014 11:31:05 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Mon, 17 Mar 2014 07:31:02 -0400 Received: from drall.uk.xensource.com ([10.80.16.71] helo=drall.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WPVl0-0008S4-Ve; Mon, 17 Mar 2014 11:31:03 +0000 From: Ian Campbell To: Date: Mon, 17 Mar 2014 11:31:02 +0000 Message-ID: <1395055862-23455-1-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 MIME-Version: 1.0 X-DLP: MIA2 Cc: Oleksandr Tyshchenko , julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH v2] xen: arm: increase priority of SGIs used as IPIs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.180 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Code such as on_selected_cpus expects/requires that an IPI can preempt a processor which is just handling a normal interrupt. Lacking this property can result in a deadlock between two CPUs trying to IPI each other from interrupt context. For the time being there is only two priorities, IRQ and IPI, although it is also conceivable that in the future some IPIs might be higher priority than others. This could be used to implement a better BUG() than we have now, but I haven't tackled that yet. Tested with a debug patch which sends a local IPI from a keyhandler, which is run in serial interrupt context. This should also fix the issue reported by Oleksandr in "xen/arm: maintenance_interrupt SMP fix" without resorting to trylock. Signed-off-by: Ian Campbell Acked-by: Julien Grall Cc: Oleksandr Tyshchenko --- v2: Rebased onto current staging. Split comment in two to clarify. Since this already has Julien's ack I intend to commit quote soon. --- xen/arch/arm/gic.c | 22 +++++++++++++++------- xen/arch/arm/time.c | 6 +++--- xen/include/asm-arm/gic.h | 22 ++++++++++++++++++++++ 3 files changed, 40 insertions(+), 10 deletions(-) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index f08b3b4..0513138 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -322,7 +322,8 @@ static void __init gic_dist_init(void) /* Default priority for global interrupts */ for ( i = 32; i < gic.lines; i += 4 ) - GICD[GICD_IPRIORITYR + i / 4] = 0xa0a0a0a0; + GICD[GICD_IPRIORITYR + i / 4] = + GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ; /* Disable all global interrupts */ for ( i = 32; i < gic.lines; i += 32 ) @@ -343,9 +344,14 @@ static void __cpuinit gic_cpu_init(void) * be set up here with the other per-cpu state. */ GICD[GICD_ICENABLER] = 0xffff0000; /* Disable all PPI */ GICD[GICD_ISENABLER] = 0x0000ffff; /* Enable all SGI */ - /* Set PPI and SGI priorities */ - for (i = 0; i < 32; i += 4) - GICD[GICD_IPRIORITYR + i / 4] = 0xa0a0a0a0; + /* Set SGI priorities */ + for (i = 0; i < 16; i += 4) + GICD[GICD_IPRIORITYR + i / 4] = + GIC_PRI_IPI<<24 | GIC_PRI_IPI<<16 | GIC_PRI_IPI<<8 | GIC_PRI_IPI; + /* Set PPI priorities */ + for (i = 16; i < 32; i += 4) + GICD[GICD_IPRIORITYR + i / 4] = + GIC_PRI_IRQ<<24 | GIC_PRI_IRQ<<16 | GIC_PRI_IRQ<<8 | GIC_PRI_IRQ; /* Local settings: interface controller */ GICC[GICC_PMR] = 0xff; /* Don't mask by priority */ @@ -541,7 +547,8 @@ void gic_disable_cpu(void) void gic_route_ppis(void) { /* GIC maintenance */ - gic_route_dt_irq(&gic.maintenance, cpumask_of(smp_processor_id()), 0xa0); + gic_route_dt_irq(&gic.maintenance, cpumask_of(smp_processor_id()), + GIC_PRI_IRQ); /* Route timer interrupt */ route_timer_interrupt(); } @@ -556,7 +563,8 @@ void gic_route_spis(void) if ( (irq = serial_dt_irq(seridx)) == NULL ) continue; - gic_route_dt_irq(irq, cpumask_of(smp_processor_id()), 0xa0); + gic_route_dt_irq(irq, cpumask_of(smp_processor_id()), + GIC_PRI_IRQ); } } @@ -779,7 +787,7 @@ int gic_route_irq_to_guest(struct domain *d, const struct dt_irq *irq, level = dt_irq_is_level_triggered(irq); gic_set_irq_properties(irq->irq, level, cpumask_of(smp_processor_id()), - 0xa0); + GIC_PRI_IRQ); retval = __setup_irq(desc, irq->irq, action); if (retval) { diff --git a/xen/arch/arm/time.c b/xen/arch/arm/time.c index 22e94bb..ba281e9 100644 --- a/xen/arch/arm/time.c +++ b/xen/arch/arm/time.c @@ -222,11 +222,11 @@ static void vtimer_interrupt(int irq, void *dev_id, struct cpu_user_regs *regs) void __cpuinit route_timer_interrupt(void) { gic_route_dt_irq(&timer_irq[TIMER_PHYS_NONSECURE_PPI], - cpumask_of(smp_processor_id()), 0xa0); + cpumask_of(smp_processor_id()), GIC_PRI_IRQ); gic_route_dt_irq(&timer_irq[TIMER_HYP_PPI], - cpumask_of(smp_processor_id()), 0xa0); + cpumask_of(smp_processor_id()), GIC_PRI_IRQ); gic_route_dt_irq(&timer_irq[TIMER_VIRT_PPI], - cpumask_of(smp_processor_id()), 0xa0); + cpumask_of(smp_processor_id()), GIC_PRI_IRQ); } /* Set up the timer interrupt on this CPU */ diff --git a/xen/include/asm-arm/gic.h b/xen/include/asm-arm/gic.h index 9c6f9bb..25b2b24 100644 --- a/xen/include/asm-arm/gic.h +++ b/xen/include/asm-arm/gic.h @@ -129,6 +129,28 @@ #define GICH_LR_CPUID_SHIFT 9 #define GICH_VTR_NRLRGS 0x3f +/* + * The minimum GICC_BPR is required to be in the range 0-3. We set + * GICC_BPR to 0 but we must expect that it might be 3. This means we + * can rely on premption between the following ranges: + * 0xf0..0xff + * 0xe0..0xdf + * 0xc0..0xcf + * 0xb0..0xbf + * 0xa0..0xaf + * 0x90..0x9f + * 0x80..0x8f + * + * Priorities within a range will not preempt each other. + * + * A GIC must support a mimimum of 16 priority levels. + */ +#define GIC_PRI_LOWEST 0xf0 +#define GIC_PRI_IRQ 0xa0 +#define GIC_PRI_IPI 0x90 /* IPIs must preempt normal interrupts */ +#define GIC_PRI_HIGHEST 0x80 /* Higher priorities belong to Secure-World */ + + #ifndef __ASSEMBLY__ #include