From patchwork Wed Jul 16 10:32:15 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 33724 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pd0-f198.google.com (mail-pd0-f198.google.com [209.85.192.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 30675201F1 for ; Wed, 16 Jul 2014 10:34:29 +0000 (UTC) Received: by mail-pd0-f198.google.com with SMTP id fp1sf4824836pdb.9 for ; Wed, 16 Jul 2014 03:34:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:cc:subject :precedence:list-id:list-unsubscribe:list-post:list-help :list-subscribe:mime-version:sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list:list-archive :content-type:content-transfer-encoding; bh=IebK/HnhnZtYkjnP33K85bNKPEm0+T93p0dhq5NYVb0=; b=SzzhqCN8a+/TcSUg2afMjfFK56/mIE1j5XiP3cUyZs1bms4U87XvU3L10vKduMK4T4 q29P61ehsym1HE7KJviGp/yz2eJgB1PsbPt3qMrvzwQzJ9kR/TLPjfLcFDySPxgr/8RS vrlMwXscDf4avTzP/jQJQH0Fcz4rKumf8n5dl4dNYWQWQNhqMr1C35jnBSFez3Ovt+4G R5naJWaIsnRR1EpXreDN7BeUgc8yLcpHhZjcfJz27txYd96ECbZnWN6Mt5w/N3vOYfh/ hm7F5D5t2WrUk4Bx2mISf/Ti876e0eRDdNhnIMNKdg1ChaANRXxyNh6oBLIFTk7Bk34s jWbw== X-Gm-Message-State: ALoCoQmRaPPN/l7g9T52bN13B5/OyVLscX5map/o0duY6du+kJXiTSPlWkFRIKVZMcBIAutyCrBB X-Received: by 10.70.53.202 with SMTP id d10mr14131409pdp.2.1405506868086; Wed, 16 Jul 2014 03:34:28 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.22.198 with SMTP id 64ls346261qgn.8.gmail; Wed, 16 Jul 2014 03:34:28 -0700 (PDT) X-Received: by 10.221.47.9 with SMTP id uq9mr6353709vcb.48.1405506867985; Wed, 16 Jul 2014 03:34:27 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id q19si8113655vea.10.2014.07.16.03.34.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Jul 2014 03:34:27 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id la4so1237426vcb.9 for ; Wed, 16 Jul 2014 03:34:27 -0700 (PDT) X-Received: by 10.52.120.38 with SMTP id kz6mr415391vdb.86.1405506867894; Wed, 16 Jul 2014 03:34:27 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp286144vcb; Wed, 16 Jul 2014 03:34:27 -0700 (PDT) X-Received: by 10.140.48.75 with SMTP id n69mr43067204qga.107.1405506867474; Wed, 16 Jul 2014 03:34:27 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id e7si19890720qae.30.2014.07.16.03.34.26 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 16 Jul 2014 03:34:27 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X7MVl-000060-1i; Wed, 16 Jul 2014 10:32:33 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X7MVj-00005E-Po for xen-devel@lists.xen.org; Wed, 16 Jul 2014 10:32:31 +0000 Received: from [85.158.143.35:55765] by server-3.bemta-4.messagelabs.com id D1/D2-09960-FB456C35; Wed, 16 Jul 2014 10:32:31 +0000 X-Env-Sender: anup.patel@linaro.org X-Msg-Ref: server-9.tower-21.messagelabs.com!1405506748!18101719!1 X-Originating-IP: [209.85.192.182] X-SpamReason: No, hits=0.0 required=7.0 tests=ML_RADAR_SPEW_LINKS_32, spamassassin: X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15314 invoked from network); 16 Jul 2014 10:32:30 -0000 Received: from mail-pd0-f182.google.com (HELO mail-pd0-f182.google.com) (209.85.192.182) by server-9.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 16 Jul 2014 10:32:30 -0000 Received: by mail-pd0-f182.google.com with SMTP id fp1so1025918pdb.13 for ; Wed, 16 Jul 2014 03:32:28 -0700 (PDT) X-Received: by 10.66.148.196 with SMTP id tu4mr9619764pab.122.1405506748345; Wed, 16 Jul 2014 03:32:28 -0700 (PDT) Received: from pnqlab006.amcc.com ([182.73.239.130]) by mx.google.com with ESMTPSA id r1sm22157724pdo.51.2014.07.16.03.32.24 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Jul 2014 03:32:27 -0700 (PDT) From: Anup Patel To: xen-devel@lists.xen.org Date: Wed, 16 Jul 2014 16:02:15 +0530 Message-Id: <1405506735-19025-1-git-send-email-anup.patel@linaro.org> X-Mailer: git-send-email 1.7.9.5 Cc: Ian.Campbell@citrix.com, Anup Patel , stefano.stabellini@eu.citrix.com, patches@apm.com, stefano.stabellini@citrix.com, Pranavkumar Sawargaonkar Subject: [Xen-devel] [PATCH v2] xen/arm: Trap and yield on WFE instructions X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: anup.patel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: If we have a Guest/DomU with two or more of its VCPUs running on same host CPU then it can quite likely happen that these VCPUs fight for same spinlock and one of them will waste CPU cycles in WFE instruction. This patch makes WFE instruction trap for VCPU and forces VCPU to yield its timeslice. The KVM ARM/ARM64 also does similar thing for handling WFE instructions. (Please refer, https://lists.cs.columbia.edu/pipermail/kvmarm/2013-November/006259.html) In general, this patch is more of an optimization for an oversubscribed system having number of VCPUs more than underlying host CPUs. Changes since V1: - Added separate member in union hsr for decoding WFI/WFE related info. Signed-off-by: Anup Patel Signed-off-by: Pranavkumar Sawargaonkar Tested-by: Pranavkumar Sawargaonkar --- xen/arch/arm/traps.c | 27 ++++++++++++++++----------- xen/include/asm-arm/processor.h | 9 +++++++++ 2 files changed, 25 insertions(+), 11 deletions(-) diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 686d8b7..632b8ea 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -90,7 +90,7 @@ void __cpuinit init_traps(void) /* Setup hypervisor traps */ WRITE_SYSREG(HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM| - HCR_TWI|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP, HCR_EL2); + HCR_TWE|HCR_TWI|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP, HCR_EL2); isb(); } @@ -1803,16 +1803,21 @@ asmlinkage void do_trap_hypervisor(struct cpu_user_regs *regs) advance_pc(regs, hsr); return; } - /* at the moment we only trap WFI */ - vcpu_block(); - /* The ARM spec declares that even if local irqs are masked in - * the CPSR register, an irq should wake up a cpu from WFI anyway. - * For this reason we need to check for irqs that need delivery, - * ignoring the CPSR register, *after* calling SCHEDOP_block to - * avoid races with vgic_vcpu_inject_irq. - */ - if ( local_events_need_delivery_nomask() ) - vcpu_unblock(current); + if ( hsr.wfi_wfe.ti ) { + /* Yield the VCPU for WFE */ + vcpu_force_reschedule(current); + } else { + /* Block the VCPU for WFI */ + vcpu_block(); + /* The ARM spec declares that even if local irqs are masked in + * the CPSR register, an irq should wake up a cpu from WFI anyway. + * For this reason we need to check for irqs that need delivery, + * ignoring the CPSR register, *after* calling SCHEDOP_block to + * avoid races with vgic_vcpu_inject_irq. + */ + if ( local_events_need_delivery_nomask() ) + vcpu_unblock(current); + } advance_pc(regs, hsr); break; case HSR_EC_CP15_32: diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h index bdfff4e..cd1db4d 100644 --- a/xen/include/asm-arm/processor.h +++ b/xen/include/asm-arm/processor.h @@ -276,6 +276,15 @@ union hsr { unsigned long ec:6; /* Exception Class */ } cond; + struct hsr_wfi_wfe { + unsigned long ti:1; /* Trapped instruction */ + unsigned long sbzp:19; + unsigned long cc:4; /* Condition Code */ + unsigned long ccvalid:1;/* CC Valid */ + unsigned long len:1; /* Instruction length */ + unsigned long ec:6; /* Exception Class */ + } wfi_wfe; + /* reg, reg0, reg1 are 4 bits on AArch32, the fifth bit is sbzp. */ struct hsr_cp32 { unsigned long read:1; /* Direction */