From patchwork Fri Jul 25 15:22:52 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 34290 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oi0-f72.google.com (mail-oi0-f72.google.com [209.85.218.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id D902C235AB for ; Fri, 25 Jul 2014 15:25:14 +0000 (UTC) Received: by mail-oi0-f72.google.com with SMTP id a141sf17581931oig.7 for ; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=oLSTPNpKI3cOXbhJpZ6B1fMmQaJwhquLUvRYeDAub6s=; b=SWq8v5gPjKyrjtJF5qCCvYxRLzcIMaSUg/0rFcs7l3uiOheaBFhJdH8KIXwsmTGWlI i0ZQOYs2DODx0Zo338PfEC6SwOQYzqxsLYICMDWpAt0HezwIMSjOGZr95Qzhfi6O7DRd ONE1cSrC6QL8Y2aXh3jjmJh/t9GA4idyZYsYUNHvfcnvsLBq2P4qQefD1e5/t5aYT60B VHSbLMGBGPIkFMyYakaAVNklHpS4BnEshYeE7qZQnFoiPAbSi20Y5kUpbI00K2fjE0FY /vgFqEMJ4VUUOY8iXAFUDxOsKSDvJyv5O0BfmPzLV84joIRT34Y/RQ5qvUEZg++s+SGi F39g== X-Gm-Message-State: ALoCoQkg0FGPeZFl4EiLzlhnKbdSCYCFaCZIE591HZUOLWcuhrUp/3CBTg3On3/Dsvb0BSGxXO/f X-Received: by 10.182.81.161 with SMTP id b1mr7665189oby.1.1406301914457; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.34.233 with SMTP id l96ls1241576qgl.42.gmail; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) X-Received: by 10.221.34.13 with SMTP id sq13mr22105962vcb.16.1406301914301; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx.google.com with ESMTPS id e4si7670352vdw.49.2014.07.25.08.25.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 25 Jul 2014 08:25:14 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) client-ip=209.85.220.181; Received: by mail-vc0-f181.google.com with SMTP id lf12so7708523vcb.12 for ; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) X-Received: by 10.52.129.200 with SMTP id ny8mr18204026vdb.27.1406301914186; Fri, 25 Jul 2014 08:25:14 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp46186vcb; Fri, 25 Jul 2014 08:25:13 -0700 (PDT) X-Received: by 10.51.17.104 with SMTP id gd8mr6422830igd.49.1406301912933; Fri, 25 Jul 2014 08:25:12 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id w4si3697238igr.4.2014.07.25.08.25.12 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 25 Jul 2014 08:25:12 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1XAhKm-0003g6-NZ; Fri, 25 Jul 2014 15:23:00 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1XAhKl-0003ft-99 for xen-devel@lists.xen.org; Fri, 25 Jul 2014 15:22:59 +0000 Received: from [85.158.143.35:17613] by server-3.bemta-4.messagelabs.com id 38/9D-06192-25672D35; Fri, 25 Jul 2014 15:22:58 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-13.tower-21.messagelabs.com!1406301775!12908397!2 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 9022 invoked from network); 25 Jul 2014 15:22:57 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-13.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 25 Jul 2014 15:22:57 -0000 X-IronPort-AV: E=Sophos;i="5.01,731,1400025600"; d="scan'208";a="156195396" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 25 Jul 2014 15:22:55 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Fri, 25 Jul 2014 11:22:55 -0400 Received: from drall.uk.xensource.com ([10.80.16.71]) by ukmail1.uk.xensource.com with smtp (Exim 4.69) (envelope-from ) id 1XAhKf-0007aW-Qb; Fri, 25 Jul 2014 16:22:54 +0100 Received: by drall.uk.xensource.com (sSMTP sendmail emulation); Fri, 25 Jul 2014 16:22:53 +0100 From: Ian Campbell To: Date: Fri, 25 Jul 2014 16:22:52 +0100 Message-ID: <2c06427f1180cf408a3e9750de3040dde0afe2ea.1406301772.git.ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <80a33cc325055bc9d63e4ef272c5b7f68f8fa812.1406301772.git.ian.campbell@citrix.com> References: <80a33cc325055bc9d63e4ef272c5b7f68f8fa812.1406301772.git.ian.campbell@citrix.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH 2/2] xen: arm: update arm32 assembly primitives to Linux v3.16-rc6 X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: bitops, cmpxchg, atomics: Import: c32ffce ARM: 7984/1: prefetch: add prefetchw invocations for barriered atomics Author: Will Deacon Signed-off-by: Will Deacon Signed-off-by: Russell King atomics: In addition to the above import: db38ee8 ARM: 7983/1: atomics: implement a better __atomic_add_unless for v6+ Author: Will Deacon Signed-off-by: Will Deacon Signed-off-by: Russell King spinlocks: We have diverged from Linux, so no updates but note this in the README. mem* and str*: Import: d98b90e ARM: 7990/1: asm: rename logical shift macros push pull into lspush lspull Author: Victor Kamensky Suggested-by: Will Deacon Signed-off-by: Victor Kamensky Acked-by: Nicolas Pitre Signed-off-by: Russell King For some reason str* were mentioned under mem* in the README, fix. libgcc: No changes, update baseline Signed-off-by: Ian Campbell Acked-by: Julien Grall --- xen/arch/arm/README.LinuxPrimitives | 17 +++++++-------- xen/arch/arm/arm32/lib/assembler.h | 8 +++---- xen/arch/arm/arm32/lib/bitops.h | 5 +++++ xen/arch/arm/arm32/lib/copy_template.S | 36 ++++++++++++++++---------------- xen/arch/arm/arm32/lib/memmove.S | 36 ++++++++++++++++---------------- xen/include/asm-arm/arm32/atomic.h | 32 ++++++++++++++++++++++++++++ xen/include/asm-arm/arm32/cmpxchg.h | 5 +++++ 7 files changed, 90 insertions(+), 49 deletions(-) diff --git a/xen/arch/arm/README.LinuxPrimitives b/xen/arch/arm/README.LinuxPrimitives index 69eeb70..7e15b04 100644 --- a/xen/arch/arm/README.LinuxPrimitives +++ b/xen/arch/arm/README.LinuxPrimitives @@ -65,7 +65,7 @@ linux/arch/arm64/lib/copy_page.S unused in Xen arm32 ===================================================================== -bitops: last sync @ v3.14-rc7 (last commit: b7ec699) +bitops: last sync @ v3.16-rc6 (last commit: c32ffce0f66e) linux/arch/arm/lib/bitops.h xen/arch/arm/arm32/lib/bitops.h linux/arch/arm/lib/changebit.S xen/arch/arm/arm32/lib/changebit.S @@ -83,13 +83,13 @@ done --------------------------------------------------------------------- -cmpxchg: last sync @ v3.14-rc7 (last commit: 775ebcc) +cmpxchg: last sync @ v3.16-rc6 (last commit: c32ffce0f66e) linux/arch/arm/include/asm/cmpxchg.h xen/include/asm-arm/arm32/cmpxchg.h --------------------------------------------------------------------- -atomics: last sync @ v3.14-rc7 (last commit: aed3a4e) +atomics: last sync @ v3.16-rc6 (last commit: 030d0178bdbd) linux/arch/arm/include/asm/atomic.h xen/include/asm-arm/arm32/atomic.h @@ -99,6 +99,8 @@ spinlocks: last sync: 15e7e5c1ebf5 linux/arch/arm/include/asm/spinlock.h xen/include/asm-arm/arm32/spinlock.h +*** Linux has switched to ticket locks but we still use bitlocks. + resync to v3.14-rc7: 7c8746a ARM: 7955/1: spinlock: ensure we have a compiler barrier before sev @@ -111,7 +113,7 @@ resync to v3.14-rc7: --------------------------------------------------------------------- -mem*: last sync @ v3.14-rc7 (last commit: 418df63a) +mem*: last sync @ v3.16-rc6 (last commit: d98b90ea22b0) linux/arch/arm/lib/copy_template.S xen/arch/arm/arm32/lib/copy_template.S linux/arch/arm/lib/memchr.S xen/arch/arm/arm32/lib/memchr.S @@ -120,9 +122,6 @@ linux/arch/arm/lib/memmove.S xen/arch/arm/arm32/lib/memmove.S linux/arch/arm/lib/memset.S xen/arch/arm/arm32/lib/memset.S linux/arch/arm/lib/memzero.S xen/arch/arm/arm32/lib/memzero.S -linux/arch/arm/lib/strchr.S xen/arch/arm/arm32/lib/strchr.S -linux/arch/arm/lib/strrchr.S xen/arch/arm/arm32/lib/strrchr.S - for i in copy_template.S memchr.S memcpy.S memmove.S memset.S \ memzero.S ; do diff -u linux/arch/arm/lib/$i xen/arch/arm/arm32/lib/$i @@ -130,7 +129,7 @@ done --------------------------------------------------------------------- -str*: last sync @ v3.13-rc7 (last commit: 93ed397) +str*: last sync @ v3.16-rc6 (last commit: d98b90ea22b0) linux/arch/arm/lib/strchr.S xen/arch/arm/arm32/lib/strchr.S linux/arch/arm/lib/strrchr.S xen/arch/arm/arm32/lib/strrchr.S @@ -145,7 +144,7 @@ clear_page == memset --------------------------------------------------------------------- -libgcc: last sync @ v3.14-rc7 (last commit: 01885bc) +libgcc: last sync @ v3.16-rc6 (last commit: 01885bc) linux/arch/arm/lib/lib1funcs.S xen/arch/arm/arm32/lib/lib1funcs.S linux/arch/arm/lib/lshrdi3.S xen/arch/arm/arm32/lib/lshrdi3.S diff --git a/xen/arch/arm/arm32/lib/assembler.h b/xen/arch/arm/arm32/lib/assembler.h index f8d4b3a..6de2638 100644 --- a/xen/arch/arm/arm32/lib/assembler.h +++ b/xen/arch/arm/arm32/lib/assembler.h @@ -36,8 +36,8 @@ * Endian independent macros for shifting bytes within registers. */ #ifndef __ARMEB__ -#define pull lsr -#define push lsl +#define lspull lsr +#define lspush lsl #define get_byte_0 lsl #0 #define get_byte_1 lsr #8 #define get_byte_2 lsr #16 @@ -47,8 +47,8 @@ #define put_byte_2 lsl #16 #define put_byte_3 lsl #24 #else -#define pull lsl -#define push lsr +#define lspull lsl +#define lspush lsr #define get_byte_0 lsr #24 #define get_byte_1 lsr #16 #define get_byte_2 lsr #8 diff --git a/xen/arch/arm/arm32/lib/bitops.h b/xen/arch/arm/arm32/lib/bitops.h index 25784c3..a167c2d 100644 --- a/xen/arch/arm/arm32/lib/bitops.h +++ b/xen/arch/arm/arm32/lib/bitops.h @@ -37,6 +37,11 @@ UNWIND( .fnstart ) add r1, r1, r0, lsl #2 @ Get word offset mov r3, r2, lsl r3 @ create mask smp_dmb +#if __LINUX_ARM_ARCH__ >= 7 && defined(CONFIG_SMP) + .arch_extension mp + ALT_SMP(W(pldw) [r1]) + ALT_UP(W(nop)) +#endif 1: ldrex r2, [r1] ands r0, r2, r3 @ save old value of bit \instr r2, r2, r3 @ toggle bit diff --git a/xen/arch/arm/arm32/lib/copy_template.S b/xen/arch/arm/arm32/lib/copy_template.S index 805e3f8..3bc8eb8 100644 --- a/xen/arch/arm/arm32/lib/copy_template.S +++ b/xen/arch/arm/arm32/lib/copy_template.S @@ -197,24 +197,24 @@ 12: PLD( pld [r1, #124] ) 13: ldr4w r1, r4, r5, r6, r7, abort=19f - mov r3, lr, pull #\pull + mov r3, lr, lspull #\pull subs r2, r2, #32 ldr4w r1, r8, r9, ip, lr, abort=19f - orr r3, r3, r4, push #\push - mov r4, r4, pull #\pull - orr r4, r4, r5, push #\push - mov r5, r5, pull #\pull - orr r5, r5, r6, push #\push - mov r6, r6, pull #\pull - orr r6, r6, r7, push #\push - mov r7, r7, pull #\pull - orr r7, r7, r8, push #\push - mov r8, r8, pull #\pull - orr r8, r8, r9, push #\push - mov r9, r9, pull #\pull - orr r9, r9, ip, push #\push - mov ip, ip, pull #\pull - orr ip, ip, lr, push #\push + orr r3, r3, r4, lspush #\push + mov r4, r4, lspull #\pull + orr r4, r4, r5, lspush #\push + mov r5, r5, lspull #\pull + orr r5, r5, r6, lspush #\push + mov r6, r6, lspull #\pull + orr r6, r6, r7, lspush #\push + mov r7, r7, lspull #\pull + orr r7, r7, r8, lspush #\push + mov r8, r8, lspull #\pull + orr r8, r8, r9, lspush #\push + mov r9, r9, lspull #\pull + orr r9, r9, ip, lspush #\push + mov ip, ip, lspull #\pull + orr ip, ip, lr, lspush #\push str8w r0, r3, r4, r5, r6, r7, r8, r9, ip, , abort=19f bge 12b PLD( cmn r2, #96 ) @@ -225,10 +225,10 @@ 14: ands ip, r2, #28 beq 16f -15: mov r3, lr, pull #\pull +15: mov r3, lr, lspull #\pull ldr1w r1, lr, abort=21f subs ip, ip, #4 - orr r3, r3, lr, push #\push + orr r3, r3, lr, lspush #\push str1w r0, r3, abort=21f bgt 15b CALGN( cmp r2, #0 ) diff --git a/xen/arch/arm/arm32/lib/memmove.S b/xen/arch/arm/arm32/lib/memmove.S index 4e142b8..18634c3 100644 --- a/xen/arch/arm/arm32/lib/memmove.S +++ b/xen/arch/arm/arm32/lib/memmove.S @@ -148,24 +148,24 @@ ENTRY(memmove) 12: PLD( pld [r1, #-128] ) 13: ldmdb r1!, {r7, r8, r9, ip} - mov lr, r3, push #\push + mov lr, r3, lspush #\push subs r2, r2, #32 ldmdb r1!, {r3, r4, r5, r6} - orr lr, lr, ip, pull #\pull - mov ip, ip, push #\push - orr ip, ip, r9, pull #\pull - mov r9, r9, push #\push - orr r9, r9, r8, pull #\pull - mov r8, r8, push #\push - orr r8, r8, r7, pull #\pull - mov r7, r7, push #\push - orr r7, r7, r6, pull #\pull - mov r6, r6, push #\push - orr r6, r6, r5, pull #\pull - mov r5, r5, push #\push - orr r5, r5, r4, pull #\pull - mov r4, r4, push #\push - orr r4, r4, r3, pull #\pull + orr lr, lr, ip, lspull #\pull + mov ip, ip, lspush #\push + orr ip, ip, r9, lspull #\pull + mov r9, r9, lspush #\push + orr r9, r9, r8, lspull #\pull + mov r8, r8, lspush #\push + orr r8, r8, r7, lspull #\pull + mov r7, r7, lspush #\push + orr r7, r7, r6, lspull #\pull + mov r6, r6, lspush #\push + orr r6, r6, r5, lspull #\pull + mov r5, r5, lspush #\push + orr r5, r5, r4, lspull #\pull + mov r4, r4, lspush #\push + orr r4, r4, r3, lspull #\pull stmdb r0!, {r4 - r9, ip, lr} bge 12b PLD( cmn r2, #96 ) @@ -176,10 +176,10 @@ ENTRY(memmove) 14: ands ip, r2, #28 beq 16f -15: mov lr, r3, push #\push +15: mov lr, r3, lspush #\push ldr r3, [r1, #-4]! subs ip, ip, #4 - orr lr, lr, r3, pull #\pull + orr lr, lr, r3, lspull #\pull str lr, [r0, #-4]! bgt 15b CALGN( cmp r2, #0 ) diff --git a/xen/include/asm-arm/arm32/atomic.h b/xen/include/asm-arm/arm32/atomic.h index 3d601d1..7ec712f 100644 --- a/xen/include/asm-arm/arm32/atomic.h +++ b/xen/include/asm-arm/arm32/atomic.h @@ -39,6 +39,7 @@ static inline int atomic_add_return(int i, atomic_t *v) int result; smp_mb(); + prefetchw(&v->counter); __asm__ __volatile__("@ atomic_add_return\n" "1: ldrex %0, [%3]\n" @@ -78,6 +79,7 @@ static inline int atomic_sub_return(int i, atomic_t *v) int result; smp_mb(); + prefetchw(&v->counter); __asm__ __volatile__("@ atomic_sub_return\n" "1: ldrex %0, [%3]\n" @@ -100,6 +102,7 @@ static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) unsigned long res; smp_mb(); + prefetchw(&ptr->counter); do { __asm__ __volatile__("@ atomic_cmpxchg\n" @@ -117,6 +120,35 @@ static inline int atomic_cmpxchg(atomic_t *ptr, int old, int new) return oldval; } +static inline int __atomic_add_unless(atomic_t *v, int a, int u) +{ + int oldval, newval; + unsigned long tmp; + + smp_mb(); + prefetchw(&v->counter); + + __asm__ __volatile__ ("@ atomic_add_unless\n" +"1: ldrex %0, [%4]\n" +" teq %0, %5\n" +" beq 2f\n" +" add %1, %0, %6\n" +" strex %2, %1, [%4]\n" +" teq %2, #0\n" +" bne 1b\n" +"2:" + : "=&r" (oldval), "=&r" (newval), "=&r" (tmp), "+Qo" (v->counter) + : "r" (&v->counter), "r" (u), "r" (a) + : "cc"); + + if (oldval != u) + smp_mb(); + + return oldval; +} + +#define atomic_xchg(v, new) (xchg(&((v)->counter), new)) + #define atomic_inc(v) atomic_add(1, v) #define atomic_dec(v) atomic_sub(1, v) diff --git a/xen/include/asm-arm/arm32/cmpxchg.h b/xen/include/asm-arm/arm32/cmpxchg.h index 9a511f2..03e0bed 100644 --- a/xen/include/asm-arm/arm32/cmpxchg.h +++ b/xen/include/asm-arm/arm32/cmpxchg.h @@ -1,6 +1,8 @@ #ifndef __ASM_ARM32_CMPXCHG_H #define __ASM_ARM32_CMPXCHG_H +#include + extern void __bad_xchg(volatile void *, int); static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size) @@ -9,6 +11,7 @@ static inline unsigned long __xchg(unsigned long x, volatile void *ptr, int size unsigned int tmp; smp_mb(); + prefetchw((const void *)ptr); switch (size) { case 1: @@ -56,6 +59,8 @@ static always_inline unsigned long __cmpxchg( { unsigned long oldval, res; + prefetchw((const void *)ptr); + switch (size) { case 1: do {