From patchwork Thu Mar 20 15:46:02 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 26726 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f200.google.com (mail-ie0-f200.google.com [209.85.223.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 508E720143 for ; Thu, 20 Mar 2014 15:49:44 +0000 (UTC) Received: by mail-ie0-f200.google.com with SMTP id lx4sf3572628iec.11 for ; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=MnoKLiOILXIBUPme7TlUd06g8biT+NK/BdEaiM37zZ0=; b=PR/sSvmdHZy4ZJtgvZi45b5vEQ/q2ZqvyV8IMoCMXIot2EvR8jYF69VydDCS4KqKj5 FFLXzEH7YWCYGAeTOs2NVheCHeNxde+WjYaYBDp4Hp3wQwWU/BR/X8N/VlDEfnxH4Jvm zsdrF+yNbYROXkjCJXFxxxpSztmW00fMhpVyUpn31ZtDoH9GBHEcv0aDFUVj50lgEKAL RfD0g67VRO2t5ldyGhK80hdH6zqkd7DnOi+k0ARoWv+iBJ4IJZ2YUoYP3PKLjkQ7zBFN 8Zc+ZeckVRrF9IT18gDJAgs2AXixxZ/xxj3d6OWsTusnHDbeR+zLSpsIs10mw9nb2nZ7 T1HA== X-Gm-Message-State: ALoCoQkBqEqy6eXyAv5Pt1y3bnAw8hiJniA3nzyQKmB4Iaf+jjp1N2td78FpDf7ZW4na2QE99yrW X-Received: by 10.182.79.194 with SMTP id l2mr2385669obx.43.1395330583744; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.27.138 with SMTP id 10ls292943qgx.52.gmail; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-Received: by 10.52.142.10 with SMTP id rs10mr29089203vdb.3.1395330583646; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) by mx.google.com with ESMTPS id d20si513280veu.97.2014.03.20.08.49.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 20 Mar 2014 08:49:43 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.175; Received: by mail-vc0-f175.google.com with SMTP id lh14so1170986vcb.20 for ; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-Received: by 10.220.139.198 with SMTP id f6mr894420vcu.47.1395330583557; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp398689vck; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) X-Received: by 10.140.87.151 with SMTP id r23mr43027759qgd.23.1395330583014; Thu, 20 Mar 2014 08:49:43 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id gq5si927068qab.233.2014.03.20.08.49.42 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 20 Mar 2014 08:49:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WQfCL-00059F-6k; Thu, 20 Mar 2014 15:48:01 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WQfCI-00056e-WB for xen-devel@lists.xen.org; Thu, 20 Mar 2014 15:47:59 +0000 Received: from [85.158.137.68:14612] by server-16.bemta-3.messagelabs.com id 83/69-13481-EAD0B235; Thu, 20 Mar 2014 15:47:58 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-8.tower-31.messagelabs.com!1395330472!1945661!2 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 13410 invoked from network); 20 Mar 2014 15:47:57 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-8.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 20 Mar 2014 15:47:57 -0000 X-IronPort-AV: E=Sophos;i="4.97,695,1389744000"; d="scan'208";a="113309376" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 20 Mar 2014 15:47:56 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Thu, 20 Mar 2014 11:47:56 -0400 Received: from drall.uk.xensource.com ([10.80.16.71] helo=drall.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WQfAU-0006O2-V1; Thu, 20 Mar 2014 15:46:06 +0000 From: Ian Campbell To: Date: Thu, 20 Mar 2014 15:46:02 +0000 Message-ID: <1395330365-9901-14-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1395330336.3104.12.camel@kazak.uk.xensource.com> References: <1395330336.3104.12.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA2 Cc: julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH 14/17] xen: arm64: assembly optimised mem* and str* X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Taken from Linux v3.14-rc7. Signed-off-by: Ian Campbell Acked-by: Julien Grall --- xen/arch/arm/arm64/lib/Makefile | 2 ++ xen/arch/arm/arm64/lib/memchr.S | 43 +++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memcpy.S | 52 +++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memmove.S | 56 ++++++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memset.S | 52 +++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/strchr.S | 41 ++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/strrchr.S | 42 ++++++++++++++++++++++++++++ xen/include/asm-arm/string.h | 4 +-- 8 files changed, 290 insertions(+), 2 deletions(-) create mode 100644 xen/arch/arm/arm64/lib/memchr.S create mode 100644 xen/arch/arm/arm64/lib/memcpy.S create mode 100644 xen/arch/arm/arm64/lib/memmove.S create mode 100644 xen/arch/arm/arm64/lib/memset.S create mode 100644 xen/arch/arm/arm64/lib/strchr.S create mode 100644 xen/arch/arm/arm64/lib/strrchr.S diff --git a/xen/arch/arm/arm64/lib/Makefile b/xen/arch/arm/arm64/lib/Makefile index 32c02c4..9f3b236 100644 --- a/xen/arch/arm/arm64/lib/Makefile +++ b/xen/arch/arm/arm64/lib/Makefile @@ -1 +1,3 @@ +obj-y += memcpy.o memmove.o memset.o memchr.o obj-y += bitops.o find_next_bit.o +obj-y += strchr.o strrchr.o diff --git a/xen/arch/arm/arm64/lib/memchr.S b/xen/arch/arm/arm64/lib/memchr.S new file mode 100644 index 0000000..3cc1b01 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memchr.S @@ -0,0 +1,43 @@ +/* + * Based on arch/arm/lib/memchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find a character in an area of memory. + * + * Parameters: + * x0 - buf + * x1 - c + * x2 - n + * Returns: + * x0 - address of first occurrence of 'c' or 0 + */ +ENTRY(memchr) + and w1, w1, #0xff +1: subs x2, x2, #1 + b.mi 2f + ldrb w3, [x0], #1 + cmp w3, w1 + b.ne 1b + sub x0, x0, #1 + ret +2: mov x0, #0 + ret +ENDPROC(memchr) diff --git a/xen/arch/arm/arm64/lib/memcpy.S b/xen/arch/arm/arm64/lib/memcpy.S new file mode 100644 index 0000000..c8197c6 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memcpy.S @@ -0,0 +1,52 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Copy a buffer from src to dest (alignment handled by the hardware) + * + * Parameters: + * x0 - dest + * x1 - src + * x2 - n + * Returns: + * x0 - dest + */ +ENTRY(memcpy) + mov x4, x0 + subs x2, x2, #8 + b.mi 2f +1: ldr x3, [x1], #8 + subs x2, x2, #8 + str x3, [x4], #8 + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + ldr w3, [x1], #4 + sub x2, x2, #4 + str w3, [x4], #4 +3: adds x2, x2, #2 + b.mi 4f + ldrh w3, [x1], #2 + sub x2, x2, #2 + strh w3, [x4], #2 +4: adds x2, x2, #1 + b.mi 5f + ldrb w3, [x1] + strb w3, [x4] +5: ret +ENDPROC(memcpy) diff --git a/xen/arch/arm/arm64/lib/memmove.S b/xen/arch/arm/arm64/lib/memmove.S new file mode 100644 index 0000000..1bf0936 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memmove.S @@ -0,0 +1,56 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Move a buffer from src to test (alignment handled by the hardware). + * If dest <= src, call memcpy, otherwise copy in reverse order. + * + * Parameters: + * x0 - dest + * x1 - src + * x2 - n + * Returns: + * x0 - dest + */ +ENTRY(memmove) + cmp x0, x1 + b.ls memcpy + add x4, x0, x2 + add x1, x1, x2 + subs x2, x2, #8 + b.mi 2f +1: ldr x3, [x1, #-8]! + subs x2, x2, #8 + str x3, [x4, #-8]! + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + ldr w3, [x1, #-4]! + sub x2, x2, #4 + str w3, [x4, #-4]! +3: adds x2, x2, #2 + b.mi 4f + ldrh w3, [x1, #-2]! + sub x2, x2, #2 + strh w3, [x4, #-2]! +4: adds x2, x2, #1 + b.mi 5f + ldrb w3, [x1, #-1] + strb w3, [x4, #-1] +5: ret +ENDPROC(memmove) diff --git a/xen/arch/arm/arm64/lib/memset.S b/xen/arch/arm/arm64/lib/memset.S new file mode 100644 index 0000000..25a4fb6 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memset.S @@ -0,0 +1,52 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Fill in the buffer with character c (alignment handled by the hardware) + * + * Parameters: + * x0 - buf + * x1 - c + * x2 - n + * Returns: + * x0 - buf + */ +ENTRY(memset) + mov x4, x0 + and w1, w1, #0xff + orr w1, w1, w1, lsl #8 + orr w1, w1, w1, lsl #16 + orr x1, x1, x1, lsl #32 + subs x2, x2, #8 + b.mi 2f +1: str x1, [x4], #8 + subs x2, x2, #8 + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + sub x2, x2, #4 + str w1, [x4], #4 +3: adds x2, x2, #2 + b.mi 4f + sub x2, x2, #2 + strh w1, [x4], #2 +4: adds x2, x2, #1 + b.mi 5f + strb w1, [x4] +5: ret +ENDPROC(memset) diff --git a/xen/arch/arm/arm64/lib/strchr.S b/xen/arch/arm/arm64/lib/strchr.S new file mode 100644 index 0000000..9e265e4 --- /dev/null +++ b/xen/arch/arm/arm64/lib/strchr.S @@ -0,0 +1,41 @@ +/* + * Based on arch/arm/lib/strchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find the first occurrence of a character in a string. + * + * Parameters: + * x0 - str + * x1 - c + * Returns: + * x0 - address of first occurrence of 'c' or 0 + */ +ENTRY(strchr) + and w1, w1, #0xff +1: ldrb w2, [x0], #1 + cmp w2, w1 + ccmp w2, wzr, #4, ne + b.ne 1b + sub x0, x0, #1 + cmp w2, w1 + csel x0, x0, xzr, eq + ret +ENDPROC(strchr) diff --git a/xen/arch/arm/arm64/lib/strrchr.S b/xen/arch/arm/arm64/lib/strrchr.S new file mode 100644 index 0000000..3791754 --- /dev/null +++ b/xen/arch/arm/arm64/lib/strrchr.S @@ -0,0 +1,42 @@ +/* + * Based on arch/arm/lib/strrchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find the last occurrence of a character in a string. + * + * Parameters: + * x0 - str + * x1 - c + * Returns: + * x0 - address of last occurrence of 'c' or 0 + */ +ENTRY(strrchr) + mov x3, #0 + and w1, w1, #0xff +1: ldrb w2, [x0], #1 + cbz w2, 2f + cmp w2, w1 + b.ne 1b + sub x3, x0, #1 + b 1b +2: mov x0, x3 + ret +ENDPROC(strrchr) diff --git a/xen/include/asm-arm/string.h b/xen/include/asm-arm/string.h index 7d8b35a..3242762 100644 --- a/xen/include/asm-arm/string.h +++ b/xen/include/asm-arm/string.h @@ -3,8 +3,6 @@ #include -#if defined(CONFIG_ARM_32) - /* * We don't do inline string functions, since the * optimised inline asm versions are not small. @@ -29,6 +27,8 @@ extern void * memset(void *, int, __kernel_size_t); #define __HAVE_ARCH_MEMCHR extern void * memchr(const void *, int, __kernel_size_t); +#if defined(CONFIG_ARM_32) + extern void __memzero(void *ptr, __kernel_size_t n); #define memset(p,v,n) \