From patchwork Wed Jan 13 07:08:19 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Pinski X-Patchwork-Id: 59652 Delivered-To: patch@linaro.org Received: by 10.112.130.2 with SMTP id oa2csp3187823lbb; Tue, 12 Jan 2016 23:09:53 -0800 (PST) X-Received: by 10.98.42.81 with SMTP id q78mr40160822pfq.142.1452668993525; Tue, 12 Jan 2016 23:09:53 -0800 (PST) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id tp10si32474073pac.173.2016.01.12.23.09.53; Tue, 12 Jan 2016 23:09:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dkim=pass header.i=@cavium-com.20150623.gappssmtp.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755397AbcAMHJw (ORCPT + 29 others); Wed, 13 Jan 2016 02:09:52 -0500 Received: from mail-io0-f176.google.com ([209.85.223.176]:33729 "EHLO mail-io0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755171AbcAMHI0 (ORCPT ); Wed, 13 Jan 2016 02:08:26 -0500 Received: by mail-io0-f176.google.com with SMTP id q21so409281558iod.0 for ; Tue, 12 Jan 2016 23:08:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cavium-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3X1StATHosWwy8blsyGIZshMmSZpXUh1BfLs9iG/7tc=; b=aWUQDokLvwRvGKEBt0vZSQ07V2sTKyDtTWllRfIDAMbkLdKWkLbATAy7rZeki4QlLR uTzwNsIXqDSnJCyA4s7fCTn07C7P6mKYQPfiVVUMxz/fLeGZswTu5/FD+qLiwlOlF5nx Q1bNx179mX0/jxrTGS65VN+g2A9Qlvx1cm2Ro+copXA51k7Man2v2UWtCV3laskmURE4 KWEGI/V4y88UQqoujGbmQKrTIDf8RD78hOGdXihqPzVztmAlNXAwPMKNaTt7C6Yci2T/ /VDUWD9jx4iXKtG27z+v/4Ca8lV3rlEA2R+Ubopu0aPWKUxZSdPLpcY7vwxvIhZ4F1d5 6cyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3X1StATHosWwy8blsyGIZshMmSZpXUh1BfLs9iG/7tc=; b=M9YVdnRL5AMb9QVnjvmqM0wIBIs8mMfgGqGYB8Ij/+ehqu0+IkD0rH58pTRoqp/YTY tAWvzLzda0AdLUj5EUuIa3o1hupOYSXQDYlNPEQzxcA1XJP1adBoAYFrxs0SHWLYPNIm 7nZQaIaKJeAwH8A/aGRgGB6OOsafi1RuYX5fGfrkof2DGvYWIQOqDdIm7RL+HJFiX0qM VHNeDGVk/2Z0hBbPbrFWHDv/8nHS3bgsrK27JVOmzqIBgMCYMHn7w9S5cilXqME1C6xs oon3p/4hYWEFWxb02XPBh+kp1RwZgmGDC29P4QNKFnxHafNZyNpjh7CmM2x5VyvYhP3P LMsA== X-Gm-Message-State: ALoCoQmgZItFHJCBTo0rWh2yVNHCeKtNFxRwA57VuIo6pzPoFGXZdGG/2wYY9OSU3nZH6y2HOhaZn+bvt8VSoe4XIWt5BXTzXg== X-Received: by 10.107.137.151 with SMTP id t23mr100912662ioi.172.1452668906220; Tue, 12 Jan 2016 23:08:26 -0800 (PST) Received: from localhost.localdomain ([64.2.3.194]) by smtp.gmail.com with ESMTPSA id p18sm563353ioi.1.2016.01.12.23.08.24 (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 12 Jan 2016 23:08:25 -0800 (PST) Received: from localhost.localdomain (apinskidesktop [127.0.0.1]) by localhost.localdomain (8.14.3/8.14.3/Debian-9.4) with ESMTP id u0D78NSO003626 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Tue, 12 Jan 2016 23:08:23 -0800 Received: (from apinski@localhost) by localhost.localdomain (8.14.3/8.14.3/Submit) id u0D78Nls003625; Tue, 12 Jan 2016 23:08:23 -0800 From: Andrew Pinski To: pinskia@gmail.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Will Deacon Cc: Andrew Pinski Subject: [PATCH 5/5] ARM64: Patch in prefetching in copy_template Date: Tue, 12 Jan 2016 23:08:19 -0800 Message-Id: <1452668899-3553-6-git-send-email-apinski@cavium.com> X-Mailer: git-send-email 1.7.2.5 In-Reply-To: <1452668899-3553-1-git-send-email-apinski@cavium.com> References: <1452668899-3553-1-git-send-email-apinski@cavium.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org For ThunderX T88 pass 1.x and 2.x where there is no hardware prefetcher, we want to patch in software prefetching instructions in the copy_template. This speeds up copy_to_user and copy_from_user for large size. The main use of large sizes is I/O read/writes. Signed-off-by: Andrew Pinski --- arch/arm64/lib/copy_template.S | 12 ++++++++++++ arch/arm64/lib/memcpy.S | 2 ++ 2 files changed, 14 insertions(+), 0 deletions(-) -- 1.7.2.5 diff --git a/arch/arm64/lib/copy_template.S b/arch/arm64/lib/copy_template.S index 410fbdb..3f3f0a4 100644 --- a/arch/arm64/lib/copy_template.S +++ b/arch/arm64/lib/copy_template.S @@ -163,12 +163,24 @@ D_h .req x14 */ .p2align L1_CACHE_SHIFT .Lcpy_body_large: +alternative_if_not ARM64_NEEDS_PREFETCH_128 + nop + nop +alternative_else + prfm pldl1strm, [src, #128] + prfm pldl1strm, [src, #256] +alternative_endif /* pre-get 64 bytes data. */ ldp1 A_l, A_h, src, #16 ldp1 B_l, B_h, src, #16 ldp1 C_l, C_h, src, #16 ldp1 D_l, D_h, src, #16 1: +alternative_if_not ARM64_NEEDS_PREFETCH_128 + nop +alternative_else + prfm pldl1strm, [src, #384] +alternative_endif /* * interlace the load of next 64 bytes data block with store of the last * loaded 64 bytes data. diff --git a/arch/arm64/lib/memcpy.S b/arch/arm64/lib/memcpy.S index 6761393..3a50cf8b 100644 --- a/arch/arm64/lib/memcpy.S +++ b/arch/arm64/lib/memcpy.S @@ -25,6 +25,8 @@ #include #include #include +#include +#include /* * Copy a buffer from src to dest (alignment handled by the hardware)