From patchwork Tue Dec 26 10:29:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 122729 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp779269qgn; Tue, 26 Dec 2017 02:31:15 -0800 (PST) X-Google-Smtp-Source: ACJfBouqbsRM3pqtzZCS6WVsARd5EE9cyEO/mSUBWcUFAeBIJqes1c4nWi2WyFeEihLrb21zET8D X-Received: by 10.84.204.8 with SMTP id a8mr24743313ple.399.1514284275893; Tue, 26 Dec 2017 02:31:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1514284275; cv=none; d=google.com; s=arc-20160816; b=bbc38P/7rvG3rf521X8ZWtcY4j/yjieJ5ImTtEjmzSxR3Eg/0SpNmwevwl87x3StUq BlakgSwOy/P3GpzGQkxoZfdhCAtzJPONz7Hs3yQ/vLM6G/KVWeliPOq2nNGFolDoCHAb KaG9M0SP5WHC3mHER/oPcSdH2hxj6NcvcGKeDhN054qV6n//UsbWNKfFmKbS3Pvt8NHV C+eETezXbm4XoQaF1dlupZmW1l9d0iYDmVkRGN4DUOA3v/KLNdKE9/R1egIEFj9wSbiQ yB7FPGaGylxFfOgaBnSEIzha4W6/fsHlGWgvvXD+YKO6TvptPfQ2spi3O1LLrrW2IKtv audw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=5zk1lw0Of4wZ9+5EpqGNCIfS63EqvIFydBGYbdg5eLE=; b=GfvTpGsg7H0MllLZEHCStPTonQuLCjicFRmTxJ11HB6oanWegUyeXEOvXIAEfiAlNm ACoNGiVc1EuqM289pi4l7SlKGctwJ9VY4f1rVzHBqoAbQ9ulETqCPnhQR1SN0lqdoIxc zIE1tcrn6Y7jgiB9PSUyCaKP4GZJhfOrSh9b0woagYj7eVMX0Xrdd/AJetKUMmf2Mrae 9pRPE5RrPOn0m8ya0rR58Xb1zqh20Zsdx+OPU3eBNW+Js/SZ5sUQnVVmfC1LWsspYCJD dts9tnvv1LKqxBYiZnGKsP/ijRnbt4xbZARZlb2vRLYgk8d+uCDguDz1EupoFWt8ht4A aN+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dZ4+ob3B; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d13si5555913pgu.499.2017.12.26.02.31.15; Tue, 26 Dec 2017 02:31:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dZ4+ob3B; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751241AbdLZKbK (ORCPT + 28 others); Tue, 26 Dec 2017 05:31:10 -0500 Received: from mail-wr0-f193.google.com ([209.85.128.193]:45150 "EHLO mail-wr0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751205AbdLZKbG (ORCPT ); Tue, 26 Dec 2017 05:31:06 -0500 Received: by mail-wr0-f193.google.com with SMTP id o15so2113180wrf.12 for ; Tue, 26 Dec 2017 02:31:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5zk1lw0Of4wZ9+5EpqGNCIfS63EqvIFydBGYbdg5eLE=; b=dZ4+ob3B4HsCxqdA2JHHbkiaIvB2WRogFed0Hhdh/NQQjem4CcMSHD+G1IPf0DjbHN z5tMaIsY/FS0rt0nqft44WUE6z7J40YpXKdRc2JupQ/59h6o2346etveM8WLF0+6On+U FtZ81AoRzzsRU9S8EfCU/FlFZsnKim08wpzCI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5zk1lw0Of4wZ9+5EpqGNCIfS63EqvIFydBGYbdg5eLE=; b=VHqHwSN7vLMFDmZryo9SyEo7nc4+qCa1JhNOUBhtt1lhqc3aCN12D/4THUg2ia9Uy9 jclKenO3kSLhvKbi88SAzLe/OxqTSmiMmU65NO8CJYT/36eZdRxYhpjV1wmqy/BHHudI dOYwUwd0oBLx9U4ETxVI6pqfXN+ErHa25YpaL2W1mmDW7HHQyAveOjq4Ccjf5zo0Zcpj a4cpK2LXZcLCJYBDkj/DTzbGGHBsytt4uHe+Kt7k9HTwKahNcW/hoM4owlKk8S8Ezra3 VVJQSRyrB+nv+9J+KIwmbhmS9hdq8/lYohqsEWrC2r0ZUozCE/hQdUMgxZy+LnRppdJL uDWg== X-Gm-Message-State: AKGB3mLW81wt1UdwX/DoSWQxwlGOTb86DAUGVWZmQvLQgubV7cqNjf1V pc9PX4oK6WVADRT7AM8snXv3pipTz6g= X-Received: by 10.223.151.208 with SMTP id t16mr14233358wrb.200.1514284265295; Tue, 26 Dec 2017 02:31:05 -0800 (PST) Received: from localhost.localdomain ([160.171.216.245]) by smtp.gmail.com with ESMTPSA id l142sm13974036wmb.43.2017.12.26.02.31.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Dec 2017 02:31:04 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Dave Martin , Russell King - ARM Linux , Sebastian Andrzej Siewior , Mark Rutland , linux-rt-users@vger.kernel.org, Peter Zijlstra , Catalin Marinas , Will Deacon , Steven Rostedt , Thomas Gleixner Subject: [PATCH v4 09/20] crypto: arm64/sha256-neon - play nice with CONFIG_PREEMPT kernels Date: Tue, 26 Dec 2017 10:29:29 +0000 Message-Id: <20171226102940.26908-10-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171226102940.26908-1-ard.biesheuvel@linaro.org> References: <20171226102940.26908-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Tweak the SHA256 update routines to invoke the SHA256 block transform block by block, to avoid excessive scheduling delays caused by the NEON algorithm running with preemption disabled. Also, remove a stale comment which no longer applies now that kernel mode NEON is actually disallowed in some contexts. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/sha256-glue.c | 36 +++++++++++++------- 1 file changed, 23 insertions(+), 13 deletions(-) -- 2.11.0 diff --git a/arch/arm64/crypto/sha256-glue.c b/arch/arm64/crypto/sha256-glue.c index b064d925fe2a..e8880ccdc71f 100644 --- a/arch/arm64/crypto/sha256-glue.c +++ b/arch/arm64/crypto/sha256-glue.c @@ -89,21 +89,32 @@ static struct shash_alg algs[] = { { static int sha256_update_neon(struct shash_desc *desc, const u8 *data, unsigned int len) { - /* - * Stacking and unstacking a substantial slice of the NEON register - * file may significantly affect performance for small updates when - * executing in interrupt context, so fall back to the scalar code - * in that case. - */ + struct sha256_state *sctx = shash_desc_ctx(desc); + if (!may_use_simd()) return sha256_base_do_update(desc, data, len, (sha256_block_fn *)sha256_block_data_order); - kernel_neon_begin(); - sha256_base_do_update(desc, data, len, - (sha256_block_fn *)sha256_block_neon); - kernel_neon_end(); + while (len > 0) { + unsigned int chunk = len; + + /* + * Don't hog the CPU for the entire time it takes to process all + * input when running on a preemptible kernel, but process the + * data block by block instead. + */ + if (IS_ENABLED(CONFIG_PREEMPT) && + chunk + sctx->count % SHA256_BLOCK_SIZE > SHA256_BLOCK_SIZE) + chunk = SHA256_BLOCK_SIZE - + sctx->count % SHA256_BLOCK_SIZE; + kernel_neon_begin(); + sha256_base_do_update(desc, data, chunk, + (sha256_block_fn *)sha256_block_neon); + kernel_neon_end(); + data += chunk; + len -= chunk; + } return 0; } @@ -117,10 +128,9 @@ static int sha256_finup_neon(struct shash_desc *desc, const u8 *data, sha256_base_do_finalize(desc, (sha256_block_fn *)sha256_block_data_order); } else { - kernel_neon_begin(); if (len) - sha256_base_do_update(desc, data, len, - (sha256_block_fn *)sha256_block_neon); + sha256_update_neon(desc, data, len); + kernel_neon_begin(); sha256_base_do_finalize(desc, (sha256_block_fn *)sha256_block_neon); kernel_neon_end();