From patchwork Fri Jul 27 12:59:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 143045 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp787285ljj; Fri, 27 Jul 2018 05:59:23 -0700 (PDT) X-Google-Smtp-Source: AAOMgpe7VtFXFYMO+SK7LqV+iY4Yjuh9p3Ax5znwKzcx0rzPuGgKieMXXlM0RgdBH2ba69TT2/Fi X-Received: by 2002:a17:902:243:: with SMTP id 61-v6mr5897050plc.301.1532696363010; Fri, 27 Jul 2018 05:59:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532696363; cv=none; d=google.com; s=arc-20160816; b=VbsyiD8h/9TGNhXsAhgIcVHF5kGdgPk3bd/WSAa+hvUK7HT58gn6x6eKu9SJ5XWu9N WVpKCvtw970zaq6yvdCmqBLSJtZ4JV8pXeJIQyhGByfONqAGH1KsFnMQlgWo7ljvZcze U2sF2si+NSOYe+ppkJ0yAiZJkZhhBsmRiYH75x0IEUIBwIshNagCqcyB6YCduPyT621F aktZtmJybz5DgpgMWYK+ogp3CNInQu50YimVjYKMHmrhRBleEE7JQIbvT3UMlE3WOpkt RZWL661bNHTuZCbrwjD+RZh2g58BdDnPqOLQUiBvejoFt0b0HFZ4UBcI+r6w28wL2lbl UmnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=+EkaCzWXvx1E5yZvtZ+oCJgGhAdYVL9kIhUTwAn06A4=; b=VjW5qqlVfKEXhlmuFSKZQurMBbobZkJ7+9gzHZzE+7VB2ZYFfxE7vFdBiPmdFZjMT/ 3qmcc/UqO4QpbkoTwEVTjaOBSpo31FgfGG+ZW2R4UrxdjDSPIGd2LalE1aiIpL2zbujo MG/Q/guHiMC9asjvjFEqoM6Uw5KCvjrbNHTnaMtJXWEYEB1TDtEOzPUZO5/ntxb6qIbW 8HU+G63e/HhdeKNpiN0x3W2GbFaqJP/z2xv/+UdMpxHzKIH23WtWznXLPjHDnVZI10kx mmd5HteS7D55CSHKBtoqTbMt9SS4KhstndWqCteym7B6DXZVvs2Ec5wsqVyxqAJZvsK+ ZCeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D0I63BJv; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c17-v6si3680806pgg.628.2018.07.27.05.59.22; Fri, 27 Jul 2018 05:59:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D0I63BJv; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730428AbeG0OVL (ORCPT + 1 other); Fri, 27 Jul 2018 10:21:11 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:44703 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728384AbeG0OVL (ORCPT ); Fri, 27 Jul 2018 10:21:11 -0400 Received: by mail-ed1-f66.google.com with SMTP id f23-v6so3852192edr.11 for ; Fri, 27 Jul 2018 05:59:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=+EkaCzWXvx1E5yZvtZ+oCJgGhAdYVL9kIhUTwAn06A4=; b=D0I63BJvLS4LXy12dfwzLruQESfQERU71oNyZIR6jxNEVmN+fxfnEVIQNpKEaDiho8 Xdp9BLSUuxRENMidi6TAf4ClRkHsumFi8YAOo3UfRLcvYcCW7vjwaG32auK9LyZUax/x B7pURTQ0+PYJSfoL2Rhx89Rt2wQv3fOOcRiCA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=+EkaCzWXvx1E5yZvtZ+oCJgGhAdYVL9kIhUTwAn06A4=; b=RpXcq+nedA13JLWy15nAHzxRIQSEztv0YXsuKX21BzJhYWzEvEo84pmc5jBa9ZMrbZ oq0wbxc8FLnH+9dVQcpP16CehqY/BGHqCVjXPf73S6UXQnhiPG82gwihtQFMXgfpYW20 RXDH436HXLYmyCEWBGZOJ1KxKZbMIqYHivvxTqXjrfTVuBzP0GhzcR0r5YDs837885yI 5kokwAFO9LZzEpdxV5p6iWlW1TvRXcDn3hPZQ3u72JF80sSYQy2XOI9pW/LxuMKMpN3V OYHzOp1NPrHtCfQn9eq995pLuQG+RzU3g8C/gksO/kUaKd/aAoTkBV8+IrTG8ZMbJTTs gvnQ== X-Gm-Message-State: AOUpUlEvi5ZhApGm9kOSHx8fKzQTJzs5BMRKmebVRKPB1yM/Yp18FZok O/dxbevqBRkgxLJfrjciwVdP/kKRBc1n/A== X-Received: by 2002:a50:8406:: with SMTP id 6-v6mr7341389edp.97.1532696360129; Fri, 27 Jul 2018 05:59:20 -0700 (PDT) Received: from rev02.home (b80182.upc-b.chello.nl. [212.83.80.182]) by smtp.gmail.com with ESMTPSA id a13-v6sm2414813edf.84.2018.07.27.05.59.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jul 2018 05:59:18 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, Ard Biesheuvel Subject: [PATCH] crypto/arm64: aes-ce-gcm - add missing kernel_neon_begin/end pair Date: Fri, 27 Jul 2018 14:59:15 +0200 Message-Id: <20180727125915.20326-1-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.18.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Calling pmull_gcm_encrypt_block() requires kernel_neon_begin() and kernel_neon_end() to be used since the routine touches the NEON register file. Add the missing calls. Also, since NEON register contents are not preserved outside of a kernel mode NEON region, pass the key schedule array again. Fixes: 7c50136a8aba ("crypto: arm64/aes-ghash - yield NEON after every ...") Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/ghash-ce-glue.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) -- 2.18.0 Acked-by: Herbert Xu diff --git a/arch/arm64/crypto/ghash-ce-glue.c b/arch/arm64/crypto/ghash-ce-glue.c index 7cf0b1aa6ea8..8a10f1d7199a 100644 --- a/arch/arm64/crypto/ghash-ce-glue.c +++ b/arch/arm64/crypto/ghash-ce-glue.c @@ -488,9 +488,13 @@ static int gcm_decrypt(struct aead_request *req) err = skcipher_walk_done(&walk, walk.nbytes % AES_BLOCK_SIZE); } - if (walk.nbytes) - pmull_gcm_encrypt_block(iv, iv, NULL, + if (walk.nbytes) { + kernel_neon_begin(); + pmull_gcm_encrypt_block(iv, iv, ctx->aes_key.key_enc, num_rounds(&ctx->aes_key)); + kernel_neon_end(); + } + } else { __aes_arm64_encrypt(ctx->aes_key.key_enc, tag, iv, num_rounds(&ctx->aes_key));