From patchwork Mon Jul 24 10:28:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 108560 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp3887661qge; Mon, 24 Jul 2017 03:28:58 -0700 (PDT) X-Received: by 10.84.232.130 with SMTP id i2mr17677922plk.278.1500892138643; Mon, 24 Jul 2017 03:28:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500892138; cv=none; d=google.com; s=arc-20160816; b=d3lKW56EhHHWgOUfWLf7aFlnTmNSP5DajZLwh9bK7oX2S92CnMTKJfrxwvjbBcKVj5 +nyDOWuDcFeYqw2LkE6vj8O6EstNDGNowff5njg1iNNjWp0Wd24H1AP14Uy31uCRBXAF CZvkxlKJA+5VkAsvxqroXpG08agGbYBufcr31faTfbMWrqgfrdC2RcZYQMOxuvzD8j2D YKiTjQxTsP6RsGMiXI4nyhAL5+wO+3EKpeBZZ2X/Ls+PCrIlJCl7iDWinJQwoO//ZcAp oXqPNd7pDbxyTsxiyv9oNXbjMONzi2pz5w3P5uA5HyRInPim0gFIfy04KtMCi8g0GaYN jWVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=sLsc+siiuMgNTqaIo8Ly1dTqU2/gy3Oo66nAHZOi2S0=; b=qc2Bz7oBKpkwhJ27maZFMiNC7tC2p9fsXj95aKlRHAzQGy0R8aLQj9yViQleh9XNae bk598g5j6fS9Ix/2Oad1g9VWDnQs1q/tgM/sCwHKEwfdwhuUVNkF1DW1SU7meuMkKnTn TZ/IMR3EcaR4C4EH5Z/HG4C3RzeWlYRezFw4/iVOKZssoIjVsuo9mGip6KawjjoNGp2q Ycufbdsi5b+/YMhhwBS+O06UE8dhkY+1QzxahRjtJoYU0kx8mRX7yi4rmL1rB3aKg1sQ VwEioWhgBgQxRzYsQUXEDxNtQUxiMZgYkAB+jpoD4vi+SSTZbxA7L+i+WSaTbqd7K4R4 1ltg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.b=PbEhRfrI; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 38si3348411pld.76.2017.07.24.03.28.58; Mon, 24 Jul 2017 03:28:58 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.b=PbEhRfrI; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752579AbdGXK25 (ORCPT + 1 other); Mon, 24 Jul 2017 06:28:57 -0400 Received: from mail-wr0-f180.google.com ([209.85.128.180]:34841 "EHLO mail-wr0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752552AbdGXK24 (ORCPT ); Mon, 24 Jul 2017 06:28:56 -0400 Received: by mail-wr0-f180.google.com with SMTP id k71so50755951wrc.2 for ; Mon, 24 Jul 2017 03:28:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sLsc+siiuMgNTqaIo8Ly1dTqU2/gy3Oo66nAHZOi2S0=; b=PbEhRfrIn4LX/CNRqfMmQER8d6wx9m8LQCFdbGCnxlIFoEoW79RXcyOl6l4pp5wc8E LNQA0nndWjdB+9DwcUNrpbqvByx7zvwiFlqjkRllDYJmKKpsptmsT97AbSYtivPt/Xk0 kcW7NeZ3+dtnvP4ECyfP6KjBpCR0QvUlFhITY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sLsc+siiuMgNTqaIo8Ly1dTqU2/gy3Oo66nAHZOi2S0=; b=hM312qL+6gN9F/4HFd8sHO0XH5giYC9n93wKn60u3U2LZU8UKIeKBkp4ZYg6h+Evzl ItqbUL1tACw5cJOWSfpIITCD2dhw/W3CAvbFU8J2h5OxCJhCJUIYqR0eehk30UUL3a0T AXwCRBvX8ZAYzF2o4zvx5O0KQt1onw0aTH+DvfKE5P35fjLI4GoyQLC7pWBpGkeQPsx9 1VtzEjFhZpctC0OWt4ic7BJHhXpI3vYXQBvkmJfs6JRVGdwtYsJ8vH6vUs5geGnQwtFh 4j+T+m+uKFVN0TF9O+s33EhYHl93IyhRO2KmSMuAJe6e4vAAM9PmMN1EqNWMRm1w85H0 Nqsw== X-Gm-Message-State: AIVw1115AWReH2GvjPDsUzBjQiK3fF7jpLgv5ZtuvVi58q3gktr3V/09 M6wgpSllAvU8BvRxfFYXgA== X-Received: by 10.223.151.212 with SMTP id t20mr11145498wrb.233.1500892135110; Mon, 24 Jul 2017 03:28:55 -0700 (PDT) Received: from localhost.localdomain ([105.148.195.69]) by smtp.gmail.com with ESMTPSA id v44sm13205400wrb.53.2017.07.24.03.28.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Jul 2017 03:28:54 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: herbert@gondor.apana.org.au, dave.martin@arm.com, Ard Biesheuvel Subject: [PATCH resend 13/18] crypto: arm64/aes-bs - implement non-SIMD fallback for AES-CTR Date: Mon, 24 Jul 2017 11:28:15 +0100 Message-Id: <20170724102820.16534-14-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170724102820.16534-1-ard.biesheuvel@linaro.org> References: <20170724102820.16534-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Of the various chaining modes implemented by the bit sliced AES driver, only CTR is exposed as a synchronous cipher, and requires a fallback in order to remain usable once we update the kernel mode NEON handling logic to disallow nested use. So wire up the existing CTR fallback C code. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 1 + arch/arm64/crypto/aes-neonbs-glue.c | 48 ++++++++++++++++++-- 2 files changed, 44 insertions(+), 5 deletions(-) -- 2.9.3 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index a068dcbe2518..f9e264b83366 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -89,6 +89,7 @@ config CRYPTO_AES_ARM64_BS depends on KERNEL_MODE_NEON select CRYPTO_BLKCIPHER select CRYPTO_AES_ARM64_NEON_BLK + select CRYPTO_AES_ARM64 select CRYPTO_SIMD endif diff --git a/arch/arm64/crypto/aes-neonbs-glue.c b/arch/arm64/crypto/aes-neonbs-glue.c index 9001aec16007..c55d68ccb89f 100644 --- a/arch/arm64/crypto/aes-neonbs-glue.c +++ b/arch/arm64/crypto/aes-neonbs-glue.c @@ -1,7 +1,7 @@ /* * Bit sliced AES using NEON instructions * - * Copyright (C) 2016 Linaro Ltd + * Copyright (C) 2016 - 2017 Linaro Ltd * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -9,12 +9,15 @@ */ #include +#include #include #include #include #include #include +#include "aes-ctr-fallback.h" + MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); @@ -58,6 +61,11 @@ struct aesbs_cbc_ctx { u32 enc[AES_MAX_KEYLENGTH_U32]; }; +struct aesbs_ctr_ctx { + struct aesbs_ctx key; /* must be first member */ + struct crypto_aes_ctx fallback; +}; + struct aesbs_xts_ctx { struct aesbs_ctx key; u32 twkey[AES_MAX_KEYLENGTH_U32]; @@ -196,6 +204,25 @@ static int cbc_decrypt(struct skcipher_request *req) return err; } +static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key, + unsigned int key_len) +{ + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + int err; + + err = crypto_aes_expand_key(&ctx->fallback, in_key, key_len); + if (err) + return err; + + ctx->key.rounds = 6 + key_len / 4; + + kernel_neon_begin(); + aesbs_convert_key(ctx->key.rk, ctx->fallback.key_enc, ctx->key.rounds); + kernel_neon_end(); + + return 0; +} + static int ctr_encrypt(struct skcipher_request *req) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -259,6 +286,17 @@ static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, return aesbs_setkey(tfm, in_key, key_len); } +static int ctr_encrypt_sync(struct skcipher_request *req) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + + if (!may_use_simd()) + return aes_ctr_encrypt_fallback(&ctx->fallback, req); + + return ctr_encrypt(req); +} + static int __xts_crypt(struct skcipher_request *req, void (*fn)(u8 out[], u8 const in[], u8 const rk[], int rounds, int blocks, u8 iv[])) @@ -355,7 +393,7 @@ static struct skcipher_alg aes_algs[] = { { .base.cra_driver_name = "ctr-aes-neonbs", .base.cra_priority = 250 - 1, .base.cra_blocksize = 1, - .base.cra_ctxsize = sizeof(struct aesbs_ctx), + .base.cra_ctxsize = sizeof(struct aesbs_ctr_ctx), .base.cra_module = THIS_MODULE, .min_keysize = AES_MIN_KEY_SIZE, @@ -363,9 +401,9 @@ static struct skcipher_alg aes_algs[] = { { .chunksize = AES_BLOCK_SIZE, .walksize = 8 * AES_BLOCK_SIZE, .ivsize = AES_BLOCK_SIZE, - .setkey = aesbs_setkey, - .encrypt = ctr_encrypt, - .decrypt = ctr_encrypt, + .setkey = aesbs_ctr_setkey_sync, + .encrypt = ctr_encrypt_sync, + .decrypt = ctr_encrypt_sync, }, { .base.cra_name = "__xts(aes)", .base.cra_driver_name = "__xts-aes-neonbs",