From patchwork Tue Jul 2 19:41:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168347 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653215ilk; Tue, 2 Jul 2019 12:42:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqycCwiBWZftdeAKS50SW6XLPOtlTY4a+Jw2nLojjf0Nt7SnyzXYFlmponq6jjx2Iq5rcjq/ X-Received: by 2002:a63:eb56:: with SMTP id b22mr30536892pgk.355.1562096535077; Tue, 02 Jul 2019 12:42:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096535; cv=none; d=google.com; s=arc-20160816; b=Uf2YFwHraq2MG4skfh9ndlBMEfuxjSrUnY6Vz2/VhsTaycFNfjlepaUi8Z5VaplnUN hBHIXqzVsmS2SjW1Ep5ArEETUw/WTQZE1v6vjZ3R0A4Nh7Io//U7AlFbqeixGd/KIHty aB40ONCvMkGbE8KpNhz0kSfdifVRbCUn9P3PxIEMwMwtQy/tG8rCHGglZ8+BXYoBf9d6 H0PPTe6qD5TMaqxIsYvdjNa8O0+BxIiLtA+V3RWEW3nKjg6tQ3i0D/p8p+yXyUfUiYam QSxtD9CgEXPtqDL0Fd37cUQDQ8UUuwruUzDy7dKFPyQYHB/0pRRQ3NLfrpL18Yowj6hk KH1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=trJajH9cyH+LbZxrv+7nC1tNggLxsbw2lvOtUdKDNJM=; b=PX05aaMP3xkLNRxdlco87pX6a+r02peDk69w7WbdBxMUWuvaoy/WffJdRZtne6arI0 qLqwtuFbPjxtfoeSYSLuuO0tiqreem/XHLYOwyG+at1PhJ4UkHjV7cdtD6EVLEX+7GW3 iIpf4Qeyb9B0gIomYdocNBeH7pSOvSdWH/1WAv2OhIyjZyVVa/wlQQAzSE5A3XygbsCq fkc33BVoXW65k4dn5A7Rt0nV2p6v+OJY9Xik7mrrCzNtFYSzZbbGoEvxAHWthyEodzJZ mARPAuVBxRZyKYLYySJn7ooL8Q6cxRIoa7lBreIddjT3vG/FkOzwLq6685mK4EbPaaOq Wlgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vFAIUsgB; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.14; Tue, 02 Jul 2019 12:42:15 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vFAIUsgB; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726736AbfGBTmO (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:14 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:40378 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726291AbfGBTmO (ORCPT ); Tue, 2 Jul 2019 15:42:14 -0400 Received: by mail-lj1-f193.google.com with SMTP id a21so18168051ljh.7 for ; Tue, 02 Jul 2019 12:42:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=trJajH9cyH+LbZxrv+7nC1tNggLxsbw2lvOtUdKDNJM=; b=vFAIUsgB1TrvxsRf5eFLNylXY+O9MuJSETqU8ykntVQYNNIOSNEeWMYs1l8/h7HZXB 11OOd4bGMOIKMd8pc74eJ4Thosg2H9P5QA85f+VPk+uZVy4/oMDy3vApeBm9dkhBN/bD 6hP/IOyDQ/CNTF3nWLBXseR5fx81qCfSKeRimvcXtNtK3sZuTHJzJN1gmIMfla47APn2 70yPZpEiFKk37t6e9jwUXGF5HxmIWNFRy7cqXXaBuFVhc1scMJlaR/8MkfR2EiHGMlSo Eb+K5AUT2QJTjINjqWcJr+qo/AbWk0BBdXg3SYn36zOSRJf41UxMxJ0fczXCRA1wk/xn yV2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=trJajH9cyH+LbZxrv+7nC1tNggLxsbw2lvOtUdKDNJM=; b=AXqFN9DSbLhq3pEOJstyfAqGgo2D1UtvqoorQByRkmpZC09p1J6ySclweAj9EBu7Hj SU2s1z07u4SN/95vWaXf1e8mpIHmYEWATkgWN3x9pGrCKnzhJctQNuXlsR+BRXxSEPP0 MCk1sQ3Jv3j0ypZpBbTgWS1U9JC+U8WzDqXuEQ8d6z3dKyjSqqw745XMMjjIA2yFtMtw fh9Ubm1GuDk91ZikvBlgBsYKY1cZkBM9oZqWFe7mgrj8gQDEeRsvTogQ6TvwoieaA1gf hUVlAgzN9U0HGKWvjKh3+i2/psboJZUev+xYLS0UApNHdu28bxFevTwPakYG2s9FuMMK rlPQ== X-Gm-Message-State: APjAAAXrM/loIY/m/2noltGjUG7mz3YD4W/1RGbJVnupV5PSJ/APrAe2 kw0Qu72DhcCIHIEBVJOpo3U0zGjzwerwAjGB X-Received: by 2002:a2e:8997:: with SMTP id c23mr18128703lji.158.1562096531542; Tue, 02 Jul 2019 12:42:11 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:10 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 01/32] crypto: arm/aes-ce - cosmetic/whitespace cleanup Date: Tue, 2 Jul 2019 21:41:19 +0200 Message-Id: <20190702194150.10405-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Rearrange the aes_algs[] array for legibility. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-glue.c | 116 ++++++++++---------- 1 file changed, 56 insertions(+), 60 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index 5affb8482379..04ba66903674 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -337,69 +337,65 @@ static int xts_decrypt(struct skcipher_request *req) } static struct skcipher_alg aes_algs[] = { { - .base = { - .cra_name = "__ecb(aes)", - .cra_driver_name = "__ecb-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .setkey = ce_aes_setkey, - .encrypt = ecb_encrypt, - .decrypt = ecb_decrypt, + .base.cra_name = "__ecb(aes)", + .base.cra_driver_name = "__ecb-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ecb_encrypt, + .decrypt = ecb_decrypt, }, { - .base = { - .cra_name = "__cbc(aes)", - .cra_driver_name = "__cbc-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .setkey = ce_aes_setkey, - .encrypt = cbc_encrypt, - .decrypt = cbc_decrypt, + .base.cra_name = "__cbc(aes)", + .base.cra_driver_name = "__cbc-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = cbc_encrypt, + .decrypt = cbc_decrypt, }, { - .base = { - .cra_name = "__ctr(aes)", - .cra_driver_name = "__ctr-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = 1, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = AES_MIN_KEY_SIZE, - .max_keysize = AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .chunksize = AES_BLOCK_SIZE, - .setkey = ce_aes_setkey, - .encrypt = ctr_encrypt, - .decrypt = ctr_encrypt, + .base.cra_name = "__ctr(aes)", + .base.cra_driver_name = "__ctr-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .chunksize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ctr_encrypt, + .decrypt = ctr_encrypt, }, { - .base = { - .cra_name = "__xts(aes)", - .cra_driver_name = "__xts-aes-ce", - .cra_priority = 300, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_xts_ctx), - .cra_module = THIS_MODULE, - }, - .min_keysize = 2 * AES_MIN_KEY_SIZE, - .max_keysize = 2 * AES_MAX_KEY_SIZE, - .ivsize = AES_BLOCK_SIZE, - .setkey = xts_set_key, - .encrypt = xts_encrypt, - .decrypt = xts_decrypt, + .base.cra_name = "__xts(aes)", + .base.cra_driver_name = "__xts-aes-ce", + .base.cra_priority = 300, + .base.cra_flags = CRYPTO_ALG_INTERNAL, + .base.cra_blocksize = AES_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct crypto_aes_xts_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = 2 * AES_MIN_KEY_SIZE, + .max_keysize = 2 * AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = xts_set_key, + .encrypt = xts_encrypt, + .decrypt = xts_decrypt, } }; static struct simd_skcipher_alg *aes_simd_algs[ARRAY_SIZE(aes_algs)]; From patchwork Tue Jul 2 19:41:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168348 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653243ilk; Tue, 2 Jul 2019 12:42:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqyjDWkhcjOKgSVbQHf6MG0aPj9nf8DLY8Oo+iwKbR6yJ5wDnk1agAVbbKt7vY3GcAC+sRI3 X-Received: by 2002:a17:902:2aab:: with SMTP id j40mr35620819plb.76.1562096536646; Tue, 02 Jul 2019 12:42:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096536; cv=none; d=google.com; s=arc-20160816; b=YQw9b3reRkUiPkvT9mDv6zbfy4n6FqKLqcJswSeUU4uNzo7GIT/jfL0xtxDt9iehO5 foUzQLjewIR2IAkQttKVHiYfo6xecPOG3RcNpHw79gm0h2AxkHrXC8hl462UYVUUQjsP WLFUNjj5oubmv5U3Rk4fSSC+2Br6iZBaS2pZygjOJe69kIiw0ZglRlKLtjXBkUPfiF/w oI04A+ckvkWKCbQKAubgaZaulc2B4Z7q3BQB/4edLPWbAmwlNW0P9wl8B9nkRTIZD3VT NYlw+pJjC4FeuuHHqRfIBbD+D5kSuTVgDycG4nqI5cpxwavZ6KlpsKcku8ZFYvW/ZnF0 dlCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=2+eLnmRKp4RXP1xyqf/h535DDbVPM+2CtmZm4AazSXI=; b=TSijA3HRE3TtOrJ446IQIZj6bjb0Fc6V79FuMY/z7PympD6z9TZ7A7leZQWTIcQIY3 QChsz4LIrQ2vDnqXEcKkGoswgF5vkUiBqOAhnRN/3VVCz6oQYrU6CvwJ95sVxnV8Y2Np w8VBaAXuXas+/sh6aZ9tkdvfTSV7tEQa5/DIshUzc0ZrQUyio5I7vvWY69SJbeLmo9hV s2WTzydDOcSGPKjDkZmrDyBvpeqeWXvHRgZq1jeJdJkrtfyLWT4V6WMduWaswyzeNwf5 jLoyZjcQ2I+Pi8Gmkl/V01ImIl1KxHwQVttVzIUUcQyi7TAsd5h16+x5uHnz7kuEfb54 3ztg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RKQ0FGEE; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.16; Tue, 02 Jul 2019 12:42:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RKQ0FGEE; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726824AbfGBTmQ (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:16 -0400 Received: from mail-lf1-f66.google.com ([209.85.167.66]:35975 "EHLO mail-lf1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726150AbfGBTmP (ORCPT ); Tue, 2 Jul 2019 15:42:15 -0400 Received: by mail-lf1-f66.google.com with SMTP id q26so12287831lfc.3 for ; Tue, 02 Jul 2019 12:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2+eLnmRKp4RXP1xyqf/h535DDbVPM+2CtmZm4AazSXI=; b=RKQ0FGEEBrttzHzkR9FWohWLqG23UNciB8IPtB4ALc43cybdfjAo6AsB1OLOl4e8xw 1QcO1+Q+aEYErm10TZIZoGAYp8bQk5xhHk0D7VnKXDqjLd6WP3mfLkATBf0j3Yt5dV4H jd3zlxdYpJLEczF2Kiuyc97CkqpIGwC1QznkYH2lNC4RaNdg94qupJN3dXxTC0+TJEsV J6vvNXA0mLbkR0lI5Vh4iicgUaAjr9vSr7wR3mqV1AYMT+jVvnoqvOW+l/OW6yNxodS4 u8T7kuRq3dxSYoWgbFfWahxRz9QVLqz8huRP4EzuLv94xPNFXTF0ByTQacgBS+APQg8/ lbDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2+eLnmRKp4RXP1xyqf/h535DDbVPM+2CtmZm4AazSXI=; b=PNURjZ36tgkI9huVyTE/TP9iXLTOeWPx9G31L9ElrlqYFDLOvD4A+or231yZz41HPA iAGwSZAUO73YTI2/4BQ/OCnaiPbhNB8ZPZZKrlcuUeod+Kbyv1pONnpbfEXd4Bzq6/hL Rj/Q9IrtWffDNp4yJH/6bE6nTUL3LzmrvRjrtIc0wDsxTHwXnq2Wu27rhwJO7rw8rqXX /qHGjpksk6WXDdDOb5EH+MGtPVYVbyppfRsirD4g3JJfCKdUnspY5IXO3S+kAFG9wOnU 5iSqd3lCASAUOgpdLwJpmrX25jkQz3iW4wNjtVjyrXDN+IFrvWJ2vRpbvswmfYDM1GFS TboQ== X-Gm-Message-State: APjAAAXXjtYLwdot+l+FXY0R4+EJqWnQosO1WFQcCPKJluN/jyZG4DtN 7wYSFkmRS14okrrd/aIjveJTLmpbesJWF8hg X-Received: by 2002:ac2:42ca:: with SMTP id n10mr3545468lfl.121.1562096532875; Tue, 02 Jul 2019 12:42:12 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.11 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:12 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 02/32] crypto: aes - rename local routines to prevent future clashes Date: Tue, 2 Jul 2019 21:41:20 +0200 Message-Id: <20190702194150.10405-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Rename some local AES encrypt/decrypt routines so they don't clash with the names we are about to introduce for the routines exposed by the generic AES library. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-glue.c | 8 ++++---- arch/arm64/crypto/aes-cipher-glue.c | 8 ++++---- arch/sparc/crypto/aes_glue.c | 8 ++++---- arch/x86/crypto/aesni-intel_glue.c | 8 ++++---- crypto/aes_generic.c | 8 ++++---- drivers/crypto/padlock-aes.c | 8 ++++---- 6 files changed, 24 insertions(+), 24 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/aes-cipher-glue.c b/arch/arm/crypto/aes-cipher-glue.c index c222f6e072ad..f6c07867b8ff 100644 --- a/arch/arm/crypto/aes-cipher-glue.c +++ b/arch/arm/crypto/aes-cipher-glue.c @@ -19,7 +19,7 @@ EXPORT_SYMBOL(__aes_arm_encrypt); asmlinkage void __aes_arm_decrypt(u32 *rk, int rounds, const u8 *in, u8 *out); EXPORT_SYMBOL(__aes_arm_decrypt); -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -27,7 +27,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) __aes_arm_encrypt(ctx->key_enc, rounds, in, out); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -47,8 +47,8 @@ static struct crypto_alg aes_alg = { .cra_cipher.cia_min_keysize = AES_MIN_KEY_SIZE, .cra_cipher.cia_max_keysize = AES_MAX_KEY_SIZE, .cra_cipher.cia_setkey = crypto_aes_set_key, - .cra_cipher.cia_encrypt = aes_encrypt, - .cra_cipher.cia_decrypt = aes_decrypt, + .cra_cipher.cia_encrypt = aes_arm_encrypt, + .cra_cipher.cia_decrypt = aes_arm_decrypt, #ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS .cra_alignmask = 3, diff --git a/arch/arm64/crypto/aes-cipher-glue.c b/arch/arm64/crypto/aes-cipher-glue.c index 7288e7cbebff..0e90b06ebcec 100644 --- a/arch/arm64/crypto/aes-cipher-glue.c +++ b/arch/arm64/crypto/aes-cipher-glue.c @@ -18,7 +18,7 @@ EXPORT_SYMBOL(__aes_arm64_encrypt); asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); EXPORT_SYMBOL(__aes_arm64_decrypt); -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm64_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -26,7 +26,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) __aes_arm64_encrypt(ctx->key_enc, out, in, rounds); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void aes_arm64_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); int rounds = 6 + ctx->key_length / 4; @@ -46,8 +46,8 @@ static struct crypto_alg aes_alg = { .cra_cipher.cia_min_keysize = AES_MIN_KEY_SIZE, .cra_cipher.cia_max_keysize = AES_MAX_KEY_SIZE, .cra_cipher.cia_setkey = crypto_aes_set_key, - .cra_cipher.cia_encrypt = aes_encrypt, - .cra_cipher.cia_decrypt = aes_decrypt + .cra_cipher.cia_encrypt = aes_arm64_encrypt, + .cra_cipher.cia_decrypt = aes_arm64_decrypt }; static int __init aes_init(void) diff --git a/arch/sparc/crypto/aes_glue.c b/arch/sparc/crypto/aes_glue.c index a9b8b0b94a8d..1f7191c243bc 100644 --- a/arch/sparc/crypto/aes_glue.c +++ b/arch/sparc/crypto/aes_glue.c @@ -196,14 +196,14 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, return 0; } -static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void crypto_aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_sparc64_aes_ctx *ctx = crypto_tfm_ctx(tfm); ctx->ops->encrypt(&ctx->key[0], (const u32 *) src, (u32 *) dst); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void crypto_aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_sparc64_aes_ctx *ctx = crypto_tfm_ctx(tfm); @@ -395,8 +395,8 @@ static struct crypto_alg algs[] = { { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt + .cia_encrypt = crypto_aes_encrypt, + .cia_decrypt = crypto_aes_decrypt } } }, { diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c index c95bd397dc07..836d50bd096f 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -349,7 +349,7 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, return aes_set_key_common(tfm, crypto_tfm_ctx(tfm), in_key, key_len); } -static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void aesni_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); @@ -362,7 +362,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) } } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) +static void aesni_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); @@ -923,8 +923,8 @@ static struct crypto_alg aesni_cipher_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt + .cia_encrypt = aesni_encrypt, + .cia_decrypt = aesni_decrypt } } }; diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index f217568917e4..3aa4a715c216 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -1332,7 +1332,7 @@ EXPORT_SYMBOL_GPL(crypto_aes_set_key); f_rl(bo, bi, 3, k); \ } while (0) -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void crypto_aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); u32 b0[4], b1[4]; @@ -1402,7 +1402,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) i_rl(bo, bi, 3, k); \ } while (0) -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void crypto_aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); u32 b0[4], b1[4]; @@ -1454,8 +1454,8 @@ static struct crypto_alg aes_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = crypto_aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt + .cia_encrypt = crypto_aes_encrypt, + .cia_decrypt = crypto_aes_decrypt } } }; diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c index 09d823d36d3a..854539512c35 100644 --- a/drivers/crypto/padlock-aes.c +++ b/drivers/crypto/padlock-aes.c @@ -299,7 +299,7 @@ static inline u8 *padlock_xcrypt_cbc(const u8 *input, u8 *output, void *key, return iv; } -static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void padlock_aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct aes_ctx *ctx = aes_ctx(tfm); @@ -308,7 +308,7 @@ static void aes_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) padlock_store_cword(&ctx->cword.encrypt); } -static void aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) +static void padlock_aes_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { struct aes_ctx *ctx = aes_ctx(tfm); @@ -331,8 +331,8 @@ static struct crypto_alg aes_alg = { .cia_min_keysize = AES_MIN_KEY_SIZE, .cia_max_keysize = AES_MAX_KEY_SIZE, .cia_setkey = aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt, + .cia_encrypt = padlock_aes_encrypt, + .cia_decrypt = padlock_aes_decrypt, } } }; From patchwork Tue Jul 2 19:41:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168349 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653257ilk; Tue, 2 Jul 2019 12:42:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqzZF8dg/hM5kQr4HB3IGpm+pBX7u1aPWYHxV1VLn1XmGZtZlaxVQ8o5sLR+3umQGWzdhtzP X-Received: by 2002:a17:90a:270f:: with SMTP id o15mr7536658pje.56.1562096537594; Tue, 02 Jul 2019 12:42:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096537; cv=none; d=google.com; s=arc-20160816; b=PIygJNliFFa30ybKxY3xtQzqJPB17TFw+xKTPY28QxNFYA8liY6JRKfWNIitdEBHl9 xXPhXxpgf/zryDTIPGvrqZd7toESh830eGgLMnrex0or/FDO/r1YL3JJgFMilMmqXYGu 5UmJKrERciECxZTHAT3xNX1g4GMRdU344vP/KBb+Svp/T/ZyLVbSONN3dmCKfraRCl48 LpnQDBm+4hVm/EzPU+IRsQ3Gr5ZgVO2+oUF/rf1TPlQXA09fYIQZEKp8QXJgrt5W7GBE V4zVb94XrdM09aGOExuh3D5DX2Kw96cnCZV0O03yuBFCny+J6dqkuu4sMyjySKSAd6PI PaRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=pFttl+7mb3vlqjJiXTJxAK+LxJsUUATYj2v8abg1GY4=; b=k9lPTpr99QsJKJV5yAUVoQpsqpsgAFcgioc+Sm/VGNUoAbTfMweIXKfAjtK5vGhmHN 5J+95pjrivvXarnY0rw6CnKDN1G4DYJWKLHFmvmNvJd1N9uPfkiTncJHaWsI+LmfnBp7 wipwTeKpkwi9XWOihvYxPCJ9KCH+WtSO2JwkTPJsvNbz9yjAdDqdyZzZ/ucKg3Xq1Upo szhCO4R7VZgpPyG1zymJdrx6lQRsJiE306fk7D308w5KUXF/g7VEdJEMhxIffmli/guE ItNvxMT8lrUQiEMR585eJv+fmdefUia34wyVoZaSBzvCgvgTyVgh9OydrZOn4FOFKWHt pW+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j0cjnk38; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.17; Tue, 02 Jul 2019 12:42:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j0cjnk38; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726150AbfGBTmQ (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:16 -0400 Received: from mail-lf1-f65.google.com ([209.85.167.65]:42319 "EHLO mail-lf1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726291AbfGBTmQ (ORCPT ); Tue, 2 Jul 2019 15:42:16 -0400 Received: by mail-lf1-f65.google.com with SMTP id x144so12265013lfa.9 for ; Tue, 02 Jul 2019 12:42:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pFttl+7mb3vlqjJiXTJxAK+LxJsUUATYj2v8abg1GY4=; b=j0cjnk38DJZNlyZr0cmMtPWv/0AjLcJBcNbJqZp5molS9GAhnNjoOaYuG93vDUDSl+ ugTF+1mRM+yyo2Y4zeWPd3RshDA9Fyivzy14e0zmHp0hAxel58NPIQC1m+8g/5JVuhso AQe1EJNMgQbYYuHjqWOkSYDB7kZvOQElUYKTFc4LJ1xOS9YT26/1QUczclgvfmmO2HV/ kJ2hKtF8WGuLCz7qhnttgp5Onyfc8nOekaohAl7RxULopkRpFS2KlN3hcDnw/WpPsZxW zwPtGUrODx8241JB61TUv3spof8OYwNG9db2giSddAtVanWijb/Xtq43a6ASFtE6HZDX WEGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pFttl+7mb3vlqjJiXTJxAK+LxJsUUATYj2v8abg1GY4=; b=Q6EEDkocNrTFzbmqoTwavUOHaYYZ4hFESQpEsnTq0VU0vNaPXGf21SLEkg1bTx1KSz 9cLdHfrX1mVQgGdDAnbqa2I+CUiQW8dTiUftXrFzQ5YTzs4xNwSpPneJVqHLCna6caiD Z7QR/copvmjpxEYxJa2aJiUDEzsF2YIiI/T7296z+1Hll8JKs+V5weqvrlPU19Hlnj2N PrnLkauLR7LeG0SjyofA1Bnkq5NorgDwpXTZCURyhQcra8+g85ABFwW+MBXBj2XjIwbq YPDNvluVMkiZruqGYJ4++7hg9tRXuaTeSMXWt58wAckYYi9y0fBnwVK8a812HgRAINjE X97w== X-Gm-Message-State: APjAAAVC/EWMMUG9xo7cNapxWlNxRnKB3OxjkX/I2wR/mcDHiAYj8kdQ MrDEn4ddPBLKJMTztePKoGd+guUpuogAe1Mg X-Received: by 2002:a19:5e4e:: with SMTP id z14mr12899379lfi.11.1562096534300; Tue, 02 Jul 2019 12:42:14 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.12 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:13 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 03/32] crypto: aes/fixed-time - align key schedule with other implementations Date: Tue, 2 Jul 2019 21:41:21 +0200 Message-Id: <20190702194150.10405-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The fixed time AES code mangles the key schedule so that xoring the first round key with values at fixed offsets across the Sbox produces the correct value. This primes the D-cache with the entire Sbox before any data dependent lookups are done, making it more difficult to infer key bits from timing variances when the plaintext is known. The downside of this approach is that it renders the key schedule incompatible with other implementations of AES in the kernel, which makes it cumbersome to use this implementation as a fallback for SIMD based AES in contexts where this is not allowed. So let's tweak the fixed Sbox indexes so that they add up to zero under the xor operation. While at it, increase the granularity to 16 bytes so we cover the entire Sbox even on systems with 16 byte cachelines. Signed-off-by: Ard Biesheuvel --- crypto/aes_ti.c | 52 ++++++++------------ 1 file changed, 21 insertions(+), 31 deletions(-) -- 2.17.1 diff --git a/crypto/aes_ti.c b/crypto/aes_ti.c index 1ff9785b30f5..fd70dc322634 100644 --- a/crypto/aes_ti.c +++ b/crypto/aes_ti.c @@ -237,30 +237,8 @@ static int aesti_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - int err; - err = aesti_expand_key(ctx, in_key, key_len); - if (err) - return err; - - /* - * In order to force the compiler to emit data independent Sbox lookups - * at the start of each block, xor the first round key with values at - * fixed indexes in the Sbox. This will need to be repeated each time - * the key is used, which will pull the entire Sbox into the D-cache - * before any data dependent Sbox lookups are performed. - */ - ctx->key_enc[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[128]; - ctx->key_enc[1] ^= __aesti_sbox[32] ^ __aesti_sbox[160]; - ctx->key_enc[2] ^= __aesti_sbox[64] ^ __aesti_sbox[192]; - ctx->key_enc[3] ^= __aesti_sbox[96] ^ __aesti_sbox[224]; - - ctx->key_dec[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[128]; - ctx->key_dec[1] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[160]; - ctx->key_dec[2] ^= __aesti_inv_sbox[64] ^ __aesti_inv_sbox[192]; - ctx->key_dec[3] ^= __aesti_inv_sbox[96] ^ __aesti_inv_sbox[224]; - - return 0; + return aesti_expand_key(ctx, in_key, key_len); } static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) @@ -283,10 +261,16 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[128]; - st0[1] ^= __aesti_sbox[32] ^ __aesti_sbox[160]; - st0[2] ^= __aesti_sbox[64] ^ __aesti_sbox[192]; - st0[3] ^= __aesti_sbox[96] ^ __aesti_sbox[224]; + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[ 64] ^ __aesti_sbox[134] ^ __aesti_sbox[195]; + st0[1] ^= __aesti_sbox[16] ^ __aesti_sbox[ 82] ^ __aesti_sbox[158] ^ __aesti_sbox[221]; + st0[2] ^= __aesti_sbox[32] ^ __aesti_sbox[ 96] ^ __aesti_sbox[160] ^ __aesti_sbox[234]; + st0[3] ^= __aesti_sbox[48] ^ __aesti_sbox[112] ^ __aesti_sbox[186] ^ __aesti_sbox[241]; for (round = 0;; round += 2, rkp += 8) { st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; @@ -331,10 +315,16 @@ static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[128]; - st0[1] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[160]; - st0[2] ^= __aesti_inv_sbox[64] ^ __aesti_inv_sbox[192]; - st0[3] ^= __aesti_inv_sbox[96] ^ __aesti_inv_sbox[224]; + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[ 64] ^ __aesti_inv_sbox[129] ^ __aesti_inv_sbox[200]; + st0[1] ^= __aesti_inv_sbox[16] ^ __aesti_inv_sbox[ 83] ^ __aesti_inv_sbox[150] ^ __aesti_inv_sbox[212]; + st0[2] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[ 96] ^ __aesti_inv_sbox[160] ^ __aesti_inv_sbox[236]; + st0[3] ^= __aesti_inv_sbox[48] ^ __aesti_inv_sbox[112] ^ __aesti_inv_sbox[187] ^ __aesti_inv_sbox[247]; for (round = 0;; round += 2, rkp += 8) { st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; From patchwork Tue Jul 2 19:41:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168351 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653329ilk; Tue, 2 Jul 2019 12:42:21 -0700 (PDT) X-Google-Smtp-Source: APXvYqyW5GQEJvv72K2br6gPEeEFCaWokMYjmZQCmcMOk9lTzyElLA/UY9UDqGTU8jLgQGXiQKPh X-Received: by 2002:a17:90b:8cd:: with SMTP id ds13mr7100966pjb.141.1562096541494; Tue, 02 Jul 2019 12:42:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096541; cv=none; d=google.com; s=arc-20160816; b=GbEt5PadE+Lzq0l5fQIgpJyuFEOFPenyVkX7K9+v6xP0Q5JN6bnMAJDDKtC+1ehn02 FZ/QoLhS2QzB6ggLY/2UdLyWbUZjpjLz2Lp5FGpRFqY+wYXujn43DEh8cNnUPRDd8t72 JuGjNGYNQYsw1C400Sta7gNhLnVMNHTmJseT8RZoRHsfVyltQu0//eA0bA54q2ut1Vlr IXxOP7e+Jy7Ocq93ENO5WT1jQ3VWi/MS1vmTV6oe8WehPldLLdfffGOEZoF9mbGk0E5q h9sm/98r8c1r2dHTG7IH6qPSz+QdOiNEzBPDG/DhJHNrfNMuGf0tnYuWDRI/aL9ceBT6 27JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=LOAcygW/KVLwYIUDb4jgiFBRafu2szSFFvDg4iJJfEc=; b=gavAiJ4Zkvu/wBBUSYUJ14l0wyTg3WHAeKuc31xkV0FqujShoZrn0rkFtREJf1EJbJ RhyHM1jtDVbIWV3H4MbE39k+HUTys7s3hSIZpim5xLiJI8DA4wblqBfdWJdaGtsg9hpT jQKs4NfKyWJ0txDw7EsttJtMczLrYNqHZWDPzX65gH2YNixKSh2dlOyZobGSk4pM+A0y ySpa32kOfwoNzn/i6BX5ez5B+Dn+9QrvOogllrom0lgcylKCbGmTdiaO9gHtzHabf5/r gldgAY55DuT1T62CNbDs3dWZ9zzHNyQn+1i6+7UDvnOj+a7M3SG5sM8Q01kFHf7OlJyN YwgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZJdwEdSR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.21; Tue, 02 Jul 2019 12:42:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZJdwEdSR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727036AbfGBTmU (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:20 -0400 Received: from mail-lf1-f65.google.com ([209.85.167.65]:41928 "EHLO mail-lf1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726291AbfGBTmU (ORCPT ); Tue, 2 Jul 2019 15:42:20 -0400 Received: by mail-lf1-f65.google.com with SMTP id 62so2219090lfa.8 for ; Tue, 02 Jul 2019 12:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LOAcygW/KVLwYIUDb4jgiFBRafu2szSFFvDg4iJJfEc=; b=ZJdwEdSRDvTSqELd3HZ8L25tR5/kwLQUJ5qaOO24PKkO5cCRGXvH19lq0irP+oC9nc WtEDs8G8WOrB9VOmTQXsV53DmDyLBlFtmpPSloiE0FtH/BC/47Du3B+LivHFdnpPwoVh L9xt/a+FwSyNSU1n3LXeJY63a4oUhLNb9YooaGMMqdvDLXZHGsEFEI8xIjEzAZwFyJOk iph5zsRDzAibeIoUd8UsFTDw4qFXj5eYKkXlTAiC6t8563fVvJ2HEkpMW09OAbM5dise zCYDdE+HThIxGERJYbsVmoQkb9B9R+1SkmxjxQshiuCg8r53BZyJiOurNouDFNi087BN oMjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LOAcygW/KVLwYIUDb4jgiFBRafu2szSFFvDg4iJJfEc=; b=X+toyte5suRyzLJMupiI6KAX8KSa+DJbeDZCPc1fkEPA7BjoYKHp0hAmDzmeP0MW0m iJJvaZrmYxkJEIX5fJMhL3BQDHhxlD6L3PBO42rjNxWaBNstnlQWbV+bTTYz5IOst4Gc FPeqTKdrBe5Dbm8z3GmpWg3qr0nqPYgQpn7AUJqapZTkS6Fj+087Ehlj26bEnMHkXzzt uOTlI3yEGyupwOZso9mTm6zQt/USlE3c+wu/Uij5+Nfr2jaOaovmbguPOlU/6TxF2woq xAuWrOpNVqy06NwOyeaF79NSKy+pVTrAfh5ZbmKA4tRsGTvVcHV0yxiTkM4lzAnu/szC 1y+A== X-Gm-Message-State: APjAAAUVNB7j6Z4G7QMc/TvO+OVlXo7GcSzNofuaUzXWOsQpilPQyGF8 I53tmU6Itm1SYaqlLx3x6EYmsk+izrOdYaYN X-Received: by 2002:ac2:569c:: with SMTP id 28mr15381877lfr.147.1562096535827; Tue, 02 Jul 2019 12:42:15 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.14 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:15 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 04/32] crypto: aes - create AES library based on the fixed time AES code Date: Tue, 2 Jul 2019 21:41:22 +0200 Message-Id: <20190702194150.10405-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Take the existing small footprint and mostly time invariant C code and turn it into a AES library that can be used for non-performance critical, casual use of AES, and as a fallback for, e.g., SIMD code that needs a secondary path that can be taken in contexts where the SIMD unit is off limits (e.g., in hard interrupts taken from kernel context) Signed-off-by: Ard Biesheuvel --- crypto/Kconfig | 4 + crypto/aes_ti.c | 307 +---------------- include/crypto/aes.h | 34 ++ lib/crypto/Makefile | 3 + lib/crypto/aes.c | 350 ++++++++++++++++++++ 5 files changed, 395 insertions(+), 303 deletions(-) -- 2.17.1 diff --git a/crypto/Kconfig b/crypto/Kconfig index e801450bcb1c..091ebbbc9655 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1066,6 +1066,9 @@ config CRYPTO_GHASH_CLMUL_NI_INTEL comment "Ciphers" +config CRYPTO_LIB_AES + tristate + config CRYPTO_AES tristate "AES cipher algorithms" select CRYPTO_ALGAPI @@ -1089,6 +1092,7 @@ config CRYPTO_AES config CRYPTO_AES_TI tristate "Fixed time AES cipher" select CRYPTO_ALGAPI + select CRYPTO_LIB_AES help This is a generic implementation of AES that attempts to eliminate data dependent latencies as much as possible without affecting diff --git a/crypto/aes_ti.c b/crypto/aes_ti.c index fd70dc322634..339915db9aeb 100644 --- a/crypto/aes_ti.c +++ b/crypto/aes_ti.c @@ -1,259 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 /* * Scalar fixed time AES core transform * * Copyright (C) 2017 Linaro Ltd - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. */ #include #include #include -#include - -/* - * Emit the sbox as volatile const to prevent the compiler from doing - * constant folding on sbox references involving fixed indexes. - */ -static volatile const u8 __cacheline_aligned __aesti_sbox[] = { - 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, - 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76, - 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, - 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, - 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, - 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15, - 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, - 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75, - 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, - 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, - 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, - 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf, - 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, - 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8, - 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, - 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, - 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, - 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73, - 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, - 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb, - 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, - 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, - 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, - 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08, - 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, - 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a, - 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, - 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, - 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, - 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf, - 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, - 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16, -}; - -static volatile const u8 __cacheline_aligned __aesti_inv_sbox[] = { - 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, - 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb, - 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, - 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb, - 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, - 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e, - 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, - 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25, - 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, - 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92, - 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, - 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84, - 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, - 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06, - 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, - 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b, - 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, - 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73, - 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, - 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e, - 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, - 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b, - 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, - 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4, - 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, - 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f, - 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, - 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef, - 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, - 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61, - 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, - 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, -}; - -static u32 mul_by_x(u32 w) -{ - u32 x = w & 0x7f7f7f7f; - u32 y = w & 0x80808080; - - /* multiply by polynomial 'x' (0b10) in GF(2^8) */ - return (x << 1) ^ (y >> 7) * 0x1b; -} - -static u32 mul_by_x2(u32 w) -{ - u32 x = w & 0x3f3f3f3f; - u32 y = w & 0x80808080; - u32 z = w & 0x40404040; - - /* multiply by polynomial 'x^2' (0b100) in GF(2^8) */ - return (x << 2) ^ (y >> 7) * 0x36 ^ (z >> 6) * 0x1b; -} - -static u32 mix_columns(u32 x) -{ - /* - * Perform the following matrix multiplication in GF(2^8) - * - * | 0x2 0x3 0x1 0x1 | | x[0] | - * | 0x1 0x2 0x3 0x1 | | x[1] | - * | 0x1 0x1 0x2 0x3 | x | x[2] | - * | 0x3 0x1 0x1 0x2 | | x[3] | - */ - u32 y = mul_by_x(x) ^ ror32(x, 16); - - return y ^ ror32(x ^ y, 8); -} - -static u32 inv_mix_columns(u32 x) -{ - /* - * Perform the following matrix multiplication in GF(2^8) - * - * | 0xe 0xb 0xd 0x9 | | x[0] | - * | 0x9 0xe 0xb 0xd | | x[1] | - * | 0xd 0x9 0xe 0xb | x | x[2] | - * | 0xb 0xd 0x9 0xe | | x[3] | - * - * which can conveniently be reduced to - * - * | 0x2 0x3 0x1 0x1 | | 0x5 0x0 0x4 0x0 | | x[0] | - * | 0x1 0x2 0x3 0x1 | | 0x0 0x5 0x0 0x4 | | x[1] | - * | 0x1 0x1 0x2 0x3 | x | 0x4 0x0 0x5 0x0 | x | x[2] | - * | 0x3 0x1 0x1 0x2 | | 0x0 0x4 0x0 0x5 | | x[3] | - */ - u32 y = mul_by_x2(x); - - return mix_columns(x ^ y ^ ror32(y, 16)); -} - -static __always_inline u32 subshift(u32 in[], int pos) -{ - return (__aesti_sbox[in[pos] & 0xff]) ^ - (__aesti_sbox[(in[(pos + 1) % 4] >> 8) & 0xff] << 8) ^ - (__aesti_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ - (__aesti_sbox[(in[(pos + 3) % 4] >> 24) & 0xff] << 24); -} - -static __always_inline u32 inv_subshift(u32 in[], int pos) -{ - return (__aesti_inv_sbox[in[pos] & 0xff]) ^ - (__aesti_inv_sbox[(in[(pos + 3) % 4] >> 8) & 0xff] << 8) ^ - (__aesti_inv_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ - (__aesti_inv_sbox[(in[(pos + 1) % 4] >> 24) & 0xff] << 24); -} - -static u32 subw(u32 in) -{ - return (__aesti_sbox[in & 0xff]) ^ - (__aesti_sbox[(in >> 8) & 0xff] << 8) ^ - (__aesti_sbox[(in >> 16) & 0xff] << 16) ^ - (__aesti_sbox[(in >> 24) & 0xff] << 24); -} - -static int aesti_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len) -{ - u32 kwords = key_len / sizeof(u32); - u32 rc, i, j; - - if (key_len != AES_KEYSIZE_128 && - key_len != AES_KEYSIZE_192 && - key_len != AES_KEYSIZE_256) - return -EINVAL; - - ctx->key_length = key_len; - - for (i = 0; i < kwords; i++) - ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); - - for (i = 0, rc = 1; i < 10; i++, rc = mul_by_x(rc)) { - u32 *rki = ctx->key_enc + (i * kwords); - u32 *rko = rki + kwords; - - rko[0] = ror32(subw(rki[kwords - 1]), 8) ^ rc ^ rki[0]; - rko[1] = rko[0] ^ rki[1]; - rko[2] = rko[1] ^ rki[2]; - rko[3] = rko[2] ^ rki[3]; - - if (key_len == 24) { - if (i >= 7) - break; - rko[4] = rko[3] ^ rki[4]; - rko[5] = rko[4] ^ rki[5]; - } else if (key_len == 32) { - if (i >= 6) - break; - rko[4] = subw(rko[3]) ^ rki[4]; - rko[5] = rko[4] ^ rki[5]; - rko[6] = rko[5] ^ rki[6]; - rko[7] = rko[6] ^ rki[7]; - } - } - - /* - * Generate the decryption keys for the Equivalent Inverse Cipher. - * This involves reversing the order of the round keys, and applying - * the Inverse Mix Columns transformation to all but the first and - * the last one. - */ - ctx->key_dec[0] = ctx->key_enc[key_len + 24]; - ctx->key_dec[1] = ctx->key_enc[key_len + 25]; - ctx->key_dec[2] = ctx->key_enc[key_len + 26]; - ctx->key_dec[3] = ctx->key_enc[key_len + 27]; - - for (i = 4, j = key_len + 20; j > 0; i += 4, j -= 4) { - ctx->key_dec[i] = inv_mix_columns(ctx->key_enc[j]); - ctx->key_dec[i + 1] = inv_mix_columns(ctx->key_enc[j + 1]); - ctx->key_dec[i + 2] = inv_mix_columns(ctx->key_enc[j + 2]); - ctx->key_dec[i + 3] = inv_mix_columns(ctx->key_enc[j + 3]); - } - ctx->key_dec[i] = ctx->key_enc[0]; - ctx->key_dec[i + 1] = ctx->key_enc[1]; - ctx->key_dec[i + 2] = ctx->key_enc[2]; - ctx->key_dec[i + 3] = ctx->key_enc[3]; - - return 0; -} static int aesti_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len) { struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - return aesti_expand_key(ctx, in_key, key_len); + return aes_expandkey(ctx, in_key, key_len); } static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - const u32 *rkp = ctx->key_enc + 4; - int rounds = 6 + ctx->key_length / 4; - u32 st0[4], st1[4]; unsigned long flags; - int round; - - st0[0] = ctx->key_enc[0] ^ get_unaligned_le32(in); - st0[1] = ctx->key_enc[1] ^ get_unaligned_le32(in + 4); - st0[2] = ctx->key_enc[2] ^ get_unaligned_le32(in + 8); - st0[3] = ctx->key_enc[3] ^ get_unaligned_le32(in + 12); /* * Temporarily disable interrupts to avoid races where cachelines are @@ -261,36 +29,7 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - /* - * Force the compiler to emit data independent Sbox references, - * by xoring the input with Sbox values that are known to add up - * to zero. This pulls the entire Sbox into the D-cache before any - * data dependent lookups are done. - */ - st0[0] ^= __aesti_sbox[ 0] ^ __aesti_sbox[ 64] ^ __aesti_sbox[134] ^ __aesti_sbox[195]; - st0[1] ^= __aesti_sbox[16] ^ __aesti_sbox[ 82] ^ __aesti_sbox[158] ^ __aesti_sbox[221]; - st0[2] ^= __aesti_sbox[32] ^ __aesti_sbox[ 96] ^ __aesti_sbox[160] ^ __aesti_sbox[234]; - st0[3] ^= __aesti_sbox[48] ^ __aesti_sbox[112] ^ __aesti_sbox[186] ^ __aesti_sbox[241]; - - for (round = 0;; round += 2, rkp += 8) { - st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; - st1[1] = mix_columns(subshift(st0, 1)) ^ rkp[1]; - st1[2] = mix_columns(subshift(st0, 2)) ^ rkp[2]; - st1[3] = mix_columns(subshift(st0, 3)) ^ rkp[3]; - - if (round == rounds - 2) - break; - - st0[0] = mix_columns(subshift(st1, 0)) ^ rkp[4]; - st0[1] = mix_columns(subshift(st1, 1)) ^ rkp[5]; - st0[2] = mix_columns(subshift(st1, 2)) ^ rkp[6]; - st0[3] = mix_columns(subshift(st1, 3)) ^ rkp[7]; - } - - put_unaligned_le32(subshift(st1, 0) ^ rkp[4], out); - put_unaligned_le32(subshift(st1, 1) ^ rkp[5], out + 4); - put_unaligned_le32(subshift(st1, 2) ^ rkp[6], out + 8); - put_unaligned_le32(subshift(st1, 3) ^ rkp[7], out + 12); + aes_encrypt(ctx, out, in); local_irq_restore(flags); } @@ -298,16 +37,7 @@ static void aesti_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { const struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); - const u32 *rkp = ctx->key_dec + 4; - int rounds = 6 + ctx->key_length / 4; - u32 st0[4], st1[4]; unsigned long flags; - int round; - - st0[0] = ctx->key_dec[0] ^ get_unaligned_le32(in); - st0[1] = ctx->key_dec[1] ^ get_unaligned_le32(in + 4); - st0[2] = ctx->key_dec[2] ^ get_unaligned_le32(in + 8); - st0[3] = ctx->key_dec[3] ^ get_unaligned_le32(in + 12); /* * Temporarily disable interrupts to avoid races where cachelines are @@ -315,36 +45,7 @@ static void aesti_decrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ local_irq_save(flags); - /* - * Force the compiler to emit data independent Sbox references, - * by xoring the input with Sbox values that are known to add up - * to zero. This pulls the entire Sbox into the D-cache before any - * data dependent lookups are done. - */ - st0[0] ^= __aesti_inv_sbox[ 0] ^ __aesti_inv_sbox[ 64] ^ __aesti_inv_sbox[129] ^ __aesti_inv_sbox[200]; - st0[1] ^= __aesti_inv_sbox[16] ^ __aesti_inv_sbox[ 83] ^ __aesti_inv_sbox[150] ^ __aesti_inv_sbox[212]; - st0[2] ^= __aesti_inv_sbox[32] ^ __aesti_inv_sbox[ 96] ^ __aesti_inv_sbox[160] ^ __aesti_inv_sbox[236]; - st0[3] ^= __aesti_inv_sbox[48] ^ __aesti_inv_sbox[112] ^ __aesti_inv_sbox[187] ^ __aesti_inv_sbox[247]; - - for (round = 0;; round += 2, rkp += 8) { - st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; - st1[1] = inv_mix_columns(inv_subshift(st0, 1)) ^ rkp[1]; - st1[2] = inv_mix_columns(inv_subshift(st0, 2)) ^ rkp[2]; - st1[3] = inv_mix_columns(inv_subshift(st0, 3)) ^ rkp[3]; - - if (round == rounds - 2) - break; - - st0[0] = inv_mix_columns(inv_subshift(st1, 0)) ^ rkp[4]; - st0[1] = inv_mix_columns(inv_subshift(st1, 1)) ^ rkp[5]; - st0[2] = inv_mix_columns(inv_subshift(st1, 2)) ^ rkp[6]; - st0[3] = inv_mix_columns(inv_subshift(st1, 3)) ^ rkp[7]; - } - - put_unaligned_le32(inv_subshift(st1, 0) ^ rkp[4], out); - put_unaligned_le32(inv_subshift(st1, 1) ^ rkp[5], out + 4); - put_unaligned_le32(inv_subshift(st1, 2) ^ rkp[6], out + 8); - put_unaligned_le32(inv_subshift(st1, 3) ^ rkp[7], out + 12); + aes_decrypt(ctx, out, in); local_irq_restore(flags); } diff --git a/include/crypto/aes.h b/include/crypto/aes.h index 0fdb542c70cd..d0067fca0cd0 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -37,4 +37,38 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, unsigned int key_len); + +/** + * aes_expandkey - Expands the AES key as described in FIPS-197 + * @ctx: The location where the computed key will be stored. + * @in_key: The supplied key. + * @key_len: The length of the supplied key. + * + * Returns 0 on success. The function fails only if an invalid key size (or + * pointer) is supplied. + * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes + * key schedule plus a 16 bytes key which is used before the first round). + * The decryption key is prepared for the "Equivalent Inverse Cipher" as + * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is + * for the initial combination, the second slot for the first round and so on. + */ +int aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, + unsigned int key_len); + +/** + * aes_encrypt - Encrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the ciphertext + * @in: Buffer containing the plaintext + */ +void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); + +/** + * aes_decrypt - Decrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the plaintext + * @in: Buffer containing the ciphertext + */ +void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); + #endif diff --git a/lib/crypto/Makefile b/lib/crypto/Makefile index 88195c34932d..42a91c62d96d 100644 --- a/lib/crypto/Makefile +++ b/lib/crypto/Makefile @@ -1,4 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_CRYPTO_LIB_AES) += libaes.o +libaes-y := aes.o + obj-$(CONFIG_CRYPTO_LIB_ARC4) += libarc4.o libarc4-y := arc4.o diff --git a/lib/crypto/aes.c b/lib/crypto/aes.c new file mode 100644 index 000000000000..9928b23e0a8a --- /dev/null +++ b/lib/crypto/aes.c @@ -0,0 +1,350 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2017-2019 Linaro Ltd + */ + +#include +#include +#include +#include + +/* + * Emit the sbox as volatile const to prevent the compiler from doing + * constant folding on sbox references involving fixed indexes. + */ +static volatile const u8 __cacheline_aligned aes_sbox[] = { + 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5, + 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76, + 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0, + 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0, + 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc, + 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15, + 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a, + 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75, + 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0, + 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84, + 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b, + 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf, + 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85, + 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8, + 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5, + 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2, + 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17, + 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73, + 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88, + 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb, + 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c, + 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79, + 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9, + 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08, + 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6, + 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a, + 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e, + 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e, + 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94, + 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf, + 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68, + 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16, +}; + +static volatile const u8 __cacheline_aligned aes_inv_sbox[] = { + 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38, + 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb, + 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87, + 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb, + 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d, + 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e, + 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2, + 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25, + 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16, + 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92, + 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda, + 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84, + 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a, + 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06, + 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02, + 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b, + 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea, + 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73, + 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85, + 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e, + 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89, + 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b, + 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20, + 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4, + 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31, + 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f, + 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d, + 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef, + 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0, + 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61, + 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26, + 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, +}; + +static u32 mul_by_x(u32 w) +{ + u32 x = w & 0x7f7f7f7f; + u32 y = w & 0x80808080; + + /* multiply by polynomial 'x' (0b10) in GF(2^8) */ + return (x << 1) ^ (y >> 7) * 0x1b; +} + +static u32 mul_by_x2(u32 w) +{ + u32 x = w & 0x3f3f3f3f; + u32 y = w & 0x80808080; + u32 z = w & 0x40404040; + + /* multiply by polynomial 'x^2' (0b100) in GF(2^8) */ + return (x << 2) ^ (y >> 7) * 0x36 ^ (z >> 6) * 0x1b; +} + +static u32 mix_columns(u32 x) +{ + /* + * Perform the following matrix multiplication in GF(2^8) + * + * | 0x2 0x3 0x1 0x1 | | x[0] | + * | 0x1 0x2 0x3 0x1 | | x[1] | + * | 0x1 0x1 0x2 0x3 | x | x[2] | + * | 0x3 0x1 0x1 0x2 | | x[3] | + */ + u32 y = mul_by_x(x) ^ ror32(x, 16); + + return y ^ ror32(x ^ y, 8); +} + +static u32 inv_mix_columns(u32 x) +{ + /* + * Perform the following matrix multiplication in GF(2^8) + * + * | 0xe 0xb 0xd 0x9 | | x[0] | + * | 0x9 0xe 0xb 0xd | | x[1] | + * | 0xd 0x9 0xe 0xb | x | x[2] | + * | 0xb 0xd 0x9 0xe | | x[3] | + * + * which can conveniently be reduced to + * + * | 0x2 0x3 0x1 0x1 | | 0x5 0x0 0x4 0x0 | | x[0] | + * | 0x1 0x2 0x3 0x1 | | 0x0 0x5 0x0 0x4 | | x[1] | + * | 0x1 0x1 0x2 0x3 | x | 0x4 0x0 0x5 0x0 | x | x[2] | + * | 0x3 0x1 0x1 0x2 | | 0x0 0x4 0x0 0x5 | | x[3] | + */ + u32 y = mul_by_x2(x); + + return mix_columns(x ^ y ^ ror32(y, 16)); +} + +static __always_inline u32 subshift(u32 in[], int pos) +{ + return (aes_sbox[in[pos] & 0xff]) ^ + (aes_sbox[(in[(pos + 1) % 4] >> 8) & 0xff] << 8) ^ + (aes_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ + (aes_sbox[(in[(pos + 3) % 4] >> 24) & 0xff] << 24); +} + +static __always_inline u32 inv_subshift(u32 in[], int pos) +{ + return (aes_inv_sbox[in[pos] & 0xff]) ^ + (aes_inv_sbox[(in[(pos + 3) % 4] >> 8) & 0xff] << 8) ^ + (aes_inv_sbox[(in[(pos + 2) % 4] >> 16) & 0xff] << 16) ^ + (aes_inv_sbox[(in[(pos + 1) % 4] >> 24) & 0xff] << 24); +} + +static u32 subw(u32 in) +{ + return (aes_sbox[in & 0xff]) ^ + (aes_sbox[(in >> 8) & 0xff] << 8) ^ + (aes_sbox[(in >> 16) & 0xff] << 16) ^ + (aes_sbox[(in >> 24) & 0xff] << 24); +} + +/** + * aes_expandkey - Expands the AES key as described in FIPS-197 + * @ctx: The location where the computed key will be stored. + * @in_key: The supplied key. + * @key_len: The length of the supplied key. + * + * Returns 0 on success. The function fails only if an invalid key size (or + * pointer) is supplied. + * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes + * key schedule plus a 16 bytes key which is used before the first round). + * The decryption key is prepared for the "Equivalent Inverse Cipher" as + * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is + * for the initial combination, the second slot for the first round and so on. + */ +int aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, + unsigned int key_len) +{ + u32 kwords = key_len / sizeof(u32); + u32 rc, i, j; + + if (key_len != AES_KEYSIZE_128 && + key_len != AES_KEYSIZE_192 && + key_len != AES_KEYSIZE_256) + return -EINVAL; + + ctx->key_length = key_len; + + for (i = 0; i < kwords; i++) + ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); + + for (i = 0, rc = 1; i < 10; i++, rc = mul_by_x(rc)) { + u32 *rki = ctx->key_enc + (i * kwords); + u32 *rko = rki + kwords; + + rko[0] = ror32(subw(rki[kwords - 1]), 8) ^ rc ^ rki[0]; + rko[1] = rko[0] ^ rki[1]; + rko[2] = rko[1] ^ rki[2]; + rko[3] = rko[2] ^ rki[3]; + + if (key_len == AES_KEYSIZE_192) { + if (i >= 7) + break; + rko[4] = rko[3] ^ rki[4]; + rko[5] = rko[4] ^ rki[5]; + } else if (key_len == AES_KEYSIZE_256) { + if (i >= 6) + break; + rko[4] = subw(rko[3]) ^ rki[4]; + rko[5] = rko[4] ^ rki[5]; + rko[6] = rko[5] ^ rki[6]; + rko[7] = rko[6] ^ rki[7]; + } + } + + /* + * Generate the decryption keys for the Equivalent Inverse Cipher. + * This involves reversing the order of the round keys, and applying + * the Inverse Mix Columns transformation to all but the first and + * the last one. + */ + ctx->key_dec[0] = ctx->key_enc[key_len + 24]; + ctx->key_dec[1] = ctx->key_enc[key_len + 25]; + ctx->key_dec[2] = ctx->key_enc[key_len + 26]; + ctx->key_dec[3] = ctx->key_enc[key_len + 27]; + + for (i = 4, j = key_len + 20; j > 0; i += 4, j -= 4) { + ctx->key_dec[i] = inv_mix_columns(ctx->key_enc[j]); + ctx->key_dec[i + 1] = inv_mix_columns(ctx->key_enc[j + 1]); + ctx->key_dec[i + 2] = inv_mix_columns(ctx->key_enc[j + 2]); + ctx->key_dec[i + 3] = inv_mix_columns(ctx->key_enc[j + 3]); + } + + ctx->key_dec[i] = ctx->key_enc[0]; + ctx->key_dec[i + 1] = ctx->key_enc[1]; + ctx->key_dec[i + 2] = ctx->key_enc[2]; + ctx->key_dec[i + 3] = ctx->key_enc[3]; + + return 0; +} +EXPORT_SYMBOL(aes_expandkey); + +/** + * aes_encrypt - Encrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the ciphertext + * @in: Buffer containing the plaintext + */ +void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in) +{ + const u32 *rkp = ctx->key_enc + 4; + int rounds = 6 + ctx->key_length / 4; + u32 st0[4], st1[4]; + int round; + + st0[0] = ctx->key_enc[0] ^ get_unaligned_le32(in); + st0[1] = ctx->key_enc[1] ^ get_unaligned_le32(in + 4); + st0[2] = ctx->key_enc[2] ^ get_unaligned_le32(in + 8); + st0[3] = ctx->key_enc[3] ^ get_unaligned_le32(in + 12); + + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= aes_sbox[ 0] ^ aes_sbox[ 64] ^ aes_sbox[134] ^ aes_sbox[195]; + st0[1] ^= aes_sbox[16] ^ aes_sbox[ 82] ^ aes_sbox[158] ^ aes_sbox[221]; + st0[2] ^= aes_sbox[32] ^ aes_sbox[ 96] ^ aes_sbox[160] ^ aes_sbox[234]; + st0[3] ^= aes_sbox[48] ^ aes_sbox[112] ^ aes_sbox[186] ^ aes_sbox[241]; + + for (round = 0;; round += 2, rkp += 8) { + st1[0] = mix_columns(subshift(st0, 0)) ^ rkp[0]; + st1[1] = mix_columns(subshift(st0, 1)) ^ rkp[1]; + st1[2] = mix_columns(subshift(st0, 2)) ^ rkp[2]; + st1[3] = mix_columns(subshift(st0, 3)) ^ rkp[3]; + + if (round == rounds - 2) + break; + + st0[0] = mix_columns(subshift(st1, 0)) ^ rkp[4]; + st0[1] = mix_columns(subshift(st1, 1)) ^ rkp[5]; + st0[2] = mix_columns(subshift(st1, 2)) ^ rkp[6]; + st0[3] = mix_columns(subshift(st1, 3)) ^ rkp[7]; + } + + put_unaligned_le32(subshift(st1, 0) ^ rkp[4], out); + put_unaligned_le32(subshift(st1, 1) ^ rkp[5], out + 4); + put_unaligned_le32(subshift(st1, 2) ^ rkp[6], out + 8); + put_unaligned_le32(subshift(st1, 3) ^ rkp[7], out + 12); +} +EXPORT_SYMBOL(aes_encrypt); + +/** + * aes_decrypt - Decrypt a single AES block + * @ctx: Context struct containing the key schedule + * @out: Buffer to store the plaintext + * @in: Buffer containing the ciphertext + */ +void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in) +{ + const u32 *rkp = ctx->key_dec + 4; + int rounds = 6 + ctx->key_length / 4; + u32 st0[4], st1[4]; + int round; + + st0[0] = ctx->key_dec[0] ^ get_unaligned_le32(in); + st0[1] = ctx->key_dec[1] ^ get_unaligned_le32(in + 4); + st0[2] = ctx->key_dec[2] ^ get_unaligned_le32(in + 8); + st0[3] = ctx->key_dec[3] ^ get_unaligned_le32(in + 12); + + /* + * Force the compiler to emit data independent Sbox references, + * by xoring the input with Sbox values that are known to add up + * to zero. This pulls the entire Sbox into the D-cache before any + * data dependent lookups are done. + */ + st0[0] ^= aes_inv_sbox[ 0] ^ aes_inv_sbox[ 64] ^ aes_inv_sbox[129] ^ aes_inv_sbox[200]; + st0[1] ^= aes_inv_sbox[16] ^ aes_inv_sbox[ 83] ^ aes_inv_sbox[150] ^ aes_inv_sbox[212]; + st0[2] ^= aes_inv_sbox[32] ^ aes_inv_sbox[ 96] ^ aes_inv_sbox[160] ^ aes_inv_sbox[236]; + st0[3] ^= aes_inv_sbox[48] ^ aes_inv_sbox[112] ^ aes_inv_sbox[187] ^ aes_inv_sbox[247]; + + for (round = 0;; round += 2, rkp += 8) { + st1[0] = inv_mix_columns(inv_subshift(st0, 0)) ^ rkp[0]; + st1[1] = inv_mix_columns(inv_subshift(st0, 1)) ^ rkp[1]; + st1[2] = inv_mix_columns(inv_subshift(st0, 2)) ^ rkp[2]; + st1[3] = inv_mix_columns(inv_subshift(st0, 3)) ^ rkp[3]; + + if (round == rounds - 2) + break; + + st0[0] = inv_mix_columns(inv_subshift(st1, 0)) ^ rkp[4]; + st0[1] = inv_mix_columns(inv_subshift(st1, 1)) ^ rkp[5]; + st0[2] = inv_mix_columns(inv_subshift(st1, 2)) ^ rkp[6]; + st0[3] = inv_mix_columns(inv_subshift(st1, 3)) ^ rkp[7]; + } + + put_unaligned_le32(inv_subshift(st1, 0) ^ rkp[4], out); + put_unaligned_le32(inv_subshift(st1, 1) ^ rkp[5], out + 4); + put_unaligned_le32(inv_subshift(st1, 2) ^ rkp[6], out + 8); + put_unaligned_le32(inv_subshift(st1, 3) ^ rkp[7], out + 12); +} +EXPORT_SYMBOL(aes_decrypt); + +MODULE_DESCRIPTION("Generic AES library"); +MODULE_AUTHOR("Ard Biesheuvel "); +MODULE_LICENSE("GPL v2"); From patchwork Tue Jul 2 19:41:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168350 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653312ilk; Tue, 2 Jul 2019 12:42:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqyd8VkfyZhjlCzzCh8m6XQazi/Z7fg8a/JHznTk6SeTd9AgmLbp/k245waID2nrQWSuj+CB X-Received: by 2002:a17:902:2865:: with SMTP id e92mr36672241plb.264.1562096540565; Tue, 02 Jul 2019 12:42:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096540; cv=none; d=google.com; s=arc-20160816; b=u1x9HL1mjDYWdaIKnni/I2yzLGjj73OcDpe2XHLuVsaasYiyF311SQ+ZV5yVne/3tT 4/n3aelNQBbZ4lkvbtheOoRBS7idh7i5gNwvkpGdvTvm5kfW+9DN49bzro/pibAb7JLH 6M2xhgJyeZEo0GFnqgF3dk3QEedBiz/XHz63BK1Eg7gYpAyebUVIms0hHwtNF4rl1Fdk kiicFgsGLTvz3l8ZB26ZFf2nwpMmlBmTGwmZcRxK5v0MLbgmK0hZAq+CR5ifUwv4WPut hPGaQEiMXeHKQsL6kZJUVYtugphtxqpc7bGvdgu/K9XGnSm+BqYfvSlR+/H1usopQ14g KzTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=nWt11EyVqqg4tMlaa7Rrh/54/0WBS7GqXZGnOHalCUs=; b=iczdXgte+6BJ4LLnYP53Z7+52SCyt4jgVV6yu8vbRdf+fpVcd2aAGZzv6bG11pm2vf xIAeohXhIVn4WmYv1qDvTB30rhxVnkckwdS2T3zdtrgPYiI1tcS8mU2Umh+Edire3BZE e/hwj9Y4XJrePSVsX5CQrSjCnhu0UkirjfCio3InHLGRdzvqB2ia20GSwAT3LcAWpW4A 37QXm01zZE21ivxbv6meN+wCHQ/DBr4xwi51ThFweIr8301OoFsC5t/yvb7bu/N1Te8W 6906pJMIWH0upPlW5+iunFokg7o7TqnuhB2CiqeVGs3rD1+x+6rdBGavjjjVYltdxG79 rjmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fdjFlEQE; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.20; Tue, 02 Jul 2019 12:42:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fdjFlEQE; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726977AbfGBTmT (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:19 -0400 Received: from mail-lf1-f67.google.com ([209.85.167.67]:39107 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726963AbfGBTmT (ORCPT ); Tue, 2 Jul 2019 15:42:19 -0400 Received: by mail-lf1-f67.google.com with SMTP id p24so12264347lfo.6 for ; Tue, 02 Jul 2019 12:42:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nWt11EyVqqg4tMlaa7Rrh/54/0WBS7GqXZGnOHalCUs=; b=fdjFlEQE1xzcolJvutPoxozOxZTrtdgx9EtUZBqOQrW89Qg1Uw/T+KtAK5LugFEcrY jx7RI9pRVYlPM68Mw/FOiUdoquk6lSc6k7qcke7lhTFMfwo/T6pVUn7pg8FkwveiHgR5 yuC0hSyXtW9uvMjLe661Mw9dux14G3RLLJyU9U58L+3PlusSBMeEGmIb/jnexhebKgbS 8jZJOrIAOJa8l4tlVTEIafjhztSdis4QLRK6S/lCOLdfr+e81jIMZ4pxFKTK6rL0ET/J y4weDt3pZu3pOK2s9UL22Ys6Lglo6eMZu11Dzr5VIYG5QWNkfoML4H2onUYkJ/DLn/Rk 0Vgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nWt11EyVqqg4tMlaa7Rrh/54/0WBS7GqXZGnOHalCUs=; b=XScPXsTVMvA9uy+8O+4ufV+0+nux4Nz84gn3e5qSa6ols3mjyddPn+ocJiV0FlNkTm XcPqv0MIdiAJ5Decdbo/dAxKLSJUhbfrT9vBD1tJthUh+YfRPg+yvYtHrrsmo6bfEyJX +CkgrIqRZNZ1BgWn0w4xxrc5dCjyimiqz7X+6w+L6F4k+w90AX/B4zOXbFFMK/iClmBS tT+/m9cF9vyRy+ctU/r4syRANtX6evinfiTT7pnRCv9ZJsQKjAub2940Xkus2skxZZhb bk7xNbPW9/+SzDIQsJxSzaRq68lTfQ5L1ispfOawKyMt/MyWz8+aoI8Zr0q0Uondq00l PHkA== X-Gm-Message-State: APjAAAVMAhQhG8OJPf/LXT21SQVmWMYqkqy0uoBnzoPT4GqJk+1pA1Ao zXt7ygsoxHWuNkn4jG+h+nSxQbMqinNLGbO1 X-Received: by 2002:ac2:4d1c:: with SMTP id r28mr14866508lfi.159.1562096537400; Tue, 02 Jul 2019 12:42:17 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.15 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:16 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 05/32] crypto: x86/aes-ni - switch to generic for fallback and key routines Date: Tue, 2 Jul 2019 21:41:23 +0200 Message-Id: <20190702194150.10405-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AES-NI code contains fallbacks for invocations that occur from a context where the SIMD unit is unavailable, which really only occurs when running in softirq context that was entered from a hard IRQ that was taken while running kernel code that was already using the FPU. That means performance is not really a consideration, and we can just use the new library code for this use case, which has a smaller footprint and is believed to be time invariant. This will allow us to drop the non-SIMD asm routines in a subsequent patch. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/aesni-intel_glue.c | 15 +++++++-------- arch/x86/include/asm/crypto/aes.h | 12 ------------ crypto/Kconfig | 3 +-- 3 files changed, 8 insertions(+), 22 deletions(-) -- 2.17.1 diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c index 836d50bd096f..42873c1f6bb4 100644 --- a/arch/x86/crypto/aesni-intel_glue.c +++ b/arch/x86/crypto/aesni-intel_glue.c @@ -30,7 +30,6 @@ #include #include #include -#include #include #include #include @@ -333,7 +332,7 @@ static int aes_set_key_common(struct crypto_tfm *tfm, void *raw_ctx, } if (!crypto_simd_usable()) - err = crypto_aes_expand_key(ctx, in_key, key_len); + err = aes_expandkey(ctx, in_key, key_len); else { kernel_fpu_begin(); err = aesni_set_key(ctx, in_key, key_len); @@ -353,9 +352,9 @@ static void aesni_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); - if (!crypto_simd_usable()) - crypto_aes_encrypt_x86(ctx, dst, src); - else { + if (!crypto_simd_usable()) { + aes_encrypt(ctx, dst, src); + } else { kernel_fpu_begin(); aesni_enc(ctx, dst, src); kernel_fpu_end(); @@ -366,9 +365,9 @@ static void aesni_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) { struct crypto_aes_ctx *ctx = aes_ctx(crypto_tfm_ctx(tfm)); - if (!crypto_simd_usable()) - crypto_aes_decrypt_x86(ctx, dst, src); - else { + if (!crypto_simd_usable()) { + aes_decrypt(ctx, dst, src); + } else { kernel_fpu_begin(); aesni_dec(ctx, dst, src); kernel_fpu_end(); diff --git a/arch/x86/include/asm/crypto/aes.h b/arch/x86/include/asm/crypto/aes.h deleted file mode 100644 index c508521dd190..000000000000 --- a/arch/x86/include/asm/crypto/aes.h +++ /dev/null @@ -1,12 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef ASM_X86_AES_H -#define ASM_X86_AES_H - -#include -#include - -void crypto_aes_encrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, - const u8 *src); -void crypto_aes_decrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, - const u8 *src); -#endif diff --git a/crypto/Kconfig b/crypto/Kconfig index 091ebbbc9655..20af58068e6b 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1156,8 +1156,7 @@ config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 select CRYPTO_AEAD - select CRYPTO_AES_X86_64 if 64BIT - select CRYPTO_AES_586 if !64BIT + select CRYPTO_LIB_AES select CRYPTO_ALGAPI select CRYPTO_BLKCIPHER select CRYPTO_GLUE_HELPER_X86 if 64BIT From patchwork Tue Jul 2 19:41:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168354 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653379ilk; Tue, 2 Jul 2019 12:42:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqz0nhNM+LNyHLZLpTFW/AM+Wvh1CcNPxR+K3suPPaCza3iFBjxqNWI5GgB4nW9zRCrtQwEb X-Received: by 2002:a65:41c6:: with SMTP id b6mr2243849pgq.269.1562096544201; Tue, 02 Jul 2019 12:42:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096544; cv=none; d=google.com; s=arc-20160816; b=EOxrWNxEIXdvmZldgHJWLoA43uE3tvBncFt1E5oOmx+09q0eTzXNwy8eZARPr0e9Ot j7xQYqZbZysXL6OF0Vmq/dhf90bkhuzdnAAEgsr76Ss3jQ+XMayydggL0EntIYaQtL7q hziyvvMyROJzLKhm50lUyDHK1VxmD86bbd9VovVRiLKfgWCO2Pubc5iYn4jysINAO2xU ZKE/9qCWccN9jl6O1jzd9tJqy7ZyWMgXls9r/IBoY5oyF0VddbgUaotq4DOqp9TV5tAy jIBg/cf7/VoJVDxtn/nfNUZWHEjzfS8b98XsE6vNcH3FdgJ41h9P7GGC5Y9duditJb2r BgEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=h9ZOoG85itijFg1VGBYzI+TZx5pf6yqUz4SbNH/wVdE=; b=BQ8kax7nfJXT20mTSBuymgpWRlRIrAOfIfWtmJ15wwfjCQq7cZav93zFT5/NSld012 SqKtP3CFIe273YlBii8i6af914UgspCS3Mmd3NRTQmC0NbBsYpuptmnBnXVI1d+HkL9Q h3Pc4BGnTrnDQt8kNe/SvaPCBDa6eGOpX0iKfXaBhGV4sQUCM+lwckUvQ9Fjm5wG8VBy xCvzAfl3UFTN6/8trSLywAwBx2wXvLxW35zpI3CRu99f5zGzj+aK1ko1MTZDS53LdXDW 18nJk/wy4bvnP/thhw49uQ48Ky2fEya/ypQ5viiHejjyiquu+elACH0Ysw3UGL1nV48U dCCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iYWQxGWz; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.24; Tue, 02 Jul 2019 12:42:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iYWQxGWz; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726291AbfGBTmX (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:23 -0400 Received: from mail-lf1-f65.google.com ([209.85.167.65]:35622 "EHLO mail-lf1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726963AbfGBTmX (ORCPT ); Tue, 2 Jul 2019 15:42:23 -0400 Received: by mail-lf1-f65.google.com with SMTP id p197so1893199lfa.2 for ; Tue, 02 Jul 2019 12:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=h9ZOoG85itijFg1VGBYzI+TZx5pf6yqUz4SbNH/wVdE=; b=iYWQxGWz2b4512UQWjTzBp/9YsKVZhQSIN3QcnmXGFOo3aDp4QeQQ7scWYM4vkrxOw h4TuVbutePjytwdjsMqWsWNl7DijQpzcWWzpcyg4TqPSPaD9cF+I7dY2FOqQnGFeyT17 RGdhQzpliZtwaSl4Jq5HHfqk1FlfVsC4+vbIXYCWm/DMe78K12h+kSKitQ4nos4ugJTG olt5hY7GFXsLJ/n2RkC1DT5FQ4dnrJ5kc2evM7LdafDv2uI2hCn/N2WMY+PnOW/znmSu HloqT5mqlixP6ePTm9jwyupgsOdDgy1FuJm8QS/tKKjYfFi0pAbRIeDjE8BQWvx+tFa/ HVUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=h9ZOoG85itijFg1VGBYzI+TZx5pf6yqUz4SbNH/wVdE=; b=SLKVIEkZvnkpPCLOxY+w701DS1vDOga0Zt9fFlfLje1+pMaqUeCenS9WO5xdcfja9V 66uhV15fgDpvfgUKrIp9LGekauqMWakvjBZxfBxzt/yluyOmrcmOVW+/+t15Df2UhDwZ VFNXsBkq7R4L5i8xE0MvS2o/7xPZbAQmEXZaCKR0p0NT32m6cmM3SDMkosKXAkoBgyKd njbqsnCEpKNIpcmwpt4tW9+j94RoiCY7GWZ+xGZWb3J4LvFhSjwPGiErCmQ3M3HHzHUq we2pMy6tzPS7AEbTgNr15YXM9CYNpy2HBSkUt6oJcp6ZL2zzLpZCQB2vuY9WpJFbQtio vi7A== X-Gm-Message-State: APjAAAV0SiJoJ3dwlTQfihVght9cjj+sks++o/qqeF5b+9mxneVBgbFq PSmA0PFK87KiqoiKGsOfAdBcocAICE43fu+b X-Received: by 2002:ac2:48a5:: with SMTP id u5mr15947940lfg.62.1562096538775; Tue, 02 Jul 2019 12:42:18 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.17 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:18 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 06/32] crypto: x86/aes - drop scalar assembler implementations Date: Tue, 2 Jul 2019 21:41:24 +0200 Message-Id: <20190702194150.10405-7-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AES assembler code for x86 isn't actually faster than code generated by the compiler from aes_generic.c, and considering the disproportionate maintenance burden of assembler code on x86, it is better just to drop it entirely. Modern x86 systems will use AES-NI anyway, and given that the modules being removed have a dependency on aes_generic already, we can remove them without running the risk of regressions. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/Makefile | 4 - arch/x86/crypto/aes-i586-asm_32.S | 362 -------------------- arch/x86/crypto/aes-x86_64-asm_64.S | 185 ---------- arch/x86/crypto/aes_glue.c | 70 ---- crypto/Kconfig | 44 --- 5 files changed, 665 deletions(-) -- 2.17.1 diff --git a/arch/x86/crypto/Makefile b/arch/x86/crypto/Makefile index 45734e1cf967..b96a14e67ab0 100644 --- a/arch/x86/crypto/Makefile +++ b/arch/x86/crypto/Makefile @@ -14,11 +14,9 @@ sha256_ni_supported :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,yes,no) obj-$(CONFIG_CRYPTO_GLUE_HELPER_X86) += glue_helper.o -obj-$(CONFIG_CRYPTO_AES_586) += aes-i586.o obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o obj-$(CONFIG_CRYPTO_SERPENT_SSE2_586) += serpent-sse2-i586.o -obj-$(CONFIG_CRYPTO_AES_X86_64) += aes-x86_64.o obj-$(CONFIG_CRYPTO_DES3_EDE_X86_64) += des3_ede-x86_64.o obj-$(CONFIG_CRYPTO_CAMELLIA_X86_64) += camellia-x86_64.o obj-$(CONFIG_CRYPTO_BLOWFISH_X86_64) += blowfish-x86_64.o @@ -68,11 +66,9 @@ ifeq ($(avx2_supported),yes) obj-$(CONFIG_CRYPTO_MORUS1280_AVX2) += morus1280-avx2.o endif -aes-i586-y := aes-i586-asm_32.o aes_glue.o twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o -aes-x86_64-y := aes-x86_64-asm_64.o aes_glue.o des3_ede-x86_64-y := des3_ede-asm_64.o des3_ede_glue.o camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o diff --git a/arch/x86/crypto/aes-i586-asm_32.S b/arch/x86/crypto/aes-i586-asm_32.S deleted file mode 100644 index 2849dbc59e11..000000000000 --- a/arch/x86/crypto/aes-i586-asm_32.S +++ /dev/null @@ -1,362 +0,0 @@ -// ------------------------------------------------------------------------- -// Copyright (c) 2001, Dr Brian Gladman < >, Worcester, UK. -// All rights reserved. -// -// LICENSE TERMS -// -// The free distribution and use of this software in both source and binary -// form is allowed (with or without changes) provided that: -// -// 1. distributions of this source code include the above copyright -// notice, this list of conditions and the following disclaimer// -// -// 2. distributions in binary form include the above copyright -// notice, this list of conditions and the following disclaimer -// in the documentation and/or other associated materials// -// -// 3. the copyright holder's name is not used to endorse products -// built using this software without specific written permission. -// -// -// ALTERNATIVELY, provided that this notice is retained in full, this product -// may be distributed under the terms of the GNU General Public License (GPL), -// in which case the provisions of the GPL apply INSTEAD OF those given above. -// -// Copyright (c) 2004 Linus Torvalds -// Copyright (c) 2004 Red Hat, Inc., James Morris - -// DISCLAIMER -// -// This software is provided 'as is' with no explicit or implied warranties -// in respect of its properties including, but not limited to, correctness -// and fitness for purpose. -// ------------------------------------------------------------------------- -// Issue Date: 29/07/2002 - -.file "aes-i586-asm.S" -.text - -#include -#include - -#define tlen 1024 // length of each of 4 'xor' arrays (256 32-bit words) - -/* offsets to parameters with one register pushed onto stack */ -#define ctx 8 -#define out_blk 12 -#define in_blk 16 - -/* offsets in crypto_aes_ctx structure */ -#define klen (480) -#define ekey (0) -#define dkey (240) - -// register mapping for encrypt and decrypt subroutines - -#define r0 eax -#define r1 ebx -#define r2 ecx -#define r3 edx -#define r4 esi -#define r5 edi - -#define eaxl al -#define eaxh ah -#define ebxl bl -#define ebxh bh -#define ecxl cl -#define ecxh ch -#define edxl dl -#define edxh dh - -#define _h(reg) reg##h -#define h(reg) _h(reg) - -#define _l(reg) reg##l -#define l(reg) _l(reg) - -// This macro takes a 32-bit word representing a column and uses -// each of its four bytes to index into four tables of 256 32-bit -// words to obtain values that are then xored into the appropriate -// output registers r0, r1, r4 or r5. - -// Parameters: -// table table base address -// %1 out_state[0] -// %2 out_state[1] -// %3 out_state[2] -// %4 out_state[3] -// idx input register for the round (destroyed) -// tmp scratch register for the round -// sched key schedule - -#define do_col(table, a1,a2,a3,a4, idx, tmp) \ - movzx %l(idx),%tmp; \ - xor table(,%tmp,4),%a1; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+2*tlen(,%tmp,4),%a3; \ - xor table+3*tlen(,%idx,4),%a4; - -// initialise output registers from the key schedule -// NB1: original value of a3 is in idx on exit -// NB2: original values of a1,a2,a4 aren't used -#define do_fcol(table, a1,a2,a3,a4, idx, tmp, sched) \ - mov 0 sched,%a1; \ - movzx %l(idx),%tmp; \ - mov 12 sched,%a2; \ - xor table(,%tmp,4),%a1; \ - mov 4 sched,%a4; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+3*tlen(,%idx,4),%a4; \ - mov %a3,%idx; \ - mov 8 sched,%a3; \ - xor table+2*tlen(,%tmp,4),%a3; - -// initialise output registers from the key schedule -// NB1: original value of a3 is in idx on exit -// NB2: original values of a1,a2,a4 aren't used -#define do_icol(table, a1,a2,a3,a4, idx, tmp, sched) \ - mov 0 sched,%a1; \ - movzx %l(idx),%tmp; \ - mov 4 sched,%a2; \ - xor table(,%tmp,4),%a1; \ - mov 12 sched,%a4; \ - movzx %h(idx),%tmp; \ - shr $16,%idx; \ - xor table+tlen(,%tmp,4),%a2; \ - movzx %l(idx),%tmp; \ - movzx %h(idx),%idx; \ - xor table+3*tlen(,%idx,4),%a4; \ - mov %a3,%idx; \ - mov 8 sched,%a3; \ - xor table+2*tlen(,%tmp,4),%a3; - - -// original Gladman had conditional saves to MMX regs. -#define save(a1, a2) \ - mov %a2,4*a1(%esp) - -#define restore(a1, a2) \ - mov 4*a2(%esp),%a1 - -// These macros perform a forward encryption cycle. They are entered with -// the first previous round column values in r0,r1,r4,r5 and -// exit with the final values in the same registers, using stack -// for temporary storage. - -// round column values -// on entry: r0,r1,r4,r5 -// on exit: r2,r1,r4,r5 -#define fwd_rnd1(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_fcol(table, r2,r5,r4,r1, r0,r3, arg); /* idx=r0 */ \ - do_col (table, r4,r1,r2,r5, r0,r3); /* idx=r4 */ \ - restore(r0,0); \ - do_col (table, r1,r2,r5,r4, r0,r3); /* idx=r1 */ \ - restore(r0,1); \ - do_col (table, r5,r4,r1,r2, r0,r3); /* idx=r5 */ - -// round column values -// on entry: r2,r1,r4,r5 -// on exit: r0,r1,r4,r5 -#define fwd_rnd2(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_fcol(table, r0,r5,r4,r1, r2,r3, arg); /* idx=r2 */ \ - do_col (table, r4,r1,r0,r5, r2,r3); /* idx=r4 */ \ - restore(r2,0); \ - do_col (table, r1,r0,r5,r4, r2,r3); /* idx=r1 */ \ - restore(r2,1); \ - do_col (table, r5,r4,r1,r0, r2,r3); /* idx=r5 */ - -// These macros performs an inverse encryption cycle. They are entered with -// the first previous round column values in r0,r1,r4,r5 and -// exit with the final values in the same registers, using stack -// for temporary storage - -// round column values -// on entry: r0,r1,r4,r5 -// on exit: r2,r1,r4,r5 -#define inv_rnd1(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_icol(table, r2,r1,r4,r5, r0,r3, arg); /* idx=r0 */ \ - do_col (table, r4,r5,r2,r1, r0,r3); /* idx=r4 */ \ - restore(r0,0); \ - do_col (table, r1,r4,r5,r2, r0,r3); /* idx=r1 */ \ - restore(r0,1); \ - do_col (table, r5,r2,r1,r4, r0,r3); /* idx=r5 */ - -// round column values -// on entry: r2,r1,r4,r5 -// on exit: r0,r1,r4,r5 -#define inv_rnd2(arg, table) \ - save (0,r1); \ - save (1,r5); \ - \ - /* compute new column values */ \ - do_icol(table, r0,r1,r4,r5, r2,r3, arg); /* idx=r2 */ \ - do_col (table, r4,r5,r0,r1, r2,r3); /* idx=r4 */ \ - restore(r2,0); \ - do_col (table, r1,r4,r5,r0, r2,r3); /* idx=r1 */ \ - restore(r2,1); \ - do_col (table, r5,r0,r1,r4, r2,r3); /* idx=r5 */ - -// AES (Rijndael) Encryption Subroutine -/* void aes_enc_blk(struct crypto_aes_ctx *ctx, u8 *out_blk, const u8 *in_blk) */ - -.extern crypto_ft_tab -.extern crypto_fl_tab - -ENTRY(aes_enc_blk) - push %ebp - mov ctx(%esp),%ebp - -// CAUTION: the order and the values used in these assigns -// rely on the register mappings - -1: push %ebx - mov in_blk+4(%esp),%r2 - push %esi - mov klen(%ebp),%r3 // key size - push %edi -#if ekey != 0 - lea ekey(%ebp),%ebp // key pointer -#endif - -// input four columns and xor in first round key - - mov (%r2),%r0 - mov 4(%r2),%r1 - mov 8(%r2),%r4 - mov 12(%r2),%r5 - xor (%ebp),%r0 - xor 4(%ebp),%r1 - xor 8(%ebp),%r4 - xor 12(%ebp),%r5 - - sub $8,%esp // space for register saves on stack - add $16,%ebp // increment to next round key - cmp $24,%r3 - jb 4f // 10 rounds for 128-bit key - lea 32(%ebp),%ebp - je 3f // 12 rounds for 192-bit key - lea 32(%ebp),%ebp - -2: fwd_rnd1( -64(%ebp), crypto_ft_tab) // 14 rounds for 256-bit key - fwd_rnd2( -48(%ebp), crypto_ft_tab) -3: fwd_rnd1( -32(%ebp), crypto_ft_tab) // 12 rounds for 192-bit key - fwd_rnd2( -16(%ebp), crypto_ft_tab) -4: fwd_rnd1( (%ebp), crypto_ft_tab) // 10 rounds for 128-bit key - fwd_rnd2( +16(%ebp), crypto_ft_tab) - fwd_rnd1( +32(%ebp), crypto_ft_tab) - fwd_rnd2( +48(%ebp), crypto_ft_tab) - fwd_rnd1( +64(%ebp), crypto_ft_tab) - fwd_rnd2( +80(%ebp), crypto_ft_tab) - fwd_rnd1( +96(%ebp), crypto_ft_tab) - fwd_rnd2(+112(%ebp), crypto_ft_tab) - fwd_rnd1(+128(%ebp), crypto_ft_tab) - fwd_rnd2(+144(%ebp), crypto_fl_tab) // last round uses a different table - -// move final values to the output array. CAUTION: the -// order of these assigns rely on the register mappings - - add $8,%esp - mov out_blk+12(%esp),%ebp - mov %r5,12(%ebp) - pop %edi - mov %r4,8(%ebp) - pop %esi - mov %r1,4(%ebp) - pop %ebx - mov %r0,(%ebp) - pop %ebp - ret -ENDPROC(aes_enc_blk) - -// AES (Rijndael) Decryption Subroutine -/* void aes_dec_blk(struct crypto_aes_ctx *ctx, u8 *out_blk, const u8 *in_blk) */ - -.extern crypto_it_tab -.extern crypto_il_tab - -ENTRY(aes_dec_blk) - push %ebp - mov ctx(%esp),%ebp - -// CAUTION: the order and the values used in these assigns -// rely on the register mappings - -1: push %ebx - mov in_blk+4(%esp),%r2 - push %esi - mov klen(%ebp),%r3 // key size - push %edi -#if dkey != 0 - lea dkey(%ebp),%ebp // key pointer -#endif - -// input four columns and xor in first round key - - mov (%r2),%r0 - mov 4(%r2),%r1 - mov 8(%r2),%r4 - mov 12(%r2),%r5 - xor (%ebp),%r0 - xor 4(%ebp),%r1 - xor 8(%ebp),%r4 - xor 12(%ebp),%r5 - - sub $8,%esp // space for register saves on stack - add $16,%ebp // increment to next round key - cmp $24,%r3 - jb 4f // 10 rounds for 128-bit key - lea 32(%ebp),%ebp - je 3f // 12 rounds for 192-bit key - lea 32(%ebp),%ebp - -2: inv_rnd1( -64(%ebp), crypto_it_tab) // 14 rounds for 256-bit key - inv_rnd2( -48(%ebp), crypto_it_tab) -3: inv_rnd1( -32(%ebp), crypto_it_tab) // 12 rounds for 192-bit key - inv_rnd2( -16(%ebp), crypto_it_tab) -4: inv_rnd1( (%ebp), crypto_it_tab) // 10 rounds for 128-bit key - inv_rnd2( +16(%ebp), crypto_it_tab) - inv_rnd1( +32(%ebp), crypto_it_tab) - inv_rnd2( +48(%ebp), crypto_it_tab) - inv_rnd1( +64(%ebp), crypto_it_tab) - inv_rnd2( +80(%ebp), crypto_it_tab) - inv_rnd1( +96(%ebp), crypto_it_tab) - inv_rnd2(+112(%ebp), crypto_it_tab) - inv_rnd1(+128(%ebp), crypto_it_tab) - inv_rnd2(+144(%ebp), crypto_il_tab) // last round uses a different table - -// move final values to the output array. CAUTION: the -// order of these assigns rely on the register mappings - - add $8,%esp - mov out_blk+12(%esp),%ebp - mov %r5,12(%ebp) - pop %edi - mov %r4,8(%ebp) - pop %esi - mov %r1,4(%ebp) - pop %ebx - mov %r0,(%ebp) - pop %ebp - ret -ENDPROC(aes_dec_blk) diff --git a/arch/x86/crypto/aes-x86_64-asm_64.S b/arch/x86/crypto/aes-x86_64-asm_64.S deleted file mode 100644 index 8739cf7795de..000000000000 --- a/arch/x86/crypto/aes-x86_64-asm_64.S +++ /dev/null @@ -1,185 +0,0 @@ -/* AES (Rijndael) implementation (FIPS PUB 197) for x86_64 - * - * Copyright (C) 2005 Andreas Steinmetz, - * - * License: - * This code can be distributed under the terms of the GNU General Public - * License (GPL) Version 2 provided that the above header down to and - * including this sentence is retained in full. - */ - -.extern crypto_ft_tab -.extern crypto_it_tab -.extern crypto_fl_tab -.extern crypto_il_tab - -.text - -#include -#include - -#define R1 %rax -#define R1E %eax -#define R1X %ax -#define R1H %ah -#define R1L %al -#define R2 %rbx -#define R2E %ebx -#define R2X %bx -#define R2H %bh -#define R2L %bl -#define R3 %rcx -#define R3E %ecx -#define R3X %cx -#define R3H %ch -#define R3L %cl -#define R4 %rdx -#define R4E %edx -#define R4X %dx -#define R4H %dh -#define R4L %dl -#define R5 %rsi -#define R5E %esi -#define R6 %rdi -#define R6E %edi -#define R7 %r9 /* don't use %rbp; it breaks stack traces */ -#define R7E %r9d -#define R8 %r8 -#define R10 %r10 -#define R11 %r11 - -#define prologue(FUNC,KEY,B128,B192,r1,r2,r5,r6,r7,r8,r9,r10,r11) \ - ENTRY(FUNC); \ - movq r1,r2; \ - leaq KEY+48(r8),r9; \ - movq r10,r11; \ - movl (r7),r5 ## E; \ - movl 4(r7),r1 ## E; \ - movl 8(r7),r6 ## E; \ - movl 12(r7),r7 ## E; \ - movl 480(r8),r10 ## E; \ - xorl -48(r9),r5 ## E; \ - xorl -44(r9),r1 ## E; \ - xorl -40(r9),r6 ## E; \ - xorl -36(r9),r7 ## E; \ - cmpl $24,r10 ## E; \ - jb B128; \ - leaq 32(r9),r9; \ - je B192; \ - leaq 32(r9),r9; - -#define epilogue(FUNC,r1,r2,r5,r6,r7,r8,r9) \ - movq r1,r2; \ - movl r5 ## E,(r9); \ - movl r6 ## E,4(r9); \ - movl r7 ## E,8(r9); \ - movl r8 ## E,12(r9); \ - ret; \ - ENDPROC(FUNC); - -#define round(TAB,OFFSET,r1,r2,r3,r4,r5,r6,r7,r8,ra,rb,rc,rd) \ - movzbl r2 ## H,r5 ## E; \ - movzbl r2 ## L,r6 ## E; \ - movl TAB+1024(,r5,4),r5 ## E;\ - movw r4 ## X,r2 ## X; \ - movl TAB(,r6,4),r6 ## E; \ - roll $16,r2 ## E; \ - shrl $16,r4 ## E; \ - movzbl r4 ## L,r7 ## E; \ - movzbl r4 ## H,r4 ## E; \ - xorl OFFSET(r8),ra ## E; \ - xorl OFFSET+4(r8),rb ## E; \ - xorl TAB+3072(,r4,4),r5 ## E;\ - xorl TAB+2048(,r7,4),r6 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r4 ## E; \ - movl TAB+1024(,r4,4),r4 ## E;\ - movw r3 ## X,r1 ## X; \ - roll $16,r1 ## E; \ - shrl $16,r3 ## E; \ - xorl TAB(,r7,4),r5 ## E; \ - movzbl r3 ## L,r7 ## E; \ - movzbl r3 ## H,r3 ## E; \ - xorl TAB+3072(,r3,4),r4 ## E;\ - xorl TAB+2048(,r7,4),r5 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r3 ## E; \ - shrl $16,r1 ## E; \ - xorl TAB+3072(,r3,4),r6 ## E;\ - movl TAB+2048(,r7,4),r3 ## E;\ - movzbl r1 ## L,r7 ## E; \ - movzbl r1 ## H,r1 ## E; \ - xorl TAB+1024(,r1,4),r6 ## E;\ - xorl TAB(,r7,4),r3 ## E; \ - movzbl r2 ## H,r1 ## E; \ - movzbl r2 ## L,r7 ## E; \ - shrl $16,r2 ## E; \ - xorl TAB+3072(,r1,4),r3 ## E;\ - xorl TAB+2048(,r7,4),r4 ## E;\ - movzbl r2 ## H,r1 ## E; \ - movzbl r2 ## L,r2 ## E; \ - xorl OFFSET+8(r8),rc ## E; \ - xorl OFFSET+12(r8),rd ## E; \ - xorl TAB+1024(,r1,4),r3 ## E;\ - xorl TAB(,r2,4),r4 ## E; - -#define move_regs(r1,r2,r3,r4) \ - movl r3 ## E,r1 ## E; \ - movl r4 ## E,r2 ## E; - -#define entry(FUNC,KEY,B128,B192) \ - prologue(FUNC,KEY,B128,B192,R2,R8,R1,R3,R4,R6,R10,R5,R11) - -#define return(FUNC) epilogue(FUNC,R8,R2,R5,R6,R3,R4,R11) - -#define encrypt_round(TAB,OFFSET) \ - round(TAB,OFFSET,R1,R2,R3,R4,R5,R6,R7,R10,R5,R6,R3,R4) \ - move_regs(R1,R2,R5,R6) - -#define encrypt_final(TAB,OFFSET) \ - round(TAB,OFFSET,R1,R2,R3,R4,R5,R6,R7,R10,R5,R6,R3,R4) - -#define decrypt_round(TAB,OFFSET) \ - round(TAB,OFFSET,R2,R1,R4,R3,R6,R5,R7,R10,R5,R6,R3,R4) \ - move_regs(R1,R2,R5,R6) - -#define decrypt_final(TAB,OFFSET) \ - round(TAB,OFFSET,R2,R1,R4,R3,R6,R5,R7,R10,R5,R6,R3,R4) - -/* void aes_enc_blk(stuct crypto_tfm *tfm, u8 *out, const u8 *in) */ - - entry(aes_enc_blk,0,.Le128,.Le192) - encrypt_round(crypto_ft_tab,-96) - encrypt_round(crypto_ft_tab,-80) -.Le192: encrypt_round(crypto_ft_tab,-64) - encrypt_round(crypto_ft_tab,-48) -.Le128: encrypt_round(crypto_ft_tab,-32) - encrypt_round(crypto_ft_tab,-16) - encrypt_round(crypto_ft_tab, 0) - encrypt_round(crypto_ft_tab, 16) - encrypt_round(crypto_ft_tab, 32) - encrypt_round(crypto_ft_tab, 48) - encrypt_round(crypto_ft_tab, 64) - encrypt_round(crypto_ft_tab, 80) - encrypt_round(crypto_ft_tab, 96) - encrypt_final(crypto_fl_tab,112) - return(aes_enc_blk) - -/* void aes_dec_blk(struct crypto_tfm *tfm, u8 *out, const u8 *in) */ - - entry(aes_dec_blk,240,.Ld128,.Ld192) - decrypt_round(crypto_it_tab,-96) - decrypt_round(crypto_it_tab,-80) -.Ld192: decrypt_round(crypto_it_tab,-64) - decrypt_round(crypto_it_tab,-48) -.Ld128: decrypt_round(crypto_it_tab,-32) - decrypt_round(crypto_it_tab,-16) - decrypt_round(crypto_it_tab, 0) - decrypt_round(crypto_it_tab, 16) - decrypt_round(crypto_it_tab, 32) - decrypt_round(crypto_it_tab, 48) - decrypt_round(crypto_it_tab, 64) - decrypt_round(crypto_it_tab, 80) - decrypt_round(crypto_it_tab, 96) - decrypt_final(crypto_il_tab,112) - return(aes_dec_blk) diff --git a/arch/x86/crypto/aes_glue.c b/arch/x86/crypto/aes_glue.c deleted file mode 100644 index e26984f7ab8d..000000000000 --- a/arch/x86/crypto/aes_glue.c +++ /dev/null @@ -1,70 +0,0 @@ -/* - * Glue Code for the asm optimized version of the AES Cipher Algorithm - * - */ - -#include -#include -#include - -asmlinkage void aes_enc_blk(struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); -asmlinkage void aes_dec_blk(struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); - -void crypto_aes_encrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, const u8 *src) -{ - aes_enc_blk(ctx, dst, src); -} -EXPORT_SYMBOL_GPL(crypto_aes_encrypt_x86); - -void crypto_aes_decrypt_x86(struct crypto_aes_ctx *ctx, u8 *dst, const u8 *src) -{ - aes_dec_blk(ctx, dst, src); -} -EXPORT_SYMBOL_GPL(crypto_aes_decrypt_x86); - -static void aes_encrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) -{ - aes_enc_blk(crypto_tfm_ctx(tfm), dst, src); -} - -static void aes_decrypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src) -{ - aes_dec_blk(crypto_tfm_ctx(tfm), dst, src); -} - -static struct crypto_alg aes_alg = { - .cra_name = "aes", - .cra_driver_name = "aes-asm", - .cra_priority = 200, - .cra_flags = CRYPTO_ALG_TYPE_CIPHER, - .cra_blocksize = AES_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct crypto_aes_ctx), - .cra_module = THIS_MODULE, - .cra_u = { - .cipher = { - .cia_min_keysize = AES_MIN_KEY_SIZE, - .cia_max_keysize = AES_MAX_KEY_SIZE, - .cia_setkey = crypto_aes_set_key, - .cia_encrypt = aes_encrypt, - .cia_decrypt = aes_decrypt - } - } -}; - -static int __init aes_init(void) -{ - return crypto_register_alg(&aes_alg); -} - -static void __exit aes_fini(void) -{ - crypto_unregister_alg(&aes_alg); -} - -module_init(aes_init); -module_exit(aes_fini); - -MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm, asm optimized"); -MODULE_LICENSE("GPL"); -MODULE_ALIAS_CRYPTO("aes"); -MODULE_ALIAS_CRYPTO("aes-asm"); diff --git a/crypto/Kconfig b/crypto/Kconfig index 20af58068e6b..df6f0be66574 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1108,50 +1108,6 @@ config CRYPTO_AES_TI block. Interrupts are also disabled to avoid races where cachelines are evicted when the CPU is interrupted to do something else. -config CRYPTO_AES_586 - tristate "AES cipher algorithms (i586)" - depends on (X86 || UML_X86) && !64BIT - select CRYPTO_ALGAPI - select CRYPTO_AES - help - AES cipher algorithms (FIPS-197). AES uses the Rijndael - algorithm. - - Rijndael appears to be consistently a very good performer in - both hardware and software across a wide range of computing - environments regardless of its use in feedback or non-feedback - modes. Its key setup time is excellent, and its key agility is - good. Rijndael's very low memory requirements make it very well - suited for restricted-space environments, in which it also - demonstrates excellent performance. Rijndael's operations are - among the easiest to defend against power and timing attacks. - - The AES specifies three key sizes: 128, 192 and 256 bits - - See for more information. - -config CRYPTO_AES_X86_64 - tristate "AES cipher algorithms (x86_64)" - depends on (X86 || UML_X86) && 64BIT - select CRYPTO_ALGAPI - select CRYPTO_AES - help - AES cipher algorithms (FIPS-197). AES uses the Rijndael - algorithm. - - Rijndael appears to be consistently a very good performer in - both hardware and software across a wide range of computing - environments regardless of its use in feedback or non-feedback - modes. Its key setup time is excellent, and its key agility is - good. Rijndael's very low memory requirements make it very well - suited for restricted-space environments, in which it also - demonstrates excellent performance. Rijndael's operations are - among the easiest to defend against power and timing attacks. - - The AES specifies three key sizes: 128, 192 and 256 bits - - See for more information. - config CRYPTO_AES_NI_INTEL tristate "AES cipher algorithms (AES-NI)" depends on X86 From patchwork Tue Jul 2 19:41:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168352 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653362ilk; Tue, 2 Jul 2019 12:42:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqyNaLhZe8xB8sSWaW+C02taQioUTd+1zkXH7r6Q2cUCO1g0m8QgzJuHOx+zzp4q6RzROR8r X-Received: by 2002:a65:55c9:: with SMTP id k9mr19699119pgs.142.1562096543140; Tue, 02 Jul 2019 12:42:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096543; cv=none; d=google.com; s=arc-20160816; b=f2LAk8c5pVLrf02fi1RGEUaR3DlFIZTvR4HAdCoIeai0w5vybj3YlbjlLYLxPUaeZj 9cixJcmvBXq18WsMMFyKIfVrGf6zjCUhIb6llJWqqgtHSTNR4qs3fucE99Io4llojVNg gUPiwb49zaJC7SiAuOhly663N/SCkevZ5J3t2IuXlrS9uz8P1F/rpxSRSupwaEUbE6UB Blza5NJ/p/3aOXWKQYBDBWcp5WfytqPwy3ZnI7HIfsa9VuUouG4bmyHqEyRfBIHAtZHa Gqbvbw2+TYp3CpJXuhN73KCplOvHMInn0Fs6MIrHQcTpTjls0VS4N5O5ylmRLPZjg1Tz oT4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=U0sLNlYCVozO8actpiST83AyF3eeiGkgBZDkGubqHmw=; b=NEWjzAg3VqcFpf+Q2aEOmPLJjOukeJskRjLyMT5nhIEz2jLJshzTKnuXGi0zzRuDC3 5MmqbaSAaUbnyqi6NetYkWx5fgG9k2YgZtj6F+g8gNkyJU+o8xMXWTdIMp3LeF/5ez7B OVzNM6SQ2Q/rQoK6x+Rd54uG9QeMUZJrhLN+dttFf4o55ff+t7FMEjKao6hgtgFgEl3V zsM4C1jyiJtSwqGfFmnk2YZQThoY3yFy4NcB7GT/xu0FSkgEZWnqFTYHrBABBdMnI0fZ CSnjKIW3Fe0G8JlA9D/xlXxMzq5ecY1gwMDPxOc+3Ayg/HygfROQonpnfNreMly7GaP8 by5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Vxqnv0i6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.22; Tue, 02 Jul 2019 12:42:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Vxqnv0i6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727050AbfGBTmW (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:22 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:44673 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726291AbfGBTmW (ORCPT ); Tue, 2 Jul 2019 15:42:22 -0400 Received: by mail-lj1-f195.google.com with SMTP id k18so18151455ljc.11 for ; Tue, 02 Jul 2019 12:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=U0sLNlYCVozO8actpiST83AyF3eeiGkgBZDkGubqHmw=; b=Vxqnv0i6HLhhHtnZqH5SYpNKc3mpt0B/YWNZRJtJaF19W4MOYixSmL4gdgiXw1bE9V kNIFupb2ENlK83FmFFd8pPO35y/Sq8SlB8P/aXBE0Dp4lybGvuNgTAHcjQExaMOYTF7m iH9lsos06HGhzyb3DEvUa0bhiy91ba/d+yQbPDoZwlX+dmzNpPGurYCch3bo7FvEB/VF Sjl0vpnLejbE1VYZ/vn2rDEl/Bmr7lJFOe/bvXVqqwiOs91ttH53CLP20qJosoHqwLTM fJYI4ZZkk6CGh54jH93s5Zutv6AS+uN8bqG1J9VegnRaIO3pH+SFVgS6gr8JfA8A6Xn1 Y5qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=U0sLNlYCVozO8actpiST83AyF3eeiGkgBZDkGubqHmw=; b=M8MlfFMAVgQVRqpDl9wBFZi23SA5U3PRFOsh7r1Ltsh5JfSTNwHaQ9osPNZ39fKZVg c77wI8IbdzUvGCxWMHbcdNWyIZN23ezL+30D3QxYMTOWqiLif1OKqcFBx+EW6HqDBdjc AA7l9Vd9JbUsmM3PN15Xg/yz7FNyunjJji03mXwNdR/GycfJ9pGkXbOIEhi1n0bahJHa yeLZfEIqYf5XyPI11Mx4aqNcJ2AKk+vLCaNbrSNUyf3GA6CiYQJvnxYdpK6VzWWfHmBT KlG+ZKZMThNNLSMo8UBDH1CB6BLZaYirBphAKwx/jgT3j03R91y5emaIHzZxc7N7IYCU bWxQ== X-Gm-Message-State: APjAAAVcOcXp4alfa4HExUfJS4EmkAVnYlObssnsUXtTeJpjZDDiBBJb v53C2drzYm83p8DY7YiiJC0VxgkKl3mTTKmr X-Received: by 2002:a2e:8e90:: with SMTP id z16mr18658353ljk.4.1562096539979; Tue, 02 Jul 2019 12:42:19 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.18 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:19 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 07/32] crypto: padlock/aes - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:25 +0200 Message-Id: <20190702194150.10405-8-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/padlock-aes.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 67af688d7d84..3fca5f7e38f0 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -26,7 +26,7 @@ config CRYPTO_DEV_PADLOCK_AES tristate "PadLock driver for AES algorithm" depends on CRYPTO_DEV_PADLOCK select CRYPTO_BLKCIPHER - select CRYPTO_AES + select CRYPTO_LIB_AES help Use VIA PadLock for AES algorithm. diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c index 854539512c35..af90138eddb7 100644 --- a/drivers/crypto/padlock-aes.c +++ b/drivers/crypto/padlock-aes.c @@ -144,7 +144,7 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, ctx->cword.encrypt.keygen = 1; ctx->cword.decrypt.keygen = 1; - if (crypto_aes_expand_key(&gen_aes, in_key, key_len)) { + if (aes_expandkey(&gen_aes, in_key, key_len)) { *flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } From patchwork Tue Jul 2 19:41:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168353 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653383ilk; Tue, 2 Jul 2019 12:42:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqw3NG7fabcwZlI8cTkfEoh8LkdGJq6dfXEBm+Aj+kA9yeYA6fLnxkT9ylcHXZ4vg5yTpqkL X-Received: by 2002:a17:90a:ba93:: with SMTP id t19mr7426191pjr.139.1562096544765; Tue, 02 Jul 2019 12:42:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096544; cv=none; d=google.com; s=arc-20160816; b=bj0F9Xj/Adsiqm5t8cPbQHZT+8Q+5Ievxu8ZyfLTED4qg+YJ3V8YuSacFkR5RLkvHW jALzf0l858HJg4c/j5WvjODMdK5FvCnNnYr2pcJSFf+e7ykWGxnfylXd0uQOAeDI0cVc O37Gz7rPqM39H4LE1Om2gXno7YA11sNZsac9FWLdG2QlhiDfeDeqFJZWcfRLoUFogQHQ r5HZU0azmK4SedvXdAzVbnwHxcZQ0wllqUKQ1GUm9hS+6nSfLH9wutTHZlTEM+T3JaDI B9kufFEE8TZ/pNuN/N8ZvOFz9E2kQuOG3GSO/OjGQT6bZ3O1uSJYF1NUo8erNj84THNO UCPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=2vu90pozuzPI+4NwppRxnbZtYKOaoe2IbHpzwdsfy4U=; b=RAXAdwmJc1WUnoJeamFqwBgFAHRefQe/WPdBhfwXT4dNAWz7Ls8YGzLo4Fosj4IRIn CAMpdVDFrJIhId56aEZlnkawcn9Y0mOTYeQD8aTS5YW9eWOxn2aV0nlvricN6VzBwL5t 4S5rT3xJZq6AvxvYQ0q+AAc7PLcnhvkZu+R+ZNn1UucdLTdPUJHPtYRUJ2yXWIg57vTX IsugdpFUwkp/t4Z1F5bRUuauJh62uQDm4ZzOLRx6Oa/8K+RQZetAagsmr37l+k64K2xV qYedMVaR01/qmHOyQwQM7H4fz9VLe2LH07koOda22pr1T0eFlIy6DFLfbByirEZXjUuT rvtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="bgemgjj/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.24; Tue, 02 Jul 2019 12:42:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="bgemgjj/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727051AbfGBTmY (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:24 -0400 Received: from mail-lj1-f196.google.com ([209.85.208.196]:36632 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727049AbfGBTmX (ORCPT ); Tue, 2 Jul 2019 15:42:23 -0400 Received: by mail-lj1-f196.google.com with SMTP id i21so18199590ljj.3 for ; Tue, 02 Jul 2019 12:42:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2vu90pozuzPI+4NwppRxnbZtYKOaoe2IbHpzwdsfy4U=; b=bgemgjj/Z8o864DCjP3XY81G24w7BW3XOxgNRvKtwcYqQd7DeqA2OMuDc3wF+cKdTE JXIfYaSod1nr8sPXEwlPeNlScTEZY4zke+aPMAvfbv3sUHr6C1L5IAfnfNN2bRljgMcy 7/U03X/wtFjl4cYcvZKe+zM0yF3Fblk+4HW5O95SIHyLtEGhySx9p3LZbGtFzRobdo1K UCM+jiy75eUH2HD24zs3Ke/l5xa3O2CzvLVZS5RVRfiLI7s6JZQdHpyDpuKXUTqopjPt l00bKwcOd8eWaJq98njJ/tvGRt8fvh90uS09AKKo8xKovvwZEAD4biWmoTkC1+EyXc02 JWLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2vu90pozuzPI+4NwppRxnbZtYKOaoe2IbHpzwdsfy4U=; b=kLyz6Ov6/4bt9bztxBmm9LJ3C2ej0IWSiMApbXZX3pqRjjMmbfDkbSRzZxf7RN6jGX kOkz9XK9Tedy8ZbL0OW+IizmGZMdE6nnIz1jL1nLCnY5A2S6FJidfLD60Nerh9J4MXSI E3ukDWhsbAvaSYuHiBhEGrQuvkwODVVIAWbIB3VBs/kEErRBflbqwrTctpoxPftpZI/W AoxklhE9xKnGviUnf7t8C+V+pAxl06W2eb1idkNgpKbftJ5KKVbOeOrb+3kyO33/94aV DIA7d/eLiNvgLpd8wmkAqCcAGpgVxBoEsvYpiUOBkcLhyL1dBHJT9wyPKUruiAQbeblY FmiQ== X-Gm-Message-State: APjAAAXkWkco6mdsWiViLaTVXOCZ4PSMpeg23qtBhj/h+jYEXcWDMm/m 9jOpVn/JwRLaZG34IKWqwmMwpBoZRWlvRQdU X-Received: by 2002:a2e:50e:: with SMTP id 14mr18798392ljf.5.1562096541399; Tue, 02 Jul 2019 12:42:21 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:20 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 08/32] crypto: cesa/aes - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:26 +0200 Message-Id: <20190702194150.10405-9-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/marvell/cipher.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index 3fca5f7e38f0..fdccadc94819 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -213,7 +213,7 @@ config CRYPTO_CRC32_S390 config CRYPTO_DEV_MARVELL_CESA tristate "Marvell's Cryptographic Engine driver" depends on PLAT_ORION || ARCH_MVEBU - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_DES select CRYPTO_BLKCIPHER select CRYPTO_HASH diff --git a/drivers/crypto/marvell/cipher.c b/drivers/crypto/marvell/cipher.c index 2fd936b19c6d..debe7d9f00ae 100644 --- a/drivers/crypto/marvell/cipher.c +++ b/drivers/crypto/marvell/cipher.c @@ -257,7 +257,7 @@ static int mv_cesa_aes_setkey(struct crypto_skcipher *cipher, const u8 *key, int ret; int i; - ret = crypto_aes_expand_key(&ctx->aes, key, len); + ret = aes_expandkey(&ctx->aes, key, len); if (ret) { crypto_skcipher_set_flags(cipher, CRYPTO_TFM_RES_BAD_KEY_LEN); return ret; From patchwork Tue Jul 2 19:41:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168355 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653400ilk; Tue, 2 Jul 2019 12:42:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqyiXdhaA1D+IB/0868Q4cvRMij3n/gYyZM46IZdp3Cu3WQtWyPxOMsAws4WN9Y+F3/+o4vA X-Received: by 2002:a63:e142:: with SMTP id h2mr4274431pgk.391.1562096545906; Tue, 02 Jul 2019 12:42:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096545; cv=none; d=google.com; s=arc-20160816; b=VC+vsmOObMnuEwC1F91T7XMFJw3NA6ebvRZDujslLqCZCzx1C4Q03qLlhCeCfTBnVm LBBdxUWy3bqdb5MD27fUlDA3OHO2sarVZGb4nBaG9yhSrTAVneCILbKfmZ2CHjhvynGz ZHwA/41A+y4jbNki7MYhzFnBuNzBAjIcQXiUisY7ggxUSjWl8pvksANSYHbCSFC3h+yf TDpo8drz1rjiaqDl0pTIAoakPu6mCj/D1Ef+IuD6ttuBERygXRz544CDVR1wHWRZWZpg 3vm4JSaYML7NMOhwimuU0Tc9W8cD8I6H5jdgluqg3o1xQ7c444JB4wQF84o5lMrjW++9 4szA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=yr+3Q9TRlnFwRP1ePoUuE5UXQ0fVFG8IySkaq4i3ZqU=; b=J3WmKtRN8Ng1s/0aa19ClPSPwX3sjjLjYYe95GJLCThW09mM3YDvW7+hynwmHpVBB0 iQaKB0e06Euds9NXp4lsT7Ljcn8WZNXCKfmlywEBapDYv53JadqRjLxqNMI91/HTgT98 osGtubo6KIWP78aGYi4CHiAqIW14CvvQpTYWn4SyD3RD/sowUEsNAfyJE79SmvaBhlVj VbgOC2RzqyBcrL9MDXdLoZuhCPt2jMpAnd127lfTAsCFfgBuevj00CTSP7gj3bey8Ilc YjyWn5S5jw3Q7DwyQCv6Vjf+VrivhNzS3ileh6EGQwBluFbZmESXeha5bj8WuyhGZ7E0 eM5w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="B/8h1V42"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.25; Tue, 02 Jul 2019 12:42:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="B/8h1V42"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727055AbfGBTmZ (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:25 -0400 Received: from mail-lf1-f68.google.com ([209.85.167.68]:38238 "EHLO mail-lf1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726963AbfGBTmZ (ORCPT ); Tue, 2 Jul 2019 15:42:25 -0400 Received: by mail-lf1-f68.google.com with SMTP id b11so12267780lfa.5 for ; Tue, 02 Jul 2019 12:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=yr+3Q9TRlnFwRP1ePoUuE5UXQ0fVFG8IySkaq4i3ZqU=; b=B/8h1V42vB/KOIakIJ//DCOWkkqLanAZVkjURxJlw5bg+u8PUbrzmtBO6GRJggmRKJ TLBIOUmrqFJ6RZUZIV2SynJPPYI29sId4CH9q6ld2aYsS1WsTkIvViuuBCvMlbzGIQTX TCuOTE/G0OM9d9R1WfmDUF5m+JYoa+MYm1p5NEveAy3oAE6iqyrzQUXeycgHaJlRKgiu 8jyrGMP+QNILHIdrJZ33mpWlJxayjJwH08rPya/qVp7rlBj6bJHJyFs6aF9t9DHXs52X YkLhEZr5u1cmnQd5tkpbcprnHqG7fq30l2CJERsQkh4ZHFHksg4qXPtAKeczXizY3lH1 Q5ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=yr+3Q9TRlnFwRP1ePoUuE5UXQ0fVFG8IySkaq4i3ZqU=; b=PED8soIbi72WDp+q0rba7Hd8HKwAd7k49/+ed84LKZt0t8rjOT6/2xYRWo6PyFmr5m Rv+i22oLEavBqwdjH3Mb0C21HTh8c+TA0c5Uf/XIXu21ihKX3lLDWtHTYkjpLV0d1Z3L VSxMhjdtsEhPTIBEEXbEXhamS1XmAquxUVEss/ngyImNfS7eYdJGv+Pz0ujgB5QqkH90 /1nswsT0mG7Y54gbjSBLDG3DrGWm8VuW6WcNDmDUeN1CorWXm0JxJ5e54mjG0b7k06Do ploB63+AyKdamQHGtHHSLkwzH+HvRsbqY3fuiVB+Rd46lGvqBawzKF/yI9edcjJ/N/7+ PRKQ== X-Gm-Message-State: APjAAAWpkILwP2JVlzTPUNOIgK6z1tRjrh5piVcxQdJS01v9gbOkVL6d r7OV6hx28Usd+XQpLCcjeyO4ADY+Kbwi99/C X-Received: by 2002:ac2:4a78:: with SMTP id q24mr14501522lfp.59.1562096542780; Tue, 02 Jul 2019 12:42:22 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.21 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:21 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 09/32] crypto: safexcel/aes - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:27 +0200 Message-Id: <20190702194150.10405-10-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/inside-secure/safexcel_cipher.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index fdccadc94819..b30b84089d11 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -718,7 +718,7 @@ config CRYPTO_DEV_SAFEXCEL tristate "Inside Secure's SafeXcel cryptographic engine driver" depends on OF depends on (ARM64 && ARCH_MVEBU) || (COMPILE_TEST && 64BIT) - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_AUTHENC select CRYPTO_BLKCIPHER select CRYPTO_DES diff --git a/drivers/crypto/inside-secure/safexcel_cipher.c b/drivers/crypto/inside-secure/safexcel_cipher.c index 8cdbdbe35681..19ec086dce4f 100644 --- a/drivers/crypto/inside-secure/safexcel_cipher.c +++ b/drivers/crypto/inside-secure/safexcel_cipher.c @@ -178,7 +178,7 @@ static int safexcel_skcipher_aes_setkey(struct crypto_skcipher *ctfm, struct crypto_aes_ctx aes; int ret, i; - ret = crypto_aes_expand_key(&aes, key, len); + ret = aes_expandkey(&aes, key, len); if (ret) { crypto_skcipher_set_flags(ctfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return ret; From patchwork Tue Jul 2 19:41:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168356 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653428ilk; Tue, 2 Jul 2019 12:42:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqzh1JtiypmH4j6giRIN+3+p8exfMyO3ZCLqTpWMv8i4AlXPgyj8kBvUwssHu2a1hjOfqbsn X-Received: by 2002:a17:902:8bc1:: with SMTP id r1mr24088811plo.42.1562096547660; Tue, 02 Jul 2019 12:42:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096547; cv=none; d=google.com; s=arc-20160816; b=HnrZP/SB3K45V84m2PIulmW0q04vT+kXgecY8Nrtvgj2lmZ8UiIzlzW0eM7yMOdLBa 4MkCyrXWj6h1U3d2x86fJz/MK2vVseQYrVQ+sBcA0/vSh1R5OGHgtrrCH8jfZQpVK1yp gS/PSiVIxDGqwPOaRyekj6XrHz914VMNtarRkBJ2U+oHQOxQz8uBPu54g2iOXu8OoYZR /qKW+n9nvxljFHOUgROPaICVCz0FWGjokEY1v5JuGepymsgSp/8ngHFRQUYHKZVG5K96 YID58q18Vu6b6FHVe0xEyfDt/aaut7b2nnIOZ8Do0FWmnkGu4fUn2baNXdbAxP7+L5sI tHCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=ess0mOhZtdacgukC/NjpqGdseB8Y22UnFMYYev7cVNc=; b=YUe9CO4GrTXrETlgqGGhSAmT3/c0UrcvNU3yV7P8Lwj/sQqXn0woScRcv2rNqOptWQ qfnGBjdW5Vui+Q4Mq+UbnaJeoPffTRLWoA9Tpyripje0uyxcI7HxGzyGVdVtuCelLgDQ oFTc/HPhLvbH9MJ22zsep7ChUxp//fheJXWW52MAwrMl6Cumn9K3Rrl+Nv5nBMvbxa99 a5HYh3/5i6J4IPzzzU7+rhivaqz3yO80sADMlIpOnqK/Z2gV/RMcn1uUVxkoLb8Y9VCJ rQ8eNlLiu74vG3cWYCTFi2QcsQpnDYNDdQX99hFs/6yTsdDGikY/UtvRne82G6uV/xkO HXcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pSZFdLA6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.27; Tue, 02 Jul 2019 12:42:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pSZFdLA6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726963AbfGBTm1 (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:27 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:37753 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727049AbfGBTm0 (ORCPT ); Tue, 2 Jul 2019 15:42:26 -0400 Received: by mail-lj1-f194.google.com with SMTP id 131so18175742ljf.4 for ; Tue, 02 Jul 2019 12:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ess0mOhZtdacgukC/NjpqGdseB8Y22UnFMYYev7cVNc=; b=pSZFdLA6eORzgQv41SSsA0mO6SCQHY/ZclAh2KiY6yeCRnt/857PHmofjy6sMsdDjT ph2AhqhQf/3k+ddM7MXUO37wT+QrGUQLbDgTsWNb7cwSXj0RdweWX4Lqf2D0FoBqrMwX g2TAajtPfZykdJ0OtLvlg+WVbExMosB65ynL/uTJM9+HBlkRvPLqCAovuDwDpDRgDjNX uiNYAxpiqqFA9KaW5BVlMR+etINtksrrUyIMczU8d3AsILkBEjZx7QpUdcHyIUjxdtdX zIuarEZxsTMCOesQQxiA3rloxJh5HLVI+FFwGHluNmSPYE+F7cdoSKVOGe8ZdcEX26BZ nzwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ess0mOhZtdacgukC/NjpqGdseB8Y22UnFMYYev7cVNc=; b=l0KLumbJoVXoi07y+quM0Mm8H1mL08R+R3cT1Oyy2J3iHa1ao6EHGhP5PYnNn9F1JF OofMEPHw/q1VIl3nvJqbLNebK4S3KpScnWflY/oJMXRDi8wApyLnUx+hRjkhg3XIRLln L6FM9bXrt30qlqaxWNk/vmLWaB8005NnUZZhPVlysOHIqhChnb22FsrEg7Gyson57/vo YvRPtLgYYdKmRbBxCmUfC9q+heBnWWW6tBAZgEL6S5y4IFbczy8qfuNt+pbZafeOn0uO pwwhpNZ5rGxIt8wJNasnGKbSBuWHlmaCpMRekwo/++tX6sUeDCc0giJcK1zDeCVJqw2C EGJg== X-Gm-Message-State: APjAAAX2HKV8CH2D+H9jHzS+QzmB4+yVdVRt7TJLaVbY4TPaYyVXLNG2 bWu8aud+DLYJrJXoNO47ElTVRFaCgZbw+hvl X-Received: by 2002:a2e:5b0f:: with SMTP id p15mr18078724ljb.82.1562096544052; Tue, 02 Jul 2019 12:42:24 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.22 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:23 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 10/32] crypto: arm64/ghash - switch to AES library Date: Tue, 2 Jul 2019 21:41:28 +0200 Message-Id: <20190702194150.10405-11-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The GHASH code uses the generic AES key expansion routines, and calls directly into the scalar table based AES cipher for arm64 from the fallback path, and since this implementation is known to be non-time invariant, doing so from a time invariant SIMD cipher is a bit nasty. So let's switch to the AES library - this makes the code more robust, and drops the dependency on the generic AES cipher, allowing us to omit it entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 3 +- arch/arm64/crypto/ghash-ce-glue.c | 30 +++++++------------- 2 files changed, 11 insertions(+), 22 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index d9a523ecdd83..1762055e7093 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -58,8 +58,7 @@ config CRYPTO_GHASH_ARM64_CE depends on KERNEL_MODE_NEON select CRYPTO_HASH select CRYPTO_GF128MUL - select CRYPTO_AES - select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES config CRYPTO_CRCT10DIF_ARM64_CE tristate "CRCT10DIF digest algorithm using PMULL instructions" diff --git a/arch/arm64/crypto/ghash-ce-glue.c b/arch/arm64/crypto/ghash-ce-glue.c index b39ed99b06fb..90496765d22f 100644 --- a/arch/arm64/crypto/ghash-ce-glue.c +++ b/arch/arm64/crypto/ghash-ce-glue.c @@ -73,8 +73,6 @@ asmlinkage void pmull_gcm_decrypt(int blocks, u64 dg[], u8 dst[], asmlinkage void pmull_gcm_encrypt_block(u8 dst[], u8 const src[], u32 const rk[], int rounds); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - static int ghash_init(struct shash_desc *desc) { struct ghash_desc_ctx *ctx = shash_desc_ctx(desc); @@ -312,14 +310,13 @@ static int gcm_setkey(struct crypto_aead *tfm, const u8 *inkey, u8 key[GHASH_BLOCK_SIZE]; int ret; - ret = crypto_aes_expand_key(&ctx->aes_key, inkey, keylen); + ret = aes_expandkey(&ctx->aes_key, inkey, keylen); if (ret) { tfm->base.crt_flags |= CRYPTO_TFM_RES_BAD_KEY_LEN; return -EINVAL; } - __aes_arm64_encrypt(ctx->aes_key.key_enc, key, (u8[AES_BLOCK_SIZE]){}, - num_rounds(&ctx->aes_key)); + aes_encrypt(&ctx->aes_key, key, (u8[AES_BLOCK_SIZE]){}); return __ghash_setkey(&ctx->ghash_key, key, sizeof(be128)); } @@ -470,7 +467,7 @@ static int gcm_encrypt(struct aead_request *req) rk = ctx->aes_key.key_enc; } while (walk.nbytes >= 2 * AES_BLOCK_SIZE); } else { - __aes_arm64_encrypt(ctx->aes_key.key_enc, tag, iv, nrounds); + aes_encrypt(&ctx->aes_key, tag, iv); put_unaligned_be32(2, iv + GCM_IV_SIZE); while (walk.nbytes >= (2 * AES_BLOCK_SIZE)) { @@ -481,8 +478,7 @@ static int gcm_encrypt(struct aead_request *req) int remaining = blocks; do { - __aes_arm64_encrypt(ctx->aes_key.key_enc, - ks, iv, nrounds); + aes_encrypt(&ctx->aes_key, ks, iv); crypto_xor_cpy(dst, src, ks, AES_BLOCK_SIZE); crypto_inc(iv, AES_BLOCK_SIZE); @@ -498,13 +494,10 @@ static int gcm_encrypt(struct aead_request *req) walk.nbytes % (2 * AES_BLOCK_SIZE)); } if (walk.nbytes) { - __aes_arm64_encrypt(ctx->aes_key.key_enc, ks, iv, - nrounds); + aes_encrypt(&ctx->aes_key, ks, iv); if (walk.nbytes > AES_BLOCK_SIZE) { crypto_inc(iv, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->aes_key.key_enc, - ks + AES_BLOCK_SIZE, iv, - nrounds); + aes_encrypt(&ctx->aes_key, ks + AES_BLOCK_SIZE, iv); } } } @@ -608,7 +601,7 @@ static int gcm_decrypt(struct aead_request *req) rk = ctx->aes_key.key_enc; } while (walk.nbytes >= 2 * AES_BLOCK_SIZE); } else { - __aes_arm64_encrypt(ctx->aes_key.key_enc, tag, iv, nrounds); + aes_encrypt(&ctx->aes_key, tag, iv); put_unaligned_be32(2, iv + GCM_IV_SIZE); while (walk.nbytes >= (2 * AES_BLOCK_SIZE)) { @@ -621,8 +614,7 @@ static int gcm_decrypt(struct aead_request *req) pmull_ghash_update_p64); do { - __aes_arm64_encrypt(ctx->aes_key.key_enc, - buf, iv, nrounds); + aes_encrypt(&ctx->aes_key, buf, iv); crypto_xor_cpy(dst, src, buf, AES_BLOCK_SIZE); crypto_inc(iv, AES_BLOCK_SIZE); @@ -640,11 +632,9 @@ static int gcm_decrypt(struct aead_request *req) memcpy(iv2, iv, AES_BLOCK_SIZE); crypto_inc(iv2, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->aes_key.key_enc, iv2, - iv2, nrounds); + aes_encrypt(&ctx->aes_key, iv2, iv2); } - __aes_arm64_encrypt(ctx->aes_key.key_enc, iv, iv, - nrounds); + aes_encrypt(&ctx->aes_key, iv, iv); } } From patchwork Tue Jul 2 19:41:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168357 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653445ilk; Tue, 2 Jul 2019 12:42:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqzz9M+TOjlp44qHmK7dB4C/4Sst/CHnakxh+41F59vHMorkPltFHYVg9sv+QrLnaMj2uyAQ X-Received: by 2002:a17:902:1003:: with SMTP id b3mr37620553pla.172.1562096548739; Tue, 02 Jul 2019 12:42:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096548; cv=none; d=google.com; s=arc-20160816; b=SNPZeG+AlftK01F8tVr4GGfvxBu1RSdtYPw0W7079X7tpCzBfB3bxjaXLmmgmfO+fc /oBNIjJ4dqWS7ATH87EsjQfwIPvEqa4GVArt2KtMs7GNNFUk6l4S9VHWSFEhkLNSq1zR HvYjfX39uCkB7r7LKj3/r36BnLBL177qUlJMI6XJGjLaSidV2xynGwcQ89yW03brNBr8 /8d2+ZeLQizhNAEPA5atQs1KguvB97sd2evwm3BwdFl2ik3AQertk9nlg5CEvnpixvsC j0tOn3FHHoleoqnYVg8t5qGv8l+whMF8F5CiKpzJ626ak80AxO3lyblM6/SYSA6Ig+/u lp7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=mFytONJYgeLPEAdaeV/79gIWsnl1yOK0v5PQmjMRL3M=; b=KITMPZ2bCpOS8vJtTJRkxg7vbJBirvbQsO+bhP/pSd+x2zcMty5sdjRpcQuE+oHYN6 AQWdcYWUGWemc2oelmuHn73e3BTlfyJE+sw/sXTSbMPNBDm21ygTFzvO9zAVTYrv+5qH imBKL1pDseH6IcxgUFV+UcYHACmixxe1oD1tSJsXlxpK5bda4690d8j6NQzAUsBFdiSc BOk31HcBUpAECqInBIx/Qz8gtp5pbFX7e4Jcp7fGF4Yy9/NMYxkYY87DjdqBHj0jQT/Q 977yAxYDZQMiT5b+BDIurpcuc95nOshEKmuZrs0v6BTKO7Rjr1/MwL+htBtBhib/GxVi akgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nWgAdFGp; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.28; Tue, 02 Jul 2019 12:42:28 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nWgAdFGp; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727065AbfGBTm2 (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:28 -0400 Received: from mail-lf1-f66.google.com ([209.85.167.66]:40012 "EHLO mail-lf1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726329AbfGBTm1 (ORCPT ); Tue, 2 Jul 2019 15:42:27 -0400 Received: by mail-lf1-f66.google.com with SMTP id a9so12297007lff.7 for ; Tue, 02 Jul 2019 12:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mFytONJYgeLPEAdaeV/79gIWsnl1yOK0v5PQmjMRL3M=; b=nWgAdFGpodhwE7+93cpf8hVe5RVv3EzevLafUKHyHxj2qLtuO7g3rMpYF0Uc8G7EST xMlgMgsEIl9/8rIwSbK7RP/Ectctkb0o4dvaDxowAW+yfl5CDlp7wsUO5gVjpFYJlf36 bQkN76++aoAlH/rYuZKwb9fGGb5Xio/JnxFethzXtBrCPKmxrKOCRovjmDdj8AV1UlD/ 9m49x0JXosLMzeaUjP+LeM2IKR8/pgOeIdy9A3lf7oKkTphUG5Z3JbH6yMxNQBBz0zfJ X2mFysrmK7dh/g3FZWhi9stGL3ymth+ae/D2x38j7jMy5Cp/tAu+xxqtiidLAaDjuhh0 FJWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mFytONJYgeLPEAdaeV/79gIWsnl1yOK0v5PQmjMRL3M=; b=sarS+xCxMhRJaTCVJuBnR0A0FQ813awpxmNS46Z01O1nL28CeBWzz3JRCs8GVS8Sha pG/63ppW0KykYhR0yXXgM81u5hsXtAUVSh4MMNCwtAYlwbJjybLFMtgC+PV+hY9sjE43 2xcqnCWJBnDM1jaaaGbXpjy++of9ACkX3jqWCO6gfaaIUgGCz4lpMZ9wLYYnkLKbJ9xU xtHjq7lBuDTwZv2gDpIHkYvgJ1mm5t0qH7AYXkdaFE0TALKjOBhzTu1m0lbm/1W7bAps JGYEYkRExW9/Wxd23CJRqw+X/OcWZmMCQjVP70I1mXmOgfZuf53ae3OHt+7w4Cly7ofQ 4O3Q== X-Gm-Message-State: APjAAAX+7R6JffqGXiQcBjxPg2yFrHZtjXTeqc33SJ4mEzOwRxln6Ygu AgQz2PvlvFKya1G6ppXF7Dov6g5vL7DzSQcO X-Received: by 2002:ac2:514b:: with SMTP id q11mr15753843lfd.33.1562096545325; Tue, 02 Jul 2019 12:42:25 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.24 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:24 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 11/32] crypto: arm/aes-neonbs - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:29 +0200 Message-Id: <20190702194150.10405-12-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/Kconfig | 2 +- arch/arm/crypto/aes-neonbs-glue.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/Kconfig b/arch/arm/crypto/Kconfig index a95322b59799..b24df84a1d7a 100644 --- a/arch/arm/crypto/Kconfig +++ b/arch/arm/crypto/Kconfig @@ -82,8 +82,8 @@ config CRYPTO_AES_ARM_BS tristate "Bit sliced AES using NEON instructions" depends on KERNEL_MODE_NEON select CRYPTO_BLKCIPHER + select CRYPTO_LIB_AES select CRYPTO_SIMD - select CRYPTO_AES help Use a faster and more secure NEON based implementation of AES in CBC, CTR and XTS modes diff --git a/arch/arm/crypto/aes-neonbs-glue.c b/arch/arm/crypto/aes-neonbs-glue.c index 617c2c99ebfb..f43c9365b6a9 100644 --- a/arch/arm/crypto/aes-neonbs-glue.c +++ b/arch/arm/crypto/aes-neonbs-glue.c @@ -64,7 +64,7 @@ static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -123,7 +123,7 @@ static int aesbs_cbc_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; From patchwork Tue Jul 2 19:41:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168358 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653456ilk; Tue, 2 Jul 2019 12:42:29 -0700 (PDT) X-Google-Smtp-Source: APXvYqxvbKioawmUXK5Ug1e85UaENIvAfq66Ey7Txe7JjNfK09l1myj0aU3kjGh2JYw8FuPEW4+x X-Received: by 2002:a17:90a:35e6:: with SMTP id r93mr7603627pjb.20.1562096549639; Tue, 02 Jul 2019 12:42:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096549; cv=none; d=google.com; s=arc-20160816; b=w3N0pZA1b1WFaU6L+3jEmPL+JbnPRMbHaLVCeSOP/79mQXSGMiAc6l/XuiTMV6tTj8 8Yo25dWULuFoJSblw9Lvbyxm9G5ZoN1ptHHUhxUsvI/xeGsH8fHiVBdCTjWhFNvcGHbh zOKsSz6/yJIFGdTILZGzOQLwrwLxAjzTo9HOKzdJ9edKX5mSlwCEMfdR+ihJtT4407nI x71EaqMeIglCkBm0cYT9h/5V9b9gD5E6lOWmtBqS0prUkkTCzm9rsknv3M/OIhNXbxOU YXD7hHLaKMR2nckmg0Ns/5zVaF+G4tWJ7g0YPIayXqhGmJ+XJw/cZ0Vpb2YfSSdqjSNq ywMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=Ad0DoiAKzwKoTLL/HCg690cbeFEcYdQtGzvCJEzt4Xw=; b=mp24Fhyr97kDrfOaZvWOuxRyHfvszt/hpIcJbz3/XMRANPHkxFzS1SAirSJ2fBHRcJ j1NjC27ft6aT4UGLCXLJjAjYDiqwDlRvLwvALpo8nxHfM/4ZDPXyY2VLkM1LwSt23xK3 D5s+8NxPR8n9nzHmYA+o9btp2cEacwWUs3CBaFbOWmgL1psjcX3T7VqUoU3GNnp6qXXz 0FMpxo4lK+gOmhR+WHirZy1jpcY2dcF2aVwj4A2hc6HB2HB/pKqSD++vbI2/9PIRJNsr RTawagsJCOrVhVz/RKTuV2A6N3aphy5NZYHgPJ67U4okIEerbHPZQxf1XhtiVObNDB4X +JVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lxWeD94g; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.29; Tue, 02 Jul 2019 12:42:29 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lxWeD94g; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726329AbfGBTm3 (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:29 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:39750 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727049AbfGBTm2 (ORCPT ); Tue, 2 Jul 2019 15:42:28 -0400 Received: by mail-lj1-f193.google.com with SMTP id v18so18158402ljh.6 for ; Tue, 02 Jul 2019 12:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ad0DoiAKzwKoTLL/HCg690cbeFEcYdQtGzvCJEzt4Xw=; b=lxWeD94g5yFO8oJjW/CzUVUNo7qwef+GL23iStNXnZM45mM94ZjY7gdpwoFSkheOwf tp7HPHofcHOl2i8JmwJoOXxzZLSJ18UFGTnZHNuqEfgwuF8qkkpX4i9i1m5kJ3JGgZLf 3RcgGloEzBvatSyE01v/iIwNrTvm0SmzbgroWWLXZwSIARChYPX+85VL7hwJTCWsZIdp ncW8T4jQheH4O2h1wxS4oTuHj8+qXKCk/pD/Gz/Fmj1NOZmop5SybaDanT1OmuZSLa3U p+YvF5uXagnoQxvvqaDcuhG3RZV9/HMYmYFCpCmlri4dzyvEqy0UQOyrfwZYeXWOLXIF zEHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ad0DoiAKzwKoTLL/HCg690cbeFEcYdQtGzvCJEzt4Xw=; b=gKiLSaCybINuM8OfdBWXP53Ul6zNpwsquUiLsREJd/wQPnTXYw+jrgvJbKvjGLYTZs cBiBZwYVtoka+6ZT5x0u6TLyGTPbpYtSYtOEDLXeH7ZOyYVZon4rxtgCHT3tas+T0a4V 2RPOQgBnphxgSLsEzfwAOQHlXpwi6rwd9j1oSQL0um9SKoPpTDk2Gret3XwDeus2ThhB tfo0qM9Gcl+15mxIlmnHa7M52wBiW4XFd8qCp97ZazUTfpkAiAsYPD1oIBE332H6eSTm kQHQPSq9Ob1UoksKxggL6y6rOZ25ayK6qw3C5KjFaiId9SzeGeS4ZneZsjrWSJK4yze5 50pQ== X-Gm-Message-State: APjAAAXFJYrg3dvM3J3Uqsv34UB+4BaSgySuyWQESuTRYs1o3921rhzv uQlPKRnQylL2GhLFqsN+GBzHseqWBa6RETAz X-Received: by 2002:a2e:5bdd:: with SMTP id m90mr17802446lje.46.1562096546620; Tue, 02 Jul 2019 12:42:26 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.25 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:25 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 12/32] crypto: arm64/aes-ccm - switch to AES library Date: Tue, 2 Jul 2019 21:41:30 +0200 Message-Id: <20190702194150.10405-13-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The CCM code calls directly into the scalar table based AES cipher for arm64 from the fallback path, and since this implementation is known to be non-time invariant, doing so from a time invariant SIMD cipher is a bit nasty. So let's switch to the AES library - this makes the code more robust, and drops the dependency on the generic AES cipher, allowing us to omit it entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-ce-ccm-glue.c | 18 ++++++------------ 2 files changed, 7 insertions(+), 13 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 1762055e7093..c6032bfb44fb 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -80,8 +80,8 @@ config CRYPTO_AES_ARM64_CE_CCM depends on ARM64 && KERNEL_MODE_NEON select CRYPTO_ALGAPI select CRYPTO_AES_ARM64_CE - select CRYPTO_AES_ARM64 select CRYPTO_AEAD + select CRYPTO_LIB_AES config CRYPTO_AES_ARM64_CE_BLK tristate "AES in ECB/CBC/CTR/XTS modes using ARMv8 Crypto Extensions" diff --git a/arch/arm64/crypto/aes-ce-ccm-glue.c b/arch/arm64/crypto/aes-ce-ccm-glue.c index cb89c80800b5..b9b7cf4b5a8f 100644 --- a/arch/arm64/crypto/aes-ce-ccm-glue.c +++ b/arch/arm64/crypto/aes-ce-ccm-glue.c @@ -46,8 +46,6 @@ asmlinkage void ce_aes_ccm_decrypt(u8 out[], u8 const in[], u32 cbytes, asmlinkage void ce_aes_ccm_final(u8 mac[], u8 const ctr[], u32 const rk[], u32 rounds); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - static int ccm_setkey(struct crypto_aead *tfm, const u8 *in_key, unsigned int key_len) { @@ -127,8 +125,7 @@ static void ccm_update_mac(struct crypto_aes_ctx *key, u8 mac[], u8 const in[], } while (abytes >= AES_BLOCK_SIZE) { - __aes_arm64_encrypt(key->key_enc, mac, mac, - num_rounds(key)); + aes_encrypt(key, mac, mac); crypto_xor(mac, in, AES_BLOCK_SIZE); in += AES_BLOCK_SIZE; @@ -136,8 +133,7 @@ static void ccm_update_mac(struct crypto_aes_ctx *key, u8 mac[], u8 const in[], } if (abytes > 0) { - __aes_arm64_encrypt(key->key_enc, mac, mac, - num_rounds(key)); + aes_encrypt(key, mac, mac); crypto_xor(mac, in, abytes); *macp = abytes; } @@ -209,10 +205,8 @@ static int ccm_crypt_fallback(struct skcipher_walk *walk, u8 mac[], u8 iv0[], bsize = nbytes; crypto_inc(walk->iv, AES_BLOCK_SIZE); - __aes_arm64_encrypt(ctx->key_enc, buf, walk->iv, - num_rounds(ctx)); - __aes_arm64_encrypt(ctx->key_enc, mac, mac, - num_rounds(ctx)); + aes_encrypt(ctx, buf, walk->iv); + aes_encrypt(ctx, mac, mac); if (enc) crypto_xor(mac, src, bsize); crypto_xor_cpy(dst, src, buf, bsize); @@ -227,8 +221,8 @@ static int ccm_crypt_fallback(struct skcipher_walk *walk, u8 mac[], u8 iv0[], } if (!err) { - __aes_arm64_encrypt(ctx->key_enc, buf, iv0, num_rounds(ctx)); - __aes_arm64_encrypt(ctx->key_enc, mac, mac, num_rounds(ctx)); + aes_encrypt(ctx, buf, iv0); + aes_encrypt(ctx, mac, mac); crypto_xor(mac, buf, AES_BLOCK_SIZE); } return err; From patchwork Tue Jul 2 19:41:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168359 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653474ilk; Tue, 2 Jul 2019 12:42:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqwFtLJ46FsvF+VIH8f4mJquChamZ13c7EJOA4r4Q+WXR55GWSexCYNUqydowAr7+qyCOERC X-Received: by 2002:a17:902:fe86:: with SMTP id x6mr15002949plm.67.1562096551225; Tue, 02 Jul 2019 12:42:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096551; cv=none; d=google.com; s=arc-20160816; b=FOz4FbaU0rN4T3219wuuDV+lzAGOI9cn0on7xPapPL281CQyyRjujm7bv0xAcdwwm3 tGB9jcNzbxjTA0O1buDvAFEB8BRA1NR2Lf+AEK2udjHoaU81cxbuIDGdfN1WcsYmYX5u 8EIz7YFaObTQ6f9WCSaZPEeSDfEMrMVz5kutfUCmmSao/V5rM5I7K6iHHTqbKQFkc/Bh wt23HrwcuQO38Ov3Ny56s87hU5/Sm0vT2ld9B+dHy1UTfqEuRHFDVa6eyuPfJkqs9Ngk 1bnVjxvDXlKCzAelxP/Z2rPIagmqROAkS5kq3DZELGriYh8YUDzJZqHb6lSuWoTbZH7Z cVYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=JyLJkqgd2qHLXLovdFPdmSe6yZQW5DMw8EodB7HW1iY=; b=ZGOL6FWkmn47/1yZEbpjsSdfgZMo9Su/yOV6wkrKSXUxVvZbUIC31Pj961O93LxkX8 OdZVkXmIBDJGHxhBiIuV22iK96xN7nKHJRNwPg4ZyDppY2oFVNEIeY+oNTzSCNTnzC78 peCf71OmlYpNtvfW5kZaPpeV9c03FGBLlPPt35BLzVGRNukfGoA439klbpEQv1YI2X0r X5QByWxIjUjWj9m+xgK31YQwqBGJwO4pO3R2A3Mwmr/05OBU3cmC7NFLvD3olDcuW5y0 wVSIFN/GUb4FqndmOgTTt6dRmWFTOIr7aR4WjACOG7+Sf12WFXVUxaayoBpdVKkaMOJC MJ7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j9AdDAXF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.31; Tue, 02 Jul 2019 12:42:31 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=j9AdDAXF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727072AbfGBTma (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:30 -0400 Received: from mail-lf1-f66.google.com ([209.85.167.66]:39118 "EHLO mail-lf1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727049AbfGBTma (ORCPT ); Tue, 2 Jul 2019 15:42:30 -0400 Received: by mail-lf1-f66.google.com with SMTP id p24so12264678lfo.6 for ; Tue, 02 Jul 2019 12:42:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JyLJkqgd2qHLXLovdFPdmSe6yZQW5DMw8EodB7HW1iY=; b=j9AdDAXFB7D5cxPfRsER1dOZWlG17/GiNQSXf3XBF91g/VaDnKGYmWABb2MhnL3+up 9g26LDey9j755oxYSfQTG7IcGKLdkcOjFzC0AQeEmntKmc2dS1Ra4YOjm4yGJU+t/T2U 7yahW7dCY2y+J3VHxbGBFtGQxPI0xy9ID33hgX1M660rcR5m4/W2JwsQCF85s88Wtpqv 5pfZVEd2LEQvGPW6hmcJruCCfQ4cu4O13MxlJa9HAQPrc9C69k7pmyYkVa5/XZj6G9Du M6qVtj+utEQJXJPK+2d+A2CTtx0KpJ4nOBE7IeVQmQkN+t+3PPfdWKE4pmooks1TV4Hn C9QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JyLJkqgd2qHLXLovdFPdmSe6yZQW5DMw8EodB7HW1iY=; b=IqHFUJIYCqf+BolFCUmiyjuWyyvrnsHdYpsxgx6hKj0Ol3Ew//H7DpPwX5nYRYsQp5 L5IPwn+92sSLzc3o7nRJYxVTqlQuiOduO2ywv8f2AiCCf2vCaytIdz+SmVxLFULI22oi sYTtjxjtGufuXdoqYN6gSzT0OFzAcIXBWmZvtsvplG/ccYGjuk3tlLkOU2BH1x2jKxja nuVQ61NXgbVSuwxO7u13abHNtgoll1eV6Zf7DN9B9p4pcwYOxAoWp3pCDED8IVgF54Ko bUwfaMcHgPYwH05cC1hOxHtCXp8IJjgSxxsfHHA2wYjd4SEMtM/otwA6GkU9AquAn5tX VmLQ== X-Gm-Message-State: APjAAAWPBiujw8nWaG6fW7LNDSHVbdN10f1p1ZI1Y7qWVNrRvtN+/QNx SMhPzTfAff2W3fKh3hgHshXEIPGmrPMURfPx X-Received: by 2002:ac2:4ace:: with SMTP id m14mr14146722lfp.99.1562096547981; Tue, 02 Jul 2019 12:42:27 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.26 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:27 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 13/32] crypto: arm64/aes-neonbs - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:31 +0200 Message-Id: <20190702194150.10405-14-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 1 + arch/arm64/crypto/aes-neonbs-glue.c | 8 ++++---- 2 files changed, 5 insertions(+), 4 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index c6032bfb44fb..17bf5dc10aad 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -116,6 +116,7 @@ config CRYPTO_AES_ARM64_BS select CRYPTO_BLKCIPHER select CRYPTO_AES_ARM64_NEON_BLK select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES select CRYPTO_SIMD endif diff --git a/arch/arm64/crypto/aes-neonbs-glue.c b/arch/arm64/crypto/aes-neonbs-glue.c index 02b65d9eb947..cb8d90f795a0 100644 --- a/arch/arm64/crypto/aes-neonbs-glue.c +++ b/arch/arm64/crypto/aes-neonbs-glue.c @@ -77,7 +77,7 @@ static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -136,7 +136,7 @@ static int aesbs_cbc_setkey(struct crypto_skcipher *tfm, const u8 *in_key, struct crypto_aes_ctx rk; int err; - err = crypto_aes_expand_key(&rk, in_key, key_len); + err = aes_expandkey(&rk, in_key, key_len); if (err) return err; @@ -208,7 +208,7 @@ static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key, struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); int err; - err = crypto_aes_expand_key(&ctx->fallback, in_key, key_len); + err = aes_expandkey(&ctx->fallback, in_key, key_len); if (err) return err; @@ -274,7 +274,7 @@ static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, return err; key_len /= 2; - err = crypto_aes_expand_key(&rk, in_key + key_len, key_len); + err = aes_expandkey(&rk, in_key + key_len, key_len); if (err) return err; From patchwork Tue Jul 2 19:41:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168360 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653489ilk; Tue, 2 Jul 2019 12:42:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqzctiNrE+YAHJQyP1SEZQU6fE7zS7yDwqkaLqmBJM5WipCxi2NBTT/VQtyKDIsfqKRULQOD X-Received: by 2002:a17:90b:f0e:: with SMTP id br14mr7471547pjb.117.1562096552260; Tue, 02 Jul 2019 12:42:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096552; cv=none; d=google.com; s=arc-20160816; b=UDEJELjl0KEOqQCPMcezK5DcWIohGqR15fWZBT69JPbIG3kBpos8bB6GGVdUSOSRPJ 0sdfRFuPJS4/PUWdnxnn+hYsAEt2VXmj/9KnP8oLDBMapzhE+JDv4E87WzIYHlfzQBj9 kEa3CbhSCDVWPm0zhbJgjcjhhGd5sUuu3uD1bjUrzxL7ccUp6RRrmqcfL29E5iGVLoyn dac943JipVyDNnGX124+aEii/4dvYbfr5UFu3qmjfbPivlbEK0DbwTxOmZDmBzPqIgde 7qOHV38gGg/SoEruq0P3Oig1KzgMEXRh/XnGYJBRWAk5IhOowX1LXrDYhaNCsPR5inPU h8FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=VJ8dT+03h95rj29GtE59SlVplLQkYEqzcrRrbYuxp50=; b=gZuMCKLW7T7QSptv7wqnAuvMJIJ1SiCyJuVaa9t2xC2WP2ftMZdmcuJFDtV28ezWW5 8s8X6aQho6+1MYg4PIPqVoV8ysWkrgjIt/RbFfZnVZNBjTc1dQECC9oFqtqAsmZd8Q5H HCFb0aZ3Bu92r30buYoSofqFfpS1tiBc1wxhAn0UMtTW4JZzOW6pIzazW6FQiST0pmb5 +EptcoVaXXHI4f5bLx+ACwkrjdJfQpWoGr6gc4g5CInTFCI/MWXCYq3z8zrSie/bIh/N BeIge5LQkxsJuEJwMhayV5uDuRGxicJYQnztMKIr6ogQKXFBVQr/ZLvMjPj0HQLzRw0k F4FQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PbegUQsr; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.32; Tue, 02 Jul 2019 12:42:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PbegUQsr; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727049AbfGBTmb (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:31 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:42977 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmb (ORCPT ); Tue, 2 Jul 2019 15:42:31 -0400 Received: by mail-lj1-f195.google.com with SMTP id t28so18176753lje.9 for ; Tue, 02 Jul 2019 12:42:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VJ8dT+03h95rj29GtE59SlVplLQkYEqzcrRrbYuxp50=; b=PbegUQsrlKcn6CxE553AjHD/kRwxIvh7GkW9LBBOqfLlZfAmDEavHpLWOExPDshaRH I//3CADJbdzJ1pJbFS5QdbAdKoG9R7cKBZW8qSQfV36aKZUy2MhPSn+Avn5449g9ox4W Qh21rHXr0l81nlMZms5K9ICP1MYFu9BorR/W3xY1e1zlWPi5T57mySIVtmXY0p2uicwY +cUhT1vd91uS6xDJCDhQPzSusFyptLBtQ4J0MFEGqOqjL4ZgROCti+qbDbn1bi+6gGVr YccK0OhDCsfwb1sRAso7yc8fG27JuHWtL4CApz/nSyaPnrcH4WMbxf8xMRdfV3RiIfys DooQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VJ8dT+03h95rj29GtE59SlVplLQkYEqzcrRrbYuxp50=; b=gbqHcMFmGF42sbCaTX0z9kP1c81N6AP0d1ulHG6GLe/f6bM1mwZXMgKnC4nGM5pY4W ZtE+QiKmxEyHk67r2juTYplwrZ5T+XwvwldsGaqIiQGQLItfK+u8rwIpox6RofNapwBJ xy/T1QBstFWBGHvnQgzovy8sq0hJP+zRRb016OH/9s8oEByr5mAwk95ck719dgdtRHUu IouAPanKhbqnlRy7IGY3iyKITUERdaKsFmnccj8yB2pxs10KAeWxeXJFS6KX0sN9kMpV LIoIO8egOqqTbiV+5yv8bC0W9whu7WcIDKyMoOjQ5OdfD3c+j9sRHLN6kH6ty4YK3uDE /kLw== X-Gm-Message-State: APjAAAUtVkkBaOmfUozFWjrc5B+PpTIgOCckgHMs388b7IYWKZlWIInP BkM/2F+ctm8Jl+H0wHd4ES0bw8lXuaPMmYiM X-Received: by 2002:a2e:894a:: with SMTP id b10mr3036421ljk.99.1562096549392; Tue, 02 Jul 2019 12:42:29 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.28 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:28 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 14/32] crypto: arm64/aes-ce - switch to library version of key expansion routine Date: Tue, 2 Jul 2019 21:41:32 +0200 Message-Id: <20190702194150.10405-15-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Switch to the new AES library that also provides an implementation of the AES key expansion routine. This removes the dependency on the generic AES cipher, allowing it to be omitted entirely in the future. While at it, remove some references to the table based arm64 version of AES and replace them with AES library calls as well. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-glue.c | 17 ++++++++++------- 2 files changed, 11 insertions(+), 8 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 17bf5dc10aad..66dea518221c 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -96,7 +96,7 @@ config CRYPTO_AES_ARM64_NEON_BLK depends on KERNEL_MODE_NEON select CRYPTO_BLKCIPHER select CRYPTO_AES_ARM64 - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_SIMD config CRYPTO_CHACHA20_NEON diff --git a/arch/arm64/crypto/aes-glue.c b/arch/arm64/crypto/aes-glue.c index f0ceb545bd1e..3c80345d914f 100644 --- a/arch/arm64/crypto/aes-glue.c +++ b/arch/arm64/crypto/aes-glue.c @@ -26,7 +26,6 @@ #ifdef USE_V8_CRYPTO_EXTENSIONS #define MODE "ce" #define PRIO 300 -#define aes_setkey ce_aes_setkey #define aes_expandkey ce_aes_expandkey #define aes_ecb_encrypt ce_aes_ecb_encrypt #define aes_ecb_decrypt ce_aes_ecb_decrypt @@ -42,8 +41,6 @@ MODULE_DESCRIPTION("AES-ECB/CBC/CTR/XTS using ARMv8 Crypto Extensions"); #else #define MODE "neon" #define PRIO 200 -#define aes_setkey crypto_aes_set_key -#define aes_expandkey crypto_aes_expand_key #define aes_ecb_encrypt neon_aes_ecb_encrypt #define aes_ecb_decrypt neon_aes_ecb_decrypt #define aes_cbc_encrypt neon_aes_cbc_encrypt @@ -121,7 +118,14 @@ struct mac_desc_ctx { static int skcipher_aes_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { - return aes_setkey(crypto_skcipher_tfm(tfm), in_key, key_len); + struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + int ret; + + ret = aes_expandkey(ctx, in_key, key_len); + if (ret) + crypto_skcipher_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); + + return ret; } static int xts_set_key(struct crypto_skcipher *tfm, const u8 *in_key, @@ -649,15 +653,14 @@ static void mac_do_update(struct crypto_aes_ctx *ctx, u8 const in[], int blocks, kernel_neon_end(); } else { if (enc_before) - __aes_arm64_encrypt(ctx->key_enc, dg, dg, rounds); + aes_encrypt(ctx, dg, dg); while (blocks--) { crypto_xor(dg, in, AES_BLOCK_SIZE); in += AES_BLOCK_SIZE; if (blocks || enc_after) - __aes_arm64_encrypt(ctx->key_enc, dg, dg, - rounds); + aes_encrypt(ctx, dg, dg); } } } From patchwork Tue Jul 2 19:41:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168361 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653526ilk; Tue, 2 Jul 2019 12:42:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqy79uAAVuj3ocBJCJ/wmp1ZYh+O3xus7P9BRCbLuyYHBdQqtwFpb/bCUxyc7THEMCvY2UfZ X-Received: by 2002:a17:902:1101:: with SMTP id d1mr37540922pla.212.1562096554902; Tue, 02 Jul 2019 12:42:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096554; cv=none; d=google.com; s=arc-20160816; b=dW6qePD04+k9K4E2oK+736i6/IRLUNXcZUUyepAX7CipqWNRVG3Nae96gD9EyGODpB Ugir51iQBaqr4TcvkcNi3+Pp4gBdIoTiZAt8KJQBeduqg8fgyaePoP3+fwkcWMWiPiMu h09BfEjQP3RLK/Zy2XTlsG6Ouljb5O6urR3C63alXwUR4A25LEPdTvYRy1RF4IgAAqhH WIPe66i9LK/wfBCGJZwSe3bzEv2nMR9vUE9FPNA3wbLYOjZveMXHQzs6Z7BqA0NIZloc qOYsnoT8ziEpWHpo5fQVXptfDbg9FyOd7w8/4seoYK785Fn20MgRtekT0kwrt4HDpJY/ 20FA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=EQ+7881FqzZH4XJUNCnQNTsoZJakZlk4h+tK69pjZnw=; b=F0u/oQhkNgSvAyOX1Wgk/Tske6wQNILQZ1e6S7h/p0273q0zMe7aPvaev+XZPMD47l aO26/qLuzuAEqz6yYWAr5JMgL5JKOnB5lz0afobLkTAauQm34dxZI8T4RO3KCiDy5YeH yngrUqes8Zfu51xLNYgV+tOl7H98MeoMLWG8t54FxYPepQcexX3ZavlCXuXm0YeQKF+8 fA/xyZR4u1bm3AMr8aeYZ7unm7xbwEX5Ag6Jk6mo2cCMyu0syxJRtlbwQdfgAe0aNWSh zKyo+Bm4HxEcGatQtOGWSd1YfawWBLMVCBcseAEuuLOulfWIf10qXnkn/ix64R7A0ZWt t0bQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KSqQ31mT; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.34; Tue, 02 Jul 2019 12:42:34 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KSqQ31mT; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727076AbfGBTme (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:34 -0400 Received: from mail-lf1-f65.google.com ([209.85.167.65]:47059 "EHLO mail-lf1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727074AbfGBTmd (ORCPT ); Tue, 2 Jul 2019 15:42:33 -0400 Received: by mail-lf1-f65.google.com with SMTP id z15so12235871lfh.13 for ; Tue, 02 Jul 2019 12:42:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EQ+7881FqzZH4XJUNCnQNTsoZJakZlk4h+tK69pjZnw=; b=KSqQ31mTMR68uqoDDMtL18VTyUwNomEO6rBJl3cZAEG8HW6OPTdeCayCfVuFkuHCiJ 630Ksb8d2TGsFnmZ14/Egsfh2/kWZpjTlN9hHrT1hKOeOh2PtX65ABGdqEMc+8TxOI7z 2YRmE7kkF/Ft27kDoEFqsvuFJAOiqGc8oNiiQzBv75y4NnMUw4A5tIimYsmR9VOswKsa 6NS0SuhhqOOPXhfAvBnuiSq3+JwvpqsBb24gTZDWmnF3h4Brn0hY/pYtRSYQsVPDlF+H OEwKTaqu4bAnFnnlzpxkMjpKqqb4GjPKFZA+RD+MtGCRTnETa9XlToUqN4lwt4OT3Gn+ vPEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EQ+7881FqzZH4XJUNCnQNTsoZJakZlk4h+tK69pjZnw=; b=FCuZ9vwaZBdTgE8aWSZ5kf1MLOadg28WAlmOXVPq2VU+YxXCGT8DuC7v12xVhPVgmt lfEs856aqMVpkcmfAr19hTy11oBH2bSb1Dqz5nPxBS0xvaJBkdll7krBwud3+BVWllfV UlauB8zOrJ3Uh9S82ve431x8HZKGS9vsi5snh2Zr+dvn1ThuJx6LAreH8ScCL21L8eE0 e2700uUJL8ivoT1TOtFvp0iqssZ4PsCRHspiiCI463PlrOkdj0vyuOcLL6dsKx16OPww fKEy+eB3ZM0uKAz1u9CCzHRCCgpxMwhl8peQ2OxSMuVHPWkDKtJZk4tl6NANCbYIjaV7 fwkQ== X-Gm-Message-State: APjAAAW2tRkgpd/+2ae3YU3KNLlW8XV56R0qlougptFRRaaP5H2PukD9 +DuomV40ChxqcoGrVea4wC4J2OzmXUk/uqaY X-Received: by 2002:a19:6602:: with SMTP id a2mr15069293lfc.25.1562096550597; Tue, 02 Jul 2019 12:42:30 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.29 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:29 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 15/32] crypto: generic/aes - drop key expansion routine in favor of library version Date: Tue, 2 Jul 2019 21:41:33 +0200 Message-Id: <20190702194150.10405-16-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Drop aes-generic's version of crypto_aes_expand_key(), and switch to the key expansion routine provided by the AES library. AES key expansion is not performance critical, and it is better to have a single version shared by all AES implementations. Signed-off-by: Ard Biesheuvel --- crypto/Kconfig | 1 + crypto/aes_generic.c | 153 +------------------- include/crypto/aes.h | 2 - 3 files changed, 3 insertions(+), 153 deletions(-) -- 2.17.1 diff --git a/crypto/Kconfig b/crypto/Kconfig index df6f0be66574..80ea118600ab 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -1072,6 +1072,7 @@ config CRYPTO_LIB_AES config CRYPTO_AES tristate "AES cipher algorithms" select CRYPTO_ALGAPI + select CRYPTO_LIB_AES help AES cipher algorithms (FIPS-197). AES uses the Rijndael algorithm. diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index 3aa4a715c216..426deb437f19 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -1125,155 +1125,6 @@ EXPORT_SYMBOL_GPL(crypto_fl_tab); EXPORT_SYMBOL_GPL(crypto_it_tab); EXPORT_SYMBOL_GPL(crypto_il_tab); -/* initialise the key schedule from the user supplied key */ - -#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b) - -#define imix_col(y, x) do { \ - u = star_x(x); \ - v = star_x(u); \ - w = star_x(v); \ - t = w ^ (x); \ - (y) = u ^ v ^ w; \ - (y) ^= ror32(u ^ t, 8) ^ \ - ror32(v ^ t, 16) ^ \ - ror32(t, 24); \ -} while (0) - -#define ls_box(x) \ - crypto_fl_tab[0][byte(x, 0)] ^ \ - crypto_fl_tab[1][byte(x, 1)] ^ \ - crypto_fl_tab[2][byte(x, 2)] ^ \ - crypto_fl_tab[3][byte(x, 3)] - -#define loop4(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[4 * i]; \ - ctx->key_enc[4 * i + 4] = t; \ - t ^= ctx->key_enc[4 * i + 1]; \ - ctx->key_enc[4 * i + 5] = t; \ - t ^= ctx->key_enc[4 * i + 2]; \ - ctx->key_enc[4 * i + 6] = t; \ - t ^= ctx->key_enc[4 * i + 3]; \ - ctx->key_enc[4 * i + 7] = t; \ -} while (0) - -#define loop6(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[6 * i]; \ - ctx->key_enc[6 * i + 6] = t; \ - t ^= ctx->key_enc[6 * i + 1]; \ - ctx->key_enc[6 * i + 7] = t; \ - t ^= ctx->key_enc[6 * i + 2]; \ - ctx->key_enc[6 * i + 8] = t; \ - t ^= ctx->key_enc[6 * i + 3]; \ - ctx->key_enc[6 * i + 9] = t; \ - t ^= ctx->key_enc[6 * i + 4]; \ - ctx->key_enc[6 * i + 10] = t; \ - t ^= ctx->key_enc[6 * i + 5]; \ - ctx->key_enc[6 * i + 11] = t; \ -} while (0) - -#define loop8tophalf(i) do { \ - t = ror32(t, 8); \ - t = ls_box(t) ^ rco_tab[i]; \ - t ^= ctx->key_enc[8 * i]; \ - ctx->key_enc[8 * i + 8] = t; \ - t ^= ctx->key_enc[8 * i + 1]; \ - ctx->key_enc[8 * i + 9] = t; \ - t ^= ctx->key_enc[8 * i + 2]; \ - ctx->key_enc[8 * i + 10] = t; \ - t ^= ctx->key_enc[8 * i + 3]; \ - ctx->key_enc[8 * i + 11] = t; \ -} while (0) - -#define loop8(i) do { \ - loop8tophalf(i); \ - t = ctx->key_enc[8 * i + 4] ^ ls_box(t); \ - ctx->key_enc[8 * i + 12] = t; \ - t ^= ctx->key_enc[8 * i + 5]; \ - ctx->key_enc[8 * i + 13] = t; \ - t ^= ctx->key_enc[8 * i + 6]; \ - ctx->key_enc[8 * i + 14] = t; \ - t ^= ctx->key_enc[8 * i + 7]; \ - ctx->key_enc[8 * i + 15] = t; \ -} while (0) - -/** - * crypto_aes_expand_key - Expands the AES key as described in FIPS-197 - * @ctx: The location where the computed key will be stored. - * @in_key: The supplied key. - * @key_len: The length of the supplied key. - * - * Returns 0 on success. The function fails only if an invalid key size (or - * pointer) is supplied. - * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes - * key schedule plus a 16 bytes key which is used before the first round). - * The decryption key is prepared for the "Equivalent Inverse Cipher" as - * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is - * for the initial combination, the second slot for the first round and so on. - */ -int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len) -{ - u32 i, t, u, v, w, j; - - if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 && - key_len != AES_KEYSIZE_256) - return -EINVAL; - - ctx->key_length = key_len; - - ctx->key_enc[0] = get_unaligned_le32(in_key); - ctx->key_enc[1] = get_unaligned_le32(in_key + 4); - ctx->key_enc[2] = get_unaligned_le32(in_key + 8); - ctx->key_enc[3] = get_unaligned_le32(in_key + 12); - - ctx->key_dec[key_len + 24] = ctx->key_enc[0]; - ctx->key_dec[key_len + 25] = ctx->key_enc[1]; - ctx->key_dec[key_len + 26] = ctx->key_enc[2]; - ctx->key_dec[key_len + 27] = ctx->key_enc[3]; - - switch (key_len) { - case AES_KEYSIZE_128: - t = ctx->key_enc[3]; - for (i = 0; i < 10; ++i) - loop4(i); - break; - - case AES_KEYSIZE_192: - ctx->key_enc[4] = get_unaligned_le32(in_key + 16); - t = ctx->key_enc[5] = get_unaligned_le32(in_key + 20); - for (i = 0; i < 8; ++i) - loop6(i); - break; - - case AES_KEYSIZE_256: - ctx->key_enc[4] = get_unaligned_le32(in_key + 16); - ctx->key_enc[5] = get_unaligned_le32(in_key + 20); - ctx->key_enc[6] = get_unaligned_le32(in_key + 24); - t = ctx->key_enc[7] = get_unaligned_le32(in_key + 28); - for (i = 0; i < 6; ++i) - loop8(i); - loop8tophalf(i); - break; - } - - ctx->key_dec[0] = ctx->key_enc[key_len + 24]; - ctx->key_dec[1] = ctx->key_enc[key_len + 25]; - ctx->key_dec[2] = ctx->key_enc[key_len + 26]; - ctx->key_dec[3] = ctx->key_enc[key_len + 27]; - - for (i = 4; i < key_len + 24; ++i) { - j = key_len + 24 - (i & ~3) + (i & 3); - imix_col(ctx->key_dec[j], ctx->key_enc[i]); - } - return 0; -} -EXPORT_SYMBOL_GPL(crypto_aes_expand_key); - /** * crypto_aes_set_key - Set the AES key. * @tfm: The %crypto_tfm that is used in the context. @@ -1281,7 +1132,7 @@ EXPORT_SYMBOL_GPL(crypto_aes_expand_key); * @key_len: The size of the key. * * Returns 0 on success, on failure the %CRYPTO_TFM_RES_BAD_KEY_LEN flag in tfm - * is set. The function uses crypto_aes_expand_key() to expand the key. + * is set. The function uses aes_expand_key() to expand the key. * &crypto_aes_ctx _must_ be the private data embedded in @tfm which is * retrieved with crypto_tfm_ctx(). */ @@ -1292,7 +1143,7 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, u32 *flags = &tfm->crt_flags; int ret; - ret = crypto_aes_expand_key(ctx, in_key, key_len); + ret = aes_expandkey(ctx, in_key, key_len); if (!ret) return 0; diff --git a/include/crypto/aes.h b/include/crypto/aes.h index d0067fca0cd0..0a64a977f9b3 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -35,8 +35,6 @@ extern const u32 crypto_il_tab[4][256] ____cacheline_aligned; int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); -int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key, - unsigned int key_len); /** * aes_expandkey - Expands the AES key as described in FIPS-197 From patchwork Tue Jul 2 19:41:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168362 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653529ilk; Tue, 2 Jul 2019 12:42:35 -0700 (PDT) X-Google-Smtp-Source: APXvYqynZNEWIPtnlBJUhri5st2X3e6igrATNhRQ4D5wXkK7OePsw7EHqtXCC8J7P2VONZuLHePP X-Received: by 2002:a17:90a:8c90:: with SMTP id b16mr7448758pjo.133.1562096555179; Tue, 02 Jul 2019 12:42:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096555; cv=none; d=google.com; s=arc-20160816; b=wE0I3g7VzSlDDq/9uwBidMv69/yXIsaFS19fPhkh/0v53t2BeP0TPiaZ+qSc4vzTtp Xwwc2yX2SHe4bd27czRM7o+J44AtEYfiMIbUSXQjUefjOYb639Cs/xYMBTg2CXsokQqZ zVD+UO0wTF6ra9MXliKNXMQsCv0nJTwLBPE1PvARd0llbuaSnpulbkEdh3O4/+4ndlV2 cT8idKePRKkonB70GGCTyUBeClHjK0b22WardfWcRyoRLdPQb85lBgLhjxxHDZEX8Eb5 jt55ctuPPfEyAYmEEMdrbrRevr6LSXvtr1CajFCeFQy4EC77h9MQJhnzpBWlpgdoeXDD RXXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=cNds9JAEq1hTUuemrXbQh9sj10to3vETzPkNzlmk2jM=; b=GghvJt62TIytoIDYLpGkkWD09NnDDjbBsHf8T85m+ACYzQzuvvTNHzIAmBqrXH2yd/ IT/RaGIKQLpaYtcW4S/MD/9TGmr3M7WQ3oSzh1MT8DtYp0Qb0OgrI1pY2PX0ZB8gkH5+ 6Y470LDEYHJPBxwnUR63Lb6feIqG4QeBJE+Lnd786E9q8zmTD/ZkBqvovD3Lhah12Mqt 3rdKcVPiE3BTUgXm77M/9es51J7Noi9AF9PpOycBTgfHFWaxbFpS4xvsoCyP6YNqaXHO zJNyFPZ0H1xb5fb9gBHUoX6n0A1UFaW4324PFL2UXUsvGGMGmK/QSCppM6ztYOZ8MZp4 9EJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u38mRObk; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.34; Tue, 02 Jul 2019 12:42:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u38mRObk; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727074AbfGBTme (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:34 -0400 Received: from mail-lf1-f67.google.com ([209.85.167.67]:34597 "EHLO mail-lf1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTme (ORCPT ); Tue, 2 Jul 2019 15:42:34 -0400 Received: by mail-lf1-f67.google.com with SMTP id b29so2309621lfq.1 for ; Tue, 02 Jul 2019 12:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cNds9JAEq1hTUuemrXbQh9sj10to3vETzPkNzlmk2jM=; b=u38mRObkPGCNBpkUfqFku8ZoUy0UumUJnobS0qIyeuCnatakwjiXwIbv5q+4gcD1/f hYpaqvE42a5zxZ/usTOvnSI8b3jOwElOCHwPQYQW3ROwY/T2690uL/bfxsmeTasOeTP3 9usz4Xg7U/zL5Xtoa92i8QccFmxZu5LiBt6Ipvj3Y2kEaHVTy6XUT+GE07S2tHMP/m+W YafR6IEU/XjJV87YhXqBOdImyaydmRwGupfGaakTMPiY7kH/v5V/mJoPhiR1Jz3TrGom WMZlYooS06YsixisHAUprj9ISqe2Sl8EmSfT7rydvgUt5MBpqvgTW13w//FgPcJYvY5R tmLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cNds9JAEq1hTUuemrXbQh9sj10to3vETzPkNzlmk2jM=; b=TxQpWk94ywE5dwYcUijjmJ7td3fO8M8iHFNOpZejxv5quDMZlk93ADJS21iNaNk9F0 gbi2+NXVA8je/YlbJGmrW32tbaKvDWpvWREoRHaBZc9VxkNgCWeMKxDAwH8ACIZekLyX wuSgjXQUSSmc2hLg20qfu4CBAf+C7O6HjXqfa26LZSLlCZubgUQBW6y2NiQyqJoT20QN NDgHoznDaJUH69hwtDuxrYon2mRxCtMLtILdhZBvCaXNaFBPGTaUfDqib8GLqlRSVIB1 XEdjCUg6aaqUxdPinyTBBdfMPtQrmwmvNXdUV0NSpdh03Bo8CsOl4b2YgRXPJRSH7zAU bARA== X-Gm-Message-State: APjAAAWmanSc4WxJi6hCn5+oL3h4wCghKfeVWq7qh1IecxpMZWCGwOdU 8Q28y8O0zXHCXqpoBgVAfAnWs25A4NWA8OnF X-Received: by 2002:ac2:52b7:: with SMTP id r23mr1871076lfm.120.1562096551941; Tue, 02 Jul 2019 12:42:31 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.30 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:31 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 16/32] crypto: ctr - add helper for performing a CTR encryption walk Date: Tue, 2 Jul 2019 21:41:34 +0200 Message-Id: <20190702194150.10405-17-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Add a static inline helper modeled after crypto_cbc_encrypt_walk() that can be reused for SIMD algorithms that need to implement a non-SIMD fallback for performing CTR encryption. Signed-off-by: Ard Biesheuvel --- include/crypto/ctr.h | 50 ++++++++++++++++++++ 1 file changed, 50 insertions(+) -- 2.17.1 diff --git a/include/crypto/ctr.h b/include/crypto/ctr.h index 4180fc080e3b..d64017fae41c 100644 --- a/include/crypto/ctr.h +++ b/include/crypto/ctr.h @@ -13,8 +13,58 @@ #ifndef _CRYPTO_CTR_H #define _CRYPTO_CTR_H +#include +#include +#include +#include + #define CTR_RFC3686_NONCE_SIZE 4 #define CTR_RFC3686_IV_SIZE 8 #define CTR_RFC3686_BLOCK_SIZE 16 +static inline int crypto_ctr_encrypt_walk(struct skcipher_request *req, + void (*fn)(struct crypto_skcipher *, + const u8 *, u8 *)) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); + int blocksize = crypto_skcipher_chunksize(tfm); + u8 buf[MAX_CIPHER_BLOCKSIZE]; + struct skcipher_walk walk; + int err; + + /* avoid integer division due to variable blocksize parameter */ + if (WARN_ON_ONCE(!is_power_of_2(blocksize))) + return -EINVAL; + + err = skcipher_walk_virt(&walk, req, false); + + while (walk.nbytes > 0) { + u8 *dst = walk.dst.virt.addr; + u8 *src = walk.src.virt.addr; + int nbytes = walk.nbytes; + int tail = 0; + + if (nbytes < walk.total) { + tail = walk.nbytes & (blocksize - 1); + nbytes -= tail; + } + + do { + int bsize = min(nbytes, blocksize); + + fn(tfm, walk.iv, buf); + + crypto_xor_cpy(dst, src, buf, bsize); + crypto_inc(walk.iv, blocksize); + + dst += bsize; + src += bsize; + nbytes -= bsize; + } while (nbytes > 0); + + err = skcipher_walk_done(&walk, tail); + } + return err; +} + #endif /* _CRYPTO_CTR_H */ From patchwork Tue Jul 2 19:41:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168364 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653579ilk; Tue, 2 Jul 2019 12:42:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqzdRGXYklTe+54tMMA+k8pa4HVie4vt0w/NwgD+O5TLQq31yV4FvBI7gJE2j0DXMDcFO1lY X-Received: by 2002:a65:5c47:: with SMTP id v7mr10456991pgr.44.1562096557924; Tue, 02 Jul 2019 12:42:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096557; cv=none; d=google.com; s=arc-20160816; b=Fm56qvQEhlGY0WY6b9dnB5eNDo+Hru/tv4Umuy1ATmET4yEgD4pCqN/HJbskXPePMf PrV9h7hljoBnYQCMsro0M1UprWN96NRiZv5cmGQ4e0QEuJSj5Z7b9nKVZhttCA2G0v1S 7nmDZ6jYjMHkYUfJT9AYnOCIHkbV3P7N3q3F/VdJsUMsRpSifm/rQ3q4O+5wD6najytQ UlghdktZMllWM3pWAF2SVXTr+mxIxHH4cCJznhEY0LOwECUM7cSRKF0YSAl2qNXYWdxX DAl4o+GcKN40PJDDlh1St0Fh+ual2pF/a/HAtHtHV0LYidEp1uyRa0hvzw9IAX8M7SFN 7x5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=0MCWUw3vDUq5xLL2N3UUfYVxMc1CoRLbEbf+zv4bCoE=; b=ETo1JQh4KO7ppMMn2zusneE15/DDw7GULEZYoAveKD5ZSuAFgOjWoqjJFxg8rkORER PmhqqcgjiQrInj7hr8FM83kFr/qVImxnybs04ZBUssI5tuyXuxsKKVeY3yCO6uVrv/3i OEt/lN1YY3YPYN3MHgTad0n4+W8I2icMsF8n+Sze/S+6wKZxrz6uaeHF3R4ABO+QnDm7 U0nO5Laz2Kg+c91YVrdspK8urB6/ssQ5avPOLjbAmlRW3kYU0hczF/5yU33/Vpv7xxH8 dyTgf1ak0JPyfoHKJeTOTnFzNTd7Yw88mRJj68UGcA19SGVJWiEp6M5HF3HpBBUWBcOJ m+Gw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i7+sZ9Ig; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.37; Tue, 02 Jul 2019 12:42:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i7+sZ9Ig; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727082AbfGBTmh (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:37 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:46350 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmh (ORCPT ); Tue, 2 Jul 2019 15:42:37 -0400 Received: by mail-lj1-f193.google.com with SMTP id v24so18173079ljg.13 for ; Tue, 02 Jul 2019 12:42:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0MCWUw3vDUq5xLL2N3UUfYVxMc1CoRLbEbf+zv4bCoE=; b=i7+sZ9Igg4lFlDQipgk5rFK5Kut6D04gI5g4VSt8nyeAs9CnyeExlo369v4slcQel0 3msnORzG9uRH+7JRWmobhA1wREy8DMgFzO80oCFY/7WezPLU7CXPUyquTmEbFrKsLj/d hB9uQXvY6qeoWa+RV0K3L+Sx/KHGeXuNqMdwQE2iGagm3C9fNtzIc6ujO6wOzGmj2VG9 E0Lry3+EwvQE63pH1EcA4ExhBqGL7FZkCczZKpxBTYX0PManuADqNOJpk/xoI2xr+0o5 z3TcEsUIUb2ZA+hvckFl1vZ6p5B56tX4Gu1FvenrIAUxag9tRRAntC5HsKjU5DtL+Mba VmAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0MCWUw3vDUq5xLL2N3UUfYVxMc1CoRLbEbf+zv4bCoE=; b=gMYJioALH8CtQlRUMqeWJkKuBYO4Y4iO0hodSsvwrEUqcfFbvi6m1iVaiFJGA4S5+l KrFnjLvSrZajj3oNfc184HLU5SIwMbTDIEApmYzdLSX+6Mpk0E1VAxcAOapy3sdHkV+G QLdgNmBTqxM0k0UnEmDC9z+7c3/iiwcznF64wuZx42X7vyDWI7ntdF3QAo4bHjj4wyde 7KGsbjlXOn8WSZNmw+YdAs2eA59DQKI4/zBYGbBiS1Agpz/R9C3Lqotpiyiwm1cxAKx2 lMzp4E3ufS6wg809NVtcjhulOnlN3o0gyM4st6tdevvWTyoHzfAkWjOSS1uYh/iLdyv0 Zr7Q== X-Gm-Message-State: APjAAAUPggCkVv4l+WJVmFclV52sbdzXBXC0CJuRmqvBAC3UpT21Jghj xL0yFN//YoPRZo7NBfAgWxNqPLNxJAv4YCXC X-Received: by 2002:a2e:3602:: with SMTP id d2mr18733543lja.112.1562096553145; Tue, 02 Jul 2019 12:42:33 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.32 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:32 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 17/32] crypto: aes - move sync ctr(aes) to AES library and generic helper Date: Tue, 2 Jul 2019 21:41:35 +0200 Message-Id: <20190702194150.10405-18-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org In preparation of duplicating the sync ctr(aes) functionality to modules under arch/arm, move the helper function from a inline .h file to the AES library, which is already depended upon by the drivers that use this fallback. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-ctr-fallback.h | 53 -------------------- arch/arm64/crypto/aes-glue.c | 22 ++++++-- arch/arm64/crypto/aes-neonbs-glue.c | 21 ++++++-- 3 files changed, 33 insertions(+), 63 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/aes-ctr-fallback.h b/arch/arm64/crypto/aes-ctr-fallback.h deleted file mode 100644 index c9285717b6b5..000000000000 --- a/arch/arm64/crypto/aes-ctr-fallback.h +++ /dev/null @@ -1,53 +0,0 @@ -/* - * Fallback for sync aes(ctr) in contexts where kernel mode NEON - * is not allowed - * - * Copyright (C) 2017 Linaro Ltd - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - */ - -#include -#include - -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - -static inline int aes_ctr_encrypt_fallback(struct crypto_aes_ctx *ctx, - struct skcipher_request *req) -{ - struct skcipher_walk walk; - u8 buf[AES_BLOCK_SIZE]; - int err; - - err = skcipher_walk_virt(&walk, req, true); - - while (walk.nbytes > 0) { - u8 *dst = walk.dst.virt.addr; - u8 *src = walk.src.virt.addr; - int nbytes = walk.nbytes; - int tail = 0; - - if (nbytes < walk.total) { - nbytes = round_down(nbytes, AES_BLOCK_SIZE); - tail = walk.nbytes % AES_BLOCK_SIZE; - } - - do { - int bsize = min(nbytes, AES_BLOCK_SIZE); - - __aes_arm64_encrypt(ctx->key_enc, buf, walk.iv, - 6 + ctx->key_length / 4); - crypto_xor_cpy(dst, src, buf, bsize); - crypto_inc(walk.iv, AES_BLOCK_SIZE); - - dst += AES_BLOCK_SIZE; - src += AES_BLOCK_SIZE; - nbytes -= AES_BLOCK_SIZE; - } while (nbytes > 0); - - err = skcipher_walk_done(&walk, tail); - } - return err; -} diff --git a/arch/arm64/crypto/aes-glue.c b/arch/arm64/crypto/aes-glue.c index 3c80345d914f..60303ea625e6 100644 --- a/arch/arm64/crypto/aes-glue.c +++ b/arch/arm64/crypto/aes-glue.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -21,7 +22,6 @@ #include #include "aes-ce-setkey.h" -#include "aes-ctr-fallback.h" #ifdef USE_V8_CRYPTO_EXTENSIONS #define MODE "ce" @@ -404,13 +404,25 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } -static int ctr_encrypt_sync(struct skcipher_request *req) +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) { - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); - struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + const struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * cachelines are evicted when the CPU is interrupted + * to do something else. + */ + local_irq_save(flags); + aes_encrypt(ctx, dst, src); + local_irq_restore(flags); +} +static int ctr_encrypt_sync(struct skcipher_request *req) +{ if (!crypto_simd_usable()) - return aes_ctr_encrypt_fallback(ctx, req); + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); return ctr_encrypt(req); } diff --git a/arch/arm64/crypto/aes-neonbs-glue.c b/arch/arm64/crypto/aes-neonbs-glue.c index cb8d90f795a0..73c17ccb347d 100644 --- a/arch/arm64/crypto/aes-neonbs-glue.c +++ b/arch/arm64/crypto/aes-neonbs-glue.c @@ -11,13 +11,12 @@ #include #include #include +#include #include #include #include #include -#include "aes-ctr-fallback.h" - MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); @@ -283,13 +282,25 @@ static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, return aesbs_setkey(tfm, in_key, key_len); } -static int ctr_encrypt_sync(struct skcipher_request *req) +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) { - struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * cachelines are evicted when the CPU is interrupted + * to do something else. + */ + local_irq_save(flags); + aes_encrypt(&ctx->fallback, dst, src); + local_irq_restore(flags); +} +static int ctr_encrypt_sync(struct skcipher_request *req) +{ if (!crypto_simd_usable()) - return aes_ctr_encrypt_fallback(&ctx->fallback, req); + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); return ctr_encrypt(req); } From patchwork Tue Jul 2 19:41:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168363 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653567ilk; Tue, 2 Jul 2019 12:42:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqw+njFHm62oU3WwYg8JZ0mD2xYf2vNOzA1rAssVWWVfe3wEo22TCvNi2MlJ3BC2b+kxlNwk X-Received: by 2002:a17:902:1003:: with SMTP id b3mr37621093pla.172.1562096557169; Tue, 02 Jul 2019 12:42:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096557; cv=none; d=google.com; s=arc-20160816; b=nFzXAVuM+nBQXrsPADFDWtDeBuhSRF1z9aaGO/XKqZpeiSBNu5L84NtghiKr6hR6/e acHjchBwqI8Q9B8pn6DAJ+ARE0rVl9ENtdX2wIwccy/AlW9uEKU24Lz/WT7YebdwHS6e oINIwBRVCJ7qlnVR9giBTpf73L5BOakckLwDHP9iXghWUn3DYtKxIey/DBf+fz85/LEv Ytfy24lya6NKpjygGEn0rhZ9trdQUkXIGOhHUjxSjzbYzkUF1M81yC2D391NZldKTpZE A19LU7vYKYbbNHS4OILEA+oLWgjXqERSHZbLDkAmAxE/21A1TxI0eQAOKtlbWD7l1gH9 BYtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=RA/9IzM/wteRuTKu0pLDxMNR1Wzx064tA8Ih0Blx/i0=; b=XTLUGd/gAtlkIrTMOGcvq0Q2fOGzCmWefzRa1zST59KL8ICCfS2/XlvRmONSrU2oa9 EaeJw/uQT0HutIZXjwAkcRI46/PUjt2A1GCLB44ksHjUANVWzj3v0DWtkDKUF5OReYha 4E1vTAtYFxkuGE0ihyQhZyc8ByKPO2jaVXfQpj5io1eQji58LCicz8UvQ1wWcRgLgWat NNlWFMYylWsJBRLeSBLg/yhhWgEDyZ0EJKkL+sy64JFa66LX5qyyIbOaKTO7rUGK5L8m cWHPujVic4LzFedmlqL6PlbWgpijW8MnOpAUHW6uVDwzhuL24CzvjC2QHukwD6m1t2LK cZBQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="cFSPCIG/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.36; Tue, 02 Jul 2019 12:42:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="cFSPCIG/"; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727080AbfGBTmg (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:36 -0400 Received: from mail-lj1-f196.google.com ([209.85.208.196]:43128 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727077AbfGBTmg (ORCPT ); Tue, 2 Jul 2019 15:42:36 -0400 Received: by mail-lj1-f196.google.com with SMTP id 16so18134504ljv.10 for ; Tue, 02 Jul 2019 12:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RA/9IzM/wteRuTKu0pLDxMNR1Wzx064tA8Ih0Blx/i0=; b=cFSPCIG/bffIkSOQesv7FkfPabeFVhhkSZpvMaJ1VeLDzo7FC5pD7m6sQB3EJMQBSO r2GYUbQ0o51V6I4iDxEk3PTvelCfsdtr6JBBVTHt1KVlGIuGgbbPQHoNEfZX+oGzUqLQ 4ubA3HfxM6y+lDhP8dcgOYQUnGBhZcV83YuXXNKq0GlXSJCWvShzeGL1A2e3/xfgB9Tz PK9i1cDfy3MuSDa9fs9IQogmvpLGe/VMI/XwcZbu8eD4cVgocfUIWuH1p1kq4PcwlxvC ztqvhoN9z/yYEgAoRs7DBZdkKnjLvjkDmRIoSm6wPd8/g1gf4IFesjFS4pbcJDQllhrO T9Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RA/9IzM/wteRuTKu0pLDxMNR1Wzx064tA8Ih0Blx/i0=; b=X9939y6R63Z4MwTIob/wtmKD9gORjKAskZD8xQFrzzv4LXoz3gagc+N0m2/meH6lvl KIXXcefIYaEyQ8DLQWwLZvIv9hEyRPybMPM/eu9+NUCmlvXVEpWYisoKkFusWL7RJWpH HmN82UlFJhC5Qkv7BEI6zlQVjWQbapv7ls4TUMsxuIYtkz7JcNGurrYjdQJkc6SHA/yE r3FzkT5gQsUg3Fj/MQlHUteTAFaoGSO8OZ2LXB2qmmLk/V05EtSkLa1G1+vJtVUU4M7w 0WQjmavNxQjMSwlauVTToEDByAusEHT8AT3p9cxpJuNRxhhD8rsV2UXesqOFgxDe7FaP 3S1A== X-Gm-Message-State: APjAAAWizuGs8LTjVC816Vl9AfV23F8B6oR39R60TptaC98Snbps5j29 h3A0jiQ73T3tLLZ7KN+gOSVxCYPujGI6qJeX X-Received: by 2002:a2e:3e01:: with SMTP id l1mr18413773lja.208.1562096554329; Tue, 02 Jul 2019 12:42:34 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.33 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:33 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 18/32] crypto: arm64/aes-ce-cipher - use AES library as fallback Date: Tue, 2 Jul 2019 21:41:36 +0200 Message-Id: <20190702194150.10405-19-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Instead of calling into the table based scalar AES code in situations where the SIMD unit may not be used, use the generic AES code, which is more appropriate since it is less likely to be susceptible to timing attacks. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Kconfig | 2 +- arch/arm64/crypto/aes-ce-glue.c | 7 ++----- arch/arm64/crypto/aes-cipher-glue.c | 3 --- 3 files changed, 3 insertions(+), 9 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/Kconfig b/arch/arm64/crypto/Kconfig index 66dea518221c..4922c4451e7c 100644 --- a/arch/arm64/crypto/Kconfig +++ b/arch/arm64/crypto/Kconfig @@ -73,7 +73,7 @@ config CRYPTO_AES_ARM64_CE tristate "AES core cipher using ARMv8 Crypto Extensions" depends on ARM64 && KERNEL_MODE_NEON select CRYPTO_ALGAPI - select CRYPTO_AES_ARM64 + select CRYPTO_LIB_AES config CRYPTO_AES_ARM64_CE_CCM tristate "AES in CCM mode using ARMv8 Crypto Extensions" diff --git a/arch/arm64/crypto/aes-ce-glue.c b/arch/arm64/crypto/aes-ce-glue.c index 3213843fcb46..6890e003b8f1 100644 --- a/arch/arm64/crypto/aes-ce-glue.c +++ b/arch/arm64/crypto/aes-ce-glue.c @@ -23,9 +23,6 @@ MODULE_DESCRIPTION("Synchronous AES cipher using ARMv8 Crypto Extensions"); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); -asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); - struct aes_block { u8 b[AES_BLOCK_SIZE]; }; @@ -54,7 +51,7 @@ static void aes_cipher_encrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[]) struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); if (!crypto_simd_usable()) { - __aes_arm64_encrypt(ctx->key_enc, dst, src, num_rounds(ctx)); + aes_encrypt(ctx, dst, src); return; } @@ -68,7 +65,7 @@ static void aes_cipher_decrypt(struct crypto_tfm *tfm, u8 dst[], u8 const src[]) struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm); if (!crypto_simd_usable()) { - __aes_arm64_decrypt(ctx->key_dec, dst, src, num_rounds(ctx)); + aes_decrypt(ctx, dst, src); return; } diff --git a/arch/arm64/crypto/aes-cipher-glue.c b/arch/arm64/crypto/aes-cipher-glue.c index 0e90b06ebcec..bf32cc6489e1 100644 --- a/arch/arm64/crypto/aes-cipher-glue.c +++ b/arch/arm64/crypto/aes-cipher-glue.c @@ -13,10 +13,7 @@ #include asmlinkage void __aes_arm64_encrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -EXPORT_SYMBOL(__aes_arm64_encrypt); - asmlinkage void __aes_arm64_decrypt(u32 *rk, u8 *out, const u8 *in, int rounds); -EXPORT_SYMBOL(__aes_arm64_decrypt); static void aes_arm64_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) { From patchwork Tue Jul 2 19:41:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168365 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653595ilk; Tue, 2 Jul 2019 12:42:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqz//hI0bOH4A1wCrfsGfSX3fu7s608fIQ+pSpctW2rtNJyYCYaY3iRl8wkaBabYaxIppWGz X-Received: by 2002:a17:902:4b:: with SMTP id 69mr36738381pla.89.1562096559107; Tue, 02 Jul 2019 12:42:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096559; cv=none; d=google.com; s=arc-20160816; b=iX+TGxYdjgdJOg5069KG/cqrEIEPAIcsQ/iTuzySpN/BxtZvVu2SNp4FmbAo+6mPom 0BD6b//oB71Ti8dUuTYcx4ybk5bzbYGaBxb6FtkQ4KSAZ1zM+/FAZDFhCdc+tdmEBUSh 76EywsKR2Qn44uOa/kKTirKuFNTtTLG58kTqzQOjHvIfGL/94RNNA51Rg4SsN58VB3wo cmiJfZoeQZnkPdIwxheJN5E+ObVOxYmYIIwN97L32umsP1QUKnZ/5R2J8SkmrfhUDZkz QcItIQH9pUt4RclwgXWudYEqLvUVgdYk721Z6sxsHwljBUQAH2FmYnz4FJxpy/yY8Reh zmjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=s+NhQ00hAhL+HJIqCr189mBOpkGb5PuAQij3uwTGDWg=; b=bE6gFNJMFkY69nTUXbOQ5UKvuKQlt1iDPDD/wGLsUt7BoQG/LoDnKHRkytGqe2DikR 7kqHK86lWVQlotUlQTENpt4lJ+xMNQqubxgD6wOODzKa/cXgEZGswf7Xaq6D/niTEnFB wcEHj2WajYEJ5gQBtqsfHHUD5WrbdItD3+TvX/bkZFxBGQfJ84Zv7LDK0y9wMKX3Owkr LwSm5YoNoSt4RWCGd/ur+0zbrXqVPC9S8kzFAXa5UFH6OVGBvPMkqdcS8zQf5tkt1cvS rql4TisGSKXmuqDWVHAozXzss53G6t25Nref/IxBop9OB/FIBNyIrkJckwZcyjSUrn2n M5Bw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vQm1OGzh; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.38; Tue, 02 Jul 2019 12:42:39 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vQm1OGzh; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727084AbfGBTmi (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:38 -0400 Received: from mail-lf1-f47.google.com ([209.85.167.47]:38246 "EHLO mail-lf1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727077AbfGBTmh (ORCPT ); Tue, 2 Jul 2019 15:42:37 -0400 Received: by mail-lf1-f47.google.com with SMTP id b11so12268243lfa.5 for ; Tue, 02 Jul 2019 12:42:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=s+NhQ00hAhL+HJIqCr189mBOpkGb5PuAQij3uwTGDWg=; b=vQm1OGzhhvdX92Ew1zFqtbcLd9L5I8rOiqRKh5575j6jviesSFNzqvLTdWHuISXm/p Yl5ccJ2UCXeKUrBO7xeSAP1Jf5//rR2pKLchEwVD975XpdkRHpzX8xgA5p8pOnrjqzvP lrMMXcd+y94W41YoalMOKFrvHvXy9ybe3bVMCSwMtRI8aXf7X+ZKyLbLqmuwPQM7KPCD PyLWB66suIQNrZHbCASVUG1/+fMnu6xk+5mGG2aJ+TekYEkYRumVjOihPfw67uHPNLn5 m78j1ydBk5NWVvypJ9azy1mKRxn7BeuLNySUSwXaDTIMZSSeVhutLETxh5pNPNQYgT6q WiPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=s+NhQ00hAhL+HJIqCr189mBOpkGb5PuAQij3uwTGDWg=; b=pNiP/ZmCnhL7MQ8kO0AmCy20JCKTAREFCcY4eKCpMPYgzRfckW63gKEUu0GAFK4NLV i11G6JcA4sFruyRGe718eBAfjG+n9eOyk9CSiBIg69dUNcm6o28IMDkT3PcNne1LIqkh Q7Tivk3xnm/TB15fDT0+whri9eXr+rt48e0uA9k4fYEWkYg4vNemb1AGcDhs5c/lSlHP hYQp/Vu3GKRQriCkaqggGhzQjAy95BCOGzQtb3R1/muugCfQLno6SPN9oS2oxxc875o7 jyU9Hpvl1q6houKKYaAB1k8DT4CNLwtgO0UYtsmchNQt3dJjHh3ezmpkJBGgYXIm0FxO Zymw== X-Gm-Message-State: APjAAAXEksUpur6PSfOGiUnRA0L2eD/5rigBOr+tqjZR+cgHOaKw+e/r JbYOm8wzbxnMyLSqe/w8HNx5/x5iKDX1d8yY X-Received: by 2002:a19:80c4:: with SMTP id b187mr1104007lfd.122.1562096555686; Tue, 02 Jul 2019 12:42:35 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.34 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:34 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 19/32] crypto: aes/arm - use native endiannes for key schedule Date: Tue, 2 Jul 2019 21:41:37 +0200 Message-Id: <20190702194150.10405-20-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Align ARM's hw instruction based AES implementation with other versions that keep the key schedule in native endianness. This will allow us to merge the various implementations going forward. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-core.S | 20 ++++++++++---------- arch/arm/crypto/aes-ce-glue.c | 9 +++------ 2 files changed, 13 insertions(+), 16 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/aes-ce-core.S b/arch/arm/crypto/aes-ce-core.S index bc53bcaa772e..3692b8735ef7 100644 --- a/arch/arm/crypto/aes-ce-core.S +++ b/arch/arm/crypto/aes-ce-core.S @@ -91,19 +91,19 @@ .macro do_block, dround, fround cmp r3, #12 @ which key size? - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q8, q9 - vld1.8 {q12-q13}, [ip]! + vld1.32 {q12-q13}, [ip]! \dround q10, q11 - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q12, q13 - vld1.8 {q12-q13}, [ip]! + vld1.32 {q12-q13}, [ip]! \dround q10, q11 blo 0f @ AES-128: 10 rounds - vld1.8 {q10-q11}, [ip]! + vld1.32 {q10-q11}, [ip]! \dround q12, q13 beq 1f @ AES-192: 12 rounds - vld1.8 {q12-q13}, [ip] + vld1.32 {q12-q13}, [ip] \dround q10, q11 0: \fround q12, q13, q14 bx lr @@ -152,8 +152,8 @@ ENDPROC(aes_decrypt_3x) .macro prepare_key, rk, rounds add ip, \rk, \rounds, lsl #4 - vld1.8 {q8-q9}, [\rk] @ load first 2 round keys - vld1.8 {q14}, [ip] @ load last round key + vld1.32 {q8-q9}, [\rk] @ load first 2 round keys + vld1.32 {q14}, [ip] @ load last round key .endm /* @@ -508,8 +508,8 @@ ENDPROC(ce_aes_sub) * operation on round key *src */ ENTRY(ce_aes_invert) - vld1.8 {q0}, [r1] + vld1.32 {q0}, [r1] aesimc.8 q0, q0 - vst1.8 {q0}, [r0] + vst1.32 {q0}, [r0] bx lr ENDPROC(ce_aes_invert) diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index 04ba66903674..e6da3e30018b 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -80,21 +81,17 @@ static int ce_aes_expandkey(struct crypto_aes_ctx *ctx, const u8 *in_key, key_len != AES_KEYSIZE_256) return -EINVAL; - memcpy(ctx->key_enc, in_key, key_len); ctx->key_length = key_len; + for (i = 0; i < kwords; i++) + ctx->key_enc[i] = get_unaligned_le32(in_key + i * sizeof(u32)); kernel_neon_begin(); for (i = 0; i < sizeof(rcon); i++) { u32 *rki = ctx->key_enc + (i * kwords); u32 *rko = rki + kwords; -#ifndef CONFIG_CPU_BIG_ENDIAN rko[0] = ror32(ce_aes_sub(rki[kwords - 1]), 8); rko[0] = rko[0] ^ rki[0] ^ rcon[i]; -#else - rko[0] = rol32(ce_aes_sub(rki[kwords - 1]), 8); - rko[0] = rko[0] ^ rki[0] ^ (rcon[i] << 24); -#endif rko[1] = rko[0] ^ rki[1]; rko[2] = rko[1] ^ rki[2]; rko[3] = rko[2] ^ rki[3]; From patchwork Tue Jul 2 19:41:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168366 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653613ilk; Tue, 2 Jul 2019 12:42:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqzhmVbQCrst3lcnkQjR4QKjw94yUZYzsiswUri71phtoOS1HCzSn25AXRO3+PR4pKG1ndOG X-Received: by 2002:a17:90a:de02:: with SMTP id m2mr7541135pjv.18.1562096560266; Tue, 02 Jul 2019 12:42:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096560; cv=none; d=google.com; s=arc-20160816; b=q0uZ3gXdc8sCqNE6KkDowrFTcWQ6XtmtfYoRTDCFmC2vQrlpNJtNvZZN33U10wBaSy +S36a06jZig/ns0mBg5zjLK/+dNVdf0B4G8tVUWeoKvG83Q2OShA34zsbVafXePXcvEF ifkCZiz32K2ct+C5iQ3q1Gq57niK6GZK+DVdx5xNasaUepU9Tleg1e8wf72mhp9MRruH fO0qkhBbEHPEb6JvtyfZ/zCNHMJf9m9y5KMk2HQ9eioKhNofmagUtEzUdv3T5QYVa3Ui PMPAMbFXCRIQo6zQbJmi8RvtWx+gWhEpCVKcFC32GUoUP95Eajb34MnAqP7+hK5279fD M+fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=9R634f5tUJpvqy1jJF6+TjW/ZQOJkrwjqwK24wF4gE4=; b=HwBgcWckanczUhWvQdqtS33nUG/+ts7FprGt5oNId9M/ysXKRrf3Uu+gYpVANOQkgk 3hhCFH+ekSJlLxC3joKTSxqU9C9VGqyi090lLHa+RQjdOYb8lGBzlFOZxjtt4lkhOgoH 3cKZ5EHqLF1N52ZSmNimAOsPuROxf9cwqUT+itgTXSnd7Jgqzw9b1caPOrHiFiPdHQRb q0zCFVt1m/ai4jkB7UKPjEesJITdmoVCG0/TnM47sAQVMjdVb7yQv1DK0II49ZZM69lG 0Ahaujjk0l8kPbvx3KP50aiyfKOkOE/216MkUH1cG5s1PrupuZbz+6VSAVrWyVfq/MtW nWwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XNauRQ8V; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.40; Tue, 02 Jul 2019 12:42:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XNauRQ8V; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727087AbfGBTmj (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:39 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:37922 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmj (ORCPT ); Tue, 2 Jul 2019 15:42:39 -0400 Received: by mail-lj1-f194.google.com with SMTP id r9so18181160ljg.5 for ; Tue, 02 Jul 2019 12:42:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9R634f5tUJpvqy1jJF6+TjW/ZQOJkrwjqwK24wF4gE4=; b=XNauRQ8VO6r2djNte89iUcIO+FGn0a93v3jW5RP0j4XZji/tyXbEkoalx5RSJKGzCR 1D4GgukKAfMxPJsaEnlvIhCS1MdbiNwtwQPWCQ/llgvgB6ltZmxNftph0BvIeV3T3QkD M3/YaEYkRTg3vvnjqYTY/xwPzZxQjftwfca7T8F8FNy5trVQpWOPL1aJah34GVzVu6Qv NVDTvj5L5iMarSbkd8PwZKA4mN2+8K20pv+RerxLGO6Xn/0/sgGY0s7s3FCOQIfoI/eF /o2Y8jUAlG2M6+jG1CDi5dc6jL8NOJEmqqwzlOkaKfiJYRckpUUxXRSIa9YE1Tp1FB4s LedA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9R634f5tUJpvqy1jJF6+TjW/ZQOJkrwjqwK24wF4gE4=; b=JC07bGJll8+xcLuOdHDhFaMuOY4NJoT9R/jOFVek1qfpqyWNC4X97nVgYin5HZG6mf hb6ZZTLCFi8+V9caEkmxAGHbt5AqfANVqE5G+xlEs1nmSgcwVEOGbKrMyDTI/r4cZWd7 p9kUBrCtRHqFa9MqfjDGw86eE08+dEUxjY/9WFJA+C3hMHH+EWx0/c2d56Aucbq2jIHy Zj6rlynqAHweCfXSdmSAiXIqbxhnltT1pZ4f5qUV7y+U5wWtLpdSY3djAXnxQp0tU3Rh JWqFDnLuf+kSahYW/8KxtRMJ/AxlYIJzxwsUX93MpuRgAS31gR99DIY4yUyKCcLlvzgK 8xOA== X-Gm-Message-State: APjAAAV2pfpzJUpLEZ6lag1Sje0TINyyEC7VkqvyLuvzvBVEcA186gGa SHvoacaZdymnyp52pjOmDgJ532TYjcN7Jfss X-Received: by 2002:a2e:a0cf:: with SMTP id f15mr18583670ljm.180.1562096557122; Tue, 02 Jul 2019 12:42:37 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.35 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:36 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 20/32] crypto: arm/aes-ce - provide a synchronous version of ctr(aes) Date: Tue, 2 Jul 2019 21:41:38 +0200 Message-Id: <20190702194150.10405-21-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org AES in CTR mode is used by modes such as GCM and CCM, which are often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of ctr(aes) based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. We have a helper for this now in the AES library, so wire that up. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-ce-glue.c | 43 ++++++++++++++++++++ 1 file changed, 43 insertions(+) -- 2.17.1 diff --git a/arch/arm/crypto/aes-ce-glue.c b/arch/arm/crypto/aes-ce-glue.c index e6da3e30018b..1d93da29d03a 100644 --- a/arch/arm/crypto/aes-ce-glue.c +++ b/arch/arm/crypto/aes-ce-glue.c @@ -10,8 +10,10 @@ #include #include +#include #include #include +#include #include #include #include @@ -289,6 +291,29 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) +{ + struct crypto_aes_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * cachelines are evicted when the CPU is interrupted + * to do something else. + */ + local_irq_save(flags); + aes_encrypt(ctx, dst, src); + local_irq_restore(flags); +} + +static int ctr_encrypt_sync(struct skcipher_request *req) +{ + if (!crypto_simd_usable()) + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); + + return ctr_encrypt(req); +} + static int xts_encrypt(struct skcipher_request *req) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -378,6 +403,21 @@ static struct skcipher_alg aes_algs[] = { { .setkey = ce_aes_setkey, .encrypt = ctr_encrypt, .decrypt = ctr_encrypt, +}, { + .base.cra_name = "ctr(aes)", + .base.cra_driver_name = "ctr-aes-ce-sync", + .base.cra_priority = 300 - 1, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct crypto_aes_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .chunksize = AES_BLOCK_SIZE, + .setkey = ce_aes_setkey, + .encrypt = ctr_encrypt_sync, + .decrypt = ctr_encrypt_sync, }, { .base.cra_name = "__xts(aes)", .base.cra_driver_name = "__xts-aes-ce", @@ -421,6 +461,9 @@ static int __init aes_init(void) return err; for (i = 0; i < ARRAY_SIZE(aes_algs); i++) { + if (!(aes_algs[i].base.cra_flags & CRYPTO_ALG_INTERNAL)) + continue; + algname = aes_algs[i].base.cra_name + 2; drvname = aes_algs[i].base.cra_driver_name + 2; basename = aes_algs[i].base.cra_driver_name; From patchwork Tue Jul 2 19:41:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168367 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653639ilk; Tue, 2 Jul 2019 12:42:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqzSJ2nCi7xlvYzBb3qiPS9J2l3o9m5ADC6VX5dVcRG59tPK4qAf+H+NLzIINM5BqvnUFMwU X-Received: by 2002:a17:902:2865:: with SMTP id e92mr36673637plb.264.1562096561867; Tue, 02 Jul 2019 12:42:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096561; cv=none; d=google.com; s=arc-20160816; b=0iX4hZ9he5KhAeyvimsgdxzUcFyR2agGPdGY8H7rdjArA8247QCwhSp7y3VareuziF 5QuTOEKgUTjOzDThx8LQiasCAmfwxKrYhNZvfoNqRIR5cL2OhjFbrbBmvdvB6mV76+o6 5ry+CwET2cBvYKfBCNAdNz3f5Gq9zbX0RXMiIRYe6Rr2dkJNuFTbwRZ7ecnH/9Xhk7YW mdVfJBozLoGnYSHTHwx1g0pDs07NV30Gi0g/tjeRa2cpv2arLBnJhK/QJzTck68yzXEb w8by3nqp4y0UF6HkgMQXzBhI0L8U8bOfNk4JQ4bQiaea99t++Zn6Hc+Sff3xWWuvuV/4 R5YA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=bK4LustB8GmckyS2nPrFKDpnC0XD07k3hKGC7VgrKjI=; b=FGKjmwPXZvIxIc/hzbI1WQ/cEh+0+IS+ZWQgW2ON7CC21zxX14eBc3EFHEZ8Uegrzc 411LdhE3FpLgI5j/1J9nwP5Z+uuQByCSCNFb+qHWX0fEMmrlLPMlaSBqWLEnlgyOp7iS x/TjbQZXjTEmZhOnlLCjjNj1SGFdmnGuwTyOxYUa4S7sz1DiSksniEylcFyErFYmsK5c AdfPpPFjVjfqcpZhQFwK9ppI8rU6d1biCXbCF4QVHQjc9jVBfPu7sqbYzFoNupeICBmN UzY9p3Yfwo2DbzZyQ46CuAzjMtTG6y4QgkGzwou4/Vb9stU8vFx2KkGv04IN0f0TWxAw 9Ypg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JN9Jgah2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.41; Tue, 02 Jul 2019 12:42:41 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JN9Jgah2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727089AbfGBTml (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:41 -0400 Received: from mail-lf1-f66.google.com ([209.85.167.66]:41957 "EHLO mail-lf1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727077AbfGBTml (ORCPT ); Tue, 2 Jul 2019 15:42:41 -0400 Received: by mail-lf1-f66.google.com with SMTP id 62so2219768lfa.8 for ; Tue, 02 Jul 2019 12:42:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bK4LustB8GmckyS2nPrFKDpnC0XD07k3hKGC7VgrKjI=; b=JN9Jgah2WGcVN504HWo6oO8sZaZHO954dgn+nY0MTg68xJHIiFTw5pIg31ihwH3nD8 Tb5T03mnm7Kn8YEhiFhsMMDyDBZR97UFbmZnULjgrlbst1/4ag0p/7H+/fAKNrEEsKm8 f4kIb8f8YWRYaWdQivY7rIL20c1D1mJWMp39+izc17g1wa4KhAy7TkWRANkk9NQF/8am a1+VyrIoJ41DDSneKIWXBVI5fk2aWyDEaFYuNnGeBHCWHgHyhYEaTyV/pEH/NN966972 aVAlA72NPTCdQakDc9IIXh7YQJ3jvmMVWmf8siUWfWuUXLMr87EbcVk9vg9a4cQ633hB 8Zxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bK4LustB8GmckyS2nPrFKDpnC0XD07k3hKGC7VgrKjI=; b=kN/+3vRIrU0f9q+hN/46ySYqPpj23Wo8ybPS5w+OrcIAKB8+7hID76IkLh7GyPfMvl U1dkiTd40BQ43JFNEog+NtnnsNIo8IslQ8229ip96J0tdVNckcXqKbHmSa46ZiVsH5ge 8x0B8Ao9hiRZ4blIpoNPGbwEvJnwx7jIvWgeN+humVZldkMDUMTg8nfa8d1v2bm4Ryce n/OYOqQe3F12/acfri7Ag9AlFZb4l7dNHNdMDYH8x8mJ/UWaQDOk+qrJ6mSbaE9gFcjN wIy4c2OY33jv/exjkb/rPLSb/7n/ayYRaTw7TlXNkNcgRRwSk+F996EWwHjjy5xcZoh7 qN4w== X-Gm-Message-State: APjAAAVQq3Lip6aK7E3ztT2rH40SU6BzbZfEkx+8xfOdlop4Ryv7ujsR m2qthOJoUbRdUbFfFYblTL5PVaaLkYGAEEK+ X-Received: by 2002:a19:41cc:: with SMTP id o195mr14128017lfa.166.1562096558298; Tue, 02 Jul 2019 12:42:38 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.37 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:37 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 21/32] crypto: arm/aes-neonbs - provide a synchronous version of ctr(aes) Date: Tue, 2 Jul 2019 21:41:39 +0200 Message-Id: <20190702194150.10405-22-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org AES in CTR mode is used by modes such as GCM and CCM, which are often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of ctr(aes) based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. We have a helper for this now in the AES library, so wire that up. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-neonbs-glue.c | 65 ++++++++++++++++++++ 1 file changed, 65 insertions(+) -- 2.17.1 diff --git a/arch/arm/crypto/aes-neonbs-glue.c b/arch/arm/crypto/aes-neonbs-glue.c index f43c9365b6a9..6eecdbb7e9b6 100644 --- a/arch/arm/crypto/aes-neonbs-glue.c +++ b/arch/arm/crypto/aes-neonbs-glue.c @@ -9,8 +9,10 @@ */ #include +#include #include #include +#include #include #include #include @@ -57,6 +59,11 @@ struct aesbs_xts_ctx { struct crypto_cipher *tweak_tfm; }; +struct aesbs_ctr_ctx { + struct aesbs_ctx key; /* must be first member */ + struct crypto_aes_ctx fallback; +}; + static int aesbs_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { @@ -192,6 +199,25 @@ static void cbc_exit(struct crypto_tfm *tfm) crypto_free_cipher(ctx->enc_tfm); } +static int aesbs_ctr_setkey_sync(struct crypto_skcipher *tfm, const u8 *in_key, + unsigned int key_len) +{ + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + int err; + + err = aes_expandkey(&ctx->fallback, in_key, key_len); + if (err) + return err; + + ctx->key.rounds = 6 + key_len / 4; + + kernel_neon_begin(); + aesbs_convert_key(ctx->key.rk, ctx->fallback.key_enc, ctx->key.rounds); + kernel_neon_end(); + + return 0; +} + static int ctr_encrypt(struct skcipher_request *req) { struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req); @@ -234,6 +260,29 @@ static int ctr_encrypt(struct skcipher_request *req) return err; } +static void ctr_encrypt_one(struct crypto_skcipher *tfm, const u8 *src, u8 *dst) +{ + struct aesbs_ctr_ctx *ctx = crypto_skcipher_ctx(tfm); + unsigned long flags; + + /* + * Temporarily disable interrupts to avoid races where + * cachelines are evicted when the CPU is interrupted + * to do something else. + */ + local_irq_save(flags); + aes_encrypt(&ctx->fallback, dst, src); + local_irq_restore(flags); +} + +static int ctr_encrypt_sync(struct skcipher_request *req) +{ + if (!crypto_simd_usable()) + return crypto_ctr_encrypt_walk(req, ctr_encrypt_one); + + return ctr_encrypt(req); +} + static int aesbs_xts_setkey(struct crypto_skcipher *tfm, const u8 *in_key, unsigned int key_len) { @@ -361,6 +410,22 @@ static struct skcipher_alg aes_algs[] = { { .setkey = aesbs_setkey, .encrypt = ctr_encrypt, .decrypt = ctr_encrypt, +}, { + .base.cra_name = "ctr(aes)", + .base.cra_driver_name = "ctr-aes-neonbs-sync", + .base.cra_priority = 250 - 1, + .base.cra_blocksize = 1, + .base.cra_ctxsize = sizeof(struct aesbs_ctr_ctx), + .base.cra_module = THIS_MODULE, + + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .chunksize = AES_BLOCK_SIZE, + .walksize = 8 * AES_BLOCK_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = aesbs_ctr_setkey_sync, + .encrypt = ctr_encrypt_sync, + .decrypt = ctr_encrypt_sync, }, { .base.cra_name = "__xts(aes)", .base.cra_driver_name = "__xts-aes-neonbs", From patchwork Tue Jul 2 19:41:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168368 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653666ilk; Tue, 2 Jul 2019 12:42:43 -0700 (PDT) X-Google-Smtp-Source: APXvYqxc69pbpLeQZxHjS2Oou85uLddMlChouRyThO0pCG5F2mvr365fmd4qJ1QminyI3JW8IcmS X-Received: by 2002:a65:41c6:: with SMTP id b6mr2244964pgq.269.1562096563060; Tue, 02 Jul 2019 12:42:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096563; cv=none; d=google.com; s=arc-20160816; b=G7JzxEdI99cYZH5Ycm9OJhpc0LE9GN/kRrsuJLuMG+KGUhhW5CHd/cqcoK27Y9m42o wEBuHRlgyKvrJ6E28n/53GgJPwSN3OgAaXV21spbS8okLt0GI/yItsZAe1uw5ydhh3Fb 194jlGROgaxlVKDC4M6DZuZef4bpbVWOfp0LOTze4bFAu4lmw/q4GTtjbns+1aAm/MMZ JTUOoYWyHwlrTVW9dgkBGjGYRXUTtgeXp/O1YesMm8lYb18HzUIBMpa1xGXdnM/H2+z3 EZJ6Lb/TI/4TQ8PNxsgCTixOAGRkED/lPSg1U+LbIYg1S1ooCs0+zxdcjqh5Uxhc+ppX vlOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=VA1NX4OTJyuJ+sHjTv3EgQQCR56i+Qi8C4n3+5icP3I=; b=MjrBoVe/EqlpMtn9VNXnRqeFMc5qp7ryf63XOrWn9UPmXzTT5bURIcNQkxVHwuxHIm 9G4Ycu0fkwIRwc/0BjL3MYiCb/bRviUqG/u6DJXmr4vF9yiCXI6WSx/Dpa0K8O2rElfT o3Fak8hgPsvc+CzSHJxEIXXs4SGuHSn1F6MGXANWyStErIJd0+K2ZmNOTA5ZKEfX+gt9 3/Jq9z/iPy/J6V/JtX9791H3ix/dPygS8jNUcB2y/ubVh4HzwC4PdEB3ARWTTE15sYqL 0UnbRmsrjOjcRQsMvahVLVS2rkBYJTQTaE1bdqxd6OR5YQvWd0Z743uB9WH17LnDgrlv hQcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gubS8BLH; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.42; Tue, 02 Jul 2019 12:42:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gubS8BLH; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727092AbfGBTmm (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:42 -0400 Received: from mail-lj1-f196.google.com ([209.85.208.196]:40428 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmm (ORCPT ); Tue, 2 Jul 2019 15:42:42 -0400 Received: by mail-lj1-f196.google.com with SMTP id a21so18169330ljh.7 for ; Tue, 02 Jul 2019 12:42:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VA1NX4OTJyuJ+sHjTv3EgQQCR56i+Qi8C4n3+5icP3I=; b=gubS8BLHsB5NbC39fRYRNOlmUqVxLMuCXolIY+O+z7WZMFn4CibcpVlUlw0snTj7hC S1MHCspjJWaQXoACWaw1T4lBLRAJyvLX55lbQcIwcU0Ji1DMdw32MZ5u3L4APxZICP5g bWQ2CklluUVYKdYfJTOWou+0m7mp/aM38Fs29RdRaooiO2geHajp2L7lIVdKXlIbnAg5 sc9XlEZtoLMcsSi3QfBSYo7cUNvGn5PFmB6Tu1eb+TLp1Fl5Y2T/kyIcFpkQYC6Pxkjv MMBA6H8Ebzxxj1uqw7SBZnDjXmYmsYF6eSSnJPMzA/UOl65abwHf8fjQ+Gu2Xjbx/Ac9 X5UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VA1NX4OTJyuJ+sHjTv3EgQQCR56i+Qi8C4n3+5icP3I=; b=YjCHgmK2C1ePflwAxnfBw8aD0wDrUIpvT/wNbb/Kgbsr+SXC77n7yfIs6uYxjie//F rlcux5rzH8JM1ZDo/RB7cKutbiDpOQW/EIwtkWjYIKuvnn3Q6ayHbsk54032smkk7LWr 5Ph+aNzVWalYZ8UGIunWFkO+CirjnO4Y5iuwaL2pjO24lYC06mM8aOxVMxHLAbUk+wcN bLF/1eUo6KbHPVkfLhz+s+jvBBRiYS1TjtUm3W5GL0O0TXU+LGiFXXt6DBmlrL5iZevm MEuWfwT6O6N7vsop7J9rFtH8sgmgUwOkaGrb7d4MWas5EM82bk1aLtbNqFTm1hMexhTW otRA== X-Gm-Message-State: APjAAAXpZZcoKrm6X8sGCObcemXVpuGfLck8MpR/lpRU/0Bsjx+RUjZU 4mROwLKTK16vhan8LCdiLUgWsXYusvA++b1D X-Received: by 2002:a2e:5d46:: with SMTP id r67mr17553560ljb.187.1562096559561; Tue, 02 Jul 2019 12:42:39 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.38 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:38 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 22/32] crypto: arm/ghash - provide a synchronous version Date: Tue, 2 Jul 2019 21:41:40 +0200 Message-Id: <20190702194150.10405-23-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org GHASH is used by the GCM mode, which is often used in contexts where only synchronous ciphers are permitted. So provide a synchronous version of GHASH based on the existing code. This requires a non-SIMD fallback to deal with invocations occurring from a context where SIMD instructions may not be used. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/ghash-ce-glue.c | 78 +++++++++++++------- 1 file changed, 52 insertions(+), 26 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/ghash-ce-glue.c b/arch/arm/crypto/ghash-ce-glue.c index 39d1ccec1aab..ebb237ca874b 100644 --- a/arch/arm/crypto/ghash-ce-glue.c +++ b/arch/arm/crypto/ghash-ce-glue.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -33,6 +34,8 @@ struct ghash_key { u64 h2[2]; u64 h3[2]; u64 h4[2]; + + be128 k; }; struct ghash_desc_ctx { @@ -65,6 +68,36 @@ static int ghash_init(struct shash_desc *desc) return 0; } +static void ghash_do_update(int blocks, u64 dg[], const char *src, + struct ghash_key *key, const char *head) +{ + if (likely(crypto_simd_usable())) { + kernel_neon_begin(); + pmull_ghash_update(blocks, dg, src, key, head); + kernel_neon_end(); + } else { + be128 dst = { cpu_to_be64(dg[1]), cpu_to_be64(dg[0]) }; + + do { + const u8 *in = src; + + if (head) { + in = head; + blocks++; + head = NULL; + } else { + src += GHASH_BLOCK_SIZE; + } + + crypto_xor((u8 *)&dst, in, GHASH_BLOCK_SIZE); + gf128mul_lle(&dst, &key->k); + } while (--blocks); + + dg[0] = be64_to_cpu(dst.b); + dg[1] = be64_to_cpu(dst.a); + } +} + static int ghash_update(struct shash_desc *desc, const u8 *src, unsigned int len) { @@ -88,10 +121,8 @@ static int ghash_update(struct shash_desc *desc, const u8 *src, blocks = len / GHASH_BLOCK_SIZE; len %= GHASH_BLOCK_SIZE; - kernel_neon_begin(); - pmull_ghash_update(blocks, ctx->digest, src, key, - partial ? ctx->buf : NULL); - kernel_neon_end(); + ghash_do_update(blocks, ctx->digest, src, key, + partial ? ctx->buf : NULL); src += blocks * GHASH_BLOCK_SIZE; partial = 0; } @@ -109,9 +140,7 @@ static int ghash_final(struct shash_desc *desc, u8 *dst) struct ghash_key *key = crypto_shash_ctx(desc->tfm); memset(ctx->buf + partial, 0, GHASH_BLOCK_SIZE - partial); - kernel_neon_begin(); - pmull_ghash_update(1, ctx->digest, ctx->buf, key, NULL); - kernel_neon_end(); + ghash_do_update(1, ctx->digest, ctx->buf, key, NULL); } put_unaligned_be64(ctx->digest[1], dst); put_unaligned_be64(ctx->digest[0], dst + 8); @@ -135,24 +164,25 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *inkey, unsigned int keylen) { struct ghash_key *key = crypto_shash_ctx(tfm); - be128 h, k; + be128 h; if (keylen != GHASH_BLOCK_SIZE) { crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return -EINVAL; } - memcpy(&k, inkey, GHASH_BLOCK_SIZE); - ghash_reflect(key->h, &k); + /* needed for the fallback */ + memcpy(&key->k, inkey, GHASH_BLOCK_SIZE); + ghash_reflect(key->h, &key->k); - h = k; - gf128mul_lle(&h, &k); + h = key->k; + gf128mul_lle(&h, &key->k); ghash_reflect(key->h2, &h); - gf128mul_lle(&h, &k); + gf128mul_lle(&h, &key->k); ghash_reflect(key->h3, &h); - gf128mul_lle(&h, &k); + gf128mul_lle(&h, &key->k); ghash_reflect(key->h4, &h); return 0; @@ -165,15 +195,13 @@ static struct shash_alg ghash_alg = { .final = ghash_final, .setkey = ghash_setkey, .descsize = sizeof(struct ghash_desc_ctx), - .base = { - .cra_name = "__ghash", - .cra_driver_name = "__driver-ghash-ce", - .cra_priority = 0, - .cra_flags = CRYPTO_ALG_INTERNAL, - .cra_blocksize = GHASH_BLOCK_SIZE, - .cra_ctxsize = sizeof(struct ghash_key), - .cra_module = THIS_MODULE, - }, + + .base.cra_name = "ghash", + .base.cra_driver_name = "ghash-ce-sync", + .base.cra_priority = 300 - 1, + .base.cra_blocksize = GHASH_BLOCK_SIZE, + .base.cra_ctxsize = sizeof(struct ghash_key), + .base.cra_module = THIS_MODULE, }; static int ghash_async_init(struct ahash_request *req) @@ -288,9 +316,7 @@ static int ghash_async_init_tfm(struct crypto_tfm *tfm) struct cryptd_ahash *cryptd_tfm; struct ghash_async_ctx *ctx = crypto_tfm_ctx(tfm); - cryptd_tfm = cryptd_alloc_ahash("__driver-ghash-ce", - CRYPTO_ALG_INTERNAL, - CRYPTO_ALG_INTERNAL); + cryptd_tfm = cryptd_alloc_ahash("ghash-ce-sync", 0, 0); if (IS_ERR(cryptd_tfm)) return PTR_ERR(cryptd_tfm); ctx->cryptd_tfm = cryptd_tfm; From patchwork Tue Jul 2 19:41:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168369 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653694ilk; Tue, 2 Jul 2019 12:42:44 -0700 (PDT) X-Google-Smtp-Source: APXvYqwuVuC5Ce+FG9RPHRrKrlXPhf6QG5bPt1yo2pwLXnFpOgK7ZYuLH8bUpWHMAIIyO7tl54Eg X-Received: by 2002:a17:902:788e:: with SMTP id q14mr37436315pll.234.1562096564726; Tue, 02 Jul 2019 12:42:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096564; cv=none; d=google.com; s=arc-20160816; b=qJqxKqMf8aQD7yPbkmVtFuWWlL5YhtRAey//6P4hBzwQaGKC8gji3LZYX7dE5xwCEx 7L6iZ6vwyX8ZrFMtMC23VtkAByMXQUYxdmVLBbq2CGS74jxmybfQh7Mc5onsWNEp+ycE h4zwBK06D1c+fOrWxrcRuzgviBn/WnJabte9sJpfECExNzjKKTFDHildIWVvAYU9rNWY KXi0adGeFf7xgtYwhyPVyNtNCOdgl3EJocGCbNfOxGKfo1pztp71g8lS+wIwtTd743rr 76+TKfIxZL4NiROHiBdQrYGNgtE81HLzXFRo61hCOicteM9/ZFrQMTH5ox1wfFR71Ze+ c80w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=Jt4bAVgVJA8IbVq4C0AE6fZbzc8C6v8wBDp3u5p2nOc=; b=DHdkTZzRKjTaKHlaGFwybDguGntkhx13oEXCpyg1ytyX+y/Ekg7+CqPYQuA6nLA9el jc13Eh4zitSH4t2pZIH9sZm07yID0dMZ5SK2KvOdVDqViF2LRihTtfRLY1s0eQFWeNjT TkidU25HM8qbgE/sKX9+d9SW64YwDaOcuwzKGt6ywJk7WEFwpsNaGmo9gJaPfq5zQaHY jdShQahIoMaigs3aGI4aiaUdh+FJAFBx3AXzzbYYUkbX9pH8/rw/wZaXfPTS0qjn3MQn IJ1CNB99z+oVWCdAoxInsp9j+7V83/hXGDqSlh+rO93MeC6eylWnbiE3b0tEcZhQFz0L d4Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=et725Ef6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.44; Tue, 02 Jul 2019 12:42:44 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=et725Ef6; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727093AbfGBTmo (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:44 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:39773 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727077AbfGBTmn (ORCPT ); Tue, 2 Jul 2019 15:42:43 -0400 Received: by mail-lj1-f193.google.com with SMTP id v18so18159055ljh.6 for ; Tue, 02 Jul 2019 12:42:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Jt4bAVgVJA8IbVq4C0AE6fZbzc8C6v8wBDp3u5p2nOc=; b=et725Ef6Yc9E+ED1+jfB44d22UPH3opVE73txPGE6aSgUDFqH/JjmktVqte/6nEUUL sx+DW1Y3VkjlHo1+F5oT7xPlU/6sG/OPu5ZO+Axly2eU8mpsvseFN9nRAr605o/7LtaS 28t3DmmwGCYu3Eh+pjDPgKylMUOV4SAT0edLPKtmEQ6fcp8wiwPCgfvB44k6a/YV+Yjo tyCxqRr76G98YIKJMo1peVytuK9uB/oXis/+FOu+gnWZg9qVpaBIfWtORekcaHfsmsmj rGECZ4NRe4ePewg3r0Yhky45NiJYNxKIL6RJhfKVgzfV87ZuC8xxYL2++MNHMkEqXEB/ zfVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Jt4bAVgVJA8IbVq4C0AE6fZbzc8C6v8wBDp3u5p2nOc=; b=grCUrKWkYmTc7FQ5NuZqYa08DKTVUQjrjQl3zGNSBCyBIgGcmw+fwOFTdosQnuf5xD WORVfl46X/tVM1ovXSYz4C+PRRc4B1y3sfZKv+yK4JdQCJJ+eUqPiH90vEC1XbHaSgqS ORIeyi37EHaijKMh6xrcSF66o36cBdH3lyv/3YcZv66yXqX7g6bKd3hKKvtc+L/wxz7Y Wx5DvuKX8C8Vdoj7qBv5KL4nN5CPTRFTutNgBwayRSiPe7OoAevtnWjeDtwUyFMfin8X XIjXy9fH3K0Q3n3T+pRRl6qDFZQR9Os1zWxs7ayYIzKmIS0KVMf8SEEl693b9QRa0bMR yCJw== X-Gm-Message-State: APjAAAVHYg6D5DjQkOOmLn5hNHPw5omG+3KoF1gAF4R1a+8UxMd8mb6K CkRyyzGBbrlhG/9mrilC1MY5bzDI5Iy2bgL1 X-Received: by 2002:a2e:8082:: with SMTP id i2mr18637146ljg.121.1562096560883; Tue, 02 Jul 2019 12:42:40 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.39 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:40 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 23/32] bluetooth: switch to AES library Date: Tue, 2 Jul 2019 21:41:41 +0200 Message-Id: <20190702194150.10405-24-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The bluetooth code uses a bare AES cipher for the encryption operations. Given that it carries out a set_key() operation right before every encryption operation, this is clearly not a hot path, and so the use of the cipher interface (which provides the best implementation available on the system) is not really required. In fact, when using a cipher like AES-NI or AES-CE, both the set_key() and the encrypt() operations involve en/disabling preemption as well as stacking and unstacking the SIMD context, and this is most certainly not worth it for encrypting 16 bytes of data. So let's switch to the new lightweight library interface instead. Signed-off-by: Ard Biesheuvel --- net/bluetooth/Kconfig | 3 +- net/bluetooth/smp.c | 103 ++++++-------------- 2 files changed, 33 insertions(+), 73 deletions(-) -- 2.17.1 diff --git a/net/bluetooth/Kconfig b/net/bluetooth/Kconfig index db82a40875e8..a9d83ec4ee33 100644 --- a/net/bluetooth/Kconfig +++ b/net/bluetooth/Kconfig @@ -9,7 +9,8 @@ menuconfig BT select CRC16 select CRYPTO select CRYPTO_BLKCIPHER - select CRYPTO_AES + select CRYPTO_LIB_AES + imply CRYPTO_AES select CRYPTO_CMAC select CRYPTO_ECB select CRYPTO_SHA256 diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c index e68c715f8d37..b5045b57ead3 100644 --- a/net/bluetooth/smp.c +++ b/net/bluetooth/smp.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -88,7 +89,6 @@ struct smp_dev { u8 local_rand[16]; bool debug_key; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; }; @@ -127,7 +127,6 @@ struct smp_chan { u8 dhkey[32]; u8 mackey[16]; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; }; @@ -377,22 +376,18 @@ static int smp_h7(struct crypto_shash *tfm_cmac, const u8 w[16], * s1 and ah. */ -static int smp_e(struct crypto_cipher *tfm, const u8 *k, u8 *r) +static int smp_e(const u8 *k, u8 *r) { + struct crypto_aes_ctx ctx; uint8_t tmp[16], data[16]; int err; SMP_DBG("k %16phN r %16phN", k, r); - if (!tfm) { - BT_ERR("tfm %p", tfm); - return -EINVAL; - } - /* The most significant octet of key corresponds to k[0] */ swap_buf(k, tmp, 16); - err = crypto_cipher_setkey(tfm, tmp, 16); + err = aes_expandkey(&ctx, tmp, 16); if (err) { BT_ERR("cipher setkey failed: %d", err); return err; @@ -401,17 +396,18 @@ static int smp_e(struct crypto_cipher *tfm, const u8 *k, u8 *r) /* Most significant octet of plaintextData corresponds to data[0] */ swap_buf(r, data, 16); - crypto_cipher_encrypt_one(tfm, data, data); + aes_encrypt(&ctx, data, data); /* Most significant octet of encryptedData corresponds to data[0] */ swap_buf(data, r, 16); SMP_DBG("r %16phN", r); + memzero_explicit(&ctx, sizeof (ctx)); return err; } -static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], +static int smp_c1(const u8 k[16], const u8 r[16], const u8 preq[7], const u8 pres[7], u8 _iat, const bdaddr_t *ia, u8 _rat, const bdaddr_t *ra, u8 res[16]) { @@ -436,7 +432,7 @@ static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], u128_xor((u128 *) res, (u128 *) r, (u128 *) p1); /* res = e(k, res) */ - err = smp_e(tfm_aes, k, res); + err = smp_e(k, res); if (err) { BT_ERR("Encrypt data error"); return err; @@ -453,14 +449,14 @@ static int smp_c1(struct crypto_cipher *tfm_aes, const u8 k[16], u128_xor((u128 *) res, (u128 *) res, (u128 *) p2); /* res = e(k, res) */ - err = smp_e(tfm_aes, k, res); + err = smp_e(k, res); if (err) BT_ERR("Encrypt data error"); return err; } -static int smp_s1(struct crypto_cipher *tfm_aes, const u8 k[16], +static int smp_s1(const u8 k[16], const u8 r1[16], const u8 r2[16], u8 _r[16]) { int err; @@ -469,15 +465,14 @@ static int smp_s1(struct crypto_cipher *tfm_aes, const u8 k[16], memcpy(_r, r2, 8); memcpy(_r + 8, r1, 8); - err = smp_e(tfm_aes, k, _r); + err = smp_e(k, _r); if (err) BT_ERR("Encrypt data error"); return err; } -static int smp_ah(struct crypto_cipher *tfm, const u8 irk[16], - const u8 r[3], u8 res[3]) +static int smp_ah(const u8 irk[16], const u8 r[3], u8 res[3]) { u8 _res[16]; int err; @@ -486,7 +481,7 @@ static int smp_ah(struct crypto_cipher *tfm, const u8 irk[16], memcpy(_res, r, 3); memset(_res + 3, 0, 13); - err = smp_e(tfm, irk, _res); + err = smp_e(irk, _res); if (err) { BT_ERR("Encrypt error"); return err; @@ -518,7 +513,7 @@ bool smp_irk_matches(struct hci_dev *hdev, const u8 irk[16], BT_DBG("RPA %pMR IRK %*phN", bdaddr, 16, irk); - err = smp_ah(smp->tfm_aes, irk, &bdaddr->b[3], hash); + err = smp_ah(irk, &bdaddr->b[3], hash); if (err) return false; @@ -541,7 +536,7 @@ int smp_generate_rpa(struct hci_dev *hdev, const u8 irk[16], bdaddr_t *rpa) rpa->b[5] &= 0x3f; /* Clear two most significant bits */ rpa->b[5] |= 0x40; /* Set second most significant bit */ - err = smp_ah(smp->tfm_aes, irk, &rpa->b[3], rpa->b); + err = smp_ah(irk, &rpa->b[3], rpa->b); if (err < 0) return err; @@ -768,7 +763,6 @@ static void smp_chan_destroy(struct l2cap_conn *conn) kzfree(smp->slave_csrk); kzfree(smp->link_key); - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); @@ -957,7 +951,7 @@ static u8 smp_confirm(struct smp_chan *smp) BT_DBG("conn %p", conn); - ret = smp_c1(smp->tfm_aes, smp->tk, smp->prnd, smp->preq, smp->prsp, + ret = smp_c1(smp->tk, smp->prnd, smp->preq, smp->prsp, conn->hcon->init_addr_type, &conn->hcon->init_addr, conn->hcon->resp_addr_type, &conn->hcon->resp_addr, cp.confirm_val); @@ -983,12 +977,9 @@ static u8 smp_random(struct smp_chan *smp) u8 confirm[16]; int ret; - if (IS_ERR_OR_NULL(smp->tfm_aes)) - return SMP_UNSPECIFIED; - BT_DBG("conn %p %s", conn, conn->hcon->out ? "master" : "slave"); - ret = smp_c1(smp->tfm_aes, smp->tk, smp->rrnd, smp->preq, smp->prsp, + ret = smp_c1(smp->tk, smp->rrnd, smp->preq, smp->prsp, hcon->init_addr_type, &hcon->init_addr, hcon->resp_addr_type, &hcon->resp_addr, confirm); if (ret) @@ -1005,7 +996,7 @@ static u8 smp_random(struct smp_chan *smp) __le64 rand = 0; __le16 ediv = 0; - smp_s1(smp->tfm_aes, smp->tk, smp->rrnd, smp->prnd, stk); + smp_s1(smp->tk, smp->rrnd, smp->prnd, stk); if (test_and_set_bit(HCI_CONN_ENCRYPT_PEND, &hcon->flags)) return SMP_UNSPECIFIED; @@ -1021,7 +1012,7 @@ static u8 smp_random(struct smp_chan *smp) smp_send_cmd(conn, SMP_CMD_PAIRING_RANDOM, sizeof(smp->prnd), smp->prnd); - smp_s1(smp->tfm_aes, smp->tk, smp->prnd, smp->rrnd, stk); + smp_s1(smp->tk, smp->prnd, smp->rrnd, stk); if (hcon->pending_sec_level == BT_SECURITY_HIGH) auth = 1; @@ -1389,16 +1380,10 @@ static struct smp_chan *smp_chan_create(struct l2cap_conn *conn) if (!smp) return NULL; - smp->tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(smp->tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - goto zfree_smp; - } - smp->tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(smp->tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - goto free_cipher; + goto zfree_smp; } smp->tfm_ecdh = crypto_alloc_kpp("ecdh", CRYPTO_ALG_INTERNAL, 0); @@ -1420,8 +1405,6 @@ static struct smp_chan *smp_chan_create(struct l2cap_conn *conn) free_shash: crypto_free_shash(smp->tfm_cmac); -free_cipher: - crypto_free_cipher(smp->tfm_aes); zfree_smp: kzfree(smp); return NULL; @@ -3219,7 +3202,6 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) { struct l2cap_chan *chan; struct smp_dev *smp; - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; @@ -3232,17 +3214,9 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) if (!smp) return ERR_PTR(-ENOMEM); - tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - kzfree(smp); - return ERR_CAST(tfm_aes); - } - tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - crypto_free_cipher(tfm_aes); kzfree(smp); return ERR_CAST(tfm_cmac); } @@ -3251,13 +3225,11 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) if (IS_ERR(tfm_ecdh)) { BT_ERR("Unable to create ECDH crypto context"); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); kzfree(smp); return ERR_CAST(tfm_ecdh); } smp->local_oob = false; - smp->tfm_aes = tfm_aes; smp->tfm_cmac = tfm_cmac; smp->tfm_ecdh = tfm_ecdh; @@ -3265,7 +3237,6 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) chan = l2cap_chan_create(); if (!chan) { if (smp) { - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); kzfree(smp); @@ -3313,7 +3284,6 @@ static void smp_del_chan(struct l2cap_chan *chan) smp = chan->data; if (smp) { chan->data = NULL; - crypto_free_cipher(smp->tfm_aes); crypto_free_shash(smp->tfm_cmac); crypto_free_kpp(smp->tfm_ecdh); kzfree(smp); @@ -3569,7 +3539,7 @@ static int __init test_debug_key(struct crypto_kpp *tfm_ecdh) return 0; } -static int __init test_ah(struct crypto_cipher *tfm_aes) +static int __init test_ah(void) { const u8 irk[16] = { 0x9b, 0x7d, 0x39, 0x0a, 0xa6, 0x10, 0x10, 0x34, @@ -3579,7 +3549,7 @@ static int __init test_ah(struct crypto_cipher *tfm_aes) u8 res[3]; int err; - err = smp_ah(tfm_aes, irk, r, res); + err = smp_ah(irk, r, res); if (err) return err; @@ -3589,7 +3559,7 @@ static int __init test_ah(struct crypto_cipher *tfm_aes) return 0; } -static int __init test_c1(struct crypto_cipher *tfm_aes) +static int __init test_c1(void) { const u8 k[16] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -3609,7 +3579,7 @@ static int __init test_c1(struct crypto_cipher *tfm_aes) u8 res[16]; int err; - err = smp_c1(tfm_aes, k, r, preq, pres, _iat, &ia, _rat, &ra, res); + err = smp_c1(k, r, preq, pres, _iat, &ia, _rat, &ra, res); if (err) return err; @@ -3619,7 +3589,7 @@ static int __init test_c1(struct crypto_cipher *tfm_aes) return 0; } -static int __init test_s1(struct crypto_cipher *tfm_aes) +static int __init test_s1(void) { const u8 k[16] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, @@ -3634,7 +3604,7 @@ static int __init test_s1(struct crypto_cipher *tfm_aes) u8 res[16]; int err; - err = smp_s1(tfm_aes, k, r1, r2, res); + err = smp_s1(k, r1, r2, res); if (err) return err; @@ -3815,8 +3785,7 @@ static const struct file_operations test_smp_fops = { .llseek = default_llseek, }; -static int __init run_selftests(struct crypto_cipher *tfm_aes, - struct crypto_shash *tfm_cmac, +static int __init run_selftests(struct crypto_shash *tfm_cmac, struct crypto_kpp *tfm_ecdh) { ktime_t calltime, delta, rettime; @@ -3831,19 +3800,19 @@ static int __init run_selftests(struct crypto_cipher *tfm_aes, goto done; } - err = test_ah(tfm_aes); + err = test_ah(); if (err) { BT_ERR("smp_ah test failed"); goto done; } - err = test_c1(tfm_aes); + err = test_c1(); if (err) { BT_ERR("smp_c1 test failed"); goto done; } - err = test_s1(tfm_aes); + err = test_s1(); if (err) { BT_ERR("smp_s1 test failed"); goto done; @@ -3900,21 +3869,13 @@ static int __init run_selftests(struct crypto_cipher *tfm_aes, int __init bt_selftest_smp(void) { - struct crypto_cipher *tfm_aes; struct crypto_shash *tfm_cmac; struct crypto_kpp *tfm_ecdh; int err; - tfm_aes = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(tfm_aes)) { - BT_ERR("Unable to create AES crypto context"); - return PTR_ERR(tfm_aes); - } - tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0); if (IS_ERR(tfm_cmac)) { BT_ERR("Unable to create CMAC crypto context"); - crypto_free_cipher(tfm_aes); return PTR_ERR(tfm_cmac); } @@ -3922,14 +3883,12 @@ int __init bt_selftest_smp(void) if (IS_ERR(tfm_ecdh)) { BT_ERR("Unable to create ECDH crypto context"); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); return PTR_ERR(tfm_ecdh); } - err = run_selftests(tfm_aes, tfm_cmac, tfm_ecdh); + err = run_selftests(tfm_cmac, tfm_ecdh); crypto_free_shash(tfm_cmac); - crypto_free_cipher(tfm_aes); crypto_free_kpp(tfm_ecdh); return err; From patchwork Tue Jul 2 19:41:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168370 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653703ilk; Tue, 2 Jul 2019 12:42:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqzkhr6T6NKRr8usAPciFRcY85JQVbzJLqp8VBeN//00xB6s+FuMShuhyrx3eD2jSskZnrdA X-Received: by 2002:a17:902:a5c7:: with SMTP id t7mr33690527plq.288.1562096565382; Tue, 02 Jul 2019 12:42:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096565; cv=none; d=google.com; s=arc-20160816; b=rMhuD9HUxxEdpHUYct3euSZ21QFf6noYBXsh2pHfnNTRu3o/oAw6S21t9q/qncj2mR UDlpEEr+/mYeFTeA8o9xfXim4QPvZLg+5M9fFpd88kBlMpk4XPmYR14ykWXzIHVA4r6F Sf5+OyDtV+RdprK19iP+j+WhOeZte7dQTtmnqaLvPsyESsI7pG1qQY5qTKYcfGgOUxxs a5zumy689OhrYtf1Zlq4vy+u8i8PR137y2EGsQL6Up0xOM5SN2+/Ge8d+NRIN0gyrzLb Fm2e9A1n192hUr6bcFi1JVoKQjFq1Ry9v7gY7VzOH7Hy2loOpMovxU6MHmwVkufP1ipf WWNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=VQ3v2advvdDgCgxLjFb+IrRE2Pm4FCw8P9b47+HGrWM=; b=Obdmte5zjf8VWVhfRcmpdZhLruy6+Wcva86FNOx8mtV7pxyHroz0M4B5P8VQBskeCS M7xYXX0oIK9mcv3WyTReHczSGB5kQScaLAtuFn4sMmM8VlEsIZI+JOqevFhe/dgvyQps j/vytcmNcvbfklH1uLkkXl24PjjU2OD00w+4HZobgSMAiXivZRTQehU0ndK8aOuJIdPV SAIVMRpltRx2J7FHi0VYv2Q1jY1lrbtf/KHt6QUJJjMMcIOHS/QeDacIuEzgtmv5Qtcf WW4mbiZez5DH4lYRuwVI8m7dWOZFVC6E6O9+fW62Tb4lhULLqaWA48bmyLgo6tYpSsKV hm6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a5jJXAy9; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si13373092pgh.116.2019.07.02.12.42.45; Tue, 02 Jul 2019 12:42:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=a5jJXAy9; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727077AbfGBTmo (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:44 -0400 Received: from mail-lj1-f196.google.com ([209.85.208.196]:33804 "EHLO mail-lj1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmo (ORCPT ); Tue, 2 Jul 2019 15:42:44 -0400 Received: by mail-lj1-f196.google.com with SMTP id p17so18207336ljg.1 for ; Tue, 02 Jul 2019 12:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VQ3v2advvdDgCgxLjFb+IrRE2Pm4FCw8P9b47+HGrWM=; b=a5jJXAy92Km3JOrJb1aLFv81FKj0y0B7GbXDsQYA6S9sCF7LQ+4cgq2FzJ0aWjb2IO jOC4O2jvHItRVgMr6MNswrwrvMeThRyJahksL6CaXzHKeidi9tS7FIKY4h3euKKm/rd1 AUJAzD7foHXEnzcD+8KF52+idI7CA6Hz/8vLNgFRqVVl/lzqVGyCewaS0qcIxxCYJjJx zD2m5wvm8N7E8CJK8tXItzf1VH2gYxsMOQsiCuldxQFSA3lQHMqbfjxFT2YzO9u0uY3v 37Nu9xVjcQ0RARCYxPPpwBZzW+y5eCXS3+S8hS/QSgdJURoYc6EF6LTtSXb5R089xFuj 8kuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VQ3v2advvdDgCgxLjFb+IrRE2Pm4FCw8P9b47+HGrWM=; b=tcLadTcVuxirM7gylzF7BfQ5uv9xD/Ct6Fq8wqey1cYWxFWctC0RnNEbUbl9ZEz8dh u2FK8UlYsMDqLpK+auuuIxIuW+MDTvRHeGXdDQt1B3bFmzySdbwLkl3rftWyjRaTBGmQ XLd5kEEyctybCBwRotKDF74wZ2zjPL70O3+G0Gn2KSZUDl3gxydlTP9OAJH0pVXEEOxJ u0AT9bBTPK1klFbjgHkPUHyZSU3bnw3YFWKOdiKcIOeKl6Fe1ND3e4mNwzAut4Xhwgos 5t5nvct7FqGLCh+lcNhkRA1ajMISsYYEf6Fv9qmXwfozlnHy8EtmFoFcUIxrVJRjisxt 3byw== X-Gm-Message-State: APjAAAUYb3Dgw2B5XDTH4IWlIJsCIzCpW/KL/SE5PLXe0Rtblri3EYNs sVvKJfgEY3nweM7ip4UWHFSkrd9AVjzVuIdj X-Received: by 2002:a2e:3c1a:: with SMTP id j26mr18680446lja.230.1562096562363; Tue, 02 Jul 2019 12:42:42 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.40 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:41 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 24/32] crypto: amcc/aes - switch to AES library for GCM key derivation Date: Tue, 2 Jul 2019 21:41:42 +0200 Message-Id: <20190702194150.10405-25-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The AMCC code for GCM key derivation allocates a AES cipher to perform a single block encryption. So let's switch to the new and more lightweight AES library instead. Signed-off-by: Ard Biesheuvel --- drivers/crypto/Kconfig | 2 +- drivers/crypto/amcc/crypto4xx_alg.c | 24 +++++++------------- 2 files changed, 9 insertions(+), 17 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig index b30b84089d11..c7ac1e6d23d4 100644 --- a/drivers/crypto/Kconfig +++ b/drivers/crypto/Kconfig @@ -311,7 +311,7 @@ config CRYPTO_DEV_PPC4XX depends on PPC && 4xx select CRYPTO_HASH select CRYPTO_AEAD - select CRYPTO_AES + select CRYPTO_LIB_AES select CRYPTO_CCM select CRYPTO_CTR select CRYPTO_GCM diff --git a/drivers/crypto/amcc/crypto4xx_alg.c b/drivers/crypto/amcc/crypto4xx_alg.c index 26f86fd7532b..d3660703a36c 100644 --- a/drivers/crypto/amcc/crypto4xx_alg.c +++ b/drivers/crypto/amcc/crypto4xx_alg.c @@ -536,28 +536,20 @@ static int crypto4xx_aes_gcm_validate_keylen(unsigned int keylen) static int crypto4xx_compute_gcm_hash_key_sw(__le32 *hash_start, const u8 *key, unsigned int keylen) { - struct crypto_cipher *aes_tfm = NULL; + struct crypto_aes_ctx ctx; uint8_t src[16] = { 0 }; - int rc = 0; - - aes_tfm = crypto_alloc_cipher("aes", 0, CRYPTO_ALG_NEED_FALLBACK); - if (IS_ERR(aes_tfm)) { - rc = PTR_ERR(aes_tfm); - pr_warn("could not load aes cipher driver: %d\n", rc); - return rc; - } + int rc; - rc = crypto_cipher_setkey(aes_tfm, key, keylen); + rc = aes_expandkey(&ctx, key, keylen); if (rc) { - pr_err("setkey() failed: %d\n", rc); - goto out; + pr_err("aes_expandkey() failed: %d\n", rc); + return rc; } - crypto_cipher_encrypt_one(aes_tfm, src, src); + aes_encrypt(&ctx, src, src); crypto4xx_memcpy_to_le32(hash_start, src, 16); -out: - crypto_free_cipher(aes_tfm); - return rc; + memzero_explicit(&ctx, sizeof(ctx)); + return 0; } int crypto4xx_setkey_aes_gcm(struct crypto_aead *cipher, From patchwork Tue Jul 2 19:41:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168372 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653776ilk; Tue, 2 Jul 2019 12:42:50 -0700 (PDT) X-Google-Smtp-Source: APXvYqyRdxea9Tjyt6cmydIGHyjJ0+u0979+nmd6L3gVPVupJf1210T2XX5NfPhFFOalg8TpCDc/ X-Received: by 2002:a63:1d2:: with SMTP id 201mr1100922pgb.232.1562096569905; Tue, 02 Jul 2019 12:42:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096569; cv=none; d=google.com; s=arc-20160816; b=ruYjnbshFh7JalvrlvOT4m33UqlRarPLu8IUR8hD9U96nuyGLRyclIxiQoAGIi+wE9 yMd83eCdZuhSskDAVyEWQVmfouXpK+JxIUPqm7oQmG/L9g6MhAYH5PovPypUYE/qBEeG iruFora66L05yWN9AIWL6AGYZU5QWghC8cqeSCZ9JUesBeBKhSBmQ5wkNAdNOTEd6EcO zYKqftkcNw11XetJxGVCveYgZC+6KqZW9QKsr8liSXuXPpEFNO9Nw80qUr6mRwfAfd80 Qkw/lnUL5Aa8Vc1sLsx74eKf8Czkrg6FlXu4TOjkI7L804WJ3bxkFE8AQ76cIZPRmMmh IPpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=n5l3F17uXNQenDSvhlR5E0fuSKbS0aVchlvnJvykzg4=; b=gqB2Ds96/nPPBHa6ft5RV1KmJ/DvEPkCclQYtaUfod+4H56FH9wzXbruRzgMdKmBX8 2/nF/6I7NIySGeIK+L+spmdZksBuJJuslDaKntuinLHMd+0T/xwlzOyIiFAPTyBEB0ap yWS/29M4w0UrRgtrQZNGTsGFEDOCXnh9Wg3vxC4mUXGZdM3FrU1t/VAbwzkZMxrFC9hs uef4BL8ydxJ2ECGDYiFZb7nTPS9+oZwEczGdvctEIUAtiKBFbtsf80sNwI/eLo2Qg3gs CW5kevG9vUWd8c2DEu+7iokLTRT2WE+oyQVdLywQUG0RBcW6CrzRuRGbG11L+hDq4SqD 5bpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EERhzbs2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.49; Tue, 02 Jul 2019 12:42:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EERhzbs2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727103AbfGBTmq (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:46 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:37814 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727094AbfGBTmp (ORCPT ); Tue, 2 Jul 2019 15:42:45 -0400 Received: by mail-lj1-f195.google.com with SMTP id 131so18176823ljf.4 for ; Tue, 02 Jul 2019 12:42:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=n5l3F17uXNQenDSvhlR5E0fuSKbS0aVchlvnJvykzg4=; b=EERhzbs2UHUCyBRzT2cFOtGhmJ+RKgtq244yJuVTRXkE8UxD/VtbE8nRMvbUiSIQuH moKG/y6WoFwOG0I3PTOOyrJUb70sOSCT04quKkU/fMp/dMetea/EnG9tt2uepbm2FFry D8fS0PueVm81/oiqsGenEaZmngLw2KZPCI6TPIiSPWOjxg+9H0OC6xlLiiEIayy6VXJL Ue+jdpAe8mY/dMThZXqbgZIus4ZnUnvLNq13V7y5/Q+tllqIpyeyb2s0mev6DVV5NNwU +sSmFzaQfZO/FWMLRHF5urCZSywC6RyrlDAebo3OeOC4+FtZmPfftOHekaClwu+aJBbI eKrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=n5l3F17uXNQenDSvhlR5E0fuSKbS0aVchlvnJvykzg4=; b=ZjY8zPZupxWP5yyPpxFB2DkjNfYTx//L8qaBcVmBE45HkqbpXsPwrjukHTqkaRfyY1 7z2+osjHHjGvh6QgxMuUoazGgDpeEsDyg7wgRfsjZkB3wNDQ+eRxLgkNSd+PHcv+jHpk FMHVfd3WHs+5mD7mNuhSfqJx6MCQyST5YN+VFPBUV1BQBhfRpZ1tUAv9zXer861hiFmX nlMvqhOJ1gjfi852ZTqiHKNj09GwI3xzWf5hb67yIecguVJhEiR++ReBxPgZFRZHbNIR 8cX7YMoehMIeSev3E9O87t4+bZlRd8LCJf3PRXeapYdbp1F/vDpnWbQ21md5tWvUC/m1 hWMQ== X-Gm-Message-State: APjAAAWxhYbrNvAqDoRmpyFkg0DFZIETQ3jMtpyZM+0rgrjnsM5lUqeu 9b9ndkCoCVZEOZq0tvRQdjJ2uL6XvsglJH2D X-Received: by 2002:a2e:7614:: with SMTP id r20mr18761048ljc.42.1562096563562; Tue, 02 Jul 2019 12:42:43 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.42 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:42 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 25/32] crypto: ccp - move to AES library for CMAC key derivation Date: Tue, 2 Jul 2019 21:41:43 +0200 Message-Id: <20190702194150.10405-26-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Use the AES library instead of the cipher interface to perform the single block of AES processing involved in updating the key of the cmac(aes) hash. Signed-off-by: Ard Biesheuvel --- drivers/crypto/ccp/Kconfig | 1 + drivers/crypto/ccp/ccp-crypto-aes-cmac.c | 25 ++++---------------- drivers/crypto/ccp/ccp-crypto.h | 3 --- 3 files changed, 5 insertions(+), 24 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/ccp/Kconfig b/drivers/crypto/ccp/Kconfig index b9dfae47aefd..ee06d0fccdb5 100644 --- a/drivers/crypto/ccp/Kconfig +++ b/drivers/crypto/ccp/Kconfig @@ -29,6 +29,7 @@ config CRYPTO_DEV_CCP_CRYPTO select CRYPTO_BLKCIPHER select CRYPTO_AUTHENC select CRYPTO_RSA + select CRYPTO_LIB_AES help Support for using the cryptographic API with the AMD Cryptographic Coprocessor. This module supports offload of SHA and AES algorithms. diff --git a/drivers/crypto/ccp/ccp-crypto-aes-cmac.c b/drivers/crypto/ccp/ccp-crypto-aes-cmac.c index f6e252c1d6fb..c8f4b29bf044 100644 --- a/drivers/crypto/ccp/ccp-crypto-aes-cmac.c +++ b/drivers/crypto/ccp/ccp-crypto-aes-cmac.c @@ -264,6 +264,7 @@ static int ccp_aes_cmac_setkey(struct crypto_ahash *tfm, const u8 *key, ccp_crypto_ahash_alg(crypto_ahash_tfm(tfm)); u64 k0_hi, k0_lo, k1_hi, k1_lo, k2_hi, k2_lo; u64 rb_hi = 0x00, rb_lo = 0x87; + struct crypto_aes_ctx aes; __be64 *gk; int ret; @@ -287,14 +288,14 @@ static int ccp_aes_cmac_setkey(struct crypto_ahash *tfm, const u8 *key, ctx->u.aes.key_len = 0; /* Set the key for the AES cipher used to generate the keys */ - ret = crypto_cipher_setkey(ctx->u.aes.tfm_cipher, key, key_len); + ret = aes_expandkey(&aes, key, key_len); if (ret) return ret; /* Encrypt a block of zeroes - use key area in context */ memset(ctx->u.aes.key, 0, sizeof(ctx->u.aes.key)); - crypto_cipher_encrypt_one(ctx->u.aes.tfm_cipher, ctx->u.aes.key, - ctx->u.aes.key); + aes_encrypt(&aes, ctx->u.aes.key, ctx->u.aes.key); + memzero_explicit(&aes, sizeof(aes)); /* Generate K1 and K2 */ k0_hi = be64_to_cpu(*((__be64 *)ctx->u.aes.key)); @@ -339,32 +340,15 @@ static int ccp_aes_cmac_cra_init(struct crypto_tfm *tfm) { struct ccp_ctx *ctx = crypto_tfm_ctx(tfm); struct crypto_ahash *ahash = __crypto_ahash_cast(tfm); - struct crypto_cipher *cipher_tfm; ctx->complete = ccp_aes_cmac_complete; ctx->u.aes.key_len = 0; crypto_ahash_set_reqsize(ahash, sizeof(struct ccp_aes_cmac_req_ctx)); - cipher_tfm = crypto_alloc_cipher("aes", 0, CRYPTO_ALG_NEED_FALLBACK); - if (IS_ERR(cipher_tfm)) { - pr_warn("could not load aes cipher driver\n"); - return PTR_ERR(cipher_tfm); - } - ctx->u.aes.tfm_cipher = cipher_tfm; - return 0; } -static void ccp_aes_cmac_cra_exit(struct crypto_tfm *tfm) -{ - struct ccp_ctx *ctx = crypto_tfm_ctx(tfm); - - if (ctx->u.aes.tfm_cipher) - crypto_free_cipher(ctx->u.aes.tfm_cipher); - ctx->u.aes.tfm_cipher = NULL; -} - int ccp_register_aes_cmac_algs(struct list_head *head) { struct ccp_crypto_ahash_alg *ccp_alg; @@ -404,7 +388,6 @@ int ccp_register_aes_cmac_algs(struct list_head *head) base->cra_ctxsize = sizeof(struct ccp_ctx); base->cra_priority = CCP_CRA_PRIORITY; base->cra_init = ccp_aes_cmac_cra_init; - base->cra_exit = ccp_aes_cmac_cra_exit; base->cra_module = THIS_MODULE; ret = crypto_register_ahash(alg); diff --git a/drivers/crypto/ccp/ccp-crypto.h b/drivers/crypto/ccp/ccp-crypto.h index 28819e11db96..9100df77a7b3 100644 --- a/drivers/crypto/ccp/ccp-crypto.h +++ b/drivers/crypto/ccp/ccp-crypto.h @@ -90,9 +90,6 @@ struct ccp_aes_ctx { /* Fallback cipher for XTS with unsupported unit sizes */ struct crypto_sync_skcipher *tfm_skcipher; - /* Cipher used to generate CMAC K1/K2 keys */ - struct crypto_cipher *tfm_cipher; - enum ccp_engine engine; enum ccp_aes_type type; enum ccp_aes_mode mode; From patchwork Tue Jul 2 19:41:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168371 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653763ilk; Tue, 2 Jul 2019 12:42:49 -0700 (PDT) X-Google-Smtp-Source: APXvYqzEp6GVmn0vOxnqxEjW+1NIncncdD8a00XOF/j6tZ5pa7rlYj5rIe+xEP3WjspNCZCG5nMY X-Received: by 2002:a65:4045:: with SMTP id h5mr25941441pgp.247.1562096569054; Tue, 02 Jul 2019 12:42:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096569; cv=none; d=google.com; s=arc-20160816; b=KWwi4VmNYYHo2iWpapWf/BN3OqKFmQqcD2E85Yj2V1jjV+0ORy6wuci5eaTjZrtvnE oA8e/NJnGSf3Y7cISthRW8W0HyAzz0SVy8/1+GWB5zsjOtwT3qMyc90+6tOe5rCizCCr iggKC8eh3VAIlbotHWG2sk6zD5SqcQ47v1aY/dAa3/viGO80h24ZiBXE/3jatMtBNlM1 ei16n8mQRNGZAX7CHBBXkrGBgaQTegtINGn9PDpwEd6u9DyHWu4fSHPPRnClIcKIOW2+ kZBtvDKSujz6Blnx39Fw+lPUP9KvP1ohjlmL48m0kpunO/t1B9VApVvBx0taLemdUv87 Qc8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=k85BuBqTS4QnHHKvJGZeRTIiktlXAufYc+fdc/GM7hc=; b=DrKYWuPPj3xW1h8kjsIIIi1azR0BCPn28HQFpQO7i+Dr8VTitewC4u69ZGVhsaTNnJ 4IyChRZAQxtJK94EqkJEE5jSjzetxBdW7/AQ3ZhLuyLbc12L3ESP8wNdusCq48nzHPw/ wONVhjnAM3Kv14VtY+Tqefvvom1puqSfv2W6UBwY0PfJggGI1pdUOyWkAkqhL9HEEaeU UA1vwA6AN5vmJ3TGJP7DJMDwhvotGWibzvW5nXttGEnMgvPBkNKeVAI1FgGznpO5krwU YB/TVuDwWZEt37Cri5z7Xi+KUfS1rc1xVN1zKmOPHLF2weO4TNNd0dxJSKWa1RI2pxtz Ovvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XwQyVakf; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.48; Tue, 02 Jul 2019 12:42:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XwQyVakf; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727117AbfGBTms (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:48 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:39804 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727101AbfGBTmr (ORCPT ); Tue, 2 Jul 2019 15:42:47 -0400 Received: by mail-lj1-f193.google.com with SMTP id v18so18159360ljh.6 for ; Tue, 02 Jul 2019 12:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=k85BuBqTS4QnHHKvJGZeRTIiktlXAufYc+fdc/GM7hc=; b=XwQyVakfS2MsDBlX+7hNIYUBZc7U36OH8US9LRD0upZFq+OSdzfTxlHIQf3H01WYUl 8OiEthnZ0z2bQt6ERIHAhozAdhMBovIaU2RyREBLW/FsJ83gF3ph15uXVjWnAsi7xSa/ 1HlxquQSrSYBY+fdOXu9U7hNs+4IhOUB3t/OOo2WD0DuUaq3Pym1LY5qMIMRQBW1FyQi E+9v8RuOtHm8Pm+UcC7wLyzXHYpxXmZ9gwb08yE0poaGHuMkO1Dd/Vc3uHLR9ULFLi0J 05ePtEH8KONkSiigpy7VTESCHphlb1Up1lR2zfnFuRlujE4BfesacuR4Ct8QtZKTUtW1 LxmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=k85BuBqTS4QnHHKvJGZeRTIiktlXAufYc+fdc/GM7hc=; b=D7HTxC6WiMdvqGYLim1xAHJAJ6JbqNX1f8qPb22dOIUgjdIfHVDojIjVwVfHetBQ7g xR5DwEX6zI+oI8XggYFC4TzolqL1BXMHBwWGHX/Q9rBuzogwmNw7zx0XWU+itEZOrMVW qHQsU9wkeM9AZOajZcHoQ0X/lncyjcBaQeIckXKiyyTVrPSPAjQ8adWrzEXyc6Gklq18 uXeiuQa2ULJb1nEdBK7r1YSggoKVssamThdBEM2e+NSIZwgLP06oz+65MkvMXGWSK1l0 u0Obuo9geKPd/b9NGVc0DUb5UuB1UchdOdQtoj9jj523/tKQV6Kr0ceNFXzdRrfcv8Bv 7R9Q== X-Gm-Message-State: APjAAAXEqURxTPAQpHY+AoUfek8OvZ1rg14yrbLlkdYEL++AZRq8BTon 05yHBFBxRetvFyXQG1hVDqRHz5D3LrQvdBGe X-Received: by 2002:a2e:1290:: with SMTP id 16mr17669109ljs.88.1562096564836; Tue, 02 Jul 2019 12:42:44 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.43 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:44 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 26/32] crypto: chelsio/aes - replace AES cipher calls with library calls Date: Tue, 2 Jul 2019 21:41:44 +0200 Message-Id: <20190702194150.10405-27-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Replace a couple of occurrences where the "aes-generic" cipher is instantiated explicitly and only used for encryption of a single block. Use AES library calls instead. Signed-off-by: Ard Biesheuvel --- drivers/crypto/chelsio/Kconfig | 1 + drivers/crypto/chelsio/chcr_algo.c | 46 ++++++-------------- drivers/crypto/chelsio/chcr_crypto.h | 1 - drivers/crypto/chelsio/chcr_ipsec.c | 19 +++----- drivers/crypto/chelsio/chtls/chtls_hw.c | 20 +++------ 5 files changed, 26 insertions(+), 61 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/chelsio/Kconfig b/drivers/crypto/chelsio/Kconfig index 930d82d991f2..36402ba63b50 100644 --- a/drivers/crypto/chelsio/Kconfig +++ b/drivers/crypto/chelsio/Kconfig @@ -1,6 +1,7 @@ config CRYPTO_DEV_CHELSIO tristate "Chelsio Crypto Co-processor Driver" depends on CHELSIO_T4 + select CRYPTO_LIB_AES select CRYPTO_SHA1 select CRYPTO_SHA256 select CRYPTO_SHA512 diff --git a/drivers/crypto/chelsio/chcr_algo.c b/drivers/crypto/chelsio/chcr_algo.c index 177f572b9589..38ee38b37ae6 100644 --- a/drivers/crypto/chelsio/chcr_algo.c +++ b/drivers/crypto/chelsio/chcr_algo.c @@ -1023,22 +1023,21 @@ static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv, struct crypto_ablkcipher *tfm = crypto_ablkcipher_reqtfm(req); struct ablk_ctx *ablkctx = ABLK_CTX(c_ctx(tfm)); struct chcr_blkcipher_req_ctx *reqctx = ablkcipher_request_ctx(req); - struct crypto_cipher *cipher; + struct crypto_aes_ctx aes; int ret, i; u8 *key; unsigned int keylen; int round = reqctx->last_req_len / AES_BLOCK_SIZE; int round8 = round / 8; - cipher = ablkctx->aes_generic; memcpy(iv, reqctx->iv, AES_BLOCK_SIZE); keylen = ablkctx->enckey_len / 2; key = ablkctx->key + keylen; - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) - goto out; - crypto_cipher_encrypt_one(cipher, iv, iv); + return ret; + aes_encrypt(&aes, iv, iv); for (i = 0; i < round8; i++) gf128mul_x8_ble((le128 *)iv, (le128 *)iv); @@ -1046,9 +1045,10 @@ static int chcr_update_tweak(struct ablkcipher_request *req, u8 *iv, gf128mul_x_ble((le128 *)iv, (le128 *)iv); if (!isfinal) - crypto_cipher_decrypt_one(cipher, iv, iv); -out: - return ret; + aes_decrypt(&aes, iv, iv); + + memzero_explicit(&aes, sizeof(aes)); + return 0; } static int chcr_update_cipher_iv(struct ablkcipher_request *req, @@ -1411,16 +1411,6 @@ static int chcr_cra_init(struct crypto_tfm *tfm) return PTR_ERR(ablkctx->sw_cipher); } - if (get_cryptoalg_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_XTS) { - /* To update tweak*/ - ablkctx->aes_generic = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(ablkctx->aes_generic)) { - pr_err("failed to allocate aes cipher for tweak\n"); - return PTR_ERR(ablkctx->aes_generic); - } - } else - ablkctx->aes_generic = NULL; - tfm->crt_ablkcipher.reqsize = sizeof(struct chcr_blkcipher_req_ctx); return chcr_device_init(crypto_tfm_ctx(tfm)); } @@ -1451,8 +1441,6 @@ static void chcr_cra_exit(struct crypto_tfm *tfm) struct ablk_ctx *ablkctx = ABLK_CTX(ctx); crypto_free_sync_skcipher(ablkctx->sw_cipher); - if (ablkctx->aes_generic) - crypto_free_cipher(ablkctx->aes_generic); } static int get_alg_config(struct algo_param *params, @@ -3364,9 +3352,9 @@ static int chcr_gcm_setkey(struct crypto_aead *aead, const u8 *key, { struct chcr_aead_ctx *aeadctx = AEAD_CTX(a_ctx(aead)); struct chcr_gcm_ctx *gctx = GCM_CTX(aeadctx); - struct crypto_cipher *cipher; unsigned int ck_size; int ret = 0, key_ctx_size = 0; + struct crypto_aes_ctx aes; aeadctx->enckey_len = 0; crypto_aead_clear_flags(aeadctx->sw_cipher, CRYPTO_TFM_REQ_MASK); @@ -3409,23 +3397,15 @@ static int chcr_gcm_setkey(struct crypto_aead *aead, const u8 *key, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(cipher)) { - aeadctx->enckey_len = 0; - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) { aeadctx->enckey_len = 0; - goto out1; + goto out; } memset(gctx->ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, gctx->ghash_h, gctx->ghash_h); + aes_encrypt(&aes, gctx->ghash_h, gctx->ghash_h); + memzero_explicit(&aes, sizeof(aes)); -out1: - crypto_free_cipher(cipher); out: return ret; } diff --git a/drivers/crypto/chelsio/chcr_crypto.h b/drivers/crypto/chelsio/chcr_crypto.h index 655606f2e4d0..993c97e70565 100644 --- a/drivers/crypto/chelsio/chcr_crypto.h +++ b/drivers/crypto/chelsio/chcr_crypto.h @@ -172,7 +172,6 @@ static inline struct chcr_context *h_ctx(struct crypto_ahash *tfm) struct ablk_ctx { struct crypto_sync_skcipher *sw_cipher; - struct crypto_cipher *aes_generic; __be32 key_ctx_hdr; unsigned int enckey_len; unsigned char ciph_mode; diff --git a/drivers/crypto/chelsio/chcr_ipsec.c b/drivers/crypto/chelsio/chcr_ipsec.c index f429aae72542..24355680f30a 100644 --- a/drivers/crypto/chelsio/chcr_ipsec.c +++ b/drivers/crypto/chelsio/chcr_ipsec.c @@ -132,11 +132,11 @@ static inline int chcr_ipsec_setauthsize(struct xfrm_state *x, static inline int chcr_ipsec_setkey(struct xfrm_state *x, struct ipsec_sa_entry *sa_entry) { - struct crypto_cipher *cipher; int keylen = (x->aead->alg_key_len + 7) / 8; unsigned char *key = x->aead->alg_key; int ck_size, key_ctx_size = 0; unsigned char ghash_h[AEAD_H_SIZE]; + struct crypto_aes_ctx aes; int ret = 0; if (keylen > 3) { @@ -170,26 +170,19 @@ static inline int chcr_ipsec_setkey(struct xfrm_state *x, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes-generic", 0, 0); - if (IS_ERR(cipher)) { - sa_entry->enckey_len = 0; - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) { sa_entry->enckey_len = 0; - goto out1; + goto out; } memset(ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, ghash_h, ghash_h); + aes_encrypt(&aes, ghash_h, ghash_h); + memzero_explicit(&aes, sizeof(aes)); + memcpy(sa_entry->key + (DIV_ROUND_UP(sa_entry->enckey_len, 16) * 16), ghash_h, AEAD_H_SIZE); sa_entry->kctx_len = ((DIV_ROUND_UP(sa_entry->enckey_len, 16)) << 4) + AEAD_H_SIZE; -out1: - crypto_free_cipher(cipher); out: return ret; } diff --git a/drivers/crypto/chelsio/chtls/chtls_hw.c b/drivers/crypto/chelsio/chtls/chtls_hw.c index 490960755864..a6f0278f3597 100644 --- a/drivers/crypto/chelsio/chtls/chtls_hw.c +++ b/drivers/crypto/chelsio/chtls/chtls_hw.c @@ -216,8 +216,8 @@ static int chtls_key_info(struct chtls_sock *csk, unsigned char key[AES_KEYSIZE_128]; struct tls12_crypto_info_aes_gcm_128 *gcm_ctx; unsigned char ghash_h[AEAD_H_SIZE]; - struct crypto_cipher *cipher; int ck_size, key_ctx_size; + struct crypto_aes_ctx aes; int ret; gcm_ctx = (struct tls12_crypto_info_aes_gcm_128 *) @@ -237,18 +237,13 @@ static int chtls_key_info(struct chtls_sock *csk, /* Calculate the H = CIPH(K, 0 repeated 16 times). * It will go in key context */ - cipher = crypto_alloc_cipher("aes", 0, 0); - if (IS_ERR(cipher)) { - ret = -ENOMEM; - goto out; - } - - ret = crypto_cipher_setkey(cipher, key, keylen); + ret = aes_expandkey(&aes, key, keylen); if (ret) - goto out1; + return ret; memset(ghash_h, 0, AEAD_H_SIZE); - crypto_cipher_encrypt_one(cipher, ghash_h, ghash_h); + aes_encrypt(&aes, ghash_h, ghash_h); + memzero_explicit(&aes, sizeof(aes)); csk->tlshws.keylen = key_ctx_size; /* Copy the Key context */ @@ -272,10 +267,7 @@ static int chtls_key_info(struct chtls_sock *csk, /* erase key info from driver */ memset(gcm_ctx->key, 0, keylen); -out1: - crypto_free_cipher(cipher); -out: - return ret; + return 0; } static void chtls_set_scmd(struct chtls_sock *csk) From patchwork Tue Jul 2 19:41:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168374 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653808ilk; Tue, 2 Jul 2019 12:42:51 -0700 (PDT) X-Google-Smtp-Source: APXvYqxJMy1XygRF4VBuEEU1JTI8uQx8qlFINXAIcxYRcnpD9BWjvbVGk6UFeejx3LJl3I09CDOn X-Received: by 2002:a17:902:9689:: with SMTP id n9mr37818511plp.241.1562096571735; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096571; cv=none; d=google.com; s=arc-20160816; b=I5iZ+AJwnsBqtuKSO99G1HS1PdNo+nDQKUupyL0ajQ6UEXmDeIO10/x+KVPcH3/CIa LD8UzLiekNxLf9rA6CONaJsvBkRhqJwAAYBy1xxFSswgh2ddRisFMMy2odlWbU2E5qhF yJb75uPc+GWXJb2v6ZxzqgNb0iZnqRT+GnniUvSdBR70qTml4kDQmcgr4meLOV2QE7U8 h9U6Vxy3+dMEhB9eIkXsRZrt8wCn+1uWu6NwxgNVprBgujK4wRzCYzMc1hXvDX643Uy8 BGwtadyBrHXXF5v3plxTO+y3sgtuydvamu98yJhjo+0wMh/CNO7Qh097etNIBBIKCiAO FOmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=AD8PvOEZeWj16u5SSLkHbNcyzb21rPobxHD6kFBr7Fk=; b=seC7S5pDEcuJFmm+0r54wM8ToRGIGLYLRq3ORjsM8JIL8mLC6Y2eEjgjDdBBDQUwwJ Ok3smq6lweCe0bXOzr8Tb0wFKZXAKirImYFiNbVOm3xLjyyE+0e1FjHHBMHktfun3kSq +tNUfhT9fs+qoCqBzhqjKFE/giU7JaCNbL+BjgzvEGzzMMMglJRqj3rGMqyBlU+SgVnQ CQmfL4Debw6tekg+JMUWWblcZsMBWAaQrHX/+D9O7TcXIzJwxAsjcl2SrY572zkxac4H EfKbMSNw05eqriEtCyq/Fo++CJ9V7e2pwlLOMk3v5ThUf3+JrcETKSrPuXVQ/DbjYGK5 R8Eg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oKXQkqHU; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.51; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oKXQkqHU; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727101AbfGBTmu (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:50 -0400 Received: from mail-lj1-f193.google.com ([209.85.208.193]:45687 "EHLO mail-lj1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTms (ORCPT ); Tue, 2 Jul 2019 15:42:48 -0400 Received: by mail-lj1-f193.google.com with SMTP id m23so18148492lje.12 for ; Tue, 02 Jul 2019 12:42:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AD8PvOEZeWj16u5SSLkHbNcyzb21rPobxHD6kFBr7Fk=; b=oKXQkqHUsHz/edKeiBwZP+iX+iVXjat0a0wxD9FYSK6I34WTJH5GyBr8PLx+Lc0BqW i7Osdo+uG4VQ63D9ByGN9aD75YW9OtbSL0U77kSTqqZKJkbZhjcTNI66bGnEAetkaVsF ir/kL7d1uEZfPoC3j53N0sQSrFs+lN/ebS6OFrJbgPFWcOY9EIwaK7CTNbAP6f9EbEdT d8Naz+IAft9IASgma7UWg+rTD1V8bjU74UUlhk5uH+fmLnI/6umDv7+e9ElSEmKFquqh 35u3JVZCr9+aIRBq2f50nUO3R7BZ30AdGp+eJy95L8qIO7junMmtlz6BWpPl/NwMsmkk POBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AD8PvOEZeWj16u5SSLkHbNcyzb21rPobxHD6kFBr7Fk=; b=HQUVaQxwX10F61ykFj3biXPsUcJm4cIOLuAb+H+ftbomPFq71hsU2za4ZoUSI9FxyS YjE14mQE46HAYGkebrrL5VvnTrSsG+2Sf9d3CJsHZ2M5E7cOeE8JCjefLvf2AqH6RkdG M5E1TXqNoJkJv/NFNxEr9XHnhM+RoSzZNTU7uTUuTqXAUKa8Xh2XSKJ7o+Med7wTp0fy IrjaArN6HRMAtFAY9b94l/97+CpWUEzHO3ZQYLOsFxtO0CzPr2j2kSiXG21oFbUh2kjo To8GNZWOdRPuZmCHpz8YON89Zh5ZNw2VKwXZzbsEkVAygtlXbGZWPOV6zUroIlSIy9Yy JjmA== X-Gm-Message-State: APjAAAWBWIkjkfQuSl+HIdsoi+q3Q5sbuagB3xpGkw//j78+maUZyLEo 6ApkZhGGe38jXHdsI+YBTXyhwk0Tof1YO8wc X-Received: by 2002:a2e:9643:: with SMTP id z3mr18859184ljh.43.1562096566046; Tue, 02 Jul 2019 12:42:46 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.44 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:45 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 27/32] crypto: aes/generic - unexport last-round AES tables Date: Tue, 2 Jul 2019 21:41:45 +0200 Message-Id: <20190702194150.10405-28-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The versions of the AES lookup tables that are only used during the last round are never used outside of the driver, so there is no need to export their symbols. Signed-off-by: Ard Biesheuvel --- crypto/aes_generic.c | 6 ++---- include/crypto/aes.h | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-) -- 2.17.1 diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c index 426deb437f19..71a5c190d360 100644 --- a/crypto/aes_generic.c +++ b/crypto/aes_generic.c @@ -328,7 +328,7 @@ __visible const u32 crypto_ft_tab[4][256] ____cacheline_aligned = { } }; -__visible const u32 crypto_fl_tab[4][256] ____cacheline_aligned = { +static const u32 crypto_fl_tab[4][256] ____cacheline_aligned = { { 0x00000063, 0x0000007c, 0x00000077, 0x0000007b, 0x000000f2, 0x0000006b, 0x0000006f, 0x000000c5, @@ -856,7 +856,7 @@ __visible const u32 crypto_it_tab[4][256] ____cacheline_aligned = { } }; -__visible const u32 crypto_il_tab[4][256] ____cacheline_aligned = { +static const u32 crypto_il_tab[4][256] ____cacheline_aligned = { { 0x00000052, 0x00000009, 0x0000006a, 0x000000d5, 0x00000030, 0x00000036, 0x000000a5, 0x00000038, @@ -1121,9 +1121,7 @@ __visible const u32 crypto_il_tab[4][256] ____cacheline_aligned = { }; EXPORT_SYMBOL_GPL(crypto_ft_tab); -EXPORT_SYMBOL_GPL(crypto_fl_tab); EXPORT_SYMBOL_GPL(crypto_it_tab); -EXPORT_SYMBOL_GPL(crypto_il_tab); /** * crypto_aes_set_key - Set the AES key. diff --git a/include/crypto/aes.h b/include/crypto/aes.h index 0a64a977f9b3..df8426fd8051 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -29,9 +29,7 @@ struct crypto_aes_ctx { }; extern const u32 crypto_ft_tab[4][256] ____cacheline_aligned; -extern const u32 crypto_fl_tab[4][256] ____cacheline_aligned; extern const u32 crypto_it_tab[4][256] ____cacheline_aligned; -extern const u32 crypto_il_tab[4][256] ____cacheline_aligned; int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key, unsigned int key_len); From patchwork Tue Jul 2 19:41:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168373 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653800ilk; Tue, 2 Jul 2019 12:42:51 -0700 (PDT) X-Google-Smtp-Source: APXvYqziA4+pg3oKA3hze8ZQO5/FfhVQEwbaKHasF+qFsFcPMpl1e2iddPExc557W7GtHJqreKkj X-Received: by 2002:a17:90a:4803:: with SMTP id a3mr7715027pjh.58.1562096571398; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096571; cv=none; d=google.com; s=arc-20160816; b=kytcRZo+lPsExWb/qffJSnlhy4I4gx7iesX3sstbl2WleoUr2QG/ak/wmQ8JM18zmS IXWClnQDvEZY9yWP94aPEbxYVfraxGWr/SEU777LYUGxmcuVdMNL3dhaNPzPkE2OIQqw VVwo4zQQAo1MV8oi8DZL+K5loCgXKkRdMahwE6jZn9T1gT1t2fUi5HD/GRd7rAbQj6BM 6Y1epJuy8Dpw29GKULH+j8uYbvPGL3GjfNDwoGOm5zXONir3W4JzUZLnKwP5bJZ3Ghk3 T6czjB0dOkNL07E9syNEPODWqsq9VkJCWrY2VtDD3WL+bcSOph4bK/C2SGrBtDNFAXfH twpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=II4IaI2l8ysBnoBGFcY14xsKlW45HqjpdlGpUkQ7rBQ=; b=ajmZnKa8K984guiXyuBQBzdlhe2PPUs/YIcqQlR60Qz8G6T9z1zuTxcz0vHv37Dwqd frcFdOCRSXSpXu6ZMAb1Zeic8gzyspegjPaaTy0rsCd2xCDwUJg2NYUrrNXvK0mZNyKa mtIdi5A1Ic+0EFMHBuLHSwR1081BNvFTGsMiwTdkEnNC99VlQbGOwkn/A4gBd+Ujz36+ c1387CsQz0RnqKsvqbtw3uPFAjPt7B6jE2BL6qpzcRKzR9V8FKPdJKuj0Qb5PpAad+M1 SOdyDvzcIapaVtTCQSbiSVKI/8XAVooDkvz3baia27GOZIlYBBKMkjU8qxPh0e0WAQ2H Gwsg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y4AyQA13; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.51; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y4AyQA13; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727119AbfGBTmu (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:50 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:37963 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727101AbfGBTmt (ORCPT ); Tue, 2 Jul 2019 15:42:49 -0400 Received: by mail-lj1-f195.google.com with SMTP id r9so18181790ljg.5 for ; Tue, 02 Jul 2019 12:42:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=II4IaI2l8ysBnoBGFcY14xsKlW45HqjpdlGpUkQ7rBQ=; b=Y4AyQA13rcUlRIXMIl6Mk3NNAkJq4EQnuk82FBjoq/Qv+Zwd6RhPeTxLusCR8WustC r1+2xoT5ECWF5WuYISoiLeM5jQWSXe//Swi827ZKtsu0IFTo4xkstqpFoTQEjgoQB+rS t+pYWKzVpFB3TNV6s6VKBeLJZE9cKibesYLscNDUbJ/dsusDr/ZWmIY7n3cUy47hDSr7 0gHhD3GM6YZqhE7e2G0NkSuENaFunDkqJpWKM2ndKGoyfcEv4deJYy04KjAtp6GACG/U H0/KIEKUScE3ovwYMDBIQme49SbhUYvyMKsC4iDjmujVPtagpRgMFM+lfsDxmNu9aHrk QEyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=II4IaI2l8ysBnoBGFcY14xsKlW45HqjpdlGpUkQ7rBQ=; b=elXG9DY4wVFFmQZHbepx6/e4fwz5BbtOygLi7EiPBVQVI+raEUr0Zuk4Lv2k0xVRMQ /mJXVRqXKR7J99ffRVz748mgOFnEM/htqi4lroKPn16ah+3b4PJ3MuwaDXk7lwPmNsnz xpzQxGcrkOgtCkYR1TKAX+YMpvadW7sp6c+qzSEsQkKvaBQ4/GZKOXhfA3URjfOTdLe0 68wjXi2h65zLijCm4R1oJJNvy+qn7DwS0t6QYCmT+x8kw1l1XGajiIIQrHBXGVNgcISM oeLblpOL0ppQy2wjAhaQvr+toxSrjvNyLuPYiWYVLP3cMlOgEuM7dhuhu9nF4z6uRIVw Hp6g== X-Gm-Message-State: APjAAAUx5H7AsxLyMrzh0H7b2iJWGl3V2+YwPSsJp9vEQaw4QsOkwVBg YggTCKo0ES3TrJZ7QYiGIQ8fFmRYq6qVU86b X-Received: by 2002:a2e:b1c1:: with SMTP id e1mr18245038lja.228.1562096567433; Tue, 02 Jul 2019 12:42:47 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.46 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:46 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 28/32] crypto: lib/aes - export sbox and inverse sbox Date: Tue, 2 Jul 2019 21:41:46 +0200 Message-Id: <20190702194150.10405-29-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org There are a few copies of the AES S-boxes floating around, so export the ones from the AES library so that we can reuse them in other modules. Signed-off-by: Ard Biesheuvel --- include/crypto/aes.h | 3 +++ lib/crypto/aes.c | 6 ++++++ 2 files changed, 9 insertions(+) -- 2.17.1 diff --git a/include/crypto/aes.h b/include/crypto/aes.h index df8426fd8051..8e0f4cf948e5 100644 --- a/include/crypto/aes.h +++ b/include/crypto/aes.h @@ -67,4 +67,7 @@ void aes_encrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); */ void aes_decrypt(const struct crypto_aes_ctx *ctx, u8 *out, const u8 *in); +extern const u8 crypto_aes_sbox[]; +extern const u8 crypto_aes_inv_sbox[]; + #endif diff --git a/lib/crypto/aes.c b/lib/crypto/aes.c index 9928b23e0a8a..4e100af38c51 100644 --- a/lib/crypto/aes.c +++ b/lib/crypto/aes.c @@ -82,6 +82,12 @@ static volatile const u8 __cacheline_aligned aes_inv_sbox[] = { 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d, }; +extern const u8 crypto_aes_sbox[256] __alias(aes_sbox); +extern const u8 crypto_aes_inv_sbox[256] __alias(aes_inv_sbox); + +EXPORT_SYMBOL(crypto_aes_sbox); +EXPORT_SYMBOL(crypto_aes_inv_sbox); + static u32 mul_by_x(u32 w) { u32 x = w & 0x7f7f7f7f; From patchwork Tue Jul 2 19:41:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168375 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653831ilk; Tue, 2 Jul 2019 12:42:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqx/7rKoqtBhIMXx2rioOPj1y8xSn9E/o9I/jRbELmd+rtb57XQTI+/ebEXFYKVhw61bCVsT X-Received: by 2002:a65:6694:: with SMTP id b20mr2551900pgw.155.1562096572923; Tue, 02 Jul 2019 12:42:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096572; cv=none; d=google.com; s=arc-20160816; b=wRifMttP1FofwfEQDY0MeoNvcHQ2iceyiNlHGDk9ENolbFgiTp6VVggHk2aDBS2EBY 8Ue+Bo8s8Vo8kzZ6f9/nFHsR3vYVgaOcDimJJuw7nsltX0ngJ7OScBKPPSXzP/wH7wiC OHKPEL3WXqn+feE4995ifCAgiVtfLIb+V2PDJ/v3nxiHZKOmSzgxyoIzyU9vEpS0h8xo /hSUhussYWwudoMf3ieaEQjmsNLyWRjxW1f2Nw62ehbwi0YpOS1FwprNROHTEXiBr6n1 A/rQybODVH3eOcFoXbL/bgp9JUK5HsS89jW33ofOF/aD5jq221GTc4ZydcyTvsXNtTOv ygsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=LHaRCJS1W8CsyX/XEABuzLRpJ3/LjklfBPli38T3UqI=; b=i/p822iGl05IAptNq1bIzBZxuJEg7d0/DFJDKDGFRCxGpMN9GhsLAd0NS6tL6DfmeS YN5zSRaNtDL2ULxSzx8umt4XVfFXOuSyRbapQyg5pp1ehUMFYkqDs58SGFpULKhJQwz7 ogtFyYwpUucLBIIeTFUfT4Y1OSl0gLMTrfEe0RTY6TdPhuO1bM4vdYH1W5J3cMfGgmlK po778AepL5v6QZYH40x/04I7jFboH09A8SBs2oTFrcUVmoJnTW7BfbG1CREKsWV18kG/ R8dBsdW3ZO12OvLhR+nO/G9bsBgrK5lnUYUu9A9dQeMBjrWxs7WUdR2gkkenTnA4kq5D wpZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VG17cYdF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.52; Tue, 02 Jul 2019 12:42:52 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VG17cYdF; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727123AbfGBTmw (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:52 -0400 Received: from mail-lj1-f195.google.com ([209.85.208.195]:40463 "EHLO mail-lj1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727118AbfGBTmv (ORCPT ); Tue, 2 Jul 2019 15:42:51 -0400 Received: by mail-lj1-f195.google.com with SMTP id a21so18169870ljh.7 for ; Tue, 02 Jul 2019 12:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LHaRCJS1W8CsyX/XEABuzLRpJ3/LjklfBPli38T3UqI=; b=VG17cYdF84n7ReYIZM2oIYqY0uRAfbEKYCvDYiheEFeGixc0cTkXktMBGC568NjmRK LE/L6mCz9HSxRxQw66Mi+6u1r2vFlQbXMDEsuZdyEEBhH9ZzaAHnKrodxwcUJqZmHcD1 1I2PscaV8huajLIbrKZV73hDQyZshGaXE2po6CMrrLkASrIkOQQmu4uGv3SxXsUnkNjO i9/N0Pxsw9ob7UtDBc5xkh6i9Bpqxf07Het6HIko70vK8bD/ziPtPZBXMUR7K438Snaw YunMI5D00ztGyUqrwH7MMY/J8BK2kFOrWaiC7C1px5tb5WZHP50ct7qCLFUzIGzXUMXA 7HMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LHaRCJS1W8CsyX/XEABuzLRpJ3/LjklfBPli38T3UqI=; b=khfctVWk8j+bEATrLdtb6Y32iT58t4IPTys9jR3tOuwNzOeSzqL4XQWOzNIlwvBhC1 NqYkm5vhrkv/Z/VCDNQzuD2Zvb/ByCAFtVXnWNDqBJIPaia4FoUzc8b9YMwAh1BBrc9m 8CBONnTK8t3l/nl6fhRz/B7rJ4GJX1D7cRf7WEFpQEOYHnuQOtI6JMV9zBaC2lmNYicm SIIAQFuYcWUxrC4blI1+ix/P/VLlen+FUPEHeXJttHgk4czFpJLfE1zPul7CLZJdJ2Cc /b2H7m6S3kJPOCNJgMt71eRNAluyxpKVtmNtX36Z00tuLu71Wk6QiPhzV3xkD+zuvxyd OQyA== X-Gm-Message-State: APjAAAVRB5gEIclA+Yo3FCdweHshuk4ro/+PmGRB7MqtOWFqe9iEkMDC x8q84cQD3UnoNKlBdKiAXnk3Lg2rOVhZwmlx X-Received: by 2002:a2e:2b19:: with SMTP id q25mr18548220lje.127.1562096568637; Tue, 02 Jul 2019 12:42:48 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.47 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:48 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 29/32] crypto: arm64/aes-neon - switch to shared AES Sboxes Date: Tue, 2 Jul 2019 21:41:47 +0200 Message-Id: <20190702194150.10405-30-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-neon.S | 74 +------------------- 1 file changed, 3 insertions(+), 71 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/aes-neon.S b/arch/arm64/crypto/aes-neon.S index 29100f692e8a..169e86d8ae36 100644 --- a/arch/arm64/crypto/aes-neon.S +++ b/arch/arm64/crypto/aes-neon.S @@ -50,7 +50,7 @@ /* do preload for encryption */ .macro enc_prepare, ignore0, ignore1, temp - prepare .LForward_Sbox, .LForward_ShiftRows, \temp + prepare crypto_aes_sbox, .LForward_ShiftRows, \temp .endm .macro enc_switch_key, ignore0, ignore1, temp @@ -59,7 +59,7 @@ /* do preload for decryption */ .macro dec_prepare, ignore0, ignore1, temp - prepare .LReverse_Sbox, .LReverse_ShiftRows, \temp + prepare crypto_aes_inv_sbox, .LReverse_ShiftRows, \temp .endm /* apply SubBytes transformation using the the preloaded Sbox */ @@ -279,75 +279,7 @@ #include "aes-modes.S" .section ".rodata", "a" - .align 6 -.LForward_Sbox: - .byte 0x63, 0x7c, 0x77, 0x7b, 0xf2, 0x6b, 0x6f, 0xc5 - .byte 0x30, 0x01, 0x67, 0x2b, 0xfe, 0xd7, 0xab, 0x76 - .byte 0xca, 0x82, 0xc9, 0x7d, 0xfa, 0x59, 0x47, 0xf0 - .byte 0xad, 0xd4, 0xa2, 0xaf, 0x9c, 0xa4, 0x72, 0xc0 - .byte 0xb7, 0xfd, 0x93, 0x26, 0x36, 0x3f, 0xf7, 0xcc - .byte 0x34, 0xa5, 0xe5, 0xf1, 0x71, 0xd8, 0x31, 0x15 - .byte 0x04, 0xc7, 0x23, 0xc3, 0x18, 0x96, 0x05, 0x9a - .byte 0x07, 0x12, 0x80, 0xe2, 0xeb, 0x27, 0xb2, 0x75 - .byte 0x09, 0x83, 0x2c, 0x1a, 0x1b, 0x6e, 0x5a, 0xa0 - .byte 0x52, 0x3b, 0xd6, 0xb3, 0x29, 0xe3, 0x2f, 0x84 - .byte 0x53, 0xd1, 0x00, 0xed, 0x20, 0xfc, 0xb1, 0x5b - .byte 0x6a, 0xcb, 0xbe, 0x39, 0x4a, 0x4c, 0x58, 0xcf - .byte 0xd0, 0xef, 0xaa, 0xfb, 0x43, 0x4d, 0x33, 0x85 - .byte 0x45, 0xf9, 0x02, 0x7f, 0x50, 0x3c, 0x9f, 0xa8 - .byte 0x51, 0xa3, 0x40, 0x8f, 0x92, 0x9d, 0x38, 0xf5 - .byte 0xbc, 0xb6, 0xda, 0x21, 0x10, 0xff, 0xf3, 0xd2 - .byte 0xcd, 0x0c, 0x13, 0xec, 0x5f, 0x97, 0x44, 0x17 - .byte 0xc4, 0xa7, 0x7e, 0x3d, 0x64, 0x5d, 0x19, 0x73 - .byte 0x60, 0x81, 0x4f, 0xdc, 0x22, 0x2a, 0x90, 0x88 - .byte 0x46, 0xee, 0xb8, 0x14, 0xde, 0x5e, 0x0b, 0xdb - .byte 0xe0, 0x32, 0x3a, 0x0a, 0x49, 0x06, 0x24, 0x5c - .byte 0xc2, 0xd3, 0xac, 0x62, 0x91, 0x95, 0xe4, 0x79 - .byte 0xe7, 0xc8, 0x37, 0x6d, 0x8d, 0xd5, 0x4e, 0xa9 - .byte 0x6c, 0x56, 0xf4, 0xea, 0x65, 0x7a, 0xae, 0x08 - .byte 0xba, 0x78, 0x25, 0x2e, 0x1c, 0xa6, 0xb4, 0xc6 - .byte 0xe8, 0xdd, 0x74, 0x1f, 0x4b, 0xbd, 0x8b, 0x8a - .byte 0x70, 0x3e, 0xb5, 0x66, 0x48, 0x03, 0xf6, 0x0e - .byte 0x61, 0x35, 0x57, 0xb9, 0x86, 0xc1, 0x1d, 0x9e - .byte 0xe1, 0xf8, 0x98, 0x11, 0x69, 0xd9, 0x8e, 0x94 - .byte 0x9b, 0x1e, 0x87, 0xe9, 0xce, 0x55, 0x28, 0xdf - .byte 0x8c, 0xa1, 0x89, 0x0d, 0xbf, 0xe6, 0x42, 0x68 - .byte 0x41, 0x99, 0x2d, 0x0f, 0xb0, 0x54, 0xbb, 0x16 - -.LReverse_Sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - + .align 4 .LForward_ShiftRows: .octa 0x0b06010c07020d08030e09040f0a0500 From patchwork Tue Jul 2 19:41:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168376 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653841ilk; Tue, 2 Jul 2019 12:42:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqx5AMSuT+rpht4ULQg1JJObphT17etQkJJte0OAyeXb3pD68pB90h3CkBG5TfC2XGSSoK6B X-Received: by 2002:a63:790c:: with SMTP id u12mr14066566pgc.424.1562096573602; Tue, 02 Jul 2019 12:42:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096573; cv=none; d=google.com; s=arc-20160816; b=nwPvP8VZe9KyKgvIP4yiPRohJnpFmvMK7pOaHe9Zwh5ImTdhDF7n5vxoRq54sQv1wG w3cgO1cVLkAXOOkBW6BXifFfGONW9gLcaRwRxuh7sklXqg7eXtVm+zZunFATlTzO4NAo kfnhSA0c3kYDaadpBVHHYHxDLm3HSbZAuIv7vgVKCRM9EBd9f8KoVMetL3cYEHd8/e6+ FTl0YsT8MBM0ApkYu0Jjdhu51WDruDXK0vsBOKIJlwbirrwbbTrKGU4qR6PqbEQo1dDZ FyGVMBTxqDQhkRvBsb7FoEN+cJXcRQQ+mv4ojru8rC8ylXuTi48k+sB2qb/k/LRHM+Ue wkUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=QBCdXIHrY/tobdMXALDGrMgM1Ga6gEjW1DZkTfwKxtw=; b=uhSaZ0SR8rDHaty+wRiGw/W3BklNY9meKEOBJSPpFoW2r3yMfvpjEYESeYU3pAgcIc am6pt8WPA/ISFVOBfqPjessSgQ6EHxFKSQAhzfydaAiIIyWoGSydZV0uxI5ba1WC5jE+ cTXTzZya3tmD0rTDnhTFxNHMsndQqjhrmizDu4iLTm9A37hv7bhrkjCKiDtE/9fFYz0v zG0TvykR2w82v8modMnDiaBVKsg06JueaiWHLDjAl0sklVfYE+ooXqpm4aYUN3k7+MfK kjc9baYEneNs1IRkqWadeVqUmYXBLuqwHqFgCy09ykAzpbCkf4Z+Tki7koxLwctDAnNT w29g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=awJ3a9BX; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.53; Tue, 02 Jul 2019 12:42:53 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=awJ3a9BX; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727124AbfGBTmw (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:52 -0400 Received: from mail-lj1-f180.google.com ([209.85.208.180]:37029 "EHLO mail-lj1-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmw (ORCPT ); Tue, 2 Jul 2019 15:42:52 -0400 Received: by mail-lj1-f180.google.com with SMTP id 131so18177199ljf.4 for ; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QBCdXIHrY/tobdMXALDGrMgM1Ga6gEjW1DZkTfwKxtw=; b=awJ3a9BXCD4kKi0fCWNLbKLQ4dHYurRXSOI2PLB+TG7l44SowikrmS3utbETW0erzK miGvgr/+4GynNabx3y6prtFKPSrwhJsmlrQ7zbcNEqqRTIj4ZddMBq+o/PQg37n24WBN DxeIrlrkZmAkypEykddgnhumwgL+j343PtvV82BhiCKyioZPFsRUIlfy5NubKPG+4i4P F1TVqTvSBSBNllmzUhvTELNeeNFPEFrOdBacD0a7bavgsC5q4y/mfP7ZtADFuzb+wysC LEhJnRir03oOYDPJHlzb+JJ1xaFaO8ZikVxzOFQESUgWwfugQtKrwgpXDs8mVSb4FNY/ 1X2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QBCdXIHrY/tobdMXALDGrMgM1Ga6gEjW1DZkTfwKxtw=; b=PrIjW5mooha+Usbjkghxp+MBurpSPwUb5Tx0Uo9j8Gp33we72gFfuBckAMGkPSMUkw U3cQ+ixTpUrw8O0I5IiMQnFpApcip8duaVHrOX8YyjdfDTXaK2FQ5rO0jG7QYhiaBzid i9kboEGMobHqrEhOszVKgvJY5h87Dt1iEQ41nK8qxGNQFxa1ZazesqJsMvbDD1D83bC/ fVh8JB2xkBLED4IKt3u+MRzVyHdZszSLg0fNt5i9+mmO7GejQw98fau/z9wEAcAC7cwX E1EUmyXz55UCy9/43/HImIIZ4JLVoS+LXEATtGD+Gc/6Myyh4E3/TRJ1FnCi0K4GSJse jgHA== X-Gm-Message-State: APjAAAX8vdYObvvw3fx/jSY+3U6Nm92ql/O2IqAcvDr99YUMb3HH3snN agHTqTOsCvmLhZ4CzLHDNk5x9dzeJzJxlP6i X-Received: by 2002:a2e:9951:: with SMTP id r17mr18066950ljj.125.1562096570015; Tue, 02 Jul 2019 12:42:50 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.48 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:49 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 30/32] crypto: arm/aes-cipher - switch to shared AES inverse Sbox Date: Tue, 2 Jul 2019 21:41:48 +0200 Message-Id: <20190702194150.10405-31-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-core.S | 40 +------------------- 1 file changed, 1 insertion(+), 39 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/aes-cipher-core.S b/arch/arm/crypto/aes-cipher-core.S index f2d67c095e59..180d8555a09c 100644 --- a/arch/arm/crypto/aes-cipher-core.S +++ b/arch/arm/crypto/aes-cipher-core.S @@ -222,43 +222,5 @@ ENDPROC(__aes_arm_encrypt) .align 5 ENTRY(__aes_arm_decrypt) - do_crypt iround, crypto_it_tab, __aes_arm_inverse_sbox, 0 + do_crypt iround, crypto_it_tab, crypto_aes_inv_sbox, 0 ENDPROC(__aes_arm_decrypt) - - .section ".rodata", "a" - .align L1_CACHE_SHIFT - .type __aes_arm_inverse_sbox, %object -__aes_arm_inverse_sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - .size __aes_arm_inverse_sbox, . - __aes_arm_inverse_sbox From patchwork Tue Jul 2 19:41:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168377 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653856ilk; Tue, 2 Jul 2019 12:42:54 -0700 (PDT) X-Google-Smtp-Source: APXvYqzwOxtunMHA0BmERCtY9D9URzNDgPWQqTNirfujUZKGdeo4d7WKzZjV2msvL2Os0w9wPwbd X-Received: by 2002:a63:f807:: with SMTP id n7mr33528110pgh.119.1562096574673; Tue, 02 Jul 2019 12:42:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096574; cv=none; d=google.com; s=arc-20160816; b=rw5yEnnqJOYxjwanV2/88uWrurzaXI5k44O+i3Az87ICOU+Kuh5wEajXBczXiQ63Yd Ji1+5pmNiTgf5EHYitbqS3nsXLoqci/+3VmUF4uJSiS2BYNeMyaAydAM8mFgkNE9RuVw Oi5rbBV9qPYy2UV3gJ10yEl33GecPPhMC0KGAwZNcYkteUWjEqOtTGfIfdQI9D9IIsry yEG0HlxSyh6HMWxHZuJpDd1VSINai3rCRVDAEU7q4NbzQnOW1+Grg03Z2PSyzk5p5l2L JMtabolpQhOlBJmF9CFsuFBigqPRQS4Pri4nesAcgwkxHlozHMk5GLGTbIH0v07FXe4e xhXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=y8bW1Oz9Gva4RHYCivhqTUhkqRq1s3Ji3bb3o0vH+Kk=; b=MbxUe/xLDJrlIPUqfN+xZpex5hVToMfesrsBLWAw+8J2U/tvKMubPprD8Vq/rPZUQ1 JUBQMpFPkleC6BKqiXHLnD7JEsBDyqsgsqopkWCE+8T46YEItiodm1Z/a3DsOqyrVEVg zPKDsOhq5DY2PEl+IZ1Yi1eS3vvYtWmFQjkpbAk2f39xcxCUzV5/ZbZfgAGRDJ8F+F4J bBZd0mCX2e7K0BilgCgg9xYpW5Aeph4aM4G4tD4s+SRdqLUkmNCH8KRNb8Nby4w/W4fa Vp62RcENNrikKmduEhpHImCbfMpNGcXpiO72WlsQ00u40sXf5fDcvYd4/DxIcOVyg6XT Lyeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qB2oh01+; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.54; Tue, 02 Jul 2019 12:42:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qB2oh01+; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727121AbfGBTmy (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:54 -0400 Received: from mail-lj1-f178.google.com ([209.85.208.178]:40536 "EHLO mail-lj1-f178.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727118AbfGBTmx (ORCPT ); Tue, 2 Jul 2019 15:42:53 -0400 Received: by mail-lj1-f178.google.com with SMTP id a21so18169992ljh.7 for ; Tue, 02 Jul 2019 12:42:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=y8bW1Oz9Gva4RHYCivhqTUhkqRq1s3Ji3bb3o0vH+Kk=; b=qB2oh01+ax1n1uskOVzYmXlI6RiNJYNT5T3orVG7VOHRda+I0Xgm7sIvWKthtxymhf wuCS1H1f1izwXMeuLa1hJJqP6wpxoZ1m9UQu4b5jvc6r6NTLDrEViVl/PGCRdOfqvLfa TPKxaRGYLeZSo1wcSIMCbbUve3Vya3IcQOc3OHkAYCtIykuTFIB7YRdK6HlSipqVpjdd eXYnNAB+70VsXwlNll+WTdvhakFqxBaW5CUe7y+HaNkEajywN4TjVcHoWujzy/y5GRNJ luymSMqAdrb3uzo5luodCVZ+UX+oQhDcrEbNZJJL6qKto07risN1QYCNVFrzxdazDShx 0svQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=y8bW1Oz9Gva4RHYCivhqTUhkqRq1s3Ji3bb3o0vH+Kk=; b=kiSiqk9LI+En7pAyZ6Qv6PnYbVUuDaQCrgY096xmu1OsI8I4B5m5isHG7wpYFcbrQG wmbdVMZIHDqOZ9HORJyWlAPbfdxOyhiy4JwpGBh+PJwmFZCB3nojGIkCqWc9dWNHI14U +ziUJy990sBpe5yYXtflY3QfzlxEufgk1EEVuxjYjuRweaqxhbA0Jm1hJgCm7vmkFys2 6M2JfaPU/HcIqUqsnq6yUArQoE0I/baBChyrYOtdwA0NNKJ5hkYRhnXv7iKfTND2ldjY BrDCcYRGbF2XDlxvyMJDqMPK0f3PKmFIvAXp9vdwqpuph8cBO9a1QBrz2m+e6uM6QQBX Qf1Q== X-Gm-Message-State: APjAAAVUbUfpCIuSqI7EWlnKUsgb5D72dgKF8hyUy8Hx6W2fEU74rW30 c73BfvGlWumbZOsIIt6cldyTpSCLYwQkMK+9 X-Received: by 2002:a2e:9b84:: with SMTP id z4mr18657031lji.75.1562096571298; Tue, 02 Jul 2019 12:42:51 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.50 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:50 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 31/32] crypto: arm64/aes-cipher - switch to shared AES inverse Sbox Date: Tue, 2 Jul 2019 21:41:49 +0200 Message-Id: <20190702194150.10405-32-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/aes-cipher-core.S | 40 +------------------- 1 file changed, 1 insertion(+), 39 deletions(-) -- 2.17.1 diff --git a/arch/arm64/crypto/aes-cipher-core.S b/arch/arm64/crypto/aes-cipher-core.S index 3a44eada2347..27dac259b359 100644 --- a/arch/arm64/crypto/aes-cipher-core.S +++ b/arch/arm64/crypto/aes-cipher-core.S @@ -131,43 +131,5 @@ ENDPROC(__aes_arm64_encrypt) .align 5 ENTRY(__aes_arm64_decrypt) - do_crypt iround, crypto_it_tab, __aes_arm64_inverse_sbox, 0 + do_crypt iround, crypto_it_tab, crypto_aes_inv_sbox, 0 ENDPROC(__aes_arm64_decrypt) - - .section ".rodata", "a" - .align L1_CACHE_SHIFT - .type __aes_arm64_inverse_sbox, %object -__aes_arm64_inverse_sbox: - .byte 0x52, 0x09, 0x6a, 0xd5, 0x30, 0x36, 0xa5, 0x38 - .byte 0xbf, 0x40, 0xa3, 0x9e, 0x81, 0xf3, 0xd7, 0xfb - .byte 0x7c, 0xe3, 0x39, 0x82, 0x9b, 0x2f, 0xff, 0x87 - .byte 0x34, 0x8e, 0x43, 0x44, 0xc4, 0xde, 0xe9, 0xcb - .byte 0x54, 0x7b, 0x94, 0x32, 0xa6, 0xc2, 0x23, 0x3d - .byte 0xee, 0x4c, 0x95, 0x0b, 0x42, 0xfa, 0xc3, 0x4e - .byte 0x08, 0x2e, 0xa1, 0x66, 0x28, 0xd9, 0x24, 0xb2 - .byte 0x76, 0x5b, 0xa2, 0x49, 0x6d, 0x8b, 0xd1, 0x25 - .byte 0x72, 0xf8, 0xf6, 0x64, 0x86, 0x68, 0x98, 0x16 - .byte 0xd4, 0xa4, 0x5c, 0xcc, 0x5d, 0x65, 0xb6, 0x92 - .byte 0x6c, 0x70, 0x48, 0x50, 0xfd, 0xed, 0xb9, 0xda - .byte 0x5e, 0x15, 0x46, 0x57, 0xa7, 0x8d, 0x9d, 0x84 - .byte 0x90, 0xd8, 0xab, 0x00, 0x8c, 0xbc, 0xd3, 0x0a - .byte 0xf7, 0xe4, 0x58, 0x05, 0xb8, 0xb3, 0x45, 0x06 - .byte 0xd0, 0x2c, 0x1e, 0x8f, 0xca, 0x3f, 0x0f, 0x02 - .byte 0xc1, 0xaf, 0xbd, 0x03, 0x01, 0x13, 0x8a, 0x6b - .byte 0x3a, 0x91, 0x11, 0x41, 0x4f, 0x67, 0xdc, 0xea - .byte 0x97, 0xf2, 0xcf, 0xce, 0xf0, 0xb4, 0xe6, 0x73 - .byte 0x96, 0xac, 0x74, 0x22, 0xe7, 0xad, 0x35, 0x85 - .byte 0xe2, 0xf9, 0x37, 0xe8, 0x1c, 0x75, 0xdf, 0x6e - .byte 0x47, 0xf1, 0x1a, 0x71, 0x1d, 0x29, 0xc5, 0x89 - .byte 0x6f, 0xb7, 0x62, 0x0e, 0xaa, 0x18, 0xbe, 0x1b - .byte 0xfc, 0x56, 0x3e, 0x4b, 0xc6, 0xd2, 0x79, 0x20 - .byte 0x9a, 0xdb, 0xc0, 0xfe, 0x78, 0xcd, 0x5a, 0xf4 - .byte 0x1f, 0xdd, 0xa8, 0x33, 0x88, 0x07, 0xc7, 0x31 - .byte 0xb1, 0x12, 0x10, 0x59, 0x27, 0x80, 0xec, 0x5f - .byte 0x60, 0x51, 0x7f, 0xa9, 0x19, 0xb5, 0x4a, 0x0d - .byte 0x2d, 0xe5, 0x7a, 0x9f, 0x93, 0xc9, 0x9c, 0xef - .byte 0xa0, 0xe0, 0x3b, 0x4d, 0xae, 0x2a, 0xf5, 0xb0 - .byte 0xc8, 0xeb, 0xbb, 0x3c, 0x83, 0x53, 0x99, 0x61 - .byte 0x17, 0x2b, 0x04, 0x7e, 0xba, 0x77, 0xd6, 0x26 - .byte 0xe1, 0x69, 0x14, 0x63, 0x55, 0x21, 0x0c, 0x7d - .size __aes_arm64_inverse_sbox, . - __aes_arm64_inverse_sbox From patchwork Tue Jul 2 19:41:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 168378 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4653901ilk; Tue, 2 Jul 2019 12:42:57 -0700 (PDT) X-Google-Smtp-Source: APXvYqyai5+JZ3t0ylgq7zRAkZluq0HPDIbC9ZNyKZqO3Ec+zdkyFFZGC6c4/ETxHLoNTuH0GUpj X-Received: by 2002:a17:902:a9ca:: with SMTP id b10mr32324781plr.69.1562096577296; Tue, 02 Jul 2019 12:42:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562096577; cv=none; d=google.com; s=arc-20160816; b=Qb7jNDZ4ZQCrHx6X3xKJrTrDzKl7+4Cvmcdum0NebFx6hfjE0t/c5fqYNqZ2UP5BvZ +krtNPHn++rl/e74m2PxvuBNfGBhAMLfIANhPgdloJFh4zEuc4ZcA+hY3dQq+z3ohNIm Du6IC+uXMkqXdVC5i/owukTMG5Mkxf614Swfg12u0F+k7HIfdLlH3RlMUErDhF0EUgGI mF4srbrFkBbjhbDDRc1K6tT6zxu6W03nfpK1i0gQ/3H/mxSlXli9+DOoqzAxOimP/A90 i3j5flukmQXqfiI0fp+bG8wxKFp2vBEQeSKG8lM+F8h6PgJEiiD7p+Vu7kNPRoqEYQjv SqTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=8YyWMi1jO77As8cVPxq35RAv7TeBcYZ2R24IVOGIcG4=; b=rwsQuXs8UE49VeoOiIg4JHl/iezXE04BCoVK+FEvOSRGyxazaOKVaPU6buRRZf2Rnf rWp7TVd5TTtaVArD9HEuZRR2xAmreAwXikg6AEOhlIV2r8l6mVqUTV4qPYbULqvfi+Vs Z6DcIjycpMLNu3fQR3KacxenLE3qFzmQ4ZfAuMqYO1rUAe/9FfJ0PuKk/ql1EYMue0nG em/dlV32PmLaRxux6p/JqL7lZyjqrbBuvbU2kRy+o5nzKPCHYJ9IZ3xk0h2etHM2E5HV tpadv3VtPKupoMqt8Rb3L8ck1A0PliFkxzG4ON+R0kz8jEgceedrHrpKvLfT3uZZRf00 vC9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OGT3WCj2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f1si13733891plf.87.2019.07.02.12.42.57; Tue, 02 Jul 2019 12:42:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OGT3WCj2; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727118AbfGBTm4 (ORCPT + 3 others); Tue, 2 Jul 2019 15:42:56 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:37970 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727069AbfGBTmy (ORCPT ); Tue, 2 Jul 2019 15:42:54 -0400 Received: by mail-lj1-f194.google.com with SMTP id r9so18182002ljg.5 for ; Tue, 02 Jul 2019 12:42:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8YyWMi1jO77As8cVPxq35RAv7TeBcYZ2R24IVOGIcG4=; b=OGT3WCj2HtxECpg7NfJSpD1KmuRBU7NWq738rqhEeeKUAcPzuh4+Diky8oujOL9mj4 v7wQBrB9gMcs4OCtaBgMt4XVu1Xq4xFvqZNzyFeTL1I6pt1sTUSCOdGVKlAgny9aXQR4 QXP6c9NNP9e46o8tCroK/X6g0IIEwf9yFtiCeXs86wotXZNYjC+AbsEdJVKhiYGE6S4i h5v2inineSzvJF1S8CdrrtZZ6snzRDJ4KrduWH7nt+tffAph98tE+KhOkIhE6mb7MlQ5 NdrAO4LyMyzbUYTG9GlyKBZ35b69yl7vJ1T17XRkpG5oc/6ZzGAEkkSr2aFd/9ElbeBB vVqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8YyWMi1jO77As8cVPxq35RAv7TeBcYZ2R24IVOGIcG4=; b=uY37dpZMAjNE91kZ5yihpXMXc70meT/3DQuMxu+bVBnAodSXutCCowzwvMJfsUUea/ rUEfH1elPa8CS0jrYlNoFobH8Suop2czVQc7LTDrBJPWIuDHm6ypwsiMNW95YTOageAf oHcwhVQ17c8eXLbHNOTRyx43KPlmjpty02c4xZQtn1D3DUbLJd7VM7QPuN9yflnLKrJL N1jU4MhbeA0QT+hAuBPc1t6fWmYHkb3VIEOgh/gsB95axWMOi5DjS1AnSmKLv2cUk88u 5oW9YunyHS8fIRmrVJHEtjJ+vykqzY+gbFRcJUlXRankRISv+N7gQp9hFBhJexCFOALh pxLQ== X-Gm-Message-State: APjAAAW8Ls9cmSB/2NnEuIs83IeICX4glRaEWTWHp9SGZebusfO/i/N7 TlqIlzSxCNLI1htADOiLOWiey4zRkKCcLTc3 X-Received: by 2002:a2e:8e90:: with SMTP id z16mr18659630ljk.4.1562096572467; Tue, 02 Jul 2019 12:42:52 -0700 (PDT) Received: from e111045-lin.arm.com (89-212-78-239.static.t-2.net. [89.212.78.239]) by smtp.gmail.com with ESMTPSA id 24sm4475163ljs.63.2019.07.02.12.42.51 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 02 Jul 2019 12:42:51 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, ebiggers@google.com, Ard Biesheuvel Subject: [PATCH v4 32/32] crypto: arm/aes-scalar - unexport en/decryption routines Date: Tue, 2 Jul 2019 21:41:50 +0200 Message-Id: <20190702194150.10405-33-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702194150.10405-1-ard.biesheuvel@linaro.org> References: <20190702194150.10405-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The scalar table based AES routines are not used by other drivers, so let's keep it that way and unexport the symbols. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/aes-cipher-glue.c | 3 --- 1 file changed, 3 deletions(-) -- 2.17.1 diff --git a/arch/arm/crypto/aes-cipher-glue.c b/arch/arm/crypto/aes-cipher-glue.c index f6c07867b8ff..26a2b81c2c12 100644 --- a/arch/arm/crypto/aes-cipher-glue.c +++ b/arch/arm/crypto/aes-cipher-glue.c @@ -14,10 +14,7 @@ #include asmlinkage void __aes_arm_encrypt(u32 *rk, int rounds, const u8 *in, u8 *out); -EXPORT_SYMBOL(__aes_arm_encrypt); - asmlinkage void __aes_arm_decrypt(u32 *rk, int rounds, const u8 *in, u8 *out); -EXPORT_SYMBOL(__aes_arm_decrypt); static void aes_arm_encrypt(struct crypto_tfm *tfm, u8 *out, const u8 *in) {