From patchwork Fri Jun 21 08:09:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 167363 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp418362ilk; Fri, 21 Jun 2019 01:09:35 -0700 (PDT) X-Google-Smtp-Source: APXvYqwCBdT4rDDKeHYhxG8u72Z2vhZJ6DO7BrIV/00nDDytsB+sxQFBd18YBoC6W3t3LDfxBjZ2 X-Received: by 2002:a63:81c6:: with SMTP id t189mr4750007pgd.15.1561104575266; Fri, 21 Jun 2019 01:09:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561104575; cv=none; d=google.com; s=arc-20160816; b=f1KxVF0GtjNKP5h8aXyx2gtSk8Wn2yd+MOXcuk3c9VazGCxVPDv+lWE1eBfGPmd2Of FtrdvOluYVzvBenX9bF4H3I99J7pSth33zB9Mqpl3eAKSPP3GoGzYCJZBcGd9eFojyss Q3saDJPfPEmpjAbr9Nfr8e7hKUo9P7ruiYiD5rz5saRNygnTxGlu6h5hZEdGOIxmkG3Y doMryifBglYTTlRHlwnzSFKE+kp02VCFfvPq7TbuzK2nGsMCdfGBMNFQSkzSYezZ04kz fv1nlgZAP10zB9nigViPYYlP3i639BQom29bz2R9A3NbI/1c7xQ3LFU7JxfUp+AESyXp Q7NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:reply-to:message-id:date:subject:cc:to :from; bh=lOG3FBpjlXW8uXdI9OAtsZb26m18EZni5K9vjsg09Z0=; b=Pt28a5oXl7Pp1sScqugjizupT0m764yPIO1q2e6wfsK2PqYQc4TZ2/q8kxlXenPjHs j3+0FeoJh0uOKp7Ch/vYDp2UjiCFtbhybYGmHVZIjoVK9Lz4XZQYGjFjEylZPcNigcfJ ZrpoIBTQabIlH4EPLo8ED1U4C1pj5Xu1JxsqYSoMFKbSiLBQpxXF/GkcQDOfmSPknTrP I3HROvvhw1P0bfUlKsJdATOxPBXpHoAAu+MK9bb0NqcGyHXKDWGKGxANEKksVaooFv2l thoxgZjo6Vvd8SyIUSdJ1On6bUux0/+cTlG0JZxN9xoulKZiZFf94HNJJDx2wlWALnp6 GI7g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f63si2062053plf.224.2019.06.21.01.09.34; Fri, 21 Jun 2019 01:09:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726285AbfFUIJd (ORCPT + 3 others); Fri, 21 Jun 2019 04:09:33 -0400 Received: from foss.arm.com ([217.140.110.172]:49712 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726030AbfFUIJd (ORCPT ); Fri, 21 Jun 2019 04:09:33 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 220072B; Fri, 21 Jun 2019 01:09:32 -0700 (PDT) Received: from e111045-lin.arm.com (unknown [10.37.10.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 757403F246; Fri, 21 Jun 2019 01:09:30 -0700 (PDT) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: Ard Biesheuvel , Herbert Xu , Eric Biggers , dm-devel@redhat.com, linux-fscrypt@vger.kernel.org, Gilad Ben-Yossef , Milan Broz Subject: [PATCH v4 0/6] crypto: switch to crypto API for ESSIV generation Date: Fri, 21 Jun 2019 10:09:12 +0200 Message-Id: <20190621080918.22809-1-ard.biesheuvel@arm.com> X-Mailer: git-send-email 2.17.1 Reply-To: ard.biesheuvel@linaro.org Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Ard Biesheuvel This series creates an ESSIV template that produces a skcipher or AEAD transform based on a tuple of the form ',,' (or ',,' for the AEAD case). It exposes the encapsulated sync or async skcipher/aead by passing through all operations, while using the cipher/shash pair to transform the input IV into an ESSIV output IV. This matches what both users of ESSIV in the kernel do, and so it is proposed as a replacement for those, in patches #2 and #4. This code has been tested using the fscrypt test suggested by Eric (generic/549), as well as the mode-test script suggested by Milan for the dm-crypt case. I also tested the aead case in a virtual machine, but it definitely needs some wider testing from the dm-crypt experts. Open issues (more discussion needed): - given that hardware already exists that can perform en/decryption including ESSIV generation of a range of blocks, it would be useful to encapsulate this in the ESSIV template, and teach at least dm-crypt how to use it (given that it often processes 8 512-byte sectors at a time) - given the above, it may or may not make sense to keep the accelerated implementation in patch #6 (and teach it to increment the LE counter after each sector) Changes since v3: - address various review comments from Eric on patch #1 - use Kconfig's 'imply' instead of 'select' to permit CRYPTO_ESSIV to be enabled as a module or disabled entirely even if fscrypt is compiled in (#2) - fix an issue in the AEAD encrypt path caused by the IV being clobbered by the inner skcipher before the hmac was being calculated Changes since v2: - fixed a couple of bugs that snuck in after I'd done the bulk of my testing - some cosmetic tweaks to the ESSIV template skcipher setkey function to align it with the aead one - add a test case for essiv(cbc(aes),aes,sha256) - add an accelerated implementation for arm64 that combines the IV derivation and the actual en/decryption in a single asm routine Scroll down for tcrypt speed test result comparing the essiv template with the asm implementation. Bare cbc(aes) tests included for reference as well. Taken on a 2GHz Cortex-A57 (AMD Seattle) Code can be found here https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=essiv-v4 Cc: Herbert Xu Cc: Eric Biggers Cc: dm-devel@redhat.com Cc: linux-fscrypt@vger.kernel.org Cc: Gilad Ben-Yossef Cc: Milan Broz Ard Biesheuvel (6): crypto: essiv - create wrapper template for ESSIV generation fs: crypto: invoke crypto API for ESSIV handling md: dm-crypt: infer ESSIV block cipher from cipher string directly md: dm-crypt: switch to ESSIV crypto API template crypto: essiv - add test vector for essiv(cbc(aes),aes,sha256) crypto: arm64/aes - implement accelerated ESSIV/CBC mode arch/arm64/crypto/aes-glue.c | 129 ++++ arch/arm64/crypto/aes-modes.S | 99 +++ crypto/Kconfig | 4 + crypto/Makefile | 1 + crypto/essiv.c | 639 ++++++++++++++++++++ crypto/tcrypt.c | 9 + crypto/testmgr.c | 6 + crypto/testmgr.h | 208 +++++++ drivers/md/Kconfig | 1 + drivers/md/dm-crypt.c | 237 ++------ fs/crypto/Kconfig | 1 + fs/crypto/crypto.c | 5 - fs/crypto/fscrypt_private.h | 9 - fs/crypto/keyinfo.c | 88 +-- 14 files changed, 1141 insertions(+), 295 deletions(-) create mode 100644 crypto/essiv.c -- 2.17.1 testing speed of async essiv(cbc(aes),aes,sha256) (essiv(cbc-aes-ce,aes-ce,sha256-ce)) encryption tcrypt: test 0 (128 bit key, 16 byte blocks): 3140785 ops/s ( 50252560 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 2672908 ops/s (171066112 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 1632811 ops/s (417999616 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 665980 ops/s (681963520 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 495180 ops/s (728904960 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 99329 ops/s (813703168 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 3106888 ops/s ( 49710208 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 2582682 ops/s (165291648 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 1511160 ops/s (386856960 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 589841 ops/s (603997184 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 435094 ops/s (640458368 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 82997 ops/s (679911424 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 3058592 ops/s ( 48937472 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 2496988 ops/s (159807232 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 1438355 ops/s (368218880 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 528902 ops/s (541595648 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 387861 ops/s (570931392 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 75444 ops/s (618037248 bytes) testing speed of async essiv(cbc(aes),aes,sha256) (essiv(cbc-aes-ce,aes-ce,sha256-ce)) decryption tcrypt: test 0 (128 bit key, 16 byte blocks): 3164752 ops/s ( 50636032 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 2975874 ops/s ( 190455936 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 2393123 ops/s ( 612639488 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 1314745 ops/s (1346298880 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 1050717 ops/s (1546655424 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 246457 ops/s (2018975744 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 3117489 ops/s ( 49879824 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 2922089 ops/s ( 187013696 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 2292023 ops/s ( 586757888 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 1207942 ops/s (1236932608 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 955598 ops/s (1406640256 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 195198 ops/s (1599062016 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 3081935 ops/s ( 49310960 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 2883181 ops/s ( 184523584 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 2205147 ops/s ( 564517632 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 1119468 ops/s (1146335232 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 877017 ops/s (1290969024 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 195255 ops/s (1599528960 bytes) testing speed of async essiv(cbc(aes),aes,sha256) (essiv-cbc-aes-sha256-ce) encryption tcrypt: test 0 (128 bit key, 16 byte blocks): 5037539 ops/s ( 80600624 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 3884302 ops/s (248595328 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 2014999 ops/s (515839744 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 721147 ops/s (738454528 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 525262 ops/s (773185664 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 100453 ops/s (822910976 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 4972667 ops/s ( 79562672 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 3721788 ops/s (238194432 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 1835967 ops/s (470007552 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 633524 ops/s (648728576 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 458306 ops/s (674626432 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 83595 ops/s (684810240 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 4975101 ops/s ( 79601616 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 3581137 ops/s (229192768 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 1741799 ops/s (445900544 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 565340 ops/s (578908160 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 407040 ops/s (599162880 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 76092 ops/s (623345664 bytes) testing speed of async essiv(cbc(aes),aes,sha256) (essiv-cbc-aes-sha256-ce) decryption tcrypt: test 0 (128 bit key, 16 byte blocks): 5122947 ops/s ( 81967152 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 4546576 ops/s ( 290980864 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 3314744 ops/s ( 848574464 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 1550823 ops/s (1588042752 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 1197388 ops/s (1762555136 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 253661 ops/s (2077990912 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 5040644 ops/s ( 80650304 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 4442490 ops/s ( 284319360 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 3138199 ops/s ( 803378944 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 1406038 ops/s (1439782912 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 1075658 ops/s (1583368576 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 199652 ops/s (1635549184 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 4979432 ops/s ( 79670912 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 4394406 ops/s ( 281241984 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 2999511 ops/s ( 767874816 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 1294498 ops/s (1325565952 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 981009 ops/s (1444045248 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 200463 ops/s (1642192896 bytes) testing speed of async cbc(aes) (cbc-aes-ce) encryption tcrypt: test 0 (128 bit key, 16 byte blocks): 5895884 ops/s ( 94334144 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 4347437 ops/s (278235968 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 2135454 ops/s (546676224 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 736839 ops/s (754523136 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 533261 ops/s (784960192 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 100850 ops/s (826163200 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 5745691 ops/s ( 91931056 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 4113271 ops/s (263249344 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 1932208 ops/s (494645248 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 644555 ops/s (660024320 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 464237 ops/s (683356864 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 84019 ops/s (688283648 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 5620065 ops/s ( 89921040 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 3982991 ops/s (254911424 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 1830587 ops/s (468630272 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 576151 ops/s (589978624 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 412487 ops/s (607180864 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 76378 ops/s (625688576 bytes) testing speed of async cbc(aes) (cbc-aes-ce) decryption tcrypt: test 0 (128 bit key, 16 byte blocks): 5821314 ops/s ( 93141024 bytes) tcrypt: test 1 (128 bit key, 64 byte blocks): 5248040 ops/s ( 335874560 bytes) tcrypt: test 2 (128 bit key, 256 byte blocks): 3677701 ops/s ( 941491456 bytes) tcrypt: test 3 (128 bit key, 1024 byte blocks): 1650808 ops/s (1690427392 bytes) tcrypt: test 4 (128 bit key, 1472 byte blocks): 1256545 ops/s (1849634240 bytes) tcrypt: test 5 (128 bit key, 8192 byte blocks): 257922 ops/s (2112897024 bytes) tcrypt: test 6 (192 bit key, 16 byte blocks): 5690108 ops/s ( 91041728 bytes) tcrypt: test 7 (192 bit key, 64 byte blocks): 5086441 ops/s ( 325532224 bytes) tcrypt: test 8 (192 bit key, 256 byte blocks): 3447562 ops/s ( 882575872 bytes) tcrypt: test 9 (192 bit key, 1024 byte blocks): 1490136 ops/s (1525899264 bytes) tcrypt: test 10 (192 bit key, 1472 byte blocks): 1124620 ops/s (1655440640 bytes) tcrypt: test 11 (192 bit key, 8192 byte blocks): 201222 ops/s (1648410624 bytes) tcrypt: test 12 (256 bit key, 16 byte blocks): 5567247 ops/s ( 89075952 bytes) tcrypt: test 13 (256 bit key, 64 byte blocks): 5050010 ops/s ( 323200640 bytes) tcrypt: test 14 (256 bit key, 256 byte blocks): 3290422 ops/s ( 842348032 bytes) tcrypt: test 15 (256 bit key, 1024 byte blocks): 1359439 ops/s (1392065536 bytes) tcrypt: test 16 (256 bit key, 1472 byte blocks): 1017751 ops/s (1498129472 bytes) tcrypt: test 17 (256 bit key, 8192 byte blocks): 201492 ops/s (1650622464 bytes)