[v3,0/6] crypto: switch to crypto API for ESSIV generation

This series creates an ESSIV template that produces a skcipher or AEAD
transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'
(or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the
encapsulated sync or async skcipher/aead by passing through all operations,
while using the cipher/shash pair to transform the input IV into an ESSIV
output IV.

This matches what both users of ESSIV in the kernel do, and so it is proposed
as a replacement for those, in patches #2 and #4.

This code has been tested using the fscrypt test suggested by Eric
(generic/549), as well as the mode-test script suggested by Milan for
the dm-crypt case. I also tested the aead case in a virtual machine,
but it definitely needs some wider testing from the dm-crypt experts.

Changes since v2:
- fixed a couple of bugs that snuck in after I'd done the bulk of my
  testing
- some cosmetic tweaks to the ESSIV template skcipher setkey function
  to align it with the aead one
- add a test case for essiv(cbc(aes),aes,sha256)
- add an accelerated implementation for arm64 that combines the IV
  derivation and the actual en/decryption in a single asm routine

Scroll down for tcrypt speed test result comparing the essiv template
with the asm implementation. Bare cbc(aes) tests included for reference
as well. Taken on a 2GHz Cortex-A57 (AMD Seattle)

Code can be found here
https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=essiv-v3

Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Eric Biggers <ebiggers@google.com>
Cc: dm-devel@redhat.com
Cc: linux-fscrypt@vger.kernel.org
Cc: Gilad Ben-Yossef <gilad@benyossef.com>
Cc: Milan Broz <gmazyland@gmail.com>

Ard Biesheuvel (6):
  crypto: essiv - create wrapper template for ESSIV generation
  fs: crypto: invoke crypto API for ESSIV handling
  md: dm-crypt: infer ESSIV block cipher from cipher string directly
  md: dm-crypt: switch to ESSIV crypto API template
  crypto: essiv - add test vector for essiv(cbc(aes),aes,sha256)
  crypto: arm64/aes - implement accelerated ESSIV/CBC mode

 arch/arm64/crypto/aes-glue.c  | 129 ++++
 arch/arm64/crypto/aes-modes.S |  99 +++
 crypto/Kconfig                |   4 +
 crypto/Makefile               |   1 +
 crypto/essiv.c                | 630 ++++++++++++++++++++
 crypto/tcrypt.c               |   9 +
 crypto/testmgr.c              |   6 +
 crypto/testmgr.h              | 208 +++++++
 drivers/md/Kconfig            |   1 +
 drivers/md/dm-crypt.c         | 237 ++------
 fs/crypto/Kconfig             |   1 +
 fs/crypto/crypto.c            |   5 -
 fs/crypto/fscrypt_private.h   |   9 -
 fs/crypto/keyinfo.c           |  88 +--
 14 files changed, 1132 insertions(+), 295 deletions(-)
 create mode 100644 crypto/essiv.c

-- 
2.20.1

testing speed of async essiv(cbc(aes),aes,sha256) (essiv(cbc-aes-ce,aes-ce,sha256-ce)) encryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 3140785 ops/s ( 50252560 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 2672908 ops/s (171066112 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 1632811 ops/s (417999616 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks):  665980 ops/s (681963520 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks):  495180 ops/s (728904960 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):   99329 ops/s (813703168 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 3106888 ops/s ( 49710208 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 2582682 ops/s (165291648 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 1511160 ops/s (386856960 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks):  589841 ops/s (603997184 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks):  435094 ops/s (640458368 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):   82997 ops/s (679911424 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 3058592 ops/s ( 48937472 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 2496988 ops/s (159807232 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 1438355 ops/s (368218880 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks):  528902 ops/s (541595648 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks):  387861 ops/s (570931392 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):   75444 ops/s (618037248 bytes)

testing speed of async essiv(cbc(aes),aes,sha256) (essiv(cbc-aes-ce,aes-ce,sha256-ce)) decryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 3164752 ops/s (  50636032 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 2975874 ops/s ( 190455936 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 2393123 ops/s ( 612639488 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks): 1314745 ops/s (1346298880 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks): 1050717 ops/s (1546655424 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):  246457 ops/s (2018975744 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 3117489 ops/s (  49879824 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 2922089 ops/s ( 187013696 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 2292023 ops/s ( 586757888 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks): 1207942 ops/s (1236932608 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks):  955598 ops/s (1406640256 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):  195198 ops/s (1599062016 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 3081935 ops/s (  49310960 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 2883181 ops/s ( 184523584 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 2205147 ops/s ( 564517632 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks): 1119468 ops/s (1146335232 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks):  877017 ops/s (1290969024 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):  195255 ops/s (1599528960 bytes)


testing speed of async essiv(cbc(aes),aes,sha256) (essiv-cbc-aes-sha256-ce) encryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 5037539 ops/s ( 80600624 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 3884302 ops/s (248595328 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 2014999 ops/s (515839744 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks):  721147 ops/s (738454528 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks):  525262 ops/s (773185664 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):  100453 ops/s (822910976 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 4972667 ops/s ( 79562672 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 3721788 ops/s (238194432 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 1835967 ops/s (470007552 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks):  633524 ops/s (648728576 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks):  458306 ops/s (674626432 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):   83595 ops/s (684810240 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 4975101 ops/s ( 79601616 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 3581137 ops/s (229192768 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 1741799 ops/s (445900544 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks):  565340 ops/s (578908160 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks):  407040 ops/s (599162880 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):   76092 ops/s (623345664 bytes)

testing speed of async essiv(cbc(aes),aes,sha256) (essiv-cbc-aes-sha256-ce) decryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 5122947 ops/s (  81967152 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 4546576 ops/s ( 290980864 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 3314744 ops/s ( 848574464 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks): 1550823 ops/s (1588042752 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks): 1197388 ops/s (1762555136 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):  253661 ops/s (2077990912 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 5040644 ops/s (  80650304 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 4442490 ops/s ( 284319360 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 3138199 ops/s ( 803378944 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks): 1406038 ops/s (1439782912 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks): 1075658 ops/s (1583368576 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):  199652 ops/s (1635549184 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 4979432 ops/s (  79670912 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 4394406 ops/s ( 281241984 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 2999511 ops/s ( 767874816 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks): 1294498 ops/s (1325565952 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks):  981009 ops/s (1444045248 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):  200463 ops/s (1642192896 bytes)

testing speed of async cbc(aes) (cbc-aes-ce) encryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 5895884 ops/s ( 94334144 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 4347437 ops/s (278235968 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 2135454 ops/s (546676224 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks):  736839 ops/s (754523136 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks):  533261 ops/s (784960192 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):  100850 ops/s (826163200 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 5745691 ops/s ( 91931056 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 4113271 ops/s (263249344 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 1932208 ops/s (494645248 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks):  644555 ops/s (660024320 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks):  464237 ops/s (683356864 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):   84019 ops/s (688283648 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 5620065 ops/s ( 89921040 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 3982991 ops/s (254911424 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 1830587 ops/s (468630272 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks):  576151 ops/s (589978624 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks):  412487 ops/s (607180864 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):   76378 ops/s (625688576 bytes)

testing speed of async cbc(aes) (cbc-aes-ce) decryption
tcrypt: test  0 (128 bit key,   16 byte blocks): 5821314 ops/s (  93141024 bytes)
tcrypt: test  1 (128 bit key,   64 byte blocks): 5248040 ops/s ( 335874560 bytes)
tcrypt: test  2 (128 bit key,  256 byte blocks): 3677701 ops/s ( 941491456 bytes)
tcrypt: test  3 (128 bit key, 1024 byte blocks): 1650808 ops/s (1690427392 bytes)
tcrypt: test  4 (128 bit key, 1472 byte blocks): 1256545 ops/s (1849634240 bytes)
tcrypt: test  5 (128 bit key, 8192 byte blocks):  257922 ops/s (2112897024 bytes)
tcrypt: test  6 (192 bit key,   16 byte blocks): 5690108 ops/s (  91041728 bytes)
tcrypt: test  7 (192 bit key,   64 byte blocks): 5086441 ops/s ( 325532224 bytes)
tcrypt: test  8 (192 bit key,  256 byte blocks): 3447562 ops/s ( 882575872 bytes)
tcrypt: test  9 (192 bit key, 1024 byte blocks): 1490136 ops/s (1525899264 bytes)
tcrypt: test 10 (192 bit key, 1472 byte blocks): 1124620 ops/s (1655440640 bytes)
tcrypt: test 11 (192 bit key, 8192 byte blocks):  201222 ops/s (1648410624 bytes)
tcrypt: test 12 (256 bit key,   16 byte blocks): 5567247 ops/s (  89075952 bytes)
tcrypt: test 13 (256 bit key,   64 byte blocks): 5050010 ops/s ( 323200640 bytes)
tcrypt: test 14 (256 bit key,  256 byte blocks): 3290422 ops/s ( 842348032 bytes)
tcrypt: test 15 (256 bit key, 1024 byte blocks): 1359439 ops/s (1392065536 bytes)
tcrypt: test 16 (256 bit key, 1472 byte blocks): 1017751 ops/s (1498129472 bytes)
tcrypt: test 17 (256 bit key, 8192 byte blocks):  201492 ops/s (1650622464 bytes)

On Wed, Jun 19, 2019 at 7:29 PM Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
>

> This series creates an ESSIV template that produces a skcipher or AEAD

> transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

> (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

> encapsulated sync or async skcipher/aead by passing through all operations,

> while using the cipher/shash pair to transform the input IV into an ESSIV

> output IV.

>

> This matches what both users of ESSIV in the kernel do, and so it is proposed

> as a replacement for those, in patches #2 and #4.

>

> This code has been tested using the fscrypt test suggested by Eric

> (generic/549), as well as the mode-test script suggested by Milan for

> the dm-crypt case. I also tested the aead case in a virtual machine,

> but it definitely needs some wider testing from the dm-crypt experts.

>

> Changes since v2:

> - fixed a couple of bugs that snuck in after I'd done the bulk of my

>   testing

> - some cosmetic tweaks to the ESSIV template skcipher setkey function

>   to align it with the aead one

> - add a test case for essiv(cbc(aes),aes,sha256)

> - add an accelerated implementation for arm64 that combines the IV

>   derivation and the actual en/decryption in a single asm routine

>

> Scroll down for tcrypt speed test result comparing the essiv template

> with the asm implementation. Bare cbc(aes) tests included for reference

> as well. Taken on a 2GHz Cortex-A57 (AMD Seattle)

>

> Code can be found here

> https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=essiv-v3



Thank you Ard for this work. It is very useful. I am testing this now
with the essiv implementation inside CryptoCell.

One possible future optimization this opens the door for is having the
template auto-increment the sector number.

This will allow the device manager or fscrypt code to ask for crypto
services on buffer spanning over a single sector size
and have the crypto code automatically increment the sector number
when processing the buffer.

This may potentially shave a few cycles because it can potentially
turn multiple calls into the crypto API in one, giving
the crypto code a larger buffer to work on.

This is actually supported by CryptoCell hardware and to the best of
my knowledge also by a similar HW from Qualcomm
via out-of-tree patches found in the Android tree.

If this makes sense to you perhaps it is a good idea to have the
template format be:

<skcipher>,<cipher>,<shash>, <sector size>

Where for now we will only support a sector size of '0' (i.e. do not
auto-increment) and later extend or am I over engineering? :-)

Thanks,
Gilad


-- 
Gilad Ben-Yossef
Chief Coffee Drinker

values of β will give rise to dom!

On 19/06/2019 18:29, Ard Biesheuvel wrote:
> This series creates an ESSIV template that produces a skcipher or AEAD

> transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

> (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

> encapsulated sync or async skcipher/aead by passing through all operations,

> while using the cipher/shash pair to transform the input IV into an ESSIV

> output IV.

> 

> This matches what both users of ESSIV in the kernel do, and so it is proposed

> as a replacement for those, in patches #2 and #4.

> 

> This code has been tested using the fscrypt test suggested by Eric

> (generic/549), as well as the mode-test script suggested by Milan for

> the dm-crypt case. I also tested the aead case in a virtual machine,

> but it definitely needs some wider testing from the dm-crypt experts.

> 

> Changes since v2:

> - fixed a couple of bugs that snuck in after I'd done the bulk of my

>   testing

> - some cosmetic tweaks to the ESSIV template skcipher setkey function

>   to align it with the aead one

> - add a test case for essiv(cbc(aes),aes,sha256)

> - add an accelerated implementation for arm64 that combines the IV

>   derivation and the actual en/decryption in a single asm routine


I run tests for the whole patchset, including some older scripts and seems
it works for dm-crypt now.

For the new CRYPTO_ESSIV option - dm-crypt must unconditionally
select it (we rely on all IV generators availability in userspace),
but that's already done in patch 4.

Thanks,
Milan

On Thu, 20 Jun 2019 at 13:22, Milan Broz <gmazyland@gmail.com> wrote:
>

> On 19/06/2019 18:29, Ard Biesheuvel wrote:

> > This series creates an ESSIV template that produces a skcipher or AEAD

> > transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

> > (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

> > encapsulated sync or async skcipher/aead by passing through all operations,

> > while using the cipher/shash pair to transform the input IV into an ESSIV

> > output IV.

> >

> > This matches what both users of ESSIV in the kernel do, and so it is proposed

> > as a replacement for those, in patches #2 and #4.

> >

> > This code has been tested using the fscrypt test suggested by Eric

> > (generic/549), as well as the mode-test script suggested by Milan for

> > the dm-crypt case. I also tested the aead case in a virtual machine,

> > but it definitely needs some wider testing from the dm-crypt experts.

> >

> > Changes since v2:

> > - fixed a couple of bugs that snuck in after I'd done the bulk of my

> >   testing

> > - some cosmetic tweaks to the ESSIV template skcipher setkey function

> >   to align it with the aead one

> > - add a test case for essiv(cbc(aes),aes,sha256)

> > - add an accelerated implementation for arm64 that combines the IV

> >   derivation and the actual en/decryption in a single asm routine

>

> I run tests for the whole patchset, including some older scripts and seems

> it works for dm-crypt now.

>


Thanks Milan, that is really helpful.

Does this include configurations that combine authenc with essiv?

> For the new CRYPTO_ESSIV option - dm-crypt must unconditionally

> select it (we rely on all IV generators availability in userspace),

> but that's already done in patch 4.

>


Indeed.

On 20/06/2019 13:54, Ard Biesheuvel wrote:
> On Thu, 20 Jun 2019 at 13:22, Milan Broz <gmazyland@gmail.com> wrote:

>>

>> On 19/06/2019 18:29, Ard Biesheuvel wrote:

>>> This series creates an ESSIV template that produces a skcipher or AEAD

>>> transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

>>> (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

>>> encapsulated sync or async skcipher/aead by passing through all operations,

>>> while using the cipher/shash pair to transform the input IV into an ESSIV

>>> output IV.

>>>

>>> This matches what both users of ESSIV in the kernel do, and so it is proposed

>>> as a replacement for those, in patches #2 and #4.

>>>

>>> This code has been tested using the fscrypt test suggested by Eric

>>> (generic/549), as well as the mode-test script suggested by Milan for

>>> the dm-crypt case. I also tested the aead case in a virtual machine,

>>> but it definitely needs some wider testing from the dm-crypt experts.

>>>

>>> Changes since v2:

>>> - fixed a couple of bugs that snuck in after I'd done the bulk of my

>>>   testing

>>> - some cosmetic tweaks to the ESSIV template skcipher setkey function

>>>   to align it with the aead one

>>> - add a test case for essiv(cbc(aes),aes,sha256)

>>> - add an accelerated implementation for arm64 that combines the IV

>>>   derivation and the actual en/decryption in a single asm routine

>>

>> I run tests for the whole patchset, including some older scripts and seems

>> it works for dm-crypt now.

>>

> 

> Thanks Milan, that is really helpful.

> 

> Does this include configurations that combine authenc with essiv?


Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

I also used this older test
https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

(just aes-gcm-random need to be commented out, we never supported this format, it was
written for some devel version)

But seems ESSIV is there tested only without AEAD composition...

So yes, this AEAD part need more testing.

Milan

On 20/06/2019 14:09, Milan Broz wrote:
> On 20/06/2019 13:54, Ard Biesheuvel wrote:

>> On Thu, 20 Jun 2019 at 13:22, Milan Broz <gmazyland@gmail.com> wrote:

>>>

>>> On 19/06/2019 18:29, Ard Biesheuvel wrote:

>>>> This series creates an ESSIV template that produces a skcipher or AEAD

>>>> transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

>>>> (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

>>>> encapsulated sync or async skcipher/aead by passing through all operations,

>>>> while using the cipher/shash pair to transform the input IV into an ESSIV

>>>> output IV.

>>>>

>>>> This matches what both users of ESSIV in the kernel do, and so it is proposed

>>>> as a replacement for those, in patches #2 and #4.

>>>>

>>>> This code has been tested using the fscrypt test suggested by Eric

>>>> (generic/549), as well as the mode-test script suggested by Milan for

>>>> the dm-crypt case. I also tested the aead case in a virtual machine,

>>>> but it definitely needs some wider testing from the dm-crypt experts.

>>>>

>>>> Changes since v2:

>>>> - fixed a couple of bugs that snuck in after I'd done the bulk of my

>>>>   testing

>>>> - some cosmetic tweaks to the ESSIV template skcipher setkey function

>>>>   to align it with the aead one

>>>> - add a test case for essiv(cbc(aes),aes,sha256)

>>>> - add an accelerated implementation for arm64 that combines the IV

>>>>   derivation and the actual en/decryption in a single asm routine

>>>

>>> I run tests for the whole patchset, including some older scripts and seems

>>> it works for dm-crypt now.

>>>

>>

>> Thanks Milan, that is really helpful.

>>

>> Does this include configurations that combine authenc with essiv?

> 

> Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

> 

> I also used this older test

> https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

> 

> (just aes-gcm-random need to be commented out, we never supported this format, it was

> written for some devel version)

> 

> But seems ESSIV is there tested only without AEAD composition...

> 

> So yes, this AEAD part need more testing.


And unfortunately it does not work - it returns EIO on sectors where it should not be data corruption.

I added few lines with length-preserving mode with ESSIV + AEAD, please could you run luks2-integrity-test
in cryptsetup upstream?

This patch adds the tests:
https://gitlab.com/cryptsetup/cryptsetup/commit/4c74ff5e5ae328cb61b44bf99f98d08ffee3366a

It is ok on mainline kernel, fails with the patchset:

# ./luks2-integrity-test 
[aes-cbc-essiv:sha256:hmac-sha256:128:512][FORMAT][ACTIVATE]sha256sum: /dev/mapper/dmi_test: Input/output error
[FAIL]
 Expecting ee501705a084cd0ab6f4a28014bcf62b8bfa3434de00b82743c50b3abf06232c got .

FAILED backtrace:
77 ./luks2-integrity-test
112 intformat ./luks2-integrity-test
127 main ./luks2-integrity-test

Milan

On Thu, 20 Jun 2019 at 15:14, Milan Broz <gmazyland@gmail.com> wrote:
>

> On 20/06/2019 14:09, Milan Broz wrote:

> > On 20/06/2019 13:54, Ard Biesheuvel wrote:

> >> On Thu, 20 Jun 2019 at 13:22, Milan Broz <gmazyland@gmail.com> wrote:

> >>>

> >>> On 19/06/2019 18:29, Ard Biesheuvel wrote:

> >>>> This series creates an ESSIV template that produces a skcipher or AEAD

> >>>> transform based on a tuple of the form '<skcipher>,<cipher>,<shash>'

> >>>> (or '<aead>,<cipher>,<shash>' for the AEAD case). It exposes the

> >>>> encapsulated sync or async skcipher/aead by passing through all operations,

> >>>> while using the cipher/shash pair to transform the input IV into an ESSIV

> >>>> output IV.

> >>>>

> >>>> This matches what both users of ESSIV in the kernel do, and so it is proposed

> >>>> as a replacement for those, in patches #2 and #4.

> >>>>

> >>>> This code has been tested using the fscrypt test suggested by Eric

> >>>> (generic/549), as well as the mode-test script suggested by Milan for

> >>>> the dm-crypt case. I also tested the aead case in a virtual machine,

> >>>> but it definitely needs some wider testing from the dm-crypt experts.

> >>>>

> >>>> Changes since v2:

> >>>> - fixed a couple of bugs that snuck in after I'd done the bulk of my

> >>>>   testing

> >>>> - some cosmetic tweaks to the ESSIV template skcipher setkey function

> >>>>   to align it with the aead one

> >>>> - add a test case for essiv(cbc(aes),aes,sha256)

> >>>> - add an accelerated implementation for arm64 that combines the IV

> >>>>   derivation and the actual en/decryption in a single asm routine

> >>>

> >>> I run tests for the whole patchset, including some older scripts and seems

> >>> it works for dm-crypt now.

> >>>

> >>

> >> Thanks Milan, that is really helpful.

> >>

> >> Does this include configurations that combine authenc with essiv?

> >

> > Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

> >

> > I also used this older test

> > https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

> >

> > (just aes-gcm-random need to be commented out, we never supported this format, it was

> > written for some devel version)

> >

> > But seems ESSIV is there tested only without AEAD composition...

> >

> > So yes, this AEAD part need more testing.

>

> And unfortunately it does not work - it returns EIO on sectors where it should not be data corruption.

>

> I added few lines with length-preserving mode with ESSIV + AEAD, please could you run luks2-integrity-test

> in cryptsetup upstream?

>

> This patch adds the tests:

> https://gitlab.com/cryptsetup/cryptsetup/commit/4c74ff5e5ae328cb61b44bf99f98d08ffee3366a

>

> It is ok on mainline kernel, fails with the patchset:

>

> # ./luks2-integrity-test

> [aes-cbc-essiv:sha256:hmac-sha256:128:512][FORMAT][ACTIVATE]sha256sum: /dev/mapper/dmi_test: Input/output error

> [FAIL]

>  Expecting ee501705a084cd0ab6f4a28014bcf62b8bfa3434de00b82743c50b3abf06232c got .

>

> FAILED backtrace:

> 77 ./luks2-integrity-test

> 112 intformat ./luks2-integrity-test

> 127 main ./luks2-integrity-test

>


OK, I will investigate.

I did my testing in a VM using a volume that was created using a
distro kernel, and mounted and used it using a kernel with these
changes applied.

Likewise, if I take a working key.img and mode-test.img, i can mount
it and use it on the system running these patches.

I noticed that this test uses algif_skcipher not algif_aead when it
formats the volume, and so I wonder if the way userland creates the
image is affected by this?

On 20/06/2019 15:52, Ard Biesheuvel wrote:
>>>> Does this include configurations that combine authenc with essiv?

>>>

>>> Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

>>>

>>> I also used this older test

>>> https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

>>>

>>> (just aes-gcm-random need to be commented out, we never supported this format, it was

>>> written for some devel version)

>>>

>>> But seems ESSIV is there tested only without AEAD composition...

>>>

>>> So yes, this AEAD part need more testing.

>>

>> And unfortunately it does not work - it returns EIO on sectors where it should not be data corruption.

>>

>> I added few lines with length-preserving mode with ESSIV + AEAD, please could you run luks2-integrity-test

>> in cryptsetup upstream?

>>

>> This patch adds the tests:

>> https://gitlab.com/cryptsetup/cryptsetup/commit/4c74ff5e5ae328cb61b44bf99f98d08ffee3366a

>>

>> It is ok on mainline kernel, fails with the patchset:

>>

>> # ./luks2-integrity-test

>> [aes-cbc-essiv:sha256:hmac-sha256:128:512][FORMAT][ACTIVATE]sha256sum: /dev/mapper/dmi_test: Input/output error

>> [FAIL]

>>  Expecting ee501705a084cd0ab6f4a28014bcf62b8bfa3434de00b82743c50b3abf06232c got .

>>

>> FAILED backtrace:

>> 77 ./luks2-integrity-test

>> 112 intformat ./luks2-integrity-test

>> 127 main ./luks2-integrity-test

>>

> 

> OK, I will investigate.

> 

> I did my testing in a VM using a volume that was created using a

> distro kernel, and mounted and used it using a kernel with these

> changes applied.

> 

> Likewise, if I take a working key.img and mode-test.img, i can mount

> it and use it on the system running these patches.

> 

> I noticed that this test uses algif_skcipher not algif_aead when it

> formats the volume, and so I wonder if the way userland creates the

> image is affected by this?


Not sure if I understand the question, but I do not think userspace even touch data area here
(except direct-io wiping after the format, but it does not read it back).

It only encrypts keyslots - and here we cannot use AEAD (in fact it is already
authenticated by a LUKS digest).

So if the data area uses AEAD (or composition of length-preserving mode and
some authentication tag like HMAC), we fallback to non-AEAD for keyslot encryption.

In short, to test it, you need to activate device (that works ok with your patches)
and *access* the data, testing LUKS format and just keyslot access will never use AEAD.

So init the data by direct-io writes, and try to read them back (with dd).

For testing data on dm-integrity (or dm-crypt with AEAD encryption stacked oved dm-integrity)
I used small utility, maybe it could be useful https://github.com/mbroz/dm_int_tools 

Milan

On Fri, 21 Jun 2019 at 09:01, Milan Broz <gmazyland@gmail.com> wrote:
>

> On 20/06/2019 15:52, Ard Biesheuvel wrote:

> >>>> Does this include configurations that combine authenc with essiv?

> >>>

> >>> Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

> >>>

> >>> I also used this older test

> >>> https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

> >>>

> >>> (just aes-gcm-random need to be commented out, we never supported this format, it was

> >>> written for some devel version)

> >>>

> >>> But seems ESSIV is there tested only without AEAD composition...

> >>>

> >>> So yes, this AEAD part need more testing.

> >>

> >> And unfortunately it does not work - it returns EIO on sectors where it should not be data corruption.

> >>

> >> I added few lines with length-preserving mode with ESSIV + AEAD, please could you run luks2-integrity-test

> >> in cryptsetup upstream?

> >>

> >> This patch adds the tests:

> >> https://gitlab.com/cryptsetup/cryptsetup/commit/4c74ff5e5ae328cb61b44bf99f98d08ffee3366a

> >>

> >> It is ok on mainline kernel, fails with the patchset:

> >>

> >> # ./luks2-integrity-test

> >> [aes-cbc-essiv:sha256:hmac-sha256:128:512][FORMAT][ACTIVATE]sha256sum: /dev/mapper/dmi_test: Input/output error

> >> [FAIL]

> >>  Expecting ee501705a084cd0ab6f4a28014bcf62b8bfa3434de00b82743c50b3abf06232c got .

> >>

> >> FAILED backtrace:

> >> 77 ./luks2-integrity-test

> >> 112 intformat ./luks2-integrity-test

> >> 127 main ./luks2-integrity-test

> >>

> >

> > OK, I will investigate.

> >

> > I did my testing in a VM using a volume that was created using a

> > distro kernel, and mounted and used it using a kernel with these

> > changes applied.

> >

> > Likewise, if I take a working key.img and mode-test.img, i can mount

> > it and use it on the system running these patches.

> >

> > I noticed that this test uses algif_skcipher not algif_aead when it

> > formats the volume, and so I wonder if the way userland creates the

> > image is affected by this?

>

> Not sure if I understand the question, but I do not think userspace even touch data area here

> (except direct-io wiping after the format, but it does not read it back).

>

> It only encrypts keyslots - and here we cannot use AEAD (in fact it is already

> authenticated by a LUKS digest).

>

> So if the data area uses AEAD (or composition of length-preserving mode and

> some authentication tag like HMAC), we fallback to non-AEAD for keyslot encryption.

>

> In short, to test it, you need to activate device (that works ok with your patches)

> and *access* the data, testing LUKS format and just keyslot access will never use AEAD.

>

> So init the data by direct-io writes, and try to read them back (with dd).

>

> For testing data on dm-integrity (or dm-crypt with AEAD encryption stacked oved dm-integrity)

> I used small utility, maybe it could be useful https://github.com/mbroz/dm_int_tools

>


Thanks.

It appears that my code generates the wrong authentication tags on
encryption, but on decryption it works fine.
I'll keep digging ...

On Fri, 21 Jun 2019 at 09:06, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>

> On Fri, 21 Jun 2019 at 09:01, Milan Broz <gmazyland@gmail.com> wrote:

> >

> > On 20/06/2019 15:52, Ard Biesheuvel wrote:

> > >>>> Does this include configurations that combine authenc with essiv?

> > >>>

> > >>> Hm, seems that we are missing these in luks2-integrity-test. I'll add them there.

> > >>>

> > >>> I also used this older test

> > >>> https://gitlab.com/omos/dm-crypt-test-scripts/blob/master/root/test_dmintegrity.sh

> > >>>

> > >>> (just aes-gcm-random need to be commented out, we never supported this format, it was

> > >>> written for some devel version)

> > >>>

> > >>> But seems ESSIV is there tested only without AEAD composition...

> > >>>

> > >>> So yes, this AEAD part need more testing.

> > >>

> > >> And unfortunately it does not work - it returns EIO on sectors where it should not be data corruption.

> > >>

> > >> I added few lines with length-preserving mode with ESSIV + AEAD, please could you run luks2-integrity-test

> > >> in cryptsetup upstream?

> > >>

> > >> This patch adds the tests:

> > >> https://gitlab.com/cryptsetup/cryptsetup/commit/4c74ff5e5ae328cb61b44bf99f98d08ffee3366a

> > >>

> > >> It is ok on mainline kernel, fails with the patchset:

> > >>

> > >> # ./luks2-integrity-test

> > >> [aes-cbc-essiv:sha256:hmac-sha256:128:512][FORMAT][ACTIVATE]sha256sum: /dev/mapper/dmi_test: Input/output error

> > >> [FAIL]

> > >>  Expecting ee501705a084cd0ab6f4a28014bcf62b8bfa3434de00b82743c50b3abf06232c got .

> > >>

> > >> FAILED backtrace:

> > >> 77 ./luks2-integrity-test

> > >> 112 intformat ./luks2-integrity-test

> > >> 127 main ./luks2-integrity-test

> > >>

> > >

> > > OK, I will investigate.

> > >

> > > I did my testing in a VM using a volume that was created using a

> > > distro kernel, and mounted and used it using a kernel with these

> > > changes applied.

> > >

> > > Likewise, if I take a working key.img and mode-test.img, i can mount

> > > it and use it on the system running these patches.

> > >

> > > I noticed that this test uses algif_skcipher not algif_aead when it

> > > formats the volume, and so I wonder if the way userland creates the

> > > image is affected by this?

> >

> > Not sure if I understand the question, but I do not think userspace even touch data area here

> > (except direct-io wiping after the format, but it does not read it back).

> >

> > It only encrypts keyslots - and here we cannot use AEAD (in fact it is already

> > authenticated by a LUKS digest).

> >

> > So if the data area uses AEAD (or composition of length-preserving mode and

> > some authentication tag like HMAC), we fallback to non-AEAD for keyslot encryption.

> >

> > In short, to test it, you need to activate device (that works ok with your patches)

> > and *access* the data, testing LUKS format and just keyslot access will never use AEAD.

> >

> > So init the data by direct-io writes, and try to read them back (with dd).

> >

> > For testing data on dm-integrity (or dm-crypt with AEAD encryption stacked oved dm-integrity)

> > I used small utility, maybe it could be useful https://github.com/mbroz/dm_int_tools

> >

>

> Thanks.

>

> It appears that my code generates the wrong authentication tags on

> encryption, but on decryption it works fine.

> I'll keep digging ...


OK, mystery solved.

The skcipher inside authenc() was corrupting the IV before the hmac
got a chance to read it.

I'll send out an updated version of the series.