From patchwork Wed Jun 1 18:26:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Loic Poulain X-Patchwork-Id: 577823 Delivered-To: patch@linaro.org Received: by 2002:a05:7000:8f0c:0:0:0:0 with SMTP id mq12csp856327mab; Wed, 1 Jun 2022 11:27:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyuueVzuwndslPItQ+l6o/9jzleWsqCNNVZpQGIff3S5j1QL9AiNwYtH+hlVcApMwOuBw59 X-Received: by 2002:a05:6638:16d6:b0:32e:1d23:da71 with SMTP id g22-20020a05663816d600b0032e1d23da71mr859892jat.195.1654108064042; Wed, 01 Jun 2022 11:27:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1654108064; cv=none; d=google.com; s=arc-20160816; b=ClbfBbQ+bFQcTpeWhiZjonUWZnXOpXqNC4JUnpd+Gz9IoqYT/e5mQUSeq99cmzsdu2 FHI6SbB5FI5+zBQ3f/HK9jOWEDN6Nf1nvLaPlAxRkgSLzLkK0cdwoSl0TKX36fYrnxUq KcAWAlZr5o6NBiPfvBIz3x3UcU74cD30JQ/SP0FixFW8Lx62s2hPQLDyusH9YXJts2vj cK6OMZ97slwS/gsHpHSE241EK2A0fy9+zl1ajfxaQIDIzjJ2pGziqOiwjSc+wZtiPy/b GmbhhuQXrQ9KwfDA8ot6KCU4KkrhekLh0/oGtZupJ282cry+Ye01BrhGqAOWIVWbKQtE 0HWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=IqoggFa0bpk378ZDhWjg79ElSHZ5vkFsTdicvgfmxYI=; b=qCaqRYBnNoBJymN4hdpuU7wf9Jng9mUg0ngYzHMSKbSDVuAvpTpD9VYOWF2YQ0k1ff DONcFn7OQTw09gtRiSYyvQ9BvtY7jOJ6djBggM5ugP6PfTFLnlr1glvpVOzpkMZYcjwu q2LCoT0uUpyzaODQ0MUiTJPTM95/3Vbd3/mCRMzPrrFO3VwZs7Tsmnu80MeIaYehxIVB IIX587Ilg9bDj9Ye47rbIpGLZ6R1P6M44D10Qb/UDpZ+aBoaPbzcN71YF8u82rd8ahac X+/D6sXabAKhdkPDR1YXQmXwuy1JKoWXYbwTHznbooWGso2cGuFcwJzoAX/3qO/0G2LZ aJfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HR2k8bSM; spf=pass (google.com: domain of u-boot-bounces@lists.denx.de designates 85.214.62.61 as permitted sender) smtp.mailfrom=u-boot-bounces@lists.denx.de; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from phobos.denx.de (phobos.denx.de. [85.214.62.61]) by mx.google.com with ESMTPS id h29-20020a056e021d9d00b002cf473192casi3339761ila.107.2022.06.01.11.27.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Jun 2022 11:27:44 -0700 (PDT) Received-SPF: pass (google.com: domain of u-boot-bounces@lists.denx.de designates 85.214.62.61 as permitted sender) client-ip=85.214.62.61; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HR2k8bSM; spf=pass (google.com: domain of u-boot-bounces@lists.denx.de designates 85.214.62.61 as permitted sender) smtp.mailfrom=u-boot-bounces@lists.denx.de; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 7EA38842FE; Wed, 1 Jun 2022 20:27:04 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="HR2k8bSM"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 1950483F24; Wed, 1 Jun 2022 20:26:46 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 04170842D4 for ; Wed, 1 Jun 2022 20:26:40 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=loic.poulain@linaro.org Received: by mail-wr1-x432.google.com with SMTP id s24so3462947wrb.10 for ; Wed, 01 Jun 2022 11:26:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IqoggFa0bpk378ZDhWjg79ElSHZ5vkFsTdicvgfmxYI=; b=HR2k8bSM4Kymhv3Fp6uqrwF+qzeCSEOss7Lmav9TG76tn8K9Bdwm/VcpZ4HLWII86w ajsn9W1e59KFoMQUKmICGKGBKZXfVpviSy/YbOKLuyQPTFp0NYFYneE2VYrD6IYPGxti T0mMHquIzAj+aUDSixv331C1u0iYz/h0DnAF4m41nW7kZt3gdd5uueGAqUpfSOQWfgrV EHYJPEwBSWFUnpMSrKcvnsQCE8ui8Khypx9eQzn9bZHhgCJPCp0BVOdZO4ElwQxLNyaI 9IcdDehThoJtoUQkVpeSk1O+rXkmiXGSvYDl55I82YFewZDaFYRvaywlslL9YXpfJcUy X7Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IqoggFa0bpk378ZDhWjg79ElSHZ5vkFsTdicvgfmxYI=; b=zoCHqs6g1h6iB0AQ3VrbpFon+QNBFae8iotoSxdGi1gGfyqaGEITVOgi0H4+SsigAX 9S3YIqRpcLEo3XzTGSQoNpqjiUiQXcRtmJQ7NtS15i+mq660SuuKV8AyWImMJSO2iDyb tDfVAlJZVDk13s8AZOK5wb4dcftYCCtzJam4cbBOaVSVwQBYfh6/g2tLdH5gqnJZGv59 STVgsN/KyuvmASvHlfmog9rLjgWvXyCmbD09I/ntGsU6uR4I0BU3eOnqZ5GN9O2U9lmo FozdMoPC6+zm0jt5uP3E9cSZR8kSC90pdiPOqSp2dHrS611dFuHwglVkFM4BctqYVbuh BkiQ== X-Gm-Message-State: AOAM531EQD2T8sS6jt1DZvTmnYbqvuPyIKXAR5lZvNYY/sL3wl1QdGm7 0w192eWY5fgERHVVua6QQEqRZA== X-Received: by 2002:a5d:584a:0:b0:20f:dec9:7597 with SMTP id i10-20020a5d584a000000b0020fdec97597mr590603wrf.157.1654107999393; Wed, 01 Jun 2022 11:26:39 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:82c:5f0:58:ccc3:729d:30c2]) by smtp.gmail.com with ESMTPSA id m30-20020a05600c3b1e00b00395f15d993fsm6672837wms.5.2022.06.01.11.26.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Jun 2022 11:26:38 -0700 (PDT) From: Loic Poulain To: trini@konsulko.com Cc: u-boot@lists.denx.de, michal.simek@xilinx.com, kettenis@openbsd.org, Loic Poulain Subject: [PATCH v2 5/5] armv8 SHA-256 using ARMv8 Crypto Extensions Date: Wed, 1 Jun 2022 20:26:31 +0200 Message-Id: <1654107991-598-6-git-send-email-loic.poulain@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1654107991-598-1-git-send-email-loic.poulain@linaro.org> References: <1654107991-598-1-git-send-email-loic.poulain@linaro.org> X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.5 at phobos.denx.de X-Virus-Status: Clean This patch adds support for the SHA-256 Secure Hash Algorithm for CPUs that have support for the SHA-256 part of the ARM v8 Crypto Extensions. It greatly improves sha-256 based operations, about 17x faster on iMX8M evk board. ~12ms vs ~208ms for a 20MiB kernel sha-256 verification. asm implementation is a simplified version of the Linux version (from Ard Biesheuvel). Signed-off-by: Loic Poulain --- arch/arm/cpu/armv8/Kconfig | 4 ++ arch/arm/cpu/armv8/Makefile | 1 + arch/arm/cpu/armv8/sha256_ce_core.S | 134 ++++++++++++++++++++++++++++++++++++ arch/arm/cpu/armv8/sha256_ce_glue.c | 21 ++++++ 4 files changed, 160 insertions(+) create mode 100644 arch/arm/cpu/armv8/sha256_ce_core.S create mode 100644 arch/arm/cpu/armv8/sha256_ce_glue.c diff --git a/arch/arm/cpu/armv8/Kconfig b/arch/arm/cpu/armv8/Kconfig index 0b11ca8..0494a08 100644 --- a/arch/arm/cpu/armv8/Kconfig +++ b/arch/arm/cpu/armv8/Kconfig @@ -180,6 +180,10 @@ config ARMV8_CE_SHA1 bool "SHA-1 digest algorithm (ARMv8 Crypto Extensions)" default y if SHA1 +config ARMV8_CE_SHA256 + bool "SHA-256 digest algorithm (ARMv8 Crypto Extensions)" + default y if SHA256 + endif endif diff --git a/arch/arm/cpu/armv8/Makefile b/arch/arm/cpu/armv8/Makefile index ff2495c..2e4bf9e 100644 --- a/arch/arm/cpu/armv8/Makefile +++ b/arch/arm/cpu/armv8/Makefile @@ -45,3 +45,4 @@ obj-$(CONFIG_ARMV8_PSCI) += psci.o obj-$(CONFIG_TARGET_BCMNS3) += bcmns3/ obj-$(CONFIG_XEN) += xen/ obj-$(CONFIG_ARMV8_CE_SHA1) += sha1_ce_glue.o sha1_ce_core.o +obj-$(CONFIG_ARMV8_CE_SHA256) += sha256_ce_glue.o sha256_ce_core.o diff --git a/arch/arm/cpu/armv8/sha256_ce_core.S b/arch/arm/cpu/armv8/sha256_ce_core.S new file mode 100644 index 0000000..fbae3ca --- /dev/null +++ b/arch/arm/cpu/armv8/sha256_ce_core.S @@ -0,0 +1,134 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * sha256-ce-core.S - core SHA-256 transform using v8 Crypto Extensions + * + * Copyright (C) 2014 Linaro Ltd + * Copyright (C) 2022 Linaro Ltd + */ + + #include + #include + #include + #include + + .text + .arch armv8-a+crypto + + dga .req q20 + dgav .req v20 + dgb .req q21 + dgbv .req v21 + + t0 .req v22 + t1 .req v23 + + dg0q .req q24 + dg0v .req v24 + dg1q .req q25 + dg1v .req v25 + dg2q .req q26 + dg2v .req v26 + + .macro add_only, ev, rc, s0 + mov dg2v.16b, dg0v.16b + .ifeq \ev + add t1.4s, v\s0\().4s, \rc\().4s + sha256h dg0q, dg1q, t0.4s + sha256h2 dg1q, dg2q, t0.4s + .else + .ifnb \s0 + add t0.4s, v\s0\().4s, \rc\().4s + .endif + sha256h dg0q, dg1q, t1.4s + sha256h2 dg1q, dg2q, t1.4s + .endif + .endm + + .macro add_update, ev, rc, s0, s1, s2, s3 + sha256su0 v\s0\().4s, v\s1\().4s + add_only \ev, \rc, \s1 + sha256su1 v\s0\().4s, v\s2\().4s, v\s3\().4s + .endm + + /* + * The SHA-256 round constants + */ + .align 4 +.Lsha2_rcon: + .word 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5 + .word 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5 + .word 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3 + .word 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174 + .word 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc + .word 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da + .word 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7 + .word 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967 + .word 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13 + .word 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85 + .word 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3 + .word 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070 + .word 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5 + .word 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3 + .word 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208 + .word 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2 + + /* + * void sha256_armv8_ce_process(struct sha256_ce_state *sst, + * uint8_t const *src, uint32_t blocks) + */ +ENTRY(sha256_armv8_ce_process) + /* load round constants */ + adr x8, .Lsha2_rcon + ld1 { v0.4s- v3.4s}, [x8], #64 + ld1 { v4.4s- v7.4s}, [x8], #64 + ld1 { v8.4s-v11.4s}, [x8], #64 + ld1 {v12.4s-v15.4s}, [x8] + + /* load state */ + ldp dga, dgb, [x0] + + /* load input */ +0: ld1 {v16.4s-v19.4s}, [x1], #64 + sub w2, w2, #1 +#if __BYTE_ORDER == __LITTLE_ENDIAN + rev32 v16.16b, v16.16b + rev32 v17.16b, v17.16b + rev32 v18.16b, v18.16b + rev32 v19.16b, v19.16b +#endif + +1: add t0.4s, v16.4s, v0.4s + mov dg0v.16b, dgav.16b + mov dg1v.16b, dgbv.16b + + add_update 0, v1, 16, 17, 18, 19 + add_update 1, v2, 17, 18, 19, 16 + add_update 0, v3, 18, 19, 16, 17 + add_update 1, v4, 19, 16, 17, 18 + + add_update 0, v5, 16, 17, 18, 19 + add_update 1, v6, 17, 18, 19, 16 + add_update 0, v7, 18, 19, 16, 17 + add_update 1, v8, 19, 16, 17, 18 + + add_update 0, v9, 16, 17, 18, 19 + add_update 1, v10, 17, 18, 19, 16 + add_update 0, v11, 18, 19, 16, 17 + add_update 1, v12, 19, 16, 17, 18 + + add_only 0, v13, 17 + add_only 1, v14, 18 + add_only 0, v15, 19 + add_only 1 + + /* update state */ + add dgav.4s, dgav.4s, dg0v.4s + add dgbv.4s, dgbv.4s, dg1v.4s + + /* handled all input blocks? */ + cbnz w2, 0b + + /* store new state */ +3: stp dga, dgb, [x0] + ret +ENDPROC(sha256_armv8_ce_process) diff --git a/arch/arm/cpu/armv8/sha256_ce_glue.c b/arch/arm/cpu/armv8/sha256_ce_glue.c new file mode 100644 index 0000000..67dd796 --- /dev/null +++ b/arch/arm/cpu/armv8/sha256_ce_glue.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * sha256_ce_glue.c - SHA-256 secure hash using ARMv8 Crypto Extensions + * + * Copyright (C) 2022 Linaro Ltd + */ + +#include +#include + +extern void sha256_armv8_ce_process(uint32_t state[8], uint8_t const *src, + uint32_t blocks); + +void sha256_process(sha256_context *ctx, const unsigned char *data, + unsigned int blocks) +{ + if (!blocks) + return; + + sha256_armv8_ce_process(ctx->state, data, blocks); +}