From patchwork Fri Sep 6 18:45:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Corentin Labbe X-Patchwork-Id: 173280 Delivered-To: patch@linaro.org Received: by 2002:a05:6e02:ce:0:0:0:0 with SMTP id r14csp1107082ilq; Fri, 6 Sep 2019 11:46:54 -0700 (PDT) X-Google-Smtp-Source: APXvYqyVWl6flNMzFA1byEsHDD8RdL2CodAxhVpC78INSgg97QpjSMcQWXXzf7gTQscd6c7j98K0 X-Received: by 2002:a63:607:: with SMTP id 7mr9139785pgg.240.1567795614036; Fri, 06 Sep 2019 11:46:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1567795614; cv=none; d=google.com; s=arc-20160816; b=kFNIGlXkIwT5DXLdFx94DndxJBmAHiC7K6SXkKjRDUzIM9MQdyxClIifU9eyYNT9ZX iWpfEc9QiCrwPJs3CvyjQpHP4QrweUyFGkBoLS/QXm1gF+ZYx7w/HkeZWV8hW6HOu3S8 Ul202slfVLwKPd6aFjdXzQDkprLzpTWf1LCsPI44REv7NB/DuU8sYjM1q7fgN43onW4i 0uNtSCnjnUcaBoJ4rFWOPuEwTmAtzAFJ5TMdNlXLGozg2meq+rMysJMOvclh5+iCKsFX ZoSTr6bk+GvJBk9uEjdlL3jNpWE9s1wjF8+ws7rDLUVWKPngA5OCc3Y3AKpZnjCnbVp+ dkIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=gEQgUSBQCxuVE1wg76TJ7aomsqut3kVHgaiDJRHUfKg=; b=V2W5zUBgUUVeORD5rDFcrshgl8BUCoESSGsgDnhkW4RhNrxlhjQZ0TDdev7Yw1v+9t SikhivIDxi+/S519RM2JFX4h6e+KPV4hTr7VDjfHt6VT416lErh42wqJqPAn5qA4y3lF tY4OLPUSKI4h9oxkONK1PDUjzYYnzcY2EBU1DAbCI3gOUBDjIknqiHOfgaOP3g+r4+zV Ks2w6duf2fTumQQrcc5iJiObI9Bsu9V9B1hKjJRpP+ORcyKvh88nhLBRmJrbMig8i+7W WC3DT98+jA7f91KDdAwNpL8hvP8Fbz7L5CraBJN4NjEUXseOnyrSQ+MqakorT5yYMFRT Zr3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=kKRpoEwr; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i62si6411323pfb.138.2019.09.06.11.46.53; Fri, 06 Sep 2019 11:46:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=kKRpoEwr; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436661AbfIFSqw (ORCPT + 3 others); Fri, 6 Sep 2019 14:46:52 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:34011 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395368AbfIFSqL (ORCPT ); Fri, 6 Sep 2019 14:46:11 -0400 Received: by mail-wm1-f68.google.com with SMTP id y135so8012970wmc.1; Fri, 06 Sep 2019 11:46:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gEQgUSBQCxuVE1wg76TJ7aomsqut3kVHgaiDJRHUfKg=; b=kKRpoEwrliVBBCH57DTvcRNglHxpwcwRhxKv/MWz4W2ZKW77NLluslz9vF3u/IzC4v VJueOo0iLDkrLq9vWAQK4GujbInid/r7guH1JbusfmaR1vmC0DNFQEAGorOj1YRrBM0j ZyIOTnsm9Vr5w995E73g+8mQIfIkxxTnTbkAeq9i3Dvc9FZEJ6j3yG9NcdK5OrtBqCz7 FiMzjXOGOir3XTXeST7YxH4q1mlN0SYRr9VyEla8Gv206Suig/rNEXAXtj+GKPpHhePT yHxWOrs4MewNec0REhiZndQJeUDIu8UgqZTBcsnRHAJ0WNg2mGDKTACfO0hBY4mc5ekZ SQ8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gEQgUSBQCxuVE1wg76TJ7aomsqut3kVHgaiDJRHUfKg=; b=bZlR0VBdxTl9xNCUGgBA+8AHsCzL+0OEAE6BIY9Cwig2bSx8F+YIU3KcDNQ2EnKPA/ v0hG5cIcOGawei0s5/1fVBRfvixbVqPZYQ+w6PcTf/UIgRhL/CSYJL7OZROHoi5kZI0a u3gKjs2NrLauwU0vmTXX5Zd/86LHe7DGVVDNWhwpdBfhIqBk8wQdAO3zcqoR9teksQQl ZcX+br6/ygL1xOvFmsG9O+3N8q6BDk0CUBE5nQwAr0V9SZ7LBgknm6N/s+awONFn+ijR azdkZSlMJJU/kAlBxGYi9sq4sRHYK9Ou6iNeIF4areFzU0DQ7aaSY9xZykFmhqMR37EV tspw== X-Gm-Message-State: APjAAAUfDbqVQcNhBKJsUCuPXrqdFnIWXpSc4BzdIdOTG9IOnpwSmia9 kMMFDwXWfZwsXw+++z2arEQ= X-Received: by 2002:a1c:7c1a:: with SMTP id x26mr9264378wmc.115.1567795566441; Fri, 06 Sep 2019 11:46:06 -0700 (PDT) Received: from Red.localdomain ([2a01:cb1d:147:7200:2e56:dcff:fed2:c6d6]) by smtp.googlemail.com with ESMTPSA id j1sm8677577wrg.24.2019.09.06.11.46.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Sep 2019 11:46:05 -0700 (PDT) From: Corentin Labbe To: davem@davemloft.net, herbert@gondor.apana.org.au, linux@armlinux.org.uk, mark.rutland@arm.com, mripard@kernel.org, robh+dt@kernel.org, wens@csie.org Cc: devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, linux-sunxi@googlegroups.com, Corentin Labbe Subject: [PATCH 2/9] crypto: Add Allwinner sun8i-ce Crypto Engine Date: Fri, 6 Sep 2019 20:45:44 +0200 Message-Id: <20190906184551.17858-3-clabbe.montjoie@gmail.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190906184551.17858-1-clabbe.montjoie@gmail.com> References: <20190906184551.17858-1-clabbe.montjoie@gmail.com> MIME-Version: 1.0 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The Crypto Engine is an hardware cryptographic offloader present on all recent Allwinner SoCs H2+, H3, R40, A64, H5, H6 This driver supports AES cipher in CBC/ECB mode. Signed-off-by: Corentin Labbe --- drivers/crypto/allwinner/Kconfig | 26 + drivers/crypto/allwinner/Makefile | 1 + drivers/crypto/allwinner/sun8i-ce/Makefile | 2 + .../allwinner/sun8i-ce/sun8i-ce-cipher.c | 390 +++++++++++ .../crypto/allwinner/sun8i-ce/sun8i-ce-core.c | 630 ++++++++++++++++++ drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h | 256 +++++++ 6 files changed, 1305 insertions(+) create mode 100644 drivers/crypto/allwinner/Makefile create mode 100644 drivers/crypto/allwinner/sun8i-ce/Makefile create mode 100644 drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c create mode 100644 drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c create mode 100644 drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h -- 2.21.0 diff --git a/drivers/crypto/allwinner/Kconfig b/drivers/crypto/allwinner/Kconfig index 0c8a99f7959d..742a08b64c90 100644 --- a/drivers/crypto/allwinner/Kconfig +++ b/drivers/crypto/allwinner/Kconfig @@ -4,3 +4,29 @@ config CRYPTO_DEV_ALLWINNER default y if ARCH_SUNXI help Say Y here to get to see options for Allwinner hardware crypto devices + +config CRYPTO_DEV_SUN8I_CE + tristate "Support for Allwinner Crypto Engine cryptographic offloader" + select CRYPTO_BLKCIPHER + select CRYPTO_ENGINE + select CRYPTO_ECB + select CRYPTO_CBC + select CRYPTO_AES + select CRYPTO_DES + depends on CRYPTO_DEV_ALLWINNER + help + Select y here for having support for the crypto Engine availlable on + Allwinner SoC H2+, H3, H5, H6, R40 and A64. + The Crypto Engine handle AES/3DES ciphers in ECB/CBC mode. + + To compile this driver as a module, choose M here: the module + will be called sun8i-ce. + +config CRYPTO_DEV_SUN8I_CE_DEBUG + bool "Enabled sun8i-ce stats" + depends on CRYPTO_DEV_SUN8I_CE + depends on DEBUG_FS + help + Say y to enabled sun8i-ce debug stats. + This will create /sys/kernel/debug/sun8i-ce/stats for displaying + the number of requests per flow and per algorithm. diff --git a/drivers/crypto/allwinner/Makefile b/drivers/crypto/allwinner/Makefile new file mode 100644 index 000000000000..11f02db9ee06 --- /dev/null +++ b/drivers/crypto/allwinner/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_CRYPTO_DEV_SUN8I_CE) += sun8i-ce/ diff --git a/drivers/crypto/allwinner/sun8i-ce/Makefile b/drivers/crypto/allwinner/sun8i-ce/Makefile new file mode 100644 index 000000000000..08b68c3c1ca9 --- /dev/null +++ b/drivers/crypto/allwinner/sun8i-ce/Makefile @@ -0,0 +1,2 @@ +obj-$(CONFIG_CRYPTO_DEV_SUN8I_CE) += sun8i-ce.o +sun8i-ce-y += sun8i-ce-core.o sun8i-ce-cipher.o diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c new file mode 100644 index 000000000000..c22f0592f168 --- /dev/null +++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c @@ -0,0 +1,390 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * sun8i-ce-cipher.c - hardware cryptographic offloader for + * Allwinner H3/A64/H5/H2+/H6/R40 SoC + * + * Copyright (C) 2016-2019 Corentin LABBE + * + * This file add support for AES cipher with 128,192,256 bits keysize in + * CBC and ECB mode. + * + * You could find a link for the datasheet in Documentation/arm/sunxi/README + */ + +#include +#include +#include +#include +#include +#include +#include "sun8i-ce.h" + +static int sun8i_ce_cipher(struct skcipher_request *areq) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq); + struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm); + struct sun8i_ce_dev *ce = op->ce; + struct sun8i_cipher_req_ctx *rctx = skcipher_request_ctx(areq); + struct skcipher_alg *alg = crypto_skcipher_alg(tfm); + struct sun8i_ce_alg_template *algt; + struct sun8i_ce_flow *chan; + struct ce_task *cet; + struct scatterlist *sg; + bool need_fallback = false; + unsigned int todo, len, offset, ivsize; + void *backup_iv = NULL; + int flow, i; + int nr_sgs = 0; + int nr_sgd = 0; + int err = 0; + + algt = container_of(alg, struct sun8i_ce_alg_template, alg.skcipher); + + dev_dbg(ce->dev, "%s %s %u %x IV(%p %u) key=%u\n", __func__, + crypto_tfm_alg_name(areq->base.tfm), + areq->cryptlen, + rctx->op_dir, areq->iv, crypto_skcipher_ivsize(tfm), + op->keylen); + + if (sg_nents(areq->src) > MAX_SG || sg_nents(areq->dst) > MAX_SG) + need_fallback = true; + + if (areq->cryptlen < crypto_skcipher_ivsize(tfm)) + need_fallback = true; + + if (areq->cryptlen == 0) + need_fallback = true; + if (areq->cryptlen % algt->alg.skcipher.base.cra_blocksize) + need_fallback = true; + + sg = areq->src; + while (sg && !need_fallback) { + if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32))) { + need_fallback = true; + break; + } + sg = sg_next(sg); + } + sg = areq->dst; + while (sg && !need_fallback) { + if (sg->length % 4 || !IS_ALIGNED(sg->offset, sizeof(u32))) { + need_fallback = true; + break; + } + sg = sg_next(sg); + } + +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + algt->stat_req++; +#endif + + if (need_fallback) { + SYNC_SKCIPHER_REQUEST_ON_STACK(subreq, op->fallback_tfm); +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + algt->stat_fb++; +#endif + skcipher_request_set_sync_tfm(subreq, op->fallback_tfm); + skcipher_request_set_callback(subreq, areq->base.flags, NULL, + NULL); + skcipher_request_set_crypt(subreq, areq->src, areq->dst, + areq->cryptlen, areq->iv); + if (rctx->op_dir & CE_DECRYPTION) + err = crypto_skcipher_decrypt(subreq); + else + err = crypto_skcipher_encrypt(subreq); + skcipher_request_zero(subreq); + return err; + } + + flow = rctx->flow; + + chan = &ce->chanlist[flow]; + mutex_lock(&chan->lock); + + cet = chan->tl; + memset(cet, 0, sizeof(struct ce_task)); + + cet->t_id = flow; + cet->t_common_ctl = ce->variant->alg_cipher[algt->ce_algo_id]; + cet->t_common_ctl |= rctx->op_dir | CE_COMM_INT; + cet->t_dlen = areq->cryptlen / 4; + /* CTS and recent CE (H6) need length in bytes, in word otherwise */ + if (ce->variant->model == CE_v2) + cet->t_dlen = areq->cryptlen; + + cet->t_sym_ctl = ce->variant->op_mode[algt->ce_blockmode]; + len = op->keylen; + switch (len) { + case 128 / 8: + cet->t_sym_ctl |= CE_AES_128BITS; + break; + case 192 / 8: + cet->t_sym_ctl |= CE_AES_192BITS; + break; + case 256 / 8: + cet->t_sym_ctl |= CE_AES_256BITS; + break; + } + + cet->t_asym_ctl = 0; + + chan->op_mode = ce->variant->op_mode[algt->ce_blockmode]; + chan->op_dir = rctx->op_dir; + chan->method = ce->variant->alg_cipher[algt->ce_algo_id]; + chan->keylen = op->keylen; + + cet->t_key = dma_map_single(ce->dev, op->key, op->keylen, + DMA_TO_DEVICE); + if (dma_mapping_error(ce->dev, cet->t_key)) { + dev_err(ce->dev, "Cannot DMA MAP KEY\n"); + err = -EFAULT; + goto theend; + } + + ivsize = crypto_skcipher_ivsize(tfm); + if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) { + chan->ivlen = ivsize; + chan->bounce_iv = kzalloc(ivsize, GFP_KERNEL | GFP_DMA); + if (!chan->bounce_iv) { + err = -ENOMEM; + goto theend_key; + } + if (rctx->op_dir & CE_DECRYPTION) { + backup_iv = kzalloc(ivsize, GFP_KERNEL); + if (!backup_iv) { + err = -ENOMEM; + goto theend_key; + } + offset = areq->cryptlen - ivsize; + scatterwalk_map_and_copy(backup_iv, areq->src, offset, + ivsize, 0); + } + memcpy(chan->bounce_iv, areq->iv, ivsize); + } + + if (areq->src == areq->dst) { + nr_sgs = dma_map_sg(ce->dev, areq->src, sg_nents(areq->src), + DMA_BIDIRECTIONAL); + if (nr_sgs <= 0 || nr_sgs > MAX_SG) { + dev_err(ce->dev, "Invalid sg number %d\n", nr_sgs); + err = -EINVAL; + goto theend_iv; + } + nr_sgd = nr_sgs; + } else { + nr_sgs = dma_map_sg(ce->dev, areq->src, sg_nents(areq->src), + DMA_TO_DEVICE); + if (nr_sgs <= 0 || nr_sgs > MAX_SG) { + dev_err(ce->dev, "Invalid sg number %d\n", nr_sgs); + err = -EINVAL; + goto theend_iv; + } + nr_sgd = dma_map_sg(ce->dev, areq->dst, sg_nents(areq->dst), + DMA_FROM_DEVICE); + if (nr_sgd <= 0 || nr_sgd > MAX_SG) { + dev_err(ce->dev, "Invalid sg number %d\n", nr_sgd); + err = -EINVAL; + goto theend_sgs; + } + } + + len = areq->cryptlen; + for_each_sg(areq->src, sg, nr_sgs, i) { + cet->t_src[i].addr = sg_dma_address(sg); + todo = min(len, sg_dma_len(sg)); + cet->t_src[i].len = todo / 4; + dev_dbg(ce->dev, "%s total=%u SG(%d %u off=%d) todo=%u\n", __func__, + areq->cryptlen, i, cet->t_src[i].len, sg->offset, todo); + len -= todo; + } + if (len > 0) + dev_err(ce->dev, "remaining len %d\n", len); + + len = areq->cryptlen; + for_each_sg(areq->dst, sg, nr_sgd, i) { + cet->t_dst[i].addr = sg_dma_address(sg); + todo = min(len, sg_dma_len(sg)); + cet->t_dst[i].len = todo / 4; + dev_dbg(ce->dev, "%s total=%u SG(%d %u off=%d) todo=%u\n", __func__, + areq->cryptlen, i, cet->t_dst[i].len, sg->offset, todo); + len -= todo; + } + if (len > 0) + dev_err(ce->dev, "remaining len %d\n", len); + + chan->timeout = areq->cryptlen; + err = sun8i_ce_run_task(ce, flow, "cipher"); + if (err) + dev_err(ce->dev, "Error with len=%u\n", areq->cryptlen); + +theend_sgs: + if (areq->src == areq->dst) { + dma_unmap_sg(ce->dev, areq->src, nr_sgs, DMA_BIDIRECTIONAL); + } else { + if (nr_sgs > 0) + dma_unmap_sg(ce->dev, areq->src, nr_sgs, DMA_TO_DEVICE); + dma_unmap_sg(ce->dev, areq->dst, nr_sgd, DMA_FROM_DEVICE); + } + +theend_iv: + if (areq->iv && ivsize > 0) { + offset = areq->cryptlen - ivsize; + if (rctx->op_dir & CE_DECRYPTION) { + memcpy(areq->iv, backup_iv, ivsize); + kzfree(backup_iv); + } else { + scatterwalk_map_and_copy(areq->iv, areq->dst, offset, + ivsize, 0); + } + kfree(chan->bounce_iv); + } + +theend_key: + dma_unmap_single(ce->dev, cet->t_key, op->keylen, DMA_TO_DEVICE); + +theend: + mutex_unlock(&chan->lock); + + return err; +} + +static int handle_cipher_request(struct crypto_engine *engine, void *areq) +{ + int err; + struct skcipher_request *breq = container_of(areq, struct skcipher_request, base); + + err = sun8i_ce_cipher(breq); + crypto_finalize_skcipher_request(engine, breq, err); + + return 0; +} + +int sun8i_ce_skdecrypt(struct skcipher_request *areq) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq); + struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm); + struct sun8i_cipher_req_ctx *rctx = skcipher_request_ctx(areq); + int e = sun8i_ce_get_engine_number(op->ce); + struct crypto_engine *engine = op->ce->chanlist[e].engine; + + rctx->op_dir = CE_DECRYPTION; + rctx->flow = e; + + return crypto_transfer_skcipher_request_to_engine(engine, areq); +} + +int sun8i_ce_skencrypt(struct skcipher_request *areq) +{ + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq); + struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm); + struct sun8i_cipher_req_ctx *rctx = skcipher_request_ctx(areq); + int e = sun8i_ce_get_engine_number(op->ce); + struct crypto_engine *engine = op->ce->chanlist[e].engine; + + rctx->op_dir = CE_ENCRYPTION; + rctx->flow = e; + + return crypto_transfer_skcipher_request_to_engine(engine, areq); +} + +int sun8i_ce_cipher_init(struct crypto_tfm *tfm) +{ + struct sun8i_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm); + struct sun8i_ce_alg_template *algt; + const char *name = crypto_tfm_alg_name(tfm); + struct crypto_skcipher *sktfm = __crypto_skcipher_cast(tfm); + struct skcipher_alg *alg = crypto_skcipher_alg(sktfm); + + memset(op, 0, sizeof(struct sun8i_cipher_tfm_ctx)); + + algt = container_of(alg, struct sun8i_ce_alg_template, alg.skcipher); + op->ce = algt->ce; + + sktfm->reqsize = sizeof(struct sun8i_cipher_req_ctx); + + op->fallback_tfm = crypto_alloc_sync_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK); + if (IS_ERR(op->fallback_tfm)) { + dev_err(op->ce->dev, "ERROR: Cannot allocate fallback for %s %ld\n", + name, PTR_ERR(op->fallback_tfm)); + return PTR_ERR(op->fallback_tfm); + } + + dev_info(op->ce->dev, "Fallback is %s\n", crypto_tfm_alg_driver_name(crypto_skcipher_tfm(&op->fallback_tfm->base))); + + op->enginectx.op.do_one_request = handle_cipher_request; + op->enginectx.op.prepare_request = NULL; + op->enginectx.op.unprepare_request = NULL; + + return 0; +} + +void sun8i_ce_cipher_exit(struct crypto_tfm *tfm) +{ + struct sun8i_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm); + + if (op->key) { + memzero_explicit(op->key, op->keylen); + kfree(op->key); + } + crypto_free_sync_skcipher(op->fallback_tfm); +} + +int sun8i_ce_aes_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keylen) +{ + struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm); + struct sun8i_ce_dev *ce = op->ce; + + switch (keylen) { + case 128 / 8: + break; + case 192 / 8: + break; + case 256 / 8: + break; + default: + dev_dbg(ce->dev, "ERROR: Invalid keylen %u\n", keylen); + crypto_skcipher_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); + return -EINVAL; + } + if (op->key) { + memzero_explicit(op->key, op->keylen); + kfree(op->key); + } + op->keylen = keylen; + op->key = kmalloc(keylen, GFP_KERNEL | GFP_DMA); + if (!op->key) + return -ENOMEM; + memcpy(op->key, key, keylen); + + crypto_sync_skcipher_clear_flags(op->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_sync_skcipher_set_flags(op->fallback_tfm, tfm->base.crt_flags & CRYPTO_TFM_REQ_MASK); + + return crypto_sync_skcipher_setkey(op->fallback_tfm, key, keylen); +} + +int sun8i_ce_des3_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keylen) +{ + struct sun8i_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm); + int err; + + err = verify_skcipher_des3_key(tfm, key); + if (err) + return err; + + if (op->key) { + memzero_explicit(op->key, op->keylen); + kfree(op->key); + } + op->keylen = keylen; + op->key = kmalloc(keylen, GFP_KERNEL | GFP_DMA); + if (!op->key) + return -ENOMEM; + memcpy(op->key, key, keylen); + + crypto_sync_skcipher_clear_flags(op->fallback_tfm, CRYPTO_TFM_REQ_MASK); + crypto_sync_skcipher_set_flags(op->fallback_tfm, tfm->base.crt_flags & CRYPTO_TFM_REQ_MASK); + + return crypto_sync_skcipher_setkey(op->fallback_tfm, key, keylen); +} diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c new file mode 100644 index 000000000000..c609b9943296 --- /dev/null +++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c @@ -0,0 +1,630 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * sun8i-ce-core.c - hardware cryptographic offloader for + * Allwinner H3/A64/H5/H2+/H6/R40 SoC + * + * Copyright (C) 2015-2019 Corentin Labbe + * + * Core file which registers crypto algorithms supported by the CryptoEngine. + * + * You could find a link for the datasheet in Documentation/arm/sunxi/README + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "sun8i-ce.h" + +static const struct ce_variant ce_h3_variant = { + .alg_cipher = { CE_ID_NOTSUPP, CE_ALG_AES, CE_ALG_DES, CE_ALG_3DES, + CE_ID_NOTSUPP, + }, + .op_mode = { CE_ID_NOTSUPP, CE_OP_ECB, CE_OP_CBC + }, + .intreg = CE_ISR, + .maxflow = 4, + .ce_clks = { + { "ahb", 200000000 }, + { "mod", 48000000 }, + } +}; + +static const struct ce_variant ce_h5_variant = { + .alg_cipher = { CE_ID_NOTSUPP, CE_ALG_AES, CE_ALG_DES, CE_ALG_3DES, + CE_ID_NOTSUPP, + }, + .op_mode = { CE_ID_NOTSUPP, CE_OP_ECB, CE_OP_CBC + }, + .intreg = CE_ISR, + .maxflow = 4, + .ce_clks = { + { "ahb", 200000000 }, + { "mod", 300000000 }, + } +}; + +static const struct ce_variant ce_h6_variant = { + .alg_cipher = { CE_ID_NOTSUPP, CE_ALG_AES, CE_ALG_DES, CE_ALG_3DES, + CE_ALG_RAES, + }, + .op_mode = { CE_ID_NOTSUPP, CE_OP_ECB, CE_OP_CBC + }, + .model = CE_v2, + .intreg = CE_ISR, + .maxflow = 4, + .ce_clks = { + { "ahb", 200000000 }, + { "mod", 300000000 }, + { "mbus", 400000000 }, + } +}; + +static const struct ce_variant ce_a64_variant = { + .alg_cipher = { CE_ID_NOTSUPP, CE_ALG_AES, CE_ALG_DES, CE_ALG_3DES, + CE_ID_NOTSUPP, + }, + .op_mode = { CE_ID_NOTSUPP, CE_OP_ECB, CE_OP_CBC + }, + .intreg = CE_ISR, + .maxflow = 4, + .ce_clks = { + { "ahb", 200000000 }, + { "mod", 300000000 }, + } +}; + +static const struct ce_variant ce_r40_variant = { + .alg_cipher = { CE_ID_NOTSUPP, CE_ALG_AES, CE_ALG_DES, CE_ALG_3DES, + CE_ID_NOTSUPP, + }, + .op_mode = { CE_ID_NOTSUPP, CE_OP_ECB, CE_OP_CBC + }, + .intreg = CE_ISR, + .maxflow = 4, + .ce_clks = { + { "ahb", 200000000 }, + { "mod", 300000000 }, + } +}; + +int sun8i_ce_get_engine_number(struct sun8i_ce_dev *ce) +{ + return atomic_inc_return(&ce->flow) % ce->variant->maxflow; +} + +int sun8i_ce_run_task(struct sun8i_ce_dev *ce, int flow, const char *name) +{ + u32 v; + int err = 0; + struct ce_task *cet = ce->chanlist[flow].tl; + + if (ce->chanlist[flow].bounce_iv) { + cet->t_iv = dma_map_single(ce->dev, + ce->chanlist[flow].bounce_iv, + ce->chanlist[flow].ivlen, + DMA_TO_DEVICE); + if (dma_mapping_error(ce->dev, cet->t_iv)) { + dev_err(ce->dev, "Cannot DMA MAP IV\n"); + return -EFAULT; + } + } + +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + ce->chanlist[flow].stat_req++; +#endif + + mutex_lock(&ce->mlock); + + v = readl(ce->base + CE_ICR); + v |= 1 << flow; + writel(v, ce->base + CE_ICR); + + reinit_completion(&ce->chanlist[flow].complete); + writel(ce->chanlist[flow].t_phy, ce->base + CE_TDQ); + + ce->chanlist[flow].status = 0; + /* Be sure all data is written before enabling the task */ + wmb(); + + v = 1 | (ce->chanlist[flow].tl->t_common_ctl & 0x7F) << 8; + writel(v, ce->base + CE_TLR); + mutex_unlock(&ce->mlock); + + wait_for_completion_interruptible_timeout(&ce->chanlist[flow].complete, + msecs_to_jiffies(ce->chanlist[flow].timeout)); + + if (ce->chanlist[flow].status == 0) { + dev_err(ce->dev, "DMA timeout for %s\n", name); + err = -EFAULT; + } + /* No need to lock for this read, the channel is locked so + * nothing could modify the error value for this channel + */ + v = readl(ce->base + CE_ESR); + if (v) { + v >>= (flow * 4); + v &= 0xFF; + if (v) { + dev_err(ce->dev, "CE ERROR: %x for flow %x\n", v, flow); + err = -EFAULT; + } + if (v & CE_ERR_ALGO_NOTSUP) + dev_err(ce->dev, "CE ERROR: algorithm not supported\n"); + if (v & CE_ERR_DATALEN) + dev_err(ce->dev, "CE ERROR: data length error\n"); + if (v & CE_ERR_KEYSRAM) + dev_err(ce->dev, "CE ERROR: keysram access error for AES\n"); + if (v & CE_ERR_ADDR_INVALID) + dev_err(ce->dev, "CE ERROR: address invalid\n"); + } + + if (ce->chanlist[flow].bounce_iv) { + dma_unmap_single(ce->dev, cet->t_iv, + ce->chanlist[flow].ivlen, + DMA_TO_DEVICE); + } + + return err; +} + +static irqreturn_t ce_irq_handler(int irq, void *data) +{ + struct sun8i_ce_dev *ce = (struct sun8i_ce_dev *)data; + int flow = 0; + u32 p; + + p = readl(ce->base + ce->variant->intreg); + for (flow = 0; flow < ce->variant->maxflow; flow++) { + if (p & (BIT(flow))) { + writel(BIT(flow), ce->base + ce->variant->intreg); + ce->chanlist[flow].status = 1; + complete(&ce->chanlist[flow].complete); + } + } + + return IRQ_HANDLED; +} + +static struct sun8i_ce_alg_template ce_algs[] = { +{ + .type = CRYPTO_ALG_TYPE_SKCIPHER, + .ce_algo_id = CE_ID_CIPHER_AES, + .ce_blockmode = CE_ID_OP_CBC, + .alg.skcipher = { + .base = { + .cra_name = "cbc(aes)", + .cra_driver_name = "cbc-aes-sun8i-ce", + .cra_priority = 400, + .cra_blocksize = AES_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0xf, + .cra_init = sun8i_ce_cipher_init, + .cra_exit = sun8i_ce_cipher_exit, + }, + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .ivsize = AES_BLOCK_SIZE, + .setkey = sun8i_ce_aes_setkey, + .encrypt = sun8i_ce_skencrypt, + .decrypt = sun8i_ce_skdecrypt, + } +}, +{ + .type = CRYPTO_ALG_TYPE_SKCIPHER, + .ce_algo_id = CE_ID_CIPHER_AES, + .ce_blockmode = CE_ID_OP_ECB, + .alg.skcipher = { + .base = { + .cra_name = "ecb(aes)", + .cra_driver_name = "ecb-aes-sun8i-ce", + .cra_priority = 400, + .cra_blocksize = AES_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0xf, + .cra_init = sun8i_ce_cipher_init, + .cra_exit = sun8i_ce_cipher_exit, + }, + .min_keysize = AES_MIN_KEY_SIZE, + .max_keysize = AES_MAX_KEY_SIZE, + .setkey = sun8i_ce_aes_setkey, + .encrypt = sun8i_ce_skencrypt, + .decrypt = sun8i_ce_skdecrypt, + } +}, +{ + .type = CRYPTO_ALG_TYPE_SKCIPHER, + .ce_algo_id = CE_ID_CIPHER_DES3, + .ce_blockmode = CE_ID_OP_CBC, + .alg.skcipher = { + .base = { + .cra_name = "cbc(des3_ede)", + .cra_driver_name = "cbc-des3-sun8i-ce", + .cra_priority = 400, + .cra_blocksize = DES3_EDE_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0xf, + .cra_init = sun8i_ce_cipher_init, + .cra_exit = sun8i_ce_cipher_exit, + }, + .min_keysize = DES3_EDE_KEY_SIZE, + .max_keysize = DES3_EDE_KEY_SIZE, + .ivsize = DES3_EDE_BLOCK_SIZE, + .setkey = sun8i_ce_des3_setkey, + .encrypt = sun8i_ce_skencrypt, + .decrypt = sun8i_ce_skdecrypt, + } +}, +{ + .type = CRYPTO_ALG_TYPE_SKCIPHER, + .ce_algo_id = CE_ID_CIPHER_DES3, + .ce_blockmode = CE_ID_OP_ECB, + .alg.skcipher = { + .base = { + .cra_name = "ecb(des3_ede)", + .cra_driver_name = "ecb-des3-sun8i-ce", + .cra_priority = 400, + .cra_blocksize = DES3_EDE_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER | + CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK, + .cra_ctxsize = sizeof(struct sun8i_cipher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0xf, + .cra_init = sun8i_ce_cipher_init, + .cra_exit = sun8i_ce_cipher_exit, + }, + .min_keysize = DES3_EDE_KEY_SIZE, + .max_keysize = DES3_EDE_KEY_SIZE, + .setkey = sun8i_ce_des3_setkey, + .encrypt = sun8i_ce_skencrypt, + .decrypt = sun8i_ce_skdecrypt, + } +}, +}; + +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG +static int sun8i_ce_dbgfs_read(struct seq_file *seq, void *v) +{ + struct sun8i_ce_dev *ce = seq->private; + int i; + + for (i = 0; i < ce->variant->maxflow; i++) + seq_printf(seq, "Channel %d: req %lu\n", i, ce->chanlist[i].stat_req); + + for (i = 0; i < ARRAY_SIZE(ce_algs); i++) { + if (!ce_algs[i].ce) + continue; + switch (ce_algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + seq_printf(seq, "%s %s %lu %lu\n", + ce_algs[i].alg.skcipher.base.cra_driver_name, + ce_algs[i].alg.skcipher.base.cra_name, + ce_algs[i].stat_req, ce_algs[i].stat_fb); + break; + } + } + return 0; +} + +static int sun8i_ce_dbgfs_open(struct inode *inode, struct file *file) +{ + return single_open(file, sun8i_ce_dbgfs_read, inode->i_private); +} + +static const struct file_operations sun8i_ce_debugfs_fops = { + .owner = THIS_MODULE, + .open = sun8i_ce_dbgfs_open, + .read = seq_read, + .llseek = seq_lseek, + .release = single_release, +}; +#endif + +static int sun8i_ce_probe(struct platform_device *pdev) +{ + struct resource *res; + u32 v; + int err, i, ce_method, id, irq; + unsigned long cr; + struct sun8i_ce_dev *ce; + + if (!pdev->dev.of_node) + return -ENODEV; + + ce = devm_kzalloc(&pdev->dev, sizeof(*ce), GFP_KERNEL); + if (!ce) + return -ENOMEM; + + ce->variant = of_device_get_match_data(&pdev->dev); + if (!ce->variant) { + dev_err(&pdev->dev, "Missing Crypto Engine variant\n"); + return -EINVAL; + } + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + ce->base = devm_ioremap_resource(&pdev->dev, res); + if (IS_ERR(ce->base)) { + err = PTR_ERR(ce->base); + dev_err(&pdev->dev, "Cannot request MMIO err=%d\n", err); + return err; + } + + for (i = 0; i < CE_MAX_CLOCKS; i++) { + if (!ce->variant->ce_clks[i].name) + continue; + dev_info(&pdev->dev, "Get %s clock\n", ce->variant->ce_clks[i].name); + ce->ceclks[i] = devm_clk_get(&pdev->dev, ce->variant->ce_clks[i].name); + if (IS_ERR(ce->ceclks[i])) { + err = PTR_ERR(ce->ceclks[i]); + dev_err(&pdev->dev, "Cannot get %s CE clock err=%d\n", + ce->variant->ce_clks[i].name, err); + } + cr = clk_get_rate(ce->ceclks[i]); + if (ce->variant->ce_clks[i].freq) { + dev_info(&pdev->dev, "Set %s clock to %lu (%lu Mhz) from %lu (%lu Mhz)\n", + ce->variant->ce_clks[i].name, + ce->variant->ce_clks[i].freq, + ce->variant->ce_clks[i].freq / 1000000, + cr, + cr / 1000000); + err = clk_set_rate(ce->ceclks[i], ce->variant->ce_clks[i].freq); + if (err) + dev_err(&pdev->dev, "Fail to set %s clk speed to %lu\n", + ce->variant->ce_clks[i].name, + ce->variant->ce_clks[i].freq); + } else { + dev_info(&pdev->dev, "%s run at %lu\n", + ce->variant->ce_clks[i].name, cr); + } + err = clk_prepare_enable(ce->ceclks[i]); + if (err) { + dev_err(&pdev->dev, "Cannot prepare_enable %s\n", + ce->variant->ce_clks[i].name); + return err; + } + } + + /* Get Non Secure IRQ */ + irq = platform_get_irq(pdev, 0); + if (irq < 0) { + dev_err(ce->dev, "Cannot get NS IRQ\n"); + return irq; + } + + err = devm_request_irq(&pdev->dev, irq, ce_irq_handler, 0, + "sun8i-ce-ns", ce); + if (err < 0) { + dev_err(ce->dev, "Cannot request NS IRQ\n"); + return err; + } + + ce->reset = devm_reset_control_get_optional(&pdev->dev, "ahb"); + if (IS_ERR(ce->reset)) { + if (PTR_ERR(ce->reset) == -EPROBE_DEFER) + return PTR_ERR(ce->reset); + dev_info(&pdev->dev, "No reset control found\n"); + ce->reset = NULL; + } + + err = reset_control_deassert(ce->reset); + if (err) { + dev_err(&pdev->dev, "Cannot deassert reset control\n"); + goto error_clk; + } + + v = readl(ce->base + CE_CTR); + v >>= 16; + v &= 0x07; + dev_info(&pdev->dev, "CE_NS Die ID %x\n", v); + + ce->dev = &pdev->dev; + platform_set_drvdata(pdev, ce); + + mutex_init(&ce->mlock); + + ce->chanlist = devm_kcalloc(ce->dev, ce->variant->maxflow, + sizeof(struct sun8i_ce_flow), GFP_KERNEL); + if (!ce->chanlist) { + err = -ENOMEM; + goto error_flow; + } + + for (i = 0; i < ce->variant->maxflow; i++) { + init_completion(&ce->chanlist[i].complete); + mutex_init(&ce->chanlist[i].lock); + + ce->chanlist[i].engine = crypto_engine_alloc_init(ce->dev, true); + if (!ce->chanlist[i].engine) { + dev_err(ce->dev, "Cannot allocate engine\n"); + i--; + goto error_engine; + } + err = crypto_engine_start(ce->chanlist[i].engine); + if (err) { + dev_err(ce->dev, "Cannot start engine\n"); + goto error_engine; + } + ce->chanlist[i].tl = dma_alloc_coherent(ce->dev, + sizeof(struct ce_task), + &ce->chanlist[i].t_phy, + GFP_KERNEL); + if (!ce->chanlist[i].tl) { + dev_err(ce->dev, "Cannot get DMA memory for task %d\n", + i); + err = -ENOMEM; + goto error_engine; + } + } + +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + ce->dbgfs_dir = debugfs_create_dir("sun8i-ce", NULL); + if (IS_ERR_OR_NULL(ce->dbgfs_dir)) { + dev_err(ce->dev, "Fail to create debugfs dir"); + err = -ENOMEM; + goto error_engine; + } + ce->dbgfs_stats = debugfs_create_file("stats", 0444, + ce->dbgfs_dir, ce, + &sun8i_ce_debugfs_fops); + if (IS_ERR_OR_NULL(ce->dbgfs_stats)) { + dev_err(ce->dev, "Fail to create debugfs stat"); + err = -ENOMEM; + goto error_debugfs; + } +#endif + for (i = 0; i < ARRAY_SIZE(ce_algs); i++) { + ce_algs[i].ce = ce; + switch (ce_algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + id = ce_algs[i].ce_algo_id; + ce_method = ce->variant->alg_cipher[id]; + if (ce_method == CE_ID_NOTSUPP) { + dev_info(ce->dev, + "DEBUG: Algo of %s not supported\n", + ce_algs[i].alg.skcipher.base.cra_name); + ce_algs[i].ce = NULL; + break; + } + id = ce_algs[i].ce_blockmode; + ce_method = ce->variant->op_mode[id]; + if (ce_method == CE_ID_NOTSUPP) { + dev_info(ce->dev, "DEBUG: Blockmode of %s not supported\n", + ce_algs[i].alg.skcipher.base.cra_name); + ce_algs[i].ce = NULL; + break; + } + dev_info(ce->dev, "DEBUG: Register %s\n", + ce_algs[i].alg.skcipher.base.cra_name); + err = crypto_register_skcipher(&ce_algs[i].alg.skcipher); + if (err) { + dev_err(ce->dev, "Fail to register %s\n", + ce_algs[i].alg.skcipher.base.cra_name); + ce_algs[i].ce = NULL; + goto error_alg; + } + break; + default: + dev_err(ce->dev, "ERROR: tryed to register an unknown algo\n"); + } + } + + return 0; +error_alg: + i--; + for (; i >= 0; i--) { + switch (ce_algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + if (ce_algs[i].ce) + crypto_unregister_skcipher(&ce_algs[i].alg.skcipher); + break; + } + } +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG +error_debugfs: + debugfs_remove_recursive(ce->dbgfs_dir); +#endif + i = ce->variant->maxflow; +error_engine: + while (i >= 0) { + crypto_engine_exit(ce->chanlist[i].engine); + if (ce->chanlist[i].tl) + dma_free_coherent(ce->dev, sizeof(struct ce_task), + ce->chanlist[i].tl, + ce->chanlist[i].t_phy); + i--; + } +error_flow: + reset_control_assert(ce->reset); +error_clk: + for (i = 0; i < CE_MAX_CLOCKS; i++) + clk_disable_unprepare(ce->ceclks[i]); + return err; +} + +static int sun8i_ce_remove(struct platform_device *pdev) +{ + int i, timeout; + struct sun8i_ce_dev *ce = platform_get_drvdata(pdev); + + for (i = 0; i < ARRAY_SIZE(ce_algs); i++) { + switch (ce_algs[i].type) { + case CRYPTO_ALG_TYPE_SKCIPHER: + if (ce_algs[i].ce) { + dev_dbg(ce->dev, "Unregister %d %s\n", i, + ce_algs[i].alg.skcipher.base.cra_name); + crypto_unregister_skcipher(&ce_algs[i].alg.skcipher); + } + break; + } + } + +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + debugfs_remove_recursive(ce->dbgfs_dir); +#endif + + for (i = 0; i < ce->variant->maxflow; i++) { + crypto_engine_exit(ce->chanlist[i].engine); + timeout = 0; + while (mutex_is_locked(&ce->chanlist[i].lock) && timeout < 10) { + dev_info(ce->dev, "Wait for %d %d\n", i, timeout); + timeout++; + msleep(20); + } + dma_free_coherent(ce->dev, sizeof(struct ce_task), + ce->chanlist[i].tl, + ce->chanlist[i].t_phy); + } + + reset_control_assert(ce->reset); + for (i = 0; i < CE_MAX_CLOCKS; i++) + clk_disable_unprepare(ce->ceclks[i]); + return 0; +} + +static const struct of_device_id sun8i_ce_crypto_of_match_table[] = { + { .compatible = "allwinner,sun8i-h3-crypto", + .data = &ce_h3_variant }, + { .compatible = "allwinner,sun50i-h5-crypto", + .data = &ce_h5_variant }, + { .compatible = "allwinner,sun50i-h6-crypto", + .data = &ce_h6_variant }, + { .compatible = "allwinner,sun50i-a64-crypto", + .data = &ce_a64_variant }, + { .compatible = "allwinner,sun8i-r40-crypto", + .data = &ce_r40_variant }, + {} +}; +MODULE_DEVICE_TABLE(of, sun8i_ce_crypto_of_match_table); + +static struct platform_driver sun8i_ce_driver = { + .probe = sun8i_ce_probe, + .remove = sun8i_ce_remove, + .driver = { + .name = "sun8i-ce", + .of_match_table = sun8i_ce_crypto_of_match_table, + }, +}; + +module_platform_driver(sun8i_ce_driver); + +MODULE_DESCRIPTION("Allwinner Crypto Engine cryptographic offloader"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Corentin Labbe "); diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h new file mode 100644 index 000000000000..fe51f9569fa0 --- /dev/null +++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce.h @@ -0,0 +1,256 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * sun8i-ce.h - hardware cryptographic offloader for + * Allwinner H3/A64/H5/H2+/H6 SoC + * + * Copyright (C) 2016-2019 Corentin LABBE + */ +#include +#include +#include +#include +#include +#include +#include + +#define MAX_SG 8 + +#define CE_STD 0 +#define CE_v2 1 + +#define CE_MAX_CLOCKS 3 + +/* CE Registers */ +#define CE_TDQ 0x00 +#define CE_CTR 0x04 +#define CE_ICR 0x08 +#define CE_ISR 0x0C +#define CE_TLR 0x10 +#define CE_TSR 0x14 +#define CE_ESR 0x18 +#define CE_CSSGR 0x1C +#define CE_CDSGR 0x20 +#define CE_CSAR 0x24 +#define CE_CDAR 0x28 +#define CE_TPR 0x2C + +/* Used in struct ce_task */ +/* ce_task common */ +#define CE_ENCRYPTION 0 +#define CE_DECRYPTION BIT(8) + +#define CE_COMM_INT BIT(31) + +/* ce_task symmetric */ +#define CE_AES_128BITS 0 +#define CE_AES_192BITS 1 +#define CE_AES_256BITS 2 + +#define CE_OP_ECB 0 +#define CE_OP_CBC (1 << 8) + +#define CE_ALG_AES 0 +#define CE_ALG_DES 1 +#define CE_ALG_3DES 2 +#define CE_ALG_RAES 48 + +/* Used in ce_variant */ +#define CE_ID_NOTSUPP 0xFF + +#define CE_ID_CIPHER_AES 1 +#define CE_ID_CIPHER_DES 2 +#define CE_ID_CIPHER_DES3 3 +#define CE_ID_CIPHER_RAES 4 +#define CE_ID_CIPHER_MAX 5 + +#define CE_ID_OP_ECB 1 +#define CE_ID_OP_CBC 2 +#define CE_ID_OP_MAX 3 + +/* Used in CE registers */ +#define CE_ERR_ALGO_NOTSUP BIT(0) +#define CE_ERR_DATALEN BIT(1) +#define CE_ERR_KEYSRAM BIT(2) +#define CE_ERR_ADDR_INVALID BIT(5) +#define CE_ERR_KEYLADDER BIT(6) + +/* + * struct ce_clock - Describe clocks used by sun8i-ce + * @name: Name of clock needed by this variant + * @freq: Maximum frequency for each clock + */ +struct ce_clock { + const char *name; + unsigned long freq; +}; + +/* + * struct ce_variant - Describe CE capability for each variant hardware + * @alg_cipher: list of supported ciphers. for each CE_ID_ this will give the + * coresponding CE_ALG_XXX/SS_ALG_XXX value + * @op_mode: list of supported block modes + * @model: The minor variant CE_STD/CE_SS/CE_v2 + * @intreg: reg offset for Interrupt register + * @maxflow: Numbers of flow for the current engine + */ +struct ce_variant { + char alg_cipher[CE_ID_CIPHER_MAX]; + u32 op_mode[CE_ID_OP_MAX]; + int model; + u32 intreg; + unsigned int maxflow; + struct ce_clock ce_clks[CE_MAX_CLOCKS]; +}; + +struct sginfo { + u32 addr; + u32 len; +} __packed; + +/* + * struct ce_task - CE Task descriptor + * The structure of this descriptor could be found in the datasheet + */ +struct ce_task { + u32 t_id; + u32 t_common_ctl; + u32 t_sym_ctl; + u32 t_asym_ctl; + u32 t_key; + u32 t_iv; + u32 t_ctr; + u32 t_dlen; + struct sginfo t_src[MAX_SG]; + struct sginfo t_dst[MAX_SG]; + u32 next; + u32 reserved[3]; +} __packed __aligned(8); + +/* + * struct sun8i_ce_flow - Information used by each flow + * @lock: lock protecting access of sun8i_ce_flow + * @engine: ptr to the crypto_engine for this flow + * @bounce_iv: buffer which contain the IV + * @ivlen: size of bounce_iv + * @keylen: keylen for this flow operation + * @complete: completion for the current task on this flow + * @status: set to 1 by interrupt if task is done + * @method: current method for flow + * @op_dir: direction (encrypt vs decrypt) of this flow + * @op_mode: op_mode for this flow + * @t_phy: Physical address of task + * @tl: pointer to the current ce_task for this flow + * @stat_req: number of request done by this flow + */ +struct sun8i_ce_flow { + struct mutex lock; + struct crypto_engine *engine; + void *bounce_iv; + unsigned int ivlen; + unsigned int keylen; + struct completion complete; + int status; + u32 method; + u32 op_dir; + u32 op_mode; + dma_addr_t t_phy; + int timeout; + struct ce_task *tl; +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + unsigned long stat_req; +#endif +}; + +/* + * struct sun8i_ce_dev - main container for all this driver information + * @base: base address of SS/CE + * @ceclks: clocks used by SS/CE + * @reset: pointer to reset controller + * @dev: the platform device + * @mlock: Control access to device registers + * @chanlist: array of all flow + * @flow: flow to use in next request + * @variant: pointer to variant specific data + * @dbgfs_dir: Debugfs dentry for statistic directory + * @dbgfs_stats: Debugfs dentry for statistic counters + */ +struct sun8i_ce_dev { + void __iomem *base; + struct clk *ceclks[CE_MAX_CLOCKS]; + struct reset_control *reset; + struct device *dev; + struct mutex mlock; + struct sun8i_ce_flow *chanlist; + atomic_t flow; + const struct ce_variant *variant; +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + struct dentry *dbgfs_dir; + struct dentry *dbgfs_stats; +#endif +}; + +/* + * struct sun8i_cipher_req_ctx - context for a skcipher request + * @op_dir: direction (encrypt vs decrypt) for this request + * @flow: the flow to use for this request + */ +struct sun8i_cipher_req_ctx { + u32 op_dir; + int flow; +}; + +/* + * struct sun8i_cipher_tfm_ctx - context for a skcipher TFM + * @enginectx: crypto_engine used by this TFM + * @key: pointer to key data + * @keylen: len of the key + * @ce: pointer to the private data of driver handling this TFM + * @fallback_tfm: pointer to the fallback TFM + */ +struct sun8i_cipher_tfm_ctx { + struct crypto_engine_ctx enginectx; + u32 *key; + u32 keylen; + struct sun8i_ce_dev *ce; + struct crypto_sync_skcipher *fallback_tfm; +}; + +/* + * struct sun8i_ce_alg_template - crypto_alg template + * @type: the CRYPTO_ALG_TYPE for this template + * @ce_algo_id: the CE_ID for this template + * @ce_blockmode: the type of block operation CE_ID + * @ce: pointer to the sun8i_ce_dev structure associated with + * this template + * @alg: one of sub struct must be used + * @stat_req: number of request done on this template + * @stat_fb: total of all data len done on this template + */ +struct sun8i_ce_alg_template { + u32 type; + u32 ce_algo_id; + u32 ce_blockmode; + struct sun8i_ce_dev *ce; + union { + struct skcipher_alg skcipher; + } alg; +#ifdef CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG + unsigned long stat_req; + unsigned long stat_fb; +#endif +}; + +int sun8i_ce_enqueue(struct crypto_async_request *areq, u32 type); + +int sun8i_ce_aes_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keylen); +int sun8i_ce_des3_setkey(struct crypto_skcipher *tfm, const u8 *key, + unsigned int keylen); +int sun8i_ce_cipher_init(struct crypto_tfm *tfm); +void sun8i_ce_cipher_exit(struct crypto_tfm *tfm); +int sun8i_ce_skdecrypt(struct skcipher_request *areq); +int sun8i_ce_skencrypt(struct skcipher_request *areq); + +int sun8i_ce_get_engine_number(struct sun8i_ce_dev *ce); + +int sun8i_ce_run_task(struct sun8i_ce_dev *ce, int flow, const char *name);