From patchwork Wed May 27 12:24:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tero Kristo X-Patchwork-Id: 197707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7468DC433E1 for ; Wed, 27 May 2020 12:24:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49F0A20899 for ; Wed, 27 May 2020 12:24:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="mum9C7zN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729318AbgE0MYr (ORCPT ); Wed, 27 May 2020 08:24:47 -0400 Received: from lelv0143.ext.ti.com ([198.47.23.248]:41250 "EHLO lelv0143.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729288AbgE0MYq (ORCPT ); Wed, 27 May 2020 08:24:46 -0400 Received: from fllv0035.itg.ti.com ([10.64.41.0]) by lelv0143.ext.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOeuU078813; Wed, 27 May 2020 07:24:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1590582280; bh=os45j/rYJrQJWrLHX/zyT3cvsjFLUypv+9XW8LkowNw=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=mum9C7zNiGIsyz0RgL/jULwEEISURCMgTLQejBhoCUyUYEymPLLriL0OnkufuOBpd aeYJv25xIZ8uFhKkJkE9FlzUd1Ezpy68Lgq7Kln8WjQRFi1h6i5A6kwL9GGrvMSZsQ x4zTVfRY/Ok2MHLjRNo9mcOTRu29M+mathUMusMk= Received: from DLEE109.ent.ti.com (dlee109.ent.ti.com [157.170.170.41]) by fllv0035.itg.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOeC7064282; Wed, 27 May 2020 07:24:40 -0500 Received: from DLEE104.ent.ti.com (157.170.170.34) by DLEE109.ent.ti.com (157.170.170.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3; Wed, 27 May 2020 07:24:40 -0500 Received: from fllv0040.itg.ti.com (10.64.41.20) by DLEE104.ent.ti.com (157.170.170.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3 via Frontend Transport; Wed, 27 May 2020 07:24:40 -0500 Received: from sokoban.ti.com (ileax41-snat.itg.ti.com [10.172.224.153]) by fllv0040.itg.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOYRg101979; Wed, 27 May 2020 07:24:39 -0500 From: Tero Kristo To: , , CC: Subject: [RESEND PATCHv3 3/7] crypto: omap-crypto: fix userspace copied buffer access Date: Wed, 27 May 2020 15:24:25 +0300 Message-ID: <20200527122429.14888-4-t-kristo@ti.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200527122429.14888-1-t-kristo@ti.com> References: <20200527122429.14888-1-t-kristo@ti.com> MIME-Version: 1.0 X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org In case buffers are copied from userspace, directly accessing the page will most likely fail because it hasn't been mapped into the kernel memory space. Fix the issue by forcing a kmap / kunmap within the cleanup functionality. Signed-off-by: Tero Kristo --- v3: - Added PageSlab() check to the cache flushing portion, and changed the used flush API to be flush_kernel_dcache_page() drivers/crypto/omap-crypto.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/omap-crypto.c b/drivers/crypto/omap-crypto.c index cc88b7362bc2..94b2dba90f0d 100644 --- a/drivers/crypto/omap-crypto.c +++ b/drivers/crypto/omap-crypto.c @@ -178,11 +178,17 @@ static void omap_crypto_copy_data(struct scatterlist *src, amt = min(src->length - srco, dst->length - dsto); amt = min(len, amt); - srcb = sg_virt(src) + srco; - dstb = sg_virt(dst) + dsto; + srcb = kmap_atomic(sg_page(src)) + srco + src->offset; + dstb = kmap_atomic(sg_page(dst)) + dsto + dst->offset; memcpy(dstb, srcb, amt); + if (!PageSlab(sg_page(dst))) + flush_kernel_dcache_page(sg_page(dst)); + + kunmap_atomic(srcb); + kunmap_atomic(dstb); + srco += amt; dsto += amt; len -= amt; From patchwork Wed May 27 12:24:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tero Kristo X-Patchwork-Id: 197706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33F3EC433E1 for ; Wed, 27 May 2020 12:24:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BA4D20899 for ; Wed, 27 May 2020 12:24:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="uOHn1K/c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729396AbgE0MYt (ORCPT ); Wed, 27 May 2020 08:24:49 -0400 Received: from fllv0016.ext.ti.com ([198.47.19.142]:37686 "EHLO fllv0016.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729352AbgE0MYs (ORCPT ); Wed, 27 May 2020 08:24:48 -0400 Received: from lelv0266.itg.ti.com ([10.180.67.225]) by fllv0016.ext.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOgv1026175; Wed, 27 May 2020 07:24:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1590582282; bh=D+FkBTnlxhFt8rwKSbgbew41pOYqmQRAEPiRqzTmXOQ=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=uOHn1K/coqZHF5iy0g7XcARebKAI7VNebv7o5r1ZT1+bRcbyRvQP0KAlVggYkrHau 36yQWfSi5Rfae03d77wuKBaVNQfgc9eBrFAS4jrJ4qiOjKyQRbVGOKXTzpvJw0PHUp Lgmqp40hpaaibJVekvWQtItkb7G/rh7X6pKZ09zI= Received: from DFLE101.ent.ti.com (dfle101.ent.ti.com [10.64.6.22]) by lelv0266.itg.ti.com (8.15.2/8.15.2) with ESMTPS id 04RCOg1G039936 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 27 May 2020 07:24:42 -0500 Received: from DFLE110.ent.ti.com (10.64.6.31) by DFLE101.ent.ti.com (10.64.6.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3; Wed, 27 May 2020 07:24:41 -0500 Received: from fllv0040.itg.ti.com (10.64.41.20) by DFLE110.ent.ti.com (10.64.6.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3 via Frontend Transport; Wed, 27 May 2020 07:24:41 -0500 Received: from sokoban.ti.com (ileax41-snat.itg.ti.com [10.172.224.153]) by fllv0040.itg.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOYRh101979; Wed, 27 May 2020 07:24:40 -0500 From: Tero Kristo To: , , CC: Subject: [RESEND PATCHv3 4/7] crypto: omap-sham: huge buffer access fixes Date: Wed, 27 May 2020 15:24:26 +0300 Message-ID: <20200527122429.14888-5-t-kristo@ti.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200527122429.14888-1-t-kristo@ti.com> References: <20200527122429.14888-1-t-kristo@ti.com> MIME-Version: 1.0 X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The ctx internal buffer can only hold buflen amount of data, don't try to copy over more than that. Also, initialize the context sg pointer if we only have data in the context internal buffer, this can happen when closing a hash with certain data amounts. Signed-off-by: Tero Kristo --- drivers/crypto/omap-sham.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c index 0c837bbd8f0c..9823d7dfca9c 100644 --- a/drivers/crypto/omap-sham.c +++ b/drivers/crypto/omap-sham.c @@ -751,8 +751,15 @@ static int omap_sham_align_sgs(struct scatterlist *sg, int offset = rctx->offset; int bufcnt = rctx->bufcnt; - if (!sg || !sg->length || !nbytes) + if (!sg || !sg->length || !nbytes) { + if (bufcnt) { + sg_init_table(rctx->sgl, 1); + sg_set_buf(rctx->sgl, rctx->dd->xmit_buf, bufcnt); + rctx->sg = rctx->sgl; + } + return 0; + } new_len = nbytes; @@ -896,7 +903,7 @@ static int omap_sham_prepare_request(struct ahash_request *req, bool update) if (hash_later < 0) hash_later = 0; - if (hash_later) { + if (hash_later && hash_later <= rctx->buflen) { scatterwalk_map_and_copy(rctx->buffer, req->src, req->nbytes - hash_later, From patchwork Wed May 27 12:24:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tero Kristo X-Patchwork-Id: 197705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0300C433E0 for ; Wed, 27 May 2020 12:24:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E19420899 for ; Wed, 27 May 2020 12:24:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="v930t/NN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729410AbgE0MYy (ORCPT ); Wed, 27 May 2020 08:24:54 -0400 Received: from fllv0016.ext.ti.com ([198.47.19.142]:37690 "EHLO fllv0016.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729407AbgE0MYx (ORCPT ); Wed, 27 May 2020 08:24:53 -0400 Received: from fllv0035.itg.ti.com ([10.64.41.0]) by fllv0016.ext.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOl31026187; Wed, 27 May 2020 07:24:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1590582287; bh=DmuTt/oLL9bLxYzTZA57/askvQvzJwvHmWkkULIzNXk=; h=From:To:CC:Subject:Date:In-Reply-To:References; b=v930t/NNvDfqHLXHVdPhA3194dE7FyckD0FvWWopISDVCnIKNzzfRP5LUwPYduIxu fXBHszBZL5YKLboUj0LcYnc6RVemKZ0tKSwqLa3Z7FYg+B+ADaOney2vYtc1I5PAxb rXyoq45A5+0LstYtwv16EmEh/aP0DlFX/6bW7fuc= Received: from DLEE115.ent.ti.com (dlee115.ent.ti.com [157.170.170.26]) by fllv0035.itg.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOlYj064346; Wed, 27 May 2020 07:24:47 -0500 Received: from DLEE112.ent.ti.com (157.170.170.23) by DLEE115.ent.ti.com (157.170.170.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3; Wed, 27 May 2020 07:24:46 -0500 Received: from fllv0040.itg.ti.com (10.64.41.20) by DLEE112.ent.ti.com (157.170.170.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1979.3 via Frontend Transport; Wed, 27 May 2020 07:24:46 -0500 Received: from sokoban.ti.com (ileax41-snat.itg.ti.com [10.172.224.153]) by fllv0040.itg.ti.com (8.15.2/8.15.2) with ESMTP id 04RCOYRk101979; Wed, 27 May 2020 07:24:45 -0500 From: Tero Kristo To: , , CC: Subject: [RESEND PATCHv3 7/7] crypto: omap-sham: add proper load balancing support for multicore Date: Wed, 27 May 2020 15:24:29 +0300 Message-ID: <20200527122429.14888-8-t-kristo@ti.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200527122429.14888-1-t-kristo@ti.com> References: <20200527122429.14888-1-t-kristo@ti.com> MIME-Version: 1.0 X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org The current implementation of the multiple accelerator core support for OMAP SHA does not work properly. It always picks up the first probed accelerator core if this is available, and rest of the book keeping also gets confused if there are two cores available. Add proper load balancing support for SHA, and also fix any bugs related to the multicore support while doing it. Signed-off-by: Tero Kristo --- drivers/crypto/omap-sham.c | 64 ++++++++++++++++++-------------------- 1 file changed, 31 insertions(+), 33 deletions(-) diff --git a/drivers/crypto/omap-sham.c b/drivers/crypto/omap-sham.c index 86949f1ac6a7..0651604161ea 100644 --- a/drivers/crypto/omap-sham.c +++ b/drivers/crypto/omap-sham.c @@ -169,8 +169,6 @@ struct omap_sham_hmac_ctx { }; struct omap_sham_ctx { - struct omap_sham_dev *dd; - unsigned long flags; /* fallback stuff */ @@ -935,27 +933,35 @@ static int omap_sham_update_dma_stop(struct omap_sham_dev *dd) return 0; } +struct omap_sham_dev *omap_sham_find_dev(struct omap_sham_reqctx *ctx) +{ + struct omap_sham_dev *dd; + + if (ctx->dd) + return ctx->dd; + + spin_lock_bh(&sham.lock); + dd = list_first_entry(&sham.dev_list, struct omap_sham_dev, list); + list_move_tail(&dd->list, &sham.dev_list); + ctx->dd = dd; + spin_unlock_bh(&sham.lock); + + return dd; +} + static int omap_sham_init(struct ahash_request *req) { struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); struct omap_sham_ctx *tctx = crypto_ahash_ctx(tfm); struct omap_sham_reqctx *ctx = ahash_request_ctx(req); - struct omap_sham_dev *dd = NULL, *tmp; + struct omap_sham_dev *dd; int bs = 0; - spin_lock_bh(&sham.lock); - if (!tctx->dd) { - list_for_each_entry(tmp, &sham.dev_list, list) { - dd = tmp; - break; - } - tctx->dd = dd; - } else { - dd = tctx->dd; - } - spin_unlock_bh(&sham.lock); + ctx->dd = NULL; - ctx->dd = dd; + dd = omap_sham_find_dev(ctx); + if (!dd) + return -ENODEV; ctx->flags = 0; @@ -1225,8 +1231,7 @@ static int omap_sham_handle_queue(struct omap_sham_dev *dd, static int omap_sham_enqueue(struct ahash_request *req, unsigned int op) { struct omap_sham_reqctx *ctx = ahash_request_ctx(req); - struct omap_sham_ctx *tctx = crypto_tfm_ctx(req->base.tfm); - struct omap_sham_dev *dd = tctx->dd; + struct omap_sham_dev *dd = ctx->dd; ctx->op = op; @@ -1236,7 +1241,7 @@ static int omap_sham_enqueue(struct ahash_request *req, unsigned int op) static int omap_sham_update(struct ahash_request *req) { struct omap_sham_reqctx *ctx = ahash_request_ctx(req); - struct omap_sham_dev *dd = ctx->dd; + struct omap_sham_dev *dd = omap_sham_find_dev(ctx); if (!req->nbytes) return 0; @@ -1340,21 +1345,8 @@ static int omap_sham_setkey(struct crypto_ahash *tfm, const u8 *key, struct omap_sham_hmac_ctx *bctx = tctx->base; int bs = crypto_shash_blocksize(bctx->shash); int ds = crypto_shash_digestsize(bctx->shash); - struct omap_sham_dev *dd = NULL, *tmp; int err, i; - spin_lock_bh(&sham.lock); - if (!tctx->dd) { - list_for_each_entry(tmp, &sham.dev_list, list) { - dd = tmp; - break; - } - tctx->dd = dd; - } else { - dd = tctx->dd; - } - spin_unlock_bh(&sham.lock); - err = crypto_shash_setkey(tctx->fallback, key, keylen); if (err) return err; @@ -1372,7 +1364,7 @@ static int omap_sham_setkey(struct crypto_ahash *tfm, const u8 *key, memset(bctx->ipad + keylen, 0, bs - keylen); - if (!test_bit(FLAGS_AUTO_XOR, &dd->flags)) { + if (!test_bit(FLAGS_AUTO_XOR, &sham.flags)) { memcpy(bctx->opad, bctx->ipad, bs); for (i = 0; i < bs; i++) { @@ -2184,6 +2176,7 @@ static int omap_sham_probe(struct platform_device *pdev) } dd->flags |= dd->pdata->flags; + sham.flags |= dd->pdata->flags; pm_runtime_use_autosuspend(dev); pm_runtime_set_autosuspend_delay(dev, DEFAULT_AUTOSUSPEND_DELAY); @@ -2211,6 +2204,9 @@ static int omap_sham_probe(struct platform_device *pdev) spin_unlock(&sham.lock); for (i = 0; i < dd->pdata->algs_info_size; i++) { + if (dd->pdata->algs_info[i].registered) + break; + for (j = 0; j < dd->pdata->algs_info[i].size; j++) { struct ahash_alg *alg; @@ -2262,9 +2258,11 @@ static int omap_sham_remove(struct platform_device *pdev) list_del(&dd->list); spin_unlock(&sham.lock); for (i = dd->pdata->algs_info_size - 1; i >= 0; i--) - for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) + for (j = dd->pdata->algs_info[i].registered - 1; j >= 0; j--) { crypto_unregister_ahash( &dd->pdata->algs_info[i].algs_list[j]); + dd->pdata->algs_info[i].registered--; + } tasklet_kill(&dd->done_task); pm_runtime_disable(&pdev->dev);