From patchwork Wed Aug 25 07:15:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hemant Agrawal X-Patchwork-Id: 502243 Delivered-To: patch@linaro.org Received: by 2002:a02:6f15:0:0:0:0:0 with SMTP id x21csp346016jab; Wed, 25 Aug 2021 00:20:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzpUWVHuyCHU4KtEjOx/kC5vw+dgZK2ozMxrVjtTw0jfp/N7TQw+Lq2m+WKcqgmpwszgfKX X-Received: by 2002:a17:906:b745:: with SMTP id fx5mr23709041ejb.515.1629876008774; Wed, 25 Aug 2021 00:20:08 -0700 (PDT) Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id o7si3546316edc.530.2021.08.25.00.20.08; Wed, 25 Aug 2021 00:20:08 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; dkim=fail header.i=@nxp.com header.s=selector2 header.b="ceGw/3vV"; arc=fail (signature failed); spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 99C8641212; Wed, 25 Aug 2021 09:19:18 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00065.outbound.protection.outlook.com [40.107.0.65]) by mails.dpdk.org (Postfix) with ESMTP id 1476540140 for ; Wed, 25 Aug 2021 09:19:14 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U4v4t0WjfCKCIhjBNbgXZOT+ErpMamu1LpfwDb/85ZMDNrFz7EFcANiOMpFW0UtzIFIaunn959b6sbvazj7Bf+0wueYMFnkn/LPLZgvj8mNcuQAzluugHwHg6oeMEq9hL95JEJEKf/PHG0XBve3CwB1dc0gaFPTWzrLCGhD9iN3ftq1ZO/Tgd9VXITmFB+/BZeySUVPiRNR3cxIcPS7iz276ekCffAytvUoJBIT0SgXfjGz1yuQE+Ubl0LlYxjQWAmLKBqMcP+kvGlPseDFUCruD8jB6gm2Dk+brHthEpARoWgZhHyR+tTUdYbdDbWefbUDVO7ogzA5pLPLDtAvn5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YqaXcWZmzes9i4Dns5vYrmIBnikEsdWyaDkbLdillH4=; b=PTzfImz4x/6olLxI26y32JsXMM1dkTEAtAoLq0y2SIPSJeT6QDWykRX88W4QyVBpz/+CUVaqCZ01pYLnKYPJLiOjIdTU3oC2+NsymRUn+aWJi4nd1kPEZ+l4symWvsKdCoXR4tdtFv4BdjHLSJemcxNGiQPD699AnjvROXdE2Cy6+/YMu5QaBQ8v0DycEZxD1RSu0W4GaUctYHelQc/n+rN4uagyQnTawj9Ew7bG2/3sZy0mBhx25XPTGzsj96FpZPN+VKgVgp3se9nbqSwCN2poKwPG+z1XjuzrKs26Buer6PkKvfMak2M8yJteMV2Wmydz4cQ1jc2yTqh1E2CEgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YqaXcWZmzes9i4Dns5vYrmIBnikEsdWyaDkbLdillH4=; b=ceGw/3vVwxlbVJXEberFqKVTSXPHr9jomaD+teJgIB0o61n2J6oScZZHVeG3SQOHmnh3AQ1nBrK7FrNw6+iyLpJafDfLA8Kvr52mRiUw+XEapQ2BMM09vQiGx6Xrw2Mnrs5tmUJtjnCnQvHJWPgakAQP5QN3kJYSvKEBTOsJkIo= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=nxp.com; Received: from DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) by DU2PR04MB8872.eurprd04.prod.outlook.com (2603:10a6:10:2e3::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19; Wed, 25 Aug 2021 07:19:12 +0000 Received: from DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::2590:514a:df1f:a793]) by DU2PR04MB8630.eurprd04.prod.outlook.com ([fe80::2590:514a:df1f:a793%8]) with mapi id 15.20.4436.025; Wed, 25 Aug 2021 07:19:12 +0000 From: Hemant Agrawal To: dev@dpdk.org, gakhil@marvell.com Cc: konstantin.ananyev@intel.com, roy.fan.zhang@intel.com, Gagandeep Singh Date: Wed, 25 Aug 2021 12:45:07 +0530 Message-Id: <20210825071510.7913-13-hemant.agrawal@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210825071510.7913-1-hemant.agrawal@nxp.com> References: <20210812071244.28799-1-hemant.agrawal@nxp.com> <20210825071510.7913-1-hemant.agrawal@nxp.com> X-ClientProxiedBy: SG2PR01CA0126.apcprd01.prod.exchangelabs.com (2603:1096:4:40::30) To DU2PR04MB8630.eurprd04.prod.outlook.com (2603:10a6:10:2dd::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from dpdk-xeon.ap.freescale.net (92.120.0.67) by SG2PR01CA0126.apcprd01.prod.exchangelabs.com (2603:1096:4:40::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19 via Frontend Transport; Wed, 25 Aug 2021 07:19:10 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aab1b1b8-0314-411e-d199-08d96798a384 X-MS-TrafficTypeDiagnostic: DU2PR04MB8872: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1443; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WHcjxJN8S/dHAGeyn4XiJddsbRYTffrwfeZaYPnZx5t9K9Df0sEswNZuxbR34J9Zz0Qx/BAQ9+SYZA7my3m2W9/MqQkTWNUmGlIA6+iduEohcO3h725WS6bjIgYJj4S2oaiNzFzz1HWyX6Y9LPkqjZbOllRvYqX/25oxy3hauHLjDcgBI/zXlm9Xhf6dmX1HY85IzM4u+KZpbFEs1p8mz72k3PyKRMNrRlU50iLt0pBoiIjpe58nxQU3QxwbLuW5CUPRCeN711PTbCTbooA1SryyAMxE6ST84hLdwhosbT9eSpTyiOiu6prBie6yVwi4vxy7VorhNhkw1RbpsvCr4gYbI7+yc4YljWdFhArrgEYdqkdJYmPlXiVS5q2CK7o8mY/lQithrzgpqglc3/j07yCUMNDawAxBagH4rMYqhDAT1tvCaljNQBKLMjWuZ0lBpa73E1pl5O21GQ7JBytPrmp0EszcJNU8PfXqHcsA7mA4wCiwv9zbZ4etf0iyMCIl3zLHi2gWvTY5lqaKLuXuNFZ+dOQnVfBfJHF6lH66g06OG9Ogu8tyUP3rGio0dkN9wW6kWw0kLxwuTOcl0bELn2ay+m4dWlNJAztni+32zSx4IF2pXK8+cMZP50JVpxvkWdJKeQ/7QVv5MQcSMZ/5sX6PWMVh0wksHQHsv1zwfY8qNDTq7BxreOa4HvS5Mbi9gbgScr4na2SLqBB3/ZA45A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DU2PR04MB8630.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(366004)(39860400002)(376002)(396003)(26005)(1076003)(478600001)(8936002)(38350700002)(38100700002)(5660300002)(316002)(8676002)(6666004)(2906002)(6486002)(956004)(83380400001)(186003)(52116002)(2616005)(86362001)(6512007)(4326008)(36756003)(6506007)(66946007)(44832011)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BW1b5X4oDO38DXekqebAlkVBlUmYMrP/mu0L78MuQZiJwuqY/cuJieLBaCW0H0TBQCoeqVHcmUTyS7EpHsEo28kbstcCtgUOgOMInAxKjn8uGqIlmutQoiFs2zNiz1VrUD6eHqFx2E6oJ8PQ8BiBH8bpwHEAnnWk4A/cHsLezsNvtUFWMsVLo74MqeGduYY8wTK0Auj+myQovo+ud2Kqa1o+9FOxrNyrORcFIbTsx7QJg7nb+llV5As5klt9KDvyuP2XH8J6NxP+8sXA/S/pUuoEGRgPFGfKwddIUXzBjbhjdep4ZmSo0jF9v6HvsHBIWKLSDji6GWv32cSQrLq3dx5XqqkHGCJD8GAqpnq7g1e7ypRc32s944JrIPE5m+gGJankFat1QItVheH0F8Qqf8ef73NQHhg3S56XeYoZMlMMX6jvjDLLzXLZ3NbMyEjvvSgSmjFIEKw3nTLoNimhMRY5t36x5FURsYMDB/qzFUs+kIfY5SyTxEHGdUW2uekt8iTYXnGIhErwNxmoQqFmonjgoe21QE1l8fh57T5MdZ+7oQGSvrTfLvE4Qx825NTPbjUYtb4krNs+zltr5wwsNrdgspNHLDXovSy1OTTR5Lgghm+AEDnRSCM2gjfGwKd0zAKj/PxQ8TfHEAIA57NwnULorBZUybl5IJqA7fvRGVgtBz/5ocpgY1t2sbn3Usf6OeUOXWNaiz9ereKtyUHYYbbN44BadpV6f9b3F+YQPuPGxadwikXQP9GU5HK2d5AXJZLvBgMIvkJjHlw6hTvjTzAlzWMWv3QlkD8Aq6qFg9+Db6PYz/KpLcBhudUJsuxLlhWxaFKCf2OaNBWlcArR5mQcmSVPYl9ZpoUYcPS+9niOh6xiMGoAS7oV3n7dPoHjg3AyDBy53fmEk+gpV9Qm8LgXHJfqUTEfPUW5gap5THwDkG5uHPlOdzmY1RhGMNEYOmc7h2l6EYyd+dqAcZuxzKomN2ruHrK7LNMgDtiEyREIkFfOvRQRX/dQDbm1luBE7iw+p1JzoJa2Palmo5nk5eNlKXk5VraQ/ilcRtZuSw4qfYBRFCK8D2/5QJeH+Ev9Rozx/kiBGlAvYwJ3pmqx7ctVKwqOxSNpiHD8BxC2cFyR+/hfNAPv3KDGKVoiu+bkDKMWS/i99/To+bj1QP7F4pc0KYZGs89YL4TIlGwq15YteFChBJl4C5/QOlF7OpfzDeQyQzuluGMs2Pd3M0CAJCeSykm4fETnyaEWxRkYTbLlLnqMInHK/quvufbDdOLpEUrCLYWJExqm4n2O8x+fwPP5OldBWUQx4UbbCSFBW6WlIjd5r7/pi5svWcjEvQrt X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: aab1b1b8-0314-411e-d199-08d96798a384 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8630.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2021 07:19:12.7886 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yRivt5UB7FSBrY4G6pQ4Qgmc5ZPNgTZUM+lZFbHvaj0TOJSTnrLJTd2klz29mb0xtwtkcMlW3NINbRUq4E6smw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8872 Subject: [dpdk-dev] [PATCH 12/15] crypto/dpaa_sec: support authonly and chain with raw APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Gagandeep Singh This patch improves the raw vector support in dpaa_sec driver for authonly and chain usecase. Signed-off-by: Gagandeep Singh --- drivers/crypto/dpaa_sec/dpaa_sec.h | 3 +- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 296 +++++++++++++++++++++- 2 files changed, 287 insertions(+), 12 deletions(-) -- 2.17.1 diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.h b/drivers/crypto/dpaa_sec/dpaa_sec.h index f6e83d46e7..2e0ab93ff0 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.h +++ b/drivers/crypto/dpaa_sec/dpaa_sec.h @@ -135,7 +135,8 @@ typedef struct dpaa_sec_job* (*dpaa_sec_build_raw_dp_fd_t)(uint8_t *drv_ctx, struct rte_crypto_va_iova_ptr *digest, struct rte_crypto_va_iova_ptr *auth_iv, union rte_crypto_sym_ofs ofs, - void *userdata); + void *userdata, + struct qm_fd *fd); typedef struct dpaa_sec_session_entry { struct sec_cdb cdb; /**< cmd block associated with qp */ diff --git a/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c b/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c index ee0ca2e0d5..4e34629f18 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c @@ -12,6 +12,7 @@ #endif /* RTA header files */ +#include #include #include @@ -26,6 +27,17 @@ struct dpaa_sec_raw_dp_ctx { uint16_t cached_dequeue; }; +static inline int +is_encode(dpaa_sec_session *ses) +{ + return ses->dir == DIR_ENC; +} + +static inline int is_decode(dpaa_sec_session *ses) +{ + return ses->dir == DIR_DEC; +} + static __rte_always_inline int dpaa_sec_raw_enqueue_done(void *qp_data, uint8_t *drv_ctx, uint32_t n) { @@ -82,18 +94,276 @@ build_dpaa_raw_dp_auth_fd(uint8_t *drv_ctx, struct rte_crypto_va_iova_ptr *digest, struct rte_crypto_va_iova_ptr *auth_iv, union rte_crypto_sym_ofs ofs, - void *userdata) + void *userdata, + struct qm_fd *fd) { - RTE_SET_USED(drv_ctx); - RTE_SET_USED(sgl); RTE_SET_USED(dest_sgl); RTE_SET_USED(iv); - RTE_SET_USED(digest); RTE_SET_USED(auth_iv); - RTE_SET_USED(ofs); - RTE_SET_USED(userdata); + RTE_SET_USED(fd); - return NULL; + dpaa_sec_session *ses = + ((struct dpaa_sec_raw_dp_ctx *)drv_ctx)->session; + struct dpaa_sec_job *cf; + struct dpaa_sec_op_ctx *ctx; + struct qm_sg_entry *sg, *out_sg, *in_sg; + phys_addr_t start_addr; + uint8_t *old_digest, extra_segs; + int data_len, data_offset, total_len = 0; + unsigned int i; + + for (i = 0; i < sgl->num; i++) + total_len += sgl->vec[i].len; + + data_len = total_len - ofs.ofs.auth.head - ofs.ofs.auth.tail; + data_offset = ofs.ofs.auth.head; + + /* Support only length in bits for SNOW3G and ZUC */ + + if (is_decode(ses)) + extra_segs = 3; + else + extra_segs = 2; + + if (sgl->num > MAX_SG_ENTRIES) { + DPAA_SEC_DP_ERR("Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + ctx = dpaa_sec_alloc_raw_ctx(ses, sgl->num * 2 + extra_segs); + if (!ctx) + return NULL; + + cf = &ctx->job; + ctx->userdata = (void *)userdata; + old_digest = ctx->digest; + + /* output */ + out_sg = &cf->sg[0]; + qm_sg_entry_set64(out_sg, digest->iova); + out_sg->length = ses->digest_length; + cpu_to_hw_sg(out_sg); + + /* input */ + in_sg = &cf->sg[1]; + /* need to extend the input to a compound frame */ + in_sg->extension = 1; + in_sg->final = 1; + in_sg->length = data_len; + qm_sg_entry_set64(in_sg, rte_dpaa_mem_vtop(&cf->sg[2])); + + /* 1st seg */ + sg = in_sg + 1; + + if (ses->iv.length) { + uint8_t *iv_ptr; + + iv_ptr = rte_crypto_op_ctod_offset(userdata, uint8_t *, + ses->iv.offset); + + if (ses->auth_alg == RTE_CRYPTO_AUTH_SNOW3G_UIA2) { + iv_ptr = conv_to_snow_f9_iv(iv_ptr); + sg->length = 12; + } else if (ses->auth_alg == RTE_CRYPTO_AUTH_ZUC_EIA3) { + iv_ptr = conv_to_zuc_eia_iv(iv_ptr); + sg->length = 8; + } else { + sg->length = ses->iv.length; + } + qm_sg_entry_set64(sg, rte_dpaa_mem_vtop(iv_ptr)); + in_sg->length += sg->length; + cpu_to_hw_sg(sg); + sg++; + } + + qm_sg_entry_set64(sg, sgl->vec[0].iova); + sg->offset = data_offset; + + if (data_len <= (int)(sgl->vec[0].len - data_offset)) { + sg->length = data_len; + } else { + sg->length = sgl->vec[0].len - data_offset; + + /* remaining i/p segs */ + for (i = 1; i < sgl->num; i++) { + cpu_to_hw_sg(sg); + sg++; + qm_sg_entry_set64(sg, sgl->vec[i].iova); + if (data_len > (int)sgl->vec[i].len) + sg->length = sgl->vec[0].len; + else + sg->length = data_len; + + data_len = data_len - sg->length; + if (data_len < 1) + break; + } + } + + if (is_decode(ses)) { + /* Digest verification case */ + cpu_to_hw_sg(sg); + sg++; + rte_memcpy(old_digest, digest->va, + ses->digest_length); + start_addr = rte_dpaa_mem_vtop(old_digest); + qm_sg_entry_set64(sg, start_addr); + sg->length = ses->digest_length; + in_sg->length += ses->digest_length; + } + sg->final = 1; + cpu_to_hw_sg(sg); + cpu_to_hw_sg(in_sg); + + return cf; +} + +static inline struct dpaa_sec_job * +build_dpaa_raw_dp_chain_fd(uint8_t *drv_ctx, + struct rte_crypto_sgl *sgl, + struct rte_crypto_sgl *dest_sgl, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *auth_iv, + union rte_crypto_sym_ofs ofs, + void *userdata, + struct qm_fd *fd) +{ + RTE_SET_USED(auth_iv); + + dpaa_sec_session *ses = + ((struct dpaa_sec_raw_dp_ctx *)drv_ctx)->session; + struct dpaa_sec_job *cf; + struct dpaa_sec_op_ctx *ctx; + struct qm_sg_entry *sg, *out_sg, *in_sg; + uint8_t *IV_ptr = iv->va; + unsigned int i; + uint16_t auth_hdr_len = ofs.ofs.cipher.head - + ofs.ofs.auth.head; + uint16_t auth_tail_len = ofs.ofs.auth.tail; + uint32_t auth_only_len = (auth_tail_len << 16) | auth_hdr_len; + int data_len = 0, auth_len = 0, cipher_len = 0; + + for (i = 0; i < sgl->num; i++) + data_len += sgl->vec[i].len; + + cipher_len = data_len - ofs.ofs.cipher.head - ofs.ofs.cipher.tail; + auth_len = data_len - ofs.ofs.auth.head - ofs.ofs.auth.tail; + + if (sgl->num > MAX_SG_ENTRIES) { + DPAA_SEC_DP_ERR("Cipher-Auth: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = dpaa_sec_alloc_raw_ctx(ses, sgl->num * 2 + 4); + if (!ctx) + return NULL; + + cf = &ctx->job; + ctx->userdata = (void *)userdata; + + rte_prefetch0(cf->sg); + + /* output */ + out_sg = &cf->sg[0]; + out_sg->extension = 1; + if (is_encode(ses)) + out_sg->length = cipher_len + ses->digest_length; + else + out_sg->length = cipher_len; + + /* output sg entries */ + sg = &cf->sg[2]; + qm_sg_entry_set64(out_sg, rte_dpaa_mem_vtop(sg)); + cpu_to_hw_sg(out_sg); + + /* 1st seg */ + if (dest_sgl) { + qm_sg_entry_set64(sg, dest_sgl->vec[0].iova); + sg->length = dest_sgl->vec[0].len - ofs.ofs.cipher.head; + sg->offset = ofs.ofs.cipher.head; + + /* Successive segs */ + for (i = 1; i < dest_sgl->num; i++) { + cpu_to_hw_sg(sg); + sg++; + qm_sg_entry_set64(sg, dest_sgl->vec[i].iova); + sg->length = dest_sgl->vec[i].len; + } + } else { + qm_sg_entry_set64(sg, sgl->vec[0].iova); + sg->length = sgl->vec[0].len - ofs.ofs.cipher.head; + sg->offset = ofs.ofs.cipher.head; + + /* Successive segs */ + for (i = 1; i < sgl->num; i++) { + cpu_to_hw_sg(sg); + sg++; + qm_sg_entry_set64(sg, sgl->vec[i].iova); + sg->length = sgl->vec[i].len; + } + } + + if (is_encode(ses)) { + cpu_to_hw_sg(sg); + /* set auth output */ + sg++; + qm_sg_entry_set64(sg, digest->iova); + sg->length = ses->digest_length; + } + sg->final = 1; + cpu_to_hw_sg(sg); + + /* input */ + in_sg = &cf->sg[1]; + in_sg->extension = 1; + in_sg->final = 1; + if (is_encode(ses)) + in_sg->length = ses->iv.length + auth_len; + else + in_sg->length = ses->iv.length + auth_len + + ses->digest_length; + + /* input sg entries */ + sg++; + qm_sg_entry_set64(in_sg, rte_dpaa_mem_vtop(sg)); + cpu_to_hw_sg(in_sg); + + /* 1st seg IV */ + qm_sg_entry_set64(sg, rte_dpaa_mem_vtop(IV_ptr)); + sg->length = ses->iv.length; + cpu_to_hw_sg(sg); + + /* 2 seg */ + sg++; + qm_sg_entry_set64(sg, sgl->vec[0].iova); + sg->length = sgl->vec[0].len - ofs.ofs.auth.head; + sg->offset = ofs.ofs.auth.head; + + /* Successive segs */ + for (i = 1; i < sgl->num; i++) { + cpu_to_hw_sg(sg); + sg++; + qm_sg_entry_set64(sg, sgl->vec[i].iova); + sg->length = sgl->vec[i].len; + } + + if (is_decode(ses)) { + cpu_to_hw_sg(sg); + sg++; + memcpy(ctx->digest, digest->va, + ses->digest_length); + qm_sg_entry_set64(sg, rte_dpaa_mem_vtop(ctx->digest)); + sg->length = ses->digest_length; + } + sg->final = 1; + cpu_to_hw_sg(sg); + + if (auth_only_len) + fd->cmd = 0x80000000 | auth_only_len; + + return cf; } static struct dpaa_sec_job * @@ -104,10 +374,13 @@ build_dpaa_raw_dp_cipher_fd(uint8_t *drv_ctx, struct rte_crypto_va_iova_ptr *digest, struct rte_crypto_va_iova_ptr *auth_iv, union rte_crypto_sym_ofs ofs, - void *userdata) + void *userdata, + struct qm_fd *fd) { RTE_SET_USED(digest); RTE_SET_USED(auth_iv); + RTE_SET_USED(fd); + dpaa_sec_session *ses = ((struct dpaa_sec_raw_dp_ctx *)drv_ctx)->session; struct dpaa_sec_job *cf; @@ -264,15 +537,14 @@ dpaa_sec_raw_enqueue_burst(void *qp_data, uint8_t *drv_ctx, &vec->digest[loop], &vec->auth_iv[loop], ofs, - user_data[loop]); + user_data[loop], + fd); if (!cf) { DPAA_SEC_ERR("error: Improper packet contents" " for crypto operation"); goto skip_tx; } inq[loop] = ses->inq[rte_lcore_id() % MAX_DPAA_CORES]; - fd->opaque_addr = 0; - fd->cmd = 0; qm_fd_addr_set64(fd, rte_dpaa_mem_vtop(cf->sg)); fd->_format1 = qm_fd_compound; fd->length29 = 2 * sizeof(struct qm_sg_entry); @@ -470,6 +742,8 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess->build_raw_dp_fd = build_dpaa_raw_dp_cipher_fd; else if (sess->ctxt == DPAA_SEC_AUTH) sess->build_raw_dp_fd = build_dpaa_raw_dp_auth_fd; + else if (sess->ctxt == DPAA_SEC_CIPHER_HASH) + sess->build_raw_dp_fd = build_dpaa_raw_dp_chain_fd; else return -ENOTSUP; dp_ctx = (struct dpaa_sec_raw_dp_ctx *)raw_dp_ctx->drv_ctx_data;