From patchwork Mon May 22 11:00:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 100287 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp191940qge; Mon, 22 May 2017 04:15:02 -0700 (PDT) X-Received: by 10.36.87.84 with SMTP id u81mr40904189ita.35.1495451701966; Mon, 22 May 2017 04:15:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1495451701; cv=none; d=google.com; s=arc-20160816; b=o8W1SN0aecGw0o/qKVQG0VDIoiWhREvDq32xUm0nT3ZvB6JCQd3jcDwZJGc0jLl+Om E4T5Wv5kYiRc796czEqcX7OpSTL6S1q462qkY9GpH4YzbUwZL6z2JaQ9sJzSp3sYrwqy XssjMElMS4cJN+0WHLgxEvrqj8yiFd9vCUplmR5k8NedHzFUPUfSgQjBnExIqS3yTnOU cpf8vKFxY0lslsW+NLrfiSiDUKe6WBRkY53gwKIDLGkEGzoa8QTquBH2lVRl333Zwgdq jFdvulTcw0CELhkImBUmZJ5LSGAuT+BdcDNYSsvHiDr473kABMa7n7ZyAKuTvo/RkpiW n0bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=LsUfzXgCsqg83pSLur8sw6njV/R1AMaPUzQ6E/SZJRA=; b=jG2sDo/piqo45YdKzEhzrLy+tTzXkuNH8kKOM3OveuKv99wQAFPAuqp62cmkOG+8ZH jWv22fMZ9oesQvBnfEkfvq6nGNkej6Vp1Cz5/fiUvkgtstV3bdZTGgBKfwmDHIZp22F+ PynpvI2wUtKi5A3MeK5rObfisWWEgJjglj3Sih121U6x1tb1TMFo+I5us6UMZOj6xHLa gcv6hJgSOLWv4km1JC9okf2Dt75sh7/yTHjUmqDLM3c7rN6B16E9SWH3NQDC18/tqGlX XIHu7xkXSaSEQydsbReB5M0O99lL5YNKUVSllhrpZXyy1+Xjev78xeSo05vs6CvFtG4G H3ZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id d80si32090209itc.51.2017.05.22.04.15.01; Mon, 22 May 2017 04:15:01 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 01EA060A4A; Mon, 22 May 2017 11:15:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 3F42560B15; Mon, 22 May 2017 11:04:44 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0C2D160B04; Mon, 22 May 2017 11:03:32 +0000 (UTC) Received: from forward1m.cmail.yandex.net (forward1m.cmail.yandex.net [5.255.216.2]) by lists.linaro.org (Postfix) with ESMTPS id 50F146079A for ; Mon, 22 May 2017 11:00:55 +0000 (UTC) Received: from smtp3o.mail.yandex.net (smtp3o.mail.yandex.net [37.140.190.28]) by forward1m.cmail.yandex.net (Yandex) with ESMTP id AAE7420FD1 for ; Mon, 22 May 2017 14:00:53 +0300 (MSK) Received: from smtp3o.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp3o.mail.yandex.net (Yandex) with ESMTP id 888A52940EF0 for ; Mon, 22 May 2017 14:00:43 +0300 (MSK) Received: by smtp3o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id EyK36iliXq-0hp8usqP; Mon, 22 May 2017 14:00:43 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Mon, 22 May 2017 14:00:06 +0300 Message-Id: <1495450806-27703-17-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1495450806-27703-1-git-send-email-odpbot@yandex.ru> References: <1495450806-27703-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 34 Subject: [lng-odp] [PATCH API-NEXT v1 16/16] linux-generic: crypto: make en/decryption work across packet segments X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Dmitry Eremin-Solenikov Signed-off-by: Dmitry Eremin-Solenikov --- /** Email created from pull request 34 (lumag:crypto-update-main-new) ** https://github.com/Linaro/odp/pull/34 ** Patch: https://github.com/Linaro/odp/pull/34.patch ** Base sha: 826ee894aa0ebd09d42a17e1de077c46bc5b366a ** Merge commit sha: 225ede5cef6eb46fc9e1d1d385ac8be0b57c64f5 **/ platform/linux-generic/odp_crypto.c | 162 ++++++++++++++++++++++++++---------- 1 file changed, 116 insertions(+), 46 deletions(-) diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index a147f271..e5468c40 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -195,15 +195,101 @@ odp_crypto_alg_err_t auth_check(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_NONE; } +#define IN_CONSUME(len) \ + in_pos += (len); \ + in_len -= (len); \ + insegaddr += (len); \ + inseglen -= (len); + +static +int internal_encrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + EVP_EncryptUpdate(ctx, insegaddr, &cipher_len, insegaddr, part); + IN_CONSUME(part); + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + EVP_EncryptUpdate(ctx, block, &cipher_len, insegaddr, part); + IN_CONSUME(part); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_EncryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + +static +int internal_decrypt(EVP_CIPHER_CTX *ctx, odp_crypto_op_param_t *param) +{ + odp_packet_t pkt = param->out_pkt; + unsigned in_pos = param->cipher_range.offset; + unsigned out_pos = param->cipher_range.offset; + unsigned in_len = param->cipher_range.length; + uint8_t block[2 * EVP_MAX_BLOCK_LENGTH]; + unsigned block_len = EVP_CIPHER_block_size(EVP_CIPHER_CTX_cipher(ctx)); + int cipher_len; + int ret; + + while (in_len > 0) { + uint32_t seglen = 0; /* GCC */ + uint8_t *insegaddr = odp_packet_offset(pkt, in_pos, &seglen, NULL); + unsigned inseglen = in_len < seglen ? in_len : seglen; + + /* There should be at least 1 additional block in out buffer */ + if (inseglen > block_len) { + unsigned part = inseglen - block_len; + EVP_DecryptUpdate(ctx, insegaddr, &cipher_len, insegaddr, part); + IN_CONSUME(part); + out_pos += cipher_len; + } + + /* Use temporal storage */ + if (inseglen > 0) { + unsigned part = inseglen; + EVP_DecryptUpdate(ctx, block, &cipher_len, insegaddr, part); + IN_CONSUME(part); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + out_pos += cipher_len; + } + } + + ret = EVP_DecryptFinal_ex(ctx, block, &cipher_len); + odp_packet_copy_from_mem(pkt, out_pos, cipher_len, block); + + return ret; +} + static odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; void *iv_ptr; - int cipher_len = 0; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -212,9 +298,6 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -222,13 +305,12 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); - - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); + ret = internal_encrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -236,10 +318,8 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; - int plain_len = 0; void *iv_ptr; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -248,9 +328,6 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -258,13 +335,12 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len); + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static int process_cipher_param(odp_crypto_generic_session_t *session, @@ -299,13 +375,12 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t plain_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; void *iv_ptr; - int cipher_len = 0; - uint8_t *tag = data + param->hash_result_offset; + int dummy_len = 0; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -314,9 +389,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Encrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -328,18 +400,20 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_EncryptUpdate(ctx, NULL, &cipher_len, + EVP_EncryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_EncryptUpdate(ctx, data, &cipher_len, data, plain_len); + ret = internal_encrypt(ctx, param); - EVP_EncryptFinal_ex(ctx, data + cipher_len, &cipher_len); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_GET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); + odp_packet_copy_from_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : + ODP_CRYPTO_ALG_ERR_NONE; } static @@ -347,13 +421,12 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { EVP_CIPHER_CTX *ctx; - uint8_t *data = odp_packet_data(param->out_pkt); - uint32_t cipher_len = param->cipher_range.length; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; - int plain_len = 0; + int dummy_len = 0; void *iv_ptr; - uint8_t *tag = data + param->hash_result_offset; + uint8_t block[EVP_MAX_MD_SIZE]; + int ret; if (param->override_iv_ptr) iv_ptr = param->override_iv_ptr; @@ -362,9 +435,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, else return ODP_CRYPTO_ALG_ERR_IV_INVALID; - /* Adjust pointer for beginning of area to cipher/auth */ - data += param->cipher_range.offset; - /* Decrypt it */ ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, @@ -374,22 +444,22 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); EVP_CIPHER_CTX_set_padding(ctx, 0); + odp_packet_copy_to_mem(param->out_pkt, param->hash_result_offset, + session->p.auth_digest_len, block); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, - session->p.auth_digest_len, tag); + session->p.auth_digest_len, block); /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) - EVP_DecryptUpdate(ctx, NULL, &plain_len, + EVP_DecryptUpdate(ctx, NULL, &dummy_len, aad_head, aad_len); - EVP_DecryptUpdate(ctx, data, &plain_len, data, cipher_len); - - if (EVP_DecryptFinal_ex(ctx, data + plain_len, &plain_len) <= 0) - return ODP_CRYPTO_ALG_ERR_ICV_CHECK; + ret = internal_decrypt(ctx, param); EVP_CIPHER_CTX_free(ctx); - return ODP_CRYPTO_ALG_ERR_NONE; + return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK: + ODP_CRYPTO_ALG_ERR_NONE; } static int process_aes_gcm_param(odp_crypto_generic_session_t *session,