From patchwork Thu Dec 14 02:00:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 121885 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp6231156qgn; Wed, 13 Dec 2017 18:01:24 -0800 (PST) X-Google-Smtp-Source: ACJfBougfEAErB2yHT1QTASVWAWImf4nYWVffD4UMAODAEzDftLYlIw4+zzrg+jBhsKhUFqbl4ds X-Received: by 10.55.19.158 with SMTP id 30mr13954980qkt.0.1513216884021; Wed, 13 Dec 2017 18:01:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1513216884; cv=none; d=google.com; s=arc-20160816; b=XiXrfg4j/OXKXRUVSfh1QWmiS0UNWxq6xGBDe/UUAo0c13NNsSwZiSKT+K3zTV7eef 5YBomtzelPSny3g+CmUsPmUJy8hm/yQNwZH1oorDOUNjzFgw/b4zpyD7j79YxpVICyQq SOUc76kh1niI/rIU3bLkaKdPKx/VVWWtYdT7f0oXA2x/bgvVm3r2w6kXt3P/qs3djytL nFnDrUaN1wDp1jr9Z1CPwDkQpZSfPTVefHfNx+m/+CUmbnAIJKWGljERtpg9U071DBYA pnOQrL04REqPKWYMk9oum8Xfm1TRRrkxaCKOg1SYZ/KJh3VluYjkuCrBnUlAYssOplmh aLVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=fvdNjIMQgCuwCPrNfekHz51qQn6b46812Faa+aMeNt4=; b=a62x5PxUm/962pAY5xPj3FUNav1ojirQKFgImiIi4eDSkAPMRrIHzGrdhPGufcyjrp bjH4Me8eXpua9LvorxapIOc+DVJXttnk3A5ui1YTuYN1RtGGJOB4MF9lD1XxnlB+4ufX ecfCAqWu5OPGEgRTPq8/GSEMLuDZnyE72JQ++A520P6bFEXoralcZlvS69YAOkrzJnzf 9krOhHrrkO53LXncePyMXW92CT+wsIeiSEIyfu/cuB1yebKUl84rt8QzkJLqN4IHTl75 /qCWBUwiPggHt+vtnThXswiD/oGc9OPi3171MhYFyiCFiD1xqQTtWec+Vl+umbqRcZcg gYNw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id t123si3335533qkf.99.2017.12.13.18.01.23; Wed, 13 Dec 2017 18:01:24 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id BB1DF6091B; Thu, 14 Dec 2017 02:01:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id CCD1560918; Thu, 14 Dec 2017 02:00:31 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0CBFE608F6; Thu, 14 Dec 2017 02:00:24 +0000 (UTC) Received: from forward104o.mail.yandex.net (forward104o.mail.yandex.net [37.140.190.179]) by lists.linaro.org (Postfix) with ESMTPS id D38E560732 for ; Thu, 14 Dec 2017 02:00:19 +0000 (UTC) Received: from mxback4j.mail.yandex.net (mxback4j.mail.yandex.net [IPv6:2a02:6b8:0:1619::10d]) by forward104o.mail.yandex.net (Yandex) with ESMTP id 617D9705C0B for ; Thu, 14 Dec 2017 05:00:18 +0300 (MSK) Received: from smtp2o.mail.yandex.net (smtp2o.mail.yandex.net [2a02:6b8:0:1a2d::26]) by mxback4j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id w9lrLnjVwS-0I3GbU2Z; Thu, 14 Dec 2017 05:00:18 +0300 Received: by smtp2o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 7ke2R8DbjY-0HtWEuXC; Thu, 14 Dec 2017 05:00:17 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 14 Dec 2017 05:00:09 +0300 Message-Id: <1513216810-9419-3-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1513216810-9419-1-git-send-email-odpbot@yandex.ru> References: <1513216810-9419-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 342 Subject: [lng-odp] [PATCH v1] [RFC 2/2] linux-gen: speedup crypto module X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Dmitry Eremin-Solenikov Do not allocate/free crypto contexts for each operation, providing slight speed increase. Signed-off-by: Dmitry Eremin-Solenikov --- /** Email created from pull request 342 (lumag:openssl-ctx) ** https://github.com/Linaro/odp/pull/342 ** Patch: https://github.com/Linaro/odp/pull/342.patch ** Base sha: 6b5cdc77eb9759a2349b10372a964648559bc92c ** Merge commit sha: fd5d6b2ed05a28c6f333aa0077b5eb4d8e4aa462 **/ platform/linux-generic/include/odp_internal.h | 2 + platform/linux-generic/odp_crypto.c | 88 ++++++++++++++++----------- platform/linux-generic/odp_init.c | 13 ++++ 3 files changed, 67 insertions(+), 36 deletions(-) diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index fc69cd0c5..72fe1acfa 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -108,6 +108,8 @@ int odp_queue_term_global(void); int odp_crypto_init_global(void); int odp_crypto_term_global(void); +int odp_crypto_init_local(void); +int odp_crypto_term_local(void); int odp_timer_init_global(void); int odp_timer_term_global(void); diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index 4c708931a..ce17e2bbb 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -86,6 +86,52 @@ struct odp_crypto_global_s { static odp_crypto_global_t *global; +typedef struct crypto_local_t { + HMAC_CTX *hmac; + EVP_CIPHER_CTX *cipher; +#if OPENSSL_VERSION_NUMBER < 0x10100000L + HMAC_CTX hmac_ctx; +#endif +} crypto_local_t; + +static __thread crypto_local_t local; + +#if OPENSSL_VERSION_NUMBER < 0x10100000L +int odp_crypto_init_local(void) +{ + local.hmac = &local.hmac_ctx; + HMAC_CTX_init(local.hmac); + local.cipher = EVP_CIPHER_CTX_new(); + + return ((local.hmac == NULL) || (local.cipher == NULL)) ? -1 : 0; +} + +#define HMAC_CTX_reset(ctx) HMAC_CTX_cleanup(ctx) + +int odp_crypto_term_local(void) +{ + EVP_CIPHER_CTX_free(local.cipher); + + return 0; +} +#else +int odp_crypto_init_local(void) +{ + local.hmac = HMAC_CTX_new(); + local.cipher = EVP_CIPHER_CTX_new(); + + return ((local.hmac == NULL) || (local.cipher == NULL)) ? -1 : 0; +} + +int odp_crypto_term_local(void) +{ + HMAC_CTX_free(local.hmac); + EVP_CIPHER_CTX_free(local.cipher); + + return 0; +} +#endif + static odp_crypto_generic_op_result_t *get_op_result_from_event(odp_event_t ev) { @@ -157,33 +203,15 @@ void packet_hmac_calculate(HMAC_CTX *ctx, HMAC_Final(ctx, hash, NULL); } -#if OPENSSL_VERSION_NUMBER < 0x10100000L -static -void packet_hmac(odp_crypto_op_param_t *param, - odp_crypto_generic_session_t *session, - uint8_t *hash) -{ - HMAC_CTX ctx; - - /* Hash it */ - HMAC_CTX_init(&ctx); - packet_hmac_calculate(&ctx, param, session, hash); - HMAC_CTX_cleanup(&ctx); -} -#else static void packet_hmac(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session, uint8_t *hash) { - HMAC_CTX *ctx; - /* Hash it */ - ctx = HMAC_CTX_new(); - packet_hmac_calculate(ctx, param, session, hash); - HMAC_CTX_free(ctx); + packet_hmac_calculate(local.hmac, param, session, hash); + HMAC_CTX_reset(local.hmac); } -#endif static odp_crypto_alg_err_t auth_gen(odp_crypto_op_param_t *param, @@ -347,7 +375,7 @@ static odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.cipher; void *iv_ptr; int ret; @@ -359,7 +387,6 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Encrypt it */ - ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, session->cipher.key_data, NULL); EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); @@ -367,8 +394,6 @@ odp_crypto_alg_err_t cipher_encrypt(odp_crypto_op_param_t *param, ret = internal_encrypt(ctx, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -377,7 +402,7 @@ static odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.cipher; void *iv_ptr; int ret; @@ -389,7 +414,6 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Decrypt it */ - ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, session->cipher.key_data, NULL); EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); @@ -397,8 +421,6 @@ odp_crypto_alg_err_t cipher_decrypt(odp_crypto_op_param_t *param, ret = internal_decrypt(ctx, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -434,7 +456,7 @@ static odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.cipher; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; void *iv_ptr; @@ -450,7 +472,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Encrypt it */ - ctx = EVP_CIPHER_CTX_new(); EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, session->cipher.key_data, NULL); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, @@ -470,8 +491,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_crypto_op_param_t *param, odp_packet_copy_from_mem(param->out_pkt, param->hash_result_offset, session->p.auth_digest_len, block); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -480,7 +499,7 @@ static odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.cipher; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = param->aad.length; int dummy_len = 0; @@ -496,7 +515,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Decrypt it */ - ctx = EVP_CIPHER_CTX_new(); EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, session->cipher.key_data, NULL); EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, @@ -516,8 +534,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_crypto_op_param_t *param, ret = internal_decrypt(ctx, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK : ODP_CRYPTO_ALG_ERR_NONE; } diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index fe03709b7..809769187 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -300,6 +300,12 @@ int odp_init_local(odp_instance_t instance, odp_thread_type_t thr_type) } stage = PKTIO_INIT; + if (odp_crypto_init_local()) { + ODP_ERR("ODP crypto local init failed.\n"); + goto init_fail; + } + stage = CRYPTO_INIT; + if (odp_pool_init_local()) { ODP_ERR("ODP pool local init failed.\n"); goto init_fail; @@ -352,6 +358,13 @@ int _odp_term_local(enum init_stage stage) } /* Fall through */ + case CRYPTO_INIT: + if (odp_crypto_term_local()) { + ODP_ERR("ODP crypto local term failed.\n"); + rc = -1; + } + /* Fall through */ + case POOL_INIT: if (odp_pool_term_local()) { ODP_ERR("ODP buffer pool local term failed.\n");