From patchwork Thu Dec 14 13:00:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 121922 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp6776132qgn; Thu, 14 Dec 2017 05:04:15 -0800 (PST) X-Google-Smtp-Source: ACJfBov0eVt4lIfVz0ImZ512erCqcQOI5nes+XAM8Kk0jU6/hNEawBoT0Gje6mCU9jv2M+PPr+Dq X-Received: by 10.129.146.85 with SMTP id j82mr4464220ywg.148.1513256655201; Thu, 14 Dec 2017 05:04:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1513256655; cv=none; d=google.com; s=arc-20160816; b=rCUHNJHyVV828z2jt7snvFSL3M4TmO2wq7fqT3UnlHO+UvQOtnpz7rWPpnZpvxcw+g jUTScEF4ZY5a6c4kCcRB6lrvBZA9VuUNQTRYN3AYZ7nzFatsSfhnky5i13r6TuCoYMXh efC3iJj9bvh0D+GF8eGIQX0Cd4Rdz9J9o9o6UVJcLFhGXcl89UBVmHdD4rHTt9QKdgJo tKlajSAD5YPonRbwafAm/LFlcS88LEU5a+blntHVcknhmLZRGCauYP7fA/iD8fGHlW1c 0WNsvmCLbFkMN1vZrBnbxtytHw+hqIw/VEPHogWgX0Bvjj0gIYmcrabXrVHskzJilnbY mC3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=8ivhWSNYJYl6+nzX/2vyRPmRoXFzjzjdBJXHzS9SxW8=; b=gpuIW+7/qTlcEhh1GtapwL6wu7z640eztIwvZBwoEn5ok4Xq13+ICNXXtD5vLV3FO0 2MEqO96x3eVZPXZALL5Ri/TeCYvf7EkfMthZ+3KQ0O6Wm4Iy2SkVjt+dZJZC9emhbRxR 9kRCZqlshsMhvNk55q+wpBA0/gn/J8kH4QbDgoA9w7pPGoyJLPi98E4c9rqx4apnjtPH QJkdOC+Gf2LdwXu4/A3X3b2HQbKXhDOlPiiMJMr4Ke1NIZt+MW+wKLAHb3JTboLFBZsU WuywPbMORLKT8mGBcxCN4q9a4+ak15DuXj+ZXh4Mbr8S6mj/oBa/tzljFvbHuY9wkooS bucQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id x71si1499936qka.19.2017.12.14.05.04.14; Thu, 14 Dec 2017 05:04:15 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id DACCF60925; Thu, 14 Dec 2017 13:04:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id E0F406094B; Thu, 14 Dec 2017 13:00:55 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id A77AF6090A; Thu, 14 Dec 2017 13:00:21 +0000 (UTC) Received: from forward101j.mail.yandex.net (forward101j.mail.yandex.net [5.45.198.241]) by lists.linaro.org (Postfix) with ESMTPS id 85E786090A for ; Thu, 14 Dec 2017 13:00:14 +0000 (UTC) Received: from mxback6j.mail.yandex.net (mxback6j.mail.yandex.net [IPv6:2a02:6b8:0:1619::10f]) by forward101j.mail.yandex.net (Yandex) with ESMTP id B67C31246869 for ; Thu, 14 Dec 2017 16:00:12 +0300 (MSK) Received: from smtp4o.mail.yandex.net (smtp4o.mail.yandex.net [2a02:6b8:0:1a2d::28]) by mxback6j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id MTwUY800ic-0CHaUNYa; Thu, 14 Dec 2017 16:00:12 +0300 Received: by smtp4o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id dS96mFDDb1-0BJeGUxn; Thu, 14 Dec 2017 16:00:11 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 14 Dec 2017 16:00:08 +0300 Message-Id: <1513256408-27608-4-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1513256408-27608-1-git-send-email-odpbot@yandex.ru> References: <1513256408-27608-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 344 Subject: [lng-odp] [PATCH API-NEXT v2 3/3] linux-gen: crypto: significant speedup of all operations X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Dmitry Eremin-Solenikov Per idea of Janne Peltonen, do not allocate/free crypto contexts for each operation, providing significant speed increase. Each thread on startup allocates hmac+cipher contexts pair for each crypto session. Then they are initialized on demand, when session is first executed using this thread. Signed-off-by: Dmitry Eremin-Solenikov --- /** Email created from pull request 344 (lumag:openssl-ctx-api-next) ** https://github.com/Linaro/odp/pull/344 ** Patch: https://github.com/Linaro/odp/pull/344.patch ** Base sha: da9d347e0bbe337ea86448306432080206f3ef49 ** Merge commit sha: ee2d929ea3a3400d5cd76fbdbb29eafa6f80978f **/ platform/linux-generic/include/odp_internal.h | 2 + platform/linux-generic/odp_crypto.c | 234 ++++++++++++++++---------- platform/linux-generic/odp_init.c | 13 ++ 3 files changed, 159 insertions(+), 90 deletions(-) diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index d01b4f155..5eb92f188 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -113,6 +113,8 @@ int odp_queue_term_global(void); int odp_crypto_init_global(void); int odp_crypto_term_global(void); +int odp_crypto_init_local(void); +int odp_crypto_term_local(void); int odp_timer_init_global(const odp_init_t *params); int odp_timer_term_global(void); diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index 811d3fc03..d855f13d7 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -129,8 +129,18 @@ struct odp_crypto_generic_session_t { }; crypto_func_t func; } auth; + + uint32_t cipher_valid[(ODP_THREAD_COUNT_MAX + 32 - 1) / 32]; + uint32_t hmac_valid[(ODP_THREAD_COUNT_MAX + 32 - 1) / 32]; + unsigned idx; }; +#define IS_VALID(session, type, id) \ + (session->type ## _valid[id / 32] & (1 << (id % 32))) + +#define SET_VALID(session, type, id) \ + session->type ## _valid[id / 32] |= (1 << (id % 32)) + typedef struct odp_crypto_global_s odp_crypto_global_t; struct odp_crypto_global_s { @@ -142,6 +152,16 @@ struct odp_crypto_global_s { static odp_crypto_global_t *global; +typedef struct crypto_local_t { + struct { + HMAC_CTX *hmac; + EVP_CIPHER_CTX *cipher; + } ctx[MAX_SESSIONS]; + int id; +} crypto_local_t; + +static __thread crypto_local_t local; + static odp_crypto_generic_session_t *alloc_session(void) { @@ -155,6 +175,8 @@ odp_crypto_generic_session_t *alloc_session(void) } odp_spinlock_unlock(&global->lock); + session->idx = (session - global->sessions) / sizeof(*session); + return session; } @@ -175,24 +197,51 @@ null_crypto_routine(odp_packet_t pkt ODP_UNUSED, return ODP_CRYPTO_ALG_ERR_NONE; } +/* Mimic new OpenSSL 1.1.y API */ +#if OPENSSL_VERSION_NUMBER < 0x10100000L +static HMAC_CTX *HMAC_CTX_new(void) +{ + HMAC_CTX *ctx = malloc(sizeof(*ctx)); + + HMAC_CTX_init(ctx); + return ctx; +} + +static void HMAC_CTX_free(HMAC_CTX *ctx) +{ + HMAC_CTX_cleanup(ctx); + free(ctx); +} +#endif + static -void packet_hmac_calculate(HMAC_CTX *ctx, - odp_packet_t pkt, - const odp_crypto_packet_op_param_t *param, - odp_crypto_generic_session_t *session, - uint8_t *hash) +void packet_hmac(odp_packet_t pkt, + const odp_crypto_packet_op_param_t *param, + odp_crypto_generic_session_t *session, + uint8_t *hash) { + HMAC_CTX *ctx = local.ctx[session->idx].hmac; uint32_t offset = param->auth_range.offset; uint32_t len = param->auth_range.length; ODP_ASSERT(offset + len <= odp_packet_len(pkt)); - HMAC_Init_ex(ctx, - session->auth.key, - session->auth.key_length, - session->auth.evp_md, - NULL); + if (!IS_VALID(session, hmac, local.id)) { + SET_VALID(session, hmac, local.id); + HMAC_Init_ex(ctx, + session->auth.key, + session->auth.key_length, + session->auth.evp_md, + NULL); + } else { + HMAC_Init_ex(ctx, + NULL, + session->auth.key_length, + NULL, + NULL); + } + /* Hash it */ while (len > 0) { uint32_t seglen = 0; /* GCC */ void *mapaddr = odp_packet_offset(pkt, offset, &seglen, NULL); @@ -206,36 +255,6 @@ void packet_hmac_calculate(HMAC_CTX *ctx, HMAC_Final(ctx, hash, NULL); } -#if OPENSSL_VERSION_NUMBER < 0x10100000L -static -void packet_hmac(odp_packet_t pkt, - const odp_crypto_packet_op_param_t *param, - odp_crypto_generic_session_t *session, - uint8_t *hash) -{ - HMAC_CTX ctx; - - /* Hash it */ - HMAC_CTX_init(&ctx); - packet_hmac_calculate(&ctx, pkt, param, session, hash); - HMAC_CTX_cleanup(&ctx); -} -#else -static -void packet_hmac(odp_packet_t pkt, - const odp_crypto_packet_op_param_t *param, - odp_crypto_generic_session_t *session, - uint8_t *hash) -{ - HMAC_CTX *ctx; - - /* Hash it */ - ctx = HMAC_CTX_new(); - packet_hmac_calculate(ctx, pkt, param, session, hash); - HMAC_CTX_free(ctx); -} -#endif - static odp_crypto_alg_err_t auth_gen(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, @@ -430,7 +449,7 @@ odp_crypto_alg_err_t cipher_encrypt(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; void *iv_ptr; int ret; @@ -442,16 +461,16 @@ odp_crypto_alg_err_t cipher_encrypt(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Encrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, - session->cipher.key_data, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, + session->cipher.key_data, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); ret = internal_encrypt(ctx, pkt, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -461,7 +480,7 @@ odp_crypto_alg_err_t cipher_decrypt(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; void *iv_ptr; int ret; @@ -473,16 +492,16 @@ odp_crypto_alg_err_t cipher_decrypt(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Decrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, - session->cipher.key_data, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, + session->cipher.key_data, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); ret = internal_decrypt(ctx, pkt, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -519,7 +538,7 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = session->p.auth_aad_len; void *iv_ptr; @@ -535,13 +554,15 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Encrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, - session->cipher.key_data, NULL); - EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, - session->p.iv.length, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_EncryptInit_ex(ctx, session->cipher.evp_cipher, NULL, + session->cipher.key_data, NULL); + EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, + session->p.iv.length, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); /* Authenticate header data (if any) without encrypting them */ if (aad_len > 0) @@ -555,8 +576,6 @@ odp_crypto_alg_err_t aes_gcm_encrypt(odp_packet_t pkt, odp_packet_copy_from_mem(pkt, param->hash_result_offset, session->p.auth_digest_len, block); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -566,7 +585,7 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; const uint8_t *aad_head = param->aad.ptr; uint32_t aad_len = session->p.auth_aad_len; int dummy_len = 0; @@ -582,13 +601,15 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Decrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, - session->cipher.key_data, NULL); - EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, - session->p.iv.length, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_DecryptInit_ex(ctx, session->cipher.evp_cipher, NULL, + session->cipher.key_data, NULL); + EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, + session->p.iv.length, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); odp_packet_copy_to_mem(pkt, param->hash_result_offset, session->p.auth_digest_len, block); @@ -602,8 +623,6 @@ odp_crypto_alg_err_t aes_gcm_decrypt(odp_packet_t pkt, ret = internal_decrypt(ctx, pkt, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK : ODP_CRYPTO_ALG_ERR_NONE; } @@ -635,7 +654,7 @@ odp_crypto_alg_err_t aes_gmac_gen(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; void *iv_ptr; uint8_t block[EVP_MAX_MD_SIZE]; int ret; @@ -648,13 +667,15 @@ odp_crypto_alg_err_t aes_gmac_gen(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Encrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_EncryptInit_ex(ctx, session->auth.evp_cipher, NULL, - session->auth.key, NULL); - EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, - session->p.iv.length, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_EncryptInit_ex(ctx, session->auth.evp_cipher, NULL, + session->auth.key, NULL); + EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, + session->p.iv.length, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); ret = internal_aad(ctx, pkt, param); @@ -663,8 +684,6 @@ odp_crypto_alg_err_t aes_gmac_gen(odp_packet_t pkt, odp_packet_copy_from_mem(pkt, param->hash_result_offset, session->p.auth_digest_len, block); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_DATA_SIZE : ODP_CRYPTO_ALG_ERR_NONE; } @@ -674,7 +693,7 @@ odp_crypto_alg_err_t aes_gmac_check(odp_packet_t pkt, const odp_crypto_packet_op_param_t *param, odp_crypto_generic_session_t *session) { - EVP_CIPHER_CTX *ctx; + EVP_CIPHER_CTX *ctx = local.ctx[session->idx].cipher; void *iv_ptr; uint8_t block[EVP_MAX_MD_SIZE]; int ret; @@ -687,13 +706,15 @@ odp_crypto_alg_err_t aes_gmac_check(odp_packet_t pkt, return ODP_CRYPTO_ALG_ERR_IV_INVALID; /* Decrypt it */ - ctx = EVP_CIPHER_CTX_new(); - EVP_DecryptInit_ex(ctx, session->auth.evp_cipher, NULL, - session->auth.key, NULL); - EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, - session->p.iv.length, NULL); + if (!IS_VALID(session, cipher, local.id)) { + SET_VALID(session, cipher, local.id); + EVP_DecryptInit_ex(ctx, session->auth.evp_cipher, NULL, + session->auth.key, NULL); + EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_IVLEN, + session->p.iv.length, NULL); + EVP_CIPHER_CTX_set_padding(ctx, 0); + } EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv_ptr); - EVP_CIPHER_CTX_set_padding(ctx, 0); odp_packet_copy_to_mem(pkt, param->hash_result_offset, session->p.auth_digest_len, block); @@ -704,8 +725,6 @@ odp_crypto_alg_err_t aes_gmac_check(odp_packet_t pkt, ret = internal_aad(ctx, pkt, param); - EVP_CIPHER_CTX_free(ctx); - return ret <= 0 ? ODP_CRYPTO_ALG_ERR_ICV_CHECK : ODP_CRYPTO_ALG_ERR_NONE; } @@ -1213,6 +1232,41 @@ int odp_crypto_term_global(void) return rc; } +int odp_crypto_init_local(void) +{ + unsigned i; + + memset(&local, 0, sizeof(local)); + + for (i = 0; i < MAX_SESSIONS; i++) { + local.ctx[i].hmac = HMAC_CTX_new(); + local.ctx[i].cipher = EVP_CIPHER_CTX_new(); + + if (local.ctx[i].hmac == NULL || local.ctx[i].cipher == NULL) { + odp_crypto_term_local(); + return -1; + } + } + + local.id = odp_thread_id(); + + return 0; +} + +int odp_crypto_term_local(void) +{ + unsigned i; + + for (i = 0; i < MAX_SESSIONS; i++) { + if (local.ctx[i].hmac != NULL) + HMAC_CTX_free(local.ctx[i].hmac); + if (local.ctx[i].cipher != NULL) + EVP_CIPHER_CTX_free(local.ctx[i].cipher); + } + + return 0; +} + odp_random_kind_t odp_random_max_kind(void) { return ODP_RANDOM_CRYPTO; diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 1412c03a0..5c2503402 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -418,6 +418,12 @@ int odp_init_local(odp_instance_t instance, odp_thread_type_t thr_type) } stage = PKTIO_INIT; + if (odp_crypto_init_local()) { + ODP_ERR("ODP crypto local init failed.\n"); + goto init_fail; + } + stage = CRYPTO_INIT; + if (odp_pool_init_local()) { ODP_ERR("ODP pool local init failed.\n"); goto init_fail; @@ -470,6 +476,13 @@ int _odp_term_local(enum init_stage stage) } /* Fall through */ + case CRYPTO_INIT: + if (odp_crypto_term_local()) { + ODP_ERR("ODP crypto local term failed.\n"); + rc = -1; + } + /* Fall through */ + case POOL_INIT: if (odp_pool_term_local()) { ODP_ERR("ODP buffer pool local term failed.\n");