From patchwork Tue Oct 18 14:08:33 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 78061 Delivered-To: patch@linaro.org Received: by 10.140.97.247 with SMTP id m110csp904009qge; Tue, 18 Oct 2016 07:08:55 -0700 (PDT) X-Received: by 10.99.62.205 with SMTP id l196mr680523pga.161.1476799735651; Tue, 18 Oct 2016 07:08:55 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j64si32584251pgd.126.2016.10.18.07.08.55; Tue, 18 Oct 2016 07:08:55 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-wireless-owner@vger.kernel.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760199AbcJROIv (ORCPT + 1 other); Tue, 18 Oct 2016 10:08:51 -0400 Received: from mail-qk0-f172.google.com ([209.85.220.172]:33770 "EHLO mail-qk0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759959AbcJROIp (ORCPT ); Tue, 18 Oct 2016 10:08:45 -0400 Received: by mail-qk0-f172.google.com with SMTP id n189so294584654qke.0 for ; Tue, 18 Oct 2016 07:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3qFHe4Z8zUbjIoDugcV+WPcwaO+6LPYubj+9lXxJcDE=; b=F3GcBEdEiwknIETCjATYM+mR1dLECmz8sefs0r1165X2EZpRvBTKdkovPThNvgBLZm kKzSCOls/H+tLxT/WEcatx5fXJYfKZ/caiiZ2NW9ykxgSVpBUQeTjmfzkj+pmchoAS5z 9JvIlW6ywZ0icj9s08IgHkanx5bCx7PXDGXhE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3qFHe4Z8zUbjIoDugcV+WPcwaO+6LPYubj+9lXxJcDE=; b=cQB5s1pYCs8q20EwsXfKcqBSD+AWcwdM6wExb7ph/dbpnryWlFi1tHvdUzuFEvDDlf znvYnw3mayas2n1N4k+YNjqMOMz4/xpvrvfXRlmZrNlaxqy3zj7yVA7TRYAXuMoflEfJ DImog3ST5V13CpwU1P1McLszo++8tD2p+usnQi+38Ma26CQlB/874aMEBfdX7AR8A7Z5 NUBoS05Qbo4m/ZaO68Jvzf/GOWsdembxrsONPDbF5ZTKiFC59Y5zAPcLcBgreW2zmlNg LCPyfg6nHFHSh4y75t4TsmvwmsQEjNJcTGTx2GIazMlp6jhy0/tCs6vbyDxH5A4C7Wg7 Q5jg== X-Gm-Message-State: AA6/9RkC1h6KKwUz5j6HTMHwMWvMxRFAsDvMtIsyKRMRTaJESVvaoo2Me/LzTO66bQSIfpTA X-Received: by 10.194.178.200 with SMTP id da8mr362646wjc.157.1476799723519; Tue, 18 Oct 2016 07:08:43 -0700 (PDT) Received: from localhost.localdomain ([105.137.38.75]) by smtp.gmail.com with ESMTPSA id za1sm59123596wjb.8.2016.10.18.07.08.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 18 Oct 2016 07:08:42 -0700 (PDT) From: Ard Biesheuvel To: linux-wireless@vger.kernel.org, johannes@sipsolutions.net, netdev@vger.kernel.org Cc: herbert@gondor.apana.org.au, j@w1.fi, luto@amacapital.net, Ard Biesheuvel Subject: [RFC PATCH 2/2] mac80211: aes_ccm: cache AEAD request structures per CPU Date: Tue, 18 Oct 2016 15:08:33 +0100 Message-Id: <1476799713-16188-3-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1476799713-16188-1-git-send-email-ard.biesheuvel@linaro.org> References: <1476799713-16188-1-git-send-email-ard.biesheuvel@linaro.org> Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Now that we can no longer invoke AEAD transforms with the aead_request structure allocated on the stack, we perform a kmalloc/kfree for every packet, which is expensive. Since the CCMP routines execute in softirq context, we know there can never be more than one request in flight on each CPU, and so we can simply keep a cached aead_request structure per CPU, and deallocate all of them when deallocating the AEAD transform. Signed-off-by: Ard Biesheuvel --- net/mac80211/aes_ccm.c | 49 ++++++++++++++------ net/mac80211/key.h | 1 + 2 files changed, 36 insertions(+), 14 deletions(-) -- 2.7.4 diff --git a/net/mac80211/aes_ccm.c b/net/mac80211/aes_ccm.c index 58e0338a2c34..2cb5ee4868ea 100644 --- a/net/mac80211/aes_ccm.c +++ b/net/mac80211/aes_ccm.c @@ -28,9 +28,14 @@ int ieee80211_aes_ccm_encrypt(struct ieee80211_ccmp_aead *ccmp, u8 *b_0, int reqsize = sizeof(*aead_req) + crypto_aead_reqsize(ccmp->tfm); u8 *__aad; - aead_req = kzalloc(reqsize + CCM_AAD_LEN, GFP_ATOMIC); - if (!aead_req) - return -ENOMEM; + aead_req = *this_cpu_ptr(ccmp->reqs); + if (!aead_req) { + aead_req = kzalloc(reqsize + CCM_AAD_LEN, GFP_ATOMIC); + if (!aead_req) + return -ENOMEM; + *this_cpu_ptr(ccmp->reqs) = aead_req; + aead_request_set_tfm(aead_req, ccmp->tfm); + } __aad = (u8 *)aead_req + reqsize; memcpy(__aad, aad, CCM_AAD_LEN); @@ -40,12 +45,10 @@ int ieee80211_aes_ccm_encrypt(struct ieee80211_ccmp_aead *ccmp, u8 *b_0, sg_set_buf(&sg[1], data, data_len); sg_set_buf(&sg[2], mic, mic_len); - aead_request_set_tfm(aead_req, ccmp->tfm); aead_request_set_crypt(aead_req, sg, sg, data_len, b_0); aead_request_set_ad(aead_req, sg[0].length); crypto_aead_encrypt(aead_req); - kzfree(aead_req); return 0; } @@ -58,14 +61,18 @@ int ieee80211_aes_ccm_decrypt(struct ieee80211_ccmp_aead *ccmp, u8 *b_0, struct aead_request *aead_req; int reqsize = sizeof(*aead_req) + crypto_aead_reqsize(ccmp->tfm); u8 *__aad; - int err; if (data_len == 0) return -EINVAL; - aead_req = kzalloc(reqsize + CCM_AAD_LEN, GFP_ATOMIC); - if (!aead_req) - return -ENOMEM; + aead_req = *this_cpu_ptr(ccmp->reqs); + if (!aead_req) { + aead_req = kzalloc(reqsize + CCM_AAD_LEN, GFP_ATOMIC); + if (!aead_req) + return -ENOMEM; + *this_cpu_ptr(ccmp->reqs) = aead_req; + aead_request_set_tfm(aead_req, ccmp->tfm); + } __aad = (u8 *)aead_req + reqsize; memcpy(__aad, aad, CCM_AAD_LEN); @@ -75,14 +82,10 @@ int ieee80211_aes_ccm_decrypt(struct ieee80211_ccmp_aead *ccmp, u8 *b_0, sg_set_buf(&sg[1], data, data_len); sg_set_buf(&sg[2], mic, mic_len); - aead_request_set_tfm(aead_req, ccmp->tfm); aead_request_set_crypt(aead_req, sg, sg, data_len + mic_len, b_0); aead_request_set_ad(aead_req, sg[0].length); - err = crypto_aead_decrypt(aead_req); - kzfree(aead_req); - - return err; + return crypto_aead_decrypt(aead_req); } int ieee80211_aes_key_setup_encrypt(struct ieee80211_ccmp_aead *ccmp, @@ -91,6 +94,7 @@ int ieee80211_aes_key_setup_encrypt(struct ieee80211_ccmp_aead *ccmp, size_t mic_len) { struct crypto_aead *tfm; + struct aead_request **r; int err; tfm = crypto_alloc_aead("ccm(aes)", 0, CRYPTO_ALG_ASYNC); @@ -104,6 +108,14 @@ int ieee80211_aes_key_setup_encrypt(struct ieee80211_ccmp_aead *ccmp, if (err) goto free_aead; + /* allow a struct aead_request to be cached per cpu */ + r = alloc_percpu(struct aead_request *); + if (!r) { + err = -ENOMEM; + goto free_aead; + } + + ccmp->reqs = r; ccmp->tfm = tfm; return 0; @@ -114,5 +126,14 @@ int ieee80211_aes_key_setup_encrypt(struct ieee80211_ccmp_aead *ccmp, void ieee80211_aes_key_free(struct ieee80211_ccmp_aead *ccmp) { + struct aead_request *req; + int cpu; + + for_each_possible_cpu(cpu) { + req = *per_cpu_ptr(ccmp->reqs, cpu); + if (req) + kzfree(req); + } + free_percpu(ccmp->reqs); crypto_free_aead(ccmp->tfm); } diff --git a/net/mac80211/key.h b/net/mac80211/key.h index 1ec7a737ab79..f99ec46dc08f 100644 --- a/net/mac80211/key.h +++ b/net/mac80211/key.h @@ -89,6 +89,7 @@ struct ieee80211_key { */ u8 rx_pn[IEEE80211_NUM_TIDS + 1][IEEE80211_CCMP_PN_LEN]; struct crypto_aead *tfm; + struct aead_request * __percpu *reqs; u32 replays; /* dot11RSNAStatsCCMPReplays */ } ccmp; struct {