From patchwork Fri Apr 18 09:22:45 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiri Slaby X-Patchwork-Id: 28628 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 826E620548 for ; Fri, 18 Apr 2014 09:29:26 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id hw13sf4637571qab.3 for ; Fri, 18 Apr 2014 02:29:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:in-reply-to:references :sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=SRaamSZThqq5EGjNclncLidAc7Yn7+Bx56SgZ8WJ19w=; b=DEdo1ZbvHbRx7WVopS+JaPdR0m9wYygDTa5LLPZRYB18INwi6Eg26EmknQJpCwIGng BZ5wyhGOQu2OqYh1BR2wI7Hdz6QhKNf8Kpv8zjoMkl1B6BctJ26OPjud8fWjpNbtbCSx hTXD3ZYu0qOMHIbFkc8iAQAwA2RNtWvRkgdaRmDJMMHJ/j6+hW1pjtSVtbg9WdwDPw0h S065ZaSZ+1wFGn7H2zFmuoTx1ckWyaAOmfiZMzBpvmEXDVLd+NW/mRoGt3tZstscsoll cj2r8xuYIdBctjXY6cK6CICoq3bGZ3hzZDO0rbK/DwZG3oSfY2yrp+PCo/J7iTrU9YD/ eXNw== X-Gm-Message-State: ALoCoQlai6R/+FfhDqWUIDlkYD4yXO1aXTaae6fzIzzpofUQnOJg9dB1pOSPiieJtIUEY7LmapkA X-Received: by 10.58.168.137 with SMTP id zw9mr10837466veb.15.1397813366089; Fri, 18 Apr 2014 02:29:26 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.101.245 with SMTP id u108ls1389058qge.79.gmail; Fri, 18 Apr 2014 02:29:26 -0700 (PDT) X-Received: by 10.58.243.72 with SMTP id ww8mr111284vec.49.1397813365971; Fri, 18 Apr 2014 02:29:25 -0700 (PDT) Received: from mail-ve0-f181.google.com (mail-ve0-f181.google.com [209.85.128.181]) by mx.google.com with ESMTPS id si3si4879710vcb.136.2014.04.18.02.29.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 18 Apr 2014 02:29:25 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.181 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.181; Received: by mail-ve0-f181.google.com with SMTP id oy12so2399385veb.40 for ; Fri, 18 Apr 2014 02:29:25 -0700 (PDT) X-Received: by 10.220.83.4 with SMTP id d4mr1703300vcl.39.1397813365878; Fri, 18 Apr 2014 02:29:25 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp99720vcb; Fri, 18 Apr 2014 02:29:25 -0700 (PDT) X-Received: by 10.66.66.66 with SMTP id d2mr20856822pat.36.1397813365031; Fri, 18 Apr 2014 02:29:25 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u6si16061611paa.216.2014.04.18.02.29.24; Fri, 18 Apr 2014 02:29:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753569AbaDRJ0V (ORCPT + 26 others); Fri, 18 Apr 2014 05:26:21 -0400 Received: from ip4-83-240-18-248.cust.nbox.cz ([83.240.18.248]:46058 "EHLO ip4-83-240-18-248.cust.nbox.cz" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752467AbaDRJWu (ORCPT ); Fri, 18 Apr 2014 05:22:50 -0400 Received: from ku by ip4-83-240-18-248.cust.nbox.cz with local (Exim 4.80.1) (envelope-from ) id 1Wb50T-0004Ky-Eq; Fri, 18 Apr 2014 11:22:49 +0200 From: Jiri Slaby To: stable@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Ard Biesheuvel , Herbert Xu , Jiri Slaby Subject: [PATCH 3.12 72/72] crypto: ghash-clmulni-intel - use C implementation for setkey() Date: Fri, 18 Apr 2014 11:22:45 +0200 Message-Id: X-Mailer: git-send-email 1.9.2 In-Reply-To: <3389f243c528afc7c7300c83b8f296290cd3656d.1397812482.git.jslaby@suse.cz> References: <3389f243c528afc7c7300c83b8f296290cd3656d.1397812482.git.jslaby@suse.cz> In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: linux-kernel-owner@vger.kernel.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.181 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Ard Biesheuvel 3.12-stable review patch. If anyone has any objections, please let me know. =============== commit 8ceee72808d1ae3fb191284afc2257a2be964725 upstream. The GHASH setkey() function uses SSE registers but fails to call kernel_fpu_begin()/kernel_fpu_end(). Instead of adding these calls, and then having to deal with the restriction that they cannot be called from interrupt context, move the setkey() implementation to the C domain. Note that setkey() does not use any particular SSE features and is not expected to become a performance bottleneck. Signed-off-by: Ard Biesheuvel Acked-by: H. Peter Anvin Fixes: 0e1227d356e9b (crypto: ghash - Add PCLMULQDQ accelerated implementation) Signed-off-by: Herbert Xu Signed-off-by: Jiri Slaby --- arch/x86/crypto/ghash-clmulni-intel_asm.S | 29 ----------------------------- arch/x86/crypto/ghash-clmulni-intel_glue.c | 14 +++++++++++--- 2 files changed, 11 insertions(+), 32 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 586f41aac361..185fad49d86f 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -24,10 +24,6 @@ .align 16 .Lbswap_mask: .octa 0x000102030405060708090a0b0c0d0e0f -.Lpoly: - .octa 0xc2000000000000000000000000000001 -.Ltwo_one: - .octa 0x00000001000000000000000000000001 #define DATA %xmm0 #define SHASH %xmm1 @@ -134,28 +130,3 @@ ENTRY(clmul_ghash_update) .Lupdate_just_ret: ret ENDPROC(clmul_ghash_update) - -/* - * void clmul_ghash_setkey(be128 *shash, const u8 *key); - * - * Calculate hash_key << 1 mod poly - */ -ENTRY(clmul_ghash_setkey) - movaps .Lbswap_mask, BSWAP - movups (%rsi), %xmm0 - PSHUFB_XMM BSWAP %xmm0 - movaps %xmm0, %xmm1 - psllq $1, %xmm0 - psrlq $63, %xmm1 - movaps %xmm1, %xmm2 - pslldq $8, %xmm1 - psrldq $8, %xmm2 - por %xmm1, %xmm0 - # reduction - pshufd $0b00100100, %xmm2, %xmm1 - pcmpeqd .Ltwo_one, %xmm1 - pand .Lpoly, %xmm1 - pxor %xmm1, %xmm0 - movups %xmm0, (%rdi) - ret -ENDPROC(clmul_ghash_setkey) diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 6759dd1135be..d785cf2c529c 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -30,8 +30,6 @@ void clmul_ghash_mul(char *dst, const be128 *shash); void clmul_ghash_update(char *dst, const char *src, unsigned int srclen, const be128 *shash); -void clmul_ghash_setkey(be128 *shash, const u8 *key); - struct ghash_async_ctx { struct cryptd_ahash *cryptd_tfm; }; @@ -58,13 +56,23 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) { struct ghash_ctx *ctx = crypto_shash_ctx(tfm); + be128 *x = (be128 *)key; + u64 a, b; if (keylen != GHASH_BLOCK_SIZE) { crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return -EINVAL; } - clmul_ghash_setkey(&ctx->shash, key); + /* perform multiplication by 'x' in GF(2^128) */ + a = be64_to_cpu(x->a); + b = be64_to_cpu(x->b); + + ctx->shash.a = (__be64)((b << 1) | (a >> 63)); + ctx->shash.b = (__be64)((a << 1) | (b >> 63)); + + if (a >> 63) + ctx->shash.b ^= cpu_to_be64(0xc2); return 0; }