From patchwork Fri Oct 5 08:13:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148150 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151070lji; Fri, 5 Oct 2018 01:13:51 -0700 (PDT) X-Google-Smtp-Source: ACcGV60/4bMLJCWG83Ctv5o63udxe1Re7hvQpdhiBFsGarepJQYFJhKvHCo34GxVJckNq+cXXpIn X-Received: by 2002:a17:902:29e3:: with SMTP id h90-v6mr10453458plb.215.1538727231714; Fri, 05 Oct 2018 01:13:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727231; cv=none; d=google.com; s=arc-20160816; b=O0y93c51nKtcCLRZASNvvJkE+ePJotZAd581JeLEGhWhI3kRm0bmqxSK7nJiV+rJgC c9MExKoP4shxfBBXjRhKckiQjab1kRpOTzuFrXiQkesFvGmfMi2fiJncGVKyofon38Hi hBBQe742HRoo2aE4rpiv+j5sZWumnrmM6Jfj91DnwdAhx6qkLajTSqOvltokDp3ZloKX ROXOG0hIcF+n1j3n9ODDVYv5Z35z2piasqzggoNnsRZoeHUm2UmuaIDVCX0b5Hmp/1dn NVLRmvx3EB5CaYD9+pHMcJFLngYHsrPxphR4hrL7tqooNxryaVkXHnYGPWxIkEn5JvY1 b/MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=a4rjYB3iYTIV+tC4OFyOVJ+Wv4RwjebsyoAYasZ50dIEoMqjdcqBN90fWu2of/bZUW BvNeYzgo1zSDIIr4H9APlKOS2ngk2NHv6IgDsSr92Ogim/s8AjcQW6is1dzaC31QRQV0 Ipyc5S2jyk715YdFCCa/WrGkr9EUjOB7xUD6tbspAy/2d5djBhVksygzB+STOw+Sa4Mf bySPxTaN4/JdKmFXeJqGsGYvvBvGhtdSkvJYhIyeZewaK8IgWWzaH9jLYA4HLBTf2wdt jcU3Y8fNM9obE6AV4S4L6sdI62xJXh8Z4BBcjMeqpMqIvWDP32+VMkXcd5mzyhpLQwbS 1ErA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de+kc9fs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v14-v6si7754716plo.208.2018.10.05.01.13.51; Fri, 05 Oct 2018 01:13:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de+kc9fs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728550AbeJEPLZ (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:25 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:52838 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728495AbeJEPLY (ORCPT ); Fri, 5 Oct 2018 11:11:24 -0400 Received: by mail-wm1-f65.google.com with SMTP id 189-v6so986022wmw.2 for ; Fri, 05 Oct 2018 01:13:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=de+kc9fsZ4BjBKbbhzZrYUMq4lHYmA2vJFroD9TaHnog+QwlRkYWgz+hr1/fdeK2OC IyauWZM3k2PDGmd0msOw0d3zhlUGxbRNkeYk0lR3xcdIIZuvqGI85mWFDFE6IkdE4zfj NpLaGFTYQMdPLGYkiqxBV67e30UPsXDrw4d4s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=KV6RdefJhlVokYUxOfUyo71DzANRGw7rByiq+LMXzQqI+DOEsiGM0/UtmZxOs+2Cwj dxAVIL7lRuEc3LfNLMd0lyrgz2nr5Wjcb8HP2VW8sC3FXqJy0+oQtv2DddsVnUoQe0tk wesG/3JpXf8IltIjZ4KG7m3DxyEofwvUzFreKNkfqYaDeRYLTPKO1oyEgYjSCm98vJYR mIe9RFrU3+WSgkZkqHMyL9wXl0k+B6fV9zPISayRhEO095DJZeFAUYo8wqTRA1ahZJWk lVyTX7WZ30u5LD0o7neuNDHT/gq2Z3BcNTGerEIppaA7voIeie21C8Vn3DarnRQhtFgJ Uilw== X-Gm-Message-State: ABuFfoix6WqBHvs6HIVE1SnUaYruECpQt5Al5bu2oQlqXijS1FAaUrK5 PuHsjOxTMkqJChyyFZaHzBm9x36bLYk= X-Received: by 2002:a1c:af07:: with SMTP id y7-v6mr7208461wme.12.1538727226568; Fri, 05 Oct 2018 01:13:46 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:45 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 5/9] crypto: crc-t10dif/arm64 - move PMULL based code into core library Date: Fri, 5 Oct 2018 10:13:29 +0200 Message-Id: <20181005081333.15018-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the PMULL based routines out of the crypto API into the core CRC-T10DIF library. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/crct10dif-ce-glue.c | 61 +++++--------------- 1 file changed, 14 insertions(+), 47 deletions(-) -- 2.11.0 diff --git a/arch/arm64/crypto/crct10dif-ce-glue.c b/arch/arm64/crypto/crct10dif-ce-glue.c index b461d62023f2..c67db86aba2c 100644 --- a/arch/arm64/crypto/crct10dif-ce-glue.c +++ b/arch/arm64/crypto/crct10dif-ce-glue.c @@ -10,13 +10,12 @@ #include #include +#include #include #include #include #include -#include - #include #include @@ -25,27 +24,17 @@ asmlinkage u16 crc_t10dif_pmull_p64(u16 init_crc, const u8 buf[], u64 len); asmlinkage u16 crc_t10dif_pmull_p8(u16 init_crc, const u8 buf[], u64 len); -static u16 (*crc_t10dif_pmull)(u16 init_crc, const u8 buf[], u64 len); - -static int crct10dif_init(struct shash_desc *desc) -{ - u16 *crc = shash_desc_ctx(desc); - - *crc = 0; - return 0; -} +DEFINE_FFP(crc_t10dif_pmull, crc_t10dif_pmull_p8); -static int crct10dif_update(struct shash_desc *desc, const u8 *data, - unsigned int length) +static __u16 crct10dif_update_arm64(u16 crc, const u8 *data, unsigned int length) { - u16 *crc = shash_desc_ctx(desc); unsigned int l; if (unlikely((u64)data % CRC_T10DIF_PMULL_CHUNK_SIZE)) { l = min_t(u32, length, CRC_T10DIF_PMULL_CHUNK_SIZE - ((u64)data % CRC_T10DIF_PMULL_CHUNK_SIZE)); - *crc = crc_t10dif_generic(*crc, data, l); + crc = crc_t10dif_generic(crc, data, l); length -= l; data += l; @@ -54,51 +43,31 @@ static int crct10dif_update(struct shash_desc *desc, const u8 *data, if (length > 0) { if (may_use_simd()) { kernel_neon_begin(); - *crc = crc_t10dif_pmull(*crc, data, length); + crc = crc_t10dif_pmull(crc, data, length); kernel_neon_end(); } else { - *crc = crc_t10dif_generic(*crc, data, length); + crc = crc_t10dif_generic(crc, data, length); } } - return 0; -} - -static int crct10dif_final(struct shash_desc *desc, u8 *out) -{ - u16 *crc = shash_desc_ctx(desc); - - *(u16 *)out = *crc; - return 0; + return crc; } -static struct shash_alg crc_t10dif_alg = { - .digestsize = CRC_T10DIF_DIGEST_SIZE, - .init = crct10dif_init, - .update = crct10dif_update, - .final = crct10dif_final, - .descsize = CRC_T10DIF_DIGEST_SIZE, - - .base.cra_name = "crct10dif", - .base.cra_driver_name = "crct10dif-arm64-ce", - .base.cra_priority = 200, - .base.cra_blocksize = CRC_T10DIF_BLOCK_SIZE, - .base.cra_module = THIS_MODULE, -}; - static int __init crc_t10dif_mod_init(void) { if (elf_hwcap & HWCAP_PMULL) - crc_t10dif_pmull = crc_t10dif_pmull_p64; - else - crc_t10dif_pmull = crc_t10dif_pmull_p8; + SET_FFP(crc_t10dif_pmull, crc_t10dif_pmull_p64); - return crypto_register_shash(&crc_t10dif_alg); + SET_FFP(crc_t10dif_update_arch, crct10dif_update_arm64); + + return 0; } static void __exit crc_t10dif_mod_exit(void) { - crypto_unregister_shash(&crc_t10dif_alg); + RESET_FFP(crc_t10dif_update_arch); + /* wait for all users to switch back to the old version */ + synchronize_rcu(); } module_cpu_feature_match(ASIMD, crc_t10dif_mod_init); @@ -106,5 +75,3 @@ module_exit(crc_t10dif_mod_exit); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); -MODULE_ALIAS_CRYPTO("crct10dif"); -MODULE_ALIAS_CRYPTO("crct10dif-arm64-ce");