From patchwork Fri Oct 5 08:13:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148147 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp150966lji; Fri, 5 Oct 2018 01:13:46 -0700 (PDT) X-Google-Smtp-Source: ACcGV6359EQyChauPxgLvPrm2mTmCaTSSofdB3cV2SYF271SF46+JoWUY/zJeBWeyWKwWRxzkPzK X-Received: by 2002:a17:902:7283:: with SMTP id d3-v6mr10347960pll.326.1538727226227; Fri, 05 Oct 2018 01:13:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727226; cv=none; d=google.com; s=arc-20160816; b=cFhhfVgRmh6lYE83/5+Fay4YfhA7kCCDgPBpPoq3YZs0Yvk3MYzuddZ7MXzkmpe75F lEwD6n+BcRUeXaL1MQiH3JuzccEiV/F/sehcZjl4iE5eVih9hT2AKRj5421IZ3qTaIoo Wnw9LVMwPjp+Gu5CjPx9Oa2fcKJXpJ5wTybJUCgnTGR7/+mofIrCI+jbcG4JOHM1T//T Bh7QJNDQyXEzQXdkmOEjVWYJ6BkQ8GurhJSNSnFmtipBP77XPGFApGfT2vG3bc3OxH4c EMylf5iPoCrJX0fydNhgLcymWASRYxMHYjSk1h94xuO+VBm829coWYeOeKaDLaqkY1LO zHiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=M974k2iyDJIOjsCt9yr954r2LChEqIlYPLi3JqN8ezM=; b=IkUlIvG/0kHaakUy3lx4uKxqbo3GNtiipvMY04eMYgE80RKUqhsSSwHRHvIMfpnY8b +RGiz6yWshoixldhr6dBAXs9ltZUoyw0pfga8TlLfL+jsjQ9UkM4e3pKd5RAUV/YsbFl ejp37d0sA8X6oQw3Sz40TpJpruTvcmUF80Q7PI0JPNapDde97fI+akF02Q9KaFbKOsxg 8fWznPutdgFIGk4WoNPoLxFUyNp1tamclE+4TvmkrlGN8BcOZFmi5HdJU8fzkdEAmjxK XPsUfkXDWgampbfxrCtR4I1d+5m5bYYs1UuAATSUBqu9nk0Ofip89I/3phPNoS0gF+wp P03A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MWd+hOYP; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y30-v6si7612388pgk.13.2018.10.05.01.13.45; Fri, 05 Oct 2018 01:13:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MWd+hOYP; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728407AbeJEPLS (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:18 -0400 Received: from mail-wm1-f66.google.com ([209.85.128.66]:33122 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728259AbeJEPLR (ORCPT ); Fri, 5 Oct 2018 11:11:17 -0400 Received: by mail-wm1-f66.google.com with SMTP id y140-v6so3198694wmd.0 for ; Fri, 05 Oct 2018 01:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=M974k2iyDJIOjsCt9yr954r2LChEqIlYPLi3JqN8ezM=; b=MWd+hOYP/XT8JhDN37zPYJpwEJYaMjz+DGw/Gv25gw+eg0s/a5gjnsTq2fxMFRjXfa XOMVBcpP7CyWqNHbBWmey7hjHyMNnG+gxrzc/jDFOObGBsroeG0GPzJLFwCtev3yPYQp vVyxleV0C7JUqQAW22vpLbZCcIWLOpw1cxhDg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=M974k2iyDJIOjsCt9yr954r2LChEqIlYPLi3JqN8ezM=; b=S3OJbaNXf01x3dL6PRKNk8CArExdDMpYeJpJq/XQR+7f45+SusiWL042j6mqyINvU7 gpHa80Fk/AfbI7EWHH5bNrmitfP9aTX/v8RzMhiRajh6wFHpk6gmuGcWSMWGxYwCIoeZ B4+JDokLGGScpnj/yHStic4YejoPHfun5az9Y7otvUwWSBcur/KiJwwCvhLllidcQVw/ 3Q+HrYEl2PxfGPU65VznPrfxr2CpANaKLPrObl53ECA8Mb/KraE+YPCVV5+HkwX81B0L I67kBBhq+CfbA2knGym9+ZwTOx5FbAFg74CsIi0KlWqLrebLzrhHgLW8tEhIAfw6GdIY a8Kg== X-Gm-Message-State: ABuFfoggDQ5Rtptf68p3s9W20q/Imk7whzT0fB/ecLiHP8BOqCnvq5hF QKmSQhgDIBLCHADrZZ9I152gncywoc4= X-Received: by 2002:a1c:9901:: with SMTP id b1-v6mr4381093wme.15.1538727220155; Fri, 05 Oct 2018 01:13:40 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:39 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 1/9] kernel: add support for patchable function pointers Date: Fri, 5 Oct 2018 10:13:25 +0200 Message-Id: <20181005081333.15018-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add a function pointer abstraction that can be implemented by the arch in a manner that avoids the downsides of function pointers, i.e., the fact that they are typically located in a writable data section, and their vulnerability to Spectre like defects. The FFP (or fast function pointer) is callable as a function, since the generic incarnation is simply that. However, due to the fact that C does not distinguish between functions and function pointers at the call site, the architecture can instead emit it as a patchable sequence of instructions consisting of ordinary branches. Signed-off-by: Ard Biesheuvel --- arch/Kconfig | 3 ++ include/linux/ffp.h | 43 ++++++++++++++++++++ 2 files changed, 46 insertions(+) -- 2.11.0 diff --git a/arch/Kconfig b/arch/Kconfig index 6801123932a5..2af3442a61d3 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -862,6 +862,9 @@ config HAVE_ARCH_PREL32_RELOCATIONS architectures, and don't require runtime relocation on relocatable kernels. +config HAVE_ARCH_FFP + bool + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig" diff --git a/include/linux/ffp.h b/include/linux/ffp.h new file mode 100644 index 000000000000..8fc3b4c9b38f --- /dev/null +++ b/include/linux/ffp.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __LINUX_FFP_H +#define __LINUX_FFP_H + +#include +#include + +#ifdef CONFIG_HAVE_ARCH_FFP +#include +#else + +struct ffp { + void (**fn)(void); + void (*default_fn)(void); +}; + +#define DECLARE_FFP(_fn, _def) \ + extern typeof(_def) *_fn; \ + extern struct ffp const __ffp_ ## _fn + +#define DEFINE_FFP(_fn, _def) \ + typeof(_def) *_fn = &_def; \ + struct ffp const __ffp_ ## _fn \ + = { (void(**)(void))&_fn, (void(*)(void))&_def }; \ + EXPORT_SYMBOL(__ffp_ ## _fn) + +static inline void ffp_set_target(const struct ffp *m, void *new_fn) +{ + WRITE_ONCE(*m->fn, new_fn); +} + +static inline void ffp_reset_target(const struct ffp *m) +{ + WRITE_ONCE(*m->fn, m->default_fn); +} + +#endif + +#define SET_FFP(_fn, _new) ffp_set_target(&__ffp_ ## _fn, _new) +#define RESET_FFP(_fn) ffp_reset_target(&__ffp_ ## _fn) + +#endif From patchwork Fri Oct 5 08:13:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148149 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151026lji; Fri, 5 Oct 2018 01:13:49 -0700 (PDT) X-Google-Smtp-Source: ACcGV62U/l6brmqGllndP+PnSwFEGkZs5/6uTthh9dbkMwRdQ5X8mTObbf8vZ95xeD5DkUWCOQss X-Received: by 2002:a63:f848:: with SMTP id v8-v6mr8970692pgj.82.1538727229410; Fri, 05 Oct 2018 01:13:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727229; cv=none; d=google.com; s=arc-20160816; b=VgEtV3Sb35ib+20AbZtK68aZcpKW4sbLehAoPQcXuYVLJbQQabFjpZUbRDC31jeil5 ZEpAdvR1VRjqeKjSnqMUwwBcJALIRKL9WIsONg7GFpflUa1ukft2tCN83hGnHn8S3f7D AdezM5T8a9kGacme6HEi9rW+XM4SiKFjCBfecbRfQmEZHQcb3OHeM/uZpFSuwQXDr+MP 7AyfWDsUfEXSDr9d3fzMN6ULgPubyDM32tXmPOqHll0JhGcusXTgtA2FI8LywGLPxE0S 0HIa/ynCCyJxXepdb3Vd5sRgztPjHvuTEnwPQJ+ygCuCmCuXLpCscdjJPMdGBnsfVSaD zFWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=9pEAraO+w5FxZNKwOwU68gCXSvwsp2RrSj23yV0hAkg=; b=mjhR/SpHcTO/22rJ0yoh2yA6qugikzG9v9JvpwG4QFAA7AOUZ05SkVAhBtYOlybYy2 0kNGvrymxmpW88c4vSVvyH3CQGb+zmurRTccYluBtONOCD2nyYWCwqVMVDjN1Al6RcVS H+pyum/NNWgCFLfGhk/0lenaUGTmJ5kl1R9pentIuQAAJvfc29SrwiIgmXfc1Z6QRi1r 5nsJEAajZxviMWkMZBAbq69CkUMvalMFgZJ3Kbh/p1w78WXhMN/Yn207CqUf2zlxda7q FUZckXwhZAkJTY5T07n0MsnUdl8Z6KjgzxnX55I+tOGDlFPjzRF7xNyvAf/sq2pb34k/ yh2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e+cypP9O; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v14-v6si7754716plo.208.2018.10.05.01.13.49; Fri, 05 Oct 2018 01:13:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e+cypP9O; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728491AbeJEPLV (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:21 -0400 Received: from mail-wr1-f66.google.com ([209.85.221.66]:36442 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728200AbeJEPLU (ORCPT ); Fri, 5 Oct 2018 11:11:20 -0400 Received: by mail-wr1-f66.google.com with SMTP id y16so12578534wrw.3 for ; Fri, 05 Oct 2018 01:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9pEAraO+w5FxZNKwOwU68gCXSvwsp2RrSj23yV0hAkg=; b=e+cypP9O6uanr3tyDWz0c6akRdpCzHEznDk/UGqdh0XmBEz5LaG5BHA8TQpA89x4OX 1yb2GPuTj7EKXgJXcoo8GSSuT10ecGI2deQSCUBfSElkNv+n92lTnZFWQxHmqCgl4FEe ouWG6NoD/o9bqb2rorXsdLVQB0OmZTM4XGPlM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9pEAraO+w5FxZNKwOwU68gCXSvwsp2RrSj23yV0hAkg=; b=Ln62Z9yIE4QMGbmj3gtbmQEyoQ9sDcRS65rWz8CMIrdcXgzAgsyShlU+WcK3AwGie+ rCpuOUc+/2TwSsvH2X+Uh4S13UaAVfGdgmgOQC/LrcLxUxsPG43I/f3bQQmbem8vFFzp 81fDgf8DvqysjjvK/wTdbBaZfIqc0XkgDT3LBuYP3anZjM8myBMUZc2L5v5mZQKSSJLE 74WvEDjaT3h/UEv/86KQQTqOcaFzIrWjhWvTbYnB5St0M1AL31T8d2Mkb/4RR7wYnzzF p4NTcZq+sR5MFv4SE3tMKXhsbVcxiG2wCE13ViaVXvV5mhCtl08N+zlZPpq2sHDfgJlN NKMA== X-Gm-Message-State: ABuFfoietSWkFIbTeUf+GYbV9B3FSw2qPAb/94IhmC2JEFJ7RRQpWO1c rjjujdH2JUCQGdZI6/ejIzmSkwJYt64= X-Received: by 2002:a5d:610a:: with SMTP id v10-v6mr1391403wrt.308.1538727223269; Fri, 05 Oct 2018 01:13:43 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:42 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 3/9] crypto: crc-t10dif - make crc_t10dif a static inline Date: Fri, 5 Oct 2018 10:13:27 +0200 Message-Id: <20181005081333.15018-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org crc_t10dif() is a trivial wrapper around crc_t10dif_update() so move it into the header file as a static inline function. Signed-off-by: Ard Biesheuvel --- include/linux/crc-t10dif.h | 6 +++++- lib/crc-t10dif.c | 6 ------ 2 files changed, 5 insertions(+), 7 deletions(-) -- 2.11.0 diff --git a/include/linux/crc-t10dif.h b/include/linux/crc-t10dif.h index 6bb0c0bf357b..4dfe09ff4cf2 100644 --- a/include/linux/crc-t10dif.h +++ b/include/linux/crc-t10dif.h @@ -10,7 +10,11 @@ extern __u16 crc_t10dif_generic(__u16 crc, const unsigned char *buffer, size_t len); -extern __u16 crc_t10dif(unsigned char const *, size_t); extern __u16 crc_t10dif_update(__u16 crc, unsigned char const *, size_t); +static inline __u16 crc_t10dif(const unsigned char *buffer, size_t len) +{ + return crc_t10dif_update(0, buffer, len); +} + #endif diff --git a/lib/crc-t10dif.c b/lib/crc-t10dif.c index 4d0d47c1ffbd..036ee664c9e1 100644 --- a/lib/crc-t10dif.c +++ b/lib/crc-t10dif.c @@ -81,12 +81,6 @@ __u16 crc_t10dif_update(__u16 crc, const unsigned char *buffer, size_t len) } EXPORT_SYMBOL(crc_t10dif_update); -__u16 crc_t10dif(const unsigned char *buffer, size_t len) -{ - return crc_t10dif_update(0, buffer, len); -} -EXPORT_SYMBOL(crc_t10dif); - static int __init crc_t10dif_mod_init(void) { crypto_register_notifier(&crc_t10dif_nb); From patchwork Fri Oct 5 08:13:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148150 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151070lji; Fri, 5 Oct 2018 01:13:51 -0700 (PDT) X-Google-Smtp-Source: ACcGV60/4bMLJCWG83Ctv5o63udxe1Re7hvQpdhiBFsGarepJQYFJhKvHCo34GxVJckNq+cXXpIn X-Received: by 2002:a17:902:29e3:: with SMTP id h90-v6mr10453458plb.215.1538727231714; Fri, 05 Oct 2018 01:13:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727231; cv=none; d=google.com; s=arc-20160816; b=O0y93c51nKtcCLRZASNvvJkE+ePJotZAd581JeLEGhWhI3kRm0bmqxSK7nJiV+rJgC c9MExKoP4shxfBBXjRhKckiQjab1kRpOTzuFrXiQkesFvGmfMi2fiJncGVKyofon38Hi hBBQe742HRoo2aE4rpiv+j5sZWumnrmM6Jfj91DnwdAhx6qkLajTSqOvltokDp3ZloKX ROXOG0hIcF+n1j3n9ODDVYv5Z35z2piasqzggoNnsRZoeHUm2UmuaIDVCX0b5Hmp/1dn NVLRmvx3EB5CaYD9+pHMcJFLngYHsrPxphR4hrL7tqooNxryaVkXHnYGPWxIkEn5JvY1 b/MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=a4rjYB3iYTIV+tC4OFyOVJ+Wv4RwjebsyoAYasZ50dIEoMqjdcqBN90fWu2of/bZUW BvNeYzgo1zSDIIr4H9APlKOS2ngk2NHv6IgDsSr92Ogim/s8AjcQW6is1dzaC31QRQV0 Ipyc5S2jyk715YdFCCa/WrGkr9EUjOB7xUD6tbspAy/2d5djBhVksygzB+STOw+Sa4Mf bySPxTaN4/JdKmFXeJqGsGYvvBvGhtdSkvJYhIyeZewaK8IgWWzaH9jLYA4HLBTf2wdt jcU3Y8fNM9obE6AV4S4L6sdI62xJXh8Z4BBcjMeqpMqIvWDP32+VMkXcd5mzyhpLQwbS 1ErA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de+kc9fs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v14-v6si7754716plo.208.2018.10.05.01.13.51; Fri, 05 Oct 2018 01:13:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=de+kc9fs; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728550AbeJEPLZ (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:25 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:52838 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728495AbeJEPLY (ORCPT ); Fri, 5 Oct 2018 11:11:24 -0400 Received: by mail-wm1-f65.google.com with SMTP id 189-v6so986022wmw.2 for ; Fri, 05 Oct 2018 01:13:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=de+kc9fsZ4BjBKbbhzZrYUMq4lHYmA2vJFroD9TaHnog+QwlRkYWgz+hr1/fdeK2OC IyauWZM3k2PDGmd0msOw0d3zhlUGxbRNkeYk0lR3xcdIIZuvqGI85mWFDFE6IkdE4zfj NpLaGFTYQMdPLGYkiqxBV67e30UPsXDrw4d4s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BPVA95HGlQkQ8R4OJEDdFwk26jpvhoTlnJX6hJlBlqw=; b=KV6RdefJhlVokYUxOfUyo71DzANRGw7rByiq+LMXzQqI+DOEsiGM0/UtmZxOs+2Cwj dxAVIL7lRuEc3LfNLMd0lyrgz2nr5Wjcb8HP2VW8sC3FXqJy0+oQtv2DddsVnUoQe0tk wesG/3JpXf8IltIjZ4KG7m3DxyEofwvUzFreKNkfqYaDeRYLTPKO1oyEgYjSCm98vJYR mIe9RFrU3+WSgkZkqHMyL9wXl0k+B6fV9zPISayRhEO095DJZeFAUYo8wqTRA1ahZJWk lVyTX7WZ30u5LD0o7neuNDHT/gq2Z3BcNTGerEIppaA7voIeie21C8Vn3DarnRQhtFgJ Uilw== X-Gm-Message-State: ABuFfoix6WqBHvs6HIVE1SnUaYruECpQt5Al5bu2oQlqXijS1FAaUrK5 PuHsjOxTMkqJChyyFZaHzBm9x36bLYk= X-Received: by 2002:a1c:af07:: with SMTP id y7-v6mr7208461wme.12.1538727226568; Fri, 05 Oct 2018 01:13:46 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:45 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 5/9] crypto: crc-t10dif/arm64 - move PMULL based code into core library Date: Fri, 5 Oct 2018 10:13:29 +0200 Message-Id: <20181005081333.15018-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the PMULL based routines out of the crypto API into the core CRC-T10DIF library. Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/crct10dif-ce-glue.c | 61 +++++--------------- 1 file changed, 14 insertions(+), 47 deletions(-) -- 2.11.0 diff --git a/arch/arm64/crypto/crct10dif-ce-glue.c b/arch/arm64/crypto/crct10dif-ce-glue.c index b461d62023f2..c67db86aba2c 100644 --- a/arch/arm64/crypto/crct10dif-ce-glue.c +++ b/arch/arm64/crypto/crct10dif-ce-glue.c @@ -10,13 +10,12 @@ #include #include +#include #include #include #include #include -#include - #include #include @@ -25,27 +24,17 @@ asmlinkage u16 crc_t10dif_pmull_p64(u16 init_crc, const u8 buf[], u64 len); asmlinkage u16 crc_t10dif_pmull_p8(u16 init_crc, const u8 buf[], u64 len); -static u16 (*crc_t10dif_pmull)(u16 init_crc, const u8 buf[], u64 len); - -static int crct10dif_init(struct shash_desc *desc) -{ - u16 *crc = shash_desc_ctx(desc); - - *crc = 0; - return 0; -} +DEFINE_FFP(crc_t10dif_pmull, crc_t10dif_pmull_p8); -static int crct10dif_update(struct shash_desc *desc, const u8 *data, - unsigned int length) +static __u16 crct10dif_update_arm64(u16 crc, const u8 *data, unsigned int length) { - u16 *crc = shash_desc_ctx(desc); unsigned int l; if (unlikely((u64)data % CRC_T10DIF_PMULL_CHUNK_SIZE)) { l = min_t(u32, length, CRC_T10DIF_PMULL_CHUNK_SIZE - ((u64)data % CRC_T10DIF_PMULL_CHUNK_SIZE)); - *crc = crc_t10dif_generic(*crc, data, l); + crc = crc_t10dif_generic(crc, data, l); length -= l; data += l; @@ -54,51 +43,31 @@ static int crct10dif_update(struct shash_desc *desc, const u8 *data, if (length > 0) { if (may_use_simd()) { kernel_neon_begin(); - *crc = crc_t10dif_pmull(*crc, data, length); + crc = crc_t10dif_pmull(crc, data, length); kernel_neon_end(); } else { - *crc = crc_t10dif_generic(*crc, data, length); + crc = crc_t10dif_generic(crc, data, length); } } - return 0; -} - -static int crct10dif_final(struct shash_desc *desc, u8 *out) -{ - u16 *crc = shash_desc_ctx(desc); - - *(u16 *)out = *crc; - return 0; + return crc; } -static struct shash_alg crc_t10dif_alg = { - .digestsize = CRC_T10DIF_DIGEST_SIZE, - .init = crct10dif_init, - .update = crct10dif_update, - .final = crct10dif_final, - .descsize = CRC_T10DIF_DIGEST_SIZE, - - .base.cra_name = "crct10dif", - .base.cra_driver_name = "crct10dif-arm64-ce", - .base.cra_priority = 200, - .base.cra_blocksize = CRC_T10DIF_BLOCK_SIZE, - .base.cra_module = THIS_MODULE, -}; - static int __init crc_t10dif_mod_init(void) { if (elf_hwcap & HWCAP_PMULL) - crc_t10dif_pmull = crc_t10dif_pmull_p64; - else - crc_t10dif_pmull = crc_t10dif_pmull_p8; + SET_FFP(crc_t10dif_pmull, crc_t10dif_pmull_p64); - return crypto_register_shash(&crc_t10dif_alg); + SET_FFP(crc_t10dif_update_arch, crct10dif_update_arm64); + + return 0; } static void __exit crc_t10dif_mod_exit(void) { - crypto_unregister_shash(&crc_t10dif_alg); + RESET_FFP(crc_t10dif_update_arch); + /* wait for all users to switch back to the old version */ + synchronize_rcu(); } module_cpu_feature_match(ASIMD, crc_t10dif_mod_init); @@ -106,5 +75,3 @@ module_exit(crc_t10dif_mod_exit); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); -MODULE_ALIAS_CRYPTO("crct10dif"); -MODULE_ALIAS_CRYPTO("crct10dif-arm64-ce"); From patchwork Fri Oct 5 08:13:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148152 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151128lji; Fri, 5 Oct 2018 01:13:54 -0700 (PDT) X-Google-Smtp-Source: ACcGV60sI0etu5y0zpI0PfsiL+7p1ieO9AzTfNjjgbZmZHoqYhjGkz34wmehnwYmpHTtAIxNJKDb X-Received: by 2002:a62:8490:: with SMTP id k138-v6mr10635731pfd.177.1538727234673; Fri, 05 Oct 2018 01:13:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727234; cv=none; d=google.com; s=arc-20160816; b=HFdc+TIaBMHJfRUKAcNR+FcdmyzjC+1kygxdsBlomCJC8ewn4vC//ki/Iqhs6iRc87 aU4jXnzFRBOSCI35dilWhRy87iXlfd7euMxAaa2T6y+hnjKBnwFvb5Eq2uznf+e6PBy9 M1A/hYs1ojSquMNINJl1newo2++FHAKul8w8OcIHcgpGb9oHY334dsjHBlHvAqTlahCw 4kOKIMCIufVzNyssguh5LkLXimtxGwjyCUvq8EPffPybbSz6fIrHjqNXZ5CK3ZEF33ii J0PcI8E0HAJXnngQyuNlNuOuZyDqswXo2sVphtxCD06pGxZlrr3LtX7Yulxtq+403J80 pXuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=W4XNrL/HVz1JFq0EC5F2/AltrGoh1ZboRDNEWdxrwVU=; b=k3XcmP98MZGWY6YJ+P1+UFgY2tGq+xwXG+MC/b2Zu5mXLd157GEZiEnnlirqbz7mWq SaqhEEHAEdwU1sAA21m9lZESFPoV9MkBj8QoVwGKetvO3WVMM3SPrCMiVH9XT8MJKCun 3JsrmXqd4Wv4Gk+sifvAjhRFt6C2l0WEO8NHMTsq2QePi+vBpzuXDb1G+3nEM+ME3ALC T3I9NkL4ph1euyi4QLMS2S7GOieDmbG2ooVD3JZx7j6HQaTHiwfh+rYwjBSGlFwoBp+X Ze21a4r9SHXr4/IDZB/ix+LPa4P8uvDgqUYGH/h2C1X9dL8YHiSiBzdvP+Q83MwInHiJ WoZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jnwyIatH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g15-v6si7691811plo.284.2018.10.05.01.13.54; Fri, 05 Oct 2018 01:13:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jnwyIatH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728577AbeJEPL0 (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:26 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:46040 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728498AbeJEPL0 (ORCPT ); Fri, 5 Oct 2018 11:11:26 -0400 Received: by mail-wr1-f67.google.com with SMTP id q5-v6so12553743wrw.12 for ; Fri, 05 Oct 2018 01:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=W4XNrL/HVz1JFq0EC5F2/AltrGoh1ZboRDNEWdxrwVU=; b=jnwyIatHaYA2n3cTx6guJoyPbr+L3fOA9AOBqx0Y2oMaeF96hobAONVdvSUned1y3s StByQ8z+FybmB6IbckDsErTVElZPazLc7iw2ihFw0/S6pZiXC3Q6LPbxewFO5smDCYZf uOH6PjRG3nJAjWfuYmqeU3mfvrJ5BOnGliwTE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=W4XNrL/HVz1JFq0EC5F2/AltrGoh1ZboRDNEWdxrwVU=; b=OIf9aQMLs19DcnnidxZLwv6T347RFRoG/dXws7ToGCpsQuLOTufIvcJBq25vFubTpR ZIHXHXs4DWI9FAUZ2Igw5W9+hj2R0Vb19jyu5e0YYv2wB+TSyltgos6IBRBNshCwiJ7J RPCTEsDBD7Q22f8+rOfU+u7D+hf0jivHi+iTjdH7dUiWSONHuZe7x9XC72rQPOu2Qwd8 qb9ARxsI1XPqBTYw+sHmSTmheMOsJD3dMt77IC116tqOpn+CVVubRY/hv19CJHNUc3FH cvqTK0kxyRqJ/enswfsk8WrLj8vGjX7bq6zNLXfrMUE46M7xRr+eRUjPcnDidXO/zF5f S8qg== X-Gm-Message-State: ABuFfoj9sFlN0Etl3LbvdhiSS7IwyDAPKV19SPoPQT2nes31YYugfcPi KCZJ0hlF00HzKtmW1PuRrxlvNVRRkj8= X-Received: by 2002:a5d:40cd:: with SMTP id b13-v6mr7838866wrq.133.1538727228189; Fri, 05 Oct 2018 01:13:48 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:47 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 6/9] crypto: crc-t10dif/arm - move PMULL based code into core library Date: Fri, 5 Oct 2018 10:13:30 +0200 Message-Id: <20181005081333.15018-7-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the PMULL based routines out of the crypto API into the core CRC-T10DIF library. Signed-off-by: Ard Biesheuvel --- arch/arm/crypto/crct10dif-ce-glue.c | 78 ++++++-------------- 1 file changed, 22 insertions(+), 56 deletions(-) -- 2.11.0 diff --git a/arch/arm/crypto/crct10dif-ce-glue.c b/arch/arm/crypto/crct10dif-ce-glue.c index d428355cf38d..2020d9c29ff1 100644 --- a/arch/arm/crypto/crct10dif-ce-glue.c +++ b/arch/arm/crypto/crct10dif-ce-glue.c @@ -10,12 +10,11 @@ #include #include +#include #include #include #include -#include - #include #include @@ -23,74 +22,42 @@ asmlinkage u16 crc_t10dif_pmull(u16 init_crc, const u8 buf[], u32 len); -static int crct10dif_init(struct shash_desc *desc) +static __u16 crct10dif_update_arm(__u16 crc, const u8 *data, unsigned int length) { - u16 *crc = shash_desc_ctx(desc); - - *crc = 0; - return 0; -} - -static int crct10dif_update(struct shash_desc *desc, const u8 *data, - unsigned int length) -{ - u16 *crc = shash_desc_ctx(desc); unsigned int l; - if (!may_use_simd()) { - *crc = crc_t10dif_generic(*crc, data, length); - } else { - if (unlikely((u32)data % CRC_T10DIF_PMULL_CHUNK_SIZE)) { - l = min_t(u32, length, CRC_T10DIF_PMULL_CHUNK_SIZE - - ((u32)data % CRC_T10DIF_PMULL_CHUNK_SIZE)); + if (!may_use_simd()) + return crc_t10dif_generic(crc, data, length); - *crc = crc_t10dif_generic(*crc, data, l); + if (unlikely((u32)data % CRC_T10DIF_PMULL_CHUNK_SIZE)) { + l = min_t(u32, length, CRC_T10DIF_PMULL_CHUNK_SIZE - + ((u32)data % CRC_T10DIF_PMULL_CHUNK_SIZE)); - length -= l; - data += l; - } - if (length > 0) { - kernel_neon_begin(); - *crc = crc_t10dif_pmull(*crc, data, length); - kernel_neon_end(); - } - } - return 0; -} + crc = crc_t10dif_generic(crc, data, l); -static int crct10dif_final(struct shash_desc *desc, u8 *out) -{ - u16 *crc = shash_desc_ctx(desc); - - *(u16 *)out = *crc; - return 0; + length -= l; + data += l; + } + if (length > 0) { + kernel_neon_begin(); + crc = crc_t10dif_pmull(crc, data, length); + kernel_neon_end(); + } + return crc; } -static struct shash_alg crc_t10dif_alg = { - .digestsize = CRC_T10DIF_DIGEST_SIZE, - .init = crct10dif_init, - .update = crct10dif_update, - .final = crct10dif_final, - .descsize = CRC_T10DIF_DIGEST_SIZE, - - .base.cra_name = "crct10dif", - .base.cra_driver_name = "crct10dif-arm-ce", - .base.cra_priority = 200, - .base.cra_blocksize = CRC_T10DIF_BLOCK_SIZE, - .base.cra_module = THIS_MODULE, -}; - static int __init crc_t10dif_mod_init(void) { - if (!(elf_hwcap2 & HWCAP2_PMULL)) - return -ENODEV; + SET_FFP(crc_t10dif_update_arch, crct10dif_update_arm); - return crypto_register_shash(&crc_t10dif_alg); + return 0; } static void __exit crc_t10dif_mod_exit(void) { - crypto_unregister_shash(&crc_t10dif_alg); + RESET_FFP(crc_t10dif_update_arch); + /* wait for all users to switch back to the old version */ + synchronize_rcu(); } module_init(crc_t10dif_mod_init); @@ -98,4 +65,3 @@ module_exit(crc_t10dif_mod_exit); MODULE_AUTHOR("Ard Biesheuvel "); MODULE_LICENSE("GPL v2"); -MODULE_ALIAS_CRYPTO("crct10dif"); From patchwork Fri Oct 5 08:13:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 148154 Delivered-To: patch@linaro.org Received: by 2002:a2e:8595:0:0:0:0:0 with SMTP id b21-v6csp151235lji; Fri, 5 Oct 2018 01:14:03 -0700 (PDT) X-Google-Smtp-Source: ACcGV63R3Xf62S85JjXMKvvOOfUmVtucBbg6KwCNM4ZmNWd8m81Kb2tAgVKBTVq7A/Te2fxYz7uf X-Received: by 2002:a17:902:b213:: with SMTP id t19-v6mr10586788plr.51.1538727243158; Fri, 05 Oct 2018 01:14:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538727243; cv=none; d=google.com; s=arc-20160816; b=u5LNLFxrpHHjjVqOxdBX2o+htSGLQYucI6jT/DdFGvz5L/R1monv0GTznkWECD4ly0 ff2fvmPs8nQ1UeYhSuJvto3IbS/T8kJP54geQESfmUJVJmzfCdDXtUL+qBE318CkTa0D uWO2UZTCLq88zL+Hus3vbVPm0skdKAoa2Fq06cz3TbzB49aoCCME9vsz9VdrRPLezxwk dX7qNt+O1bBFPv4S4toe021dRwiti2kBgiIcAIUlysH95L5rn6SQeb2CwjDLdIlHvwth xiX6II8GsZ6dhgeUj4NkYpJv3Z4WoHO4LgChBBf36stYhH6al1Ab/o60kfVDXBw87esH Z8OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=bI/Y1BaMDdXUGi8izs00Syk/DQmcRijPgw+/FGfCOtekx6Bc1VvhWRDT9DFHRsdnGl Iv6BLzSD4I0pBkLddCR6g22NFKtxYDw17lWUXQFt+oI+zldPOLbdsJgsOkB6Kjw/SVNO Ce+Q4mKusbw60hA7Fb6X3kyTNY3MBsc2K9Uc92xCnR3hsFqM00Aop03d9WgXZJ5I9JBe 1ROcTna1LLCNp0GGnWsy5euT+6vob/96JKBe/h9hon8tr1J01oePv92fAIq2ngW6uI5i SQxuyheeExtvpjrYcCRXSfuMj1fpb0MdXu6/T7QJOo7XRVODRUADXyBUBd0I5qCm831u YSdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJcLwtVX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n2-v6si7341775plk.255.2018.10.05.01.14.02; Fri, 05 Oct 2018 01:14:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJcLwtVX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728614AbeJEPLg (ORCPT + 32 others); Fri, 5 Oct 2018 11:11:36 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35454 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728581AbeJEPLa (ORCPT ); Fri, 5 Oct 2018 11:11:30 -0400 Received: by mail-wm1-f68.google.com with SMTP id e187-v6so1036453wmf.0 for ; Fri, 05 Oct 2018 01:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=PJcLwtVXRuyJPKWbbwJBGaYD/0+A6pH/bRN1v8Ky+MsOeDl1X8nZVqwwcKBgUCWjct PMh/eQPIxbpQRmrMSUG0NVWRyNWL60QRHPjdWY8s3j9dnprSTAVQeym8nwAdgE0Kfn39 qt/i7giQasS5aiIXOxSLGpmt7KZ1soLYUoyuA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=A+1FdRMKX5kWT68MST48ZYdoIobJ5B3kYMhv71JKOAg=; b=tluxwEiIPDSJloz+1X+IG6iOxivHJCcj7VFN3GI3k9ynxYazam65OuSDmD1tqpjFrD 5gebJftGlpeApmHmueXgA6YLtF9JNym2F31lBl4viCwjQLERmTRkb4VkDUKFewIEQ3Fs mTc1qRayiXCbguKoSCzDyAcWHc/9Fd+xCFKx2x9/S/t9vspnweixVlNdTGvSsdybj3xL M+37WceBLO/GCstMm9bUfmcJzYm+aPF0LmTCkrTAIwITLdJeGm90QPkYKvswJ6mRsaXr wcEKQZAenow33QvJfepZN0fV3NEFyqGRYFK8lxabirZYh6EbfmDWPrEQUZzr7rJFL7NE MqkQ== X-Gm-Message-State: ABuFfogglaQiDw746rUjizPb1/G/yZaL05+jdkyh9NM9AXzw0riiQoJ5 baR9nVD+s/pEQSAARAXfnWfsHrNSEVE= X-Received: by 2002:a1c:9e8f:: with SMTP id h137-v6mr380822wme.142.1538727232789; Fri, 05 Oct 2018 01:13:52 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb1d:112:6f00:697e:67d9:a05d:22c7]) by smtp.gmail.com with ESMTPSA id t4-v6sm6565620wrb.45.2018.10.05.01.13.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Oct 2018 01:13:52 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , "Jason A . Donenfeld" , Eric Biggers , Samuel Neves , Andy Lutomirski , Arnd Bergmann , Herbert Xu , "David S. Miller" , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Thomas Gleixner , Ingo Molnar , Kees Cook , "Martin K. Petersen" , Greg Kroah-Hartman , Andrew Morton , Richard Weinberger , Peter Zijlstra , linux-crypto@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org Subject: [RFC PATCH 9/9] crypto: crc-t10dif/x86 - move PMULL based code into core library Date: Fri, 5 Oct 2018 10:13:33 +0200 Message-Id: <20181005081333.15018-10-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005081333.15018-1-ard.biesheuvel@linaro.org> References: <20181005081333.15018-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Move the PMULL based routines out of the crypto API into the core CRC-T10DIF library. Signed-off-by: Ard Biesheuvel --- arch/x86/crypto/crct10dif-pclmul_glue.c | 98 +++----------------- 1 file changed, 13 insertions(+), 85 deletions(-) -- 2.11.0 diff --git a/arch/x86/crypto/crct10dif-pclmul_glue.c b/arch/x86/crypto/crct10dif-pclmul_glue.c index cd4df9322501..47f0017ad7f3 100644 --- a/arch/x86/crypto/crct10dif-pclmul_glue.c +++ b/arch/x86/crypto/crct10dif-pclmul_glue.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -36,90 +36,17 @@ asmlinkage __u16 crc_t10dif_pcl(__u16 crc, const unsigned char *buf, size_t len); -struct chksum_desc_ctx { - __u16 crc; -}; - -/* - * Steps through buffer one byte at at time, calculates reflected - * crc using table. - */ - -static int chksum_init(struct shash_desc *desc) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - ctx->crc = 0; - - return 0; -} - -static int chksum_update(struct shash_desc *desc, const u8 *data, - unsigned int length) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - if (irq_fpu_usable()) { - kernel_fpu_begin(); - ctx->crc = crc_t10dif_pcl(ctx->crc, data, length); - kernel_fpu_end(); - } else - ctx->crc = crc_t10dif_generic(ctx->crc, data, length); - return 0; -} - -static int chksum_final(struct shash_desc *desc, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - *(__u16 *)out = ctx->crc; - return 0; -} - -static int __chksum_finup(__u16 *crcp, const u8 *data, unsigned int len, - u8 *out) +static __u16 crct10dif_update_x86(__u16 crc, const u8 *data, unsigned int length) { if (irq_fpu_usable()) { kernel_fpu_begin(); - *(__u16 *)out = crc_t10dif_pcl(*crcp, data, len); + crc = crc_t10dif_pcl(crc, data, length); kernel_fpu_end(); - } else - *(__u16 *)out = crc_t10dif_generic(*crcp, data, len); - return 0; -} - -static int chksum_finup(struct shash_desc *desc, const u8 *data, - unsigned int len, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - return __chksum_finup(&ctx->crc, data, len, out); -} - -static int chksum_digest(struct shash_desc *desc, const u8 *data, - unsigned int length, u8 *out) -{ - struct chksum_desc_ctx *ctx = shash_desc_ctx(desc); - - return __chksum_finup(&ctx->crc, data, length, out); -} - -static struct shash_alg alg = { - .digestsize = CRC_T10DIF_DIGEST_SIZE, - .init = chksum_init, - .update = chksum_update, - .final = chksum_final, - .finup = chksum_finup, - .digest = chksum_digest, - .descsize = sizeof(struct chksum_desc_ctx), - .base = { - .cra_name = "crct10dif", - .cra_driver_name = "crct10dif-pclmul", - .cra_priority = 200, - .cra_blocksize = CRC_T10DIF_BLOCK_SIZE, - .cra_module = THIS_MODULE, + } else { + crc = crc_t10dif_generic(crc, data, length); } -}; + return crc; +} static const struct x86_cpu_id crct10dif_cpu_id[] = { X86_FEATURE_MATCH(X86_FEATURE_PCLMULQDQ), @@ -132,12 +59,16 @@ static int __init crct10dif_intel_mod_init(void) if (!x86_match_cpu(crct10dif_cpu_id)) return -ENODEV; - return crypto_register_shash(&alg); + SET_FFP(crc_t10dif_update_arch, crct10dif_update_x86); + + return 0; } static void __exit crct10dif_intel_mod_fini(void) { - crypto_unregister_shash(&alg); + RESET_FFP(crc_t10dif_update_arch); + /* wait for all users to switch back to the old version */ + synchronize_rcu(); } module_init(crct10dif_intel_mod_init); @@ -146,6 +77,3 @@ module_exit(crct10dif_intel_mod_fini); MODULE_AUTHOR("Tim Chen "); MODULE_DESCRIPTION("T10 DIF CRC calculation accelerated with PCLMULQDQ."); MODULE_LICENSE("GPL"); - -MODULE_ALIAS_CRYPTO("crct10dif"); -MODULE_ALIAS_CRYPTO("crct10dif-pclmul");