From patchwork Tue Feb 21 19:09:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 655481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F50FC64EC7 for ; Tue, 21 Feb 2023 19:09:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229759AbjBUTJ2 (ORCPT ); Tue, 21 Feb 2023 14:09:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229484AbjBUTJW (ORCPT ); Tue, 21 Feb 2023 14:09:22 -0500 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFFC52BEFC for ; Tue, 21 Feb 2023 11:09:20 -0800 (PST) Received: by mail-ed1-x529.google.com with SMTP id ck15so22611783edb.0 for ; Tue, 21 Feb 2023 11:09:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FYNxCoy2kO8Df1US+5U8F6a2dVH3sUiqyDcA4T+cIdk=; b=EYITKlLr72rNFqewRWkBcZIH/vOCjOWBdgaXGm/V1n/d566Z7FAxepJOy1rHCPj6dl iZPY68A8qqNpvLChuWDvncH9eb8FPlf9oUzkfajT4Y0wErspOI9prAtce1ortH1TkJMQ 0GmxfCml7vv16dNhye981zY8nbwUXe1dnK7tWuhXAjwDCius2dbj0JGupkde2qZiZOxU zOu9rPfITBTwRMfp2zkXJ2gaRU3w+V52Y3zu+OdjMeTJx6iDbp8UK3soMZ3atBudw6e6 lF/NjUB56yU5KdfYM1Y3ZQ4YTKxtInhjUAkYP6NPkVzLPR3un1vHHidHWNsE/wI+aOqq aHNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FYNxCoy2kO8Df1US+5U8F6a2dVH3sUiqyDcA4T+cIdk=; b=3q5vuv38GM3ebUXnD6F5M34pmRZY0YsUl1YW8bWB0OycDRAFsbWXlx5wkxKqWVPAS1 ZiPqPlWX2Fo7ph1mzYQRtZltg7OyOoy6A4bxjWFDYqCbKXvGwVpnC8kNBqtMncHamIR1 PZ47ZnjoNhM9Qtq/HymBoAUTPhQPN9G4mqZBJUGNAOx+ivK7iAB399wFrCnQKYB/Pt8e XgjFSRNxckZKacbYIoAxUsLYSJS+vrHSm1c3HjsViXlUTfSq+vt3vVNjqhAu6J3hgpRI Xl6/4YOucwk02xfAODj4ydm1ca6nMqqYqzj62DlUzDCxa57s6lGMh+PFog8sJrV3GsR5 wLaQ== X-Gm-Message-State: AO0yUKUWXSdFVv9l7+Y7DkvbYsGawYe0lZiCr/fMi+DvoCfcae3SWv/h JhnD0hjAhVZX9QTlxv+TbTxEJx6m0KyD/+e/ X-Google-Smtp-Source: AK7set/w5FjER/3tKGB8RECJxQK3hqRtMHjbasbAVUNEy5Da6xeEsfNxs3m4JYfZBwWCY/jOTenDiQ== X-Received: by 2002:a17:907:1c0e:b0:8b1:3159:2eb3 with SMTP id nc14-20020a1709071c0e00b008b131592eb3mr21055233ejc.43.1677006559385; Tue, 21 Feb 2023 11:09:19 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id f6-20020a170906494600b008e240be0bddsm733310ejt.140.2023.02.21.11.09.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Feb 2023 11:09:19 -0800 (PST) From: Andrew Jones To: linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: 'Rob Herring ' , 'Jisheng Zhang ' , 'Anup Patel ' , 'Conor Dooley ' , 'Krzysztof Kozlowski ' , 'Heiko Stuebner ' , 'Paul Walmsley ' , 'Palmer Dabbelt ' , 'Albert Ou ' , 'Ben Dooks ' , 'Atish Patra ' Subject: [PATCH v5 1/8] RISC-V: alternatives: Support patching multiple insns in assembly Date: Tue, 21 Feb 2023 20:09:09 +0100 Message-Id: <20230221190916.572454-2-ajones@ventanamicro.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230221190916.572454-1-ajones@ventanamicro.com> References: <20230221190916.572454-1-ajones@ventanamicro.com> MIME-Version: 1.0 Content-type: text/plain Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org As pointed out in commit d374a16539b1 ("RISC-V: fix compile error from deduplicated __ALTERNATIVE_CFG_2"), we need quotes around parameters passed to macros within macros to avoid spaces being interpreted as separators. ALT_NEW_CONTENT was trying to handle this by defining new_c has a vararg, but this isn't sufficient for calling ALTERNATIVE() from assembly with multiple instructions in the new/old sequences. Remove the vararg "hack" and use quotes. Signed-off-by: Andrew Jones Reviewed-by: Conor Dooley --- arch/riscv/include/asm/alternative-macros.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/alternative-macros.h b/arch/riscv/include/asm/alternative-macros.h index 993a44a8fdac..b8c55fb3ab2c 100644 --- a/arch/riscv/include/asm/alternative-macros.h +++ b/arch/riscv/include/asm/alternative-macros.h @@ -14,7 +14,7 @@ .4byte \patch_id .endm -.macro ALT_NEW_CONTENT vendor_id, patch_id, enable = 1, new_c : vararg +.macro ALT_NEW_CONTENT vendor_id, patch_id, enable = 1, new_c .if \enable .pushsection .alternative, "a" ALT_ENTRY 886b, 888f, \vendor_id, \patch_id, 889f - 888f @@ -41,13 +41,13 @@ \old_c .option pop 887 : - ALT_NEW_CONTENT \vendor_id, \patch_id, \enable, \new_c + ALT_NEW_CONTENT \vendor_id, \patch_id, \enable, "\new_c" .endm .macro ALTERNATIVE_CFG_2 old_c, new_c_1, vendor_id_1, patch_id_1, enable_1, \ new_c_2, vendor_id_2, patch_id_2, enable_2 ALTERNATIVE_CFG "\old_c", "\new_c_1", \vendor_id_1, \patch_id_1, \enable_1 - ALT_NEW_CONTENT \vendor_id_2, \patch_id_2, \enable_2, \new_c_2 + ALT_NEW_CONTENT \vendor_id_2, \patch_id_2, \enable_2, "\new_c_2" .endm #define __ALTERNATIVE_CFG(...) ALTERNATIVE_CFG __VA_ARGS__ From patchwork Tue Feb 21 19:09:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 655480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB710C64ED6 for ; Tue, 21 Feb 2023 19:09:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229945AbjBUTJa (ORCPT ); Tue, 21 Feb 2023 14:09:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229852AbjBUTJ2 (ORCPT ); Tue, 21 Feb 2023 14:09:28 -0500 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35E3A2F792 for ; Tue, 21 Feb 2023 11:09:25 -0800 (PST) Received: by mail-ed1-x535.google.com with SMTP id cy6so15896980edb.5 for ; Tue, 21 Feb 2023 11:09:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GXedayrqa7eBo4rkkBiYovR8Wr8pe4sKgfVmovmcu18=; b=j/982Ctm96sVfHm/ro7bM8b7YKU6uEmLnHH3zdFR0JwFQSSlkXrzD+jMoXyli3VYtR TtsAXkyj59KhFyyvK+M+XVj1TgJYLsmw/l8bW3OHD0y3porahT8MSEGSpgVLfo1Ra3b+ CwhBanLhecrJPXkkW6kidWmp+WKCWn0ilAOhBwyvHjGN3XjpTMeaG/qM0jiTcJmJnWHy Py6PrxoHnY4wFrz/zGJHmYnPXiOXSkn3BlutKP7hH7fiDQe70ZMrJUpoRSkxTYhmocBh bb+gqtQ/z/JVipRnELQd6fIOSwweIhkMAVe5PupX4tanWoik/1vKhPUC0UXVsUBLXtIP DF5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GXedayrqa7eBo4rkkBiYovR8Wr8pe4sKgfVmovmcu18=; b=14h558SS2nb90QldkRIBtlBfsQApC7YsHXll5tmuPTYKFTMTDQGvEMajy+2xLEUFWc Sf+uQeUY9MChVHjkeDr2twC4IWcsULm/0Hr70YMTNkxA3OHRgIR24ZZ4z9fzD/nCRDbx NLIKQQbl/mMn9Vlefji5yEyUODxP5J5ZFzELMhlMX8meJdE4AEiHbfV2pKOVFPr30tBU j/h5rSdt2amPLQFLrKW13zHUU9bwdKT872KKqcJCwkoCZoDZaV9CyPhFI+EB1RwNZUXn luodRmPMPhf75PBZ6IMFFsyBp0CzkhdN46K+e+22S08z30HmQHOWED2EPYTLK06ZLdOU /cdQ== X-Gm-Message-State: AO0yUKUTAA6sWyY6rYd0lUIs4BChFsqbjhufc6BmdBoXAPskBLBPg0ZT LFeBmHCQb7WgFrsZK7EQQshJEA== X-Google-Smtp-Source: AK7set80pg34Z/dTD1nm35A+9eoM84IWV71wFC0+PR8j5t8jwA+a78wLQabIFXgkq+hRLQaeiCQT/w== X-Received: by 2002:a17:907:d25:b0:878:5c36:a14a with SMTP id gn37-20020a1709070d2500b008785c36a14amr19466820ejc.23.1677006563769; Tue, 21 Feb 2023 11:09:23 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id y14-20020a1709064b0e00b008c979c74732sm3931632eju.156.2023.02.21.11.09.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Feb 2023 11:09:23 -0800 (PST) From: Andrew Jones To: linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: 'Rob Herring ' , 'Jisheng Zhang ' , 'Anup Patel ' , 'Conor Dooley ' , 'Krzysztof Kozlowski ' , 'Heiko Stuebner ' , 'Paul Walmsley ' , 'Palmer Dabbelt ' , 'Albert Ou ' , 'Ben Dooks ' , 'Atish Patra ' Subject: [PATCH v5 4/8] RISC-V: Add Zicboz detection and block size parsing Date: Tue, 21 Feb 2023 20:09:12 +0100 Message-Id: <20230221190916.572454-5-ajones@ventanamicro.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230221190916.572454-1-ajones@ventanamicro.com> References: <20230221190916.572454-1-ajones@ventanamicro.com> MIME-Version: 1.0 Content-type: text/plain Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Parse "riscv,cboz-block-size" from the DT by piggybacking on Zicbom's riscv_init_cbom_blocksize(). Additionally check the DT for the presence of the "zicboz" extension and, when it's present, validate the parsed cboz block size as we do Zicbom's cbom block size with riscv_isa_extension_check(). Signed-off-by: Andrew Jones Reviewed-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/include/asm/cacheflush.h | 3 ++- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 10 ++++++++++ arch/riscv/kernel/setup.c | 2 +- arch/riscv/mm/cacheflush.c | 23 +++++++++++++++-------- 6 files changed, 30 insertions(+), 10 deletions(-) diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index 03e3b95ae6da..8091b8bf4883 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -50,7 +50,8 @@ void flush_icache_mm(struct mm_struct *mm, bool local); #endif /* CONFIG_SMP */ extern unsigned int riscv_cbom_block_size; -void riscv_init_cbom_blocksize(void); +extern unsigned int riscv_cboz_block_size; +void riscv_init_cbo_blocksizes(void); #ifdef CONFIG_RISCV_DMA_NONCOHERENT void riscv_noncoherent_supported(void); diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index ee9c80fe0062..bd025e918a08 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -47,6 +47,7 @@ #define RISCV_ISA_EXT_ZBB 30 #define RISCV_ISA_EXT_ZICBOM 31 #define RISCV_ISA_EXT_ZIHINTPAUSE 32 +#define RISCV_ISA_EXT_ZICBOZ 33 #ifndef __ASSEMBLY__ diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index 420228e219f7..7a3065544dcb 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -187,6 +187,7 @@ arch_initcall(riscv_cpuinfo_init); static struct riscv_isa_ext_data isa_ext_arr[] = { __RISCV_ISA_EXT_DATA(zbb, RISCV_ISA_EXT_ZBB), __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), + __RISCV_ISA_EXT_DATA(zicboz, RISCV_ISA_EXT_ZICBOZ), __RISCV_ISA_EXT_DATA(zihintpause, RISCV_ISA_EXT_ZIHINTPAUSE), __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 151e1a715db9..6102b6bb5db3 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -73,6 +73,15 @@ static bool riscv_isa_extension_check(int id) return false; } return true; + case RISCV_ISA_EXT_ZICBOZ: + if (!riscv_cboz_block_size) { + pr_err("Zicboz detected in ISA string, but no cboz-block-size found\n"); + return false; + } else if (!is_power_of_2(riscv_cboz_block_size)) { + pr_err("cboz-block-size present, but is not a power-of-2\n"); + return false; + } + return true; } return true; @@ -221,6 +230,7 @@ void __init riscv_fill_hwcap(void) SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); SET_ISA_EXT_MAP("zbb", RISCV_ISA_EXT_ZBB); SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); + SET_ISA_EXT_MAP("zicboz", RISCV_ISA_EXT_ZICBOZ); SET_ISA_EXT_MAP("zihintpause", RISCV_ISA_EXT_ZIHINTPAUSE); } #undef SET_ISA_EXT_MAP diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index 376d2827e736..5d3184cbf518 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -297,7 +297,7 @@ void __init setup_arch(char **cmdline_p) setup_smp(); #endif - riscv_init_cbom_blocksize(); + riscv_init_cbo_blocksizes(); riscv_fill_hwcap(); apply_boot_alternatives(); if (IS_ENABLED(CONFIG_RISCV_ISA_ZICBOM) && diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index eaf23fc14966..ba4832bb949b 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -98,6 +98,9 @@ void flush_icache_pte(pte_t pte) unsigned int riscv_cbom_block_size; EXPORT_SYMBOL_GPL(riscv_cbom_block_size); +unsigned int riscv_cboz_block_size; +EXPORT_SYMBOL_GPL(riscv_cboz_block_size); + static void cbo_get_block_size(struct device_node *node, const char *name, u32 *block_size, unsigned long *first_hartid) @@ -120,19 +123,23 @@ static void cbo_get_block_size(struct device_node *node, } } -void riscv_init_cbom_blocksize(void) +void riscv_init_cbo_blocksizes(void) { + unsigned long cbom_hartid, cboz_hartid; + u32 cbom_block_size = 0, cboz_block_size = 0; struct device_node *node; - unsigned long cbom_hartid; - u32 probed_block_size; - probed_block_size = 0; for_each_of_cpu_node(node) { - /* set block-size for cbom extension if available */ + /* set block-size for cbom and/or cboz extension if available */ cbo_get_block_size(node, "riscv,cbom-block-size", - &probed_block_size, &cbom_hartid); + &cbom_block_size, &cbom_hartid); + cbo_get_block_size(node, "riscv,cboz-block-size", + &cboz_block_size, &cboz_hartid); } - if (probed_block_size) - riscv_cbom_block_size = probed_block_size; + if (cbom_block_size) + riscv_cbom_block_size = cbom_block_size; + + if (cboz_block_size) + riscv_cboz_block_size = cboz_block_size; } From patchwork Tue Feb 21 19:09:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 655479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC706C6FA99 for ; Tue, 21 Feb 2023 19:09:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229799AbjBUTJc (ORCPT ); Tue, 21 Feb 2023 14:09:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjBUTJ3 (ORCPT ); Tue, 21 Feb 2023 14:09:29 -0500 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5BDD2FCCC for ; Tue, 21 Feb 2023 11:09:25 -0800 (PST) Received: by mail-ed1-x529.google.com with SMTP id ck15so22612743edb.0 for ; Tue, 21 Feb 2023 11:09:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=34NFoNBQKweqYqqRRhyhG2y14f8qhVJaos3C6Zb1qCA=; b=NyDRWGQLHDM4WE1MYSCVp6YW+mR5oILACbgX2cpEbZ+C1DA4Z8a4rlbPwZC2sOFHwf cvtP4Kpv/sZisnUc7GoEl3D7AYs9j/bPLf7FoggpE5+IaNqHPNbdm/EF9OyKjWd9pxE2 maV7OKvNYzIJ59NRQ8+JHyuK2aCSn6MEytvF8cC2+Uta/54D4d1eKP1iOHsN3KLPE/ya jjjuouOa5gFoXNYvY+OfVjdmbCvAl0y86fEmEF2f3KSS66UsrVKNGzXOAr3YECL+G8yK l05TJwz4xo3MExkf17ynt3J7nCTQQfP/E2yDzqhZc4+RO7GrIgT12sedGQazDqKkgBte Z2rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=34NFoNBQKweqYqqRRhyhG2y14f8qhVJaos3C6Zb1qCA=; b=s3EdzCxuQAXPhQRxAAikSd2kdQoGgffiCF4ixJJXs/Bd2DZuMohx5qG/Q+kJDs537c jJRtWRDhFFFNe4iAIhtBCN2DMq+HnaX9pvz3HH8NCVPRVcmLAeXW0kTA/8VsiDIJeOlU gzYj9JfGrGdxDX5ETSxFmeLCwk0apEjERkxyAcrOEOHCKL/bC+R7Xh3XWhI3twMtkCWi bxrNxATYvS6pmIBqipPvVOZOoE9Tfc95Sz/W6bcr4PERNz3j8r5f4UfPMUwP5d0SY1V1 sNa+HfD1pWJUp9ro6xbnbMZJD7Vkrp2av7CBTwzPBYmzBIMe8jWo707gQCjOX+xRaeIV +8dQ== X-Gm-Message-State: AO0yUKX9jbbnCOfuuq8oVAPO92fZvtyucP7A49MWj6dFBUx68/5FVYjx n4+tr4pcvt+cbv4DkfDCfVeH7w== X-Google-Smtp-Source: AK7set+rFsFWbTWMLIdMY+Q2/Z4HCUYcZHJSaxWuUbHlYQ1cFc34yRDDdrx0axW4+oXkdSAylmamTQ== X-Received: by 2002:aa7:d705:0:b0:4ac:89b:b605 with SMTP id t5-20020aa7d705000000b004ac089bb605mr7342612edq.22.1677006565381; Tue, 21 Feb 2023 11:09:25 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id i25-20020a508719000000b004acb42134c4sm2040769edb.70.2023.02.21.11.09.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Feb 2023 11:09:25 -0800 (PST) From: Andrew Jones To: linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: 'Rob Herring ' , 'Jisheng Zhang ' , 'Anup Patel ' , 'Conor Dooley ' , 'Krzysztof Kozlowski ' , 'Heiko Stuebner ' , 'Paul Walmsley ' , 'Palmer Dabbelt ' , 'Albert Ou ' , 'Ben Dooks ' , 'Atish Patra ' Subject: [PATCH v5 5/8] riscv: cpufeatures: Put the upper 16 bits of patch ID to work Date: Tue, 21 Feb 2023 20:09:13 +0100 Message-Id: <20230221190916.572454-6-ajones@ventanamicro.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230221190916.572454-1-ajones@ventanamicro.com> References: <20230221190916.572454-1-ajones@ventanamicro.com> MIME-Version: 1.0 Content-type: text/plain Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org cpufeature IDs are consecutive integers starting at 26, so a 32-bit patch ID allows an aircraft carrier load of feature IDs. Repurposing the upper 16 bits still leaves a boat load of feature IDs and gains 16 bits which may be used to control patching on a per patch-site basis. This will be initially used in Zicboz's application to clear_page(), as Zicboz's block size must also be considered. In that case, the upper 16-bit value's role will be to convey the maximum block size which the Zicboz clear_page() implementation supports. cpufeature patch sites which need to check for the existence or absence of other cpufeatures may also be able to make use of this. Signed-off-by: Andrew Jones --- arch/riscv/include/asm/alternative.h | 3 +++ arch/riscv/kernel/cpufeature.c | 37 +++++++++++++++++++++++++--- 2 files changed, 36 insertions(+), 4 deletions(-) diff --git a/arch/riscv/include/asm/alternative.h b/arch/riscv/include/asm/alternative.h index 8f39d4e8598d..f2cb543b0bd2 100644 --- a/arch/riscv/include/asm/alternative.h +++ b/arch/riscv/include/asm/alternative.h @@ -17,6 +17,9 @@ #include #include +#define PATCH_ID_CPUFEATURE_ID(p) ((u16)((u32)(p) & 0xffff)) +#define PATCH_ID_CPUFEATURE_VALUE(p) ((u16)(((u32)(p) >> 16) & 0xffff)) + #define RISCV_ALTERNATIVES_BOOT 0 /* alternatives applied during regular boot */ #define RISCV_ALTERNATIVES_MODULE 1 /* alternatives applied during module-init */ #define RISCV_ALTERNATIVES_EARLY_BOOT 2 /* alternatives applied before mmu start */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 6102b6bb5db3..0594989ead63 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -273,12 +273,35 @@ void __init riscv_fill_hwcap(void) } #ifdef CONFIG_RISCV_ALTERNATIVE +/* + * Alternative patch sites consider 48 bits when determining when to patch + * the old instruction sequence with the new. These bits are broken into a + * 16-bit vendor ID and a 32-bit patch ID. A non-zero vendor ID means the + * patch site is for an erratum, identified by the 32-bit patch ID. When + * the vendor ID is zero, the patch site is for a cpufeature. cpufeatures + * further break down patch ID into two 16-bit numbers. The lower 16 bits + * are the cpufeature ID and the upper 16 bits are used for a value specific + * to the cpufeature and patch site. If the upper 16 bits are zero, then it + * implies no specific value is specified. cpufeatures that want to control + * patching on a per-site basis will provide non-zero values and implement + * checks here. The checks return true when patching should be done, and + * false otherwise. + */ +static bool riscv_cpufeature_patch_check(u16 id, u16 value) +{ + if (!value) + return true; + + return false; +} + void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end, unsigned int stage) { struct alt_entry *alt; void *oldptr, *altptr; + u16 id, value; if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) return; @@ -286,13 +309,19 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, for (alt = begin; alt < end; alt++) { if (alt->vendor_id != 0) continue; - if (alt->patch_id >= RISCV_ISA_EXT_MAX) { - WARN(1, "This extension id:%d is not in ISA extension list", - alt->patch_id); + + id = PATCH_ID_CPUFEATURE_ID(alt->patch_id); + + if (id >= RISCV_ISA_EXT_MAX) { + WARN(1, "This extension id:%d is not in ISA extension list", id); continue; } - if (!__riscv_isa_extension_available(NULL, alt->patch_id)) + if (!__riscv_isa_extension_available(NULL, id)) + continue; + + value = PATCH_ID_CPUFEATURE_VALUE(alt->patch_id); + if (!riscv_cpufeature_patch_check(id, value)) continue; oldptr = ALT_OLD_PTR(alt); From patchwork Tue Feb 21 19:09:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 655478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C9FFC6FD1A for ; Tue, 21 Feb 2023 19:09:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229944AbjBUTJf (ORCPT ); Tue, 21 Feb 2023 14:09:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229846AbjBUTJa (ORCPT ); Tue, 21 Feb 2023 14:09:30 -0500 Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B4ED2BEFC for ; Tue, 21 Feb 2023 11:09:28 -0800 (PST) Received: by mail-ed1-x532.google.com with SMTP id g1so21132426edz.7 for ; Tue, 21 Feb 2023 11:09:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8S+XYy12JZsXVFfKIyUi1nIupPzXfiQ+NXxg+VU2ykw=; b=NcjwsDPzCzzl7IU7KmAT3C0cHSa9dnBMwfvf/ePCglGQwgK299eBEIKILm/428uQya OZ51tMuy7wOQ1p2QZt3gOpMAcNjddAxUqKYEjDNLjo9xMwt8TigPZ+esVX0TCEssAocl p+mn5qDAzPt75Vtokuc9DGn0Hu2TI+VIAFE2xockIH1Zw/q/ueBq6TQcbqG6N8W0Jf8J tS4pSdSrr6tsvMJkr7FRyqCvDu/HPJIpbO6+IRa3aBcBT4582wSYtoUrSFT0PYF3EiHB 2k2AytcwUEu/iYxTsCPQ5YbLjJu6aj2N+1i5Fuqy4xJWBiTKjTgRxYWUOxCJOuUVc6w4 iFpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8S+XYy12JZsXVFfKIyUi1nIupPzXfiQ+NXxg+VU2ykw=; b=HyvNi6p/+N07lKptGn0Pd8O1YGFD81KJKYPvPgrCTTI24qVySQ8cUr9XLhVlUBP4CZ V7tTERDQ7rsattJhRhrnGEtJ/jxsB1/FYW39B+eTemwSztFhQ40iNH+/RweElPukLBab wvkI7JVrVoorajfg2FXT/qkVuL7kwiNannIu4H+zcv3poXZCYezmXxCyYWKrWAVenQvx 7kEyTRddBPj50qtZlaSRPxaky+jSowgYNT15QsBwsBL8prTamlRHvopOuaRbiitShRSP VOWzsoFBzK2OXUdKF8trVS1DrDwbd0A2guGY8ZHGrxiJQrB8LoQKAOpyZSQ6K+ZuTBwz D9Ww== X-Gm-Message-State: AO0yUKVeZSzxTpyjn+Czu42ih9gPye9KVhzYBABtuwq0CxDAfdyWcKyR x1MxIg/UX7Livkt7w8kaXR0Fkg== X-Google-Smtp-Source: AK7set8lywRDYYj/sCGuAccQ8jI4g1DwGbTEkkC3GMmLISPTXn6PeXZU4cGsZ34cUgJdVmXau3tCbg== X-Received: by 2002:a17:906:37cb:b0:879:ab3:93cd with SMTP id o11-20020a17090637cb00b008790ab393cdmr13401168ejc.46.1677006566798; Tue, 21 Feb 2023 11:09:26 -0800 (PST) Received: from localhost (cst2-173-16.cust.vodafone.cz. [31.30.173.16]) by smtp.gmail.com with ESMTPSA id lf13-20020a170906ae4d00b008b12c318622sm7502111ejb.29.2023.02.21.11.09.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Feb 2023 11:09:26 -0800 (PST) From: Andrew Jones To: linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, kvm-riscv@lists.infradead.org Cc: 'Rob Herring ' , 'Jisheng Zhang ' , 'Anup Patel ' , 'Conor Dooley ' , 'Krzysztof Kozlowski ' , 'Heiko Stuebner ' , 'Paul Walmsley ' , 'Palmer Dabbelt ' , 'Albert Ou ' , 'Ben Dooks ' , 'Atish Patra ' Subject: [PATCH v5 6/8] RISC-V: Use Zicboz in clear_page when available Date: Tue, 21 Feb 2023 20:09:14 +0100 Message-Id: <20230221190916.572454-7-ajones@ventanamicro.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230221190916.572454-1-ajones@ventanamicro.com> References: <20230221190916.572454-1-ajones@ventanamicro.com> MIME-Version: 1.0 Content-type: text/plain Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org Using memset() to zero a 4K page takes 563 total instructions, where 20 are branches. clear_page(), with Zicboz and a 64 byte block size, takes 169 total instructions, where 4 are branches and 33 are nops. Even though the block size is a variable, thanks to alternatives, we can still implement a Duff device without having to do any preliminary calculations. This is achieved by using the alternatives' cpufeature value (the upper 16 bits of patch_id). The value used is the maximum zicboz block size order accepted at the patch site. This enables us to stop patching / unrolling when 4K bytes have been zeroed (we would loop and continue after 4K if the page size would be larger) For 4K pages, unrolling 16 times allows block sizes of 64 and 128 to only loop a few times and larger block sizes to not loop at all. Since cbo.zero doesn't take an offset, we also need an 'add' after each instruction, making the loop body 112 to 160 bytes. Hopefully this is small enough to not cause icache misses. Signed-off-by: Andrew Jones Acked-by: Conor Dooley --- arch/riscv/Kconfig | 13 ++++++ arch/riscv/include/asm/insn-def.h | 4 ++ arch/riscv/include/asm/page.h | 6 ++- arch/riscv/kernel/cpufeature.c | 11 +++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/clear_page.S | 73 +++++++++++++++++++++++++++++++ 6 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/lib/clear_page.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 0c494c36e911..c9006bcf912d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -456,6 +456,19 @@ config RISCV_ISA_ZICBOM If you don't know what to do here, say Y. +config RISCV_ISA_ZICBOZ + bool "Zicboz extension support for faster zeroing of memory" + depends on !XIP_KERNEL && MMU + select RISCV_ALTERNATIVE + default y + help + Enable the use of the ZICBOZ extension (cbo.zero instruction) + when available. + + The Zicboz extension is used for faster zeroing of memory. + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZIHINTPAUSE bool default y diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h index e01ab51f50d2..6960beb75f32 100644 --- a/arch/riscv/include/asm/insn-def.h +++ b/arch/riscv/include/asm/insn-def.h @@ -192,4 +192,8 @@ INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \ RS1(base), SIMM12(2)) +#define CBO_zero(base) \ + INSN_I(OPCODE_MISC_MEM, FUNC3(2), __RD(0), \ + RS1(base), SIMM12(4)) + #endif /* __ASM_INSN_DEF_H */ diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 9f432c1b5289..ccd168fe29d2 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -49,10 +49,14 @@ #ifndef __ASSEMBLY__ +#ifdef CONFIG_RISCV_ISA_ZICBOZ +void clear_page(void *page); +#else #define clear_page(pgaddr) memset((pgaddr), 0, PAGE_SIZE) +#endif #define copy_page(to, from) memcpy((to), (from), PAGE_SIZE) -#define clear_user_page(pgaddr, vaddr, page) memset((pgaddr), 0, PAGE_SIZE) +#define clear_user_page(pgaddr, vaddr, page) clear_page(pgaddr) #define copy_user_page(vto, vfrom, vaddr, topg) \ memcpy((vto), (vfrom), PAGE_SIZE) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 0594989ead63..4a496552b812 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -292,6 +292,17 @@ static bool riscv_cpufeature_patch_check(u16 id, u16 value) if (!value) return true; + switch (id) { + case RISCV_ISA_EXT_ZICBOZ: + /* + * Zicboz alternative applications provide the maximum + * supported block size order, or zero when it doesn't + * matter. If the current block size exceeds the maximum, + * then the alternative cannot be applied. + */ + return riscv_cboz_block_size <= (1U << value); + } + return false; } diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile index 6c74b0bedd60..26cb2502ecf8 100644 --- a/arch/riscv/lib/Makefile +++ b/arch/riscv/lib/Makefile @@ -8,5 +8,6 @@ lib-y += strlen.o lib-y += strncmp.o lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o +lib-$(CONFIG_RISCV_ISA_ZICBOZ) += clear_page.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o diff --git a/arch/riscv/lib/clear_page.S b/arch/riscv/lib/clear_page.S new file mode 100644 index 000000000000..7c7fa45b5ab5 --- /dev/null +++ b/arch/riscv/lib/clear_page.S @@ -0,0 +1,73 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2023 Ventana Micro Systems Inc. + */ + +#include +#include +#include +#include +#include +#include + +#define CBOZ_ALT(order, old, new) \ + ALTERNATIVE(old, new, 0, \ + ((order) << 16) | RISCV_ISA_EXT_ZICBOZ, \ + CONFIG_RISCV_ISA_ZICBOZ) + +/* void clear_page(void *page) */ +ENTRY(__clear_page) +WEAK(clear_page) + li a2, PAGE_SIZE + + /* + * If Zicboz isn't present, or somehow has a block + * size larger than 4K, then fallback to memset. + */ + CBOZ_ALT(12, "j .Lno_zicboz", "nop") + + lw a1, riscv_cboz_block_size + add a2, a0, a2 +.Lzero_loop: + CBO_zero(a0) + add a0, a0, a1 + CBOZ_ALT(11, "bltu a0, a2, .Lzero_loop; ret", "nop; nop") + CBO_zero(a0) + add a0, a0, a1 + CBOZ_ALT(10, "bltu a0, a2, .Lzero_loop; ret", "nop; nop") + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBOZ_ALT(9, "bltu a0, a2, .Lzero_loop; ret", "nop; nop") + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBOZ_ALT(8, "bltu a0, a2, .Lzero_loop; ret", "nop; nop") + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + CBO_zero(a0) + add a0, a0, a1 + bltu a0, a2, .Lzero_loop + ret +.Lno_zicboz: + li a1, 0 + tail __memset +END(__clear_page)