From patchwork Wed Apr 12 11:43:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 672623 Delivered-To: patch@linaro.org Received: by 2002:a05:6000:184:0:0:0:0 with SMTP id p4csp288487wrx; Wed, 12 Apr 2023 04:47:32 -0700 (PDT) X-Google-Smtp-Source: AKy350aS2cCrT8e0HOScPYJ23Thcy6PDwm34xrxXNvTMos1cX87kOQAHceDojbW/TMKahP864vmI X-Received: by 2002:a05:622a:14e:b0:3e3:9275:17ad with SMTP id v14-20020a05622a014e00b003e3927517admr3580200qtw.12.1681300052026; Wed, 12 Apr 2023 04:47:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681300052; cv=none; d=google.com; s=arc-20160816; b=IXyFxcZMObfgq/s56ohc4F/FmLfCp7h/fgLR/afZAlaUzVkJf0IwdW5xuhO6wGxR34 FsLSf8SunPDeB1usZIwLurwwcDo0AmM/vCmTzYd8BsN7w5U/T4kaFTjCPiA7U/r50EBN +IPqUMqSAg39540B6HK6X7QAc4aLEIOT5kd93mcC/4MLxHadkkLBsFE1btgqMC99sh87 HN6gxPBAIZ0LxrIX6U0quarXB4LZtlBvBq7L6newveqPUaNaknURfjhLCTc299DC+mhc +jU+BrC3xwcqhtUBFHCtAqlI1J6yfC0csowCBfm6TVhkqTiXPDBDwlLQevJGcK60BuiL l0+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=vFpmN4l+aiQZzHkMQF0Is1Gbr8TTg8AJfI2hz+TEgNU=; b=Kjqfdcc0caX2gciAIcjUnnu/AORr8L9qztDSphB+GFq1UUYTFRXWcA8DsL0pony+Z/ 6yZFZ3ZQ9MNqkGCin9X13tefBXaku7oo02gI46oGDTxifFqiLDDOpWtHK5hz+fpc9T99 xPJE6sekUujS2B+7FePyY0vDLJpNO2nNtGh4n/JAEhFOC9ft7yiWrRA/gYwAtDIIaCKC WmqdUEYHGWR1TINVEiIaPJ2f4RjMtxEd/9G/5nNPJUpUtxs1FmEk5sc5HAyDxGEwlnS8 ZcjtnFLDj5Pav8k8t9FvoFBLkqcPulV/IGXiPNlHPUgN+d+cT5sFA/hCPBfYYH6gorYx qWFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nubkifGA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id bl16-20020a05620a1a9000b0074a88515827si8139103qkb.755.2023.04.12.04.47.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 12 Apr 2023 04:47:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nubkifGA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pmYtd-0005m8-Vs; Wed, 12 Apr 2023 07:44:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pmYt8-0004l5-DZ for qemu-devel@nongnu.org; Wed, 12 Apr 2023 07:43:47 -0400 Received: from mail-wr1-x432.google.com ([2a00:1450:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pmYt4-0002NE-1m for qemu-devel@nongnu.org; Wed, 12 Apr 2023 07:43:45 -0400 Received: by mail-wr1-x432.google.com with SMTP id l18so10512845wrb.9 for ; Wed, 12 Apr 2023 04:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1681299821; x=1683891821; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vFpmN4l+aiQZzHkMQF0Is1Gbr8TTg8AJfI2hz+TEgNU=; b=nubkifGATpQrBCVy4n7E0dP6pAc1PkiFqkKd3VkYpkLWWgA+z6/AHXD6SsbJB9QUxd 0Od0dfFs161Qb7JD776X970PESx5q24GgMNOZTlVNSSGnoQPm334rfc1Hy/dHrmsn09V flmFoJ6Ztu4yqFM0wwe2JXdvgWLEEG92aj+3anO/UjpqIO3Q0K8POx/UZM0X9/kIrEc4 f6Od1BxtJgiCC0yKN8PA/td1VNllIFEOve8KI39zWjkdHHg25XOYj6VTub8559IXCIB1 uKnG3YaK3AsAq0sGOTQrwDCY2NLkE3kQl4Zgn1lQQ5bQoW8Klx8pSq26bRKd5OC1Xm+A 5rGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681299821; x=1683891821; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vFpmN4l+aiQZzHkMQF0Is1Gbr8TTg8AJfI2hz+TEgNU=; b=tu78DYeATl0JJIwHDVMt9ayDDie0zz/0oIPTrKT+Zs5ve/2aoO3lWAHu0YaV/KxScv eOuM7hG/9cEbuL0DY1fbA6uaR67DOZ79QmjAxcoRoNkN2zxVWLM0i1aKNKRzEJ6wPdcG GhR7Y+Te5/JoLZ5h5dzk4ce9QNQ29qdk2wzEOIsxOvdd1q2cvW6XEI0TrCkTCSN37FnI cpzfgpBrL6iKeAM8RaElmpk8lOTCMK5fxOXdnz3Vixd2EHspdUCJmVXpQzeykLwz7xSh w1hl1hl//l+/BD/uiKO6etAtNnS2He1LShD9VL+i4ETeGxmxbKdyRrWbWph2d0zzyYuc YeKg== X-Gm-Message-State: AAQBX9c0rDSmTPhyjLPUshKuKIRZs4Y8uaSEmu+bczwo0UAF5/45qekG s733nFpvYaql+brcMuK+OIuyRVtydxBZTeMnu85g2JRc X-Received: by 2002:adf:dfcc:0:b0:2ef:af46:1278 with SMTP id q12-20020adfdfcc000000b002efaf461278mr11347284wrn.10.1681299821156; Wed, 12 Apr 2023 04:43:41 -0700 (PDT) Received: from stoup.hotel.gast (ip-037-024-010-236.um08.pools.vodafone-ip.de. [37.24.10.236]) by smtp.gmail.com with ESMTPSA id k9-20020a5d4289000000b002d21379bcabsm16980826wrq.110.2023.04.12.04.43.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Apr 2023 04:43:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alistair23@gmail.com, qemu-riscv@nongnu.org, Fei Wu , LIU Zhiwei , Alistair Francis , Weiwei Li , Daniel Henrique Barboza Subject: [PATCH v7 07/25] target/riscv: Reduce overhead of MSTATUS_SUM change Date: Wed, 12 Apr 2023 13:43:15 +0200 Message-Id: <20230412114333.118895-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230412114333.118895-1-richard.henderson@linaro.org> References: <20230412114333.118895-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-wr1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org From: Fei Wu Kernel needs to access user mode memory e.g. during syscalls, the window is usually opened up for a very limited time through MSTATUS.SUM, the overhead is too much if tlb_flush() gets called for every SUM change. This patch creates a separate MMU index for S+SUM, so that it's not necessary to flush tlb anymore when SUM changes. This is similar to how ARM handles Privileged Access Never (PAN). Result of 'pipe 10' from unixbench boosts from 223656 to 1705006. Many other syscalls benefit a lot from this too. Reviewed-by: Richard Henderson Signed-off-by: Fei Wu Message-Id: <20230324054154.414846-3-fei2.wu@intel.com> Reviewed-by: LIU Zhiwei Reviewed-by: Alistair Francis Reviewed-by: Weiwei Li Tested-by: Daniel Henrique Barboza Message-Id: <20230325105429.1142530-8-richard.henderson@linaro.org> --- target/riscv/cpu.h | 2 -- target/riscv/internals.h | 14 ++++++++++++++ target/riscv/cpu_helper.c | 17 +++++++++++++++-- target/riscv/csr.c | 3 +-- target/riscv/op_helper.c | 5 +++-- target/riscv/insn_trans/trans_rvh.c.inc | 4 ++-- 6 files changed, 35 insertions(+), 10 deletions(-) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 9b971ee1b0..6239c99f4c 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -629,8 +629,6 @@ G_NORETURN void riscv_raise_exception(CPURISCVState *env, target_ulong riscv_cpu_get_fflags(CPURISCVState *env); void riscv_cpu_set_fflags(CPURISCVState *env, target_ulong); -#define TB_FLAGS_PRIV_HYP_ACCESS_MASK (1 << 2) - #include "exec/cpu-all.h" FIELD(TB_FLAGS, MEM_IDX, 0, 3) diff --git a/target/riscv/internals.h b/target/riscv/internals.h index 5620fbffb6..b55152a7dc 100644 --- a/target/riscv/internals.h +++ b/target/riscv/internals.h @@ -21,6 +21,20 @@ #include "hw/registerfields.h" +/* + * The current MMU Modes are: + * - U 0b000 + * - S 0b001 + * - S+SUM 0b010 + * - M 0b011 + * - HLV/HLVX/HSV adds 0b100 + */ +#define MMUIdx_U 0 +#define MMUIdx_S 1 +#define MMUIdx_S_SUM 2 +#define MMUIdx_M 3 +#define MMU_HYP_ACCESS_BIT (1 << 2) + /* share data between vector helpers and decode code */ FIELD(VDATA, VM, 0, 1) FIELD(VDATA, LMUL, 1, 3) diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c index 36d6e422d7..174a77706b 100644 --- a/target/riscv/cpu_helper.c +++ b/target/riscv/cpu_helper.c @@ -21,6 +21,7 @@ #include "qemu/log.h" #include "qemu/main-loop.h" #include "cpu.h" +#include "internals.h" #include "pmu.h" #include "exec/exec-all.h" #include "instmap.h" @@ -36,7 +37,19 @@ int riscv_cpu_mmu_index(CPURISCVState *env, bool ifetch) #ifdef CONFIG_USER_ONLY return 0; #else - return env->priv; + if (ifetch) { + return env->priv; + } + + /* All priv -> mmu_idx mapping are here */ + int mode = env->priv; + if (mode == PRV_M && get_field(env->mstatus, MSTATUS_MPRV)) { + mode = get_field(env->mstatus, MSTATUS_MPP); + } + if (mode == PRV_S && get_field(env->mstatus, MSTATUS_SUM)) { + return MMUIdx_S_SUM; + } + return mode; #endif } @@ -588,7 +601,7 @@ void riscv_cpu_set_virt_enabled(CPURISCVState *env, bool enable) bool riscv_cpu_two_stage_lookup(int mmu_idx) { - return mmu_idx & TB_FLAGS_PRIV_HYP_ACCESS_MASK; + return mmu_idx & MMU_HYP_ACCESS_BIT; } int riscv_cpu_claim_interrupts(RISCVCPU *cpu, uint64_t interrupts) diff --git a/target/riscv/csr.c b/target/riscv/csr.c index 4268828dc4..124be448dc 100644 --- a/target/riscv/csr.c +++ b/target/riscv/csr.c @@ -1270,8 +1270,7 @@ static RISCVException write_mstatus(CPURISCVState *env, int csrno, val = legalize_mpp(env, get_field(mstatus, MSTATUS_MPP), val); /* flush tlb on mstatus fields that affect VM */ - if ((val ^ mstatus) & (MSTATUS_MXR | MSTATUS_MPP | MSTATUS_MPV | - MSTATUS_MPRV | MSTATUS_SUM)) { + if ((val ^ mstatus) & (MSTATUS_MXR | MSTATUS_MPV)) { tlb_flush(env_cpu(env)); } mask = MSTATUS_SIE | MSTATUS_SPIE | MSTATUS_MIE | MSTATUS_MPIE | diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c index bd21c6eeef..00bdf77f32 100644 --- a/target/riscv/op_helper.c +++ b/target/riscv/op_helper.c @@ -20,6 +20,7 @@ #include "qemu/osdep.h" #include "cpu.h" +#include "internals.h" #include "qemu/main-loop.h" #include "exec/exec-all.h" #include "exec/helper-proto.h" @@ -428,14 +429,14 @@ void helper_hyp_gvma_tlb_flush(CPURISCVState *env) target_ulong helper_hyp_hlvx_hu(CPURISCVState *env, target_ulong address) { - int mmu_idx = cpu_mmu_index(env, true) | TB_FLAGS_PRIV_HYP_ACCESS_MASK; + int mmu_idx = cpu_mmu_index(env, true) | MMU_HYP_ACCESS_BIT; return cpu_lduw_mmuidx_ra(env, address, mmu_idx, GETPC()); } target_ulong helper_hyp_hlvx_wu(CPURISCVState *env, target_ulong address) { - int mmu_idx = cpu_mmu_index(env, true) | TB_FLAGS_PRIV_HYP_ACCESS_MASK; + int mmu_idx = cpu_mmu_index(env, true) | MMU_HYP_ACCESS_BIT; return cpu_ldl_mmuidx_ra(env, address, mmu_idx, GETPC()); } diff --git a/target/riscv/insn_trans/trans_rvh.c.inc b/target/riscv/insn_trans/trans_rvh.c.inc index 4b730cd492..ae98b45e5e 100644 --- a/target/riscv/insn_trans/trans_rvh.c.inc +++ b/target/riscv/insn_trans/trans_rvh.c.inc @@ -42,7 +42,7 @@ static bool do_hlv(DisasContext *ctx, arg_r2 *a, MemOp mop) if (check_access(ctx)) { TCGv dest = dest_gpr(ctx, a->rd); TCGv addr = get_gpr(ctx, a->rs1, EXT_NONE); - int mem_idx = ctx->mem_idx | TB_FLAGS_PRIV_HYP_ACCESS_MASK; + int mem_idx = ctx->mem_idx | MMU_HYP_ACCESS_BIT; tcg_gen_qemu_ld_tl(dest, addr, mem_idx, mop); gen_set_gpr(ctx, a->rd, dest); } @@ -89,7 +89,7 @@ static bool do_hsv(DisasContext *ctx, arg_r2_s *a, MemOp mop) if (check_access(ctx)) { TCGv addr = get_gpr(ctx, a->rs1, EXT_NONE); TCGv data = get_gpr(ctx, a->rs2, EXT_NONE); - int mem_idx = ctx->mem_idx | TB_FLAGS_PRIV_HYP_ACCESS_MASK; + int mem_idx = ctx->mem_idx | MMU_HYP_ACCESS_BIT; tcg_gen_qemu_st_tl(data, addr, mem_idx, mop); } return true;