From patchwork Mon Jun 19 14:23:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 694018 Delivered-To: patch@linaro.org Received: by 2002:a5d:4d91:0:0:0:0:0 with SMTP id b17csp2399230wru; Mon, 19 Jun 2023 07:25:20 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6QyQi0f1inI98LV/VoKDXK4RGSX4VhQY3A/qdgN6g7cPnipij34YCKqkdWKHZb9iVH8Q4g X-Received: by 2002:a05:6214:411b:b0:630:18d6:18b with SMTP id kc27-20020a056214411b00b0063018d6018bmr3814271qvb.19.1687184720584; Mon, 19 Jun 2023 07:25:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687184720; cv=none; d=google.com; s=arc-20160816; b=za9sN9OnwPASFtX0LrmssQppAmUT74CqsYD0Z2n9IIAdK1dsylegDJc5u9F1/GCpcD Z9nIrR0yQ7cdBeW8etRoHsSlPE7eF7aF71ZmcQfR45FBS0NTYDTMynrGMBaRoZaLbvoy EACOM3D32TO5xeDdBa/HgSbuzFY5gLkRHzSlvdPmMA2L1YtgquqoklWSKYlal0uS7+I3 kWSbu0Esh5Q+DTynyg7w2WvX4tpYcFx6SiKvII0DvHW6cNTvwzH2Wcp2/7orMzBgupVw 138Ot4gNd7gGbUkYl4PhrgWnkr/YEsyUDaZzjS5nGMqQERYLYvosrcwE0w1v2cF7QfpP 5cgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wYaXVLSy7te17IjoR6ZeNkrXoDozUSIUDyj3cihp+Jg=; b=i13SaaNtQku1dSJk2dul8xzHMkI/QEwkiGUi3Xrwp55CW/1Qz3s1vWlP7wHah4t1x9 PyPMVLtJQM6S7NcD4CcZmk95ZUOhCWmRUia3IZmU9ZLZsFVtlX2wLaDs1vgkp0Wj860X nopARRx6tpuc9XUEgRWy9EsVyA4+cEVofXAWkdypH10VTO+Hf7hiC6mOJ9ZXommgCrQ9 P608JZqN5+O8Z20NFJHSS/egHj21KkTQn12en15CA0EtKXEkJ+Sutk6Yflxj4Gczi0T9 Ok14rsnk71tfs9T2Bz7JTzDId/uQ1U3+iJMOVrKpxiRHw5N9R+cVUfY01o7ei4VGeDEG SIgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GsMASvLt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id jo24-20020a056214501800b0062de32cc1a3si7203qvb.302.2023.06.19.07.25.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 19 Jun 2023 07:25:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GsMASvLt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qBFnK-0004FN-BH; Mon, 19 Jun 2023 10:23:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qBFnH-0004ED-I5 for qemu-devel@nongnu.org; Mon, 19 Jun 2023 10:23:47 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qBFnF-0000lV-78 for qemu-devel@nongnu.org; Mon, 19 Jun 2023 10:23:47 -0400 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-3f9b4a71623so3163855e9.1 for ; Mon, 19 Jun 2023 07:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1687184623; x=1689776623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=wYaXVLSy7te17IjoR6ZeNkrXoDozUSIUDyj3cihp+Jg=; b=GsMASvLtD8hdEgx+CsONDgKpyFnqmLl7DO3hlnrlkAPsJvr+yz8cwY3pQEzpF5xUg6 KkAMdgHaeA6+9M8d94G2SSDBZMsQVyXk5JMQrA0cGFJYMO999wCQuVxQj0XBrzybaXxq XvyymparhakZzXWSYPcjM0yYCAfsy+lOiL7NCkJ149tdIWx6+5krMAJuiBlVjHAQHLkm hhDya6PFg2DcKo5xZ2EX39tr/brnwB5jtt0mKfacqV6Ijg/U85jTW3/9FT1G6IESmmIo Dfvqz8xOPYDSFoZcvwggxl/luMOPTVKZzID/cqKZd9WLxdUiKxZjj7V9x202RAHrQ9EK d6ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687184623; x=1689776623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wYaXVLSy7te17IjoR6ZeNkrXoDozUSIUDyj3cihp+Jg=; b=DomWEa1batAHo7yy458omcnOfpabpiQTK2nKeAcYyDL3hiocSZOmOMjGwYfUoWBp/M CCG/WS5Y+sSXHJbFOaxzBtJrE/jv8pSQvntv3uNifZW4tXp813CstTPfUdbmq4abAXy1 LWoGCHeEEp+Pt6ZKexi0N7dk94kNjwYG9+wvkCxdCpI2Q7PkmiYBT5jgjk+NR/HXNbAO NTy7tAUSVT805/zWwKvwA5WGXn+5J3JA7xVEoQBbab5JFTlyoli+ndy2fN4d4cGj2DHN 6rr95gMiyK4QnhknGF1Ec8yd5TTeLfT+ISI9YeD3Rpy7IPv+3v9rdvPFbboAAdt5jzPm IuGw== X-Gm-Message-State: AC+VfDzpWf63MWFPfl5vrUwlRb+NZkUe71s14LzfJ7S1iu+Xi6KhWpBY MriEDPZ17jqYozaO4A8APiMkwxxV1zwBQMhZ3FxQ4uhD X-Received: by 2002:a05:600c:203:b0:3f7:f24b:b2ed with SMTP id 3-20020a05600c020300b003f7f24bb2edmr6990245wmi.19.1687184622904; Mon, 19 Jun 2023 07:23:42 -0700 (PDT) Received: from stoup.lan (sar95-h02-176-184-10-225.dsl.sta.abo.bbox.fr. [176.184.10.225]) by smtp.gmail.com with ESMTPSA id k25-20020a7bc419000000b003f96d10eafbsm4143467wmi.12.2023.06.19.07.23.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Jun 2023 07:23:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v3 4/5] tcg: Add host memory barriers to cpu_ldst.h interfaces Date: Mon, 19 Jun 2023 16:23:32 +0200 Message-Id: <20230619142333.429028-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230619142333.429028-1-richard.henderson@linaro.org> References: <20230619142333.429028-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=richard.henderson@linaro.org; helo=mail-wm1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Bring the helpers into line with the rest of tcg in respecting guest memory ordering. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/internal.h | 34 ++++++++++++++++++++++++++++++++++ accel/tcg/cputlb.c | 10 ++++++++++ accel/tcg/user-exec.c | 10 ++++++++++ 3 files changed, 54 insertions(+) diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h index 24f225cac7..be0c7753fb 100644 --- a/accel/tcg/internal.h +++ b/accel/tcg/internal.h @@ -78,4 +78,38 @@ extern int64_t max_advance; extern bool one_insn_per_tb; +/** + * tcg_req_mo: + * @type: TCGBar + * + * Filter @type to the barrier that is required for the guest + * memory ordering vs the host memory ordering. A non-zero + * result indicates that some barrier is required. + * + * If TCG_GUEST_DEFAULT_MO is not defined, assume that the + * guest requires strict ordering. + * + * This is a macro so that it's constant even without optimization. + */ +#ifdef TCG_GUEST_DEFAULT_MO +# define tcg_req_mo(type) \ + ((type) & TCG_GUEST_DEFAULT_MO & ~TCG_TARGET_DEFAULT_MO) +#else +# define tcg_req_mo(type) ((type) & ~TCG_TARGET_DEFAULT_MO) +#endif + +/** + * cpu_req_mo: + * @type: TCGBar + * + * If tcg_req_mo indicates a barrier for @type is required + * for the guest memory model, issue a host memory barrier. + */ +#define cpu_req_mo(type) \ + do { \ + if (tcg_req_mo(type)) { \ + smp_mb(); \ + } \ + } while (0) + #endif /* ACCEL_TCG_INTERNAL_H */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 5e2ca47243..a48e1c9693 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2342,6 +2342,7 @@ static uint8_t do_ld1_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, MMULookupLocals l; bool crosspage; + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(env, addr, oi, ra, access_type, &l); tcg_debug_assert(!crosspage); @@ -2363,6 +2364,7 @@ static uint16_t do_ld2_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, uint16_t ret; uint8_t a, b; + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(env, addr, oi, ra, access_type, &l); if (likely(!crosspage)) { return do_ld_2(env, &l.page[0], l.mmu_idx, access_type, l.memop, ra); @@ -2393,6 +2395,7 @@ static uint32_t do_ld4_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, bool crosspage; uint32_t ret; + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(env, addr, oi, ra, access_type, &l); if (likely(!crosspage)) { return do_ld_4(env, &l.page[0], l.mmu_idx, access_type, l.memop, ra); @@ -2420,6 +2423,7 @@ static uint64_t do_ld8_mmu(CPUArchState *env, target_ulong addr, MemOpIdx oi, bool crosspage; uint64_t ret; + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(env, addr, oi, ra, access_type, &l); if (likely(!crosspage)) { return do_ld_8(env, &l.page[0], l.mmu_idx, access_type, l.memop, ra); @@ -2472,6 +2476,7 @@ static Int128 do_ld16_mmu(CPUArchState *env, target_ulong addr, Int128 ret; int first; + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD, &l); if (likely(!crosspage)) { /* Perform the load host endian. */ @@ -2804,6 +2809,7 @@ void helper_stb_mmu(CPUArchState *env, uint64_t addr, uint32_t val, bool crosspage; tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_8); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); tcg_debug_assert(!crosspage); @@ -2817,6 +2823,7 @@ static void do_st2_mmu(CPUArchState *env, target_ulong addr, uint16_t val, bool crosspage; uint8_t a, b; + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); if (likely(!crosspage)) { do_st_2(env, &l.page[0], val, l.mmu_idx, l.memop, ra); @@ -2845,6 +2852,7 @@ static void do_st4_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MMULookupLocals l; bool crosspage; + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); if (likely(!crosspage)) { do_st_4(env, &l.page[0], val, l.mmu_idx, l.memop, ra); @@ -2872,6 +2880,7 @@ static void do_st8_mmu(CPUArchState *env, target_ulong addr, uint64_t val, MMULookupLocals l; bool crosspage; + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); if (likely(!crosspage)) { do_st_8(env, &l.page[0], val, l.mmu_idx, l.memop, ra); @@ -2901,6 +2910,7 @@ static void do_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, uint64_t a, b; int first; + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE, &l); if (likely(!crosspage)) { /* Swap to host endian if necessary, then store. */ diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index dc8d6b5d40..ce65021cd4 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -914,6 +914,7 @@ static uint8_t do_ld1_mmu(CPUArchState *env, abi_ptr addr, uint8_t ret; tcg_debug_assert((mop & MO_SIZE) == MO_8); + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = ldub_p(haddr); clear_helper_retaddr(); @@ -947,6 +948,7 @@ static uint16_t do_ld2_mmu(CPUArchState *env, abi_ptr addr, uint16_t ret; tcg_debug_assert((mop & MO_SIZE) == MO_16); + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_2(env, ra, haddr, mop); clear_helper_retaddr(); @@ -984,6 +986,7 @@ static uint32_t do_ld4_mmu(CPUArchState *env, abi_ptr addr, uint32_t ret; tcg_debug_assert((mop & MO_SIZE) == MO_32); + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_4(env, ra, haddr, mop); clear_helper_retaddr(); @@ -1021,6 +1024,7 @@ static uint64_t do_ld8_mmu(CPUArchState *env, abi_ptr addr, uint64_t ret; tcg_debug_assert((mop & MO_SIZE) == MO_64); + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_8(env, ra, haddr, mop); clear_helper_retaddr(); @@ -1052,6 +1056,7 @@ static Int128 do_ld16_mmu(CPUArchState *env, abi_ptr addr, Int128 ret; tcg_debug_assert((mop & MO_SIZE) == MO_128); + cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_16(env, ra, haddr, mop); clear_helper_retaddr(); @@ -1087,6 +1092,7 @@ static void do_st1_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_8); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); stb_p(haddr, val); clear_helper_retaddr(); @@ -1111,6 +1117,7 @@ static void do_st2_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_16); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); if (mop & MO_BSWAP) { @@ -1139,6 +1146,7 @@ static void do_st4_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_32); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); if (mop & MO_BSWAP) { @@ -1167,6 +1175,7 @@ static void do_st8_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_64); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); if (mop & MO_BSWAP) { @@ -1195,6 +1204,7 @@ static void do_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val, void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_128); + cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); if (mop & MO_BSWAP) {