From patchwork Tue May 8 15:14:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 135184 Delivered-To: patch@linaro.org Received: by 10.46.151.6 with SMTP id r6csp4522824lji; Tue, 8 May 2018 08:25:59 -0700 (PDT) X-Google-Smtp-Source: AB8JxZpZsALAHOwRcswWov/dYMMDSep+K6vBq+4hZAIB2zAUDhSIl4IB1eXr1tO8g1lZIhBBAfaf X-Received: by 2002:a0c:965a:: with SMTP id 26-v6mr22418040qvy.133.1525793159681; Tue, 08 May 2018 08:25:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525793159; cv=none; d=google.com; s=arc-20160816; b=R42cEzDeYS9rRo1EdrYNsinQjLolHpH/QpCs3X18iDc77EmdKXCMAuCsQULY8h8yb6 ZhbMow3PRgecUb+PXM/5QT1QY1UJGqQQvZV5kYs17GKucNdhMAQ4PwvfPkvI/wXkGhnz 8BtGHmLWA3tGZkP5m1JAQftz6M4uWaXT348SLmwfiE3F83+u+Jk1CYhhsBswaarMA5X1 CpS7mt/FtHguWC85Xgn/q2GNhFQAivDe5+E0S3PccZB6oxi3SjjQm2q81TO7RRS48/89 XUzLrdJ5xyXoz4nQukXpLUEQM36fPHwaeV9xr2cEnAR0yVBHWIllU7CW4fWuMSo9m3wc Q5dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=7A9XEgWChxB+rPK5/XKmD/6XiegwwZSHREIlzYhR5iU=; b=w+uT/I1wnec37DUGr/oHZpUrqLymU+J37Lb/7Hc9NIy6v+o3/e6yOc6hn80VRufU74 DfxA93SyTlh+1boFOkxbN8bsn3J0KfUB8zDB396dlaO4QBUkl1SS9YqTgKBV8R14wXVw IONV2ZgZBOBaEYKtBawlbeYyxjVZtzRvdGS+goQSaDtclJ3y7LCJTLh1xoHieKL5YoMm MroUNaszK8HAgz/YdFCr3ytwnYRlA7nFVwFNMkUVum90City/bhEe1Vufmio4LGtM0+G P7nX6ys1FrF5VLyjeVTrS1Yjg5F/aiPM2fIYaI9+y9lfwnSFkZgHWYQ74eMGUStkQS+Y 2ylA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WWtIr9PV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id f8-v6si1302915qvo.2.2018.05.08.08.25.59 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 08 May 2018 08:25:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WWtIr9PV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51873 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fG4V5-0007pw-2o for patch@linaro.org; Tue, 08 May 2018 11:25:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59625) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fG4KN-0006iE-En for qemu-devel@nongnu.org; Tue, 08 May 2018 11:14:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fG4KL-0008Hb-L1 for qemu-devel@nongnu.org; Tue, 08 May 2018 11:14:55 -0400 Received: from mail-pl0-x243.google.com ([2607:f8b0:400e:c01::243]:41706) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fG4KL-0008Gk-Ct for qemu-devel@nongnu.org; Tue, 08 May 2018 11:14:53 -0400 Received: by mail-pl0-x243.google.com with SMTP id az12-v6so2410949plb.8 for ; Tue, 08 May 2018 08:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7A9XEgWChxB+rPK5/XKmD/6XiegwwZSHREIlzYhR5iU=; b=WWtIr9PVwQaURAfy9c8BGaqmQDYjUd9che3/25B1CzizNlZ6d0tV4fKVHg3QGQCPkz p6a2rQTsmZzaeKq0H830xqf+NYtpwKg7+L3c6ddXtvosnYkXPD6VvXftKz2DWAL6Ovu5 bponW0ml9uI4fF18hQbg2dwIAUaLAwQ1Ixx8s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7A9XEgWChxB+rPK5/XKmD/6XiegwwZSHREIlzYhR5iU=; b=eh2Ax5bG8qcsu84NAWR614UNSOaRXAHoS4qBDO55PwW+FN8ETBvtzyUjBgosCNXLbe zcORcLyEQVfsHyet0afzdTV4Fmyu3Ymp0O7O2LanG4Yqg4jSogcQDDEfnGC91zxedIxg McUUt2F7oIsBGEeoojSgMVwU2KPpuRESgkScAQl1JPtm2zr6HECnSm5pc5kEBFSnbrBU 7p5Es0+23lkodbU+2tqXHNKH6elJOw3DikTGUkM8hD8BjTRPjhtGP0ItZdycPg87OSTS 0NgPm/aq2FBAzeAMaTnDs/996a+TfYFxnIqUMzHj3vsHb82C6aT+XkjbzJpfgJZ9lW8h oaaw== X-Gm-Message-State: ALQs6tB8dVgneNN6xhJNQY2691E2xOcknnjGm3Vf57XCYkeB+OCNUMJk g19SQCGNOedsbot+kaIa1n0Sk2RjDtI= X-Received: by 2002:a17:902:e8:: with SMTP id a95-v6mr41951106pla.274.1525792492108; Tue, 08 May 2018 08:14:52 -0700 (PDT) Received: from cloudburst.twiddle.net (97-113-2-170.tukw.qwest.net. [97.113.2.170]) by smtp.gmail.com with ESMTPSA id n10sm55598896pfj.68.2018.05.08.08.14.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 08 May 2018 08:14:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Tue, 8 May 2018 08:14:36 -0700 Message-Id: <20180508151437.4232-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180508151437.4232-1-richard.henderson@linaro.org> References: <20180508151437.4232-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::243 Subject: [Qemu-devel] [PATCH v3 09/10] target/arm: Implement CAS and CASP X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/helper-a64.h | 2 + target/arm/helper-a64.c | 43 ++++++++++++++ target/arm/translate-a64.c | 119 ++++++++++++++++++++++++++++++++++++- 3 files changed, 161 insertions(+), 3 deletions(-) -- 2.17.0 diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index ef4ddfe9d8..b8028ac98c 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -51,6 +51,8 @@ DEF_HELPER_FLAGS_4(paired_cmpxchg64_le_parallel, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_4(paired_cmpxchg64_be, TCG_CALL_NO_WG, i64, env, i64, i64, i64) DEF_HELPER_FLAGS_4(paired_cmpxchg64_be_parallel, TCG_CALL_NO_WG, i64, env, i64, i64, i64) +DEF_HELPER_5(casp_le_parallel, void, env, i32, i64, i64, i64) +DEF_HELPER_5(casp_be_parallel, void, env, i32, i64, i64, i64) DEF_HELPER_FLAGS_3(advsimd_maxh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(advsimd_minh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) DEF_HELPER_FLAGS_3(advsimd_maxnumh, TCG_CALL_NO_RWG, f16, f16, f16, ptr) diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index afb25ad20c..549ed3513e 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -636,6 +636,49 @@ uint64_t HELPER(paired_cmpxchg64_be_parallel)(CPUARMState *env, uint64_t addr, return do_paired_cmpxchg64_be(env, addr, new_lo, new_hi, true, GETPC()); } +/* Writes back the old data into Rs. */ +void HELPER(casp_le_parallel)(CPUARMState *env, uint32_t rs, uint64_t addr, + uint64_t new_lo, uint64_t new_hi) +{ + uintptr_t ra = GETPC(); +#ifndef CONFIG_ATOMIC128 + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); +#else + Int128 oldv, cmpv, newv; + + cmpv = int128_make128(env->xregs[rs], env->xregs[rs + 1]); + newv = int128_make128(new_lo, new_hi); + + int mem_idx = cpu_mmu_index(env, false); + TCGMemOpIdx oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); + oldv = helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, oi, ra); + + env->xregs[rs] = int128_getlo(oldv); + env->xregs[rs + 1] = int128_gethi(oldv); +#endif +} + +void HELPER(casp_be_parallel)(CPUARMState *env, uint32_t rs, uint64_t addr, + uint64_t new_hi, uint64_t new_lo) +{ + uintptr_t ra = GETPC(); +#ifndef CONFIG_ATOMIC128 + cpu_loop_exit_atomic(ENV_GET_CPU(env), ra); +#else + Int128 oldv, cmpv, newv; + + cmpv = int128_make128(env->xregs[rs + 1], env->xregs[rs]); + newv = int128_make128(new_lo, new_hi); + + int mem_idx = cpu_mmu_index(env, false); + TCGMemOpIdx oi = make_memop_idx(MO_LEQ | MO_ALIGN_16, mem_idx); + oldv = helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, oi, ra); + + env->xregs[rs + 1] = int128_getlo(oldv); + env->xregs[rs] = int128_gethi(oldv); +#endif +} + /* * AdvSIMD half-precision */ diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index c13858a690..e6cd88303f 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -2114,6 +2114,103 @@ static void gen_store_exclusive(DisasContext *s, int rd, int rt, int rt2, tcg_gen_movi_i64(cpu_exclusive_addr, -1); } +static void gen_compare_and_swap(DisasContext *s, int rs, int rt, + int rn, int size) +{ + TCGv_i64 tcg_rs = cpu_reg(s, rs); + TCGv_i64 tcg_rt = cpu_reg(s, rt); + int memidx = get_mem_index(s); + TCGv_i64 addr = cpu_reg_sp(s, rn); + + if (rn == 31) { + gen_check_sp_alignment(s); + } + tcg_gen_atomic_cmpxchg_i64(tcg_rs, addr, tcg_rs, tcg_rt, memidx, + size | MO_ALIGN | s->be_data); +} + +static void gen_compare_and_swap_pair(DisasContext *s, int rs, int rt, + int rn, int size) +{ + TCGv_i64 s1 = cpu_reg(s, rs); + TCGv_i64 s2 = cpu_reg(s, rs + 1); + TCGv_i64 t1 = cpu_reg(s, rt); + TCGv_i64 t2 = cpu_reg(s, rt + 1); + TCGv_i64 addr = cpu_reg_sp(s, rn); + int memidx = get_mem_index(s); + + if (rn == 31) { + gen_check_sp_alignment(s); + } + + if (size == 2) { + TCGv_i64 cmp = tcg_temp_new_i64(); + TCGv_i64 val = tcg_temp_new_i64(); + + if (s->be_data == MO_LE) { + tcg_gen_concat32_i64(val, t1, t2); + tcg_gen_concat32_i64(cmp, s1, s2); + } else { + tcg_gen_concat32_i64(val, t2, t1); + tcg_gen_concat32_i64(cmp, s2, s1); + } + + tcg_gen_atomic_cmpxchg_i64(cmp, addr, cmp, val, memidx, + MO_64 | MO_ALIGN | s->be_data); + tcg_temp_free_i64(val); + + if (s->be_data == MO_LE) { + tcg_gen_extr32_i64(s1, s2, cmp); + } else { + tcg_gen_extr32_i64(s2, s1, cmp); + } + tcg_temp_free_i64(cmp); + } else if (tb_cflags(s->base.tb) & CF_PARALLEL) { + TCGv_i32 tcg_rs = tcg_const_i32(rs); + + if (s->be_data == MO_LE) { + gen_helper_casp_le_parallel(cpu_env, tcg_rs, addr, t1, t2); + } else { + gen_helper_casp_be_parallel(cpu_env, tcg_rs, addr, t1, t2); + } + tcg_temp_free_i32(tcg_rs); + } else { + TCGv_i64 d1 = tcg_temp_new_i64(); + TCGv_i64 d2 = tcg_temp_new_i64(); + TCGv_i64 a2 = tcg_temp_new_i64(); + TCGv_i64 c1 = tcg_temp_new_i64(); + TCGv_i64 c2 = tcg_temp_new_i64(); + TCGv_i64 zero = tcg_const_i64(0); + + /* Load the two words, in memory order. */ + tcg_gen_qemu_ld_i64(d1, addr, memidx, + MO_64 | MO_ALIGN_16 | s->be_data); + tcg_gen_addi_i64(a2, addr, 8); + tcg_gen_qemu_ld_i64(d2, addr, memidx, MO_64 | s->be_data); + + /* Compare the two words, also in memory order. */ + tcg_gen_setcond_i64(TCG_COND_EQ, c1, d1, s1); + tcg_gen_setcond_i64(TCG_COND_EQ, c2, d2, s2); + tcg_gen_and_i64(c2, c2, c1); + + /* If compare equal, write back new data, else write back old data. */ + tcg_gen_movcond_i64(TCG_COND_NE, c1, c2, zero, t1, d1); + tcg_gen_movcond_i64(TCG_COND_NE, c2, c2, zero, t2, d2); + tcg_gen_qemu_st_i64(c1, addr, memidx, MO_64 | s->be_data); + tcg_gen_qemu_st_i64(c2, a2, memidx, MO_64 | s->be_data); + tcg_temp_free_i64(a2); + tcg_temp_free_i64(c1); + tcg_temp_free_i64(c2); + tcg_temp_free_i64(zero); + + /* Write back the data from memory to Rs. */ + tcg_gen_mov_i64(s1, d1); + tcg_gen_mov_i64(s2, d2); + tcg_temp_free_i64(d1); + tcg_temp_free_i64(d2); + } +} + /* Update the Sixty-Four bit (SF) registersize. This logic is derived * from the ARMv8 specs for LDR (Shared decode for all encodings). */ @@ -2214,10 +2311,16 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) gen_store_exclusive(s, rs, rt, rt2, tcg_addr, size, true); return; } - /* CASP / CASPL */ + if (rt2 == 31 + && ((rt | rs) & 1) == 0 + && arm_dc_feature(s, ARM_FEATURE_V8_ATOMICS)) { + /* CASP / CASPL */ + gen_compare_and_swap_pair(s, rs, rt, rn, size | 2); + return; + } break; - case 0x6: case 0x7: /* CASP / LDXP */ + case 0x6: case 0x7: /* CASPA / LDXP */ if (size & 2) { /* LDXP / LDAXP */ if (rn == 31) { gen_check_sp_alignment(s); @@ -2230,13 +2333,23 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) } return; } - /* CASPA / CASPAL */ + if (rt2 == 31 + && ((rt | rs) & 1) == 0 + && arm_dc_feature(s, ARM_FEATURE_V8_ATOMICS)) { + /* CASPA / CASPAL */ + gen_compare_and_swap_pair(s, rs, rt, rn, size | 2); + return; + } break; case 0xa: /* CAS */ case 0xb: /* CASL */ case 0xe: /* CASA */ case 0xf: /* CASAL */ + if (rt2 == 31 && arm_dc_feature(s, ARM_FEATURE_V8_ATOMICS)) { + gen_compare_and_swap(s, rs, rt, rn, size); + return; + } break; } unallocated_encoding(s);