From patchwork Tue Jun 26 16:19:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 140025 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp5451626lji; Tue, 26 Jun 2018 09:29:20 -0700 (PDT) X-Google-Smtp-Source: AAOMgpcJJyBzcotRUHGQcxPTtHCb3M/H5CNNTG2fJQT+BPfgLDeTnmKjUXVoiyyjRJfP5lQeJQwK X-Received: by 2002:ac8:2297:: with SMTP id f23-v6mr2128530qta.373.1530030560137; Tue, 26 Jun 2018 09:29:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530030560; cv=none; d=google.com; s=arc-20160816; b=wsPYLW5dgKX3H3aq6W8uO6oYKxj6JB699fHDeIGtGv8t54OtaxephLp899xlklUuhu /NQ/x2rZqsFzadHRC0ljmvyzSCDSP0c257irzsmQP307cib+c4d2FcET7KVDm0BY9w9a x6xEwaEsOqBswp/p3Qki1vUjALXXWEIDwV9JRkLBzZFOT3vwSiOMk7rC/wCw5wxISv1n MEKVNjacxmcr6cYAL1FNgw8efMmDHaWTMKWCoSA/9+Emuj7dXR28pLfrPhSP2rzeV6G7 mmY+cl/DPg0EL0+wpkhXrdmwUbL3YratApoRsZK4ztO+9pfr3+45ERGPn5+Q0K9vA5AK xP7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=11623Q9N35sn4n0cdJ5SZiaWyungWhuQfe2dol9yZ90=; b=b1H329X78ommnYpcbJNk6xL+cYCkTpOAPqKMwIECcfHAF/QIcqiZTZVK1VtrMQmdyG nPOIeXEeJlL7W7FfOzVcOc0TtLp3oSjq5/ZBY5WUWt1k7jmR6p+SrlqEe6fK+swqkgW4 aJfD1LyEbGD4vwLP4fWXNtvTWZKQPDD12nBitMJFByabq1BCBGI03x1eCpZ74bvRVtwy tHQW/vFTH4wUNo09JGH9EgvuRg/0PuvXsR3SAYSUgC8WpF5o65ggPOQXoF1S5Cp0p0J4 P9MxwQvZpvVhB872LbaXzLDhmSGoN27JrOVnSpFK7GSqD++SrnH5szNr49ZodkNfbKBI woig== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=czVVSGjz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id x18-v6si1873266qva.281.2018.06.26.09.29.19 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 26 Jun 2018 09:29:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=czVVSGjz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53804 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fXqqF-0008WN-Ik for patch@linaro.org; Tue, 26 Jun 2018 12:29:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42470) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fXqgp-0000vb-7p for qemu-devel@nongnu.org; Tue, 26 Jun 2018 12:19:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fXqgl-0005tp-12 for qemu-devel@nongnu.org; Tue, 26 Jun 2018 12:19:35 -0400 Received: from mail-pl0-x22d.google.com ([2607:f8b0:400e:c01::22d]:39942) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fXqgk-0005tU-Of for qemu-devel@nongnu.org; Tue, 26 Jun 2018 12:19:30 -0400 Received: by mail-pl0-x22d.google.com with SMTP id t6-v6so3431722plo.7 for ; Tue, 26 Jun 2018 09:19:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=11623Q9N35sn4n0cdJ5SZiaWyungWhuQfe2dol9yZ90=; b=czVVSGjzYHyIXvNhiRshj+Y5I+VsXZMrCrb/3GwOgM5ItP3hetBsOnnvXcnCL82/9C Zm/YhitbHO72LClY3Ef8Dx5Hgi71kFRVlY7WURGjUHSmkXXugodaV6zg2yRlSbOMyQ/w UiojWMYkEGKrN8+uypnliJTfqZBi7nqNrIhR8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=11623Q9N35sn4n0cdJ5SZiaWyungWhuQfe2dol9yZ90=; b=tVt4KXpu25NbEwdR19a6lEnrfjnDRySKMsdq0Slz6trKEdm68g2sbkQfgr+4ADrt8K TcjGZ/EDv+kl0rO8zR3G67vwiIvFRLkZoEMAPEZbzyDfx7YO/jL72sy5JE//dT7Odfb0 WHmyuQ2DVtleAwH6tBGIqOT55ZeS4QYQA9bzoITxwwKSsjfgyjNAd524H8XWVh4Lp+XK MCJWK292YHeSut1Bm13nWCYAkF3h36ZpX3/PCB7jJUzWdZDaIoHrn7MXawvGRVveAfu/ pQAsnnYVKhGGlqnKjD27PKVbOn3N/C/6XozZFQAmYy4lErqAogTVHJ9saQiLfWnB2lyj ygkw== X-Gm-Message-State: APt69E0TNNh6T2gcY67gzk1iJNg4cxtWLOwuyrDypGSuaw58paY3y1f9 +QWk0Em43Si30VDEYJec0fZCk6XTHz8= X-Received: by 2002:a17:902:d688:: with SMTP id v8-v6mr2358559ply.59.1530029969493; Tue, 26 Jun 2018 09:19:29 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id 67-v6sm5054306pfm.171.2018.06.26.09.19.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 26 Jun 2018 09:19:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Tue, 26 Jun 2018 09:19:12 -0700 Message-Id: <20180626161921.27941-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180626161921.27941-1-richard.henderson@linaro.org> References: <20180626161921.27941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22d Subject: [Qemu-devel] [PATCH 04/13] target/ppc: Use atomic cmpxchg for STQCX X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" When running in a parallel context, we must use a helper in order to perform the 128-bit atomic operation. When running in a serial context, do the compare before the store. Signed-off-by: Richard Henderson --- target/ppc/helper.h | 2 + target/ppc/mem_helper.c | 38 +++++++++++++++++ target/ppc/translate.c | 95 ++++++++++++++++++++++++++--------------- 3 files changed, 101 insertions(+), 34 deletions(-) -- 2.17.1 diff --git a/target/ppc/helper.h b/target/ppc/helper.h index cbc1228570..5706c2497f 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -807,4 +807,6 @@ DEF_HELPER_FLAGS_5(stq_le_parallel, TCG_CALL_NO_WG, void, env, tl, i64, i64, i32) DEF_HELPER_FLAGS_5(stq_be_parallel, TCG_CALL_NO_WG, void, env, tl, i64, i64, i32) +DEF_HELPER_5(stqcx_le_parallel, i32, env, tl, i64, i64, i32) +DEF_HELPER_5(stqcx_be_parallel, i32, env, tl, i64, i64, i32) #endif diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 57e301edc3..8f0d86d104 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -245,6 +245,44 @@ void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr, Int128 val = int128_make128(lo, hi); helper_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); } + +uint32_t helper_stqcx_le_parallel(CPUPPCState *env, target_ulong addr, + uint64_t new_lo, uint64_t new_hi, + uint32_t opidx) +{ + bool success = false; + + if (likely(addr == env->reserve_addr)) { + Int128 oldv, cmpv, newv; + + cmpv = int128_make128(env->reserve_val2, env->reserve_val); + newv = int128_make128(new_lo, new_hi); + oldv = helper_atomic_cmpxchgo_le_mmu(env, addr, cmpv, newv, + opidx, GETPC()); + success = int128_eq(oldv, cmpv); + } + env->reserve_addr = -1; + return env->so + success * CRF_EQ_BIT; +} + +uint32_t helper_stqcx_be_parallel(CPUPPCState *env, target_ulong addr, + uint64_t new_lo, uint64_t new_hi, + uint32_t opidx) +{ + bool success = false; + + if (likely(addr == env->reserve_addr)) { + Int128 oldv, cmpv, newv; + + cmpv = int128_make128(env->reserve_val2, env->reserve_val); + newv = int128_make128(new_lo, new_hi); + oldv = helper_atomic_cmpxchgo_be_mmu(env, addr, cmpv, newv, + opidx, GETPC()); + success = int128_eq(oldv, cmpv); + } + env->reserve_addr = -1; + return env->so + success * CRF_EQ_BIT; +} #endif /*****************************************************************************/ diff --git a/target/ppc/translate.c b/target/ppc/translate.c index 3d63a62269..c7b9d226eb 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -3332,50 +3332,77 @@ static void gen_lqarx(DisasContext *ctx) /* stqcx. */ static void gen_stqcx_(DisasContext *ctx) { - TCGv EA; - int reg = rS(ctx->opcode); - int len = 16; -#if !defined(CONFIG_USER_ONLY) - TCGLabel *l1; - TCGv gpr1, gpr2; -#endif + int rs = rS(ctx->opcode); + TCGv EA, hi, lo; - if (unlikely((rD(ctx->opcode) & 1))) { + if (unlikely(rs & 1)) { gen_inval_exception(ctx, POWERPC_EXCP_INVAL_INVAL); return; } + gen_set_access_type(ctx, ACCESS_RES); - EA = tcg_temp_local_new(); + EA = tcg_temp_new(); gen_addr_reg_index(ctx, EA); - if (len > 1) { - gen_check_align(ctx, EA, (len) - 1); - } -#if defined(CONFIG_USER_ONLY) - gen_conditional_store(ctx, EA, reg, 16); + /* Note that the low part is always in RS+1, even in LE mode. */ + lo = cpu_gpr[rs + 1]; + hi = cpu_gpr[rs]; + + if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { + TCGv_i32 oi = tcg_const_i32(DEF_MEMOP(MO_Q) | MO_ALIGN_16); +#ifdef CONFIG_ATOMIC128 + if (ctx->le_mode) { + gen_helper_stqcx_le_parallel(cpu_crf[0], cpu_env, EA, lo, hi, oi); + } else { + gen_helper_stqcx_le_parallel(cpu_crf[0], cpu_env, EA, lo, hi, oi); + } #else - tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so); - l1 = gen_new_label(); - tcg_gen_brcond_tl(TCG_COND_NE, EA, cpu_reserve, l1); - tcg_gen_ori_i32(cpu_crf[0], cpu_crf[0], CRF_EQ); - - if (unlikely(ctx->le_mode)) { - gpr1 = cpu_gpr[reg + 1]; - gpr2 = cpu_gpr[reg]; - } else { - gpr1 = cpu_gpr[reg]; - gpr2 = cpu_gpr[reg + 1]; - } - tcg_gen_qemu_st_tl(gpr1, EA, ctx->mem_idx, DEF_MEMOP(MO_Q)); - gen_addr_add(ctx, EA, EA, 8); - tcg_gen_qemu_st_tl(gpr2, EA, ctx->mem_idx, DEF_MEMOP(MO_Q)); - - gen_set_label(l1); - tcg_gen_movi_tl(cpu_reserve, -1); + /* Restart with exclusive lock. */ + gen_helper_exit_atomic(cpu_env); + ctx->base.is_jmp = DISAS_NORETURN; #endif - tcg_temp_free(EA); -} + tcg_temp_free(EA); + tcg_temp_free_i32(oi); + } else { + TCGLabel *lab_fail = gen_new_label(); + TCGLabel *lab_over = gen_new_label(); + TCGv_i64 t0 = tcg_temp_new_i64(); + TCGv_i64 t1 = tcg_temp_new_i64(); + tcg_gen_brcond_tl(TCG_COND_NE, EA, cpu_reserve, lab_fail); + tcg_temp_free(EA); + + gen_qemu_ld64_i64(ctx, t0, cpu_reserve); + tcg_gen_ld_i64(t1, cpu_env, (ctx->le_mode + ? offsetof(CPUPPCState, reserve_val2) + : offsetof(CPUPPCState, reserve_val))); + tcg_gen_brcond_i64(TCG_COND_NE, t0, t1, lab_fail); + + tcg_gen_addi_i64(t0, cpu_reserve, 8); + gen_qemu_ld64_i64(ctx, t0, t0); + tcg_gen_ld_i64(t1, cpu_env, (ctx->le_mode + ? offsetof(CPUPPCState, reserve_val) + : offsetof(CPUPPCState, reserve_val2))); + tcg_gen_brcond_i64(TCG_COND_NE, t0, t1, lab_fail); + + /* Success */ + gen_qemu_st64_i64(ctx, ctx->le_mode ? lo : hi, cpu_reserve); + tcg_gen_addi_i64(t0, cpu_reserve, 8); + gen_qemu_st64_i64(ctx, ctx->le_mode ? hi : lo, t0); + + tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so); + tcg_gen_ori_i32(cpu_crf[0], cpu_crf[0], CRF_EQ); + tcg_gen_br(lab_over); + + gen_set_label(lab_fail); + tcg_gen_trunc_tl_i32(cpu_crf[0], cpu_so); + + gen_set_label(lab_over); + tcg_gen_movi_tl(cpu_reserve, -1); + tcg_temp_free_i64(t0); + tcg_temp_free_i64(t1); + } +} #endif /* defined(TARGET_PPC64) */ /* sync */