From patchwork Sat Apr 8 02:42:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 671480 Delivered-To: patch@linaro.org Received: by 2002:a5d:4d08:0:0:0:0:0 with SMTP id z8csp608247wrt; Fri, 7 Apr 2023 19:50:36 -0700 (PDT) X-Google-Smtp-Source: AKy350YX9JtZ2fyIlEJyd/A0Q9DjcW29B7SxUxMRD2bENfXMuR907kGTY1VaHRMYV/Qjnx/LGjdM X-Received: by 2002:a05:6214:2a8b:b0:56e:f9a2:1aff with SMTP id jr11-20020a0562142a8b00b0056ef9a21affmr6881136qvb.35.1680922236425; Fri, 07 Apr 2023 19:50:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680922236; cv=none; d=google.com; s=arc-20160816; b=hjMZoSsrvF/Z4Cuz/3PfhTPWb03XYawPG2py5Lcc47yVgeWHzRRyiBjVvF4zVDizMP y47uunK4bgNF/tpa4f8dViJ3gTMbA0A0pOARok/lFR9nnM+ndEQV/x5rcSgMOYmWPuLh vYqGy9+viFV+GVljAe2gh9NBSm4im8VGlQuC6F0myYwmQnpuUZt+QznUaMPpGFIHusoX clvrw4anhaDH2L1afPeySqWBX/1nSdTqL7hBJX2Wjt4sR5GpgTX3Ge55RO9q9GtlmSvS S2j6kxyrBIqZKH5KjdZuIIpWBfovEfGyVH/WkGgwEmZjAW/3+J4tH+EX4/w7BiKiN8dm 8scQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=hlH9Syu5nBco3hVib/PJ96sBJ5uHmafkdc2luPh0/Cc=; b=DtkVHL6ohGEfe9DGm99ACM6EJYvT/RTlzp3fL1wJ2AOkVhvDvVx2zqwNxh0RgOQwcj cwEdb56fDmdVfqWFdfXKAjwAuPj4SK5E3e5iHaOlJ0cMoBKPhQ+IzWiuVHORWCcfwrK/ MfIvvnCCD8ExC+8fU5LCps9sKLoF1rGiUfeF2nB9wfU05wWKRdXBB88gvYGpSc0omRhz MDDDdxXUHG3OMFYYm1Aftb1N98BzJhpsI2Q6m2XaSwM/RMqFC+sem0CKL7m3rQlu/xF5 B+1REqvAZpkMpj0hYPngDg/0LS1WjhYuPLsIPnAU1iH8Hr0hrZREjcqNbAiSnUrx7zae HPYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oyMP87kL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c8-20020ad45ae8000000b005aba6f67e1fsi3841251qvh.401.2023.04.07.19.50.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 07 Apr 2023 19:50:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=oyMP87kL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pkyYZ-0007gl-7n; Fri, 07 Apr 2023 22:43:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pkyYX-0007dH-Bb for qemu-devel@nongnu.org; Fri, 07 Apr 2023 22:43:57 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pkyYA-0005hN-2m for qemu-devel@nongnu.org; Fri, 07 Apr 2023 22:43:57 -0400 Received: by mail-pj1-x1034.google.com with SMTP id j14-20020a17090a7e8e00b002448c0a8813so4738946pjl.0 for ; Fri, 07 Apr 2023 19:43:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1680921812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hlH9Syu5nBco3hVib/PJ96sBJ5uHmafkdc2luPh0/Cc=; b=oyMP87kLU53OJi80NV9nSKfQvn3DkDtLju5WKWBQJrf8WGcM9PrpEArmw/MaQVOZJN t5VbhOkWanFrOifO8SmWlo81VxscNmxh/COmCd9UOFv6ogaJZAhiCD/CjnBjyvrc0AMc Yq6Rm0G+rnEz/t5uf8ldEFgLkHi9qxljKufGcKwSmx7vqc+t3lteltcv8dLVt6bMQR3S +nc+3VpmcyntP6kOdhHH/cI8drPFylTs7pJISWqxrUsDboVPNq7ljEB/Q/xGe61c9qfl u2quGFvURkYuHMZbUZizcj+fwS2l8saewHODbYoM/3++XHjETh6yJL+POJt4Vk0vrhqS aygg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680921812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hlH9Syu5nBco3hVib/PJ96sBJ5uHmafkdc2luPh0/Cc=; b=yxGKFQ7xeUspH2tiOF0H7qKarUs3xF4/QHmS4VbRU08ivQUDgjM7maACq3yMzsx0oO ZQIq/ex4OxCMd7+Z4EEWQfGcKeDAuVP5jytOJwMHAk399To5kiNinLxfXrOKvT9sdYtX 0Tz4aMsi51FPLn11yJEUua5RVUU+9B4wSRblbvlHhr6j2uguy9WWQemupj4qBWomivPh m6R3NtmFUvx1sX14J6mjn9tQJNQzA39xBFvDWVXUFVAtD0mxy0425yPiEz4ALT3dYnO6 3fOkwMUFFGRBbiEsLSEZWP/dIN1S0S+DXVpsxQa1j/jnHTD0VDKpYiLjNcHV3DbTXtpL QRmw== X-Gm-Message-State: AAQBX9fiec1Rdv07lUQ9K5Vze0sSLNQ7joS5wvU9wXENSg4H+rvhmy7i +ZeCeCLx7EVTPivgX/ASNjRQA8R6NGYLp4FVrdc= X-Received: by 2002:a17:903:2310:b0:19e:ecaf:c4b4 with SMTP id d16-20020a170903231000b0019eecafc4b4mr6382021plh.4.1680921812651; Fri, 07 Apr 2023 19:43:32 -0700 (PDT) Received: from stoup.. ([2602:ae:1541:f901:8bb4:5a9d:7ab7:b4b8]) by smtp.gmail.com with ESMTPSA id d9-20020a170902c18900b0019d397b0f18sm3530780pld.214.2023.04.07.19.43.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Apr 2023 19:43:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org, qemu-riscv@nongnu.org, qemu-ppc@nongnu.org Subject: [PATCH 18/42] tcg: Introduce tcg_out_movext2 Date: Fri, 7 Apr 2023 19:42:50 -0700 Message-Id: <20230408024314.3357414-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230408024314.3357414-1-richard.henderson@linaro.org> References: <20230408024314.3357414-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, SUSPICIOUS_RECIPS=2.51 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org This is common code in most qemu_{ld,st} slow paths, moving two registers when there may be overlap between sources and destinations. At present, this is only used by 32-bit hosts for 64-bit data, but will shortly be used for more than that. Signed-off-by: Richard Henderson --- tcg/tcg.c | 50 +++++++++++++++++++++++++++++++++++---- tcg/arm/tcg-target.c.inc | 34 +++++++------------------- tcg/i386/tcg-target.c.inc | 16 ++++--------- 3 files changed, 59 insertions(+), 41 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index d82d99e1b0..1c11f15bce 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -115,8 +115,7 @@ static void tcg_out_exts_i32_i64(TCGContext *s, TCGReg ret, TCGReg arg); static void tcg_out_extu_i32_i64(TCGContext *s, TCGReg ret, TCGReg arg); static void tcg_out_extrl_i64_i32(TCGContext *s, TCGReg ret, TCGReg arg); static void tcg_out_addi_ptr(TCGContext *s, TCGReg, TCGReg, tcg_target_long); -static bool tcg_out_xchg(TCGContext *s, TCGType type, TCGReg r1, TCGReg r2) - __attribute__((unused)); +static bool tcg_out_xchg(TCGContext *s, TCGType type, TCGReg r1, TCGReg r2); static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg); static void tcg_out_goto_tb(TCGContext *s, int which); static void tcg_out_op(TCGContext *s, TCGOpcode opc, @@ -365,9 +364,8 @@ void tcg_raise_tb_overflow(TCGContext *s) * * Move or extend @src into @dst, depending on @src_ext and the types. */ -static void __attribute__((unused)) -tcg_out_movext(TCGContext *s, TCGType dst_type, TCGReg dst, - TCGType src_type, MemOp src_ext, TCGReg src) +static void tcg_out_movext(TCGContext *s, TCGType dst_type, TCGReg dst, + TCGType src_type, MemOp src_ext, TCGReg src) { switch (src_ext) { case MO_UB: @@ -413,6 +411,48 @@ tcg_out_movext(TCGContext *s, TCGType dst_type, TCGReg dst, } } +/** + * tcg_out_movext2 -- move and extend two pair + * @s: tcg context + * @d1_type: integral type for destination + * @d1: destination register + * @s1_type: integral type for source + * @s1_ext: extension to apply to source + * @s1: source register + * @d2_type: integral type for destination + * @d2: destination register + * @s2_type: integral type for source + * @s2_ext: extension to apply to source + * @s2: source register + * @scratch: temporary register, or -1 for none + * + * As tcg_out_movext, for both s1->d1 and s2->d2, caring for overlap + * between the sources and destinations. + */ +static void __attribute__((unused)) +tcg_out_movext2(TCGContext *s, TCGType d1_type, TCGReg d1, TCGType s1_type, + MemOp s1_ext, TCGReg s1, TCGType d2_type, TCGReg d2, + TCGType s2_type, MemOp s2_ext, TCGReg s2, int scratch) +{ + if (d1 != s2) { + tcg_out_movext(s, d1_type, d1, s1_type, s1_ext, s1); + tcg_out_movext(s, d2_type, d2, s2_type, s2_ext, s2); + return; + } + if (d2 == s1) { + if (tcg_out_xchg(s, MAX(s1_type, s2_type), s1, s2)) { + /* The data is now in the correct registers, now extend. */ + s1 = d1, s2 = d2; + } else { + tcg_debug_assert(scratch >= 0); + tcg_out_mov(s, s1_type, scratch, s1); + s1 = scratch; + } + } + tcg_out_movext(s, d2_type, d2, s2_type, s2_ext, s2); + tcg_out_movext(s, d1_type, d1, s1_type, s1_ext, s1); +} + #define C_PFX1(P, A) P##A #define C_PFX2(P, A, B) P##A##_##B #define C_PFX3(P, A, B, C) P##A##_##B##_##C diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 4a5d57a41c..bad1e6d399 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1545,7 +1545,7 @@ static void add_qemu_ldst_label(TCGContext *s, bool is_ld, MemOpIdx oi, static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) { - TCGReg argreg, datalo, datahi; + TCGReg argreg; MemOpIdx oi = lb->oi; MemOp opc = get_memop(oi); @@ -1565,20 +1565,11 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) /* Use the canonical unsigned helpers and minimize icache usage. */ tcg_out_call_int(s, qemu_ld_helpers[opc & MO_SIZE]); - datalo = lb->datalo_reg; - datahi = lb->datahi_reg; if ((opc & MO_SIZE) == MO_64) { - if (datalo != TCG_REG_R1) { - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_R0); - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - } else if (datahi != TCG_REG_R0) { - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_R0); - } else { - tcg_out_mov_reg(s, COND_AL, TCG_REG_TMP, TCG_REG_R0); - tcg_out_mov_reg(s, COND_AL, datahi, TCG_REG_R1); - tcg_out_mov_reg(s, COND_AL, datalo, TCG_REG_TMP); - } + tcg_out_movext2(s, TCG_TYPE_I32, lb->datalo_reg, + TCG_TYPE_I32, MO_UL, TCG_REG_R0, + TCG_TYPE_I32, lb->datahi_reg, + TCG_TYPE_I32, MO_UL, TCG_REG_R1, TCG_REG_TMP); } else { tcg_out_movext(s, TCG_TYPE_I32, lb->datalo_reg, TCG_TYPE_I32, opc & MO_SSIZE, TCG_REG_R0); @@ -1663,17 +1654,10 @@ static bool tcg_out_fail_alignment(TCGContext *s, TCGLabelQemuLdst *l) if (TARGET_LONG_BITS == 64) { /* 64-bit target address is aligned into R2:R3. */ - if (l->addrhi_reg != TCG_REG_R2) { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R2, l->addrlo_reg); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R3, l->addrhi_reg); - } else if (l->addrlo_reg != TCG_REG_R3) { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R3, l->addrhi_reg); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R2, l->addrlo_reg); - } else { - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R1, TCG_REG_R2); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R2, TCG_REG_R3); - tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R3, TCG_REG_R1); - } + tcg_out_movext2(s, TCG_TYPE_I32, TCG_REG_R2, + TCG_TYPE_I32, MO_UL, l->addrlo_reg, + TCG_TYPE_I32, TCG_REG_R3, + TCG_TYPE_I32, MO_UL, l->addrhi_reg, TCG_REG_TMP); } else { tcg_out_mov(s, TCG_TYPE_I32, TCG_REG_R1, l->addrlo_reg); } diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 7d6bf30747..54465c7f46 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -1916,7 +1916,6 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) { MemOpIdx oi = l->oi; MemOp opc = get_memop(oi); - TCGReg data_reg; tcg_insn_unit **label_ptr = &l->label_ptr[0]; /* resolve label address */ @@ -1953,18 +1952,13 @@ static bool tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) tcg_out_branch(s, 1, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); - data_reg = l->datalo_reg; if (TCG_TARGET_REG_BITS == 32 && (opc & MO_SIZE) == MO_64) { - if (data_reg == TCG_REG_EDX) { - /* xchg %edx, %eax */ - tcg_out_opc(s, OPC_XCHG_ax_r32 + TCG_REG_EDX, 0, 0, 0); - tcg_out_mov(s, TCG_TYPE_I32, l->datahi_reg, TCG_REG_EAX); - } else { - tcg_out_mov(s, TCG_TYPE_I32, data_reg, TCG_REG_EAX); - tcg_out_mov(s, TCG_TYPE_I32, l->datahi_reg, TCG_REG_EDX); - } + tcg_out_movext2(s, TCG_TYPE_I32, l->datalo_reg, + TCG_TYPE_I32, MO_UL, TCG_REG_EAX, + TCG_TYPE_I32, l->datahi_reg, + TCG_TYPE_I32, MO_UL, TCG_REG_EDX, -1); } else { - tcg_out_movext(s, l->type, data_reg, + tcg_out_movext(s, l->type, l->datalo_reg, TCG_TYPE_REG, opc & MO_SSIZE, TCG_REG_EAX); }