From patchwork Mon Jan 28 15:58:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156781 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3611755jaa; Mon, 28 Jan 2019 08:19:49 -0800 (PST) X-Google-Smtp-Source: ALg8bN63CFCZJU4F14A7w/J2aFmveo0NDpoRN0/RNDFidjLSVtUSNd7B9hGqQ6AsIuKwo1SCDgfE X-Received: by 2002:adf:e9d1:: with SMTP id l17mr20964952wrn.73.1548692389647; Mon, 28 Jan 2019 08:19:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692389; cv=none; d=google.com; s=arc-20160816; b=WPhl+3D8jfFcCtjvNIDuehAnj/IHmZCOUBtCxZj8zl+BJKBPZlU1gVuB59Ely11gt0 l9WhlAaRbbZqMTlYV1WelhCGaYZMDX+obzDIjp9eLb1OnWzNoME+VFSFF9ZWVhhSl131 IwAwww5INXk/XPPv5Gm9vqITP017syqBRD06ctnCK0x9Kz8AFR4ku2SRCsq7i2zfyks+ s6dv2nCoaDduStV9om71DJJKxcJsHmR49KkEtW2H/Hn/ViAWyEQBdb0BQdHWNxQih+57 otOVJ64AkvwhGH1I+rewR1op+fQBvsGiXnpaKjMl7aFmGqJplfw+lrNwq/0QJcCzKCaZ WJLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=GdYICPq4CpG4O+JQQaCNuqVcH+thq20RL0BQO6HO2vwFSvaTDpD2PV6Is3XPGkiQ9E HTvdAphbMNQGZ9yYwBcRvnACyhlPl7U0VDIbABwjAzBFa5XE92peIrX5+3QGWpAwoScJ pPBuT19gRLPxOxWltAV/qwEkBXr7YKMcm8uU2W12EwZarHP11aAs/ZcYSHm9DUheMpLN QqnviTZCUdhF2skgvyieh3bqJT3d83KxHHCRpCTqUAg096Ec/9Nc2bZ2cR42QBdtom/R dM+qykYqLXhC3EUjPfip3Jyw4IpTRhXHhzpruhmhqaBmqcJbQq4zBky+TsS/9q49yFaf GFPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zsr6VqYx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n1si87612890wre.10.2019.01.28.08.19.49 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:19:49 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zsr6VqYx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34424 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9dU-0002ta-Fn for patch@linaro.org; Mon, 28 Jan 2019 11:19:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33655) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K7-0005ET-H1 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007w7-V6 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:38248) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007pP-OF for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x62d.google.com with SMTP id e5so7924314plb.5 for ; Mon, 28 Jan 2019 07:59:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=Zsr6VqYxkcJI6lX9hZptFADvGzpqquZo6KTtPZ8McOxf+AO6LOEQyRWOHIMCqCbxCg ye/RYQB1EaV1kPcTJYTerPFzmdsrYJmmzQVmmUGb3BdmFEo5zvgiQEoyiaGzm8+5J+kY XqpzTYpTSOgCFqaBP2+dEkmoW4aifem7c1cyk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=LmAhQ+GeqAMKBEU1dzO8kidaD6dhdxF6Ae2jOtZYALfgQmF1daLPQLGOxcE6soCZJv 0Rg4/ZiK8V7gPAziOOuhizphvfbHyRQZzqanrawj8yMYfGJnczGTk2Ukq+7QjZL3lv7W 49ldDgyI+9Jxjjt/mW704uQST+DS2OIWCIxNQV8CY8KPnr++zuRr44tlFMc9t0hSj4Od IFNVmry4EjPA+E8v2jV4LblxX+gAVv/lNueHp6uiBaTtSS5wYaEGy8CfZWKsIieFBF6Z d64s8XwBJq39ZACMWTiLwbhEdWiWRnnrrSuz81D1TYT0Meem7RkdzpsnaBVMGdYFGjYw kzug== X-Gm-Message-State: AJcUukcucnfcRsUFzS8ckeEyXvmzz0m3b+9Duw8VcixG262mxrL2d+3y VqD5YpIJaM+8I8ulmTE4PWXw+gfry0o= X-Received: by 2002:a17:902:a5c3:: with SMTP id t3mr21943192plq.117.1548691171653; Mon, 28 Jan 2019 07:59:31 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:30 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:59 -0800 Message-Id: <20190128155907.20607-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::62d Subject: [Qemu-devel] [PULL 15/23] tcg/ppc: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.h | 2 +- tcg/ppc/tcg-target.inc.c | 91 ++++++++++++++++++++++------------------ 2 files changed, 52 insertions(+), 41 deletions(-) -- 2.17.2 diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index b51854b5cf..95b735b0bb 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -34,7 +34,7 @@ #define TCG_TARGET_NB_REGS 32 #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3, diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c index 8c1cfdd7ac..773690f1d9 100644 --- a/tcg/ppc/tcg-target.inc.c +++ b/tcg/ppc/tcg-target.inc.c @@ -327,6 +327,7 @@ static int tcg_target_const_match(tcg_target_long val, TCGType type, #define LHZ OPCD( 40) #define LHA OPCD( 42) #define LWZ OPCD( 32) +#define LWZUX XO31( 55) #define STB OPCD( 38) #define STH OPCD( 44) #define STW OPCD( 36) @@ -338,6 +339,7 @@ static int tcg_target_const_match(tcg_target_long val, TCGType type, #define LD XO58( 0) #define LDX XO31( 21) #define LDU XO58( 1) +#define LDUX XO31( 53) #define LWA XO58( 2) #define LWAX XO31(341) @@ -1503,6 +1505,10 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + /* Perform the TLB load and compare. Places the result of the comparison in CR7, loads the addend of the TLB into R3, and returns the register containing the guest address (zero-extended into R4). Clobbers R0 and R2. */ @@ -1513,61 +1519,63 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGMemOp opc, { int cmp_off = (is_read - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - TCGReg base = TCG_AREG0; + ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - /* Extract the page index, shifted into place for tlb index. */ - if (TCG_TARGET_REG_BITS == 64) { - if (TARGET_LONG_BITS == 32) { - /* Zero-extend the address into a place helpful for further use. */ - tcg_out_ext32u(s, TCG_REG_R4, addrlo); - addrlo = TCG_REG_R4; - } else { - tcg_out_rld(s, RLDICL, TCG_REG_R3, addrlo, - 64 - TARGET_PAGE_BITS, 64 - CPU_TLB_BITS); + if (table_off > 0x7fff) { + int mask_hi = mask_off - (int16_t)mask_off; + int table_hi = table_off - (int16_t)table_off; + + table_base = TCG_REG_R4; + if (mask_hi == table_hi) { + mask_base = table_base; + } else if (mask_hi) { + mask_base = TCG_REG_R3; + tcg_out32(s, ADDIS | TAI(mask_base, TCG_AREG0, mask_hi >> 16)); } + tcg_out32(s, ADDIS | TAI(table_base, TCG_AREG0, table_hi >> 16)); + mask_off -= mask_hi; + table_off -= table_hi; } - /* Compensate for very large offsets. */ - if (add_off >= 0x8000) { - int low = (int16_t)cmp_off; - int high = cmp_off - low; - assert((high & 0xffff) == 0); - assert(cmp_off - high == (int16_t)(cmp_off - high)); - assert(add_off - high == (int16_t)(add_off - high)); - tcg_out32(s, ADDIS | TAI(TCG_REG_TMP1, base, high >> 16)); - base = TCG_REG_TMP1; - cmp_off -= high; - add_off -= high; - } + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R4, table_base, table_off); - /* Extraction and shifting, part 2. */ - if (TCG_TARGET_REG_BITS == 32 || TARGET_LONG_BITS == 32) { - tcg_out_rlw(s, RLWINM, TCG_REG_R3, addrlo, - 32 - (TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS), - 32 - (CPU_TLB_BITS + CPU_TLB_ENTRY_BITS), - 31 - CPU_TLB_ENTRY_BITS); + /* Extract the page index, shifted into place for tlb index. */ + if (TCG_TARGET_REG_BITS == 32) { + tcg_out_shri32(s, TCG_REG_TMP1, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); } else { - tcg_out_shli64(s, TCG_REG_R3, TCG_REG_R3, CPU_TLB_ENTRY_BITS); + tcg_out_shri64(s, TCG_REG_TMP1, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); } + tcg_out32(s, AND | SAB(TCG_REG_R3, TCG_REG_R3, TCG_REG_TMP1)); - tcg_out32(s, ADD | TAB(TCG_REG_R3, TCG_REG_R3, base)); - - /* Load the tlb comparator. */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_R4, TCG_REG_R3, cmp_off); - tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_TMP1, TCG_REG_R3, cmp_off + 4); + /* Load the TLB comparator. */ + if (cmp_off == 0 && TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { + uint32_t lxu = (TCG_TARGET_REG_BITS == 32 || TARGET_LONG_BITS == 32 + ? LWZUX : LDUX); + tcg_out32(s, lxu | TAB(TCG_REG_TMP1, TCG_REG_R3, TCG_REG_R4)); } else { - tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP1, TCG_REG_R3, cmp_off); + tcg_out32(s, ADD | TAB(TCG_REG_R3, TCG_REG_R3, TCG_REG_R4)); + if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_TMP1, TCG_REG_R3, cmp_off + 4); + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_R4, TCG_REG_R3, cmp_off); + } else { + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP1, TCG_REG_R3, cmp_off); + } } /* Load the TLB addend for use on the fast path. Do this asap to minimize any load use delay. */ - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, TCG_REG_R3, add_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, TCG_REG_R3, + offsetof(CPUTLBEntry, addend)); /* Clear the non-page, non-alignment bits from the address */ if (TCG_TARGET_REG_BITS == 32) { @@ -1600,6 +1608,9 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGMemOp opc, if (TARGET_LONG_BITS == 32) { tcg_out_rlw(s, RLWINM, TCG_REG_R0, t, 0, (32 - a_bits) & 31, 31 - TARGET_PAGE_BITS); + /* Zero-extend the address for use in the final address. */ + tcg_out_ext32u(s, TCG_REG_R4, addrlo); + addrlo = TCG_REG_R4; } else if (a_bits == 0) { tcg_out_rld(s, RLDICR, TCG_REG_R0, t, 0, 63 - TARGET_PAGE_BITS); } else {