From patchwork Wed Sep 25 18:45:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 174411 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1025654ill; Wed, 25 Sep 2019 11:56:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqzh1qO6abW6XV56pgiBsqJwPU8jN7MRtQ1L8Y+vCxWRQNwat/tfhP+ufyOA/PV4JyQWmlVr X-Received: by 2002:a37:7403:: with SMTP id p3mr4958638qkc.366.1569437797065; Wed, 25 Sep 2019 11:56:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569437797; cv=none; d=google.com; s=arc-20160816; b=NQJVxo1P69/g0SoJH87/rahTc2vxtf/UmcsbW7cd8YUjyfTcUy9/0ACcFhME4FxuIP RNdaFhYLeNh9QOlp6sZKrEAE/cLaMDslx7QPvPXkO7q6q/c2BnMEpLQt7CRgBQ8vdFzD QHZ9U1KIRHzOY6rDsFH1C5c6/HygBV0AVU6aUcRDuHC00x1nrgukysj5aJITi5g0Vchr 7x1A2l/JukrGmcr5R1tV5nYEeUQw/FuRBfommcyQHj5hiAgF64J2o3Anrq6XbGr53RLe zAh06VoQk6d6nqbW5OKELJZ26xZPWn/UWJHtch8mVuQb6lTL9cNVVzVzSO6qYiBchnEY sh5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=thoGiFuad0+sCfOpAMLAT3aKZ6kfKYJYfdagLPNuTGE=; b=LN7I7PltIspVKy92+x9Q5+uQWqkFI/rDShrxdRQNvR/KrGccqOgWpNzH3UM7dTNN2E KWa541eTVQ4TX+2rstageMSWeOu9Masu+d0ARmAzmKaRLO7k3Tae9lV3IrPCmL6pVwIx aGCNxaRfD8b+4JKV7L31uLb279SwzAFtbb6oLC3zgRYtnaEajlCLJ1eaLwdCeWoBc1H2 MS6GfAjsgXtg00JVu1NyfkYvPtNGvr9+Zga3LM5v3WUSSew0zwvDVL93dKElzTxHXu5c 07AxOfOKlchvICEdsia4AdzXKviY4nO1zoVuFqtM7awLYMvqq/OQ3UiBPgqwivh8ujTO +kxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=obzLGpmu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w77si4682029qkb.241.2019.09.25.11.56.36 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 25 Sep 2019 11:56:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=obzLGpmu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56150 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iDCSp-0001v7-V3 for patch@linaro.org; Wed, 25 Sep 2019 14:56:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47152) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iDCIi-0001s1-OH for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iDCIg-0004L5-KO for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:08 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:43872) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iDCIg-0004K9-BU for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:06 -0400 Received: by mail-pl1-x636.google.com with SMTP id f21so539567plj.10 for ; Wed, 25 Sep 2019 11:46:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=thoGiFuad0+sCfOpAMLAT3aKZ6kfKYJYfdagLPNuTGE=; b=obzLGpmus8NXGajsJbBANMPP1Lr9oKl26c9ALC/g3ZVjwdIviv7vHovKTsUKsdr5GX 8UdKcDq2RgoDaJi5E2y8abFH2EtNZf+hGSz5aQOosle62tDMF4+U82BLZ6BlVa5i4r0f symZO6y5FgXpaecoKfYDxWzWdDftRr/NIOsRp9gQTklRBnShR244zyonBxxMv7MQ5cDd d0HczvjrDVGtojvpy9ZMHwd3EgjytwSA8HKUhltokzJOTNkHeWSFXiLzDkMhVIUjgzNE mDAkc7TfHwo+rqDLRwQE15z26wb/wdCqeR8zeEGy2DOPoX3kScAXid2FBmpV6ILNFpMy Pzag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=thoGiFuad0+sCfOpAMLAT3aKZ6kfKYJYfdagLPNuTGE=; b=QNXT1DEbvnS6qi0TjUr4GBNvTLc1E1/cQspOCK/q8bEsZuWPmPl8mf5trv53AUAJES DMqu7qjTfz9xm9CzaSDlleq+YpCgoTKhnLSEScpIN16RdyjIcA/fr0X6Dvr2UtZhFzwZ Cj7JERLIZGquDlAhRytzaWe8NvUNdj8t+mRIp/aWA8Gc437eDBZyWZjwBFKzCYaR2LmI qJOPeYfQLN3rc0rAepl0rsbwhaBdnoAkGsiyJgk1rA1qdQwAhbaHx6qJLsdOq+zswv2V fgJZxh7x5Odk62QX8o24aWit9Z9SKB3o/AdgCYv2CIBH9hiHUxd13GWpfA+nTV4OGFxg kuFQ== X-Gm-Message-State: APjAAAWy3zNX3cJdfWKQamWz0DrhEWWCwnVAd4KaD9FO/ewsMlYNnObN BNJAmgFzAOKC4YLjrlj+WFCfAT6+C+w= X-Received: by 2002:a17:902:7296:: with SMTP id d22mr10991369pll.41.1569437165034; Wed, 25 Sep 2019 11:46:05 -0700 (PDT) Received: from localhost.localdomain ([12.206.46.61]) by smtp.gmail.com with ESMTPSA id l24sm6133229pff.151.2019.09.25.11.46.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2019 11:46:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 10/16] cputlb: Partially inline memory_region_section_get_iotlb Date: Wed, 25 Sep 2019 11:45:42 -0700 Message-Id: <20190925184548.30673-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190925184548.30673-1-richard.henderson@linaro.org> References: <20190925184548.30673-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::636 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" There is only one caller, tlb_set_page_with_attrs. We cannot inline the entire function because the AddressSpaceDispatch structure is private to exec.c, and cannot easily be moved to include/exec/memory-internal.h. Compute is_ram and is_romd once within tlb_set_page_with_attrs. Fold the number of tests against these predicates. Compute cpu_physical_memory_is_clean outside of the tlb lock region. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/exec/exec-all.h | 6 +--- accel/tcg/cputlb.c | 68 ++++++++++++++++++++++++++--------------- exec.c | 22 ++----------- 3 files changed, 47 insertions(+), 49 deletions(-) -- 2.17.1 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 81b02eb2fe..49db07ba0b 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -509,11 +509,7 @@ address_space_translate_for_iotlb(CPUState *cpu, int asidx, hwaddr addr, hwaddr *xlat, hwaddr *plen, MemTxAttrs attrs, int *prot); hwaddr memory_region_section_get_iotlb(CPUState *cpu, - MemoryRegionSection *section, - target_ulong vaddr, - hwaddr paddr, hwaddr xlat, - int prot, - target_ulong *address); + MemoryRegionSection *section); #endif /* vl.c */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 7e9a0f7ac8..4f118d2cc9 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -705,13 +705,14 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, MemoryRegionSection *section; unsigned int index; target_ulong address; - target_ulong code_address; + target_ulong write_address; uintptr_t addend; CPUTLBEntry *te, tn; hwaddr iotlb, xlat, sz, paddr_page; target_ulong vaddr_page; int asidx = cpu_asidx_from_attrs(cpu, attrs); int wp_flags; + bool is_ram, is_romd; assert_cpu_is_self(cpu); @@ -740,18 +741,46 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, if (attrs.byte_swap) { address |= TLB_BSWAP; } - if (!memory_region_is_ram(section->mr) && - !memory_region_is_romd(section->mr)) { - /* IO memory case */ - address |= TLB_MMIO; - addend = 0; - } else { + + is_ram = memory_region_is_ram(section->mr); + is_romd = memory_region_is_romd(section->mr); + + if (is_ram || is_romd) { + /* RAM and ROMD both have associated host memory. */ addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat; + } else { + /* I/O does not; force the host address to NULL. */ + addend = 0; + } + + write_address = address; + if (is_ram) { + iotlb = memory_region_get_ram_addr(section->mr) + xlat; + /* + * Computing is_clean is expensive; avoid all that unless + * the page is actually writable. + */ + if (prot & PAGE_WRITE) { + if (section->readonly) { + write_address |= TLB_DISCARD_WRITE; + } else if (cpu_physical_memory_is_clean(iotlb)) { + write_address |= TLB_NOTDIRTY; + } + } + } else { + /* I/O or ROMD */ + iotlb = memory_region_section_get_iotlb(cpu, section) + xlat; + /* + * Writes to romd devices must go through MMIO to enable write. + * Reads to romd devices go through the ram_ptr found above, + * but of course reads to I/O must go through MMIO. + */ + write_address |= TLB_MMIO; + if (!is_romd) { + address = write_address; + } } - code_address = address; - iotlb = memory_region_section_get_iotlb(cpu, section, vaddr_page, - paddr_page, xlat, prot, &address); wp_flags = cpu_watchpoint_address_matches(cpu, vaddr_page, TARGET_PAGE_SIZE); @@ -791,8 +820,8 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, /* * At this point iotlb contains a physical section number in the lower * TARGET_PAGE_BITS, and either - * + the ram_addr_t of the page base of the target RAM (if NOTDIRTY or ROM) - * + the offset within section->mr of the page base (otherwise) + * + the ram_addr_t of the page base of the target RAM (RAM) + * + the offset within section->mr of the page base (I/O, ROMD) * We subtract the vaddr_page (which is page aligned and thus won't * disturb the low bits) to give an offset which can be added to the * (non-page-aligned) vaddr of the eventual memory access to get @@ -815,25 +844,14 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, } if (prot & PAGE_EXEC) { - tn.addr_code = code_address; + tn.addr_code = address; } else { tn.addr_code = -1; } tn.addr_write = -1; if (prot & PAGE_WRITE) { - tn.addr_write = address; - if (memory_region_is_romd(section->mr)) { - /* Use the MMIO path so that the device can switch states. */ - tn.addr_write |= TLB_MMIO; - } else if (memory_region_is_ram(section->mr)) { - if (section->readonly) { - tn.addr_write |= TLB_DISCARD_WRITE; - } else if (cpu_physical_memory_is_clean( - memory_region_get_ram_addr(section->mr) + xlat)) { - tn.addr_write |= TLB_NOTDIRTY; - } - } + tn.addr_write = write_address; if (prot & PAGE_WRITE_INV) { tn.addr_write |= TLB_INVALID_MASK; } diff --git a/exec.c b/exec.c index dc7001f115..961d7d6497 100644 --- a/exec.c +++ b/exec.c @@ -1459,26 +1459,10 @@ bool cpu_physical_memory_snapshot_get_dirty(DirtyBitmapSnapshot *snap, /* Called from RCU critical section */ hwaddr memory_region_section_get_iotlb(CPUState *cpu, - MemoryRegionSection *section, - target_ulong vaddr, - hwaddr paddr, hwaddr xlat, - int prot, - target_ulong *address) + MemoryRegionSection *section) { - hwaddr iotlb; - - if (memory_region_is_ram(section->mr)) { - /* Normal RAM. */ - iotlb = memory_region_get_ram_addr(section->mr) + xlat; - } else { - AddressSpaceDispatch *d; - - d = flatview_to_dispatch(section->fv); - iotlb = section - d->map.sections; - iotlb += xlat; - } - - return iotlb; + AddressSpaceDispatch *d = flatview_to_dispatch(section->fv); + return section - d->map.sections; } #endif /* defined(CONFIG_USER_ONLY) */