From patchwork Wed Sep 25 18:45:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 174409 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp1025560ill; Wed, 25 Sep 2019 11:56:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqy8ai6mzXDWkEFauNK5kQalA9PjhqnqGUM/mMeA9CCbpyWObN4Ot7v/5Pm9R6BEdJv4Hql+ X-Received: by 2002:ac8:184:: with SMTP id x4mr882858qtf.249.1569437791065; Wed, 25 Sep 2019 11:56:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569437791; cv=none; d=google.com; s=arc-20160816; b=LyeyXaR70pkmiN1Cj0NQpufOEwumS1dUwKr5/4/5bSwbWvQtda1tHBpyMifZLloPPu jsaNhI1efBLUGnWhnnCoCeh7vNIrB8ovSALJXCK3luS9TAqnbIFc3Ar0WX8bfipwcXAy /waSAXawepDZqprHcNZxjyqBCv5EbsMvhdshVBrAzgstzgFnF02Gjrpffubwqd6prZ5l Dr7X3VUJ4I53ODfSyscXLA4eVaRMbHTu30KeXUN7uO/bHmUcO4WR++B2Qd5qsBEsr/QT V86VeJDcRoEK5et2ruw7KRWULWmJMUfYCtmyS2tUFgdYkAwEYJFMcSL+/ClcpSoA3F+M 2WfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=VU8gXJ0ha+BxU/auLcZl3iqhGwLSOAQ2I5JEO3/h8Yw=; b=QKb+9/1FSaFXGPjasAMIbcwkZdA6n8G7yw7vujULblLD7fZQ1uc6nk9y7PH1IsiXvx DnA6ZiEF30/rUjexCV554ulf//MEBY5yQJNV6YuTyQlBIojZ/LtsrFk+r1ldJbN0Abra F4j2ryxeIOZaN454P8W3pde/Uz8n30G9oS/vIvlxRVorqTbk5ydzaYiRX9XzHYbNsR9l 3ePXpZs5Uy7mYI0x06hUpz/WcaOF6muWWkx6T49O7mA2STFO774DnPqtWNnhedvJ6oEn QjivIwyLOAIXnh2nFe38qBPaRQC2Tfcn/a3ycjilxOKjlx3+lKiOtD3EAM9lG/jUn4ts XZ/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yj84Hus4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v8si421446qtb.125.2019.09.25.11.56.30 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 25 Sep 2019 11:56:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=yj84Hus4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iDCSk-0001jt-6i for patch@linaro.org; Wed, 25 Sep 2019 14:56:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47016) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iDCIc-0001ku-IO for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iDCIb-0004FO-13 for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:02 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:33419) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iDCIa-0004F1-P7 for qemu-devel@nongnu.org; Wed, 25 Sep 2019 14:46:00 -0400 Received: by mail-pl1-x636.google.com with SMTP id d22so2945375pls.0 for ; Wed, 25 Sep 2019 11:46:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VU8gXJ0ha+BxU/auLcZl3iqhGwLSOAQ2I5JEO3/h8Yw=; b=yj84Hus4B03MBtSSLlSTLIPbiF7Wmw/YsBAFyO/HFgEdtQYr1YrIeSuJ45AXvu4oZq aEv7huRujb/U3ogGaliSZiU4X3Vh998j7z8VubAfWL+JU7I4SKUgzaqvnyLwNdbwhDWk GqGvnOgWQs20v9VgndldTWVJ4O58kQDrzOsV+z9u8qHpU7C/wn/C+WB3yEWxxYX/Howe bU3IAklyjegUqf7uFylZgAH0fheWQeHaBOTo7+5D30jT0jiAyOYaUqVe+/L2KD8plEdD gRTDdZ2/m1p4eGaB6l+FFwfPEMmpsiMnqJmdBO6YBk0yKQijgo5FNKdKBMr8eCexn8Pf kB5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VU8gXJ0ha+BxU/auLcZl3iqhGwLSOAQ2I5JEO3/h8Yw=; b=JFJqbMpp/hRYXE0yyJPt9jWwHEXoS9J7LXCHbP0WRXkj/dyItmsZpGcTsuBMQtTqP8 lvRB+wK8Fe6pKtgp3u7wXOvafgqSA/QkFFICXAJgJQqY/1CiJGyMEJQXQyMEsPjq1Ole aYYGF1bhSUsZCUzfUuzgDoFF2Rr8Zxijr+MiW3M3k7/rTaz+06QMVd6kNYDcFUFSdmmA el0qt0c5H8v6JcV0CBvPyDUZIOhXywXaaXCrnFbX02B3WaN3+kL0RXdlv+sb4NQudt85 R234boosdgLqUFEEJe/svCTc5yG9DwSTgBA4j0Xe3g3aNp+CK+AnvYqCWx6sgF5csRah dcXQ== X-Gm-Message-State: APjAAAUYVpTGhPCeGM8A7dMcR9FCk+W7yjsd9CEdLV0RHB3mZryKA4DM 0a8sAFzy5K9Xfk1tcnkseuUTZgzqSrA= X-Received: by 2002:a17:902:9a05:: with SMTP id v5mr10778603plp.237.1569437159291; Wed, 25 Sep 2019 11:45:59 -0700 (PDT) Received: from localhost.localdomain ([12.206.46.61]) by smtp.gmail.com with ESMTPSA id l24sm6133229pff.151.2019.09.25.11.45.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2019 11:45:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 06/16] cputlb: Introduce TLB_BSWAP Date: Wed, 25 Sep 2019 11:45:38 -0700 Message-Id: <20190925184548.30673-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190925184548.30673-1-richard.henderson@linaro.org> References: <20190925184548.30673-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::636 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Handle bswap on ram directly in load/store_helper. This fixes a bug with the previous implementation in that one cannot use the I/O path for RAM. Fixes: a26fc6f5152b47f1 Reviewed-by: Alex Bennée Reviewed-by: David Hildenbrand Signed-off-by: Richard Henderson --- include/exec/cpu-all.h | 4 ++- accel/tcg/cputlb.c | 72 +++++++++++++++++++++++++----------------- 2 files changed, 46 insertions(+), 30 deletions(-) -- 2.17.1 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index e0c8dc540c..d148bded35 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -335,12 +335,14 @@ CPUArchState *cpu_copy(CPUArchState *env); #define TLB_MMIO (1 << (TARGET_PAGE_BITS_MIN - 3)) /* Set if TLB entry contains a watchpoint. */ #define TLB_WATCHPOINT (1 << (TARGET_PAGE_BITS_MIN - 4)) +/* Set if TLB entry requires byte swap. */ +#define TLB_BSWAP (1 << (TARGET_PAGE_BITS_MIN - 5)) /* Use this mask to check interception with an alignment mask * in a TCG backend. */ #define TLB_FLAGS_MASK \ - (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT) + (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO | TLB_WATCHPOINT | TLB_BSWAP) /** * tlb_hit_page: return true if page aligned @addr is a hit against the diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index eeba8c9847..028eebcb44 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -737,8 +737,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, address |= TLB_INVALID_MASK; } if (attrs.byte_swap) { - /* Force the access through the I/O slow path. */ - address |= TLB_MMIO; + address |= TLB_BSWAP; } if (!memory_region_is_ram(section->mr) && !memory_region_is_romd(section->mr)) { @@ -901,10 +900,6 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, bool locked = false; MemTxResult r; - if (iotlbentry->attrs.byte_swap) { - op ^= MO_BSWAP; - } - section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -947,10 +942,6 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry, bool locked = false; MemTxResult r; - if (iotlbentry->attrs.byte_swap) { - op ^= MO_BSWAP; - } - section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -1133,8 +1124,8 @@ void *probe_access(CPUArchState *env, target_ulong addr, int size, wp_access, retaddr); } - if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO)) { - /* I/O access */ + /* Reject I/O access, or other required slow-path. */ + if (tlb_addr & (TLB_NOTDIRTY | TLB_MMIO | TLB_BSWAP)) { return NULL; } @@ -1344,6 +1335,7 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, /* Handle anything that isn't just a straight memory access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { CPUIOTLBEntry *iotlbentry; + bool need_swap; /* For anything that is unaligned, recurse through full_load. */ if ((addr & (size - 1)) != 0) { @@ -1357,17 +1349,27 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_READ, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &= ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) { - goto do_aligned_access; - } } + need_swap = size > 1 && (tlb_addr & TLB_BSWAP); + /* Handle I/O access. */ - return io_readx(env, iotlbentry, mmu_idx, addr, - retaddr, access_type, op); + if (likely(tlb_addr & TLB_MMIO)) { + return io_readx(env, iotlbentry, mmu_idx, addr, retaddr, + access_type, op ^ (need_swap * MO_BSWAP)); + } + + haddr = (void *)((uintptr_t)addr + entry->addend); + + /* + * Keep these two load_memop separate to ensure that the compiler + * is able to fold the entire function to a single instruction. + * There is a build-time assert inside to remind you of this. ;-) + */ + if (unlikely(need_swap)) { + return load_memop(haddr, op ^ MO_BSWAP); + } + return load_memop(haddr, op); } /* Handle slow unaligned access (it spans two pages or IO). */ @@ -1394,7 +1396,6 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, return res & MAKE_64BIT_MASK(0, size * 8); } - do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); return load_memop(haddr, op); } @@ -1591,6 +1592,7 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, /* Handle anything that isn't just a straight memory access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { CPUIOTLBEntry *iotlbentry; + bool need_swap; /* For anything that is unaligned, recurse through byte stores. */ if ((addr & (size - 1)) != 0) { @@ -1604,16 +1606,29 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_WRITE, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &= ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) { - goto do_aligned_access; - } } + need_swap = size > 1 && (tlb_addr & TLB_BSWAP); + /* Handle I/O access. */ - io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op); + if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) { + io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, + op ^ (need_swap * MO_BSWAP)); + return; + } + + haddr = (void *)((uintptr_t)addr + entry->addend); + + /* + * Keep these two store_memop separate to ensure that the compiler + * is able to fold the entire function to a single instruction. + * There is a build-time assert inside to remind you of this. ;-) + */ + if (unlikely(need_swap)) { + store_memop(haddr, val, op ^ MO_BSWAP); + } else { + store_memop(haddr, val, op); + } return; } @@ -1682,7 +1697,6 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, return; } - do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); store_memop(haddr, val, op); }