From patchwork Thu Jun 14 19:31:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 138617 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp2527454lji; Thu, 14 Jun 2018 12:39:36 -0700 (PDT) X-Google-Smtp-Source: ADUXVKLdoQFbPEGPI9xuxrSi9PWb82PZ8PWop3m39w/XyIaO1FGohmCZcXUrxTODyHlc7ijBNirV X-Received: by 2002:a37:34ca:: with SMTP id b193-v6mr3277688qka.435.1529005176328; Thu, 14 Jun 2018 12:39:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529005176; cv=none; d=google.com; s=arc-20160816; b=RB2jZTKjxVbT//f4LSSNj2XHHuREGCoN3TQZ9cQqwGbtdnCNBWGl2rPSTMSG9b/oMo AP4U1tsFDSz5TuCWCwXbtoRa2kZzITeRe1Ox3W0OHzHqUFJd9MI4UmSuHRGqwT/+KLAG xfsOC5ApzAW3oVAk/ATy2xBlrFBevahLnbB/5rvAqBWY43oe8+OzfLLPa1HrjSE4VGSN DlSCH2OvdTb2WcJhKw68z85aA/PX1yIno+vq8qfwjYPGXUEZciGhA+swuDs2klXAs2Qs YV141m201iCdsjqZNG4SNgE7BsbSSMMmDG2QvQb6bRMqP026kbJSywhS0NWjRP2RbJai TNcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=8gYiYUL8lXxTailPJ53NLxVw5b580afFuqReCeuTUJw=; b=aSRRAc/7cv6c9CvZDDRkffehsuKIv+LjP5qtRe1L1/h+O7MAzbYUVEsQH4SagT+gAM ZMMv5rwBjLeQ1Y/qnGbOn025FAkt7Kjfs4C4QUg5gdfEtAg8GCFnpYGQsYRN79FhdKad 7ZNoP6ncM0XSXtlzWYxWeNoHqyzHlUmvbrmis0LUMwK6VcAjjMyiDKqlnUPRJdw3kXxw 7fqpUUPlwgguiWFDZFk0a/0g42X/uX4OSILj/ayY+6KyDyaiP+QALex3akUpf6ZcbpR4 7eYA8LsATFAv9kpduvqjn+OjAgXLqF0yl31V3gkLFOL+T8INVIbFNojWH6stByk9zxuy G3Rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Wx3UATVz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id l5-v6si5879646qvc.98.2018.06.14.12.39.36 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 14 Jun 2018 12:39:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Wx3UATVz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42363 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTY5n-00040H-Ly for patch@linaro.org; Thu, 14 Jun 2018 15:39:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36935) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTXyZ-0006wm-Og for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fTXyW-0005kT-FD for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:07 -0400 Received: from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]:40792) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fTXyW-0005jz-73 for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:04 -0400 Received: by mail-pg0-x243.google.com with SMTP id w8-v6so330794pgp.7 for ; Thu, 14 Jun 2018 12:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8gYiYUL8lXxTailPJ53NLxVw5b580afFuqReCeuTUJw=; b=Wx3UATVzU8m6Ax4jTXq4WbpV6ShCGXiafhoUvyTfSRTMIlWhLQbEeN138zrdjPM49L Cknmhjps+vyFJTVw9Cq21fRh3aWhE3Lqz2iixfzAYYh1nkEIBZeayEmEe5MTJrXi1rkI ocMEUG4MOPnd+mQrBFO+KbSLswvXPBpUqfRCc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8gYiYUL8lXxTailPJ53NLxVw5b580afFuqReCeuTUJw=; b=R/DdTKHI2e5cLm8NkORej7NPMaW7RahyHsvYcqIBLoxVsH2esyN0uJuC07eecVxiYf kZa9Kalty88BH1w5qTiElrB0jolqE14BhX9ye+tC9A2OqNZoDSUrDuf8vzLnW7La7AVN dQX1lL15uANU9fptSTEYQeVWBADGVLg04KZ+Vd1kJWnMmnZtLoWqrO3UfXch6ec0mIK2 pOgb9jxW9PVCaXz110o4tI5P8F0DWjuo5CioPw+kHcZsbAZOo8mIl8ConVvceZOPQWFY ctk2i+sXFyyZLECUAW5on5b+hcrC7n4Cg3zUIxcckktJcVV45uJ+3QTSLS+DkopSZT47 WrkQ== X-Gm-Message-State: APt69E2LYDXk8i/AfWwQUTEdtYQ3LUGwLZ69+1X3fsYgsN1uf6jWAAhm XfOfmjcfFru+CS2K43BJfF5p0ZT/Ovk= X-Received: by 2002:a62:968f:: with SMTP id s15-v6mr10792598pfk.191.1529004722926; Thu, 14 Jun 2018 12:32:02 -0700 (PDT) Received: from cloudburst.twiddle.net (rrcs-173-198-77-219.west.biz.rr.com. [173.198.77.219]) by smtp.gmail.com with ESMTPSA id x24-v6sm11532184pfj.104.2018.06.14.12.32.01 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Jun 2018 12:32:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Thu, 14 Jun 2018 09:31:35 -1000 Message-Id: <20180614193147.29680-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180614193147.29680-1-richard.henderson@linaro.org> References: <20180614193147.29680-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::243 Subject: [Qemu-devel] [PULL 06/18] translate-all: iterate over TBs in a page with PAGE_FOR_EACH_TB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" This commit does several things, but to avoid churn I merged them all into the same commit. To wit: - Use uintptr_t instead of TranslationBlock * for the list of TBs in a page. Just like we did in (c37e6d7e "tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB"), the rationale is the same: these are tagged pointers, not pointers. So use a more appropriate type. - Only check the least significant bit of the tagged pointers. Masking with 3/~3 is unnecessary and confusing. - Introduce the TB_FOR_EACH_TAGGED macro, and use it to define PAGE_FOR_EACH_TB, which improves readability. Note that TB_FOR_EACH_TAGGED will gain another user in a subsequent patch. - Update tb_page_remove to use PAGE_FOR_EACH_TB. In case there is a bug and we attempt to remove a TB that is not in the list, instead of segfaulting (since the list is NULL-terminated) we will reach g_assert_not_reached(). Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Signed-off-by: Richard Henderson --- include/exec/exec-all.h | 2 +- accel/tcg/translate-all.c | 62 ++++++++++++++++++--------------------- 2 files changed, 30 insertions(+), 34 deletions(-) -- 2.17.1 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 7684f4cb69..54a3398374 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -360,7 +360,7 @@ struct TranslationBlock { struct TranslationBlock *orig_tb; /* first and second physical page containing code. The lower bit of the pointer tells the index in page_next[] */ - struct TranslationBlock *page_next[2]; + uintptr_t page_next[2]; tb_page_addr_t page_addr[2]; /* The following data are used to directly call another TB from diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index a9f2bfb468..52e62125ed 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -103,7 +103,7 @@ typedef struct PageDesc { /* list of TBs intersecting this ram page */ - TranslationBlock *first_tb; + uintptr_t first_tb; #ifdef CONFIG_SOFTMMU /* in order to optimize self modifying code, we count the number of lookups we do to a given page to use a bitmap */ @@ -114,6 +114,15 @@ typedef struct PageDesc { #endif } PageDesc; +/* list iterators for lists of tagged pointers in TranslationBlock */ +#define TB_FOR_EACH_TAGGED(head, tb, n, field) \ + for (n = (head) & 1, tb = (TranslationBlock *)((head) & ~1); \ + tb; tb = (TranslationBlock *)tb->field[n], n = (uintptr_t)tb & 1, \ + tb = (TranslationBlock *)((uintptr_t)tb & ~1)) + +#define PAGE_FOR_EACH_TB(pagedesc, tb, n) \ + TB_FOR_EACH_TAGGED((pagedesc)->first_tb, tb, n, page_next) + /* In system mode we want L1_MAP to be based on ram offsets, while in user mode we want it to be based on virtual addresses. */ #if !defined(CONFIG_USER_ONLY) @@ -815,7 +824,7 @@ static void page_flush_tb_1(int level, void **lp) PageDesc *pd = *lp; for (i = 0; i < V_L2_SIZE; ++i) { - pd[i].first_tb = NULL; + pd[i].first_tb = (uintptr_t)NULL; invalidate_page_bitmap(pd + i); } } else { @@ -943,21 +952,21 @@ static void tb_page_check(void) #endif /* CONFIG_USER_ONLY */ -static inline void tb_page_remove(TranslationBlock **ptb, TranslationBlock *tb) +static inline void tb_page_remove(PageDesc *pd, TranslationBlock *tb) { TranslationBlock *tb1; + uintptr_t *pprev; unsigned int n1; - for (;;) { - tb1 = *ptb; - n1 = (uintptr_t)tb1 & 3; - tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3); + pprev = &pd->first_tb; + PAGE_FOR_EACH_TB(pd, tb1, n1) { if (tb1 == tb) { - *ptb = tb1->page_next[n1]; - break; + *pprev = tb1->page_next[n1]; + return; } - ptb = &tb1->page_next[n1]; + pprev = &tb1->page_next[n1]; } + g_assert_not_reached(); } /* remove the TB from a list of TBs jumping to the n-th jump target of the TB */ @@ -1045,12 +1054,12 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr) /* remove the TB from the page list */ if (tb->page_addr[0] != page_addr) { p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } if (tb->page_addr[1] != -1 && tb->page_addr[1] != page_addr) { p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } @@ -1081,10 +1090,7 @@ static void build_page_bitmap(PageDesc *p) p->code_bitmap = bitmap_new(TARGET_PAGE_SIZE); - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + PAGE_FOR_EACH_TB(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1099,7 +1105,6 @@ static void build_page_bitmap(PageDesc *p) tb_end = ((tb->pc + tb->size) & ~TARGET_PAGE_MASK); } bitmap_set(p->code_bitmap, tb_start, tb_end - tb_start); - tb = tb->page_next[n]; } } #endif @@ -1122,9 +1127,9 @@ static inline void tb_alloc_page(TranslationBlock *tb, p = page_find_alloc(page_addr >> TARGET_PAGE_BITS, 1); tb->page_next[n] = p->first_tb; #ifndef CONFIG_USER_ONLY - page_already_protected = p->first_tb != NULL; + page_already_protected = p->first_tb != (uintptr_t)NULL; #endif - p->first_tb = (TranslationBlock *)((uintptr_t)tb | n); + p->first_tb = (uintptr_t)tb | n; invalidate_page_bitmap(p); #if defined(CONFIG_USER_ONLY) @@ -1401,7 +1406,7 @@ void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end) void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, int is_cpu_write_access) { - TranslationBlock *tb, *tb_next; + TranslationBlock *tb; tb_page_addr_t tb_start, tb_end; PageDesc *p; int n; @@ -1432,11 +1437,7 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, /* we remove all the TBs in the range [start, end[ */ /* XXX: see if in some cases it could be faster to invalidate all the code */ - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); - tb_next = tb->page_next[n]; + PAGE_FOR_EACH_TB(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1474,7 +1475,6 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, -1); } - tb = tb_next; } #if !defined(CONFIG_USER_ONLY) /* if no code remaining, no need to continue to use slow writes */ @@ -1568,18 +1568,15 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } tb_lock(); - tb = p->first_tb; #ifdef TARGET_HAS_PRECISE_SMC - if (tb && pc != 0) { + if (p->first_tb && pc != 0) { current_tb = tcg_tb_lookup(pc); } if (cpu != NULL) { env = cpu->env_ptr; } #endif - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + PAGE_FOR_EACH_TB(p, tb, n) { #ifdef TARGET_HAS_PRECISE_SMC if (current_tb == tb && (current_tb->cflags & CF_COUNT_MASK) != 1) { @@ -1596,9 +1593,8 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, addr); - tb = tb->page_next[n]; } - p->first_tb = NULL; + p->first_tb = (uintptr_t)NULL; #ifdef TARGET_HAS_PRECISE_SMC if (current_tb_modified) { /* Force execution of one insn next time. */