From patchwork Fri Jun 15 19:43:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 138758 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp1250375lji; Fri, 15 Jun 2018 12:51:14 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKL1ySQzkaB1D52pzCzzf/nCn/oe4FRmBxLAjBMnggVmYUPjOhbo5GbON+aaKY2TS8BoKfX X-Received: by 2002:a37:1807:: with SMTP id j7-v6mr2664667qkh.333.1529092274132; Fri, 15 Jun 2018 12:51:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529092274; cv=none; d=google.com; s=arc-20160816; b=wFPybmJYcejGFij8E5e2Rco7AorpLYV6jyEO33Ab6tgA9HzlZHhlg3Y91jOizQP92d 2Xf8nZsMHMmZIb9Oua9d/LzOZzjSwGksEaYd0ict4zEkW2i9/LGtsJp+mahQGkpThCrR 0Didp+oFsFdDjz5hzfxfNEo8p+anXAlaIlLeX4lxENwt+wpsHtfyAzeFaHbRrCZ7keez cxvbZSOd0v6NtNwyQn6UtmEjtStLp/R+sbKDq6da+7FuZt+62HwCWLEdJjG0fTP4I5Aa DiuOInT9TzBo88H4oziJSbv5NNukf/wX5/2j9XrXtl5Zz4TPPoNdpYkMqvBYSU70ozPf +Knw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=0FQ+ZwY1R2rA+XDd8sFaaKFP5ITR5yBL2RD0xEoLhgc=; b=VuMZ6KDjyTO1NCcKO13KKQ0iI4DLkOoDHM2tHAyIdNAFdB0JLAH460q8O255I5hBkE LfVHY+8J+ViOg/RGsdOf95XlATkhVLF6u7xiI+0LMzMjyGT/7qr0R25b6RycocbqVVJH 5DkNmWImQPXesz8Y8o1KBLZUa/YkzdTl9lPl4ZN+7PQ/5OJMKqFfBh3rVxq0LaAdTDd3 64qbZR4t4EKHqL8Mvh6pALOW3EF0ynSmZfTojrkZMcwTmRORUMbLkK7llRFPBYOKfENZ ecxkBjxcJ9xNabQoMaZY5iT09NV+nrNa8LlnvnRXiu0u1garjSsiY6FLZ9ezu3DdhigS yA1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UJZRTTkH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id d36-v6si5259509qvh.276.2018.06.15.12.51.13 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 15 Jun 2018 12:51:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UJZRTTkH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48988 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTukb-0005OF-If for patch@linaro.org; Fri, 15 Jun 2018 15:51:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51554) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTudt-0000Mu-5L for qemu-devel@nongnu.org; Fri, 15 Jun 2018 15:44:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fTudo-0002Tv-M6 for qemu-devel@nongnu.org; Fri, 15 Jun 2018 15:44:17 -0400 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]:41330) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fTudo-0002TQ-D1 for qemu-devel@nongnu.org; Fri, 15 Jun 2018 15:44:12 -0400 Received: by mail-pg0-x244.google.com with SMTP id l65-v6so4852985pgl.8 for ; Fri, 15 Jun 2018 12:44:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0FQ+ZwY1R2rA+XDd8sFaaKFP5ITR5yBL2RD0xEoLhgc=; b=UJZRTTkH0rgauJdubrnPVrx+R24hQNz6XkeWojLwctMyhoQ0fbdakaZ+JNTcvaJ+mw qXrgZvk6QJ3V1xiMGDDXTNgATaWpCK3ndwRTvLyL5vlRIFHA59loJCGy8KcTcbdNfDMy 0/ixLm0l7KIzmiBDq1ZzV4+k5aYnaCWCQfyDg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0FQ+ZwY1R2rA+XDd8sFaaKFP5ITR5yBL2RD0xEoLhgc=; b=ky4yFazfU0dvDPXoT/rsD+GGYfzzMaDAtbHry6emB2wdCxL0wqbFWrMgJ79477M/R3 VAJQO6WzmDeBqw73jfSiMQg45Xs0SAPm7pA8ZvDMDd/zMa9oAwqQ3P3FMWVikjzk496C AuQcVNgVTUpAsQRuWJSMJneVadh5DIgN4Np93toyAFb9Hj+rPiiZN+p+BeDKrsE0IHZ1 C4zvc/LSu0+v6f7ehS+L+4itymD+KEQvAKUHOVDD5DLkgh/OJlWCzhYevHGhkXfUmUKo kG9JiSJfrvsBJuXnrSZy8F6MRBKPIXZ3Gu9E2FfjKmofoCicIMGS/TOEtArep+cTUcW6 4Q9g== X-Gm-Message-State: APt69E397LWoFQ3IhSA0TtwVHdMcUS4C9YzsbYqEffpLyDCNpAkrp3et kVwKwBYjQwZw+CDShcvrzTRkfk5TdHk= X-Received: by 2002:a62:904c:: with SMTP id a73-v6mr3364885pfe.145.1529091851051; Fri, 15 Jun 2018 12:44:11 -0700 (PDT) Received: from cloudburst.twiddle.net (rrcs-173-198-77-219.west.biz.rr.com. [173.198.77.219]) by smtp.gmail.com with ESMTPSA id 29-v6sm14038360pfj.14.2018.06.15.12.44.09 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Jun 2018 12:44:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 15 Jun 2018 09:43:41 -1000 Message-Id: <20180615194354.12489-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180615194354.12489-1-richard.henderson@linaro.org> References: <20180615194354.12489-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::244 Subject: [Qemu-devel] [PULL v2 06/19] translate-all: iterate over TBs in a page with PAGE_FOR_EACH_TB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" This commit does several things, but to avoid churn I merged them all into the same commit. To wit: - Use uintptr_t instead of TranslationBlock * for the list of TBs in a page. Just like we did in (c37e6d7e "tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB"), the rationale is the same: these are tagged pointers, not pointers. So use a more appropriate type. - Only check the least significant bit of the tagged pointers. Masking with 3/~3 is unnecessary and confusing. - Introduce the TB_FOR_EACH_TAGGED macro, and use it to define PAGE_FOR_EACH_TB, which improves readability. Note that TB_FOR_EACH_TAGGED will gain another user in a subsequent patch. - Update tb_page_remove to use PAGE_FOR_EACH_TB. In case there is a bug and we attempt to remove a TB that is not in the list, instead of segfaulting (since the list is NULL-terminated) we will reach g_assert_not_reached(). Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Signed-off-by: Richard Henderson --- include/exec/exec-all.h | 2 +- accel/tcg/translate-all.c | 62 ++++++++++++++++++--------------------- 2 files changed, 30 insertions(+), 34 deletions(-) -- 2.17.1 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 8d4306ac25..07653d3c92 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -360,7 +360,7 @@ struct TranslationBlock { struct TranslationBlock *orig_tb; /* first and second physical page containing code. The lower bit of the pointer tells the index in page_next[] */ - struct TranslationBlock *page_next[2]; + uintptr_t page_next[2]; tb_page_addr_t page_addr[2]; /* The following data are used to directly call another TB from diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index a9f2bfb468..52e62125ed 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -103,7 +103,7 @@ typedef struct PageDesc { /* list of TBs intersecting this ram page */ - TranslationBlock *first_tb; + uintptr_t first_tb; #ifdef CONFIG_SOFTMMU /* in order to optimize self modifying code, we count the number of lookups we do to a given page to use a bitmap */ @@ -114,6 +114,15 @@ typedef struct PageDesc { #endif } PageDesc; +/* list iterators for lists of tagged pointers in TranslationBlock */ +#define TB_FOR_EACH_TAGGED(head, tb, n, field) \ + for (n = (head) & 1, tb = (TranslationBlock *)((head) & ~1); \ + tb; tb = (TranslationBlock *)tb->field[n], n = (uintptr_t)tb & 1, \ + tb = (TranslationBlock *)((uintptr_t)tb & ~1)) + +#define PAGE_FOR_EACH_TB(pagedesc, tb, n) \ + TB_FOR_EACH_TAGGED((pagedesc)->first_tb, tb, n, page_next) + /* In system mode we want L1_MAP to be based on ram offsets, while in user mode we want it to be based on virtual addresses. */ #if !defined(CONFIG_USER_ONLY) @@ -815,7 +824,7 @@ static void page_flush_tb_1(int level, void **lp) PageDesc *pd = *lp; for (i = 0; i < V_L2_SIZE; ++i) { - pd[i].first_tb = NULL; + pd[i].first_tb = (uintptr_t)NULL; invalidate_page_bitmap(pd + i); } } else { @@ -943,21 +952,21 @@ static void tb_page_check(void) #endif /* CONFIG_USER_ONLY */ -static inline void tb_page_remove(TranslationBlock **ptb, TranslationBlock *tb) +static inline void tb_page_remove(PageDesc *pd, TranslationBlock *tb) { TranslationBlock *tb1; + uintptr_t *pprev; unsigned int n1; - for (;;) { - tb1 = *ptb; - n1 = (uintptr_t)tb1 & 3; - tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3); + pprev = &pd->first_tb; + PAGE_FOR_EACH_TB(pd, tb1, n1) { if (tb1 == tb) { - *ptb = tb1->page_next[n1]; - break; + *pprev = tb1->page_next[n1]; + return; } - ptb = &tb1->page_next[n1]; + pprev = &tb1->page_next[n1]; } + g_assert_not_reached(); } /* remove the TB from a list of TBs jumping to the n-th jump target of the TB */ @@ -1045,12 +1054,12 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr) /* remove the TB from the page list */ if (tb->page_addr[0] != page_addr) { p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } if (tb->page_addr[1] != -1 && tb->page_addr[1] != page_addr) { p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } @@ -1081,10 +1090,7 @@ static void build_page_bitmap(PageDesc *p) p->code_bitmap = bitmap_new(TARGET_PAGE_SIZE); - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + PAGE_FOR_EACH_TB(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1099,7 +1105,6 @@ static void build_page_bitmap(PageDesc *p) tb_end = ((tb->pc + tb->size) & ~TARGET_PAGE_MASK); } bitmap_set(p->code_bitmap, tb_start, tb_end - tb_start); - tb = tb->page_next[n]; } } #endif @@ -1122,9 +1127,9 @@ static inline void tb_alloc_page(TranslationBlock *tb, p = page_find_alloc(page_addr >> TARGET_PAGE_BITS, 1); tb->page_next[n] = p->first_tb; #ifndef CONFIG_USER_ONLY - page_already_protected = p->first_tb != NULL; + page_already_protected = p->first_tb != (uintptr_t)NULL; #endif - p->first_tb = (TranslationBlock *)((uintptr_t)tb | n); + p->first_tb = (uintptr_t)tb | n; invalidate_page_bitmap(p); #if defined(CONFIG_USER_ONLY) @@ -1401,7 +1406,7 @@ void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end) void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, int is_cpu_write_access) { - TranslationBlock *tb, *tb_next; + TranslationBlock *tb; tb_page_addr_t tb_start, tb_end; PageDesc *p; int n; @@ -1432,11 +1437,7 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, /* we remove all the TBs in the range [start, end[ */ /* XXX: see if in some cases it could be faster to invalidate all the code */ - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); - tb_next = tb->page_next[n]; + PAGE_FOR_EACH_TB(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1474,7 +1475,6 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, -1); } - tb = tb_next; } #if !defined(CONFIG_USER_ONLY) /* if no code remaining, no need to continue to use slow writes */ @@ -1568,18 +1568,15 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } tb_lock(); - tb = p->first_tb; #ifdef TARGET_HAS_PRECISE_SMC - if (tb && pc != 0) { + if (p->first_tb && pc != 0) { current_tb = tcg_tb_lookup(pc); } if (cpu != NULL) { env = cpu->env_ptr; } #endif - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + PAGE_FOR_EACH_TB(p, tb, n) { #ifdef TARGET_HAS_PRECISE_SMC if (current_tb == tb && (current_tb->cflags & CF_COUNT_MASK) != 1) { @@ -1596,9 +1593,8 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, addr); - tb = tb->page_next[n]; } - p->first_tb = NULL; + p->first_tb = (uintptr_t)NULL; #ifdef TARGET_HAS_PRECISE_SMC if (current_tb_modified) { /* Force execution of one insn next time. */