From patchwork Thu Jun 14 19:31:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 138629 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp2536593lji; Thu, 14 Jun 2018 12:49:23 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIX6Hy7jTya8B3w9cULPUfIzXAqXtp/LEs01Oe1ctDsEhKDrGZn5g5I4IKGNN7oeKdpc6qh X-Received: by 2002:a0c:e2cd:: with SMTP id t13-v6mr3684327qvl.47.1529005763120; Thu, 14 Jun 2018 12:49:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529005763; cv=none; d=google.com; s=arc-20160816; b=RafnVt7mE4ow+nTyei7pRYxIASuTJftqCQ5JqmmRASg2XVKedRJxGXfx2NZz6d6DzZ AZ1pzKl4mjM1rf/Wpik3A3CUcJmAqxF2yxWZk/R/LLSkqIGBv4Fw0Ui1+1QzA3KDuA06 2eas5oCXcfFR9kOrrSgy1CBPJ6o/DbPssmje5UM+RVj4UlnyaMn5NHMLdELiLjvpBgmB BITgXvAZyCLtnFbSWp8fjr7aC4NVJqthCexxopsDT1Z7jH0CN8XfA7qdm6MRghY1hcSQ 8UUuC1kcfbc8LB6n/2u3eGSQPL09nsu2we344WteCDmOciZ2Cn+P9C+yKBM4EtEqnTCD U/6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=t7rbHIfP7bGjD+FWmMNzziJrtUs9iEaHFBrJCstC904=; b=EyTs5aNRpAQQfcVx4AM6lJgH1TEssPSgC0sboW8HLxDCuzHoM9TwMLKrIhL+zwKU8E qCst9lEIGmWnnteliSDeBpbn4dKmLgNQcCUNctRHa6FZDyqrzMr6MXvYCpJFS7m49U0D 1yhZc94UT0Lxl8ZjHuM5gqlXH2aES+vd5OSAbAGr7UYLc010y0TxqV7lStTsCzmmStqy uqGdHTEHaNQIxWY1UOX/LMfHGcjOwYAEnEbVVrZrl38R5hQ7h6o9pvgIXVvkALsksle8 uucPN0oVr0FMEFkvBXLQrmZCEJhCSGBdzXfD7MGrhvClVch2ksvXvM5eHzXh1m4rH8f+ A9tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="b/kWOCtC"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id w13-v6si2514751qto.294.2018.06.14.12.49.22 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 14 Jun 2018 12:49:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="b/kWOCtC"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42451 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTYFG-0003rs-EH for patch@linaro.org; Thu, 14 Jun 2018 15:49:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:37138) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fTXym-000781-Sz for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fTXyl-0005yd-Gp for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:20 -0400 Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:46141) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fTXyl-0005xy-8K for qemu-devel@nongnu.org; Thu, 14 Jun 2018 15:32:19 -0400 Received: by mail-pg0-x242.google.com with SMTP id d2-v6so3353563pga.13 for ; Thu, 14 Jun 2018 12:32:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=t7rbHIfP7bGjD+FWmMNzziJrtUs9iEaHFBrJCstC904=; b=b/kWOCtCeg/JosFKJt2RrFrjieqAcelcioU0X/mCdwn75tPOg6alipy8kPxKb8KED+ g1ggfZLMiytbq8jBnjwV54Gn1b9SID8Fc/IPtevFoKfCGpqCaBCc3VDWY9EfBqoc0H1E pLhq4uw//rP+n9dUBL8/QD491Kt2k4rkkVV6M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=t7rbHIfP7bGjD+FWmMNzziJrtUs9iEaHFBrJCstC904=; b=bODsCmsvpp4N0MjGKpSlkJFqJCj8o4O+FGOAyMFHgJzk4kbnvFgKYOgiZqQ4IdYjUZ qXVdjhyzDnUXaRxJuLP2owaoxGLZFXu4HpvOAOXDxXC33oXPavChjBwroi22Jbup7c0w AzP1yYy9Z5X/Wagvequqw4qJJcIa9b7HgJU3D2YJ0LvSkKlX/gI9+K4NoxN64XjJOX99 ndJQ1366AY7rSjIudv2PF8ODetbH0m+fWRxEJd8FnbU71krffgtfSAyv7L35TEaWIdqQ JbfGyvI7V8KBWBmrVwkJOvlo19RdRC6D4smrJAUAwyz8TzDY0Op8bKoXzcXTi2CJ+5NG kzVA== X-Gm-Message-State: APt69E2jcHUG0v8hE/V10MwsQjJaWPyVl1AwrUnBUrM+S0s8/iwpiDtQ ymaWH+GmLMcab7n4Xf5HyIoBkESd/v4= X-Received: by 2002:aa7:818b:: with SMTP id g11-v6mr10843367pfi.50.1529004738069; Thu, 14 Jun 2018 12:32:18 -0700 (PDT) Received: from cloudburst.twiddle.net (rrcs-173-198-77-219.west.biz.rr.com. [173.198.77.219]) by smtp.gmail.com with ESMTPSA id x24-v6sm11532184pfj.104.2018.06.14.12.32.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Jun 2018 12:32:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Thu, 14 Jun 2018 09:31:43 -1000 Message-Id: <20180614193147.29680-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180614193147.29680-1-richard.henderson@linaro.org> References: <20180614193147.29680-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::242 Subject: [Qemu-devel] [PULL 14/18] translate-all: discard TB when tb_link_page returns an existing matching TB X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" Use the recently-gained QHT feature of returning the matching TB if it already exists. This allows us to get rid of the lookup we perform right after acquiring tb_lock. Suggested-by: Richard Henderson Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Signed-off-by: Richard Henderson --- accel/tcg/cpu-exec.c | 14 ++------- accel/tcg/translate-all.c | 50 +++++++++++++++++++++++++++------ docs/devel/multi-thread-tcg.txt | 3 ++ 3 files changed, 46 insertions(+), 21 deletions(-) -- 2.17.1 Tested-by: Pavel Dovgalyuk diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index d75c35380a..45f6ebc65e 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -245,10 +245,7 @@ void cpu_exec_step_atomic(CPUState *cpu) if (tb == NULL) { mmap_lock(); tb_lock(); - tb = tb_htable_lookup(cpu, pc, cs_base, flags, cf_mask); - if (likely(tb == NULL)) { - tb = tb_gen_code(cpu, pc, cs_base, flags, cflags); - } + tb = tb_gen_code(cpu, pc, cs_base, flags, cflags); tb_unlock(); mmap_unlock(); } @@ -398,14 +395,7 @@ static inline TranslationBlock *tb_find(CPUState *cpu, tb_lock(); acquired_tb_lock = true; - /* There's a chance that our desired tb has been translated while - * taking the locks so we check again inside the lock. - */ - tb = tb_htable_lookup(cpu, pc, cs_base, flags, cf_mask); - if (likely(tb == NULL)) { - /* if no translated code available, then translate it now */ - tb = tb_gen_code(cpu, pc, cs_base, flags, cf_mask); - } + tb = tb_gen_code(cpu, pc, cs_base, flags, cf_mask); mmap_unlock(); /* We add the TB in the virtual pc hash table for the fast lookup */ diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index c75298d08a..2585e6fd3e 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -1581,18 +1581,30 @@ static inline void tb_page_add(PageDesc *p, TranslationBlock *tb, * (-1) to indicate that only one page contains the TB. * * Called with mmap_lock held for user-mode emulation. + * + * Returns a pointer @tb, or a pointer to an existing TB that matches @tb. + * Note that in !user-mode, another thread might have already added a TB + * for the same block of guest code that @tb corresponds to. In that case, + * the caller should discard the original @tb, and use instead the returned TB. */ -static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc, - tb_page_addr_t phys_page2) +static TranslationBlock * +tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc, + tb_page_addr_t phys_page2) { PageDesc *p; PageDesc *p2 = NULL; + void *existing_tb = NULL; uint32_t h; assert_memory_lock(); /* * Add the TB to the page list, acquiring first the pages's locks. + * We keep the locks held until after inserting the TB in the hash table, + * so that if the insertion fails we know for sure that the TBs are still + * in the page descriptors. + * Note that inserting into the hash table first isn't an option, since + * we can only insert TBs that are fully initialized. */ page_lock_pair(&p, phys_pc, &p2, phys_page2, 1); tb_page_add(p, tb, 0, phys_pc & TARGET_PAGE_MASK); @@ -1602,21 +1614,33 @@ static void tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc, tb->page_addr[1] = -1; } + /* add in the hash table */ + h = tb_hash_func(phys_pc, tb->pc, tb->flags, tb->cflags & CF_HASH_MASK, + tb->trace_vcpu_dstate); + qht_insert(&tb_ctx.htable, tb, h, &existing_tb); + + /* remove TB from the page(s) if we couldn't insert it */ + if (unlikely(existing_tb)) { + tb_page_remove(p, tb); + invalidate_page_bitmap(p); + if (p2) { + tb_page_remove(p2, tb); + invalidate_page_bitmap(p2); + } + tb = existing_tb; + } + if (p2) { page_unlock(p2); } page_unlock(p); - /* add in the hash table */ - h = tb_hash_func(phys_pc, tb->pc, tb->flags, tb->cflags & CF_HASH_MASK, - tb->trace_vcpu_dstate); - qht_insert(&tb_ctx.htable, tb, h, NULL); - #ifdef CONFIG_USER_ONLY if (DEBUG_TB_CHECK_GATE) { tb_page_check(); } #endif + return tb; } /* Called with mmap_lock held for user mode emulation. */ @@ -1625,7 +1649,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, uint32_t flags, int cflags) { CPUArchState *env = cpu->env_ptr; - TranslationBlock *tb; + TranslationBlock *tb, *existing_tb; tb_page_addr_t phys_pc, phys_page2; target_ulong virt_page2; tcg_insn_unit *gen_code_buf; @@ -1773,7 +1797,15 @@ TranslationBlock *tb_gen_code(CPUState *cpu, * memory barrier is required before tb_link_page() makes the TB visible * through the physical hash table and physical page list. */ - tb_link_page(tb, phys_pc, phys_page2); + existing_tb = tb_link_page(tb, phys_pc, phys_page2); + /* if the TB already exists, discard what we just translated */ + if (unlikely(existing_tb != tb)) { + uintptr_t orig_aligned = (uintptr_t)gen_code_buf; + + orig_aligned -= ROUND_UP(sizeof(*tb), qemu_icache_linesize); + atomic_set(&tcg_ctx->code_gen_ptr, orig_aligned); + return existing_tb; + } tcg_tb_insert(tb); return tb; } diff --git a/docs/devel/multi-thread-tcg.txt b/docs/devel/multi-thread-tcg.txt index faf8918b23..faf09c6069 100644 --- a/docs/devel/multi-thread-tcg.txt +++ b/docs/devel/multi-thread-tcg.txt @@ -140,6 +140,9 @@ to atomically insert new elements. The lookup caches are updated atomically and the lookup hash uses QHT which is designed for concurrent safe lookup. +Parallel code generation is supported. QHT is used at insertion time +as the synchronization point across threads, thereby ensuring that we only +keep track of a single TranslationBlock for each guest code block. Memory maps and TLBs --------------------