From patchwork Thu Jun 26 10:17:12 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 32547 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f197.google.com (mail-ie0-f197.google.com [209.85.223.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id C43D32066E for ; Thu, 26 Jun 2014 10:19:01 +0000 (UTC) Received: by mail-ie0-f197.google.com with SMTP id lx4sf18695775iec.0 for ; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=diXTX7n9YVsaDzwCCx304pdtjbDPWeTJp46GAyptY50=; b=J2Cr6cnb+ayhrJWEDOViRLojCoRnhpE+E8GKUkpuhXHofebgwYdieQwPfJQ64ICgeD JBhEJKcnGVOAqPyWgIjGk1ccm8ELqZCgf/MlkR5tBB4UAyAo4Dedlb8jnpArbGeiOTLq V73h7C6AhlmgWU2RR/pgfip/kkR0nAZl6wBe5/QXU3gqoKVi7qf4Mc2cB8fARGB8aC7m g6S5buKvNptHR8PmwDcBs3+/+5OLBE1SFVwUZSNSrzzvBoFnUq2FfJZvh8A8E5VeALV5 BNjPP+x7hfT62xo2Ck2L0XJozZmNKHQJmqdrHygxCW7F0v7te2qtvfKSE4GNaiw9z5vF Y4UQ== X-Gm-Message-State: ALoCoQl/ZDw28aK9F8lsssqYe2U19nh+A5Qxbk+Wu7mcP6kzdBZzLhwHgr598ll1guU0hXT0PoZl X-Received: by 10.50.164.202 with SMTP id ys10mr1621899igb.0.1403777941318; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.101.244 with SMTP id u107ls156835qge.52.gmail; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) X-Received: by 10.221.9.72 with SMTP id ov8mr12371274vcb.27.1403777941208; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id dr2si4044226vdb.78.2014.06.26.03.19.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 26 Jun 2014 03:19:01 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id ij19so3267427vcb.37 for ; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) X-Received: by 10.58.208.228 with SMTP id mh4mr12495469vec.22.1403777941079; Thu, 26 Jun 2014 03:19:01 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp19458vcb; Thu, 26 Jun 2014 03:19:00 -0700 (PDT) X-Received: by 10.140.24.230 with SMTP id 93mr2589736qgr.11.1403777940096; Thu, 26 Jun 2014 03:19:00 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id p18si8543434qay.3.2014.06.26.03.18.59 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 26 Jun 2014 03:19:00 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X06k8-00066r-6R; Thu, 26 Jun 2014 10:17:24 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X06k6-00065W-Hu for xen-devel@lists.xen.org; Thu, 26 Jun 2014 10:17:22 +0000 Received: from [85.158.143.35:58729] by server-1.bemta-4.messagelabs.com id B0/14-09496-133FBA35; Thu, 26 Jun 2014 10:17:21 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-2.tower-21.messagelabs.com!1403777838!5974012!3 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 12951 invoked from network); 26 Jun 2014 10:17:21 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-2.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 26 Jun 2014 10:17:21 -0000 X-IronPort-AV: E=Sophos;i="5.01,552,1400025600"; d="scan'208";a="147517491" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 26 Jun 2014 10:17:18 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Thu, 26 Jun 2014 06:17:17 -0400 Received: from marilith-n13-p0.uk.xensource.com ([10.80.229.115] helo=marilith-n13.uk.xensource.com.) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1X06k1-0004Wh-FH; Thu, 26 Jun 2014 11:17:17 +0100 From: Ian Campbell To: Date: Thu, 26 Jun 2014 11:17:12 +0100 Message-ID: <1403777837-16779-3-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1403777793.16595.21.camel@kazak.uk.xensource.com> References: <1403777793.16595.21.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: Ian Jackson , julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH v3 3/8] tools: arm: allocate large pages to guests. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Tries to allocate as many large (1G or 2M) pages as it can to the guest and tries to align to the next larger size on each attempt so that we can attempt to allocate even larger pages on the next iteration (this is currently only exercised via a compile time debug option, which is left in place under a #if 0). Since ARM page tables are consistent at each level there is a common helper function which tries to allocate a levels worth of pages. The exception to this consistency is level 0 which does not support table mappings (0.5TB superpages!). Signed-off-by: Ian Campbell Acked-by: Julien Grall Cc: Ian Jackson --- v3: Drop outdated paragraph from commit message v2: min_t defintion moved into earlier patch drop debug_iters allocate extents array explicitly instead of putting 8M on the stack. Handle OOM, xc_populate_physmap returns 0, which needs handling for L3. --- tools/libxc/xc_dom_arm.c | 119 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 105 insertions(+), 14 deletions(-) diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c index cc64363..c4db19b 100644 --- a/tools/libxc/xc_dom_arm.c +++ b/tools/libxc/xc_dom_arm.c @@ -30,6 +30,13 @@ #define CONSOLE_PFN_OFFSET 0 #define XENSTORE_PFN_OFFSET 1 +#define LPAE_SHIFT 9 + +#define PFN_4K_SHIFT (0) +#define PFN_2M_SHIFT (PFN_4K_SHIFT+LPAE_SHIFT) +#define PFN_1G_SHIFT (PFN_2M_SHIFT+LPAE_SHIFT) +#define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT) + /* get guest IO ABI protocol */ const char *xc_domain_get_native_protocol(xc_interface *xch, uint32_t domid) @@ -249,11 +256,60 @@ static int set_mode(xc_interface *xch, domid_t domid, char *guest_type) return rc; } +/* >0: success, *nr_pfns set to number actually populated + * 0: didn't try with this pfn shift (e.g. misaligned base etc) + * <0: ERROR + */ +static int populate_one_size(struct xc_dom_image *dom, int pfn_shift, + xen_pfn_t base_pfn, xen_pfn_t *nr_pfns, + xen_pfn_t *extents) +{ + uint64_t mask = ((uint64_t)1<<(pfn_shift))-1; + uint64_t next_mask = ((uint64_t)1<<(LPAE_SHIFT))-1; + int nr, i, count = *nr_pfns >> pfn_shift; + + /* No level zero super pages with current hardware */ + if ( pfn_shift == PFN_512G_SHIFT ) + return 0; + + /* Nothing to allocate */ + if ( !count ) + return 0; + + /* base is misaligned for this level */ + if ( mask & base_pfn ) + return 0; + + /* align to the end of a super page at this level */ + if ( count & next_mask ) + count &= next_mask; + + for ( i = 0 ; i < count ; i ++ ) + extents[i] = base_pfn + (i<xch, dom->guest_domid, count, + pfn_shift, 0, extents); + if ( nr <= 0 ) return nr; + DOMPRINTF("%s: populated %#x/%#x entries with shift %d", + __FUNCTION__, nr, count, pfn_shift); + + *nr_pfns = nr << pfn_shift; + + return 1; +} + static int populate_guest_memory(struct xc_dom_image *dom, xen_pfn_t base_pfn, xen_pfn_t nr_pfns) { - int rc; - xen_pfn_t allocsz, pfn; + int rc = 0; + xen_pfn_t allocsz, pfn, *extents; + + extents = calloc(1024*1024,sizeof(xen_pfn_t)); + if ( extents == NULL ) + { + DOMPRINTF("%s: Unable to allocate extent array", __FUNCTION__); + return -1; + } DOMPRINTF("%s: populating RAM @ %016"PRIx64"-%016"PRIx64" (%"PRId64"MB)", __FUNCTION__, @@ -261,21 +317,56 @@ static int populate_guest_memory(struct xc_dom_image *dom, (uint64_t)(base_pfn + nr_pfns) << XC_PAGE_SHIFT, (uint64_t)nr_pfns >> (20-XC_PAGE_SHIFT)); - for ( pfn = 0; pfn < nr_pfns; pfn++ ) - dom->p2m_host[pfn] = base_pfn + pfn; - - for ( pfn = rc = allocsz = 0; (pfn < nr_pfns) && !rc; pfn += allocsz ) + for ( pfn = 0; pfn < nr_pfns; pfn += allocsz ) { - allocsz = nr_pfns - pfn; - if ( allocsz > 1024*1024 ) - allocsz = 1024*1024; - - rc = xc_domain_populate_physmap_exact( - dom->xch, dom->guest_domid, allocsz, - 0, 0, &dom->p2m_host[pfn]); + allocsz = min_t(int, 1024*1024, nr_pfns - pfn); +#if 0 /* Enable this to exercise/debug the code which tries to realign + * to a superpage boundary, by misaligning at the start. */ + if ( pfn == 0 ) + { + allocsz = 1; + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if (rc < 0) break; + if (rc > 0) continue; + /* Failed to allocate a single page? */ + break; + } +#endif + + rc = populate_one_size(dom, PFN_512G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_1G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_2M_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc == 0 ) + { + DOMPRINTF("%s: Not enough RAM", __FUNCTION__); + errno = ENOMEM; + rc = -1; + goto out; + } } - return rc; + for ( pfn = 0; pfn < nr_pfns; pfn++ ) + dom->p2m_host[pfn] = base_pfn + pfn; + +out: + free(extents); + return rc < 0 ? rc : 0; } int arch_setup_meminit(struct xc_dom_image *dom)