From patchwork Wed Jul 9 12:07:41 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 33294 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qc0-f200.google.com (mail-qc0-f200.google.com [209.85.216.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id CB51F203F4 for ; Wed, 9 Jul 2014 12:09:51 +0000 (UTC) Received: by mail-qc0-f200.google.com with SMTP id o8sf24741810qcw.3 for ; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=DOLUqn6W8Bg03xHO3Fy8Ar8bq+npK0ypUrgQ922V37s=; b=HUKzba7cnQn6QxRddtpEgAZctH17T8pM5dvYuX1h77nvlRFoA5qpVkc0wyYulFVRE+ GPtYa+mwQhLLHHnmxS8Br1w1HPi6k+UsY79VQwvdhY7vizXrz0fmXPQgGZPeFwqskVle RvDdCimqFyO9evyZ6EqgiLGuOF7FQ0f56VE3x9CSJjJOizBUisey7PBg8uB4r/VdEXW9 UASAIbe30CeUZ03mYJ0wmDQlaUKlqPePwA9McND2XT2KWrZhvsXh8TcUKJUyS4/UWsWB XUPTnEGYBCgVYgjYMif3JmrfXA610RgT9KGDQhZNWhYleDtiWYgrSP0mJhDM5QsyImqN L+bA== X-Gm-Message-State: ALoCoQkBqDZSJFt1VSSXYDMtCesvWjTMfhvU4+g/tKlF43prIcC77fIfQ4bp3YFFsDQHlNXvRCVf X-Received: by 10.236.67.103 with SMTP id i67mr10164416yhd.56.1404907791691; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.92.110 with SMTP id a101ls2703531qge.99.gmail; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) X-Received: by 10.58.65.40 with SMTP id u8mr31503ves.53.1404907791593; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) Received: from mail-ve0-f173.google.com (mail-ve0-f173.google.com [209.85.128.173]) by mx.google.com with ESMTPS id wm5si12539637vdb.83.2014.07.09.05.09.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Jul 2014 05:09:51 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.173 as permitted sender) client-ip=209.85.128.173; Received: by mail-ve0-f173.google.com with SMTP id db11so7040611veb.18 for ; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) X-Received: by 10.52.113.37 with SMTP id iv5mr37515vdb.51.1404907791487; Wed, 09 Jul 2014 05:09:51 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp41988vcb; Wed, 9 Jul 2014 05:09:51 -0700 (PDT) X-Received: by 10.50.137.71 with SMTP id qg7mr12064444igb.24.1404907790794; Wed, 09 Jul 2014 05:09:50 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id vl9si7659084igb.39.2014.07.09.05.09.49 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 09 Jul 2014 05:09:50 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X4qfG-0006Kp-E8; Wed, 09 Jul 2014 12:07:58 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X4qfC-0006Jk-R7 for xen-devel@lists.xen.org; Wed, 09 Jul 2014 12:07:55 +0000 Received: from [85.158.139.211:35045] by server-3.bemta-5.messagelabs.com id 77/03-01676-A903DB35; Wed, 09 Jul 2014 12:07:54 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-15.tower-206.messagelabs.com!1404907671!11171997!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 4645 invoked from network); 9 Jul 2014 12:07:53 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-15.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 9 Jul 2014 12:07:53 -0000 X-IronPort-AV: E=Sophos;i="5.01,631,1400025600"; d="scan'208";a="150929461" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 09 Jul 2014 12:07:52 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Wed, 9 Jul 2014 08:07:50 -0400 Received: from marilith-n13-p0.uk.xensource.com ([10.80.229.115] helo=localhost.localdomain) by ukmail1.uk.xensource.com with smtp (Exim 4.69) (envelope-from ) id 1X4qf7-00010t-D7; Wed, 09 Jul 2014 13:07:50 +0100 Received: by localhost.localdomain (sSMTP sendmail emulation); Wed, 09 Jul 2014 13:07:49 +0100 From: Ian Campbell To: Date: Wed, 9 Jul 2014 13:07:41 +0100 Message-ID: <1404907666-8594-3-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1404907608.16789.18.camel@kazak.uk.xensource.com> References: <1404907608.16789.18.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA2 Cc: Ian Jackson , julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH v5 3/8] tools: arm: allocate large pages to guests. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.173 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Tries to allocate as many large (1G or 2M) pages as it can to the guest and tries to align to the next larger size on each attempt so that we can attempt to allocate even larger pages on the next iteration (this is currently only exercised via a compile time debug option, which is left in place under a #if 0). Since ARM page tables are consistent at each level there is a common helper function which tries to allocate a levels worth of pages. The exception to this consistency is level 0 which does not support table mappings (0.5TB superpages!). Signed-off-by: Ian Campbell Acked-by: Julien Grall Cc: Ian Jackson Acked-by: Ian Jackson --- v4: fixed the code which was supposed to try and realign things to the next superpage. constify some values v3: Drop outdated paragraph from commit message v2: min_t defintion moved into earlier patch drop debug_iters allocate extents array explicitly instead of putting 8M on the stack. Handle OOM, xc_populate_physmap returns 0, which needs handling for L3. --- tools/libxc/xc_dom_arm.c | 131 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 117 insertions(+), 14 deletions(-) diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c index 10f6bcb..9b31b1f 100644 --- a/tools/libxc/xc_dom_arm.c +++ b/tools/libxc/xc_dom_arm.c @@ -30,6 +30,13 @@ #define CONSOLE_PFN_OFFSET 0 #define XENSTORE_PFN_OFFSET 1 +#define LPAE_SHIFT 9 + +#define PFN_4K_SHIFT (0) +#define PFN_2M_SHIFT (PFN_4K_SHIFT+LPAE_SHIFT) +#define PFN_1G_SHIFT (PFN_2M_SHIFT+LPAE_SHIFT) +#define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT) + /* get guest IO ABI protocol */ const char *xc_domain_get_native_protocol(xc_interface *xch, uint32_t domid) @@ -249,11 +256,72 @@ static int set_mode(xc_interface *xch, domid_t domid, char *guest_type) return rc; } +/* >0: success, *nr_pfns set to number actually populated + * 0: didn't try with this pfn shift (e.g. misaligned base etc) + * <0: ERROR + */ +static int populate_one_size(struct xc_dom_image *dom, int pfn_shift, + xen_pfn_t base_pfn, xen_pfn_t *nr_pfns, + xen_pfn_t *extents) +{ + /* The mask for this level */ + const uint64_t mask = ((uint64_t)1<<(pfn_shift))-1; + /* The shift, mask and next boundary for the level above this one */ + const int next_shift = pfn_shift + LPAE_SHIFT; + const uint64_t next_mask = ((uint64_t)1< next_boundary ) + end_pfn = next_boundary; + + count = ( end_pfn - base_pfn ) >> pfn_shift; + + /* Nothing to allocate */ + if ( !count ) + return 0; + + for ( i = 0 ; i < count ; i ++ ) + extents[i] = base_pfn + (i<xch, dom->guest_domid, count, + pfn_shift, 0, extents); + if ( nr <= 0 ) return nr; + DOMPRINTF("%s: populated %#x/%#x entries with shift %d", + __FUNCTION__, nr, count, pfn_shift); + + *nr_pfns = nr << pfn_shift; + + return 1; +} + static int populate_guest_memory(struct xc_dom_image *dom, xen_pfn_t base_pfn, xen_pfn_t nr_pfns) { - int rc; - xen_pfn_t allocsz, pfn; + int rc = 0; + xen_pfn_t allocsz, pfn, *extents; + + extents = calloc(1024*1024,sizeof(xen_pfn_t)); + if ( extents == NULL ) + { + DOMPRINTF("%s: Unable to allocate extent array", __FUNCTION__); + return -1; + } DOMPRINTF("%s: populating RAM @ %016"PRIx64"-%016"PRIx64" (%"PRId64"MB)", __FUNCTION__, @@ -261,21 +329,56 @@ static int populate_guest_memory(struct xc_dom_image *dom, (uint64_t)(base_pfn + nr_pfns) << XC_PAGE_SHIFT, (uint64_t)nr_pfns >> (20-XC_PAGE_SHIFT)); - for ( pfn = 0; pfn < nr_pfns; pfn++ ) - dom->p2m_host[pfn] = base_pfn + pfn; - - for ( pfn = rc = allocsz = 0; (pfn < nr_pfns) && !rc; pfn += allocsz ) + for ( pfn = 0; pfn < nr_pfns; pfn += allocsz ) { - allocsz = nr_pfns - pfn; - if ( allocsz > 1024*1024 ) - allocsz = 1024*1024; - - rc = xc_domain_populate_physmap_exact( - dom->xch, dom->guest_domid, allocsz, - 0, 0, &dom->p2m_host[pfn]); + allocsz = min_t(int, 1024*1024, nr_pfns - pfn); +#if 0 /* Enable this to exercise/debug the code which tries to realign + * to a superpage boundary, by misaligning at the start. */ + if ( pfn == 0 ) + { + allocsz = 1; + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if (rc < 0) break; + if (rc > 0) continue; + /* Failed to allocate a single page? */ + break; + } +#endif + + rc = populate_one_size(dom, PFN_512G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_1G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_2M_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc == 0 ) + { + DOMPRINTF("%s: Not enough RAM", __FUNCTION__); + errno = ENOMEM; + rc = -1; + goto out; + } } - return rc; + for ( pfn = 0; pfn < nr_pfns; pfn++ ) + dom->p2m_host[pfn] = base_pfn + pfn; + +out: + free(extents); + return rc < 0 ? rc : 0; } int arch_setup_meminit(struct xc_dom_image *dom)