From patchwork Tue Jul 1 08:26:56 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 32861 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-oa0-f71.google.com (mail-oa0-f71.google.com [209.85.219.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 53EBD203C0 for ; Tue, 1 Jul 2014 08:29:32 +0000 (UTC) Received: by mail-oa0-f71.google.com with SMTP id n16sf58989375oag.2 for ; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=DOLUqn6W8Bg03xHO3Fy8Ar8bq+npK0ypUrgQ922V37s=; b=KRCaeL0F7gm7QD4fV4fSznUbVwCxVaJxLGb1mkBFuDe8gK2M6Sm88u78jIjewpY+9o GPHbkbIYx9jzFbQ//gKPkMtK4xMSvXAkcIAe+9Bj+enjDUF5ldC9SsbvEd5GHQ0D8kh5 W6gn/ysSFjsHIWNF4iDrscX97d1kn6n/Rgzxtu5IoCkGT8pbbbIJUdGENKeakzWMOzXt 7EcpwSzoZ5i1xec/Ir1q4aNG25Ntm4HKp5oUE2HicT2N6LAoUKwZ7AUW2ULTpLihms65 9R2b+Z7hVKuVR2bQFMFmDKZzmVhkTRg9pU0DiRm0qDxTyQDy0BJMQKoCXPAFxt9K/jz+ gpKQ== X-Gm-Message-State: ALoCoQl1m5A2Gu5P25nepJUbCokHlHqN0OFbYUobWow7lWb/uUQs64SErIc8lm9t45TOCK+Emj3T X-Received: by 10.182.34.200 with SMTP id b8mr24303307obj.48.1404203371960; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.91.201 with SMTP id z67ls1986492qgd.94.gmail; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) X-Received: by 10.52.244.138 with SMTP id xg10mr62927vdc.40.1404203371823; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx.google.com with ESMTPS id m4si11149987vev.44.2014.07.01.01.29.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Jul 2014 01:29:31 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) client-ip=209.85.220.181; Received: by mail-vc0-f181.google.com with SMTP id il7so8688148vcb.12 for ; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) X-Received: by 10.58.188.199 with SMTP id gc7mr39200564vec.4.1404203371694; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp198722vcb; Tue, 1 Jul 2014 01:29:31 -0700 (PDT) X-Received: by 10.42.233.12 with SMTP id jw12mr42392864icb.12.1404203371119; Tue, 01 Jul 2014 01:29:31 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id vm8si7409746igb.35.2014.07.01.01.29.30 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 01 Jul 2014 01:29:31 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X1tPe-0000m5-NP; Tue, 01 Jul 2014 08:27:38 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X1tPd-0000ki-1Z for xen-devel@lists.xen.org; Tue, 01 Jul 2014 08:27:37 +0000 Received: from [193.109.254.147:2041] by server-1.bemta-14.messagelabs.com id 7D/5B-14887-8F072B35; Tue, 01 Jul 2014 08:27:36 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-7.tower-27.messagelabs.com!1404203251!14679715!4 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 4738 invoked from network); 1 Jul 2014 08:27:35 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-7.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 1 Jul 2014 08:27:35 -0000 X-IronPort-AV: E=Sophos;i="5.01,580,1400025600"; d="scan'208";a="148822955" Received: from accessns.citrite.net (HELO FTLPEX01CL03.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 01 Jul 2014 08:27:03 +0000 Received: from ukmail1.uk.xensource.com (10.80.16.128) by smtprelay.citrix.com (10.13.107.80) with Microsoft SMTP Server id 14.3.181.6; Tue, 1 Jul 2014 04:27:02 -0400 Received: from marilith-n13-p0.uk.xensource.com ([10.80.229.115] helo=marilith-n13.uk.xensource.com.) by ukmail1.uk.xensource.com with esmtp (Exim 4.69) (envelope-from ) id 1X1tP4-0008Hn-KG; Tue, 01 Jul 2014 09:27:02 +0100 From: Ian Campbell To: Date: Tue, 1 Jul 2014 09:26:56 +0100 Message-ID: <1404203222-30111-3-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1404202883.1829.125.camel@dagon.hellion.org.uk> References: <1404202883.1829.125.camel@dagon.hellion.org.uk> MIME-Version: 1.0 X-DLP: MIA1 Cc: Ian Jackson , julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH v4 3/8] tools: arm: allocate large pages to guests. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Tries to allocate as many large (1G or 2M) pages as it can to the guest and tries to align to the next larger size on each attempt so that we can attempt to allocate even larger pages on the next iteration (this is currently only exercised via a compile time debug option, which is left in place under a #if 0). Since ARM page tables are consistent at each level there is a common helper function which tries to allocate a levels worth of pages. The exception to this consistency is level 0 which does not support table mappings (0.5TB superpages!). Signed-off-by: Ian Campbell Acked-by: Julien Grall Cc: Ian Jackson --- v4: fixed the code which was supposed to try and realign things to the next superpage. constify some values v3: Drop outdated paragraph from commit message v2: min_t defintion moved into earlier patch drop debug_iters allocate extents array explicitly instead of putting 8M on the stack. Handle OOM, xc_populate_physmap returns 0, which needs handling for L3. --- tools/libxc/xc_dom_arm.c | 131 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 117 insertions(+), 14 deletions(-) diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c index 10f6bcb..9b31b1f 100644 --- a/tools/libxc/xc_dom_arm.c +++ b/tools/libxc/xc_dom_arm.c @@ -30,6 +30,13 @@ #define CONSOLE_PFN_OFFSET 0 #define XENSTORE_PFN_OFFSET 1 +#define LPAE_SHIFT 9 + +#define PFN_4K_SHIFT (0) +#define PFN_2M_SHIFT (PFN_4K_SHIFT+LPAE_SHIFT) +#define PFN_1G_SHIFT (PFN_2M_SHIFT+LPAE_SHIFT) +#define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT) + /* get guest IO ABI protocol */ const char *xc_domain_get_native_protocol(xc_interface *xch, uint32_t domid) @@ -249,11 +256,72 @@ static int set_mode(xc_interface *xch, domid_t domid, char *guest_type) return rc; } +/* >0: success, *nr_pfns set to number actually populated + * 0: didn't try with this pfn shift (e.g. misaligned base etc) + * <0: ERROR + */ +static int populate_one_size(struct xc_dom_image *dom, int pfn_shift, + xen_pfn_t base_pfn, xen_pfn_t *nr_pfns, + xen_pfn_t *extents) +{ + /* The mask for this level */ + const uint64_t mask = ((uint64_t)1<<(pfn_shift))-1; + /* The shift, mask and next boundary for the level above this one */ + const int next_shift = pfn_shift + LPAE_SHIFT; + const uint64_t next_mask = ((uint64_t)1< next_boundary ) + end_pfn = next_boundary; + + count = ( end_pfn - base_pfn ) >> pfn_shift; + + /* Nothing to allocate */ + if ( !count ) + return 0; + + for ( i = 0 ; i < count ; i ++ ) + extents[i] = base_pfn + (i<xch, dom->guest_domid, count, + pfn_shift, 0, extents); + if ( nr <= 0 ) return nr; + DOMPRINTF("%s: populated %#x/%#x entries with shift %d", + __FUNCTION__, nr, count, pfn_shift); + + *nr_pfns = nr << pfn_shift; + + return 1; +} + static int populate_guest_memory(struct xc_dom_image *dom, xen_pfn_t base_pfn, xen_pfn_t nr_pfns) { - int rc; - xen_pfn_t allocsz, pfn; + int rc = 0; + xen_pfn_t allocsz, pfn, *extents; + + extents = calloc(1024*1024,sizeof(xen_pfn_t)); + if ( extents == NULL ) + { + DOMPRINTF("%s: Unable to allocate extent array", __FUNCTION__); + return -1; + } DOMPRINTF("%s: populating RAM @ %016"PRIx64"-%016"PRIx64" (%"PRId64"MB)", __FUNCTION__, @@ -261,21 +329,56 @@ static int populate_guest_memory(struct xc_dom_image *dom, (uint64_t)(base_pfn + nr_pfns) << XC_PAGE_SHIFT, (uint64_t)nr_pfns >> (20-XC_PAGE_SHIFT)); - for ( pfn = 0; pfn < nr_pfns; pfn++ ) - dom->p2m_host[pfn] = base_pfn + pfn; - - for ( pfn = rc = allocsz = 0; (pfn < nr_pfns) && !rc; pfn += allocsz ) + for ( pfn = 0; pfn < nr_pfns; pfn += allocsz ) { - allocsz = nr_pfns - pfn; - if ( allocsz > 1024*1024 ) - allocsz = 1024*1024; - - rc = xc_domain_populate_physmap_exact( - dom->xch, dom->guest_domid, allocsz, - 0, 0, &dom->p2m_host[pfn]); + allocsz = min_t(int, 1024*1024, nr_pfns - pfn); +#if 0 /* Enable this to exercise/debug the code which tries to realign + * to a superpage boundary, by misaligning at the start. */ + if ( pfn == 0 ) + { + allocsz = 1; + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if (rc < 0) break; + if (rc > 0) continue; + /* Failed to allocate a single page? */ + break; + } +#endif + + rc = populate_one_size(dom, PFN_512G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_1G_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_2M_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc > 0 ) continue; + + rc = populate_one_size(dom, PFN_4K_SHIFT, + base_pfn + pfn, &allocsz, extents); + if ( rc < 0 ) break; + if ( rc == 0 ) + { + DOMPRINTF("%s: Not enough RAM", __FUNCTION__); + errno = ENOMEM; + rc = -1; + goto out; + } } - return rc; + for ( pfn = 0; pfn < nr_pfns; pfn++ ) + dom->p2m_host[pfn] = base_pfn + pfn; + +out: + free(extents); + return rc < 0 ? rc : 0; } int arch_setup_meminit(struct xc_dom_image *dom)