From patchwork Tue Apr 3 14:10:09 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Szyprowski X-Patchwork-Id: 7585 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 1848C199212 for ; Tue, 3 Apr 2012 14:10:45 +0000 (UTC) Received: from mail-gx0-f180.google.com (mail-gx0-f180.google.com [209.85.161.180]) by fiordland.canonical.com (Postfix) with ESMTP id C287EA1855D for ; Tue, 3 Apr 2012 14:10:44 +0000 (UTC) Received: by mail-gx0-f180.google.com with SMTP id u1so2060958ggl.11 for ; Tue, 03 Apr 2012 07:10:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:date:from :in-reply-to:to:message-id:mime-version:x-mailer:references:cc :subject:x-beenthere:x-mailman-version:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :content-type:content-transfer-encoding:sender:errors-to :x-gm-message-state; bh=3b3wMgJvkVI71tqATGEU1qMvk8UjW+HMADG/V3kpnFg=; b=em/IG7loCBnHxTvdZacIcNxC188RwXUHy5GtqwfSIlmfwvTMpjMNak3Wbc6blL8cA8 K8wd8nWlvePgoE63829HsdnVOz2LZp9JLkYjNHimBKduSUBJl4T19ru6zq28q7PCy+fI mbeHsNzHeV4e2YOIPFCbenjUplYQsWEL5r2wbifJhlKPPhXgctGxgGktYJyZuKmRjsJj YFJnMNpw06FPi5nWXkJ3U8cSliYFZqEqLzJJzTuOODVYqqoRtr4YMEaHzCw61lOt8zb2 xfLudv+DG+yl88lutXEeqbmStPXETPZJwHLW23d/nWEtRgozrV3SZmQW9bqxs/pCIRfD vmmg== Received: by 10.50.42.132 with SMTP id o4mr9151576igl.41.1333462244339; Tue, 03 Apr 2012 07:10:44 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.5.205 with SMTP id 13csp34799ibw; Tue, 3 Apr 2012 07:10:43 -0700 (PDT) Received: by 10.14.98.75 with SMTP id u51mr2340317eef.35.1333462237650; Tue, 03 Apr 2012 07:10:37 -0700 (PDT) Received: from mombin.canonical.com (mombin.canonical.com. [91.189.95.16]) by mx.google.com with ESMTP id w71si22372678weq.52.2012.04.03.07.10.33; Tue, 03 Apr 2012 07:10:37 -0700 (PDT) Received-SPF: neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) client-ip=91.189.95.16; Authentication-Results: mx.google.com; spf=neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) smtp.mail=linaro-mm-sig-bounces@lists.linaro.org Received: from localhost ([127.0.0.1] helo=mombin.canonical.com) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SF4RM-000490-CZ; Tue, 03 Apr 2012 14:10:32 +0000 Received: from mailout2.w1.samsung.com ([210.118.77.12]) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SF4RG-00046X-Vs for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 14:10:27 +0000 Received: from euspt2 (mailout2.w1.samsung.com [210.118.77.12]) by mailout2.w1.samsung.com (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTP id <0M1W004SHQ1915@mailout2.w1.samsung.com> for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 15:10:23 +0100 (BST) Received: from linux.samsung.com ([106.116.38.10]) by spt2.w1.samsung.com (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTPA id <0M1W00INIQ19T3@spt2.w1.samsung.com> for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 15:10:22 +0100 (BST) Received: from mcdsrvbld02.digital.local (unknown [106.116.37.23]) by linux.samsung.com (Postfix) with ESMTP id 4F3B5270065; Tue, 03 Apr 2012 16:16:21 +0200 (CEST) Date: Tue, 03 Apr 2012 16:10:09 +0200 From: Marek Szyprowski In-reply-to: <1333462221-3987-1-git-send-email-m.szyprowski@samsung.com> To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-media@vger.kernel.org, linux-mm@kvack.org, linaro-mm-sig@lists.linaro.org Message-id: <1333462221-3987-5-git-send-email-m.szyprowski@samsung.com> MIME-version: 1.0 X-Mailer: git-send-email 1.7.9.1 References: <1333462221-3987-1-git-send-email-m.szyprowski@samsung.com> Cc: Ohad Ben-Cohen , Daniel Walker , Russell King , Arnd Bergmann , Jonathan Corbet , Mel Gorman , Michal Nazarewicz , Dave Hansen , Jesse Barker , Kyungmin Park , Andrew Morton , Rob Clark , KAMEZAWA Hiroyuki Subject: [Linaro-mm-sig] [PATCHv24 04/16] mm: compaction: introduce isolate_freepages_range() X-BeenThere: linaro-mm-sig@lists.linaro.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Unified memory management interest group." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linaro-mm-sig-bounces@lists.linaro.org Errors-To: linaro-mm-sig-bounces@lists.linaro.org X-Gm-Message-State: ALoCoQnE+VS/KLi5bVR7gKXlxcHJJPJ9lG+fO0NgxBKgBYJeEZpYdzZeggE+B+pbfTRsm4KMxi+L From: Michal Nazarewicz This commit introduces isolate_freepages_range() function which generalises isolate_freepages_block() so that it can be used on arbitrary PFN ranges. isolate_freepages_block() is left with only minor changes. Signed-off-by: Michal Nazarewicz Signed-off-by: Marek Szyprowski Acked-by: Mel Gorman Reviewed-by: KAMEZAWA Hiroyuki Tested-by: Rob Clark Tested-by: Ohad Ben-Cohen Tested-by: Benjamin Gaignard Tested-by: Robert Nelson Tested-by: Barry Song --- mm/compaction.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 files changed, 93 insertions(+), 18 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index d9d7b35..06b198f 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -54,24 +54,20 @@ static unsigned long release_freepages(struct list_head *freelist) return count; } -/* Isolate free pages onto a private freelist. Must hold zone->lock */ -static unsigned long isolate_freepages_block(struct zone *zone, - unsigned long blockpfn, - struct list_head *freelist) +/* + * Isolate free pages onto a private freelist. Caller must hold zone->lock. + * If @strict is true, will abort returning 0 on any invalid PFNs or non-free + * pages inside of the pageblock (even though it may still end up isolating + * some pages). + */ +static unsigned long isolate_freepages_block(unsigned long blockpfn, + unsigned long end_pfn, + struct list_head *freelist, + bool strict) { - unsigned long zone_end_pfn, end_pfn; int nr_scanned = 0, total_isolated = 0; struct page *cursor; - /* Get the last PFN we should scan for free pages at */ - zone_end_pfn = zone->zone_start_pfn + zone->spanned_pages; - end_pfn = min(blockpfn + pageblock_nr_pages, zone_end_pfn); - - /* Find the first usable PFN in the block to initialse page cursor */ - for (; blockpfn < end_pfn; blockpfn++) { - if (pfn_valid_within(blockpfn)) - break; - } cursor = pfn_to_page(blockpfn); /* Isolate free pages. This assumes the block is valid */ @@ -79,15 +75,23 @@ static unsigned long isolate_freepages_block(struct zone *zone, int isolated, i; struct page *page = cursor; - if (!pfn_valid_within(blockpfn)) + if (!pfn_valid_within(blockpfn)) { + if (strict) + return 0; continue; + } nr_scanned++; - if (!PageBuddy(page)) + if (!PageBuddy(page)) { + if (strict) + return 0; continue; + } /* Found a free page, break it into order-0 pages */ isolated = split_free_page(page); + if (!isolated && strict) + return 0; total_isolated += isolated; for (i = 0; i < isolated; i++) { list_add(&page->lru, freelist); @@ -105,6 +109,73 @@ static unsigned long isolate_freepages_block(struct zone *zone, return total_isolated; } +/** + * isolate_freepages_range() - isolate free pages. + * @start_pfn: The first PFN to start isolating. + * @end_pfn: The one-past-last PFN. + * + * Non-free pages, invalid PFNs, or zone boundaries within the + * [start_pfn, end_pfn) range are considered errors, cause function to + * undo its actions and return zero. + * + * Otherwise, function returns one-past-the-last PFN of isolated page + * (which may be greater then end_pfn if end fell in a middle of + * a free page). + */ +static unsigned long +isolate_freepages_range(unsigned long start_pfn, unsigned long end_pfn) +{ + unsigned long isolated, pfn, block_end_pfn, flags; + struct zone *zone = NULL; + LIST_HEAD(freelist); + + if (pfn_valid(start_pfn)) + zone = page_zone(pfn_to_page(start_pfn)); + + for (pfn = start_pfn; pfn < end_pfn; pfn += isolated) { + if (!pfn_valid(pfn) || zone != page_zone(pfn_to_page(pfn))) + break; + + /* + * On subsequent iterations ALIGN() is actually not needed, + * but we keep it that we not to complicate the code. + */ + block_end_pfn = ALIGN(pfn + 1, pageblock_nr_pages); + block_end_pfn = min(block_end_pfn, end_pfn); + + spin_lock_irqsave(&zone->lock, flags); + isolated = isolate_freepages_block(pfn, block_end_pfn, + &freelist, true); + spin_unlock_irqrestore(&zone->lock, flags); + + /* + * In strict mode, isolate_freepages_block() returns 0 if + * there are any holes in the block (ie. invalid PFNs or + * non-free pages). + */ + if (!isolated) + break; + + /* + * If we managed to isolate pages, it is always (1 << n) * + * pageblock_nr_pages for some non-negative n. (Max order + * page may span two pageblocks). + */ + } + + /* split_free_page does not map the pages */ + map_pages(&freelist); + + if (pfn < end_pfn) { + /* Loop terminated early, cleanup. */ + release_freepages(&freelist); + return 0; + } + + /* We don't use freelists for anything. */ + return pfn; +} + /* Returns true if the page is within a block suitable for migration to */ static bool suitable_migration_target(struct page *page) { @@ -145,7 +216,7 @@ static void isolate_freepages(struct zone *zone, struct compact_control *cc) { struct page *page; - unsigned long high_pfn, low_pfn, pfn; + unsigned long high_pfn, low_pfn, pfn, zone_end_pfn, end_pfn; unsigned long flags; int nr_freepages = cc->nr_freepages; struct list_head *freelist = &cc->freepages; @@ -165,6 +236,8 @@ static void isolate_freepages(struct zone *zone, */ high_pfn = min(low_pfn, pfn); + zone_end_pfn = zone->zone_start_pfn + zone->spanned_pages; + /* * Isolate free pages until enough are available to migrate the * pages on cc->migratepages. We stop searching if the migrate @@ -201,7 +274,9 @@ static void isolate_freepages(struct zone *zone, isolated = 0; spin_lock_irqsave(&zone->lock, flags); if (suitable_migration_target(page)) { - isolated = isolate_freepages_block(zone, pfn, freelist); + end_pfn = min(pfn + pageblock_nr_pages, zone_end_pfn); + isolated = isolate_freepages_block(pfn, end_pfn, + freelist, false); nr_freepages += isolated; } spin_unlock_irqrestore(&zone->lock, flags);