From patchwork Thu Oct 3 00:51:40 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 20761 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 09DA4238F9 for ; Thu, 3 Oct 2013 00:52:27 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id j7sf3508800qaq.7 for ; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:x-gm-message-state:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=4HTBb58OlXy902syymXHe5/kRb63n6T7TaQuiqnWisw=; b=XQ+EeMCx1XY4zkOVQHopM0G4J0YQL0E9pARMIfrLSdQsVQ30Ycdslx43gfDnebJNkU Dkihu15AhcXu1dvALLf3OW2Z/UtWtgo4suUJYXjwp9lE5AqWJ9TojT1QJZeizGKeqBAD cJU/P0nUhGTuBk0v9JAP6SRv+vvfiFlPi0pJAY2YWsoTcxc68GLBw8sUnvxXe3TZSbPZ 8IeGMyCsaFVXxuuq4atOlmOMfOiy8tQvfAq4C0tkUuzr4zMNItqGQ9vhZXJNPx3VVMhN ixkEkOLqmjRltQXEXZBNHv3ZZX193aYrPirwGhL66DpHmlMXmFGt15e9pmuaIJPXeqb/ s9Jw== X-Received: by 10.236.69.35 with SMTP id m23mr4702809yhd.6.1380761546709; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.17.98 with SMTP id n2ls695629qed.12.gmail; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) X-Received: by 10.52.100.202 with SMTP id fa10mr3935624vdb.0.1380761546608; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx.google.com with ESMTPS id gq10si1049341vdc.4.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 17:52:26 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.212.54 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.54; Received: by mail-vb0-f54.google.com with SMTP id q14so1135783vbe.41 for ; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) X-Gm-Message-State: ALoCoQnWBs3RBw8GLKF99Sv+YXZYgAjO02h/mr52x6CA4fpNxw4nsec5AfpiMge2zzjqsuNDr6AU X-Received: by 10.58.201.73 with SMTP id jy9mr4738529vec.20.1380761546466; Wed, 02 Oct 2013 17:52:26 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp137538vcz; Wed, 2 Oct 2013 17:52:25 -0700 (PDT) X-Received: by 10.66.102.66 with SMTP id fm2mr6173980pab.94.1380761545518; Wed, 02 Oct 2013 17:52:25 -0700 (PDT) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by mx.google.com with ESMTPS id l5si3939290pay.134.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 17:52:25 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.50 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=209.85.220.50; Received: by mail-pa0-f50.google.com with SMTP id fb1so1811896pad.9 for ; Wed, 02 Oct 2013 17:52:24 -0700 (PDT) X-Received: by 10.68.134.98 with SMTP id pj2mr5505244pbb.110.1380761544839; Wed, 02 Oct 2013 17:52:24 -0700 (PDT) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id gh2sm4507018pbc.40.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 17:52:23 -0700 (PDT) From: John Stultz To: LKML Cc: Minchan Kim , Andrew Morton , Android Kernel Team , Robert Love , Mel Gorman , Hugh Dickins , Dave Hansen , Rik van Riel , Dmitry Adamushko , Dave Chinner , Neil Brown , Andrea Righi , Andrea Arcangeli , "Aneesh Kumar K.V" , Mike Hommey , Taras Glek , Dhaval Giani , Jan Kara , KOSAKI Motohiro , Michel Lespinasse , Rob Clark , "linux-mm@kvack.org" , John Stultz Subject: [PATCH 11/14] vrange: Purging vrange-anon pages from shrinker Date: Wed, 2 Oct 2013 17:51:40 -0700 Message-Id: <1380761503-14509-12-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1380761503-14509-1-git-send-email-john.stultz@linaro.org> References: <1380761503-14509-1-git-send-email-john.stultz@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.54 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Minchan Kim This patch provides the logic to discard anonymous vranges from the shrinker, by generating the page list for the volatile ranges setting the ptes volatile, and discarding the pages. Cc: Andrew Morton Cc: Android Kernel Team Cc: Robert Love Cc: Mel Gorman Cc: Hugh Dickins Cc: Dave Hansen Cc: Rik van Riel Cc: Dmitry Adamushko Cc: Dave Chinner Cc: Neil Brown Cc: Andrea Righi Cc: Andrea Arcangeli Cc: Aneesh Kumar K.V Cc: Mike Hommey Cc: Taras Glek Cc: Dhaval Giani Cc: Jan Kara Cc: KOSAKI Motohiro Cc: Michel Lespinasse Cc: Rob Clark Cc: Minchan Kim Cc: linux-mm@kvack.org Signed-off-by: Minchan Kim [jstultz: Code tweaks and commit log rewording] Signed-off-by: John Stultz --- mm/vrange.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 178 insertions(+), 1 deletion(-) diff --git a/mm/vrange.c b/mm/vrange.c index e7c5a25..c6bc32f 100644 --- a/mm/vrange.c +++ b/mm/vrange.c @@ -11,6 +11,8 @@ #include #include "internal.h" #include +#include +#include static struct kmem_cache *vrange_cachep; @@ -20,6 +22,11 @@ static struct vrange_list { struct mutex lock; } vrange_list; +struct vrange_walker { + struct vm_area_struct *vma; + struct list_head *pagelist; +}; + static inline unsigned int vrange_size(struct vrange *range) { return range->node.last + 1 - range->node.start; @@ -690,11 +697,181 @@ static struct vrange *vrange_isolate(void) return vrange; } -static unsigned int discard_vrange(struct vrange *vrange) +static unsigned int discard_vrange_pagelist(struct list_head *page_list) +{ + struct page *page; + unsigned int nr_discard = 0; + LIST_HEAD(ret_pages); + LIST_HEAD(free_pages); + + while (!list_empty(page_list)) { + int err; + page = list_entry(page_list->prev, struct page, lru); + list_del(&page->lru); + if (!trylock_page(page)) { + list_add(&page->lru, &ret_pages); + continue; + } + + /* + * discard_vpage returns unlocked page if it + * is successful + */ + err = discard_vpage(page); + if (err) { + unlock_page(page); + list_add(&page->lru, &ret_pages); + continue; + } + + ClearPageActive(page); + list_add(&page->lru, &free_pages); + dec_zone_page_state(page, NR_ISOLATED_ANON); + nr_discard++; + } + + free_hot_cold_page_list(&free_pages, 1); + list_splice(&ret_pages, page_list); + return nr_discard; +} + +static void vrange_pte_entry(pte_t pteval, unsigned long address, + unsigned ptent_size, struct mm_walk *walk) +{ + struct page *page; + struct vrange_walker *vw = walk->private; + struct vm_area_struct *vma = vw->vma; + struct list_head *pagelist = vw->pagelist; + + if (pte_none(pteval)) + return; + + if (!pte_present(pteval)) + return; + + page = vm_normal_page(vma, address, pteval); + if (unlikely(!page)) + return; + + if (!PageLRU(page) || PageLocked(page)) + return; + + /* TODO : Support THP */ + if (unlikely(PageCompound(page))) + return; + + if (isolate_lru_page(page)) + return; + + list_add(&page->lru, pagelist); + + VM_BUG_ON(page_is_file_cache(page)); + inc_zone_page_state(page, NR_ISOLATED_ANON); +} + +static int vrange_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, + struct mm_walk *walk) { + pte_t *pte; + spinlock_t *ptl; + + pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); + for (; addr != end; pte++, addr += PAGE_SIZE) + vrange_pte_entry(*pte, addr, PAGE_SIZE, walk); + pte_unmap_unlock(pte - 1, ptl); + cond_resched(); + return 0; } +static unsigned int discard_vma_pages(struct mm_struct *mm, + struct vm_area_struct *vma, unsigned long start, + unsigned long end) +{ + unsigned int ret = 0; + LIST_HEAD(pagelist); + struct vrange_walker vw; + struct mm_walk vrange_walk = { + .pmd_entry = vrange_pte_range, + .mm = vma->vm_mm, + .private = &vw, + }; + + vw.pagelist = &pagelist; + vw.vma = vma; + + walk_page_range(start, end, &vrange_walk); + + if (!list_empty(&pagelist)) + ret = discard_vrange_pagelist(&pagelist); + + putback_lru_pages(&pagelist); + return ret; +} + +/* + * vrange->owner isn't stable because caller doesn't hold vrange_lock + * so avoid touching vrange->owner. + */ +static int __discard_vrange_anon(struct mm_struct *mm, struct vrange *vrange, + unsigned int *ret_discard) +{ + struct vm_area_struct *vma; + unsigned int nr_discard = 0; + unsigned long start = vrange->node.start; + unsigned long end = vrange->node.last + 1; + int ret = 0; + + /* It prevent to destroy vma when the process exist */ + if (!atomic_inc_not_zero(&mm->mm_users)) + return ret; + + if (!down_read_trylock(&mm->mmap_sem)) { + mmput(mm); + ret = -EBUSY; + goto out; /* this vrange could be retried */ + } + + vma = find_vma(mm, start); + if (!vma || (vma->vm_start >= end)) + goto out_unlock; + + for (; vma; vma = vma->vm_next) { + if (vma->vm_start >= end) + break; + BUG_ON(vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP| + VM_HUGETLB)); + cond_resched(); + nr_discard += discard_vma_pages(mm, vma, + max_t(unsigned long, start, vma->vm_start), + min_t(unsigned long, end, vma->vm_end)); + } +out_unlock: + up_read(&mm->mmap_sem); + mmput(mm); + *ret_discard = nr_discard; +out: + return ret; +} + +static int discard_vrange(struct vrange *vrange) +{ + int ret = 0; + struct mm_struct *mm; + struct vrange_root *vroot; + unsigned int nr_discard = 0; + vroot = vrange->owner; + + /* TODO : handle VRANGE_FILE */ + if (vroot->type != VRANGE_MM) + goto out; + + mm = vroot->object; + ret = __discard_vrange_anon(mm, vrange, &nr_discard); +out: + return nr_discard; +} + static int shrink_vrange(struct shrinker *s, struct shrink_control *sc) { struct vrange *range = NULL;