From patchwork Tue Oct 1 18:38:53 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 20733 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-yh0-f70.google.com (mail-yh0-f70.google.com [209.85.213.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id D8BBF23920 for ; Tue, 1 Oct 2013 18:39:14 +0000 (UTC) Received: by mail-yh0-f70.google.com with SMTP id i72sf8812033yha.9 for ; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:x-gm-message-state:delivered-to:from:to:subject:date :message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=jEdKDm++PvxKlP76UF8ELSLkk3vyiEjv4Y1ci1XeTwQ=; b=YRO0bUhLYMFCs/plnfvEA1+VdtzYj3q/A5hoSrSPpQISSCeO91TtPK6kX+D6FaLTwr uqlHmFSFBiNb9IEVFMHg6K9QAPIE6VvnNyg7zy3QjimHgiB4DEtkDJ/mAzORTTj9jfCu L2se+L7d84SpnxLwD2KkQ5cUslq2OfbpswffJ7D/rxwhevOZL/0et5+xwoOHmpiXGM8I gZZm7BGuDrb1WwZLRSjZRSA0FD3xV0YK1R2rHsdPbf40j0Al1WX4e4bPRTkgYlfpi0dn pdZyjMTTTUl1ljFAsH8hjmvM0KnBYP53aYKxl1J1TCq0qM+wPiXhFOaQWsCqkTTM8wUp aPMQ== X-Received: by 10.58.94.103 with SMTP id db7mr629262veb.40.1380652754683; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.49.196 with SMTP id w4ls187100qen.71.gmail; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) X-Received: by 10.58.201.73 with SMTP id jy9mr13664059vec.20.1380652754514; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169]) by mx.google.com with ESMTPS id gq10si1611622vdc.69.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 11:39:14 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.169; Received: by mail-ve0-f169.google.com with SMTP id db12so5760678veb.28 for ; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) X-Gm-Message-State: ALoCoQlodFBBsQbFnkFXGZz1Qx21t7XwPZkpqc7Sa3pQ3OocEfUGL3a+02tEGee4I7l33CjTzDEP X-Received: by 10.58.205.195 with SMTP id li3mr1344878vec.31.1380652754407; Tue, 01 Oct 2013 11:39:14 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp38608vcz; Tue, 1 Oct 2013 11:39:13 -0700 (PDT) X-Received: by 10.68.26.202 with SMTP id n10mr30764064pbg.97.1380652753418; Tue, 01 Oct 2013 11:39:13 -0700 (PDT) Received: from mail-pa0-f42.google.com (mail-pa0-f42.google.com [209.85.220.42]) by mx.google.com with ESMTPS id lm10si5684594pab.324.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 11:39:13 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.42 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=209.85.220.42; Received: by mail-pa0-f42.google.com with SMTP id lj1so7894693pab.29 for ; Tue, 01 Oct 2013 11:39:13 -0700 (PDT) X-Received: by 10.68.231.71 with SMTP id te7mr2565239pbc.203.1380652752993; Tue, 01 Oct 2013 11:39:12 -0700 (PDT) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id ed3sm8282606pbc.6.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 11:39:12 -0700 (PDT) From: John Stultz To: Minchan Kim , Dhaval Giani Subject: [PATCH 09/14] vrange: Add vrange LRU list for purging Date: Tue, 1 Oct 2013 11:38:53 -0700 Message-Id: <1380652738-8000-10-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1380652738-8000-1-git-send-email-john.stultz@linaro.org> References: <1380652738-8000-1-git-send-email-john.stultz@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.169 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Minchan Kim This patch adds vrange LRU list for managing vranges to purge by something (In this implementation, I will use slab shrinker introduced by upcoming patches). This is necessary to purge vranges on swapless system because currently the VM only ages anonymous pages if the system has a swap device. In this case, because we would otherwise be duplicating the page LRUs tracking of hot/cold pages, we utilize a vrange LRU, to manage the shrinking order. Thus the shrinker will discard the entire vrange at once, and vranges are purged in the order they are marked volatile. Cc: Andrew Morton Cc: Android Kernel Team Cc: Robert Love Cc: Mel Gorman Cc: Hugh Dickins Cc: Dave Hansen Cc: Rik van Riel Cc: Dmitry Adamushko Cc: Dave Chinner Cc: Neil Brown Cc: Andrea Righi Cc: Andrea Arcangeli Cc: Aneesh Kumar K.V Cc: Mike Hommey Cc: Taras Glek Cc: Dhaval Giani Cc: Jan Kara Cc: KOSAKI Motohiro Cc: Michel Lespinasse Cc: Rob Clark Cc: Minchan Kim Cc: linux-mm@kvack.org Signed-off-by: Minchan Kim Signed-off-by: John Stultz --- include/linux/vrange_types.h | 2 ++ mm/vrange.c | 61 ++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 58 insertions(+), 5 deletions(-) diff --git a/include/linux/vrange_types.h b/include/linux/vrange_types.h index 0d48b42..d7d451c 100644 --- a/include/linux/vrange_types.h +++ b/include/linux/vrange_types.h @@ -20,6 +20,8 @@ struct vrange { struct interval_tree_node node; struct vrange_root *owner; int purged; + struct list_head lru; + atomic_t refcount; }; #endif diff --git a/mm/vrange.c b/mm/vrange.c index 296ba89..28041c1 100644 --- a/mm/vrange.c +++ b/mm/vrange.c @@ -14,8 +14,21 @@ static struct kmem_cache *vrange_cachep; +static struct vrange_list { + struct list_head list; + unsigned long size; + struct mutex lock; +} vrange_list; + +static inline unsigned int vrange_size(struct vrange *range) +{ + return range->node.last + 1 - range->node.start; +} + static int __init vrange_init(void) { + INIT_LIST_HEAD(&vrange_list.list); + mutex_init(&vrange_list.lock); vrange_cachep = KMEM_CACHE(vrange, SLAB_PANIC); return 0; } @@ -27,19 +40,56 @@ static struct vrange *__vrange_alloc(gfp_t flags) if (!vrange) return vrange; vrange->owner = NULL; + INIT_LIST_HEAD(&vrange->lru); + atomic_set(&vrange->refcount, 1); + return vrange; } static void __vrange_free(struct vrange *range) { WARN_ON(range->owner); + WARN_ON(atomic_read(&range->refcount) != 0); + WARN_ON(!list_empty(&range->lru)); + kmem_cache_free(vrange_cachep, range); } +static inline void __vrange_lru_add(struct vrange *range) +{ + mutex_lock(&vrange_list.lock); + WARN_ON(!list_empty(&range->lru)); + list_add(&range->lru, &vrange_list.list); + vrange_list.size += vrange_size(range); + mutex_unlock(&vrange_list.lock); +} + +static inline void __vrange_lru_del(struct vrange *range) +{ + mutex_lock(&vrange_list.lock); + if (!list_empty(&range->lru)) { + list_del_init(&range->lru); + vrange_list.size -= vrange_size(range); + WARN_ON(range->owner); + } + mutex_unlock(&vrange_list.lock); +} + static void __vrange_add(struct vrange *range, struct vrange_root *vroot) { range->owner = vroot; interval_tree_insert(&range->node, &vroot->v_rb); + + WARN_ON(atomic_read(&range->refcount) <= 0); + __vrange_lru_add(range); +} + +static inline void __vrange_put(struct vrange *range) +{ + if (atomic_dec_and_test(&range->refcount)) { + __vrange_lru_del(range); + __vrange_free(range); + } } static void __vrange_remove(struct vrange *range) @@ -64,6 +114,7 @@ static inline void __vrange_resize(struct vrange *range, bool purged = range->purged; __vrange_remove(range); + __vrange_lru_del(range); __vrange_set(range, start_idx, end_idx, purged); __vrange_add(range, vroot); } @@ -100,7 +151,7 @@ static int vrange_add(struct vrange_root *vroot, range = vrange_from_node(node); /* old range covers new range fully */ if (node->start <= start_idx && node->last >= end_idx) { - __vrange_free(new_range); + __vrange_put(new_range); goto out; } @@ -109,7 +160,7 @@ static int vrange_add(struct vrange_root *vroot, purged |= range->purged; __vrange_remove(range); - __vrange_free(range); + __vrange_put(range); node = next; } @@ -150,7 +201,7 @@ static int vrange_remove(struct vrange_root *vroot, if (start_idx <= node->start && end_idx >= node->last) { /* argumented range covers the range fully */ __vrange_remove(range); - __vrange_free(range); + __vrange_put(range); } else if (node->start >= start_idx) { /* * Argumented range covers over the left of the @@ -181,7 +232,7 @@ static int vrange_remove(struct vrange_root *vroot, vrange_unlock(vroot); if (!used_new) - __vrange_free(new_range); + __vrange_put(new_range); return 0; } @@ -204,7 +255,7 @@ void vrange_root_cleanup(struct vrange_root *vroot) while ((node = rb_first(&vroot->v_rb))) { range = vrange_entry(node); __vrange_remove(range); - __vrange_free(range); + __vrange_put(range); } vrange_unlock(vroot); }