From patchwork Mon Feb 3 18:16:27 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 24056 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f197.google.com (mail-ie0-f197.google.com [209.85.223.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 186BC202B2 for ; Mon, 3 Feb 2014 18:16:55 +0000 (UTC) Received: by mail-ie0-f197.google.com with SMTP id e14sf28875163iej.8 for ; Mon, 03 Feb 2014 10:16:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=svq+aW7PnuxgHwP1JAMxdAPTPq0QYokmHuJow2GbW1g=; b=jDlfOd4L3/cOoRTfukrChlh/sUO3fxpmhmoF86Whgp7Ep6re5P4408MuM0v1eXSeSt 56YzlstN2FifWPgclytPv7TvJBQpIJlQPEd5WVqc9nz8HshWoY4xwE+3S+qz/8xI9CXY Eo4YuqIJey2KxOuYFfkOE7BDBaHxAekC3kFvVdG9euKGoGD7sdfVCCriYl1gjmEQ+P/k DOL/s6kMLHZlCbQERcEhDICwVatulDXO2udqkpuShayMTAWh1376n/XPgx1qGQA8ohKy okBBc0t0rGsKqoFMdpFb6FIoj285PMG6A/4MICEFIpY01HYDCdobR/xVaJOR3hteZDpq 6Y0g== X-Gm-Message-State: ALoCoQlsDaMGmjoEdyhMDPz/4D7dDncW+g1c3i+dpQbQ6bWlT63gCAGjSYLqxAYl5WJfbctp4G4D X-Received: by 10.182.65.168 with SMTP id y8mr14397449obs.38.1391451415261; Mon, 03 Feb 2014 10:16:55 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.39.134 with SMTP id v6ls1883256qgv.36.gmail; Mon, 03 Feb 2014 10:16:55 -0800 (PST) X-Received: by 10.220.131.210 with SMTP id y18mr30010281vcs.12.1391451415147; Mon, 03 Feb 2014 10:16:55 -0800 (PST) Received: from mail-ve0-f177.google.com (mail-ve0-f177.google.com [209.85.128.177]) by mx.google.com with ESMTPS id fv9si7078092vcb.20.2014.02.03.10.16.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Feb 2014 10:16:55 -0800 (PST) Received-SPF: neutral (google.com: 209.85.128.177 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.177; Received: by mail-ve0-f177.google.com with SMTP id jz11so5078597veb.8 for ; Mon, 03 Feb 2014 10:16:55 -0800 (PST) X-Received: by 10.58.187.161 with SMTP id ft1mr176067vec.54.1391451415053; Mon, 03 Feb 2014 10:16:55 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp174193vcz; Mon, 3 Feb 2014 10:16:54 -0800 (PST) X-Received: by 10.68.194.65 with SMTP id hu1mr3844256pbc.158.1391451413980; Mon, 03 Feb 2014 10:16:53 -0800 (PST) Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51]) by mx.google.com with ESMTPS id if4si21488291pbc.106.2014.02.03.10.16.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Feb 2014 10:16:53 -0800 (PST) Received-SPF: neutral (google.com: 209.85.220.51 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=209.85.220.51; Received: by mail-pa0-f51.google.com with SMTP id ld10so7364638pab.24 for ; Mon, 03 Feb 2014 10:16:53 -0800 (PST) X-Received: by 10.68.108.130 with SMTP id hk2mr39064110pbb.16.1391451413568; Mon, 03 Feb 2014 10:16:53 -0800 (PST) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id y9sm150496099pas.10.2014.02.03.10.16.52 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Feb 2014 10:16:52 -0800 (PST) From: John Stultz To: LKML Cc: Colin Cross , Greg KH , Android Kernel Team , John Stultz Subject: [PATCH 15/16] staging: ion: Move shrinker out of heaps Date: Mon, 3 Feb 2014 10:16:27 -0800 Message-Id: <1391451388-23906-16-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1391451388-23906-1-git-send-email-john.stultz@linaro.org> References: <1391451388-23906-1-git-send-email-john.stultz@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.177 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Colin Cross Every heap that uses deferred frees is going to need a shrinker to shrink the freelist under memory pressure. Rather than requiring each heap to implement a shrinker, automatically register a shrinker if the deferred free flag is set. The system heap also needs to shrink its page pools, so add a shrink function to the heap ops that will be called after shrinking the freelists. Cc: Greg KH Cc: Colin Cross Cc: Android Kernel Team Signed-off-by: Colin Cross [jstultz: Resolved big conflicts with the shrinker api change. Also minor commit subject tweak.] Signed-off-by: John Stultz --- drivers/staging/android/ion/ion.c | 3 + drivers/staging/android/ion/ion_heap.c | 50 +++++++++++++++++ drivers/staging/android/ion/ion_page_pool.c | 8 +-- drivers/staging/android/ion/ion_priv.h | 21 ++++--- drivers/staging/android/ion/ion_system_heap.c | 80 ++++++--------------------- 5 files changed, 85 insertions(+), 77 deletions(-) diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c index 1ecbf31..e00e6e8 100644 --- a/drivers/staging/android/ion/ion.c +++ b/drivers/staging/android/ion/ion.c @@ -1503,6 +1503,9 @@ void ion_device_add_heap(struct ion_device *dev, struct ion_heap *heap) if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) ion_heap_init_deferred_free(heap); + if ((heap->flags & ION_HEAP_FLAG_DEFER_FREE) || heap->ops->shrink) + ion_heap_init_shrinker(heap); + heap->dev = dev; down_write(&dev->lock); /* use negative heap->id to reverse the priority -- when traversing diff --git a/drivers/staging/android/ion/ion_heap.c b/drivers/staging/android/ion/ion_heap.c index 296c74f..f49bdc0 100644 --- a/drivers/staging/android/ion/ion_heap.c +++ b/drivers/staging/android/ion/ion_heap.c @@ -252,6 +252,56 @@ int ion_heap_init_deferred_free(struct ion_heap *heap) return 0; } +static unsigned long ion_heap_shrink_count(struct shrinker *shrinker, + struct shrink_control *sc) +{ + struct ion_heap *heap = container_of(shrinker, struct ion_heap, + shrinker); + int total = 0; + + total = ion_heap_freelist_size(heap) / PAGE_SIZE; + if (heap->ops->shrink) + total += heap->ops->shrink(heap, sc->gfp_mask, 0); + return total; +} + +static unsigned long ion_heap_shrink_scan(struct shrinker *shrinker, + struct shrink_control *sc) +{ + struct ion_heap *heap = container_of(shrinker, struct ion_heap, + shrinker); + int freed = 0; + int to_scan = sc->nr_to_scan; + + if (to_scan == 0) + return 0; + + /* + * shrink the free list first, no point in zeroing the memory if we're + * just going to reclaim it + */ + if (heap->flags & ION_HEAP_FLAG_DEFER_FREE) + freed = ion_heap_freelist_drain(heap, to_scan * PAGE_SIZE) / + PAGE_SIZE; + + to_scan -= freed; + if (to_scan <= 0) + return freed; + + if (heap->ops->shrink) + freed += heap->ops->shrink(heap, sc->gfp_mask, to_scan); + return freed; +} + +void ion_heap_init_shrinker(struct ion_heap *heap) +{ + heap->shrinker.count_objects = ion_heap_shrink_count; + heap->shrinker.scan_objects = ion_heap_shrink_scan; + heap->shrinker.seeks = DEFAULT_SEEKS; + heap->shrinker.batch = 0; + register_shrinker(&heap->shrinker); +} + struct ion_heap *ion_heap_create(struct ion_platform_heap *heap_data) { struct ion_heap *heap = NULL; diff --git a/drivers/staging/android/ion/ion_page_pool.c b/drivers/staging/android/ion/ion_page_pool.c index fa693c2..ecb5fc3 100644 --- a/drivers/staging/android/ion/ion_page_pool.c +++ b/drivers/staging/android/ion/ion_page_pool.c @@ -130,8 +130,7 @@ static int ion_page_pool_total(struct ion_page_pool *pool, bool high) int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask, int nr_to_scan) { - int nr_freed = 0; - int i; + int freed; bool high; high = !!(gfp_mask & __GFP_HIGHMEM); @@ -139,7 +138,7 @@ int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask, if (nr_to_scan == 0) return ion_page_pool_total(pool, high); - for (i = 0; i < nr_to_scan; i++) { + for (freed = 0; freed < nr_to_scan; freed++) { struct page *page; mutex_lock(&pool->mutex); @@ -153,10 +152,9 @@ int ion_page_pool_shrink(struct ion_page_pool *pool, gfp_t gfp_mask, } mutex_unlock(&pool->mutex); ion_page_pool_free_pages(pool, page); - nr_freed += (1 << pool->order); } - return nr_freed; + return freed; } struct ion_page_pool *ion_page_pool_create(gfp_t gfp_mask, unsigned int order) diff --git a/drivers/staging/android/ion/ion_priv.h b/drivers/staging/android/ion/ion_priv.h index d986739..a8d0ed7 100644 --- a/drivers/staging/android/ion/ion_priv.h +++ b/drivers/staging/android/ion/ion_priv.h @@ -113,6 +113,7 @@ struct ion_heap_ops { void (*unmap_kernel) (struct ion_heap *heap, struct ion_buffer *buffer); int (*map_user) (struct ion_heap *mapper, struct ion_buffer *buffer, struct vm_area_struct *vma); + int (*shrink)(struct ion_heap *heap, gfp_t gfp_mask, int nr_to_scan); }; /** @@ -131,10 +132,7 @@ struct ion_heap_ops { * allocating. These are specified by platform data and * MUST be unique * @name: used for debugging - * @shrinker: a shrinker for the heap, if the heap caches system - * memory, it must define a shrinker to return it on low - * memory conditions, this includes system memory cached - * in the deferred free lists for heaps that support it + * @shrinker: a shrinker for the heap * @free_list: free list head if deferred free is used * @free_list_size size of the deferred free list in bytes * @lock: protects the free list @@ -218,6 +216,16 @@ int ion_heap_buffer_zero(struct ion_buffer *buffer); int ion_heap_pages_zero(struct page *page, size_t size, pgprot_t pgprot); /** + * ion_heap_init_shrinker + * @heap: the heap + * + * If a heap sets the ION_HEAP_FLAG_DEFER_FREE flag or defines the shrink op + * this function will be called to setup a shrinker to shrink the freelists + * and call the heap's shrink op. + */ +void ion_heap_init_shrinker(struct ion_heap *heap); + +/** * ion_heap_init_deferred_free -- initialize deferred free functionality * @heap: the heap * @@ -304,13 +312,8 @@ void ion_carveout_free(struct ion_heap *heap, ion_phys_addr_t addr, * @low_count: number of lowmem items in the pool * @high_items: list of highmem items * @low_items: list of lowmem items - * @shrinker: a shrinker for the items * @mutex: lock protecting this struct and especially the count * item list - * @alloc: function to be used to allocate pageory when the pool - * is empty - * @free: function to be used to free pageory back to the system - * when the shrinker fires * @gfp_mask: gfp_mask to use from alloc * @order: order of pages in the pool * @list: plist node for list of pools diff --git a/drivers/staging/android/ion/ion_system_heap.c b/drivers/staging/android/ion/ion_system_heap.c index 9849f39..f453d97 100644 --- a/drivers/staging/android/ion/ion_system_heap.c +++ b/drivers/staging/android/ion/ion_system_heap.c @@ -231,75 +231,34 @@ static void ion_system_heap_unmap_dma(struct ion_heap *heap, return; } -static struct ion_heap_ops system_heap_ops = { - .allocate = ion_system_heap_allocate, - .free = ion_system_heap_free, - .map_dma = ion_system_heap_map_dma, - .unmap_dma = ion_system_heap_unmap_dma, - .map_kernel = ion_heap_map_kernel, - .unmap_kernel = ion_heap_unmap_kernel, - .map_user = ion_heap_map_user, -}; - -static unsigned long ion_system_heap_shrink_count(struct shrinker *shrinker, - struct shrink_control *sc) +static int ion_system_heap_shrink(struct ion_heap *heap, gfp_t gfp_mask, + int nr_to_scan) { - struct ion_heap *heap = container_of(shrinker, struct ion_heap, - shrinker); - struct ion_system_heap *sys_heap = container_of(heap, - struct ion_system_heap, - heap); + struct ion_system_heap *sys_heap; int nr_total = 0; int i; - /* total number of items is whatever the page pools are holding - plus whatever's in the freelist */ - for (i = 0; i < num_orders; i++) { - struct ion_page_pool *pool = sys_heap->pools[i]; - nr_total += ion_page_pool_shrink(pool, sc->gfp_mask, 0); - } - nr_total += ion_heap_freelist_size(heap) / PAGE_SIZE; - return nr_total; - -} - -static unsigned long ion_system_heap_shrink_scan(struct shrinker *shrinker, - struct shrink_control *sc) -{ - - struct ion_heap *heap = container_of(shrinker, struct ion_heap, - shrinker); - struct ion_system_heap *sys_heap = container_of(heap, - struct ion_system_heap, - heap); - int nr_freed = 0; - int i; - - if (sc->nr_to_scan == 0) - goto end; - - /* shrink the free list first, no point in zeroing the memory if - we're just going to reclaim it */ - nr_freed += ion_heap_freelist_drain(heap, sc->nr_to_scan * PAGE_SIZE) / - PAGE_SIZE; - - if (nr_freed >= sc->nr_to_scan) - goto end; + sys_heap = container_of(heap, struct ion_system_heap, heap); for (i = 0; i < num_orders; i++) { struct ion_page_pool *pool = sys_heap->pools[i]; - - nr_freed += ion_page_pool_shrink(pool, sc->gfp_mask, - sc->nr_to_scan); - if (nr_freed >= sc->nr_to_scan) - break; + nr_total += ion_page_pool_shrink(pool, gfp_mask, nr_to_scan); } -end: - return nr_freed; - + return nr_total; } +static struct ion_heap_ops system_heap_ops = { + .allocate = ion_system_heap_allocate, + .free = ion_system_heap_free, + .map_dma = ion_system_heap_map_dma, + .unmap_dma = ion_system_heap_unmap_dma, + .map_kernel = ion_heap_map_kernel, + .unmap_kernel = ion_heap_unmap_kernel, + .map_user = ion_heap_map_user, + .shrink = ion_system_heap_shrink, +}; + static int ion_system_heap_debug_show(struct ion_heap *heap, struct seq_file *s, void *unused) { @@ -347,11 +306,6 @@ struct ion_heap *ion_system_heap_create(struct ion_platform_heap *unused) heap->pools[i] = pool; } - heap->heap.shrinker.scan_objects = ion_system_heap_shrink_scan; - heap->heap.shrinker.count_objects = ion_system_heap_shrink_count; - heap->heap.shrinker.seeks = DEFAULT_SEEKS; - heap->heap.shrinker.batch = 0; - register_shrinker(&heap->heap.shrinker); heap->heap.debug_show = ion_system_heap_debug_show; return &heap->heap; err_create_pool: