From patchwork Wed Jun 30 01:34:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 468672 Delivered-To: patches@linaro.org Received: by 2002:a02:c94a:0:0:0:0:0 with SMTP id u10csp5477374jao; Tue, 29 Jun 2021 18:34:32 -0700 (PDT) X-Received: by 2002:a17:902:ee8d:b029:129:182:1bd9 with SMTP id a13-20020a170902ee8db029012901821bd9mr4883945pld.45.1625016871856; Tue, 29 Jun 2021 18:34:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625016871; cv=none; d=google.com; s=arc-20160816; b=m0vDJduZD8MMQ0jdlvmVaGHRWl1hoy4zcD3Gifz36uxtR+0kjmuXzBjPrq0JVDb02B 3Vg303hqVqPMdtvVjf1uwQCx2cElfIRA9m5FD6CWM1YOBfx5pmNxpkvpXaj9yL+1vAIc jjIONlawS3QqV2zxeev0Cgsg/1FC8kUGrzSmXg+IWxP7Q5EnlhAMGcCJO4uO52/BCyhQ DOGU9fzTtMYB+OvbrGewD8kirA9dWiVoAYb6nsW61fJNwB3d/KIjdeogMoglFxDNLJ+4 oXtFikKodeYFrGqChqTY+DuFz3rIk+xEwqmoUjCf/++S4TyhF+GULMK52j/gTYQZwtYI g8Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=UQZ07rpmRfbC0AgTi5yXuUE7gB8O5W4feOgGAG87EuE=; b=0sy79UaS1wRZIIpOfT/AJEUyYhLvjObxfE0rcmck83CvFrzy4HgZF0xb41dUsE1Bh7 rovyq/upWZ3umpDgVOaSLLwlCMGmZs4KPq1+2g8okTvFBbnMwNcs6OJjhnLzrRVk2iMs t/ak1AzzldD5b8mVA7gd3cvZrhp91kXVQFGOo4VQQd6KTQZDNgKBQJct45jJmp46JcIe f6o0RnLfevmPRtJS2vIwLWEJRTUtj5lVNokkyNSO6M+Nnlo/K631a34WleBnEKGZlFt8 BpGPm8g+iSKxmxciJZ5lGOHrsBZ50t4jdAedTeGh4Yku48bvyaO9TtONVblfziAvddEK cH7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lI4aTLFV; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id r5sor9530542pls.68.2021.06.29.18.34.31 for (Google Transport Security); Tue, 29 Jun 2021 18:34:31 -0700 (PDT) Received-SPF: pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lI4aTLFV; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UQZ07rpmRfbC0AgTi5yXuUE7gB8O5W4feOgGAG87EuE=; b=lI4aTLFVPfy3aDa/WPXf6U9K2UtE4k3b0Yguy0hbp6hS2U+BLPP62Hi80UtchaZxij Y1bkau+GAlBQr/JpELHPPrBoJJrbSJ+8rx3beyRdgIye5ef0nMxCF70YkuwquwrkIto3 r4SkX1A6lLFV/HGICKCzSKMpRpyOYCfvFTRaXmqegic/u8ocgz0YcpbzTQ64byDw/sFY 3uq2PGFX47h175CsbqvafjEiOfBudegy0Wj2fHQf26j3DgdQuasRbnz47xtKUchGad35 5GPa/0oWnbTxwuVRYo71q8xL6I3sTYoXMCyjUrkxG5H1AJyHNkzZOqs+lK/9x5goq0JL uSJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UQZ07rpmRfbC0AgTi5yXuUE7gB8O5W4feOgGAG87EuE=; b=kh258/F9RVY2QdB5NXdj2YB0q72DqCZCowx/igbjMLybxn5c+4GXN88xnfCDNJ4hl7 p2s3bNtb+J2Qn3ZYj6dFheeHFmMWjTGDzkZ2Cv+SMQbfwadMXVnI4/6nZ0L9iHf4YJFq bWwJP/5unv5Kpj6Pbt9p2HIod7b3k+uUNCT8eNvOGUiQKHcPH5QSHx4ZwDryQU3OWFfj tXwoHsxDm40p8MeP34YTpD5M63rfUiyLHV0dilgk3CePtyeqyhk0i+PmYuWPSk47Styl 7aJbsWHwX0nbD60XfhBYuAFzSPVdge6za0OrNZ8x/mTIAxOwN3NXQiQLgAJDaL1VrGA4 4zVQ== X-Gm-Message-State: AOAM532E9oUn9xMbF+kV4kOhNDeG1wmCuJC3cYpL8Felv+mG8yvNE7hO yWrqgk/FNjgaRMuoT5ws6uDZ6QUQ X-Google-Smtp-Source: ABdhPJzxuY8XMWMewO/Jp0JI4aaqWdpv5THu0uBWtq1Ey7qw6x5RoV/BZhpDv6UYzo2biHMOt34RBg== X-Received: by 2002:a17:902:ed95:b029:ee:aa46:547a with SMTP id e21-20020a170902ed95b02900eeaa46547amr30530303plj.27.1625016871442; Tue, 29 Jun 2021 18:34:31 -0700 (PDT) Return-Path: Received: from localhost.localdomain ([2601:1c2:680:1319:692:26ff:feda:3a81]) by smtp.gmail.com with ESMTPSA id g8sm20252901pja.14.2021.06.29.18.34.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Jun 2021 18:34:30 -0700 (PDT) From: John Stultz To: lkml Cc: John Stultz , Daniel Vetter , Christian Koenig , Sumit Semwal , Liam Mark , Chris Goldsworthy , Laura Abbott , Brian Starkey , Hridya Valsaraju , Suren Baghdasaryan , Sandeep Patil , Daniel Mentz , =?utf-8?q?=C3=98rjan_Eide?= , Robin Murphy , Ezequiel Garcia , Simon Ser , James Jones , linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Subject: [PATCH v9 1/5] drm: Add a sharable drm page-pool implementation Date: Wed, 30 Jun 2021 01:34:17 +0000 Message-Id: <20210630013421.735092-2-john.stultz@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210630013421.735092-1-john.stultz@linaro.org> References: <20210630013421.735092-1-john.stultz@linaro.org> MIME-Version: 1.0 This adds a shrinker controlled page pool, extracted out of the ttm_pool logic, and abstracted out a bit so it can be used by other non-ttm drivers. Cc: Daniel Vetter Cc: Christian Koenig Cc: Sumit Semwal Cc: Liam Mark Cc: Chris Goldsworthy Cc: Laura Abbott Cc: Brian Starkey Cc: Hridya Valsaraju Cc: Suren Baghdasaryan Cc: Sandeep Patil Cc: Daniel Mentz Cc: Ørjan Eide Cc: Robin Murphy Cc: Ezequiel Garcia Cc: Simon Ser Cc: James Jones Cc: linux-media@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Signed-off-by: John Stultz --- v8: * Completely rewritten from scratch, using only the ttm_pool logic so it can be dual licensed. v9: * Add Kerneldoc comments similar to tmm implementation as suggested by ChristianK * Mark some functions static as suggested by ChristianK * Fix locking issue ChristianK pointed out * Add methods to block the shrinker so users can make atomic calculations across multiple pools, as suggested by ChristianK * Fix up Kconfig dependency issue as Reported-by: kernel test robot --- drivers/gpu/drm/Kconfig | 3 + drivers/gpu/drm/Makefile | 2 + drivers/gpu/drm/page_pool.c | 297 ++++++++++++++++++++++++++++++++++++ include/drm/page_pool.h | 68 +++++++++ 4 files changed, 370 insertions(+) create mode 100644 drivers/gpu/drm/page_pool.c create mode 100644 include/drm/page_pool.h -- 2.25.1 diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index 3c16bd1afd87..52d9ba92b35e 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -177,6 +177,9 @@ config DRM_DP_CEC Note: not all adapters support this feature, and even for those that do support this they often do not hook up the CEC pin. +config DRM_PAGE_POOL + bool + config DRM_TTM tristate depends on DRM && MMU diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile index 5279db4392df..affa4ca3a08e 100644 --- a/drivers/gpu/drm/Makefile +++ b/drivers/gpu/drm/Makefile @@ -39,6 +39,8 @@ obj-$(CONFIG_DRM_VRAM_HELPER) += drm_vram_helper.o drm_ttm_helper-y := drm_gem_ttm_helper.o obj-$(CONFIG_DRM_TTM_HELPER) += drm_ttm_helper.o +drm-$(CONFIG_DRM_PAGE_POOL) += page_pool.o + drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o drm_dp_helper.o \ drm_dsc.o drm_probe_helper.o \ drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \ diff --git a/drivers/gpu/drm/page_pool.c b/drivers/gpu/drm/page_pool.c new file mode 100644 index 000000000000..c07bbe3afc32 --- /dev/null +++ b/drivers/gpu/drm/page_pool.c @@ -0,0 +1,297 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/* + * Sharable page pool implementation + * + * Extracted from drivers/gpu/drm/ttm/ttm_pool.c + * Copyright 2020 Advanced Micro Devices, Inc. + * Copyright 2021 Linaro Ltd. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: Christian König, John Stultz + */ + +#include +#include +#include +#include + +static unsigned long page_pool_size; /* max size of the pool */ + +MODULE_PARM_DESC(page_pool_size, "Number of pages in the drm page pool"); +module_param(page_pool_size, ulong, 0644); + +static atomic_long_t nr_managed_pages; + +static struct mutex shrinker_lock; +static struct list_head shrinker_list; +static struct shrinker mm_shrinker; + +/** + * drm_page_pool_set_max - Sets maximum size of all pools + * + * Sets the maximum number of pages allows in all pools. + * This can only be set once, and the first caller wins. + */ +void drm_page_pool_set_max(unsigned long max) +{ + if (!page_pool_size) + page_pool_size = max; +} + +/** + * drm_page_pool_get_max - Maximum size of all pools + * + * Return the maximum number of pages allows in all pools + */ +unsigned long drm_page_pool_get_max(void) +{ + return page_pool_size; +} + +/** + * drm_page_pool_get_total - Current size of all pools + * + * Return the number of pages in all managed pools + */ +unsigned long drm_page_pool_get_total(void) +{ + return atomic_long_read(&nr_managed_pages); +} + +/** + * drm_page_pool_get_size - Get the number of pages in the pool + * + * @pool: Pool to calculate the size of + * + * Return the number of pages in the specified pool + */ +unsigned long drm_page_pool_get_size(struct drm_page_pool *pool) +{ + unsigned long size; + + spin_lock(&pool->lock); + size = pool->page_count; + spin_unlock(&pool->lock); + return size; +} + +/** + * drm_page_pool_add - Add a page to a pool + * + * @pool: Pool to add page to + * @page: Page to be added to the pool + * + * Gives specified page into a specific pool + */ +void drm_page_pool_add(struct drm_page_pool *pool, struct page *p) +{ + unsigned int i, num_pages = 1 << pool->order; + + /* Make sure we won't grow larger then the max pool size */ + if (page_pool_size && + ((drm_page_pool_get_total()) + num_pages > page_pool_size)) { + pool->free(pool, p); + return; + } + + /* Be sure to zero pages before adding them to the pool */ + for (i = 0; i < num_pages; ++i) { + if (PageHighMem(p)) + clear_highpage(p + i); + else + clear_page(page_address(p + i)); + } + + spin_lock(&pool->lock); + list_add(&p->lru, &pool->pages); + pool->page_count += 1 << pool->order; + spin_unlock(&pool->lock); + atomic_long_add(1 << pool->order, &nr_managed_pages); + +} + +/** + * drm_page_pool_remove - Remove page from pool + * + * @pool: Pool to pull the page from + * + * Take pages from a specific pool, return NULL when nothing available + */ +struct page *drm_page_pool_remove(struct drm_page_pool *pool) +{ + struct page *p; + + spin_lock(&pool->lock); + p = list_first_entry_or_null(&pool->pages, typeof(*p), lru); + if (p) { + atomic_long_sub(1 << pool->order, &nr_managed_pages); + pool->page_count -= 1 << pool->order; + list_del(&p->lru); + } + spin_unlock(&pool->lock); + + return p; +} + +/** + * drm_page_pool_init - Initialize a pool + * + * @pool: the pool to initialize + * @order: page allocation order + * @free_page: function pointer to free the pool's pages + * + * Initialize and add a pool type to the global shrinker list + */ +void drm_page_pool_init(struct drm_page_pool *pool, unsigned int order, + void (*free_page)(struct drm_page_pool *pool, struct page *p)) +{ + pool->order = order; + spin_lock_init(&pool->lock); + INIT_LIST_HEAD(&pool->pages); + pool->free = free_page; + pool->page_count = 0; + + mutex_lock(&shrinker_lock); + list_add_tail(&pool->shrinker_list, &shrinker_list); + mutex_unlock(&shrinker_lock); +} + +/** + * drm_page_pool_fini - Cleanup a pool + * + * @pool: the pool to clean up + * + * Remove a pool_type from the global shrinker list and free all pages + */ +void drm_page_pool_fini(struct drm_page_pool *pool) +{ + struct page *p; + + mutex_lock(&shrinker_lock); + list_del(&pool->shrinker_list); + mutex_unlock(&shrinker_lock); + + while ((p = drm_page_pool_remove(pool))) + pool->free(pool, p); +} + +/** + * drm_page_pool_shrink - Shrink the drm page pool + * + * Free pages using the global shrinker list. Returns + * the number of pages free + */ +unsigned int drm_page_pool_shrink(void) +{ + struct drm_page_pool *pool; + unsigned int num_freed; + struct page *p; + + mutex_lock(&shrinker_lock); + pool = list_first_entry(&shrinker_list, typeof(*pool), shrinker_list); + + p = drm_page_pool_remove(pool); + if (p) { + pool->free(pool, p); + num_freed = 1 << pool->order; + } else { + num_freed = 0; + } + + list_move_tail(&pool->shrinker_list, &shrinker_list); + mutex_unlock(&shrinker_lock); + + return num_freed; +} + +/* As long as pages are available make sure to release at least one */ +static unsigned long drm_page_pool_shrinker_scan(struct shrinker *shrink, + struct shrink_control *sc) +{ + unsigned long num_freed = 0; + + do + num_freed += drm_page_pool_shrink(); + while (!num_freed && atomic_long_read(&nr_managed_pages)); + + return num_freed; +} + +/* Return the number of pages available or SHRINK_EMPTY if we have none */ +static unsigned long drm_page_pool_shrinker_count(struct shrinker *shrink, + struct shrink_control *sc) +{ + unsigned long num_pages = atomic_long_read(&nr_managed_pages); + + return num_pages ? num_pages : SHRINK_EMPTY; +} + +/** + * dma_page_pool_lock_shrinker - Take the shrinker lock + * + * Takes the shrinker lock, preventing the shrinker from making + * changes to the pools + */ +void dma_page_pool_lock_shrinker(void) +{ + mutex_lock(&shrinker_lock); +} + +/** + * dma_page_pool_unlock_shrinker - Release the shrinker lock + * + * Releases the shrinker lock, allowing the shrinker to free + * pages + */ +void dma_page_pool_unlock_shrinker(void) +{ + mutex_unlock(&shrinker_lock); +} + +/** + * drm_page_pool_shrinker_init - Initialize globals + * + * Initialize the global locks and lists for the shrinker. + */ +static int drm_page_pool_shrinker_init(void) +{ + mutex_init(&shrinker_lock); + INIT_LIST_HEAD(&shrinker_list); + + mm_shrinker.count_objects = drm_page_pool_shrinker_count; + mm_shrinker.scan_objects = drm_page_pool_shrinker_scan; + mm_shrinker.seeks = 1; + return register_shrinker(&mm_shrinker); +} + +/** + * drm_page_pool_shrinker_fini - Finalize globals + * + * Unregister the shrinker. + */ +static void drm_page_pool_shrinker_fini(void) +{ + unregister_shrinker(&mm_shrinker); + WARN_ON(!list_empty(&shrinker_list)); +} + +module_init(drm_page_pool_shrinker_init); +module_exit(drm_page_pool_shrinker_fini); +MODULE_LICENSE("Dual MIT/GPL"); diff --git a/include/drm/page_pool.h b/include/drm/page_pool.h new file mode 100644 index 000000000000..860cf6c92aab --- /dev/null +++ b/include/drm/page_pool.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ +/* + * Extracted from include/drm/ttm/ttm_pool.h + * Copyright 2020 Advanced Micro Devices, Inc. + * Copyright 2021 Linaro Ltd + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: Christian König, John Stultz + */ + +#ifndef _DRM_PAGE_POOL_H_ +#define _DRM_PAGE_POOL_H_ + +#include +#include +#include + +/** + * drm_page_pool - Page Pool for a certain memory type + * + * @order: the allocation order our pages have + * @pages: the list of pages in the pool + * @shrinker_list: our place on the global shrinker list + * @lock: protection of the page list + * @page_count: number of pages currently in the pool + * @free: Function pointer to free the page + */ +struct drm_page_pool { + unsigned int order; + struct list_head pages; + struct list_head shrinker_list; + spinlock_t lock; + + unsigned long page_count; + void (*free)(struct drm_page_pool *pool, struct page *p); +}; + +void drm_page_pool_set_max(unsigned long max); +unsigned long drm_page_pool_get_max(void); +unsigned long drm_page_pool_get_total(void); +unsigned int drm_page_pool_shrink(void); +unsigned long drm_page_pool_get_size(struct drm_page_pool *pool); +void drm_page_pool_add(struct drm_page_pool *pool, struct page *p); +struct page *drm_page_pool_remove(struct drm_page_pool *pool); +void dma_page_pool_lock_shrinker(void); +void dma_page_pool_unlock_shrinker(void); +void drm_page_pool_init(struct drm_page_pool *pool, unsigned int order, + void (*free_page)(struct drm_page_pool *pool, struct page *p)); +void drm_page_pool_fini(struct drm_page_pool *pool); + +#endif