From patchwork Mon Nov 16 17:48:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 325823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C48ECC4742C for ; Mon, 16 Nov 2020 17:47:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 667C222265 for ; Mon, 16 Nov 2020 17:47:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="L/IqNuLZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730394AbgKPRrD (ORCPT ); Mon, 16 Nov 2020 12:47:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733124AbgKPRrC (ORCPT ); Mon, 16 Nov 2020 12:47:02 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7834AC0613CF; Mon, 16 Nov 2020 09:47:02 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id 10so14845728pfp.5; Mon, 16 Nov 2020 09:47:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CjOmYSFFYsasYX9Sq8MuBrH7p/CFaq4+YCq5TRyIrqw=; b=L/IqNuLZYXlhxtLOo3MCcdORyLWunGfw44Ksysu/z6S7b1yXCzj613kMr8Pn4MADeu amRCkuB+8BUHUajukJKbGkiF0ziUiR4lmfWK+QYn+Y5uu9LaH6v3ZDaYqES/GXAgQYHa 4kdo15Ta8FAvnYL6qn4Agh28RYSIRYYXlFB1LfrQdg8jpUaiVLwPX1XQy9vErmtYcc++ XD6ZGKFI+rq9y1fEhwQAqXUfHIHTpPpSJq93Zw9qAHqGB8UilWSRP2X+6C/LVniM+Omd bx+Jojydoh9u4rcxDaQg99OO+qCM2ZWiAURuxJZ8amQjPQxcr2FG0D2J03sDfDy9P6Xq 4Jmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CjOmYSFFYsasYX9Sq8MuBrH7p/CFaq4+YCq5TRyIrqw=; b=Fl5Hm9wST45N6qgGtuDpAS8Jf2h3V0EQ96/otQ/WEWmHLBb+plU4piQaavgr7pkvJa D8SYEcVzEJvWJ/jcWZx9RoSEci/tsyorcw47RApvBrhXgRL96bGY94DZebUeYrzE+0UB H6MR2FO1wdUfUK8RApkrPau374F9iNHefsc6bpJFNzL4u0sjDOqLsZ3afThi26oHaotR LETqh5xrS1EwXz1snUUipDB/9cKaiZC+FVwVAO8CPDCk+CC3GaMewc0i9GNVqCciwMtd n7mayeUWxFgT3I4IfsKlVEiP2VH+EqF1483lCN1hVZKQwSlHu5XQGM7XvqGbysxC1BkM OTYA== X-Gm-Message-State: AOAM531OrP36GCJMxHci6oWzravUtCODIHNjJKZDJuKWj9ODA08akuVG +DekHXPxLQHyNLfF0zb25wk= X-Google-Smtp-Source: ABdhPJz7Bv774caWlw43NtNTdo+VPH1KYBjvgktOt/3s3DivG4Duu9j7KIVWfyNPKa6YdQx0b3ZrLQ== X-Received: by 2002:a63:48b:: with SMTP id 133mr324103pge.132.1605548821982; Mon, 16 Nov 2020 09:47:01 -0800 (PST) Received: from localhost (c-73-25-156-94.hsd1.or.comcast.net. [73.25.156.94]) by smtp.gmail.com with ESMTPSA id w131sm18330274pfd.14.2020.11.16.09.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 09:47:00 -0800 (PST) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Rob Clark , Rob Clark , Sean Paul , David Airlie , Daniel Vetter , Jordan Crouse , "Kristian H. Kristensen" , linux-arm-msm@vger.kernel.org (open list:DRM DRIVER FOR MSM ADRENO GPU), freedreno@lists.freedesktop.org (open list:DRM DRIVER FOR MSM ADRENO GPU), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 1/3] drm/msm: Protect obj->active_count under obj lock Date: Mon, 16 Nov 2020 09:48:49 -0800 Message-Id: <20201116174851.878426-2-robdclark@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201116174851.878426-1-robdclark@gmail.com> References: <20201116174851.878426-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Previously we only held obj lock in the _active_get() path, and relied on atomic_dec_return() to not be racy in the _active_put() path where obj lock was not held. But this is a false sense of security. Unlike obj lifetime refcnt, where you do not expect to *increase* the refcnt after the last put (which would mean that something has gone horribly wrong with the object liveness reference counting), the active_count can increase again from zero. Racing _active_put()s and _active_get()s could leave the obj on the wrong mm list. But in the retire path, immediately after the _active_put(), the _unpin_iova() would acquire obj lock. So just move the locking earlier and rely on that to protect obj->active_count. Fixes: c5c1643cef7a ("drm/msm: Drop struct_mutex from the retire path") Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 11 ++++++----- drivers/gpu/drm/msm/msm_gem.h | 5 +++-- drivers/gpu/drm/msm/msm_gpu.c | 10 ++++++---- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index a9a834bb7794..2795288b0a95 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -770,7 +770,7 @@ void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu) WARN_ON(!msm_gem_is_locked(obj)); WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED); - if (!atomic_fetch_inc(&msm_obj->active_count)) { + if (msm_obj->active_count++ == 0) { mutex_lock(&priv->mm_lock); list_del_init(&msm_obj->mm_list); list_add_tail(&msm_obj->mm_list, &gpu->active_list); @@ -784,8 +784,9 @@ void msm_gem_active_put(struct drm_gem_object *obj) struct msm_drm_private *priv = obj->dev->dev_private; might_sleep(); + WARN_ON(!msm_gem_is_locked(obj)); - if (!atomic_dec_return(&msm_obj->active_count)) { + if (--msm_obj->active_count == 0) { mutex_lock(&priv->mm_lock); list_del_init(&msm_obj->mm_list); list_add_tail(&msm_obj->mm_list, &priv->inactive_list); @@ -936,15 +937,15 @@ void msm_gem_free_object(struct drm_gem_object *obj) struct drm_device *dev = obj->dev; struct msm_drm_private *priv = dev->dev_private; - /* object should not be on active list: */ - WARN_ON(is_active(msm_obj)); - mutex_lock(&priv->mm_lock); list_del(&msm_obj->mm_list); mutex_unlock(&priv->mm_lock); msm_gem_lock(obj); + /* object should not be on active list: */ + WARN_ON(is_active(msm_obj)); + put_iova(obj); if (obj->import_attach) { diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index d79e7019cc88..3355a48a023b 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -87,7 +87,7 @@ struct msm_gem_object { char name[32]; /* Identifier to print for the debugfs files */ - atomic_t active_count; + int active_count; }; #define to_msm_bo(x) container_of(x, struct msm_gem_object, base) @@ -185,7 +185,8 @@ msm_gem_is_locked(struct drm_gem_object *obj) static inline bool is_active(struct msm_gem_object *msm_obj) { - return atomic_read(&msm_obj->active_count); + WARN_ON(!msm_gem_is_locked(&msm_obj->base)); + return msm_obj->active_count; } static inline bool is_purgeable(struct msm_gem_object *msm_obj) diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index b597213a1890..04f7ab4d63ae 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -717,11 +717,13 @@ static void retire_submit(struct msm_gpu *gpu, struct msm_ringbuffer *ring, stats->alwayson_start, stats->alwayson_end); for (i = 0; i < submit->nr_bos; i++) { - struct msm_gem_object *msm_obj = submit->bos[i].obj; + struct drm_gem_object *obj = &submit->bos[i].obj->base; - msm_gem_active_put(&msm_obj->base); - msm_gem_unpin_iova(&msm_obj->base, submit->aspace); - drm_gem_object_put(&msm_obj->base); + msm_gem_lock(obj); + msm_gem_active_put(obj); + msm_gem_unpin_iova_locked(obj, submit->aspace); + msm_gem_unlock(obj); + drm_gem_object_put(obj); } pm_runtime_mark_last_busy(&gpu->pdev->dev); From patchwork Mon Nov 16 17:48:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 324664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DFDCC61DD8 for ; Mon, 16 Nov 2020 17:47:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CD9AC222E8 for ; Mon, 16 Nov 2020 17:47:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qXgQ97Bt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387929AbgKPRrF (ORCPT ); Mon, 16 Nov 2020 12:47:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733124AbgKPRrE (ORCPT ); Mon, 16 Nov 2020 12:47:04 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7F08C0613CF; Mon, 16 Nov 2020 09:47:04 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id i8so1124820pfk.10; Mon, 16 Nov 2020 09:47:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bcNKe/onX36d4xFW2lvtFrnwC6y2tKCelaaYJA86BPE=; b=qXgQ97Btq4MYpx4N4xkEyfDI60eUkY5lBRDKlfkSAnpcboIvF4ZtrOXruiCyX4JK8r JbK/Beia5e8ZiBYP3yzJyAmoGZSxCx5TOevJFOP7sN6cCYEf7ggvTi8ED1fBvpbh+zr4 OaXNVsjhip9qktxBkFVI2yXsC0Jpl26ZqFOeA6xik8FmUgK0xRIbDSch/4/9UQ213P0h umCzjmn4mwdHwVsSEhWvCh9VCr5f02j+WSjYn33QmgQUlymSJkZJi9xIQ3XgeNHVWQXd 4e4xoKXk+FluuZc9TY4ISATinZFzLNSCl65DtShmI6nCjnmOvbuMtPUD5V6dc4ugThTR tDXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bcNKe/onX36d4xFW2lvtFrnwC6y2tKCelaaYJA86BPE=; b=JvoIslVY7wB4LDL3sCTT/HQVsUt2BizvNgmvPfMnPzCIfqlvGVuPLfOYNqC6zGStIj m7q707GLNYPwuP4b5ugjldQS0PTLveSSTRtu2+dQX02cqLaK2qLItvuqb8C1En8OXt4t bIbFRoadoKP4+MCcTBX4P5EHfzKv7k11R9mXe6+JQEXOZitYym7QsP8AgKpT/+Xyjdiz PbK6S4KWWhejAZA10oPJufypcdbiY3yzF7PM6xOK+sE2B/ZjNKfkrflonBo6q7tRJytm VrabTapKFvJxm4KCxA551Sh7iX+0YKOeqmhmTCDf2L7vQoM54lEYiJGKSEjlkn5QMQnf 0IQw== X-Gm-Message-State: AOAM533Vt9HWHJnDxbcZf/il/Z8DLkEHonvAiaVBTqFofbb04JVxazF3 VVoEajTrm1ZX7U3QdNeUb80= X-Google-Smtp-Source: ABdhPJzTgDwy8OTh1DnkmcbyfHpJ5Y/+F55k5USXK5SBh+oT0x34EyD0BNRM4vkHDniT0kdUW/xVoA== X-Received: by 2002:a63:1d12:: with SMTP id d18mr314741pgd.314.1605548824454; Mon, 16 Nov 2020 09:47:04 -0800 (PST) Received: from localhost (c-73-25-156-94.hsd1.or.comcast.net. [73.25.156.94]) by smtp.gmail.com with ESMTPSA id b5sm18698502pfr.193.2020.11.16.09.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 09:47:03 -0800 (PST) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Rob Clark , Rob Clark , Sean Paul , David Airlie , Daniel Vetter , linux-arm-msm@vger.kernel.org (open list:DRM DRIVER FOR MSM ADRENO GPU), freedreno@lists.freedesktop.org (open list:DRM DRIVER FOR MSM ADRENO GPU), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 2/3] drm/msm/shrinker: We can vmap shrink active_list too Date: Mon, 16 Nov 2020 09:48:50 -0800 Message-Id: <20201116174851.878426-3-robdclark@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201116174851.878426-1-robdclark@gmail.com> References: <20201116174851.878426-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Just because a obj is active, if the vmap_count is zero, we can still tear down the vmap. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_shrinker.c | 47 +++++++++++++++++++------- 1 file changed, 35 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 6f4b1355725f..9d51c1eb808d 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -6,6 +6,7 @@ #include "msm_drv.h" #include "msm_gem.h" +#include "msm_gpu.h" #include "msm_gpu_trace.h" static unsigned long @@ -61,17 +62,19 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) return freed; } -static int -msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) +/* since we don't know any better, lets bail after a few + * and if necessary the shrinker will be invoked again. + * Seems better than unmapping *everything* + */ +static const int vmap_shrink_limit = 15; + +static unsigned +vmap_shrink(struct list_head *mm_list) { - struct msm_drm_private *priv = - container_of(nb, struct msm_drm_private, vmap_notifier); struct msm_gem_object *msm_obj; unsigned unmapped = 0; - mutex_lock(&priv->mm_lock); - - list_for_each_entry(msm_obj, &priv->inactive_list, mm_list) { + list_for_each_entry(msm_obj, mm_list, mm_list) { if (!msm_gem_trylock(&msm_obj->base)) continue; if (is_vunmapable(msm_obj)) { @@ -80,11 +83,31 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) } msm_gem_unlock(&msm_obj->base); - /* since we don't know any better, lets bail after a few - * and if necessary the shrinker will be invoked again. - * Seems better than unmapping *everything* - */ - if (++unmapped >= 15) + if (++unmapped >= vmap_shrink_limit) + break; + } + + return unmapped; +} + +static int +msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) +{ + struct msm_drm_private *priv = + container_of(nb, struct msm_drm_private, vmap_notifier); + struct list_head *mm_lists[] = { + &priv->inactive_list, + priv->gpu ? &priv->gpu->active_list : NULL, + NULL, + }; + unsigned idx, unmapped = 0; + + mutex_lock(&priv->mm_lock); + + for (idx = 0; mm_lists[idx]; idx++) { + unmapped += vmap_shrink(mm_lists[idx]); + + if (unmapped >= vmap_shrink_limit) break; } From patchwork Mon Nov 16 17:48:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 325821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB09EC63777 for ; Mon, 16 Nov 2020 17:47:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D44B22265 for ; Mon, 16 Nov 2020 17:47:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="a3oTavea" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388190AbgKPRrI (ORCPT ); Mon, 16 Nov 2020 12:47:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1733124AbgKPRrH (ORCPT ); Mon, 16 Nov 2020 12:47:07 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0346C0613CF; Mon, 16 Nov 2020 09:47:07 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id d17so7240246plr.5; Mon, 16 Nov 2020 09:47:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nyc6Vw30SJLc6DlOelCN1vi010c5NzXSzO7wNfG6Ujs=; b=a3oTaveaAhPJXT91daEnbooRS6IbuXh/Yrt8X71y7ZR449MeLSThllKjnkoRpgB5xp bcK4XUQMwcLKwlOc7wjT5V0tGBrbfpItcIE/nf6Tlyf2OSZTjeGWPim8HOCm0ZX/HzYU PS5rz+VJxDZo3rtQD3xVwZf73emnBPFqmPoRTNvji/kevgdGG3eTZbo9q6NxP5DA2gFT rSb+kwmnkI4ALlxYDjgPnVE18TNPmSpckstWIih37NpCnt7zuHj/RzHBY1eyUQESPAg8 hYqdJd0y3vLOoKq2NlOmyO0lI+h0MZ1ll1xsOiYt75C7nQFmpHOEanoHAnaLOvKLyYlo 1UlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nyc6Vw30SJLc6DlOelCN1vi010c5NzXSzO7wNfG6Ujs=; b=OcihYV+/TJj472DUyh/gF3q7EJ5vKFlbB+RdJt9uZGLBeRWkvUgIrBWYry5y5ME8Mc gBpDKczzvFV68wKW/GuGpgmBuSfjkxA7f4gq5fR73H++fCU8aPh6GiZEVStmlmYtNEfu mVI7aU3aXJQ0LHCryw30x2HPqkeb6HPPyX1PqLhjNaT2iywvrqUk4+BD7ydbllsj1GHv pz3gfrlroojupIHdirwT9MaqjLg2Ry1e+V5Ow/hGwifw9vlu1ATAoJkFwuUe+I3eiW5j gRj6Kr2w7/H5ZvoPlzxHXSUKaqpNfXTYj9gMZEnh42oLz2j0FRAtraingrs31lt3/ryv brYA== X-Gm-Message-State: AOAM5337YnoCAazmm1s3xN8Gp5egWlOJ8a4E4Uq6f3cZQiwfyaBJur3K 2xhJkeT5wCRruk7seJUkx28= X-Google-Smtp-Source: ABdhPJxLTAZIdhvdvXTvtkmIMdbrHi2BaUQDgIlBLeI1KlW8+jMil/iZuJzLG0R7g1xO/sFEqV5ITg== X-Received: by 2002:a17:902:6b84:b029:d8:d13d:14e with SMTP id p4-20020a1709026b84b02900d8d13d014emr14345881plk.29.1605548827237; Mon, 16 Nov 2020 09:47:07 -0800 (PST) Received: from localhost (c-73-25-156-94.hsd1.or.comcast.net. [73.25.156.94]) by smtp.gmail.com with ESMTPSA id y9sm24969pjn.24.2020.11.16.09.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Nov 2020 09:47:06 -0800 (PST) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Rob Clark , Rob Clark , Sean Paul , David Airlie , Daniel Vetter , linux-arm-msm@vger.kernel.org (open list:DRM DRIVER FOR MSM ADRENO GPU), freedreno@lists.freedesktop.org (open list:DRM DRIVER FOR MSM ADRENO GPU), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 3/3] drm/msm/shrinker: Only iterate dontneed objs Date: Mon, 16 Nov 2020 09:48:51 -0800 Message-Id: <20201116174851.878426-4-robdclark@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201116174851.878426-1-robdclark@gmail.com> References: <20201116174851.878426-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark In situations where the GPU is mostly idle, all or nearly all buffer objects will be in the inactive list. But if the system is under memory pressure (from something other than GPU), we could still get a lot of shrinker calls. Which results in traversing a list of thousands of objs and in the end finding nothing to shrink. Which isn't so efficient. Instead split the inactive_list into two lists, one inactive objs which are shrinkable, and a second one for those that are not. This way we can avoid traversing objs which we know are not shrinker candidates. v2: Fix inverted logic think-o Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_debugfs.c | 3 ++- drivers/gpu/drm/msm/msm_drv.c | 3 ++- drivers/gpu/drm/msm/msm_drv.h | 8 +++--- drivers/gpu/drm/msm/msm_gem.c | 34 ++++++++++++++++++++------ drivers/gpu/drm/msm/msm_gem_shrinker.c | 7 +++--- 5 files changed, 40 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_debugfs.c b/drivers/gpu/drm/msm/msm_debugfs.c index 64afbed89821..85ad0babc326 100644 --- a/drivers/gpu/drm/msm/msm_debugfs.c +++ b/drivers/gpu/drm/msm/msm_debugfs.c @@ -124,7 +124,8 @@ static int msm_gem_show(struct drm_device *dev, struct seq_file *m) } seq_printf(m, "Inactive Objects:\n"); - msm_gem_describe_objects(&priv->inactive_list, m); + msm_gem_describe_objects(&priv->inactive_dontneed, m); + msm_gem_describe_objects(&priv->inactive_willneed, m); mutex_unlock(&priv->mm_lock); diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 4d808769e6ed..39a54f364aa8 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -465,7 +465,8 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) priv->wq = alloc_ordered_workqueue("msm", 0); - INIT_LIST_HEAD(&priv->inactive_list); + INIT_LIST_HEAD(&priv->inactive_willneed); + INIT_LIST_HEAD(&priv->inactive_dontneed); mutex_init(&priv->mm_lock); /* Teach lockdep about lock ordering wrt. shrinker: */ diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index f869ed67b5da..ed18c5bed10f 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -175,8 +175,9 @@ struct msm_drm_private { struct msm_perf_state *perf; /* - * List of inactive GEM objects. Every bo is either in the inactive_list - * or gpu->active_list (for the gpu it is active on[1]) + * Lists of inactive GEM objects. Every bo is either in one of the + * inactive lists (depending on whether or not it is shrinkable) or + * gpu->active_list (for the gpu it is active on[1]) * * These lists are protected by mm_lock. If struct_mutex is involved, it * should be aquired prior to mm_lock. One should *not* hold mm_lock in @@ -185,7 +186,8 @@ struct msm_drm_private { * [1] if someone ever added support for the old 2d cores, there could be * more than one gpu object */ - struct list_head inactive_list; + struct list_head inactive_willneed; /* inactive + !shrinkable */ + struct list_head inactive_dontneed; /* inactive + shrinkable */ struct mutex mm_lock; struct workqueue_struct *wq; diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 2795288b0a95..318ccc2fae86 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -17,6 +17,7 @@ #include "msm_gpu.h" #include "msm_mmu.h" +static void update_inactive(struct msm_gem_object *msm_obj); static dma_addr_t physaddr(struct drm_gem_object *obj) { @@ -678,6 +679,12 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) madv = msm_obj->madv; + /* If the obj is inactive, we might need to move it + * between inactive lists + */ + if (msm_obj->active_count == 0) + update_inactive(msm_obj); + msm_gem_unlock(obj); return (madv != __MSM_MADV_PURGED); @@ -781,19 +788,31 @@ void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu) void msm_gem_active_put(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); - struct msm_drm_private *priv = obj->dev->dev_private; might_sleep(); WARN_ON(!msm_gem_is_locked(obj)); if (--msm_obj->active_count == 0) { - mutex_lock(&priv->mm_lock); - list_del_init(&msm_obj->mm_list); - list_add_tail(&msm_obj->mm_list, &priv->inactive_list); - mutex_unlock(&priv->mm_lock); + update_inactive(msm_obj); } } +static void update_inactive(struct msm_gem_object *msm_obj) +{ + struct msm_drm_private *priv = msm_obj->base.dev->dev_private; + + mutex_lock(&priv->mm_lock); + WARN_ON(msm_obj->active_count != 0); + + list_del_init(&msm_obj->mm_list); + if (msm_obj->madv == MSM_MADV_WILLNEED) + list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed); + else + list_add_tail(&msm_obj->mm_list, &priv->inactive_dontneed); + + mutex_unlock(&priv->mm_lock); +} + int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout) { bool write = !!(op & MSM_PREP_WRITE); @@ -1099,7 +1118,8 @@ static struct drm_gem_object *_msm_gem_new(struct drm_device *dev, } mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_list); + /* Initially obj is idle, obj->madv == WILLNEED: */ + list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed); mutex_unlock(&priv->mm_lock); return obj; @@ -1169,7 +1189,7 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev, msm_gem_unlock(obj); mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_list); + list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed); mutex_unlock(&priv->mm_lock); return obj; diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 9d51c1eb808d..81dfa57b6a0d 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -19,7 +19,7 @@ msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) mutex_lock(&priv->mm_lock); - list_for_each_entry(msm_obj, &priv->inactive_list, mm_list) { + list_for_each_entry(msm_obj, &priv->inactive_dontneed, mm_list) { if (!msm_gem_trylock(&msm_obj->base)) continue; if (is_purgeable(msm_obj)) @@ -42,7 +42,7 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) mutex_lock(&priv->mm_lock); - list_for_each_entry(msm_obj, &priv->inactive_list, mm_list) { + list_for_each_entry(msm_obj, &priv->inactive_dontneed, mm_list) { if (freed >= sc->nr_to_scan) break; if (!msm_gem_trylock(&msm_obj->base)) @@ -96,7 +96,8 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) struct msm_drm_private *priv = container_of(nb, struct msm_drm_private, vmap_notifier); struct list_head *mm_lists[] = { - &priv->inactive_list, + &priv->inactive_dontneed, + &priv->inactive_willneed, priv->gpu ? &priv->gpu->active_list : NULL, NULL, };