From patchwork Thu Sep 26 21:16:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 830820 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77AE6186E3B; Thu, 26 Sep 2024 21:17:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385422; cv=none; b=vCoRfBEZPR/1YAcAOfibN/HFk9FlOf4z0T2hJ6lQJb+M+HOIi4bKnEFd4LXTaZseTEwQ1zd68QMGz0N1IRkeaKTfE+RWK6B+IyTJ5lLQygU7rHHICbrZrGh6lrP38SvPWO7Add1s0AMm96B/0xByf6bR3KmIPBYdddzk9PwKT1E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385422; c=relaxed/simple; bh=CepXFxjXqyEXue/YGJWfm3KBEXw1RyR6W2RTmQQLiRs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eUzWhzlIDciyuxWq8HpEGWP6Zq/0I/M+80qj9zz5uPKuRs62PV1OvJ687yf4FaSnj2zNT5i+uTzMaUiMGd81ePC0m4rMRYpZ/IrCjevhkq99wdNG4yXUHmH/3zp3GLqzW+CboOR2yj1FohjgvrQ3wLh3gpxyp5A/7gU56Cywdkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nUr2h4LS; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nUr2h4LS" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-378c16a4d3eso1423166f8f.1; Thu, 26 Sep 2024 14:17:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385419; x=1727990219; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ubh2LO1iYir2G9XehyXS+a8IHNhnU1KljQqeBF6FN+A=; b=nUr2h4LSUTeZ/DRfYhgQ3HFTKvSawnrLlBUjInhKj4hcM1qCS44jk4BqylHsZnWuEt qYZRk62x/XHUWbZx6fT5WK3Ys737QK5pqWMol6k3gfe0uXdoq32SFjFk0wXsuKaJLry+ 2VUPErPx9QGVzx4Celxo1FamAYov9l1CIC/JXty+lCBrpZsCOkSVyccTeO+4BNKq6324 MWasvpAo/zrzFFScI0Qlx6fYyZjd+dwZY4+r0dY1vCVz7lfPqv/A5JRUpBPoZYLPSt8A Yl28fVlaasDfwn1rTSFSCluVvB40XfSf0exgf+5+ceCrHIIu39NT/Ai6ruYM4FCuTiZG nByQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385419; x=1727990219; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ubh2LO1iYir2G9XehyXS+a8IHNhnU1KljQqeBF6FN+A=; b=IUfoeA0CgXDxeWWw9TSUdvy7OUm+c2EdCyRKGuPz1380tBdFdjr6rkMyiAVtrblkBP iO2+49XcrzXmiXmrob75g82wKWP6tDwvzFA4xm+sQ05wZcKy6+VuhzkHw20p7gNK1H1p MJ17PsQjJXb4shN7rM6R4W/bUMFCFfQcvU78zdf/GTpJ13D5Ol5uWs9zCRdMEBnvTDwn S0YB21vH+mWhakWO99CleKiZzHw23xsYF5iN0XcqrK+rlSwhPHpXuWAmzzsMY/ZRlRBZ VwUzitQuSGidHn8VZa83ipKj+o3hnKWsyFYWO3ujYj81vOi6vTFKhhVNVLiinwYJz5OL h1jg== X-Forwarded-Encrypted: i=1; AJvYcCUKNcmFsdr3qLbClzLT9kbRF7loAuil6MOFyzutua66v6Uo0LUso7IMkekBybDDhBbNUpxpgO1t94fVom0x@vger.kernel.org, AJvYcCW1Fqj3TQ0Os+dCxVe/bykL2YGJfvyWfAWoifdBKPUXdoti+FPqH2xIbv5TYZooYW9ceVaZCNnJY00=@vger.kernel.org X-Gm-Message-State: AOJu0YxzcK2KpwmvfdIJ420icaBHZf3MI25z/d6DL+s479OUYvSrZ0h2 9XDMfzYyh1eV/1Mn8tPk6RJmAYJK02X9oYfdxoSLawyhyWZfc7KR X-Google-Smtp-Source: AGHT+IH24YkdNigtxck4YOEZcQBP81NdikRqENUdGntnRMH26co6OtJGGXlqP4gt1P0dhUB+82atfw== X-Received: by 2002:a5d:6148:0:b0:37c:d18e:6463 with SMTP id ffacd0b85a97d-37cd5b04b6bmr602848f8f.47.1727385418639; Thu, 26 Sep 2024 14:16:58 -0700 (PDT) Received: from [192.168.1.17] (a-lu6-5.tin.it. [212.216.221.196]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c297b7d6sm38862166b.162.2024.09.26.14.16.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:16:58 -0700 (PDT) From: Antonino Maniscalco Date: Thu, 26 Sep 2024 23:16:44 +0200 Subject: [PATCH v6 02/11] drm/msm/a6xx: Track current_ctx_seqno per ring Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240926-preemption-a750-t-v6-2-7b6e1ef3648f@gmail.com> References: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> In-Reply-To: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727385413; l=8423; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=CepXFxjXqyEXue/YGJWfm3KBEXw1RyR6W2RTmQQLiRs=; b=KQ5H8zsM+CGmv8IK1+y+nHbbKRxxUrMg/Iezow2Af1oIpBNa4fz2pq/6yI6sn4H17vzL5ghtn +I2+4QEAZqYAVB1xYzohp5ZWEJw+kBRw4iPdFca0gxPrY/vlRnZYssg X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= With preemption it is not enough to track the current_ctx_seqno globally as execution might switch between rings. This is especially problematic when current_ctx_seqno is used to determine whether a page table switch is necessary as it might lead to security bugs. Track current context per ring. Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +++--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 10 ++++++---- drivers/gpu/drm/msm/msm_gpu.c | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 11 ----------- drivers/gpu/drm/msm/msm_ringbuffer.h | 10 ++++++++++ 8 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c index 0dc255ddf5ceba87090f64d5cb9f078b61104063..379a3d346c300f3ccc9e9bd08ef2a32aa3e24ceb 100644 --- a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c @@ -22,7 +22,7 @@ static void a2xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c index 5273dc8498381ce09e878894f4eb56263900be39..945fe64f835cc6094f1880ea20f20584de74a464 100644 --- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c @@ -40,7 +40,7 @@ static void a3xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c index 8b4cdf95f4453bb76e7efb93d86080ef678c9f68..50c490b492f08a1a7ebfe33b2f206cafd91a84ba 100644 --- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c @@ -34,7 +34,7 @@ static void a4xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index c0b5373e90d7139caa023aec6f272545456acb0a..80b441fe8e3a823c5bd1f74cd8c7bb418d0674fb 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -75,7 +75,7 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit *submit case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -129,7 +129,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) unsigned int i, ibs = 0; if (IS_ENABLED(CONFIG_DRM_MSM_GPU_SUDO) && submit->in_rb) { - gpu->cur_ctx_seqno = 0; + ring->cur_ctx_seqno = 0; a5xx_submit_in_rb(gpu, submit); return; } @@ -164,7 +164,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 32a4faa93d7f072ea6b8d949f4dc9d2a58cec6b9..6e065500b64d6d95599d89c33e6703c92f210047 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -109,7 +109,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, u32 asid; u64 memptr = rbmemptr(ring, ttbr0); - if (ctx->seqno == a6xx_gpu->base.base.cur_ctx_seqno) + if (ctx->seqno == ring->cur_ctx_seqno) return; if (msm_iommu_pagetable_params(ctx->aspace->mmu, &ttbr, &asid)) @@ -219,7 +219,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -305,7 +305,7 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -843,6 +843,7 @@ static int hw_init(struct msm_gpu *gpu) struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); struct a6xx_gmu *gmu = &a6xx_gpu->gmu; u64 gmem_range_min; + unsigned int i; int ret; if (!adreno_has_gmu_wrapper(adreno_gpu)) { @@ -1138,7 +1139,8 @@ static int hw_init(struct msm_gpu *gpu) /* Always come up on rb 0 */ a6xx_gpu->cur_ring = gpu->rb[0]; - gpu->cur_ctx_seqno = 0; + for (i = 0; i < gpu->nr_rings; i++) + gpu->rb[i]->cur_ctx_seqno = 0; /* Enable the SQE_to start the CP engine */ gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 3666b42b4ecd7f91b24302d7f229eeefdc3c39b7..c063b3896dc1c193e41b8fc380a91a9076376811 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -783,7 +783,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) mutex_unlock(&gpu->active_lock); gpu->funcs->submit(gpu, submit); - gpu->cur_ctx_seqno = submit->queue->ctx->seqno; + submit->ring->cur_ctx_seqno = submit->queue->ctx->seqno; pm_runtime_put(&gpu->pdev->dev); hangcheck_timer_reset(gpu); diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 1f02bb9956be2720a2760646ccdf92f8bead7dd0..7cabc8480d7c5461ab8d8726fcc21690cbaf7366 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -193,17 +193,6 @@ struct msm_gpu { */ refcount_t sysprof_active; - /** - * cur_ctx_seqno: - * - * The ctx->seqno value of the last context to submit rendering, - * and the one with current pgtables installed (for generations - * that support per-context pgtables). Tracked by seqno rather - * than pointer value to avoid dangling pointers, and cases where - * a ctx can be freed and a new one created with the same address. - */ - int cur_ctx_seqno; - /** * lock: * diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 40791b2ade46ef0e16e2a4088291a575d3be9e82..174f83137a49940ec80b1fbf548e214fa3c32784 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -100,6 +100,16 @@ struct msm_ringbuffer { * preemption. Can be aquired from irq context. */ spinlock_t preempt_lock; + + /** + * cur_ctx_seqno: + * + * The ctx->seqno value of the last context to submit to this ring + * Tracked by seqno rather than pointer value to avoid dangling + * pointers, and cases where a ctx can be freed and a new one created + * with the same address. + */ + int cur_ctx_seqno; }; struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id, From patchwork Thu Sep 26 21:16:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 830819 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD46518660A; Thu, 26 Sep 2024 21:17:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385425; cv=none; b=NO+ptzexJQLKAw3+GcDeJypoprfh+oNfCxUQCXH7Ec8XOkW8i1XG6ZHCOagwMAHhiaOfqiw90csn/SQL4OdlCL8RHIL8ajdruKDZNdnXn+6n2BpUX7kVBKpYPyq3TYqcxwDuD7l0be6gX9hi7RMp0LkuB59KxpQDU36nvzUiFUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385425; c=relaxed/simple; bh=HrcVfm6GphW+dkH2/eI9roTkaCaCbkN7M4JC5UFjlfM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OABK0x0i9Qu4yzRFyS0GRTrq3a7aZZGqI2iOHY4KFHUs3kKfmI9P2sToOghxBE0pnoms0l0b0Pi9t9iKhI5fwH4ep/KD+qv6/rTFWgEq8ahX8EWFkxuj5DpcymR/9R9vzvfIJri+m203PGFo1bPnYTG7cPECH2V+rxzz2Pqv9XY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZS3fqjA8; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZS3fqjA8" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a86e9db75b9so224103566b.1; Thu, 26 Sep 2024 14:17:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385422; x=1727990222; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=DtlKt6idkJAcSVx+NWk8duOZfO6okh0MSeZqXt1n944=; b=ZS3fqjA8UdNlWXfcZSgHCH06eKmRLaFUSE49jAg6IELxP7AHRSX0Mg8iFlb5/kLcw2 8IX5wX7FAPfHYJ7bC8nIHAD8I3+KVlzkmOXRTxvE0bP1B4rCy2j1k+3Hi+9imw9MDjHm w1P3XaHJhnUAEmJlHLavwSvVoG5Cfcj7Z8KIEfSKTnWZCycbPSTlVT30uKBHXGx+3ZgV Gm6KVimj0yzin9G9Xzd0AtprYLHegNiJEhJGeT0cruSeq+ocLBzvRmCs++Yn16CIokCs uR/RF/OHAgYThhUMRdgIDx45PUEQfERcIrz4jD3/9hbH45xk6vynLSvpxYrWD/+j5lWe qx6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385422; x=1727990222; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DtlKt6idkJAcSVx+NWk8duOZfO6okh0MSeZqXt1n944=; b=ApDLy+vI16PRIq38UKBh61dTmb/DjSywZyZr6tPaGgQWDOD9TmCLLEaaxJridxZKVi l0vjXcm9O0s7w646yGZebCqEw0tVmHaqVc3DCiYPx/dDiHupb0dN7d/H3QlVQyfcZPdy 6E5UrkLlhaiRZZQiQClDpW4nam0O/N62kn15Ss/z9JFZl6/z6QKRWfElaLy5rkLQ3imJ pMcn5q2mxGqpJhJjUGOI8P4QGwhIs64RXNM5ZuAHBxIymqf3RirzDNjJwKtYAYQc8pn4 80Q+arV9OVbEGlkxcqjEw6zUaUGROZyGjzhS5Ff1VFrq70kz2jWkjj/jqhsmzFZtIkZq gWqQ== X-Forwarded-Encrypted: i=1; AJvYcCXYNM8ZZmDR+AXAgcAv2+ZO2Iqilg5iJcWhI349WDO23xH4EiscshP1FhKj6XWmo2H7HTwmyFBNZmc=@vger.kernel.org, AJvYcCXw/gZ6Go3icsgty2Ca9wGNowi2F+MBHaZbzWnlByz0Lj3CZACX+qp7yGEAizNT/0jVsJDjieudz4lqloly@vger.kernel.org X-Gm-Message-State: AOJu0YxYzUxzd59+iWjo3zL/NIaUMh/ou+8arV/gdCRPdWWaqQ/adgmD eH/w75tV5XUi3btyJXAeh4HIMO4xk/qPEJcy57LrKgavtiVHbxCy X-Google-Smtp-Source: AGHT+IFeG3xQe2NUPb0TWY2IpSWyUWxXtqMEEggeGF8jV+8iSTppPXNsjqibA2NC+C8f+F4+2Kaaqg== X-Received: by 2002:a17:906:4788:b0:a8d:2ab2:c99e with SMTP id a640c23a62f3a-a93c4aa6a5bmr61656266b.55.1727385421887; Thu, 26 Sep 2024 14:17:01 -0700 (PDT) Received: from [192.168.1.17] (a-lu6-5.tin.it. [212.216.221.196]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c297b7d6sm38862166b.162.2024.09.26.14.17.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:17:01 -0700 (PDT) From: Antonino Maniscalco Date: Thu, 26 Sep 2024 23:16:46 +0200 Subject: [PATCH v6 04/11] drm/msm: Add CONTEXT_SWITCH_CNTL bitfields Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240926-preemption-a750-t-v6-4-7b6e1ef3648f@gmail.com> References: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> In-Reply-To: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727385413; l=1479; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=HrcVfm6GphW+dkH2/eI9roTkaCaCbkN7M4JC5UFjlfM=; b=ec7LoUmEJAjOmebgibx3u60S+UsCHv4BwpHYl/Nc/7q9K7NGp9GiEBLqx/gvyNu6Wv+J7Suz2 nQnm9jESmN5Cejmax/JV9YxHfosajsFLUgzIOl368HKPOvv+/ryayTH X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= Add missing bitfields to CONTEXT_SWITCH_CNTL in a6xx.xml. Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/registers/adreno/a6xx.xml b/drivers/gpu/drm/msm/registers/adreno/a6xx.xml index 2dfe6913ab4f52449b76c2f75b2d101c08115d16..fd31d1d7a11eef7f38dcc2975dc1034f6b7a2e41 100644 --- a/drivers/gpu/drm/msm/registers/adreno/a6xx.xml +++ b/drivers/gpu/drm/msm/registers/adreno/a6xx.xml @@ -1337,7 +1337,12 @@ to upconvert to 32b float internally? - + + + + + + From patchwork Thu Sep 26 21:16:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 830818 Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E1C94188000; Thu, 26 Sep 2024 21:17:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385428; cv=none; b=VO/T5GZmJSy6YTZC25CPnkcjg5up+FIYzPPQHbaLfWYEfaMeJCyrTyguPInYxTCPiC41of7pzG3UIsZCoGgIzFPde83ve9lVeGqWPbSh2U660j7yKMgmgz524BVRfkmuQLH+EmO7ALLHXxtQ//dXJ+F2KZWFs4BOUG22EijYiAE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385428; c=relaxed/simple; bh=BJUHjq0PmqLEqZfzgWW8mronKYe/VvIZ9f1IA7KYCEk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g0os9EbVzftV0EoJkAfowNf9JL6kfYwEf9dsvvPACy4Y1tVKN0N34Kiq54ZdlrU9FrfKQGOgKPHGWWkixK2dmNspeIm/uiMeDRiRmltwswrETuEsHOMvrJ8+dz977x4J7aec56RMBMJrYhAcXtqbQKDcuums3uZGvX/o0uOLO6I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=G8ul7BLD; arc=none smtp.client-ip=209.85.167.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G8ul7BLD" Received: by mail-lf1-f50.google.com with SMTP id 2adb3069b0e04-5356aa9a0afso2322031e87.2; Thu, 26 Sep 2024 14:17:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385424; x=1727990224; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=tciLVJ5m0bCsIVx+iFZ+9ls4/+uLx8AgJgsjhcTPMtU=; b=G8ul7BLDFw2A+XXYEsyBrmrHV9i6xTwSE+rg4AAHpcEJlDNyPl0ZhDzTjQc6B5eEOg vqwIl2zNK5jSj0kX1vqmpPTcDU4v4n2IzTdacL3FrKLkehEusZjvOhunguBEOMFoX9WB b+GUuB2mMI+eeYwcGhrVpPSSMkH2MI0nRCsG4fFjGMGzpoIw5gFnnpE27a9e95TMD8ja J0GuNMbx3V3nkkUXTi3bfKBurqcXyV5TeBDf7wiydYdMU8KPAXl0OWfsmRo8ObFrDdwh y7c+NPWwwSoJC7KVKETT16845tViEeDWg75qO3uYJOPMDxJu1dww9UplfK11MnG1dkTA XRWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385424; x=1727990224; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tciLVJ5m0bCsIVx+iFZ+9ls4/+uLx8AgJgsjhcTPMtU=; b=rmbKJ7YxYaezEFTOvSC3WepSTZC5/IR2JpJ8XgQl5oU1YYHDWf4Fkikep2hRnnKTpx zYfS7ra6i2khu/G9aZcu7dL94ImpvCqrZ+DuQM3xQPJyyiyNMufHkAPdPF3ZX1fKM3bX +YgS/1413/aQZvgmPWo0g2dLVzP/ggz68t4YFIz1Gu85VtYS0T8Vu8/TlrHgy6QZpISJ +E3gu3qKQ1NMZWeYixpWgTWIYi94ZjGrDtrnCTNqFZIT8azIx1ana8Q0m3dSji1a56Nj Tn0MOEvSRyDAxjRyJkL3Xx/CEw2wxARd4czxFzCEqDe2Jb/g1zHODy5CrrZ5oNjo/Dvh l6Ug== X-Forwarded-Encrypted: i=1; AJvYcCWBWpK4dku1VvPaF3YzGrF6etdfbqsT745m5vCS3NrdctXSjFO2EnEII1FtTU6UJsa2/gX9Lh3lcbw=@vger.kernel.org, AJvYcCXaJC9Z8eCoDYtUQBRWn1oN1tot5BBWbIXsg0KKHmgIf1KKZWlhE0iC6U7m1EsQtxJ3dIBgh4XKEwun8Kj+@vger.kernel.org X-Gm-Message-State: AOJu0Yx9aTTQRq2ufQDehKElC/wH+jodZjCjxHpuWA+7nclxOurG9pf6 8u72xVdOJ0MUMXRhl1q9Qr/asPPZpI9dwMEmvSa6QqrAJ+xGnGaJ X-Google-Smtp-Source: AGHT+IG0JVVytZkV30BIJ7O0aORweLQf3cp0Aj93r8cIkf1Ckl7fL9+IC39tOUN5v4l0E9mQA83mzA== X-Received: by 2002:ac2:4f01:0:b0:533:71f:3a3d with SMTP id 2adb3069b0e04-5389fc4690amr882031e87.24.1727385423619; Thu, 26 Sep 2024 14:17:03 -0700 (PDT) Received: from [192.168.1.17] (a-lu6-5.tin.it. [212.216.221.196]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c297b7d6sm38862166b.162.2024.09.26.14.17.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:17:03 -0700 (PDT) From: Antonino Maniscalco Date: Thu, 26 Sep 2024 23:16:47 +0200 Subject: [PATCH v6 05/11] drm/msm/a6xx: Implement preemption for a7xx targets Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240926-preemption-a750-t-v6-5-7b6e1ef3648f@gmail.com> References: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> In-Reply-To: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Akhil P Oommen , Neil Armstrong , Sharat Masetty X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727385413; l=34976; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=BJUHjq0PmqLEqZfzgWW8mronKYe/VvIZ9f1IA7KYCEk=; b=kvzCbyLjTIxJZEBcZcQq09T2My6o+doJD51m65BEjwE9hVnAzb5FX5Qb2G+Q9PxwNyTMfXPI2 911F/EMNi/1DjE2AEt/j0y71sCd6m1AvCNg5SX1v91vmF3RgZ1PvsWf X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= This patch implements preemption feature for A6xx targets, this allows the GPU to switch to a higher priority ringbuffer if one is ready. A6XX hardware as such supports multiple levels of preemption granularities, ranging from coarse grained(ringbuffer level) to a more fine grained such as draw-call level or a bin boundary level preemption. This patch enables the basic preemption level, with more fine grained preemption support to follow. Reviewed-by: Akhil P Oommen Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Sharat Masetty Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/Makefile | 1 + drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 283 +++++++++++++++++++++- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 168 +++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 377 ++++++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_ringbuffer.h | 7 + 5 files changed, 825 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index f5e2838c6a76505b353f83c9fe9c997f1c282701..32e915109a59dda96ed76ddd2b4f57bb225f4572 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -23,6 +23,7 @@ adreno-y := \ adreno/a6xx_gpu.o \ adreno/a6xx_gmu.o \ adreno/a6xx_hfi.o \ + adreno/a6xx_preempt.o \ adreno-$(CONFIG_DEBUG_FS) += adreno/a5xx_debugfs.o \ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 6e065500b64d6d95599d89c33e6703c92f210047..355a3e210335d60a5bed0ee287912271c353402a 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -16,6 +16,84 @@ #define GPU_PAS_ID 13 +/* IFPC & Preemption static powerup restore list */ +static const uint32_t a7xx_pwrup_reglist[] = { + REG_A6XX_UCHE_TRAP_BASE, + REG_A6XX_UCHE_TRAP_BASE + 1, + REG_A6XX_UCHE_WRITE_THRU_BASE, + REG_A6XX_UCHE_WRITE_THRU_BASE + 1, + REG_A6XX_UCHE_GMEM_RANGE_MIN, + REG_A6XX_UCHE_GMEM_RANGE_MIN + 1, + REG_A6XX_UCHE_GMEM_RANGE_MAX, + REG_A6XX_UCHE_GMEM_RANGE_MAX + 1, + REG_A6XX_UCHE_CACHE_WAYS, + REG_A6XX_UCHE_MODE_CNTL, + REG_A6XX_RB_NC_MODE_CNTL, + REG_A6XX_RB_CMP_DBG_ECO_CNTL, + REG_A7XX_GRAS_NC_MODE_CNTL, + REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE, + REG_A6XX_UCHE_GBIF_GX_CONFIG, + REG_A6XX_UCHE_CLIENT_PF, + REG_A6XX_TPL1_DBG_ECO_CNTL1, +}; + +static const uint32_t a7xx_ifpc_pwrup_reglist[] = { + REG_A6XX_TPL1_NC_MODE_CNTL, + REG_A6XX_SP_NC_MODE_CNTL, + REG_A6XX_CP_DBG_ECO_CNTL, + REG_A6XX_CP_PROTECT_CNTL, + REG_A6XX_CP_PROTECT(0), + REG_A6XX_CP_PROTECT(1), + REG_A6XX_CP_PROTECT(2), + REG_A6XX_CP_PROTECT(3), + REG_A6XX_CP_PROTECT(4), + REG_A6XX_CP_PROTECT(5), + REG_A6XX_CP_PROTECT(6), + REG_A6XX_CP_PROTECT(7), + REG_A6XX_CP_PROTECT(8), + REG_A6XX_CP_PROTECT(9), + REG_A6XX_CP_PROTECT(10), + REG_A6XX_CP_PROTECT(11), + REG_A6XX_CP_PROTECT(12), + REG_A6XX_CP_PROTECT(13), + REG_A6XX_CP_PROTECT(14), + REG_A6XX_CP_PROTECT(15), + REG_A6XX_CP_PROTECT(16), + REG_A6XX_CP_PROTECT(17), + REG_A6XX_CP_PROTECT(18), + REG_A6XX_CP_PROTECT(19), + REG_A6XX_CP_PROTECT(20), + REG_A6XX_CP_PROTECT(21), + REG_A6XX_CP_PROTECT(22), + REG_A6XX_CP_PROTECT(23), + REG_A6XX_CP_PROTECT(24), + REG_A6XX_CP_PROTECT(25), + REG_A6XX_CP_PROTECT(26), + REG_A6XX_CP_PROTECT(27), + REG_A6XX_CP_PROTECT(28), + REG_A6XX_CP_PROTECT(29), + REG_A6XX_CP_PROTECT(30), + REG_A6XX_CP_PROTECT(31), + REG_A6XX_CP_PROTECT(32), + REG_A6XX_CP_PROTECT(33), + REG_A6XX_CP_PROTECT(34), + REG_A6XX_CP_PROTECT(35), + REG_A6XX_CP_PROTECT(36), + REG_A6XX_CP_PROTECT(37), + REG_A6XX_CP_PROTECT(38), + REG_A6XX_CP_PROTECT(39), + REG_A6XX_CP_PROTECT(40), + REG_A6XX_CP_PROTECT(41), + REG_A6XX_CP_PROTECT(42), + REG_A6XX_CP_PROTECT(43), + REG_A6XX_CP_PROTECT(44), + REG_A6XX_CP_PROTECT(45), + REG_A6XX_CP_PROTECT(46), + REG_A6XX_CP_PROTECT(47), + REG_A6XX_CP_AHB_CNTL, +}; + + static inline bool _a6xx_check_idle(struct msm_gpu *gpu) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); @@ -68,6 +146,8 @@ static void update_shadow_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring) static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); uint32_t wptr; unsigned long flags; @@ -81,12 +161,17 @@ static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) /* Make sure to wrap wptr if we need to */ wptr = get_wptr(ring); - spin_unlock_irqrestore(&ring->preempt_lock, flags); - - /* Make sure everything is posted before making a decision */ - mb(); + /* Update HW if this is the current ring and we are not in preempt*/ + if (!a6xx_in_preempt(a6xx_gpu)) { + if (a6xx_gpu->cur_ring == ring) + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + else + ring->restore_wptr = true; + } else { + ring->restore_wptr = true; + } - gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + spin_unlock_irqrestore(&ring->preempt_lock, flags); } static void get_stats_counter(struct msm_ringbuffer *ring, u32 counter, @@ -138,12 +223,14 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, /* * Write the new TTBR0 to the memstore. This is good for debugging. + * Needed for preemption */ - OUT_PKT7(ring, CP_MEM_WRITE, 4); + OUT_PKT7(ring, CP_MEM_WRITE, 5); OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr))); OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr))); OUT_RING(ring, lower_32_bits(ttbr)); - OUT_RING(ring, (asid << 16) | upper_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + OUT_RING(ring, ctx->seqno); /* * Sync both threads after switching pagetables and enable BR only @@ -268,6 +355,34 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) a6xx_flush(gpu, ring); } +static void a6xx_emit_set_pseudo_reg(struct msm_ringbuffer *ring, + struct a6xx_gpu *a6xx_gpu, struct msm_gpu_submitqueue *queue) +{ + OUT_PKT7(ring, CP_SET_PSEUDO_REG, 12); + + OUT_RING(ring, SMMU_INFO); + /* don't save SMMU, we write the record from the kernel instead */ + OUT_RING(ring, 0); + OUT_RING(ring, 0); + + /* privileged and non secure buffer save */ + OUT_RING(ring, NON_SECURE_SAVE_ADDR); + OUT_RING(ring, lower_32_bits( + a6xx_gpu->preempt_iova[ring->id] + PREEMPT_OFFSET_PRIV_NON_SECURE)); + OUT_RING(ring, upper_32_bits( + a6xx_gpu->preempt_iova[ring->id] + PREEMPT_OFFSET_PRIV_NON_SECURE)); + + /* user context buffer save, seems to be unnused by fw */ + OUT_RING(ring, NON_PRIV_SAVE_ADDR); + OUT_RING(ring, 0); + OUT_RING(ring, 0); + + OUT_RING(ring, COUNTER); + /* seems OK to set to 0 to disable it */ + OUT_RING(ring, 0); + OUT_RING(ring, 0); +} + static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) { unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT; @@ -285,6 +400,13 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx); + /* + * If preemption is enabled, then set the pseudo register for the save + * sequence + */ + if (gpu->nr_rings > 1) + a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, submit->queue); + get_stats_counter(ring, REG_A7XX_RBBM_PERFCTR_CP(0), rbmemptr_stats(ring, index, cpcycles_start)); get_stats_counter(ring, REG_A6XX_CP_ALWAYS_ON_COUNTER, @@ -376,6 +498,8 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) OUT_RING(ring, upper_32_bits(rbmemptr(ring, bv_fence))); OUT_RING(ring, submit->seqno); + a6xx_gpu->last_seqno[ring->id] = submit->seqno; + /* write the ringbuffer timestamp */ OUT_PKT7(ring, CP_EVENT_WRITE, 4); OUT_RING(ring, CACHE_CLEAN | CP_EVENT_WRITE_0_IRQ | BIT(27)); @@ -389,10 +513,32 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) OUT_PKT7(ring, CP_SET_MARKER, 1); OUT_RING(ring, 0x100); /* IFPC enable */ + /* If preemption is enabled */ + if (gpu->nr_rings > 1) { + /* Yield the floor on command completion */ + OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4); + + /* + * If dword[2:1] are non zero, they specify an address for + * the CP to write the value of dword[3] to on preemption + * complete. Write 0 to skip the write + */ + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + /* Data value - not used if the address above is 0 */ + OUT_RING(ring, 0x01); + /* generate interrupt on preemption completion */ + OUT_RING(ring, 0x00); + } + + trace_msm_gpu_submit_flush(submit, gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER)); a6xx_flush(gpu, ring); + + /* Check to see if we need to start preemption */ + a6xx_preempt_trigger(gpu); } static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) @@ -588,6 +734,89 @@ static void a6xx_set_ubwc_config(struct msm_gpu *gpu) adreno_gpu->ubwc_config.min_acc_len << 23 | hbb_lo << 21); } +static void a7xx_patch_pwrup_reglist(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct adreno_reglist_list reglist[2]; + void *ptr = a6xx_gpu->pwrup_reglist_ptr; + struct cpu_gpu_lock *lock = ptr; + u32 *dest = (u32 *)&lock->regs[0]; + int i, j; + + lock->gpu_req = lock->cpu_req = lock->turn = 0; + lock->ifpc_list_len = ARRAY_SIZE(a7xx_ifpc_pwrup_reglist); + lock->preemption_list_len = ARRAY_SIZE(a7xx_pwrup_reglist); + + /* Static IFPC-only registers */ + reglist[0].regs = a7xx_ifpc_pwrup_reglist; + reglist[0].count = ARRAY_SIZE(a7xx_ifpc_pwrup_reglist); + lock->ifpc_list_len = reglist[0].count; + + /* Static IFPC + preemption registers */ + reglist[1].regs = a7xx_pwrup_reglist; + reglist[1].count = ARRAY_SIZE(a7xx_pwrup_reglist); + lock->preemption_list_len = reglist[1].count; + + /* + * For each entry in each of the lists, write the offset and the current + * register value into the GPU buffer + */ + for (i = 0; i < 2; i++) { + const u32 *r = reglist[i].regs; + + for (j = 0; j < reglist[i].count; j++) { + *dest++ = r[j]; + *dest++ = gpu_read(gpu, r[j]); + } + } + + /* + * The overall register list is composed of + * 1. Static IFPC-only registers + * 2. Static IFPC + preemption registers + * 3. Dynamic IFPC + preemption registers (ex: perfcounter selects) + * + * The first two lists are static. Size of these lists are stored as + * number of pairs in ifpc_list_len and preemption_list_len + * respectively. With concurrent binning, Some of the perfcounter + * registers being virtualized, CP needs to know the pipe id to program + * the aperture inorder to restore the same. Thus, third list is a + * dynamic list with triplets as + * (
), and the length is + * stored as number for triplets in dynamic_list_len. + */ + lock->dynamic_list_len = 0; +} + +static int a7xx_preempt_start(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct msm_ringbuffer *ring = gpu->rb[0]; + + if (gpu->nr_rings <= 1) + return 0; + + /* Turn CP protection off */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 0); + + a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, NULL); + + /* Yield the floor on command completion */ + OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4); + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + /* Generate interrupt on preemption completion */ + OUT_RING(ring, 0x00); + + a6xx_flush(gpu, ring); + + return a6xx_idle(gpu, ring) ? 0 : -EINVAL; +} + static int a6xx_cp_init(struct msm_gpu *gpu) { struct msm_ringbuffer *ring = gpu->rb[0]; @@ -619,6 +848,8 @@ static int a6xx_cp_init(struct msm_gpu *gpu) static int a7xx_cp_init(struct msm_gpu *gpu) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); struct msm_ringbuffer *ring = gpu->rb[0]; u32 mask; @@ -656,11 +887,11 @@ static int a7xx_cp_init(struct msm_gpu *gpu) /* *Don't* send a power up reg list for concurrent binning (TODO) */ /* Lo address */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, lower_32_bits(a6xx_gpu->pwrup_reglist_iova)); /* Hi address */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, upper_32_bits(a6xx_gpu->pwrup_reglist_iova)); /* BIT(31) set => read the regs from the list */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, BIT(31)); a6xx_flush(gpu, ring); return a6xx_idle(gpu, ring) ? 0 : -EINVAL; @@ -784,6 +1015,16 @@ static int a6xx_ucode_load(struct msm_gpu *gpu) msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow"); } + a6xx_gpu->pwrup_reglist_ptr = msm_gem_kernel_new(gpu->dev, PAGE_SIZE, + MSM_BO_WC | MSM_BO_MAP_PRIV, + gpu->aspace, &a6xx_gpu->pwrup_reglist_bo, + &a6xx_gpu->pwrup_reglist_iova); + + if (IS_ERR(a6xx_gpu->pwrup_reglist_ptr)) + return PTR_ERR(a6xx_gpu->pwrup_reglist_ptr); + + msm_gem_object_set_name(a6xx_gpu->pwrup_reglist_bo, "pwrup_reglist"); + return 0; } @@ -1128,6 +1369,8 @@ static int hw_init(struct msm_gpu *gpu) if (a6xx_gpu->shadow_bo) { gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR, shadowptr(a6xx_gpu, gpu->rb[0])); + for (unsigned int i = 0; i < gpu->nr_rings; i++) + a6xx_gpu->shadow[i] = 0; } /* ..which means "always" on A7xx, also for BV shadow */ @@ -1136,6 +1379,8 @@ static int hw_init(struct msm_gpu *gpu) rbmemptr(gpu->rb[0], bv_rptr)); } + a6xx_preempt_hw_init(gpu); + /* Always come up on rb 0 */ a6xx_gpu->cur_ring = gpu->rb[0]; @@ -1145,6 +1390,11 @@ static int hw_init(struct msm_gpu *gpu) /* Enable the SQE_to start the CP engine */ gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); + if (adreno_is_a7xx(adreno_gpu) && !a6xx_gpu->pwrup_reglist_emitted) { + a7xx_patch_pwrup_reglist(gpu); + a6xx_gpu->pwrup_reglist_emitted = true; + } + ret = adreno_is_a7xx(adreno_gpu) ? a7xx_cp_init(gpu) : a6xx_cp_init(gpu); if (ret) goto out; @@ -1182,6 +1432,10 @@ static int hw_init(struct msm_gpu *gpu) out: if (adreno_has_gmu_wrapper(adreno_gpu)) return ret; + + /* Last step - yield the ringbuffer */ + a7xx_preempt_start(gpu); + /* * Tell the GMU that we are done touching the GPU and it can start power * management @@ -1559,8 +1813,13 @@ static irqreturn_t a6xx_irq(struct msm_gpu *gpu) if (status & A6XX_RBBM_INT_0_MASK_SWFUSEVIOLATION) a7xx_sw_fuse_violation_irq(gpu); - if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) + if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) { msm_gpu_retire(gpu); + a6xx_preempt_trigger(gpu); + } + + if (status & A6XX_RBBM_INT_0_MASK_CP_SW) + a6xx_preempt_irq(gpu); return IRQ_HANDLED; } @@ -2333,6 +2592,8 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) a6xx_fault_handler); a6xx_calc_ubwc_config(adreno_gpu); + /* Set up the preemption specific bits and pieces for each ringbuffer */ + a6xx_preempt_init(gpu); return gpu; } diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h index e3e5c53ae8af2cc59a21160f05c59fd125cb94b1..7fc994121676844cc53438fb47756e7caf5eee03 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -12,6 +12,31 @@ extern bool hang_debug; +struct cpu_gpu_lock { + uint32_t gpu_req; + uint32_t cpu_req; + uint32_t turn; + union { + struct { + uint16_t list_length; + uint16_t list_offset; + }; + struct { + uint8_t ifpc_list_len; + uint8_t preemption_list_len; + uint16_t dynamic_list_len; + }; + }; + uint64_t regs[62]; +}; + +struct adreno_reglist_list { + /** @reg: List of register **/ + const u32 *regs; + /** @count: Number of registers in the list **/ + u32 count; +}; + /** * struct a6xx_info - a6xx specific information from device table * @@ -31,6 +56,20 @@ struct a6xx_gpu { uint64_t sqe_iova; struct msm_ringbuffer *cur_ring; + struct msm_ringbuffer *next_ring; + + struct drm_gem_object *preempt_bo[MSM_GPU_MAX_RINGS]; + void *preempt[MSM_GPU_MAX_RINGS]; + uint64_t preempt_iova[MSM_GPU_MAX_RINGS]; + uint32_t last_seqno[MSM_GPU_MAX_RINGS]; + + atomic_t preempt_state; + spinlock_t eval_lock; + struct timer_list preempt_timer; + + unsigned int preempt_level; + bool uses_gmem; + bool skip_save_restore; struct a6xx_gmu gmu; @@ -38,6 +77,11 @@ struct a6xx_gpu { uint64_t shadow_iova; uint32_t *shadow; + struct drm_gem_object *pwrup_reglist_bo; + void *pwrup_reglist_ptr; + uint64_t pwrup_reglist_iova; + bool pwrup_reglist_emitted; + bool has_whereami; void __iomem *llc_mmio; @@ -49,6 +93,102 @@ struct a6xx_gpu { #define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) +/* + * In order to do lockless preemption we use a simple state machine to progress + * through the process. + * + * PREEMPT_NONE - no preemption in progress. Next state START. + * PREEMPT_START - The trigger is evaluating if preemption is possible. Next + * states: TRIGGERED, NONE + * PREEMPT_FINISH - An intermediate state before moving back to NONE. Next + * state: NONE. + * PREEMPT_TRIGGERED: A preemption has been executed on the hardware. Next + * states: FAULTED, PENDING + * PREEMPT_FAULTED: A preemption timed out (never completed). This will trigger + * recovery. Next state: N/A + * PREEMPT_PENDING: Preemption complete interrupt fired - the callback is + * checking the success of the operation. Next state: FAULTED, NONE. + */ + +enum a6xx_preempt_state { + PREEMPT_NONE = 0, + PREEMPT_START, + PREEMPT_FINISH, + PREEMPT_TRIGGERED, + PREEMPT_FAULTED, + PREEMPT_PENDING, +}; + +/* + * struct a6xx_preempt_record is a shared buffer between the microcode and the + * CPU to store the state for preemption. The record itself is much larger + * (2112k) but most of that is used by the CP for storage. + * + * There is a preemption record assigned per ringbuffer. When the CPU triggers a + * preemption, it fills out the record with the useful information (wptr, ring + * base, etc) and the microcode uses that information to set up the CP following + * the preemption. When a ring is switched out, the CP will save the ringbuffer + * state back to the record. In this way, once the records are properly set up + * the CPU can quickly switch back and forth between ringbuffers by only + * updating a few registers (often only the wptr). + * + * These are the CPU aware registers in the record: + * @magic: Must always be 0xAE399D6EUL + * @info: Type of the record - written 0 by the CPU, updated by the CP + * @errno: preemption error record + * @data: Data field in YIELD and SET_MARKER packets, Written and used by CP + * @cntl: Value of RB_CNTL written by CPU, save/restored by CP + * @rptr: Value of RB_RPTR written by CPU, save/restored by CP + * @wptr: Value of RB_WPTR written by CPU, save/restored by CP + * @_pad: Reserved/padding + * @rptr_addr: Value of RB_RPTR_ADDR_LO|HI written by CPU, save/restored by CP + * @rbase: Value of RB_BASE written by CPU, save/restored by CP + * @counter: GPU address of the storage area for the preemption counters + * @bv_rptr_addr: Value of BV_RB_RPTR_ADDR_LO|HI written by CPU, save/restored by CP + */ +struct a6xx_preempt_record { + u32 magic; + u32 info; + u32 errno; + u32 data; + u32 cntl; + u32 rptr; + u32 wptr; + u32 _pad; + u64 rptr_addr; + u64 rbase; + u64 counter; + u64 bv_rptr_addr; +}; + +#define A6XX_PREEMPT_RECORD_MAGIC 0xAE399D6EUL + +#define PREEMPT_RECORD_SIZE_FALLBACK(size) \ + ((size) == 0 ? 4192 * SZ_1K : (size)) + +#define PREEMPT_OFFSET_SMMU_INFO 0 +#define PREEMPT_OFFSET_PRIV_NON_SECURE (PREEMPT_OFFSET_SMMU_INFO + 4096) +#define PREEMPT_SIZE(size) \ + (PREEMPT_OFFSET_PRIV_NON_SECURE + PREEMPT_RECORD_SIZE_FALLBACK(size)) + +/* + * The preemption counter block is a storage area for the value of the + * preemption counters that are saved immediately before context switch. We + * append it on to the end of the allocation for the preemption record. + */ +#define A6XX_PREEMPT_COUNTER_SIZE (16 * 4) + +struct a7xx_cp_smmu_info { + u32 magic; + u32 _pad4; + u64 ttbr0; + u32 asid; + u32 context_idr; + u32 context_bank; +}; + +#define GEN7_CP_SMMU_INFO_MAGIC 0x241350d5UL + /* * Given a register and a count, return a value to program into * REG_CP_PROTECT_REG(n) - this will block both reads and writes for @@ -106,6 +246,34 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node); int a6xx_gmu_wrapper_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node); void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); +void a6xx_preempt_init(struct msm_gpu *gpu); +void a6xx_preempt_hw_init(struct msm_gpu *gpu); +void a6xx_preempt_trigger(struct msm_gpu *gpu); +void a6xx_preempt_irq(struct msm_gpu *gpu); +void a6xx_preempt_fini(struct msm_gpu *gpu); +int a6xx_preempt_submitqueue_setup(struct msm_gpu *gpu, + struct msm_gpu_submitqueue *queue); +void a6xx_preempt_submitqueue_close(struct msm_gpu *gpu, + struct msm_gpu_submitqueue *queue); + +/* Return true if we are in a preempt state */ +static inline bool a6xx_in_preempt(struct a6xx_gpu *a6xx_gpu) +{ + /* + * Make sure the read to preempt_state is ordered with respect to reads + * of other variables before ... + */ + smp_rmb(); + + int preempt_state = atomic_read(&a6xx_gpu->preempt_state); + + /* ... and after. */ + smp_rmb(); + + return !(preempt_state == PREEMPT_NONE || + preempt_state == PREEMPT_FINISH); +} + void a6xx_gmu_set_freq(struct msm_gpu *gpu, struct dev_pm_opp *opp, bool suspended); unsigned long a6xx_gmu_get_freq(struct msm_gpu *gpu); diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c new file mode 100644 index 0000000000000000000000000000000000000000..38d68b341d99e9f4c39213d968de6b1e168c9786 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c @@ -0,0 +1,377 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2018, The Linux Foundation. All rights reserved. */ +/* Copyright (c) 2023 Collabora, Ltd. */ +/* Copyright (c) 2024 Valve Corporation */ + +#include "msm_gem.h" +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" +#include "msm_mmu.h" + +/* + * Try to transition the preemption state from old to new. Return + * true on success or false if the original state wasn't 'old' + */ +static inline bool try_preempt_state(struct a6xx_gpu *a6xx_gpu, + enum a6xx_preempt_state old, enum a6xx_preempt_state new) +{ + enum a6xx_preempt_state cur = atomic_cmpxchg(&a6xx_gpu->preempt_state, + old, new); + + return (cur == old); +} + +/* + * Force the preemption state to the specified state. This is used in cases + * where the current state is known and won't change + */ +static inline void set_preempt_state(struct a6xx_gpu *gpu, + enum a6xx_preempt_state new) +{ + /* + * preempt_state may be read by other cores trying to trigger a + * preemption or in the interrupt handler so barriers are needed + * before... + */ + smp_mb__before_atomic(); + atomic_set(&gpu->preempt_state, new); + /* ... and after*/ + smp_mb__after_atomic(); +} + +/* Write the most recent wptr for the given ring into the hardware */ +static inline void update_wptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + unsigned long flags; + uint32_t wptr; + + spin_lock_irqsave(&ring->preempt_lock, flags); + + if (ring->restore_wptr) { + wptr = get_wptr(ring); + + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + + ring->restore_wptr = false; + } + + spin_unlock_irqrestore(&ring->preempt_lock, flags); +} + +/* Return the highest priority ringbuffer with something in it */ +static struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + unsigned long flags; + int i; + + for (i = 0; i < gpu->nr_rings; i++) { + bool empty; + struct msm_ringbuffer *ring = gpu->rb[i]; + + spin_lock_irqsave(&ring->preempt_lock, flags); + empty = (get_wptr(ring) == gpu->funcs->get_rptr(gpu, ring)); + if (!empty && ring == a6xx_gpu->cur_ring) + empty = ring->memptrs->fence == a6xx_gpu->last_seqno[i]; + spin_unlock_irqrestore(&ring->preempt_lock, flags); + + if (!empty) + return ring; + } + + return NULL; +} + +static void a6xx_preempt_timer(struct timer_list *t) +{ + struct a6xx_gpu *a6xx_gpu = from_timer(a6xx_gpu, t, preempt_timer); + struct msm_gpu *gpu = &a6xx_gpu->base.base; + struct drm_device *dev = gpu->dev; + + if (!try_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED, PREEMPT_FAULTED)) + return; + + dev_err(dev->dev, "%s: preemption timed out\n", gpu->name); + kthread_queue_work(gpu->worker, &gpu->recover_work); +} + +void a6xx_preempt_irq(struct msm_gpu *gpu) +{ + uint32_t status; + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct drm_device *dev = gpu->dev; + + if (!try_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED, PREEMPT_PENDING)) + return; + + /* Delete the preemption watchdog timer */ + del_timer(&a6xx_gpu->preempt_timer); + + /* + * The hardware should be setting the stop bit of CP_CONTEXT_SWITCH_CNTL + * to zero before firing the interrupt, but there is a non zero chance + * of a hardware condition or a software race that could set it again + * before we have a chance to finish. If that happens, log and go for + * recovery + */ + status = gpu_read(gpu, REG_A6XX_CP_CONTEXT_SWITCH_CNTL); + if (unlikely(status & A6XX_CP_CONTEXT_SWITCH_CNTL_STOP)) { + DRM_DEV_ERROR(&gpu->pdev->dev, + "!!!!!!!!!!!!!!!! preemption faulted !!!!!!!!!!!!!! irq\n"); + set_preempt_state(a6xx_gpu, PREEMPT_FAULTED); + dev_err(dev->dev, "%s: Preemption failed to complete\n", + gpu->name); + kthread_queue_work(gpu->worker, &gpu->recover_work); + return; + } + + a6xx_gpu->cur_ring = a6xx_gpu->next_ring; + a6xx_gpu->next_ring = NULL; + + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); + + update_wptr(gpu, a6xx_gpu->cur_ring); + + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + + /* + * Retrigger preemption to avoid a deadlock that might occur when preemption + * is skipped due to it being already in flight when requested. + */ + a6xx_preempt_trigger(gpu); +} + +void a6xx_preempt_hw_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + /* No preemption if we only have one ring */ + if (gpu->nr_rings == 1) + return; + + for (i = 0; i < gpu->nr_rings; i++) { + struct a6xx_preempt_record *record_ptr = + a6xx_gpu->preempt[i] + PREEMPT_OFFSET_PRIV_NON_SECURE; + record_ptr->wptr = 0; + record_ptr->rptr = 0; + record_ptr->rptr_addr = shadowptr(a6xx_gpu, gpu->rb[i]); + record_ptr->info = 0; + record_ptr->data = 0; + record_ptr->rbase = gpu->rb[i]->iova; + } + + /* Write a 0 to signal that we aren't switching pagetables */ + gpu_write64(gpu, REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO, 0); + + /* Enable the GMEM save/restore feature for preemption */ + gpu_write(gpu, REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE, 0x1); + + /* Reset the preemption state */ + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + + spin_lock_init(&a6xx_gpu->eval_lock); + + /* Always come up on rb 0 */ + a6xx_gpu->cur_ring = gpu->rb[0]; +} + +void a6xx_preempt_trigger(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + unsigned long flags; + struct msm_ringbuffer *ring; + unsigned int cntl; + + if (gpu->nr_rings == 1) + return; + + /* + * Lock to make sure another thread attempting preemption doesn't skip it + * while we are still evaluating the next ring. This makes sure the other + * thread does start preemption if we abort it and avoids a soft lock. + */ + spin_lock_irqsave(&a6xx_gpu->eval_lock, flags); + + /* + * Try to start preemption by moving from NONE to START. If + * unsuccessful, a preemption is already in flight + */ + if (!try_preempt_state(a6xx_gpu, PREEMPT_NONE, PREEMPT_START)) { + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + return; + } + + cntl = A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL(a6xx_gpu->preempt_level); + + if (a6xx_gpu->skip_save_restore) + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_SKIP_SAVE_RESTORE; + + if (a6xx_gpu->uses_gmem) + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_USES_GMEM; + + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_STOP; + + /* Get the next ring to preempt to */ + ring = get_next_ring(gpu); + + /* + * If no ring is populated or the highest priority ring is the current + * one do nothing except to update the wptr to the latest and greatest + */ + if (!ring || (a6xx_gpu->cur_ring == ring)) { + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); + update_wptr(gpu, a6xx_gpu->cur_ring); + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + return; + } + + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + + spin_lock_irqsave(&ring->preempt_lock, flags); + + struct a7xx_cp_smmu_info *smmu_info_ptr = + a6xx_gpu->preempt[ring->id] + PREEMPT_OFFSET_SMMU_INFO; + struct a6xx_preempt_record *record_ptr = + a6xx_gpu->preempt[ring->id] + PREEMPT_OFFSET_PRIV_NON_SECURE; + u64 ttbr0 = ring->memptrs->ttbr0; + u32 context_idr = ring->memptrs->context_idr; + + smmu_info_ptr->ttbr0 = ttbr0; + smmu_info_ptr->context_idr = context_idr; + record_ptr->wptr = get_wptr(ring); + + /* + * The GPU will write the wptr we set above when we preempt. Reset + * restore_wptr to make sure that we don't write WPTR to the same + * thing twice. It's still possible subsequent submissions will update + * wptr again, in which case they will set the flag to true. This has + * to be protected by the lock for setting the flag and updating wptr + * to be atomic. + */ + ring->restore_wptr = false; + + spin_unlock_irqrestore(&ring->preempt_lock, flags); + + gpu_write64(gpu, + REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO, + a6xx_gpu->preempt_iova[ring->id] + PREEMPT_OFFSET_SMMU_INFO); + + gpu_write64(gpu, + REG_A6XX_CP_CONTEXT_SWITCH_PRIV_NON_SECURE_RESTORE_ADDR, + a6xx_gpu->preempt_iova[ring->id] + PREEMPT_OFFSET_PRIV_NON_SECURE); + + a6xx_gpu->next_ring = ring; + + /* Start a timer to catch a stuck preemption */ + mod_timer(&a6xx_gpu->preempt_timer, jiffies + msecs_to_jiffies(10000)); + + /* Set the preemption state to triggered */ + set_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED); + + /* Trigger the preemption */ + gpu_write(gpu, REG_A6XX_CP_CONTEXT_SWITCH_CNTL, cntl); +} + +static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu, + struct msm_ringbuffer *ring) +{ + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + struct drm_gem_object *bo = NULL; + phys_addr_t ttbr; + u64 iova = 0; + void *ptr; + int asid; + + ptr = msm_gem_kernel_new(gpu->dev, + PREEMPT_SIZE(adreno_gpu->info->preempt_record_size), + MSM_BO_WC | MSM_BO_MAP_PRIV, gpu->aspace, &bo, &iova); + + if (IS_ERR(ptr)) + return PTR_ERR(ptr); + + memset(ptr, 0, PREEMPT_SIZE(adreno_gpu->info->preempt_record_size)); + + msm_gem_object_set_name(bo, "preempt_record ring%d", ring->id); + + a6xx_gpu->preempt_bo[ring->id] = bo; + a6xx_gpu->preempt_iova[ring->id] = iova; + a6xx_gpu->preempt[ring->id] = ptr; + + struct a7xx_cp_smmu_info *smmu_info_ptr = ptr + PREEMPT_OFFSET_SMMU_INFO; + struct a6xx_preempt_record *record_ptr = ptr + PREEMPT_OFFSET_PRIV_NON_SECURE; + + msm_iommu_pagetable_params(gpu->aspace->mmu, &ttbr, &asid); + + smmu_info_ptr->magic = GEN7_CP_SMMU_INFO_MAGIC; + smmu_info_ptr->ttbr0 = ttbr; + smmu_info_ptr->asid = 0xdecafbad; + smmu_info_ptr->context_idr = 0; + + /* Set up the defaults on the preemption record */ + record_ptr->magic = A6XX_PREEMPT_RECORD_MAGIC; + record_ptr->info = 0; + record_ptr->data = 0; + record_ptr->rptr = 0; + record_ptr->wptr = 0; + record_ptr->cntl = MSM_GPU_RB_CNTL_DEFAULT; + record_ptr->rbase = ring->iova; + record_ptr->counter = 0; + record_ptr->bv_rptr_addr = rbmemptr(ring, bv_rptr); + + return 0; +} + +void a6xx_preempt_fini(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + for (i = 0; i < gpu->nr_rings; i++) + msm_gem_kernel_put(a6xx_gpu->preempt_bo[i], gpu->aspace); +} + +void a6xx_preempt_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + /* No preemption if we only have one ring */ + if (gpu->nr_rings <= 1) + return; + + for (i = 0; i < gpu->nr_rings; i++) { + if (preempt_init_ring(a6xx_gpu, gpu->rb[i])) + goto fail; + } + + /* TODO: make this configurable? */ + a6xx_gpu->preempt_level = 1; + a6xx_gpu->uses_gmem = 1; + a6xx_gpu->skip_save_restore = 1; + + timer_setup(&a6xx_gpu->preempt_timer, a6xx_preempt_timer, 0); + + return; +fail: + /* + * On any failure our adventure is over. Clean up and + * set nr_rings to 1 to force preemption off + */ + a6xx_preempt_fini(gpu); + gpu->nr_rings = 1; + + DRM_DEV_ERROR(&gpu->pdev->dev, + "preemption init failed, disabling preemption\n"); + + return; +} diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 174f83137a49940ec80b1fbf548e214fa3c32784..d1e49f701c8176e50d2b9a5cca35acee67f75209 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -36,6 +36,7 @@ struct msm_rbmemptrs { volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT]; volatile u64 ttbr0; + volatile u32 context_idr; }; struct msm_cp_state { @@ -101,6 +102,12 @@ struct msm_ringbuffer { */ spinlock_t preempt_lock; + /* + * Whether we skipped writing wptr and it needs to be updated in the + * future when the ring becomes current. + */ + bool restore_wptr; + /** * cur_ctx_seqno: * From patchwork Thu Sep 26 21:16:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 830817 Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99AC0188707; Thu, 26 Sep 2024 21:17:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385432; cv=none; b=Wj3Tl5wtlwrD6m6056XN8A2I7KlLy4Arkx87IMMEjQwHx4N1H6lj6G9trs1cICZb8bNbSbMfgQajP5m1RmGQC12NLfBcoIfbMblidW1hx+qFyd/JYVSVKQq6taznVw83ARG90jpvnrm2gsHStJg4ou7M43cuwz0kOtz2gzb2hd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385432; c=relaxed/simple; bh=e8NI1sPsjgdaczPyb6uHIaB/U/4w86fcwHEFY+OeNMM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=O343iEQ++OGcKcSAkaMJOeXTuhR+c5JOiOSywlnU79mulnh45/0KNEhDcVEpxlD/5SIuqgwKm5Pz4QbjvmscwLwXDdJjyF2OUIogQd5KjDodNk+n3yQE7LCtvDC4tA58hM9Iy7c9ay92Ya3JkcJTG8C8mpWos6N8iVWljSByGxg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Qijhq5th; arc=none smtp.client-ip=209.85.167.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Qijhq5th" Received: by mail-lf1-f50.google.com with SMTP id 2adb3069b0e04-5365a9574b6so2431058e87.1; Thu, 26 Sep 2024 14:17:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385429; x=1727990229; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=P5IaPT/gznJhO20fAht54q7dPGf6JrWBFw1pjTTg0Ss=; b=Qijhq5thpObWR9y2unGfNvacIGmBrq6CwjL+LyvqvCvsXedSaJTQXXoe8zHHU10rmW pKHaV8/WoZvyd5zwWqfTXpr32ry308iJt3o0VinZF+wX+v7mnpyJe8K+2i/CV0ICgBmT pxQgupHpxojZKF5u4A52yyQPbbnwy5zSCaJEN7jgDI6C7Vwn0s/H9jT0qGohvzyMqPBV njCei79FwEAUs3jrQZPaC3BSPmTsaDi9UtxYJau5B99sHMxf8f7kU6KPWq4s+Vi42Yjl g3ASjlwDUnC2w0LNmEam6q/1U9m71TRNG9yJAzKZer5ocpVxJsyUqzJd+/6R/6XKyQ+2 DRsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385429; x=1727990229; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P5IaPT/gznJhO20fAht54q7dPGf6JrWBFw1pjTTg0Ss=; b=YWpIgYgPlYfsYbyhQmtBkoeE4iwofgoPTGHlv4kfYMbG+JmIyYDGlFeV/3sx6hPGPB mV7ufrBlahSQ7zUvtuMuPRS49lvtEYUga0GuCyfKdankB75INqSJTsLyPsMKqBm1A5LS nAK+5GsZOIJsvvbGqWpeJhkrQHCtYgIIaA2AvPAZcpyd1mXrAJdA+kCahvUvlWVyjgsf +/uveNwwjXAa5wDSE0BAhtWtNY9ObW+ZGAOiDryQIb99lhk0mfrbsVyaA4pf90YKze+U QFUm+BtgN3XC1wYFqQQqmK0nL6ppmK1VJU+95uSWLDGCbflmtG46vJ2wGIXrI1b0AxSk gq2w== X-Forwarded-Encrypted: i=1; AJvYcCV2DPjRgDphpPUqdzhLsSRrhzzy8VParoIi7prFAMYZ4eGOA2CnUyGrKOd52NA2vT+KQQMLbmlfnFf5BSCX@vger.kernel.org, AJvYcCWdvVzIAAmC82WIppzWSMeCU6qUcP0Fakvb9le4fHi8V/tM59HFtAUSjZIqPbhhRxSlB3zzDcSexsc=@vger.kernel.org X-Gm-Message-State: AOJu0YxFnFOq6s7WNYHgtx3ph/0Pg6TWJU1dKkncAkR6RXtbEJyGKchA 2ruQW4fXOcXODU4GD4wo5169Nct2goc8lk1qSpdfkbllCuYG4Efi X-Google-Smtp-Source: AGHT+IEwNNAB08ANCNV/wGne7cXNeEle3eZUTerFEgNElbRY0pDwuztkgBO/PdeOyRc+y/ceeL3eTA== X-Received: by 2002:a05:6512:ba4:b0:538:9ef8:882c with SMTP id 2adb3069b0e04-5389fca4542mr893746e87.57.1727385428334; Thu, 26 Sep 2024 14:17:08 -0700 (PDT) Received: from [192.168.1.17] (a-lu6-5.tin.it. [212.216.221.196]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c297b7d6sm38862166b.162.2024.09.26.14.17.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:17:08 -0700 (PDT) From: Antonino Maniscalco Date: Thu, 26 Sep 2024 23:16:50 +0200 Subject: [PATCH v6 08/11] drm/msm/a6xx: Add traces for preemption Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240926-preemption-a750-t-v6-8-7b6e1ef3648f@gmail.com> References: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> In-Reply-To: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Akhil P Oommen , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727385413; l=2841; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=e8NI1sPsjgdaczPyb6uHIaB/U/4w86fcwHEFY+OeNMM=; b=L/zsrWgyQ1WUnaU3njAXmCrXvxP69203oSucaP7nGVBVx0l5AGhEKUdnFZCCSDBoqOFudrkrd jhgMiTIYQf0AvgqygoRCcg50ZBqlBdswFhB5qkGr62Xg9Dsw7Xl1ne+ X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= Add trace points corresponding to preemption being triggered and being completed for latency measurement purposes. Reviewed-by: Akhil P Oommen Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 6 ++++++ drivers/gpu/drm/msm/msm_gpu_trace.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c index cf75f612edc15374f3d8238e3dc0297532569607..ae471a572a7100d8a3b86a7eaa0dcbea7a728fb1 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c @@ -7,6 +7,7 @@ #include "a6xx_gpu.h" #include "a6xx_gmu.xml.h" #include "msm_mmu.h" +#include "msm_gpu_trace.h" /* * Try to transition the preemption state from old to new. Return @@ -174,6 +175,8 @@ void a6xx_preempt_irq(struct msm_gpu *gpu) set_preempt_state(a6xx_gpu, PREEMPT_NONE); + trace_msm_gpu_preemption_irq(a6xx_gpu->cur_ring->id); + /* * Retrigger preemption to avoid a deadlock that might occur when preemption * is skipped due to it being already in flight when requested. @@ -295,6 +298,9 @@ void a6xx_preempt_trigger(struct msm_gpu *gpu) */ ring->restore_wptr = false; + trace_msm_gpu_preemption_trigger(a6xx_gpu->cur_ring->id, + ring ? ring->id : -1); + spin_unlock_irqrestore(&ring->preempt_lock, flags); gpu_write64(gpu, diff --git a/drivers/gpu/drm/msm/msm_gpu_trace.h b/drivers/gpu/drm/msm/msm_gpu_trace.h index ac40d857bc4578377b03b4cedd138c87144997e4..7f863282db0d7812c8fd53b3f1fc0cd5635028ba 100644 --- a/drivers/gpu/drm/msm/msm_gpu_trace.h +++ b/drivers/gpu/drm/msm/msm_gpu_trace.h @@ -177,6 +177,34 @@ TRACE_EVENT(msm_gpu_resume, TP_printk("%u", __entry->dummy) ); +TRACE_EVENT(msm_gpu_preemption_trigger, + TP_PROTO(int ring_id_from, int ring_id_to), + TP_ARGS(ring_id_from, ring_id_to), + TP_STRUCT__entry( + __field(int, ring_id_from) + __field(int, ring_id_to) + ), + TP_fast_assign( + __entry->ring_id_from = ring_id_from; + __entry->ring_id_to = ring_id_to; + ), + TP_printk("preempting %u -> %u", + __entry->ring_id_from, + __entry->ring_id_to) +); + +TRACE_EVENT(msm_gpu_preemption_irq, + TP_PROTO(u32 ring_id), + TP_ARGS(ring_id), + TP_STRUCT__entry( + __field(u32, ring_id) + ), + TP_fast_assign( + __entry->ring_id = ring_id; + ), + TP_printk("preempted to %u", __entry->ring_id) +); + #endif #undef TRACE_INCLUDE_PATH From patchwork Thu Sep 26 21:16:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 830816 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32BD218732A; Thu, 26 Sep 2024 21:17:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385435; cv=none; b=AVuCNkipp+rHRDqPOKP54YAROGD2i4k/Kk+LigxMu1YAc5rxyCx+d2aNXCE/6VlMQdEfmv1oyUBbhan0QLD12VuJklzaSEM95uwhEz0KbLUUpDej0wlfk/FI1PE76V0OH5RHA+5Yemen7ZrCm4x2E85XJHbXNYiSrw4RJF68C3E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727385435; c=relaxed/simple; bh=tJ1kdgy3bb373sdl5Q1008Ilk7kEqd+hupf+/02stds=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=j4n33DyzwgL1WUuYHjDA/O77fMZVjCUyFFTOzvoVgyuyzIB6UYZkII2k9zbCR36vhhfaDKwiM+9V9fNhxiyIWTNrDmdnykqUxPXQ869Qdn9Lix1QEWgZP1/iD7AawYIX8xmvd+HNmA7UOe1Oq5AMUmCs8N2hk+C6364iN49jESQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jNMN92c0; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jNMN92c0" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a8a7cdfdd80so185124266b.0; Thu, 26 Sep 2024 14:17:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727385431; x=1727990231; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=aMGC6v5HDzDrNYIW2Y0FsEmwdy21TIrXb2BCqk6p3s0=; b=jNMN92c01sWcuSh65zgfpPv/8/7VBtccRgfeMi9SPtoyrkZ46v8jYaGb3yHHMXbQhL 7ejlPGog6sq6MOY4UlMRMc1xx4WSU1ZS5gklGK7raW1ekIZ11AhgW3sQs9X/IycnJra4 z6xJVdU/5x1QD0zy2+nbihb+h37vUxQc10I4ANkWvPEZ5qgDaaA5z4C9fc7nQPK3zhQ2 PnBWfFOxsaASAQYNjqBLWwH4Ir3WzcVxy4Tq4Cp/mv5jWT97tdaRg/wi3fmGAR02ezhD UPr/Xm/GoijK9vSNNAPXBDwnrrgkj+fRe2tRj+j4ILSmbklZ2xIUVNXm0vFuPvCEifnk xEcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727385431; x=1727990231; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aMGC6v5HDzDrNYIW2Y0FsEmwdy21TIrXb2BCqk6p3s0=; b=VanVQsYSYEhu3xQqlo9l5Ou0SV1LUMVqG6anafK3+pDhCjmaYn/xEXxWATgHh31Dne wI+Sk7fw9H8/qTdZLdIDuV80qQfHyGEDrTIi1+5lJBYSVhzYmRSEJjQhxgFCSegLnpZp Wr36pIEP0vpixFf2k9jSd8vf0UGef4c26g3yOv0/r+6+bDn1/kpyA8aJ8W3daAZ+6oG5 OBuZDXn40oQiSnArLg54iFgQCYhVTX50M00btIpkUi1relwmwU7aiCLm8ivqsb/JxH80 KVvPJwO5zW3HoxNoYXp8geI4GnFN5qFmtWnGoqYffoZcCNHJeoB2xoHJlfnA+ir/cLYP P/jw== X-Forwarded-Encrypted: i=1; AJvYcCVSLEUFVDEwhsRqh+1SMPgp5pylJ5CW+7G/muXs0hipv/JoAaQOqiNqxVOKifcY3QkLKYMR3ki5YdUK2x1N@vger.kernel.org, AJvYcCVoRQN0FHebdL/DCi2lPOGEqfnyFmDCFJOCE5WgeFLyjDOHDMLiKRyueXAIvEHrie9DMqoDTQuG/xg=@vger.kernel.org X-Gm-Message-State: AOJu0YxZQoGKWj/q8OrfHZKnuDwGY89jvH+d9wYOKRSrA2lQtkbhDQyL NgH89XTEearTgqHxfH3HLSo0JUBY+9rqy31ePS9JuXJR90GkGL1x X-Google-Smtp-Source: AGHT+IF3wB74LP8e01KeteynvoWXRYKiStHQCxFyGVIn7DZ24h+gS59rfuRv6C5BtBtaB1bmMgZlPw== X-Received: by 2002:a17:907:6ea1:b0:a86:7b71:7b77 with SMTP id a640c23a62f3a-a93c4a69aabmr65376766b.44.1727385431312; Thu, 26 Sep 2024 14:17:11 -0700 (PDT) Received: from [192.168.1.17] (a-lu6-5.tin.it. [212.216.221.196]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c297b7d6sm38862166b.162.2024.09.26.14.17.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2024 14:17:11 -0700 (PDT) From: Antonino Maniscalco Date: Thu, 26 Sep 2024 23:16:52 +0200 Subject: [PATCH v6 10/11] drm/msm/a6xx: Enable preemption for a750 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240926-preemption-a750-t-v6-10-7b6e1ef3648f@gmail.com> References: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> In-Reply-To: <20240926-preemption-a750-t-v6-0-7b6e1ef3648f@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727385413; l=5085; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=tJ1kdgy3bb373sdl5Q1008Ilk7kEqd+hupf+/02stds=; b=q7238TaQocQjp+PUMQZ+d4DhBkOkFQe3NljCUUn8Fz0W9ogcvQAFV8FxsHN+lAouE9LzY2IFF 11r0seMR8B4DbfsHOYJ2fpnCNrJWluBiupVTh+xu568GeiXXv+Nr03f X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= Initialize with 4 rings to enable preemption. For now only on a750 as other targets require testing. Add the "preemption_enabled" module parameter to override this for other a7xx targets. Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 3 ++- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 6 +++++- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 ++++ drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + drivers/gpu/drm/msm/msm_submitqueue.c | 6 +++++- 5 files changed, 17 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c index 316f23ca91671d973797f2a5b69344f376707325..0e3041b2941905f1acdc9e571e0549a960a7edfa 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c @@ -1240,7 +1240,8 @@ static const struct adreno_info a7xx_gpus[] = { .gmem = 3 * SZ_1M, .inactive_period = DRM_MSM_INACTIVE_PERIOD, .quirks = ADRENO_QUIRK_HAS_CACHED_COHERENT | - ADRENO_QUIRK_HAS_HW_APRIV, + ADRENO_QUIRK_HAS_HW_APRIV | + ADRENO_QUIRK_PREEMPTION, .init = a6xx_gpu_init, .zapfw = "gen70900_zap.mbn", .a6xx = &(const struct a6xx_info) { diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index edbcb6d229ba614be910ee70e75731538116e4a4..4760f9469613c0bf208f56be9608747b5aa75606 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -2529,6 +2529,7 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) struct a6xx_gpu *a6xx_gpu; struct adreno_gpu *adreno_gpu; struct msm_gpu *gpu; + extern int enable_preemption; bool is_a7xx; int ret; @@ -2567,7 +2568,10 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) return ERR_PTR(ret); } - if (is_a7xx) + if ((enable_preemption == 1) || (enable_preemption == -1 && + (config->info->quirks & ADRENO_QUIRK_PREEMPTION))) + ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs_a7xx, 4); + else if (is_a7xx) ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs_a7xx, 1); else if (adreno_has_gmu_wrapper(adreno_gpu)) ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs_gmuwrapper, 1); diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index cfc74a9e2646d3de76a06bd67457d69afa49e309..9ffe91920fbfb4841b28aabec9fbde94539fdd83 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -20,6 +20,10 @@ bool allow_vram_carveout = false; MODULE_PARM_DESC(allow_vram_carveout, "Allow using VRAM Carveout, in place of IOMMU"); module_param_named(allow_vram_carveout, allow_vram_carveout, bool, 0600); +int enable_preemption = -1; +MODULE_PARM_DESC(enable_preemption, "Enable preemption (A7xx only) (1=on , 0=disable, -1=auto (default))"); +module_param(enable_preemption, int, 0600); + extern const struct adreno_gpulist a2xx_gpulist; extern const struct adreno_gpulist a3xx_gpulist; extern const struct adreno_gpulist a4xx_gpulist; diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index 87098567483b69c21025b80f356e0a68f0e7f172..d1cd53f05de68b3873f35520655e09e82fc40449 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -56,6 +56,7 @@ enum adreno_family { #define ADRENO_QUIRK_LMLOADKILL_DISABLE BIT(2) #define ADRENO_QUIRK_HAS_HW_APRIV BIT(3) #define ADRENO_QUIRK_HAS_CACHED_COHERENT BIT(4) +#define ADRENO_QUIRK_PREEMPTION BIT(5) /* Helper for formating the chip_id in the way that userspace tools like * crashdec expect. diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c b/drivers/gpu/drm/msm/msm_submitqueue.c index 9b3ffca3f3b471f509918edd4a2fdb0f80dfeb06..2fc3eaf81f4461990d48065f67bf5477787708e9 100644 --- a/drivers/gpu/drm/msm/msm_submitqueue.c +++ b/drivers/gpu/drm/msm/msm_submitqueue.c @@ -161,6 +161,8 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx, struct msm_drm_private *priv = drm->dev_private; struct msm_gpu_submitqueue *queue; enum drm_sched_priority sched_prio; + extern int enable_preemption; + bool preemption_supported; unsigned ring_nr; int ret; @@ -170,7 +172,9 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx, if (!priv->gpu) return -ENODEV; - if (flags & MSM_SUBMITQUEUE_ALLOW_PREEMPT && priv->gpu->nr_rings == 1) + preemption_supported = priv->gpu->nr_rings == 1 && enable_preemption != 0; + + if (flags & MSM_SUBMITQUEUE_ALLOW_PREEMPT && preemption_supported) return -EINVAL; ret = msm_gpu_convert_priority(priv->gpu, prio, &ring_nr, &sched_prio);