From patchwork Mon Oct 7 08:45:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ekansh Gupta X-Patchwork-Id: 833373 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C45F18C00C; Mon, 7 Oct 2024 08:45:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290742; cv=none; b=BOgmUILqRYI7jx1cxHmxmLpJlS/Vz40V7WnaCkEizy8iLuSXmaf2nPh/+8Bg2W0Y8HlG4ojVyOtIn4t9GHUqXYEtwiY2x3N49KbTMcNEYWGvJ/eMFVtS+pgx0ABwcwKwwDYtR+KnrmrmtZ4GVr6oH/7TGxdHdO+SaGakQeC7+jw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290742; c=relaxed/simple; bh=NQVpztLiZphmfHVc6jh5dzykNAw+rNTIU6MaSQ09BrI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dTVfxoMFbaS/qjKFPs1fIdqzOnY6/dOw1Xp2DSXzuCcoq8XUEQ+IfhkQo8TzlDWClCKcV/31oZHJOijYMebkNVL3hDBQG7k89TtqBLK7Zhtr51j+5MaK0Jix7fN7MoTlOOWj/6YaZNwW3pf3rL6z09oZ62OaALmYLwKPraMSH8w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=bMPsUDnW; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="bMPsUDnW" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4974U2L5001587; Mon, 7 Oct 2024 08:45:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= nWSOYJhjzpP0W0nvkO2lgq7nzrutmo7bmQx2dGfd6IA=; b=bMPsUDnWMUhOiGxG hg3xwLwABoYUIC4pKkH3tJPF/xdp3sZ4o9M7jNbROeoY29vkuEYp7yy11gIO5G4b jgJwmNU3lQthKYWDkeFGmDURte8o0G3269+iD+rc2FMyLMFDrupRV8SZHIWlN6NJ VyxYBsQRyusIByKqj60LG5EidAS6uCdCyaONgGrqs+FRJHiE45bl2+m9udqLuHNE UvBEXaLuKSMRt6GJazCiZYMUY9poVCIQjN9omqvCWW3ly+qVtZpDRSbZFJLVP1Nq bYICcq+JRpPSzC1WYqS1+wH57ANWGLY62UiTjnwiIO1T2+gQzn27s6bEvAx1OlsT yCW+Xg== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 422xv8bff3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Oct 2024 08:45:33 +0000 (GMT) Received: from nalasex01b.na.qualcomm.com (nalasex01b.na.qualcomm.com [10.47.209.197]) by NALASPPMTA01.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4978jXZY030668 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 7 Oct 2024 08:45:33 GMT Received: from hu-ekangupt-hyd.qualcomm.com (10.80.80.8) by nalasex01b.na.qualcomm.com (10.47.209.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Mon, 7 Oct 2024 01:45:30 -0700 From: Ekansh Gupta To: , CC: , , , , , Subject: [PATCH v1 1/4] misc: fastrpc: Add CRC support using invokeV2 request Date: Mon, 7 Oct 2024 14:15:15 +0530 Message-ID: <20241007084518.3649876-2-quic_ekangupt@quicinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> References: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01b.na.qualcomm.com (10.47.209.197) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: -E70nRih5kxIgiKcs6eX7iGZMqqamnNP X-Proofpoint-GUID: -E70nRih5kxIgiKcs6eX7iGZMqqamnNP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2410070061 InvokeV2 request is intended to support multiple enhanced invoke requests like CRC check, performance counter enablement and polling mode for RPC invocations. CRC check is getting enabled as part of this patch. CRC check for input and output argument helps in ensuring data consistency over a remote call. If user intends to enable CRC check, first local user CRC is calculated at user end and a CRC buffer is passed to DSP to capture remote CRC values. DSP is expected to write to the remote CRC buffer which is then compared at user level with the local CRC values. Signed-off-by: Ekansh Gupta --- drivers/misc/fastrpc.c | 161 ++++++++++++++++++++++++------------ include/uapi/misc/fastrpc.h | 7 ++ 2 files changed, 116 insertions(+), 52 deletions(-) diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 74181b8c386b..8e817a763d1d 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -573,13 +573,15 @@ static void fastrpc_get_buff_overlaps(struct fastrpc_invoke_ctx *ctx) static struct fastrpc_invoke_ctx *fastrpc_context_alloc( struct fastrpc_user *user, u32 kernel, u32 sc, - struct fastrpc_invoke_args *args) + struct fastrpc_invoke_v2 *inv2) { struct fastrpc_channel_ctx *cctx = user->cctx; struct fastrpc_invoke_ctx *ctx = NULL; + struct fastrpc_invoke_args *args = NULL; unsigned long flags; int ret; + args = (struct fastrpc_invoke_args *)inv2->inv.args; ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); if (!ctx) return ERR_PTR(-ENOMEM); @@ -611,6 +613,7 @@ static struct fastrpc_invoke_ctx *fastrpc_context_alloc( /* Released in fastrpc_context_put() */ fastrpc_channel_ctx_get(cctx); + ctx->crc = (u32 *)(uintptr_t)inv2->crc; ctx->sc = sc; ctx->retval = -1; ctx->pid = current->pid; @@ -1070,6 +1073,7 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, struct fastrpc_invoke_buf *list; struct fastrpc_phy_page *pages; u64 *fdlist; + u32 *crclist; int i, inbufs, outbufs, handles; inbufs = REMOTE_SCALARS_INBUFS(ctx->sc); @@ -1078,6 +1082,7 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, list = fastrpc_invoke_buf_start(rpra, ctx->nscalars); pages = fastrpc_phy_page_start(list, ctx->nscalars); fdlist = (uint64_t *)(pages + inbufs + outbufs + handles); + crclist = (u32 *)(fdlist + FASTRPC_MAX_FDLIST); for (i = inbufs; i < ctx->nbufs; ++i) { if (!ctx->maps[i]) { @@ -1102,6 +1107,12 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, fastrpc_map_put(mmap); } + if (ctx->crc && crclist && rpra) { + if (copy_to_user((void __user *)ctx->crc, crclist, + FASTRPC_MAX_CRCLIST * sizeof(u32))) + return -EFAULT; + } + return 0; } @@ -1137,13 +1148,12 @@ static int fastrpc_invoke_send(struct fastrpc_session_ctx *sctx, } -static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, - u32 handle, u32 sc, - struct fastrpc_invoke_args *args) +static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct fastrpc_invoke_v2 *inv2) { struct fastrpc_invoke_ctx *ctx = NULL; struct fastrpc_buf *buf, *b; - + struct fastrpc_invoke inv; + u32 handle, sc; int err = 0; if (!fl->sctx) @@ -1152,12 +1162,15 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, if (!fl->cctx->rpdev) return -EPIPE; + inv = inv2->inv; + handle = inv.handle; + sc = inv.sc; if (handle == FASTRPC_INIT_HANDLE && !kernel) { dev_warn_ratelimited(fl->sctx->dev, "user app trying to send a kernel RPC message (%d)\n", handle); return -EPERM; } - ctx = fastrpc_context_alloc(fl, kernel, sc, args); + ctx = fastrpc_context_alloc(fl, kernel, sc, inv2); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1239,6 +1252,7 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, { struct fastrpc_init_create_static init; struct fastrpc_invoke_args *args; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_phy_page pages[1]; char *name; int err; @@ -1248,7 +1262,6 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, u32 namelen; u32 pageslen; } inbuf; - u32 sc; args = kcalloc(FASTRPC_CREATE_STATIC_PROCESS_NARGS, sizeof(*args), GFP_KERNEL); if (!args) @@ -1313,10 +1326,10 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, args[2].length = sizeof(*pages); args[2].fd = -1; - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE_STATIC, 3, 0); - - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, - sc, args); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE_STATIC, 3, 0); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (err) goto err_invoke; @@ -1357,6 +1370,7 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl, { struct fastrpc_init_create init; struct fastrpc_invoke_args *args; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_phy_page pages[1]; struct fastrpc_map *map = NULL; struct fastrpc_buf *imem = NULL; @@ -1370,7 +1384,6 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl, u32 attrs; u32 siglen; } inbuf; - u32 sc; bool unsigned_module = false; args = kcalloc(FASTRPC_CREATE_PROCESS_NARGS, sizeof(*args), GFP_KERNEL); @@ -1444,12 +1457,12 @@ static int fastrpc_init_create_process(struct fastrpc_user *fl, args[5].length = sizeof(inbuf.siglen); args[5].fd = -1; - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE, 4, 0); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE, 4, 0); if (init.attrs) - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE_ATTR, 4, 0); - - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, - sc, args); + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_CREATE_ATTR, 4, 0); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (err) goto err_invoke; @@ -1501,17 +1514,18 @@ static void fastrpc_session_free(struct fastrpc_channel_ctx *cctx, static int fastrpc_release_current_dsp_process(struct fastrpc_user *fl) { struct fastrpc_invoke_args args[1]; + struct fastrpc_invoke_v2 ioctl = {0}; int tgid = 0; - u32 sc; tgid = fl->tgid; args[0].ptr = (u64)(uintptr_t) &tgid; args[0].length = sizeof(tgid); args[0].fd = -1; - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_RELEASE, 1, 0); - return fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, - sc, &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_RELEASE, 1, 0); + ioctl.inv.args = (u64)args; + return fastrpc_internal_invoke(fl, true, &ioctl); } static int fastrpc_device_release(struct inode *inode, struct file *file) @@ -1647,45 +1661,77 @@ static int fastrpc_dmabuf_alloc(struct fastrpc_user *fl, char __user *argp) static int fastrpc_init_attach(struct fastrpc_user *fl, int pd) { struct fastrpc_invoke_args args[1]; + struct fastrpc_invoke_v2 ioctl = {0}; int tgid = fl->tgid; - u32 sc; args[0].ptr = (u64)(uintptr_t) &tgid; args[0].length = sizeof(tgid); args[0].fd = -1; - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_ATTACH, 1, 0); fl->pd = pd; - return fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, - sc, &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_ATTACH, 1, 0); + ioctl.inv.args = (u64)args; + return fastrpc_internal_invoke(fl, true, &ioctl); } -static int fastrpc_invoke(struct fastrpc_user *fl, char __user *argp) +static int fastrpc_copy_args(struct fastrpc_invoke *inv) { struct fastrpc_invoke_args *args = NULL; - struct fastrpc_invoke inv; u32 nscalars; - int err; - - if (copy_from_user(&inv, argp, sizeof(inv))) - return -EFAULT; /* nscalars is truncated here to max supported value */ - nscalars = REMOTE_SCALARS_LENGTH(inv.sc); + nscalars = REMOTE_SCALARS_LENGTH(inv->sc); if (nscalars) { args = kcalloc(nscalars, sizeof(*args), GFP_KERNEL); if (!args) return -ENOMEM; - if (copy_from_user(args, (void __user *)(uintptr_t)inv.args, + if (copy_from_user(args, (void __user *)(uintptr_t)inv->args, nscalars * sizeof(*args))) { kfree(args); return -EFAULT; } } + inv->args = args; - err = fastrpc_internal_invoke(fl, false, inv.handle, inv.sc, args); - kfree(args); + return 0; +} + +static int fastrpc_invoke(struct fastrpc_user *fl, char __user *argp) +{ + struct fastrpc_invoke_v2 ioctl = {0}; + struct fastrpc_invoke inv; + int err; + + if (copy_from_user(&inv, argp, sizeof(inv))) + return -EFAULT; + + err = fastrpc_copy_args(&inv); + if (err) + return err; + + ioctl.inv = inv; + err = fastrpc_internal_invoke(fl, false, &ioctl); + kfree(inv.args); + + return err; +} + +static int fastrpc_invokev2(struct fastrpc_user *fl, char __user *argp) +{ + struct fastrpc_invoke_v2 inv2 = {0}; + int err; + + if (copy_from_user(&inv2, argp, sizeof(inv2))) + return -EFAULT; + + err = fastrpc_copy_args(&inv2.inv); + if (err) + return err; + + err = fastrpc_internal_invoke(fl, false, &inv2); + kfree(inv2.inv.args); return err; } @@ -1694,6 +1740,7 @@ static int fastrpc_get_info_from_dsp(struct fastrpc_user *fl, uint32_t *dsp_attr uint32_t dsp_attr_buf_len) { struct fastrpc_invoke_args args[2] = { 0 }; + struct fastrpc_invoke_v2 ioctl = {0}; /* * Capability filled in userspace. This carries the information @@ -1710,8 +1757,10 @@ static int fastrpc_get_info_from_dsp(struct fastrpc_user *fl, uint32_t *dsp_attr args[1].length = dsp_attr_buf_len * sizeof(u32); args[1].fd = -1; - return fastrpc_internal_invoke(fl, true, FASTRPC_DSP_UTILITIES_HANDLE, - FASTRPC_SCALARS(0, 1, 1), args); + ioctl.inv.handle = FASTRPC_DSP_UTILITIES_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(0, 1, 1); + ioctl.inv.args = (u64)args; + return fastrpc_internal_invoke(fl, true, &ioctl); } static int fastrpc_get_info_from_kernel(struct fastrpc_ioctl_capability *cap, @@ -1798,10 +1847,10 @@ static int fastrpc_get_dsp_info(struct fastrpc_user *fl, char __user *argp) static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf *buf) { struct fastrpc_invoke_args args[1] = { [0] = { 0 } }; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_munmap_req_msg req_msg; struct device *dev = fl->sctx->dev; int err; - u32 sc; req_msg.pgid = fl->tgid; req_msg.size = buf->size; @@ -1810,9 +1859,10 @@ static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf * args[0].ptr = (u64) (uintptr_t) &req_msg; args[0].length = sizeof(req_msg); - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MUNMAP, 1, 0); - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, sc, - &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MUNMAP, 1, 0); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (!err) { dev_dbg(dev, "unmmap\tpt 0x%09lx OK\n", buf->raddr); spin_lock(&fl->lock); @@ -1856,6 +1906,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) { struct fastrpc_invoke_args args[3] = { [0 ... 2] = { 0 } }; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_buf *buf = NULL; struct fastrpc_mmap_req_msg req_msg; struct fastrpc_mmap_rsp_msg rsp_msg; @@ -1863,7 +1914,6 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) struct fastrpc_req_mmap req; struct device *dev = fl->sctx->dev; int err; - u32 sc; if (copy_from_user(&req, argp, sizeof(req))) return -EFAULT; @@ -1906,9 +1956,10 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) args[2].ptr = (u64) (uintptr_t) &rsp_msg; args[2].length = sizeof(rsp_msg); - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MMAP, 2, 1); - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, sc, - &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MMAP, 2, 1); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (err) { dev_err(dev, "mmap error (len 0x%08llx)\n", buf->size); fastrpc_buf_free(buf); @@ -1957,10 +2008,10 @@ static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) static int fastrpc_req_mem_unmap_impl(struct fastrpc_user *fl, struct fastrpc_mem_unmap *req) { struct fastrpc_invoke_args args[1] = { [0] = { 0 } }; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_map *map = NULL, *iter, *m; struct fastrpc_mem_unmap_req_msg req_msg = { 0 }; int err = 0; - u32 sc; struct device *dev = fl->sctx->dev; spin_lock(&fl->lock); @@ -1986,9 +2037,10 @@ static int fastrpc_req_mem_unmap_impl(struct fastrpc_user *fl, struct fastrpc_me args[0].ptr = (u64) (uintptr_t) &req_msg; args[0].length = sizeof(req_msg); - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MEM_UNMAP, 1, 0); - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, sc, - &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MEM_UNMAP, 1, 0); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (err) { dev_err(dev, "unmmap\tpt fd = %d, 0x%09llx error\n", map->fd, map->raddr); return err; @@ -2011,6 +2063,7 @@ static int fastrpc_req_mem_unmap(struct fastrpc_user *fl, char __user *argp) static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp) { struct fastrpc_invoke_args args[4] = { [0 ... 3] = { 0 } }; + struct fastrpc_invoke_v2 ioctl = {0}; struct fastrpc_mem_map_req_msg req_msg = { 0 }; struct fastrpc_mmap_rsp_msg rsp_msg = { 0 }; struct fastrpc_mem_unmap req_unmap = { 0 }; @@ -2019,7 +2072,6 @@ static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp) struct device *dev = fl->sctx->dev; struct fastrpc_map *map = NULL; int err; - u32 sc; if (copy_from_user(&req, argp, sizeof(req))) return -EFAULT; @@ -2055,8 +2107,10 @@ static int fastrpc_req_mem_map(struct fastrpc_user *fl, char __user *argp) args[3].ptr = (u64) (uintptr_t) &rsp_msg; args[3].length = sizeof(rsp_msg); - sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MEM_MAP, 3, 1); - err = fastrpc_internal_invoke(fl, true, FASTRPC_INIT_HANDLE, sc, &args[0]); + ioctl.inv.handle = FASTRPC_INIT_HANDLE; + ioctl.inv.sc = FASTRPC_SCALARS(FASTRPC_RMID_INIT_MEM_MAP, 3, 1); + ioctl.inv.args = (u64)args; + err = fastrpc_internal_invoke(fl, true, &ioctl); if (err) { dev_err(dev, "mem mmap error, fd %d, vaddr %llx, size %lld\n", req.fd, req.vaddrin, map->size); @@ -2096,6 +2150,9 @@ static long fastrpc_device_ioctl(struct file *file, unsigned int cmd, case FASTRPC_IOCTL_INVOKE: err = fastrpc_invoke(fl, argp); break; + case FASTRPC_IOCTL_INVOKEV2: + err = fastrpc_invokev2(fl, argp); + break; case FASTRPC_IOCTL_INIT_ATTACH: err = fastrpc_init_attach(fl, ROOT_PD); break; diff --git a/include/uapi/misc/fastrpc.h b/include/uapi/misc/fastrpc.h index f33d914d8f46..406b80555d41 100644 --- a/include/uapi/misc/fastrpc.h +++ b/include/uapi/misc/fastrpc.h @@ -17,6 +17,7 @@ #define FASTRPC_IOCTL_MEM_MAP _IOWR('R', 10, struct fastrpc_mem_map) #define FASTRPC_IOCTL_MEM_UNMAP _IOWR('R', 11, struct fastrpc_mem_unmap) #define FASTRPC_IOCTL_GET_DSP_INFO _IOWR('R', 13, struct fastrpc_ioctl_capability) +#define FASTRPC_IOCTL_INVOKEV2 _IOWR('R', 14, struct fastrpc_invoke_v2) /** * enum fastrpc_map_flags - control flags for mapping memory on DSP user process @@ -80,6 +81,12 @@ struct fastrpc_invoke { __u64 args; }; +struct fastrpc_invoke_v2 { + struct fastrpc_invoke inv; + __u64 crc; + __u32 reserved[16]; +}; + struct fastrpc_init_create { __u32 filelen; /* elf file length */ __s32 filefd; /* fd for the file */ From patchwork Mon Oct 7 08:45:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ekansh Gupta X-Patchwork-Id: 834118 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB385187353; Mon, 7 Oct 2024 08:45:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290744; cv=none; b=f+RejDWTBKZy43JTw2Uuv0BA9ZxiJt0jXENzji8Tt2vICMT3Yyt+rDDds52HCaP+l9msVsGihYs0PLveIoV8lcJrR+jdpUh9ZACKrF1pjjtmrxNS/5ABAuCVCBHldM9BKEyaRdsBehtSfZOGC/qGxIPfOAU74Fn0ZM0qpBLjTTo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290744; c=relaxed/simple; bh=4He6GAK85rGaihAzpW1qOad9XowHPfchucXUb9v6EyY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dIrmBe3a2Hxwo+bCtN7UG7GIqaxQUOe8SBWyPS76/xe8RMXD26/GtBOXPFp0R3QNDXuzMdU4CctBVq+4o8in88uR853LmD55W5rDIYEqT6msCgoNpcDXp4x3icDE7GwLL9jh7kEVCz1FPFvTVmZ2E2ZbrGbpvJTAvhNNQzpfMqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=WnIWLH/B; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="WnIWLH/B" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4974Sr8g032224; Mon, 7 Oct 2024 08:45:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= d/xYOlAfHq811yxJ/AL8jc7fZL4S8qi38cwZvN2sd8c=; b=WnIWLH/BsLAF0/gm FJWLPObfKoXjPVuPwBqLK35CmESDlIhc95B4p4EiVlneNzcEIm41eOBAndWBVqlL mIVivOwxfbWuw+Qy4T00FfMg3nYg55CqKuf7Tm5nZGlBWP29LfepvkIzgY92ZHQx 8grAY7sq9Lc69TrKTgwcr++Z9XMemahsgRZXBN1s8LyUsEWf1Xkvqg4A9Y682LPh SXhc13F+/Tgb35VRX7zJr/0Fk1vaN8bTJnWOKnmS3rCTg/yoi4PGZ9ItzlDUOFoP ZIZQueXxz2RdMWzGEoXBwfqFyg5I15K3O98y9gzx0wzVwMCMQ4CNZRzVUj1CMusl rUJm2Q== Received: from nalasppmta03.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 422xv8bff5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Oct 2024 08:45:36 +0000 (GMT) Received: from nalasex01b.na.qualcomm.com (nalasex01b.na.qualcomm.com [10.47.209.197]) by NALASPPMTA03.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4978jZwd024819 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 7 Oct 2024 08:45:35 GMT Received: from hu-ekangupt-hyd.qualcomm.com (10.80.80.8) by nalasex01b.na.qualcomm.com (10.47.209.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Mon, 7 Oct 2024 01:45:33 -0700 From: Ekansh Gupta To: , CC: , , , , , Subject: [PATCH v1 2/4] misc: fastrpc: Capture kernel and DSP performance counters Date: Mon, 7 Oct 2024 14:15:16 +0530 Message-ID: <20241007084518.3649876-3-quic_ekangupt@quicinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> References: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01b.na.qualcomm.com (10.47.209.197) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: Jf1M4Vl6VdJJcwp8A4AG2xfEFZWbs8jX X-Proofpoint-GUID: Jf1M4Vl6VdJJcwp8A4AG2xfEFZWbs8jX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2410070061 Add support to capture kernel performance counters for different kernel level operations. These counters collects the information for remote call and copies the information to a buffer shared by user. Collection of DSP performance counters is also added as part of this change. DSP updates the performance information in the metadata which is then copied to a buffer passed by the users. Signed-off-by: Ekansh Gupta --- drivers/misc/fastrpc.c | 143 ++++++++++++++++++++++++++++++++++-- include/uapi/misc/fastrpc.h | 4 +- 2 files changed, 139 insertions(+), 8 deletions(-) diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 8e817a763d1d..54a562fc94fb 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -38,6 +39,7 @@ #define FASTRPC_CTX_MAX (256) #define FASTRPC_INIT_HANDLE 1 #define FASTRPC_DSP_UTILITIES_HANDLE 2 +#define FASTRPC_MAX_STATIC_HANDLE (20) #define FASTRPC_CTXID_MASK (0xFF0) #define INIT_FILELEN_MAX (2 * 1024 * 1024) #define INIT_FILE_NAMELEN_MAX (128) @@ -106,6 +108,9 @@ #define miscdev_to_fdevice(d) container_of(d, struct fastrpc_device, miscdev) +#define FASTRPC_KERNEL_PERF_LIST (PERF_KEY_MAX) +#define FASTRPC_DSP_PERF_LIST 12 + static const char *domains[FASTRPC_DEV_MAX] = { "adsp", "mdsp", "sdsp", "cdsp", "cdsp1" }; struct fastrpc_phy_page { @@ -229,6 +234,18 @@ struct fastrpc_map { struct kref refcount; }; +struct fastrpc_perf { + u64 count; + u64 flush; + u64 map; + u64 copy; + u64 link; + u64 getargs; + u64 putargs; + u64 invargs; + u64 invoke; +}; + struct fastrpc_invoke_ctx { int nscalars; int nbufs; @@ -237,6 +254,8 @@ struct fastrpc_invoke_ctx { int tgid; u32 sc; u32 *crc; + u64 *perf_kernel; + u64 *perf_dsp; u64 ctxid; u64 msg_sz; struct kref refcount; @@ -251,6 +270,7 @@ struct fastrpc_invoke_ctx { struct fastrpc_invoke_args *args; struct fastrpc_buf_overlap *olaps; struct fastrpc_channel_ctx *cctx; + struct fastrpc_perf *perf; }; struct fastrpc_session_ctx { @@ -308,6 +328,51 @@ struct fastrpc_user { struct mutex mutex; }; +enum fastrpc_perfkeys { + PERF_COUNT = 0, + PERF_FLUSH = 1, + PERF_MAP = 2, + PERF_COPY = 3, + PERF_LINK = 4, + PERF_GETARGS = 5, + PERF_PUTARGS = 6, + PERF_INVARGS = 7, + PERF_INVOKE = 8, + PERF_KEY_MAX = 9, +}; + +#define PERF_END ((void)0) + +#define PERF(enb, cnt, ff) \ + {\ + struct timespec64 startT = {0};\ + u64 *counter = cnt;\ + if (enb && counter) {\ + ktime_get_boottime_ts64(&startT);\ + } \ + ff ;\ + if (enb && counter) {\ + *counter += getnstimediff(&startT);\ + } \ + } + +#define GET_COUNTER(perf_ptr, offset) \ + (perf_ptr != NULL ?\ + (((offset >= 0) && (offset < PERF_KEY_MAX)) ?\ + (u64 *)(perf_ptr + offset)\ + : (u64 *)NULL) : (u64 *)NULL) + +static inline s64 getnstimediff(struct timespec64 *start) +{ + s64 ns; + struct timespec64 ts, b; + + ktime_get_boottime_ts64(&ts); + b = timespec64_sub(ts, *start); + ns = timespec64_to_ns(&b); + return ns; +} + static void fastrpc_free_map(struct kref *ref) { struct fastrpc_map *map; @@ -497,6 +562,7 @@ static void fastrpc_context_free(struct kref *ref) idr_remove(&cctx->ctx_idr, ctx->ctxid >> 4); spin_unlock_irqrestore(&cctx->lock, flags); + kfree(ctx->perf); kfree(ctx->maps); kfree(ctx->olaps); kfree(ctx); @@ -614,6 +680,13 @@ static struct fastrpc_invoke_ctx *fastrpc_context_alloc( fastrpc_channel_ctx_get(cctx); ctx->crc = (u32 *)(uintptr_t)inv2->crc; + ctx->perf_dsp = (u64 *)(uintptr_t)inv2->perf_dsp; + ctx->perf_kernel = (u64 *)(uintptr_t)inv2->perf_kernel; + if (ctx->perf_kernel) { + ctx->perf = kzalloc(sizeof(*(ctx->perf)), GFP_KERNEL); + if (!ctx->perf) + return ERR_PTR(-ENOMEM); + } ctx->sc = sc; ctx->retval = -1; ctx->pid = current->pid; @@ -877,7 +950,8 @@ static int fastrpc_get_meta_size(struct fastrpc_invoke_ctx *ctx) sizeof(struct fastrpc_invoke_buf) + sizeof(struct fastrpc_phy_page)) * ctx->nscalars + sizeof(u64) * FASTRPC_MAX_FDLIST + - sizeof(u32) * FASTRPC_MAX_CRCLIST; + sizeof(u32) * FASTRPC_MAX_CRCLIST + + sizeof(u32) + sizeof(u64) * FASTRPC_DSP_PERF_LIST; return size; } @@ -951,7 +1025,9 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) metalen = fastrpc_get_meta_size(ctx); pkt_size = fastrpc_get_payload_size(ctx, metalen); + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_MAP), err = fastrpc_create_maps(ctx); + PERF_END); if (err) return err; @@ -987,6 +1063,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) continue; if (ctx->maps[i]) { + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_MAP), struct vm_area_struct *vma = NULL; rpra[i].buf.pv = (u64) ctx->args[i].ptr; @@ -1003,9 +1080,9 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) pg_end = ((ctx->args[i].ptr + len - 1) & PAGE_MASK) >> PAGE_SHIFT; pages[i].size = (pg_end - pg_start + 1) * PAGE_SIZE; - + PERF_END); } else { - + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_COPY), if (ctx->olaps[oix].offset == 0) { rlen -= ALIGN(args, FASTRPC_ALIGN) - args; args = ALIGN(args, FASTRPC_ALIGN); @@ -1027,12 +1104,14 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) pages[i].size = (pg_end - pg_start + 1) * PAGE_SIZE; args = args + mlen; rlen -= mlen; + PERF_END); } if (i < inbufs && !ctx->maps[i]) { void *dst = (void *)(uintptr_t)rpra[i].buf.pv; void *src = (void *)(uintptr_t)ctx->args[i].ptr; + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_COPY), if (!kernel) { if (copy_from_user(dst, (void __user *)src, len)) { @@ -1042,6 +1121,7 @@ static int fastrpc_get_args(u32 kernel, struct fastrpc_invoke_ctx *ctx) } else { memcpy(dst, src, len); } + PERF_END); } } @@ -1072,9 +1152,9 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, struct fastrpc_map *mmap = NULL; struct fastrpc_invoke_buf *list; struct fastrpc_phy_page *pages; - u64 *fdlist; - u32 *crclist; - int i, inbufs, outbufs, handles; + u64 *fdlist, *perf_dsp_list; + u32 *crclist, *poll; + int i, inbufs, outbufs, handles, perferr; inbufs = REMOTE_SCALARS_INBUFS(ctx->sc); outbufs = REMOTE_SCALARS_OUTBUFS(ctx->sc); @@ -1083,6 +1163,8 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, pages = fastrpc_phy_page_start(list, ctx->nscalars); fdlist = (uint64_t *)(pages + inbufs + outbufs + handles); crclist = (u32 *)(fdlist + FASTRPC_MAX_FDLIST); + poll = (u32 *)(crclist + FASTRPC_MAX_CRCLIST); + perf_dsp_list = (u64 *)(poll + 1); for (i = inbufs; i < ctx->nbufs; ++i) { if (!ctx->maps[i]) { @@ -1113,6 +1195,14 @@ static int fastrpc_put_args(struct fastrpc_invoke_ctx *ctx, return -EFAULT; } + if (ctx->perf_dsp && perf_dsp_list) { + perferr = copy_to_user((void __user *)ctx->perf_dsp, + perf_dsp_list, + FASTRPC_DSP_PERF_LIST * sizeof(u64)); + if (perferr) + dev_dbg(fl->sctx->dev, "Warning: failed to copy perf data %d\n", perferr); + } + return 0; } @@ -1148,13 +1238,35 @@ static int fastrpc_invoke_send(struct fastrpc_session_ctx *sctx, } +static void fastrpc_update_invoke_count(u32 handle, u64 *perf_counter, + struct timespec64 *invoket) +{ + /* update invoke count for dynamic handles */ + u64 *invcount, *count; + + if (handle <= FASTRPC_MAX_STATIC_HANDLE) + return; + + invcount = GET_COUNTER(perf_counter, PERF_INVOKE); + if (invcount) + *invcount += getnstimediff(invoket); + + count = GET_COUNTER(perf_counter, PERF_COUNT); + if (count) + *count++; +} + static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct fastrpc_invoke_v2 *inv2) { struct fastrpc_invoke_ctx *ctx = NULL; struct fastrpc_buf *buf, *b; struct fastrpc_invoke inv; u32 handle, sc; - int err = 0; + int err = 0, perferr = 0; + struct timespec64 invoket = {0}; + + if (inv2->perf_kernel) + ktime_get_boottime_ts64(&invoket); if (!fl->sctx) return -EINVAL; @@ -1174,14 +1286,18 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct if (IS_ERR(ctx)) return PTR_ERR(ctx); + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_GETARGS), err = fastrpc_get_args(kernel, ctx); + PERF_END); if (err) goto bail; /* make sure that all CPU memory writes are seen by DSP */ dma_wmb(); + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_LINK), /* Send invoke buffer to remote dsp */ err = fastrpc_invoke_send(fl->sctx, ctx, kernel, handle); + PERF_END); if (err) goto bail; @@ -1197,8 +1313,10 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct /* make sure that all memory writes by DSP are seen by CPU */ dma_rmb(); + PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_PUTARGS), /* populate all the output buffers with results */ err = fastrpc_put_args(ctx, kernel); + PERF_END); if (err) goto bail; @@ -1209,6 +1327,17 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct bail: if (err != -ERESTARTSYS && err != -ETIMEDOUT) { + if (ctx) { + if (ctx->perf_kernel && !err) + fastrpc_update_invoke_count(handle, (u64 *)ctx->perf, &invoket); + if (ctx->perf_kernel && ctx->perf && ctx->perf_kernel) { + perferr = copy_to_user((void __user *)ctx->perf_kernel, + ctx->perf, + FASTRPC_KERNEL_PERF_LIST * sizeof(u64)); + if (perferr) + dev_dbg(fl->sctx->dev, "Warning: failed to copy perf data %d\n", perferr); + } + } /* We are done with this compute context */ spin_lock(&fl->lock); list_del(&ctx->node); diff --git a/include/uapi/misc/fastrpc.h b/include/uapi/misc/fastrpc.h index 406b80555d41..1edc7c04b171 100644 --- a/include/uapi/misc/fastrpc.h +++ b/include/uapi/misc/fastrpc.h @@ -84,7 +84,9 @@ struct fastrpc_invoke { struct fastrpc_invoke_v2 { struct fastrpc_invoke inv; __u64 crc; - __u32 reserved[16]; + __u64 perf_kernel; + __u64 perf_dsp; + __u32 reserved[12]; }; struct fastrpc_init_create { From patchwork Mon Oct 7 08:45:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ekansh Gupta X-Patchwork-Id: 833372 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EE1A18C32F; Mon, 7 Oct 2024 08:45:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290747; cv=none; b=Yu1kvh2yxcVX09YOhxh11g2jp8pUGLOeRZOOJwvsCmsT5KNhL1asZYDjk757qyj20ccl5LuOMRMteKeiqJbpCrnbvss98YjFtsqEHgU4vINB58NTz32/Vl19ZBst2ewjCUbzdpSXrnvI7Z/cBA1rMJ8VPwwfVwLtAdAhHtkk2eM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290747; c=relaxed/simple; bh=M195vOWhBa+RcPea3T2ce880qTaxse75rPhnLOAwgZI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ucam8fEFJK9NG3/m8i7lF3L7NdhijHfna1ElKMYaTF/FRLSHlHdDIoqo5BivUNFSPMyCAlRGAtW0El80lQCHZG54kmhXJTu/yE/h6NeMj0HBGsUASb0Sxr9HnlJuwXEESte4i1Vykcw+boZUT+0oi5P3l9Hg2ek/jmIilGr50jM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=kNfSyOgD; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="kNfSyOgD" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4974TD8h032511; Mon, 7 Oct 2024 08:45:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= nBevgU321+ZkvO2ez5LU2yHZr2Ipcwcl+bm1EytCkY4=; b=kNfSyOgDES3S1VPO A4tHV5WNzMtz2aGI18AlXozElnAGF27VsOMXdNfgi+vIAyzRVSvebHtWT4mLCHCs soNdm600QXxzjpaIkIG0kiXCbsOKUluUDsUzx4v6tJX+zU78rNBkjuAdPKMMVrTV fC429j0bzUS0JgaVNs8k3gwI3ml49yYMKDFc4XmP1WGIjNa3zJFG4wSPXBEn7f+W uNsKmu8wPdOSOe9m97yicQ/6zd2VpLE8F+Pm4Zy0TGAFKY+VY50T2bG8454nFTtH pmJk91yeKYaWlDnzER7k0CruQ0FHod3FcAIcZUhovIrp7QQpywE+tOkRRIQyu+7z V4VIPw== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 422xv8bff8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Oct 2024 08:45:39 +0000 (GMT) Received: from nalasex01b.na.qualcomm.com (nalasex01b.na.qualcomm.com [10.47.209.197]) by NALASPPMTA01.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4978jc4q030796 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 7 Oct 2024 08:45:38 GMT Received: from hu-ekangupt-hyd.qualcomm.com (10.80.80.8) by nalasex01b.na.qualcomm.com (10.47.209.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Mon, 7 Oct 2024 01:45:36 -0700 From: Ekansh Gupta To: , CC: , , , , , Subject: [PATCH v1 3/4] misc: fastrpc: Modify context id calculation for poll mode Date: Mon, 7 Oct 2024 14:15:17 +0530 Message-ID: <20241007084518.3649876-4-quic_ekangupt@quicinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> References: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01b.na.qualcomm.com (10.47.209.197) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: ccl-2WawCM_VKXkgP_eX6u3SlLX3l-PU X-Proofpoint-GUID: ccl-2WawCM_VKXkgP_eX6u3SlLX3l-PU X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 impostorscore=0 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2410070061 Poll mode is a feature to be introduced which improves the fastrpc remote calls latency. For this feature, DSP expects the 4th to 7th bit of context id to be zero. Prepare context id in such a way that polling mode can be supported. Signed-off-by: Ekansh Gupta --- drivers/misc/fastrpc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index 54a562fc94fb..c008fcd95e15 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -40,7 +40,7 @@ #define FASTRPC_INIT_HANDLE 1 #define FASTRPC_DSP_UTILITIES_HANDLE 2 #define FASTRPC_MAX_STATIC_HANDLE (20) -#define FASTRPC_CTXID_MASK (0xFF0) +#define FASTRPC_CTXID_MASK (0xFF0000) #define INIT_FILELEN_MAX (2 * 1024 * 1024) #define INIT_FILE_NAMELEN_MAX (128) #define FASTRPC_DEVICE_NAME "fastrpc" @@ -559,7 +559,7 @@ static void fastrpc_context_free(struct kref *ref) fastrpc_buf_free(ctx->buf); spin_lock_irqsave(&cctx->lock, flags); - idr_remove(&cctx->ctx_idr, ctx->ctxid >> 4); + idr_remove(&cctx->ctx_idr, ctx->ctxid >> 16); spin_unlock_irqrestore(&cctx->lock, flags); kfree(ctx->perf); @@ -706,7 +706,7 @@ static struct fastrpc_invoke_ctx *fastrpc_context_alloc( spin_unlock_irqrestore(&cctx->lock, flags); goto err_idr; } - ctx->ctxid = ret << 4; + ctx->ctxid = ret << 16; spin_unlock_irqrestore(&cctx->lock, flags); kref_init(&ctx->refcount); @@ -2621,7 +2621,7 @@ static int fastrpc_rpmsg_callback(struct rpmsg_device *rpdev, void *data, if (len < sizeof(*rsp)) return -EINVAL; - ctxid = ((rsp->ctx & FASTRPC_CTXID_MASK) >> 4); + ctxid = ((rsp->ctx & FASTRPC_CTXID_MASK) >> 16); spin_lock_irqsave(&cctx->lock, flags); ctx = idr_find(&cctx->ctx_idr, ctxid); From patchwork Mon Oct 7 08:45:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ekansh Gupta X-Patchwork-Id: 834117 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC5F718C90A; Mon, 7 Oct 2024 08:45:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290750; cv=none; b=TNjIbkVs9PqGbfGQDBAp+MvAjtkzhOFswJMPWjTrYYKny3OPXRP7utF8h++39g5q8d+pyqtDfGVBVj0h+PooluXz5oX9tKKefsUyEseUe0QBFEP1QpVaJYYbvdzWZ3P5oV1tGFposFIQ6wqyBVByWexSEJnlT7gFzp/uZgjlnmc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728290750; c=relaxed/simple; bh=UG72Cpzkki87rQALIJWboNyc6S7PCXSW4dMpJctdtPw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FkAVgkRJbQo7btae8jMNb9MkCasDDj8kcc5MityonmpezbjJP2V1NVy09uZF0tHidmFSBxjziE+LJ1cpp1kbfCa4iwq/9Ah7bRlbwhxj5onEP5RaYQ9hf5wZzDH6pTFGOqT0QcIxxAO0uq8VmT0uTHTeIioJiv+p5czKJ9D5mzg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=Arln9c0R; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Arln9c0R" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4974TfF0021858; Mon, 7 Oct 2024 08:45:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= iiGu/T4YMqs9vu7zT6kk9SjYjwUUm6sCok1Z5LDxw9s=; b=Arln9c0RSPiHsmI5 SDAhItocFrrTWqOM1++5O50zknnBN0sWleYW4HCYLmkk26997xxmwDOc8T9WmkQL +XoB1XcAKCX4flOH3LyabaDf4E2O8SC+lQqwsXNvrYEMs22OmsAhpQ26cacZSEDG 4XnZwBVLi/Owszui56G/TxCmBEY5/pyrYoyREYxZXSEhuwnUFfjbGbOEXER2AbKx Fk/WLKwnxUHpKSjIgQjBUOiRRIPPUbHzEzMcRE4ihOeWa4mNcJF7MIyzTHQ0tkGz RmZJpbyETJQJOBsgQvMYxw72mhXsJ+f1bmoeaBtuZBnSBeLjHqNxj0WJiphnBfdH nqHCaA== Received: from nalasppmta05.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 422xr5kgjj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 07 Oct 2024 08:45:42 +0000 (GMT) Received: from nalasex01b.na.qualcomm.com (nalasex01b.na.qualcomm.com [10.47.209.197]) by NALASPPMTA05.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4978jfCQ010040 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 7 Oct 2024 08:45:41 GMT Received: from hu-ekangupt-hyd.qualcomm.com (10.80.80.8) by nalasex01b.na.qualcomm.com (10.47.209.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Mon, 7 Oct 2024 01:45:39 -0700 From: Ekansh Gupta To: , CC: , , , , , Subject: [PATCH v1 4/4] misc: fastrpc: Add polling mode support for fastRPC driver Date: Mon, 7 Oct 2024 14:15:18 +0530 Message-ID: <20241007084518.3649876-5-quic_ekangupt@quicinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> References: <20241007084518.3649876-1-quic_ekangupt@quicinc.com> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01b.na.qualcomm.com (10.47.209.197) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 6JLd1kRHvyREZ8gnek10FIlMq_Qsjmsf X-Proofpoint-ORIG-GUID: 6JLd1kRHvyREZ8gnek10FIlMq_Qsjmsf X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 clxscore=1015 mlxlogscore=999 priorityscore=1501 impostorscore=0 adultscore=0 mlxscore=0 phishscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2409260000 definitions=main-2410070061 For any remote call to DSP, after sending an invocation message, fastRPC driver waits for glink response and during this time the CPU can go into low power modes. Adding a polling mode support with which fastRPC driver will poll continuously on a memory after sending a message to remote subsystem which will eliminate CPU wakeup and scheduling latencies and reduce fastRPC overhead. With this change, DSP always sends a glink response which will get ignored if polling mode didn't time out. Signed-off-by: Ekansh Gupta --- drivers/misc/fastrpc.c | 127 +++++++++++++++++++++++++++++++++--- include/uapi/misc/fastrpc.h | 3 +- 2 files changed, 121 insertions(+), 9 deletions(-) diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index c008fcd95e15..a25673c22db8 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -111,6 +111,19 @@ #define FASTRPC_KERNEL_PERF_LIST (PERF_KEY_MAX) #define FASTRPC_DSP_PERF_LIST 12 +/* Poll response number from remote processor for call completion */ +#define FASTRPC_POLL_RESPONSE (0xdecaf) +/* timeout in us for polling until memory barrier */ +#define FASTRPC_POLL_TIME_MEM_UPDATE (500) + +/* Response types supported for RPC calls */ +enum fastrpc_response_flags { + /* normal job completion glink response */ + NORMAL_RESPONSE = 0, + /* process updates poll memory instead of glink response */ + POLL_MODE = 1, +}; + static const char *domains[FASTRPC_DEV_MAX] = { "adsp", "mdsp", "sdsp", "cdsp", "cdsp1" }; struct fastrpc_phy_page { @@ -258,6 +271,12 @@ struct fastrpc_invoke_ctx { u64 *perf_dsp; u64 ctxid; u64 msg_sz; + /* Threads poll for specified timeout and fall back to glink wait */ + u64 poll_timeout; + /* work done status flag */ + bool is_work_done; + /* response flags from remote processor */ + enum fastrpc_response_flags rsp_flags; struct kref refcount; struct list_head node; /* list of ctxs */ struct completion work; @@ -682,6 +701,7 @@ static struct fastrpc_invoke_ctx *fastrpc_context_alloc( ctx->crc = (u32 *)(uintptr_t)inv2->crc; ctx->perf_dsp = (u64 *)(uintptr_t)inv2->perf_dsp; ctx->perf_kernel = (u64 *)(uintptr_t)inv2->perf_kernel; + ctx->poll_timeout = (u64)inv2->poll_timeout; if (ctx->perf_kernel) { ctx->perf = kzalloc(sizeof(*(ctx->perf)), GFP_KERNEL); if (!ctx->perf) @@ -692,6 +712,8 @@ static struct fastrpc_invoke_ctx *fastrpc_context_alloc( ctx->pid = current->pid; ctx->tgid = user->tgid; ctx->cctx = cctx; + ctx->rsp_flags = NORMAL_RESPONSE; + ctx->is_work_done = false; init_completion(&ctx->work); INIT_WORK(&ctx->put_work, fastrpc_context_put_wq); @@ -1256,6 +1278,87 @@ static void fastrpc_update_invoke_count(u32 handle, u64 *perf_counter, *count++; } +static int poll_for_remote_response(struct fastrpc_invoke_ctx *ctx, u64 timeout) +{ + int err = -EIO, i, j; + u32 sc = ctx->sc; + struct fastrpc_invoke_buf *list; + struct fastrpc_phy_page *pages; + u64 *fdlist = NULL; + u32 *crclist = NULL, *poll = NULL; + unsigned int inbufs, outbufs, handles; + + /* calculate poll memory location */ + inbufs = REMOTE_SCALARS_INBUFS(sc); + outbufs = REMOTE_SCALARS_OUTBUFS(sc); + handles = REMOTE_SCALARS_INHANDLES(sc) + REMOTE_SCALARS_OUTHANDLES(sc); + list = fastrpc_invoke_buf_start(ctx->rpra, ctx->nscalars); + pages = fastrpc_phy_page_start(list, ctx->nscalars); + fdlist = (u64 *)(pages + inbufs + outbufs + handles); + crclist = (u32 *)(fdlist + FASTRPC_MAX_FDLIST); + poll = (u32 *)(crclist + FASTRPC_MAX_CRCLIST); + + /* poll on memory for DSP response. Return failure on timeout */ + for (i = 0, j = 0; i < timeout; i++, j++) { + if (*poll == FASTRPC_POLL_RESPONSE) { + err = 0; + ctx->is_work_done = true; + ctx->retval = 0; + break; + } + if (j == FASTRPC_POLL_TIME_MEM_UPDATE) { + /* make sure that all poll memory writes by DSP are seen by CPU */ + dma_rmb(); + j = 0; + } + udelay(1); + } + return err; +} + +static inline int fastrpc_wait_for_response(struct fastrpc_invoke_ctx *ctx, + u32 kernel) +{ + int err = 0; + + if (kernel) { + if (!wait_for_completion_timeout(&ctx->work, 10 * HZ)) + err = -ETIMEDOUT; + } else { + err = wait_for_completion_interruptible(&ctx->work); + } + + return err; +} + +static int fastrpc_wait_for_completion(struct fastrpc_invoke_ctx *ctx, + u32 kernel) +{ + int err; + + do { + switch (ctx->rsp_flags) { + case NORMAL_RESPONSE: + err = fastrpc_wait_for_response(ctx, kernel); + if (err || ctx->is_work_done) + return err; + break; + case POLL_MODE: + err = poll_for_remote_response(ctx, ctx->poll_timeout); + /* If polling timed out, move to normal response mode */ + if (err) + ctx->rsp_flags = NORMAL_RESPONSE; + break; + default: + err = -EBADR; + dev_dbg(ctx->fl->sctx->dev, "unsupported response type:0x%x\n", ctx->rsp_flags); + break; + } + } while (!ctx->is_work_done); + + return err; +} + static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct fastrpc_invoke_v2 *inv2) { struct fastrpc_invoke_ctx *ctx = NULL; @@ -1301,16 +1404,22 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, struct if (err) goto bail; - if (kernel) { - if (!wait_for_completion_timeout(&ctx->work, 10 * HZ)) - err = -ETIMEDOUT; - } else { - err = wait_for_completion_interruptible(&ctx->work); - } + if (ctx->poll_timeout != 0 && handle > FASTRPC_MAX_STATIC_HANDLE && + fl->cctx->domain_id == CDSP_DOMAIN_ID && + fl->pd == USER_PD) + ctx->rsp_flags = POLL_MODE; + err = fastrpc_wait_for_completion(ctx, kernel); if (err) goto bail; + if (!ctx->is_work_done) { + err = -ETIMEDOUT; + dev_dbg(fl->sctx->dev, "Invalid workdone state for handle 0x%x, sc 0x%x\n", + handle, sc); + goto bail; + } + /* make sure that all memory writes by DSP are seen by CPU */ dma_rmb(); PERF(ctx->perf_kernel, GET_COUNTER((u64 *)ctx->perf, PERF_PUTARGS), @@ -2627,12 +2736,14 @@ static int fastrpc_rpmsg_callback(struct rpmsg_device *rpdev, void *data, ctx = idr_find(&cctx->ctx_idr, ctxid); spin_unlock_irqrestore(&cctx->lock, flags); + /* Ignore this failure as context returned will be NULL for polling mode */ if (!ctx) { - dev_err(&rpdev->dev, "No context ID matches response\n"); - return -ENOENT; + dev_dbg(&rpdev->dev, "No context ID matches response\n"); + return 0; } ctx->retval = rsp->retval; + ctx->is_work_done = true; complete(&ctx->work); /* diff --git a/include/uapi/misc/fastrpc.h b/include/uapi/misc/fastrpc.h index 1edc7c04b171..dde684554011 100644 --- a/include/uapi/misc/fastrpc.h +++ b/include/uapi/misc/fastrpc.h @@ -86,7 +86,8 @@ struct fastrpc_invoke_v2 { __u64 crc; __u64 perf_kernel; __u64 perf_dsp; - __u32 reserved[12]; + __u64 poll_timeout; + __u32 reserved[10]; }; struct fastrpc_init_create {