From patchwork Tue Apr 12 05:14:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C37BC433F5 for ; Tue, 12 Apr 2022 05:15:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233238AbiDLFSG (ORCPT ); Tue, 12 Apr 2022 01:18:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347751AbiDLFSD (ORCPT ); Tue, 12 Apr 2022 01:18:03 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 999E3344EF; Mon, 11 Apr 2022 22:15:46 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23C4ZfLf028018; Tue, 12 Apr 2022 05:15:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=vM7E6r5mviZVwlwH6pQpxCFTVPHF9CArCTvbz56sfLjjr53zg4rg9Sn6NvZCsmJkj0ZE D4BGW94eZjc06hyJtOkavcLyLPk4pA81/g0XUfjLAEwBkgxb8dQKzqIP5tkS5Isd6yUS pf9ymNleAtsDOPC4gHq4+Fj6zAzpqUPRP0MOBgjOfAqXgFY6sjXdxj0auKQ5HRqr6vZZ 9/R26Uc2u7mbMegVuhUyp9ptyolPlLo8+60BF6pVTfiutLJOKK4L9iDbcYeMQQGPWV8J Tn2s3zk/6YOEXB0c+Vko50QaTjdYwiR0E3EIIAZyiiXvGNplnrXRYnMxT8AuKbnBPALR Ww== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb219wdxe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:15:44 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5FhQq016586; Tue, 12 Apr 2022 05:15:43 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2040.outbound.protection.outlook.com [104.47.66.40]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k2h5gq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:15:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q6Kzn6G6+sOcdsJzBikbcd8/TdbsZsGSDOnx2wJN+jfBut25NWq/xVGjHlVFEqh9Y4yh3uLp3eDvSNqEKX1x+yu0O5lqg2pVxHj/0GtTsNuKCLzbwwEG0CxFRxwYpFgTs7CoIu6gM/3m+qupJRbhzB00eGoTQ/geZ65jbOn2+HFoE5tcw+20oN0u/e2BneJ2K+Uezo+YT68nNFjen7I5uaMy82Q2mePsI5EsPVAuvFhJOWpgDaM6Bd3ELMu5MGLZUU9atw185ul8wW5QZsbArbDasYPaB1NmAIwqSQLiYLTTZnehXUx++5RPbLfGwFOxm3uMJIK57ol6pLmTDuQpZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=Rwwm5YUekCspByFHiYWQon7L+WOsC8NitwLNmaojs06cV+wnGLjiijsVhFh1zf+U9yULZgPZmgN6TeOhotXKWCa3FmNi0qpagTIgksPOncBKeexPZ1d3pJ97Om6OYhtktSN9rIwVWCueyHl4xCmrScKDyuCIJYK3uMhT83jjVZjOUceQqqPOmYv7Av0dBpJz3HBQLY0KPq9TuXOOEvOHtOJtsPSuFHWqAlIwY1UuOJQW5LKWhfvmR65DZVUMf5UxXCX9PoO0vHW9nlzGgHB0t3uXNamxpPd63ZPT0op4RmQtJibiSqL1mrViqFCySruoPhaZwcAtPvjC76lKjFpk8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=BP7ZL/xDCota4kFyP7LC2YgzJcMEv5NFHJupLwHR7Fq3TstbokzVkMDp8SWWKJV9I/FTOG+xleM7YFxovm56tXUN26ZJrOB08hxMUZzanrPoLCUhdEs8fwUC9kbF8hqIPq/5s3wT7D0eauIQtcgkUV0gsuk7drs5SgpfV4D/zwY= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BYAPR10MB3256.namprd10.prod.outlook.com (2603:10b6:a03:152::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:15:36 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:15:36 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 01/18 stable-5.15.y] gup: Turn fault_in_pages_{readable,writeable} into fault_in_{readable,writeable} Date: Tue, 12 Apr 2022 13:14:58 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR06CA0007.apcprd06.prod.outlook.com (2603:1096:4:186::9) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40d019c5-4c67-457b-d8a0-08da1c4379e2 X-MS-TrafficTypeDiagnostic: BYAPR10MB3256:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WKzpNcT+YLwCCrVCpvitpjxyr0QCfCOIrX/OG5SSO7URhLMvpacZxDQ11uqaKjRKAmgxJvsa+jTtHX66CtixBDwQvg+9dEgPkQLalWujxJd7lbkK4iX5ky4PayVIX4n4LHVIBeIL5TVewLIxNC2acd5r6uDrJtMyB+p2X/+JBDXnVOJ/4YmtE9BDLsi/On3sew37pmN97zJH6jBU2Ryj2I/d8vF6soO5vBLwUnY5Crhn1rpAzE8DMKXXNQOguWjx2S1d4OO3ndQPJ63XLte9hzqnxz0ZZHIncG5LVjgsyEbCRXPhJrG/3S+mftyRxF4iB4mW1DxB4CaDLVXvXv5KC9FzfGI0w+zva61nlNnarXZsXQrn7ZEbmzE5z4yceTYbdnZ8BipZqlvzHZTmTRX6tvPb1x80LNGUyId4jR/kktGvSFkqy0M0HIqkPII5UtMH47owpWwHQXtMtr82ud4H8vTwwZlBUVthyjOyPLh3wgkqlSPEV9O/3pSeQ632p24zKKOswqcJkBq5ZxDLImHlleeXtNeojpWlDVK72l4UZwovh6AtLCsgP8hcJoqMV/4LIQ0/TZPjkB850yWMt5mHVvNIuYs9djwmT7LFV83p8bZu+ReDcX4h9PTJhffxDpcXKzsKb2Vu4H6gry0TjeDctA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(38100700002)(8936002)(5660300002)(30864003)(2906002)(83380400001)(6512007)(2616005)(66476007)(44832011)(107886003)(186003)(6506007)(6666004)(508600001)(66556008)(66946007)(6486002)(86362001)(54906003)(36756003)(8676002)(4326008)(6916009)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PUoUC2qMP3LeZZVIOu5XwHCIwqH0qjwMoO8CeVKHNBUSZPF5G4fxtHcaIJYaW3Tv3viYfQUcM2on6r5SLIlEUm1QPo6JJaLPLb9WtBYxgKiqXsEJFg18fZXMhBWqe7GuGbIW0cT5XdItrz6Pc83XscCLnZQqp194dmgQm8z0bZBmOfzYpW/WpiHdwQi8kLwKLf1u7LyG94QmuIDNep/CQSQ2TPvng4REkJWBjEjSGLduKLgYhOW+jQ4JrqN554mX9P6P7cSmm5iD+WdlnwHCgo58YaqDLNML8/Z/NWGQTBzRe5fOe8HRMPGQsePBzu41fa2inZE1dHWAsr2az+/yTNXpOWKxPruonzuMBM4KuRUbty6zV6o7RI9ejVCB2/QVu/xnM5luv6hHwmtFZ/LERxfMC45w6VDsKyrDDeWsQwisIli2wyrsZM4l2aQpwZCn5sMK6qvxaIwE76dWkRCL9BQsOsqFy864vm1zgW1KPafsPKTIdpafY4z/SbjXYvmI9FOEotLWf38IDR8+m1sxhOp8g8qf+NrMcDSFFj2GiaBp68S3QfF/bGYv12Ezn7nIJTeUszNWKB+TQsqEjOKEEuPxmih1WRBI8TOKQQdG0DrX9bXrFtnU6W3wiuDzxyTpGP7JS+cX9fwp6Iy9sEhPI9PbCA13hxMTXCTBC6+ucvKCcnQwyzmGwBULCYC1dw5ZAPJYrSIUUATwdknrP8U0EQYXAM0EFIePys/auxmbkYXITBzw/fz/xHabLkFgj7UxZR96UK4GuyqoEaTGF03uVyJLixbvvSo6nKTUJz4Jleofuvzs14oqvQ20MW1D+ZvIt559/XE+1SCZvIrp6HMzzd+UPz1Qrd3IGTpWJD2yVOLRrydvNjmwqMBLrOX8BqbNLCPQrK+f/lGkFP2qeYiRqEfDV2R6H3NuJtA17mN7Vt2q5SdNkjsrVJOBuCSouk/PL1jjl1Ni8zKX4T07fNIwT6nzr60x3mhfyPdJSfjjYlOs3DYcPbtAPvAWOMAnv+e69lW2NFyqigzk1QF3vBwLqRCOt41WtlcBNfaGqmXqY3n2l2kTzaDMvz/atZZXHFDSBmsbWnUuw9RU6XMyRkxjHW7KqDb/8v65n1LthB7oDfLMrJIV49/pfMsWd9o4HimYQ+a/tnkbHEVddnCYvuNZj55ucVh4pHFGvnHzPhCx494xRpvGFqVGQs2lU6SNSNQFH7ldco5SnLz4lHVIQandytP9Z+ky5sotGsNupUJEEuCoeEj2df7ISivvrvI7dRUQkAPYggNBs0pzO+BYCX5NQtCUlUep/GJ+eJ0am+VXmMoTAuDJ+jJOyPgLIUK1FxAnk8yQz3KH+i9uVY0J9Z98mS54iTTrkVJdvm83SD43l1FlanIHGWcbNi5IMXl2+ZvjDik61zfw6P46+pGR1L9WUnU2UFHx17ZeHtN1cZYrGNVFkF0xqlGJJs2oAvB5if4ze27DtOBy15O8gbODGd64Xm2aoJwsxdqXws4lH9Isol+BU8cWAT++DRyheMwWQopZatRB7MOeslci5Sc57yHKObkk4jUGyuVFCX6K9dHDLA9ERkLTiXArCorrNHUHQ3pXpFlyg2wnf6PVYCKb7+WA3TMYQNKOR2cye73V8qkKyeJpOsi+VSwY/QsNca7kPnNe7SusVM8oaHSBZ3D3d/HzMo+hQypqkrY2ocD2EI2lEZFB5pvKiTq21VYh24CaQUfJo5YMgXiZVo1eblwb/HWL2lwrtzFJWh48oKWL6KbpO07sp3NBrtsR7r7IxFp3dLK0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 40d019c5-4c67-457b-d8a0-08da1c4379e2 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:15:36.2297 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5lJSQOtoWCkdShBc84IsZGb1D7nHuh5Tk7Rb1enpI99e2ZGVcMNkbJa9OuoQjJ7mi6yTaP1yVenP9yQCQguMkQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3256 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-GUID: wSRLVjnSgIainkyzQuWJlLg0qiCygnAU X-Proofpoint-ORIG-GUID: wSRLVjnSgIainkyzQuWJlLg0qiCygnAU Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit bb523b406c849eef8f265a07cd7f320f1f177743 upstream Turn fault_in_pages_{readable,writeable} into versions that return the number of bytes not faulted in, similar to copy_to_user, instead of returning a non-zero value when any of the requested pages couldn't be faulted in. This supports the existing users that require all pages to be faulted in as well as new users that are happy if any pages can be faulted in. Rename the functions to fault_in_{readable,writeable} to make sure this change doesn't silently break things. Neither of these functions is entirely trivial and it doesn't seem useful to inline them, so move them to mm/gup.c. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- arch/powerpc/kernel/kvm.c | 3 +- arch/powerpc/kernel/signal_32.c | 4 +- arch/powerpc/kernel/signal_64.c | 2 +- arch/x86/kernel/fpu/signal.c | 7 ++- drivers/gpu/drm/armada/armada_gem.c | 7 ++- fs/btrfs/ioctl.c | 5 +- include/linux/pagemap.h | 57 ++--------------------- lib/iov_iter.c | 10 ++-- mm/filemap.c | 2 +- mm/gup.c | 72 +++++++++++++++++++++++++++++ 10 files changed, 93 insertions(+), 76 deletions(-) diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c index d89cf802d9aa..6568823cf306 100644 --- a/arch/powerpc/kernel/kvm.c +++ b/arch/powerpc/kernel/kvm.c @@ -669,7 +669,8 @@ static void __init kvm_use_magic_page(void) on_each_cpu(kvm_map_magic_page, &features, 1); /* Quick self-test to see if the mapping works */ - if (fault_in_pages_readable((const char *)KVM_MAGIC_PAGE, sizeof(u32))) { + if (fault_in_readable((const char __user *)KVM_MAGIC_PAGE, + sizeof(u32))) { kvm_patching_worked = false; return; } diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c index f2da879264bc..3e053e2fd6b6 100644 --- a/arch/powerpc/kernel/signal_32.c +++ b/arch/powerpc/kernel/signal_32.c @@ -1048,7 +1048,7 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext __user *, old_ctx, if (new_ctx == NULL) return 0; if (!access_ok(new_ctx, ctx_size) || - fault_in_pages_readable((u8 __user *)new_ctx, ctx_size)) + fault_in_readable((char __user *)new_ctx, ctx_size)) return -EFAULT; /* @@ -1239,7 +1239,7 @@ SYSCALL_DEFINE3(debug_setcontext, struct ucontext __user *, ctx, #endif if (!access_ok(ctx, sizeof(*ctx)) || - fault_in_pages_readable((u8 __user *)ctx, sizeof(*ctx))) + fault_in_readable((char __user *)ctx, sizeof(*ctx))) return -EFAULT; /* diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c index bb9c077ac132..d1e1fc0acbea 100644 --- a/arch/powerpc/kernel/signal_64.c +++ b/arch/powerpc/kernel/signal_64.c @@ -688,7 +688,7 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext __user *, old_ctx, if (new_ctx == NULL) return 0; if (!access_ok(new_ctx, ctx_size) || - fault_in_pages_readable((u8 __user *)new_ctx, ctx_size)) + fault_in_readable((char __user *)new_ctx, ctx_size)) return -EFAULT; /* diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index 831b25c5e705..7f71bd4dcd0d 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -205,7 +205,7 @@ int copy_fpstate_to_sigframe(void __user *buf, void __user *buf_fx, int size) fpregs_unlock(); if (ret) { - if (!fault_in_pages_writeable(buf_fx, fpu_user_xstate_size)) + if (!fault_in_writeable(buf_fx, fpu_user_xstate_size)) goto retry; return -EFAULT; } @@ -278,10 +278,9 @@ static int restore_fpregs_from_user(void __user *buf, u64 xrestore, if (ret != -EFAULT) return -EINVAL; - ret = fault_in_pages_readable(buf, size); - if (!ret) + if (!fault_in_readable(buf, size)) goto retry; - return ret; + return -EFAULT; } /* diff --git a/drivers/gpu/drm/armada/armada_gem.c b/drivers/gpu/drm/armada/armada_gem.c index 21909642ee4c..8fbb25913327 100644 --- a/drivers/gpu/drm/armada/armada_gem.c +++ b/drivers/gpu/drm/armada/armada_gem.c @@ -336,7 +336,7 @@ int armada_gem_pwrite_ioctl(struct drm_device *dev, void *data, struct drm_armada_gem_pwrite *args = data; struct armada_gem_object *dobj; char __user *ptr; - int ret; + int ret = 0; DRM_DEBUG_DRIVER("handle %u off %u size %u ptr 0x%llx\n", args->handle, args->offset, args->size, args->ptr); @@ -349,9 +349,8 @@ int armada_gem_pwrite_ioctl(struct drm_device *dev, void *data, if (!access_ok(ptr, args->size)) return -EFAULT; - ret = fault_in_pages_readable(ptr, args->size); - if (ret) - return ret; + if (fault_in_readable(ptr, args->size)) + return -EFAULT; dobj = armada_gem_object_lookup(file, args->handle); if (dobj == NULL) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 6a863b3f6de0..bf53af8694f8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2258,9 +2258,8 @@ static noinline int search_ioctl(struct inode *inode, key.offset = sk->min_offset; while (1) { - ret = fault_in_pages_writeable(ubuf + sk_offset, - *buf_size - sk_offset); - if (ret) + ret = -EFAULT; + if (fault_in_writeable(ubuf + sk_offset, *buf_size - sk_offset)) break; ret = btrfs_search_forward(root, &key, path, sk->min_transid); diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 62db6b0176b9..9fe94f7a4f7e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -733,61 +733,10 @@ int wait_on_page_private_2_killable(struct page *page); extern void add_page_wait_queue(struct page *page, wait_queue_entry_t *waiter); /* - * Fault everything in given userspace address range in. + * Fault in userspace address range. */ -static inline int fault_in_pages_writeable(char __user *uaddr, size_t size) -{ - char __user *end = uaddr + size - 1; - - if (unlikely(size == 0)) - return 0; - - if (unlikely(uaddr > end)) - return -EFAULT; - /* - * Writing zeroes into userspace here is OK, because we know that if - * the zero gets there, we'll be overwriting it. - */ - do { - if (unlikely(__put_user(0, uaddr) != 0)) - return -EFAULT; - uaddr += PAGE_SIZE; - } while (uaddr <= end); - - /* Check whether the range spilled into the next page. */ - if (((unsigned long)uaddr & PAGE_MASK) == - ((unsigned long)end & PAGE_MASK)) - return __put_user(0, end); - - return 0; -} - -static inline int fault_in_pages_readable(const char __user *uaddr, size_t size) -{ - volatile char c; - const char __user *end = uaddr + size - 1; - - if (unlikely(size == 0)) - return 0; - - if (unlikely(uaddr > end)) - return -EFAULT; - - do { - if (unlikely(__get_user(c, uaddr) != 0)) - return -EFAULT; - uaddr += PAGE_SIZE; - } while (uaddr <= end); - - /* Check whether the range spilled into the next page. */ - if (((unsigned long)uaddr & PAGE_MASK) == - ((unsigned long)end & PAGE_MASK)) { - return __get_user(c, end); - } - - (void)c; - return 0; -} +size_t fault_in_writeable(char __user *uaddr, size_t size); +size_t fault_in_readable(const char __user *uaddr, size_t size); int add_to_page_cache_locked(struct page *page, struct address_space *mapping, pgoff_t index, gfp_t gfp_mask); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index c5b2f0f4b8a8..2e07a4b083ed 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -191,7 +191,7 @@ static size_t copy_page_to_iter_iovec(struct page *page, size_t offset, size_t b buf = iov->iov_base + skip; copy = min(bytes, iov->iov_len - skip); - if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_pages_writeable(buf, copy)) { + if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_writeable(buf, copy)) { kaddr = kmap_atomic(page); from = kaddr + offset; @@ -275,7 +275,7 @@ static size_t copy_page_from_iter_iovec(struct page *page, size_t offset, size_t buf = iov->iov_base + skip; copy = min(bytes, iov->iov_len - skip); - if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_pages_readable(buf, copy)) { + if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_readable(buf, copy)) { kaddr = kmap_atomic(page); to = kaddr + offset; @@ -447,13 +447,11 @@ int iov_iter_fault_in_readable(const struct iov_iter *i, size_t bytes) bytes = i->count; for (p = i->iov, skip = i->iov_offset; bytes; p++, skip = 0) { size_t len = min(bytes, p->iov_len - skip); - int err; if (unlikely(!len)) continue; - err = fault_in_pages_readable(p->iov_base + skip, len); - if (unlikely(err)) - return err; + if (fault_in_readable(p->iov_base + skip, len)) + return -EFAULT; bytes -= len; } } diff --git a/mm/filemap.c b/mm/filemap.c index 1293c3409e42..d697b3446a4a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -90,7 +90,7 @@ * ->lock_page (filemap_fault, access_process_vm) * * ->i_rwsem (generic_perform_write) - * ->mmap_lock (fault_in_pages_readable->do_page_fault) + * ->mmap_lock (fault_in_readable->do_page_fault) * * bdi->wb.list_lock * sb_lock (fs/fs-writeback.c) diff --git a/mm/gup.c b/mm/gup.c index 52f08e3177e9..e063cb2bb187 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1681,6 +1681,78 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, } #endif /* !CONFIG_MMU */ +/** + * fault_in_writeable - fault in userspace address range for writing + * @uaddr: start of address range + * @size: size of address range + * + * Returns the number of bytes not faulted in (like copy_to_user() and + * copy_from_user()). + */ +size_t fault_in_writeable(char __user *uaddr, size_t size) +{ + char __user *start = uaddr, *end; + + if (unlikely(size == 0)) + return 0; + if (!PAGE_ALIGNED(uaddr)) { + if (unlikely(__put_user(0, uaddr) != 0)) + return size; + uaddr = (char __user *)PAGE_ALIGN((unsigned long)uaddr); + } + end = (char __user *)PAGE_ALIGN((unsigned long)start + size); + if (unlikely(end < start)) + end = NULL; + while (uaddr != end) { + if (unlikely(__put_user(0, uaddr) != 0)) + goto out; + uaddr += PAGE_SIZE; + } + +out: + if (size > uaddr - start) + return size - (uaddr - start); + return 0; +} +EXPORT_SYMBOL(fault_in_writeable); + +/** + * fault_in_readable - fault in userspace address range for reading + * @uaddr: start of user address range + * @size: size of user address range + * + * Returns the number of bytes not faulted in (like copy_to_user() and + * copy_from_user()). + */ +size_t fault_in_readable(const char __user *uaddr, size_t size) +{ + const char __user *start = uaddr, *end; + volatile char c; + + if (unlikely(size == 0)) + return 0; + if (!PAGE_ALIGNED(uaddr)) { + if (unlikely(__get_user(c, uaddr) != 0)) + return size; + uaddr = (const char __user *)PAGE_ALIGN((unsigned long)uaddr); + } + end = (const char __user *)PAGE_ALIGN((unsigned long)start + size); + if (unlikely(end < start)) + end = NULL; + while (uaddr != end) { + if (unlikely(__get_user(c, uaddr) != 0)) + goto out; + uaddr += PAGE_SIZE; + } + +out: + (void)c; + if (size > uaddr - start) + return size - (uaddr - start); + return 0; +} +EXPORT_SYMBOL(fault_in_readable); + /** * get_dump_page() - pin user page in memory while writing it to core dump * @addr: user address From patchwork Tue Apr 12 05:15:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9847C433F5 for ; Tue, 12 Apr 2022 05:16:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241873AbiDLFSS (ORCPT ); Tue, 12 Apr 2022 01:18:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347740AbiDLFSQ (ORCPT ); Tue, 12 Apr 2022 01:18:16 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F9E6344EF; Mon, 11 Apr 2022 22:15:59 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23C1kKng012649; Tue, 12 Apr 2022 05:15:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=dynU+Cucx7gZAaChk2IH4uYi39A59mGDfxPi3IpHO4NQqsKsGHwa+ZP9M+PRvFBLBynA drtoSPIgxaDPKrAqR8UulXddtIBxZcdf+3rXMLUsPUmLmipIvKezInqwMQQiH6ZywAWi lDGZVz7M3cvUSkrHIHAgTn5w34XoTbfj/+Qvh/FngGuoRzP35H7Y5NluULr2rh2RCjTY GePjKb9RUvh2Vxa0KPPU+1QppSpdys31/pkvufFy56tW9vfXWlnaRRYNo6HE9CtjEYuX DM4sXGySgf6WsuT/A3scpcFyLsLS+IPsWDYvL5evBDgXn3TZe4/mEvKEIXCNq1R/RR5L CQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb2ptwnrv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:15:56 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5BErB028116; Tue, 12 Apr 2022 05:15:55 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2176.outbound.protection.outlook.com [104.47.57.176]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k23u9r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:15:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ArSqu+/sp4cQp9pucCPXPtSKrqNkM146bBJbheIGnAX3+6CSVBktQqGCTlXzdWES7BdnuM2mIPZkyxSlIqwxZ6J1jUBsTEH6DfzfDNS4ZOfwSlnL1p5zRkI7J1Ybnk7+vrDnEjTHxXbr+4Cfdq+P5AmRoVXBMN4wxhUKqNy8W4BeUSXcJqdg37JG0ieu7l9arSAEbssmicEkgyMbh9zmZePpvgnqIASI1K+cQVOTd1UXLvECat89kE9VdtSilxm1SK7ywukTqz9vF5U0MPTTBZGe1rd2lHEpZQNhV/SqJ1xWExd8bbRetSIGSr5Ee8cxk+btLem/S9sb2MdMC1Ofvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=WS3E1430vwApx51HiLg3rCITYfVt7PPQB0K/ieqnP7DX556jmR6x1NOeDMN9YC+qXWjhN19DDXSSawLNYRY+8q57lrvppQOqMVBPHWfhBGAL0IExoKiDjX98GPWeLAqfmGxl8Hu8DOFRntY66CvkiLahQluiM3ieDMrF0sHZK430DN4ynY4f0t2ybNev+9HZnzq1XQjsEOdBAwJVFxw98AROMsnZE59G2iPmeT5UaKzWAsB4mXViOWIsDsLTf/hbSw6yuDZJojpJDLG7pSopsGRnP3ds+Uww198R6kANYx/BXSyD4bE+eOWfANt/8HjCNXEZXbExJ7sa5vlneAGPxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=ewIwS8DktWjeeRDa33SPndPZgKthg9Js3eKEi3pYOoqjEu0pW3KnkvfdXxbsAIG8HZfM64vy2RODKwcJ8QAgUetRy7bJ0XItq9rFdwjMkNzLGTZVMUDUWyez1DXz+tWTzCgi0EURG26riXqdMLEM9Swp1dbunx6Ge87+bG41nvM= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:15:53 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:15:53 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 03/18 stable-5.15.y] iov_iter: Introduce fault_in_iov_iter_writeable Date: Tue, 12 Apr 2022 13:15:00 +0800 Message-Id: <567e96aa787d4f03add03832ae0a2a28bef0904d.1649733186.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SG3P274CA0003.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:be::15) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: de93bfe4-b22e-4828-3cc3-08da1c43841e X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8uP5GKl3L0YpDBQSZ8MKhUah55iRv+APIAkX5VVMvjhGl0IiYKAoPB8lBeE7d2zTr+retq8PusI4ggNGzSWOkPcwwXRVelbfbawbC4Fcvnxxr60qPwrAda/H4gc0NZQhr3qbSd9v8YqVvggWgZKExH+Rtpi8znF6mFZV0027kO3kVSaUCjOiFRUPOYiY4CLTpkU2NeNP3j2Qzwaz0XhEVHv4VAj4eB0RENeiqiM3wdCuJ0VQH8F56Ry6yi4ExX+6CYpAToEcUJLxRHfP7fOErXz9iYTqz020yMtAWdUKqOhWwW71FiwQm0RjgTJ1hxVyyGrKPPc4dHLFCRqBg18Q97FWKxCkbgRPvONRkX+0kOOIj7/02TcF+OHm6FWkXqlcgnYMU/f+tOIkw0dJ/3azgwiUM5j4YaRYXqLe64xQAvyif8SqJkAAmG3Gr5h9+k/wit4eJtTzHRftAeU1ueSpd0tZKszC5nGjIsz0GCPzEOpK4Ihyxu1x8Yf0pk8tkmk6kR6WW5e5oCivD0sEl5yjWus5ycbA2FnwJ5rTgv2Mx0Xhm9hFPiT1QvNMHI/JqIskaCwK6ZNuDiYxl4FF8y1hWZL8hmVVI4qtpaeOECuwPOc3cITLBLzYSdxGc0PYybe+QJSIBLw08jIRdgakBfxEXg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(83380400001)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YD2wVKgBO9GQjS3BbXYosofxXkfzqtMk8vF9RhGyCcZ6qilyiTa1LZviR3w14/CdxgDPTFFqrv5LwCVHqkXsKo4cYHNhsW46WI6lOEktHPNGtNGEwZeqJvxl7Tw1V+ti6JMRr6qSkkqZ3LOuaFUgYQrKEbhpnpXqhoAEm8vjvi1K5KTQkwKeOc7l0/kqGPPSrnFAcm1QsfSmx1Ch1FCunIwoRxldZiVyFYUrDvVP0NzjlxBxTXBgA7LWP8KL/YSyAhS1NKOeB+NJtTvao+c6vHako6sGdHqNIwwLPBhlVg4sFzUZe5v528k58n9b8HOHkSkwjCcuCyIrhr70j4iECE0jJ/FTQXvUo+vc2lHvIaWUAF1cnKF4z48a7Lk7NXJYEVim+00iiXfeu3TgQf5/vF66T7BtVGizVRiWy68aq3dR1vRpMJEIQJvkwyvBf8sms/daidTB53uHmkIZklpLOhti7+qz0siOv0xrh708s0RZouLsjPNe/M80vROfblG5JeC/a3ppX47v8QfXNX/4wTxOKIY2XIsceE4U5zAyWMeaigctocw+tHOl1OrlgDrNJzuAZx5khbuHQEWCourBzfYECrCkQ/a2trtK0+V5tFTk0BfXCQQ6VDQTB1BV7uUGgcv3JtZZ7aVth0Qh7fgA4w/ca3w8bMnPJBFOFrU4d2VDVDi0OsXcw9It75MVoGFHxK139WsT8A3dETJrXthJVsvhQzNRN2tA+g4WiGnOhf3tdEGDbfNcNw6JFopDhOyuSnSZ4HyIhqnNBq/rmyDZlLBf5QvIBhHuRaGFQweMqKstilUII7J6qhNIRi9vJiSeEAc3H4m8f0AFqVxOXwNC3zIoSLR7urVk9vSouianS5RrD8fqqKHg22t53zqxtCjIRVnP5q9iLcVkZANl5aoXnI3zUKdgLlTR5GT+ZFG7s1RdwxW5G6vhDbNe0WEAOvCDi6l3pFuVuWXnMZXRCjZ5BgGszRApUkwEyBpQVs3DMCGw26gilPIaGZFe5wddrqPLHm3v+gvLgHq9DuEZWodbS5WH8WGtglT1d9YHCyiU4WchGgsUTypndf7WX3MmZG1mgX0e7WNnhhh+ipfaBuLSq5HjRFqT8fPFECkLNgcwiF7OfetjY91AC+mjXEbArs7GcRDqe1cPhpq983PpGIItp0LXQIqmTwz3sotbUc3Vac3NCwibapbV15QdzkSO0PFwpgFVRSF+O53Wx40Ajmt2Sv77jZKE9ktVPAj5MRipViETPzX3HgYPlTFYLikIZS4l80cPr5N4T/uU0dRpxDVUAuWNn+ZgAHcuPeiaADP/N/rdkfqUaCsecyvodZasbWO91jG5/oeQwUoAONJ9TP3P8li/7TO8He37MTgs/ntz6cb08bF9vO8O7Tduydjy9ztzkjC0UHe24GVbRv0afuljEPBGCGCR0dWTjoE4HoMSe8/2oS06nPkqLk2WfQ8nuJrPxzPBDqFRwdEd6cY/6nHfKfXrBgvsBP/P1cEi7FU8c5iomfU/Ove2BLXWUbNZbk/sab5tCVFgsylUurcMpezkMxjrF2rYa3KI3epVumfAV6+gHs+pREw+XrCS+zr8cAEULRwmNt5gCwlJ/ka8+tOofPawJLxXZhIaqrqpTllB49peKhhTn8VTR2CUnjkmrIyGENk3nZ3FN7TUR6E+SBDGp4LUiz2+agPpFPqJxP/Z5CfY8JfTA4SG7rRW4UoUHf1pPTsqL6clseIgEcTNkNu3io/ARvUTViCu5rknXoiP7mDMEOL6VIY2LH9qJg6cnlLx X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: de93bfe4-b22e-4828-3cc3-08da1c43841e X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:15:53.4275 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7SM+mMZoZ4uXbR3xCUzNB3friBJFTmrYNwa18+rpjKdswTOAwxT7p91SKkAp7a+SJq4CZVETeGFe4BgRF365SA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120023 X-Proofpoint-ORIG-GUID: BEbe4HcuV3n_0S-dTfmMwqsn_v3yofSN X-Proofpoint-GUID: BEbe4HcuV3n_0S-dTfmMwqsn_v3yofSN Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit cdd591fc86e38ad3899196066219fbbd845f3162 upstream Introduce a new fault_in_iov_iter_writeable helper for safely faulting in an iterator for writing. Uses get_user_pages() to fault in the pages without actually writing to them, which would be destructive. We'll use fault_in_iov_iter_writeable in gfs2 once we've determined that the iterator passed to .read_iter isn't in memory. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- include/linux/pagemap.h | 1 + include/linux/uio.h | 1 + lib/iov_iter.c | 39 +++++++++++++++++++++++++ mm/gup.c | 63 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 9fe94f7a4f7e..2f7dd14083d9 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -736,6 +736,7 @@ extern void add_page_wait_queue(struct page *page, wait_queue_entry_t *waiter); * Fault in userspace address range. */ size_t fault_in_writeable(char __user *uaddr, size_t size); +size_t fault_in_safe_writeable(const char __user *uaddr, size_t size); size_t fault_in_readable(const char __user *uaddr, size_t size); int add_to_page_cache_locked(struct page *page, struct address_space *mapping, diff --git a/include/linux/uio.h b/include/linux/uio.h index d18458af6681..25d1c24fd829 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -134,6 +134,7 @@ size_t copy_page_from_iter_atomic(struct page *page, unsigned offset, void iov_iter_advance(struct iov_iter *i, size_t bytes); void iov_iter_revert(struct iov_iter *i, size_t bytes); size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t bytes); +size_t fault_in_iov_iter_writeable(const struct iov_iter *i, size_t bytes); size_t iov_iter_single_seg_count(const struct iov_iter *i); size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, struct iov_iter *i); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index b8de180420c7..b137da9afd7a 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -468,6 +468,45 @@ size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t size) } EXPORT_SYMBOL(fault_in_iov_iter_readable); +/* + * fault_in_iov_iter_writeable - fault in iov iterator for writing + * @i: iterator + * @size: maximum length + * + * Faults in the iterator using get_user_pages(), i.e., without triggering + * hardware page faults. This is primarily useful when we already know that + * some or all of the pages in @i aren't in memory. + * + * Returns the number of bytes not faulted in, like copy_to_user() and + * copy_from_user(). + * + * Always returns 0 for non-user-space iterators. + */ +size_t fault_in_iov_iter_writeable(const struct iov_iter *i, size_t size) +{ + if (iter_is_iovec(i)) { + size_t count = min(size, iov_iter_count(i)); + const struct iovec *p; + size_t skip; + + size -= count; + for (p = i->iov, skip = i->iov_offset; count; p++, skip = 0) { + size_t len = min(count, p->iov_len - skip); + size_t ret; + + if (unlikely(!len)) + continue; + ret = fault_in_safe_writeable(p->iov_base + skip, len); + count -= len - ret; + if (ret) + break; + } + return count + size; + } + return 0; +} +EXPORT_SYMBOL(fault_in_iov_iter_writeable); + void iov_iter_init(struct iov_iter *i, unsigned int direction, const struct iovec *iov, unsigned long nr_segs, size_t count) diff --git a/mm/gup.c b/mm/gup.c index e063cb2bb187..bd53a5bb715d 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1716,6 +1716,69 @@ size_t fault_in_writeable(char __user *uaddr, size_t size) } EXPORT_SYMBOL(fault_in_writeable); +/* + * fault_in_safe_writeable - fault in an address range for writing + * @uaddr: start of address range + * @size: length of address range + * + * Faults in an address range using get_user_pages, i.e., without triggering + * hardware page faults. This is primarily useful when we already know that + * some or all of the pages in the address range aren't in memory. + * + * Other than fault_in_writeable(), this function is non-destructive. + * + * Note that we don't pin or otherwise hold the pages referenced that we fault + * in. There's no guarantee that they'll stay in memory for any duration of + * time. + * + * Returns the number of bytes not faulted in, like copy_to_user() and + * copy_from_user(). + */ +size_t fault_in_safe_writeable(const char __user *uaddr, size_t size) +{ + unsigned long start = (unsigned long)untagged_addr(uaddr); + unsigned long end, nstart, nend; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = NULL; + int locked = 0; + + nstart = start & PAGE_MASK; + end = PAGE_ALIGN(start + size); + if (end < nstart) + end = 0; + for (; nstart != end; nstart = nend) { + unsigned long nr_pages; + long ret; + + if (!locked) { + locked = 1; + mmap_read_lock(mm); + vma = find_vma(mm, nstart); + } else if (nstart >= vma->vm_end) + vma = vma->vm_next; + if (!vma || vma->vm_start >= end) + break; + nend = end ? min(end, vma->vm_end) : vma->vm_end; + if (vma->vm_flags & (VM_IO | VM_PFNMAP)) + continue; + if (nstart < vma->vm_start) + nstart = vma->vm_start; + nr_pages = (nend - nstart) / PAGE_SIZE; + ret = __get_user_pages_locked(mm, nstart, nr_pages, + NULL, NULL, &locked, + FOLL_TOUCH | FOLL_WRITE); + if (ret <= 0) + break; + nend = nstart + ret * PAGE_SIZE; + } + if (locked) + mmap_read_unlock(mm); + if (nstart == end) + return 0; + return size - min_t(size_t, nstart - start, size); +} +EXPORT_SYMBOL(fault_in_safe_writeable); + /** * fault_in_readable - fault in userspace address range for reading * @uaddr: start of user address range From patchwork Tue Apr 12 05:15:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E915EC433EF for ; Tue, 12 Apr 2022 05:16:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236896AbiDLFS2 (ORCPT ); Tue, 12 Apr 2022 01:18:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239534AbiDLFS1 (ORCPT ); Tue, 12 Apr 2022 01:18:27 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F10F1344D5; Mon, 11 Apr 2022 22:16:10 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23BLtMIk029741; Tue, 12 Apr 2022 05:16:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=XShKdXazBzc8whg4ha4eM7lV02CWXU19Y963VZhMBg9KsjQh3WHGqOHZDdzQKUVvPjlc xGdgoW+leg7PEFkp6HRnbpqFvFZQpyfLGF6g91x7ysW1uDE6oSpInKsj07Dfmv3sica1 sZOZ68U4MO0aUMXSyoh6r8ZkD6ez0Bde6Ojhv9z6p1dXvZHVV93rSXWyMKJROUf3Y/kL CVYuZeoLNPwencUlGIyVn8llSRsSYhpu1iki8+UIu7svlA5vWt7occXIMgig1wpf4V/L qwji18jpJnjyiuI4OriSWEtwh0WJce7xcWyGLMVtTZTSZ9BdIKWdFbEVr+1Mot5RZCwN qA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jd5g2d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:08 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5BO6h006109; Tue, 12 Apr 2022 05:16:08 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2170.outbound.protection.outlook.com [104.47.57.170]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k2cks3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FRHfxAF+tcWTGm6Y6J/gkNoq/BPVQZj8XKZazC2ywEcSrxnKeUZ1otIMwyeJEwoc30rGwXYhlA2ibo1SjXVqynDzhe4/RYiHklg4S/dnA9bAIrQNV5caplqkx9bqIP3ZrqfA6RZWKHEWUwCElbAbCWg2j6KmyLFoS95SfbnA5BrMPB5DZyx9CNfZsIgW/i9vFDAh4GNOg9auBL5LNBkFfc6fMawE8qw1n985TBwXiZ1BNw2C3/aNlRYt1XErqDBEJ49nOPa8I/v9Kl8m9eMkA2Xi3Xpre0PrBViplNL0Bgeme7UKRCMFYFtkyRnRZkYhC/ksoIyY2VnOO+mteKk8pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=dxlIQmNzhmKDQZusaNFP7OihFI7VCUBZ5zuzXxANiFI8QRO0sXe3RC19y9slMKSHdWS0o0B89UlHxrzPLruAKFJ4WvZ6Y5B0O6yn+XRd6uOH3Ozai6ZoGtLXSgPUKy8mHuKPSiJ1stluYd20qsnuczkgxyaiAFRpB/2HpBtAhbW0q8zS+Uu2/phSGlAMQM8mW5tJFXRSvLQmaX0IbKGB1vj/gwMnyGDDGIb+4IUVMj+I9osPOiC6qdXx5zv248VO2aRm0yTeSQzQzxuZfGYu1rKd4bOHKs+gF2I37VGGFBUiP2PvIlUWlBRP/B496p2JvbW5gBhCK87MEktM2ogykA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=esrrEdRWHqtISEb6/rFeTB6UPSN93ir7Q2ObLYFa/JH0DPKmPXC1LNoHgYo2kNCbWNZQgRkjzXtwg1tEfeCF7EFtv7D/J29hq8nGigicI8txm+E65Rqr97c/F27qZgpQ17ZaFb6+ns0yuVT7VCbvIclecevX5L4u7uTOATyXMrA= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:16:06 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:16:06 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 05/18 stable-5.15.y] gfs2: Clean up function may_grant Date: Tue, 12 Apr 2022 13:15:02 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR02CA0030.apcprd02.prod.outlook.com (2603:1096:4:195::17) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ecb785dd-e195-4358-501c-08da1c438bae X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: n/yaZ6A3hz5wZUiepIpVjwuy+cSuh0w9s2fuT2JMiD6tqp30GoV2Y2wFGazv3lnn578Plfnn4rHigRd516PAKOa0AolBN3EroP2VlcVmVbZ5JeJzQArpKCbYcNtnLBVpIXr591X+kX45wyIOsAvZCBIh3lJr8zWoqGNMHTbasu/7aNscd++SZsGWKws9uoqMnsB3E/SkfYYep/iApK090NG2RhmcR1gwMzV4QjVeWnXXB7595GsREXKg/kM6dCQ8jJz7Nvqx/XyI6cuXH47CUSmd65rBxRxalMR07Vl84mWVsiSvPnGKx5oSGrAMiT5jw+7FaBgotUKDJXW47Bx/0dSAvsaQWpGc9Q2IGX84BgipkTKAtw3yGoEN1SNw2bPB+Ry05T8VoPHbyTFuyC3KssunJt6fQUI0lDlkTnjS+hi0Ylkek3t3kj572uBp1Gm365nsaB4AtrotjaLgCLmQsrQ0cwATg0upG0F44INaWQPZpR0IHtdT/glCXpHVTQRQOeqI/xKlgAFeFbAlZmEiMrcPkX+urtBHRQTDcQYZPXOvfeaLPAb0xXO1l1Tn3GvpeQMZuD9P84pvCbjOZTFLiS0rXLRQjDbcCTAycv3h4/kA4QyoThy7vq0hTLLlxd5fXnfKcyctcucxZy+Chm0LOw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(83380400001)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: lxAk5Pb5JElDTRF2uFBPDqjx3I+SJI7tUz4TzqBWqN/pQhJTb2afrIQZB+e9Dr2tGuKPQgO5uL2XGMM3Qn/WqMR5z+Iye2Oh18LxpRNVw3Q4VBZp5D4rlKP/1RMOdLZM/Zbv5Oq6TjRuu8EXffgvsC5YxIzYyLeVvoc7BxrPg/KCTIW5CSFqJD6Lq8+qJGYULqkMDFDGwZFyNail/lOefd0lHlDh67uKGlpAyhozU86MO7seV9NteCCn3L6Cb700d0zpPXhqsNE7cSrjc4/1Ra0QolIbK9tUal03cBNfNwR3feVjO4MdSagObOqUou+ZxSe4fNd00L4mXPgbYW5F/va2iKFgGpuK6nilt95e1o8lckHCPMLf4zL4VMA8zHxIuQ1w0ZOsiW/CMoDz2p/0kCTCLpO4svfk77IYWWi4PEQkT00T86BWH149V3phB4fh/jS4Bryiubles7YhflBgdF1TH9NHxrdOpAVP/+RPT+MQTaPkhF9ZHdMAsiS7aJj/wAfNPcsd++De1bd69sE+85o/Rq+1qxTDFmJS6iyb8GzH3nt0MRvt8yJfc/oatqQ/Zaj7Qn4//kyLmOug2ndeU6/ygl+5KhnylcI0cLFCvhC31EiNh++k3JL+fFpHivHzmDWXVv/oDF6rZneJbJfCDugeoxW77A7Bcxlh3OHkQ6GrmH+h5NGSbBf+3ba0NBRoSRuJWJJ33o0Td5p63Ntj0tfSWxXFJSJlKOkKmBOBk6DjgD1cKNLBQqn/hvvGHuvzQBoYkMEKbTgTxhM6Ifk0Ae16nSDhZQYCtNVIvSvw7cRpybhA2mw5PcCksTQNfuRhHeOuACN4/YDWL0I1iBLIDq7SdVjjhwehR60LqyYLrMG/dI6MlxlIgczuA0MHvfAl1WuPbv4K/BSzZeAvOcgJnI7GdokXr7f9tZ7LTOWTLcvaeLHHS+n7OsdOTVvqskXKXe+fi0ci08g011l01wXHai0O3+dH0In7koKggtyOVwIGx2XBnFhwa6TMMrxzHJEIjV2cAJUE97ZHX3EKmc2hlIATDh9biOGFHq2Tj5AXIKXWhJZWdW5BeL1OYLi3Gtfn84EUYofSziwXYE4ZOTr4slwu5wwSKuhxSqvVopEyJTZa8MuAhYZ2xHhb08qp5eLwSSPEDal9ZYvlJcccfxwD1hLA2h0aVVhM2IfBF7cVmFk82MuEBZkZ+H3vCiPU8EUkjM1tE6G3+RPI/ZdKbXNeWyO3utC8VoHnzL185gdjDEPmEKLfgyKbvaD0yKAsvrjql9p4kZ7wl4j0qR63zR6RecaDF8BgDrrslV3BTE623Obrw4GyyeZoktCOA6MeBFEx6vrPMJZZ+2GHKqjKSFFYFe+ZWfSaiKa/CBLEXcooep/4oZNsmE/lTw3RPw3mM2coL+YpU2pL34Yy18h9+WgijLHa8PV9moVAzwVynPx4SmA+vvLxhVJHGSnwWHn8MHPia3wE5d8lKXtf1KFgSN0Py7J2YmeCCf/mPe5ADaoWb5t7RlwDIMkitgo864Dsn0mpNf68uonsbGGhCTbSVnvrhxurnhm2jcku3mhG9ofP4NjA1OLJPgESivBFNS2Pet0mA3m5p12ImuusqYLAOTqHEKnBbVZ7TUKWV6rrQVxGGUKx+ot1GigCT52zE2Hk/cj80Bllq+YQhdqd0Tn36iaw5R9XDGU+YmM8ez3Luc0t/hNaTJX9RIIQpc35DPgBT5m3CToYf+If2VpDPSMNgOT4TmiFzNkSGafXcvHSWIPE+ZlLf7w/nVhEiP3sxH2lgJCM X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ecb785dd-e195-4358-501c-08da1c438bae X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:16:06.1001 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aiRUd47xagjMKsAqsowiUXLG9iI4+JwM3RMnYNUfQZKO/Vmt9D2OEKaeMZXzCmnFLjuwFkH1dBkNN3IKKV6eJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120023 X-Proofpoint-ORIG-GUID: NtsUGfmi7VjbifBEkJMwUcgvMnjlb-cn X-Proofpoint-GUID: NtsUGfmi7VjbifBEkJMwUcgvMnjlb-cn Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 6144464937fe1e6135b13a30502a339d549bf093 upstream Pass the first current glock holder into function may_grant and deobfuscate the logic there. While at it, switch from BUG_ON to GLOCK_BUG_ON in may_grant. To make that build cleanly, de-constify the may_grant arguments. We're now using function find_first_holder in do_promote, so move the function's definition above do_promote. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/glock.c | 119 ++++++++++++++++++++++++++++-------------------- 1 file changed, 69 insertions(+), 50 deletions(-) diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 02cd0ae98208..8f30ad956270 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -301,46 +301,59 @@ void gfs2_glock_put(struct gfs2_glock *gl) } /** - * may_grant - check if its ok to grant a new lock + * may_grant - check if it's ok to grant a new lock * @gl: The glock + * @current_gh: One of the current holders of @gl * @gh: The lock request which we wish to grant * - * Returns: true if its ok to grant the lock + * With our current compatibility rules, if a glock has one or more active + * holders (HIF_HOLDER flag set), any of those holders can be passed in as + * @current_gh; they are all the same as far as compatibility with the new @gh + * goes. + * + * Returns true if it's ok to grant the lock. */ -static inline int may_grant(const struct gfs2_glock *gl, const struct gfs2_holder *gh) -{ - const struct gfs2_holder *gh_head = list_first_entry(&gl->gl_holders, const struct gfs2_holder, gh_list); +static inline bool may_grant(struct gfs2_glock *gl, + struct gfs2_holder *current_gh, + struct gfs2_holder *gh) +{ + if (current_gh) { + GLOCK_BUG_ON(gl, !test_bit(HIF_HOLDER, ¤t_gh->gh_iflags)); + + switch(current_gh->gh_state) { + case LM_ST_EXCLUSIVE: + /* + * Here we make a special exception to grant holders + * who agree to share the EX lock with other holders + * who also have the bit set. If the original holder + * has the LM_FLAG_NODE_SCOPE bit set, we grant more + * holders with the bit set. + */ + return gh->gh_state == LM_ST_EXCLUSIVE && + (current_gh->gh_flags & LM_FLAG_NODE_SCOPE) && + (gh->gh_flags & LM_FLAG_NODE_SCOPE); - if (gh != gh_head) { - /** - * Here we make a special exception to grant holders who agree - * to share the EX lock with other holders who also have the - * bit set. If the original holder has the LM_FLAG_NODE_SCOPE bit - * is set, we grant more holders with the bit set. - */ - if (gh_head->gh_state == LM_ST_EXCLUSIVE && - (gh_head->gh_flags & LM_FLAG_NODE_SCOPE) && - gh->gh_state == LM_ST_EXCLUSIVE && - (gh->gh_flags & LM_FLAG_NODE_SCOPE)) - return 1; - if ((gh->gh_state == LM_ST_EXCLUSIVE || - gh_head->gh_state == LM_ST_EXCLUSIVE)) - return 0; + case LM_ST_SHARED: + case LM_ST_DEFERRED: + return gh->gh_state == current_gh->gh_state; + + default: + return false; + } } + if (gl->gl_state == gh->gh_state) - return 1; + return true; if (gh->gh_flags & GL_EXACT) - return 0; + return false; if (gl->gl_state == LM_ST_EXCLUSIVE) { - if (gh->gh_state == LM_ST_SHARED && gh_head->gh_state == LM_ST_SHARED) - return 1; - if (gh->gh_state == LM_ST_DEFERRED && gh_head->gh_state == LM_ST_DEFERRED) - return 1; + return gh->gh_state == LM_ST_SHARED || + gh->gh_state == LM_ST_DEFERRED; } - if (gl->gl_state != LM_ST_UNLOCKED && (gh->gh_flags & LM_FLAG_ANY)) - return 1; - return 0; + if (gh->gh_flags & LM_FLAG_ANY) + return gl->gl_state != LM_ST_UNLOCKED; + return false; } static void gfs2_holder_wake(struct gfs2_holder *gh) @@ -380,6 +393,24 @@ static void do_error(struct gfs2_glock *gl, const int ret) } } +/** + * find_first_holder - find the first "holder" gh + * @gl: the glock + */ + +static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) +{ + struct gfs2_holder *gh; + + if (!list_empty(&gl->gl_holders)) { + gh = list_first_entry(&gl->gl_holders, struct gfs2_holder, + gh_list); + if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + return gh; + } + return NULL; +} + /** * do_promote - promote as many requests as possible on the current queue * @gl: The glock @@ -393,14 +424,15 @@ __releases(&gl->gl_lockref.lock) __acquires(&gl->gl_lockref.lock) { const struct gfs2_glock_operations *glops = gl->gl_ops; - struct gfs2_holder *gh, *tmp; + struct gfs2_holder *gh, *tmp, *first_gh; int ret; restart: + first_gh = find_first_holder(gl); list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { if (test_bit(HIF_HOLDER, &gh->gh_iflags)) continue; - if (may_grant(gl, gh)) { + if (may_grant(gl, first_gh, gh)) { if (gh->gh_list.prev == &gl->gl_holders && glops->go_lock) { spin_unlock(&gl->gl_lockref.lock); @@ -722,23 +754,6 @@ __acquires(&gl->gl_lockref.lock) spin_lock(&gl->gl_lockref.lock); } -/** - * find_first_holder - find the first "holder" gh - * @gl: the glock - */ - -static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) -{ - struct gfs2_holder *gh; - - if (!list_empty(&gl->gl_holders)) { - gh = list_first_entry(&gl->gl_holders, struct gfs2_holder, gh_list); - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) - return gh; - } - return NULL; -} - /** * run_queue - do all outstanding tasks related to a glock * @gl: The glock in question @@ -1354,8 +1369,12 @@ __acquires(&gl->gl_lockref.lock) GLOCK_BUG_ON(gl, true); if (gh->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB)) { - if (test_bit(GLF_LOCK, &gl->gl_flags)) - try_futile = !may_grant(gl, gh); + if (test_bit(GLF_LOCK, &gl->gl_flags)) { + struct gfs2_holder *first_gh; + + first_gh = find_first_holder(gl); + try_futile = !may_grant(gl, first_gh, gh); + } if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags)) goto fail; } From patchwork Tue Apr 12 05:15:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3333C433EF for ; Tue, 12 Apr 2022 05:16:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347772AbiDLFSx (ORCPT ); Tue, 12 Apr 2022 01:18:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235348AbiDLFSm (ORCPT ); Tue, 12 Apr 2022 01:18:42 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC02834647; Mon, 11 Apr 2022 22:16:25 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23BMDCuZ003034; Tue, 12 Apr 2022 05:16:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=/iYiU6ya0Q9jSxbTwyxtIX/v7IHM+oZyFAtlR2c047o=; b=a9jMJrt6DkBV7MZ2RyrRK1eeWvZbWBJ5QSm4HqLj5EZqI4o5ZXzKWclp4VKG14X+Htm/ EzkD5yHyRBC2EQvXtHWB5pGK3/2xJ9bvK03bsI9RBMMFIkyTJ+XNXmkgnepJvKWVY1EI Cm41j21LK5X9TaCAir39TqDwjLk9zbd5lsZZ+tGjU6tXziUxUst7zcLzbLoMl41nHbn5 gupDjfdBHUPA/1WnliPz+4nT+RbkiaLFsDDCJFDN8TZTbpeB6pjdtSs/LRUGdoip4LgO CK+si04UImQe4JvsmPFI9UJdoguEsMDzIIs39CJSD4aiGWosWTd+15SFLSydU4o8Kn37 QA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2dmb4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:23 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5GMoH034441; Tue, 12 Apr 2022 05:16:22 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2173.outbound.protection.outlook.com [104.47.57.173]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9h01qp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lcIVzFGOJd6j9hCJWgRABBooPNInafZ4C6b4rgALu5ThpjqKwS3o4BsnfnEaZrwfObtqQlRkcKJ6m9673PCfTIXcn1zu8yA+ZM7B+7I8tr9RFKH41LhmE77QXU2A7hPVXwHZQzBYmNmqk3GgU1GdnH9Pt13boTEMTamUpZjZLTeDwdOjEhFtuH69NvWfvPlgYuQCnvTvAOADQiz6Ex3G1xfeHBMdn5E5WAW+Uk0ggeiaLO/XLxmbssSKzqS8bJ73wczSu/Zns0EFjy/keQ+rTXzHp/hdiVxekZbpv+2t04v8jsB9rJKnHySgcyo5u9HTpOPOd6zu4jlYyd1w0CGcFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/iYiU6ya0Q9jSxbTwyxtIX/v7IHM+oZyFAtlR2c047o=; b=EbDiZznLW9+sTXxrg9Xulfgb9E0L0hPwZG92SUjZnqvODNbWeCFBEqf1wewWySAOtdS/gRTFCRMGPoCZlsokj0+YcNjNrt1ORu7lixrxbZv5LjDHDaoATfrFuJxfB3W10UPEEnRqkVgWKQHIRJTgD9BpIknCRPXd3kmMQbM3NkgsL2CJuF0aJ6zQxvk34753DgXH6k0u4PcwyA2kzNVeiDtqBLXmz/5APE2PWvDDxuv0xvn4w2SYiVQGc587W1lqF4BasN+gmBO9Gz9eYuhXNYd5DmMIDloJmCKTouuWlpOfT5ipCQVjWk/YTC3AxoUlrUHyANx/otDq22h8XMytLw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/iYiU6ya0Q9jSxbTwyxtIX/v7IHM+oZyFAtlR2c047o=; b=Wb7gOn7mlzBWegnSLDRqI+8AWpC2fL7gU8Q4pECtXR6e558iVHY3h7VzT61XeRHlNtPXUZ6WA8gCLAYntCqhE9bEUMxs5C1oA69zGNrqZCGljHsjo3rw7lxaXijqQglOhEGGZhSZDadmVYFkA7omDBw+P757Xjf/8ZNOYs1rFWw= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:16:20 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:16:20 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 07/18 stable-5.15.y] gfs2: Eliminate ip->i_gh Date: Tue, 12 Apr 2022 13:15:04 +0800 Message-Id: <8ef6c4491dde583e6fc460286b96455b3d1671c3.1649733186.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SGBP274CA0004.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b0::16) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 350d5bf1-2b11-4365-7d03-08da1c43948e X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fEG5AzlVVyZb8iw1MmteuB8DoIcPRMVpNuRbmYHHDMmvIDc93kmxmmg78Avkv3pVdESvr1Rvpm0WqDBQ2aBJ7r9+z4Vy2DXTxruCub+g3Ldxcs8lFqcaVck8vxBrmE3LU4qgElK/CqexRHCdjuSfUyLWFty3G5lkyjmkqjBr+U8mI16ND15dBksJKCCoxWwb62TzW2ic2YSz8DIMpa+aeuhe64T0u0cYNkYtZHvZxw8syVnmpxz0xhJsi/m79nGPpQt58KSNOl8lBbXncWUlzi51rSGQmoNh7du5876F7efi4AVvkjR5t929ax5nao4ZQwFqlxUsKkTE7ma63WHdRmSmeRHqQnOaY/rJsyfyq0yt+ZeJBVbvdx/3C1CF4XilEVMwFHZP590xUZMov007s/0o1HPG5bqMKu9Bx00/s7uGeEY6xRrHr755PbJY0kwxSGV2rGM4b+0jNPwGOg2PWKvNmYuKretjDo6frcS1mg4p81agQF68iYFWDhrxFjOYiA+uYAEz1FOOC7mmmZ7Qi+avI9DJrswssKgLPvD7K9E72IS0JIx0uBRd8uE7hnFFyhMF2pZZxnNkkQlIFKgyichS8VLXHmmi+dXNk9HHBd/LYq7aP1QKJJwaEWUo3MXZpzTUD0Pn18rub1Xvgun8jg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(83380400001)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: GL9S+CYYMbqUnUqR9CcATKHzJktEWhuqSK3UD0NfdBDk3Fr9QqqyJZfogVgu3GYgrKUHf2AD9APP4Tc2xE0aAhGm81pK7GDwT/BotqXmY86MBWXOjFM8dZWUmcrtNdNBowBhN/r8qOzp5cxm/6FFWyq8xgVKCGCh1e4P9LIFi0N3Gz8eT4wK1gOImt1h0MwKqTX7fJLv+iart6Xhzpy3J+ZQZFS5S/lKTrvtP9cLKVX96PF0tim43ry7vul+WBt7fkvJM6bGxKGVnZLBBdmNEGmVcjC0ySpdqGY3+LobYF55kyiY66HXoNOENmIesSCc3MDHbOJMZ0oKGKwu0qAydluPF5T0PdldB+hj7BeYtEz4mLOZlYdXX1dOvK0aFKcEnAeGM3tKD/AjsNB2h3t7riUitUWeo4HY9xH2eDqMutvpEb6ijA7DwQRKiG7N9GrqKavWwa2ZssvnWHkreEZ3roY4NyU6hRx38CAb6552ATrGDNF3pjqnM8qdZZ/p1IPE4cTichkpcO0ogWB3C4GMjfTRLW9X9Na/yD3qOKAazuLnNnwDCU3jPuqqEJa107/erfvcOTqGsYP35EVggoOzRwkVo0i0yPr623IHf3lpKxsn48KhqN3fZBv1/Qy3QsZ7okm9yPEYwwhNdoMhOo+xMKPQ+/5VPMY/j4thKoujaaPdcMEFSVE+ppWMSaQoHKK3LW8QxLY3B2KDFezKHoCJ3PToyKuf4J2aNfmBuIl4FXGasg2HMseqgalZb5na5NZeg9iwNvqvUInhcLhBzdp79CMLN9FynDGx4dRJQINdwz3Eeo/aEmyNozn/VNQie0MRbOZUElcYwFjdqmI0y6Dei07SrLW7X8Z0GzBeOU6ykMqgkhlcx3/Tdi777ZNkLIDhOSTRm9yQL6k3AzeywJYYKa7t0p1dg13AYB9V8LcH1vr9u92CHLHzBGfZrBwyxaTHZy9qAOhnhPE8PIPsFeD79+0jTOSQnV4yI0k8NXKYz9AT3VKLcIF2iGH3xk+adZSAETNp8a8KfaVi1QziReBOoaQ9LwHnCkij2bclAj9nFuHdN9GC2YjF77wIrFpsgZQ0wXWVaksv9ounXm7VIdXimlaDlC6OJ0mIaHwW06xUNrhDPzgpTJ/QjfGvy0IfC4eDmZmAfFojWN8VPbNulbWB1KelOxLbFMzbFSmGrfxUSB7DXLZeLQ3Nj8a4eKgsT0U6gTZwLp6ehx5Yl76z+Z5v0GUx9Dw3aHhWZGF/rjMEW4RcCRShlCqRI22bL+dlTXQbM+JINZE2dFsERzcHY0T3l2vuObc9bgXukH4fKeaF8lmlREwBp5xgCy+abWUlyEZSpN1LZZRu/E0RB6wb3/zfUAljXGurjXzalZRHJvQZC9uA0rSPjzxWqKlKPZPUVEw5lf2UFUBNarxZwz7r5NLeKNygJrsWOpQotFA0Q/6lMgg10Z92W/bECu+puuWEnbw3h348pXIXt3t1WFA+nJwUjl1tlL2722lfmObvZFI2P7HTSFFcJx4/ScHr5/rjkmcY3wQVWq98mu4LD4XYzJnKGWLw+b6pktywN2rMJ9aQDqcRdIjg0vb48tQWGtzLkqbaZkEfBz8JaSK7HVQ7EJD6EjWa0QCG1IKvyT3e1WnYLo+B5wxwlqSmANUTP3ij5LnzcY5V8aqDeVQLC9f63QLlk9Xv6T7EQiCLXW4xLXdsD/0a1//8aycOB37Zb6Ag3zkjbB9PkpxF/D8rpDZxARcYJLDpMSm8MRnAlCzHiuy/kKJW/MzWVeD149/sISxZZnrg X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 350d5bf1-2b11-4365-7d03-08da1c43948e X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:16:20.8409 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5Gr9Y3nzNlzrfgjOQgbe5U6+jfLqGfnzyq9nlWU9ONhKzZCPrIF6RkcIYDlA55RPz6ND4cZsB8nXmche2zaFCQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-ORIG-GUID: ayWiFk2uXxJ2jxGA2xqOVIMaM0gyzGUw X-Proofpoint-GUID: ayWiFk2uXxJ2jxGA2xqOVIMaM0gyzGUw Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 1b223f7065bc7d89c4677c27381817cc95b117a8 upstream Now that gfs2_file_buffered_write is the only remaining user of ip->i_gh, we can move the glock holder to the stack (or rather, use the one we already have on the stack); there is no need for keeping the holder in the inode anymore. This is slightly complicated by the fact that we're using ip->i_gh for the statfs inode in gfs2_file_buffered_write as well. Writing to the statfs inode isn't very common, so allocate the statfs holder dynamically when needed. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/file.c | 34 +++++++++++++++++++++------------- fs/gfs2/incore.h | 3 +-- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index f652688716aa..288a789cb54b 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -877,16 +877,25 @@ static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return written ? written : ret; } -static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *from) +static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, + struct iov_iter *from, + struct gfs2_holder *gh) { struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); struct gfs2_inode *ip = GFS2_I(inode); struct gfs2_sbd *sdp = GFS2_SB(inode); + struct gfs2_holder *statfs_gh = NULL; ssize_t ret; - gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &ip->i_gh); - ret = gfs2_glock_nq(&ip->i_gh); + if (inode == sdp->sd_rindex) { + statfs_gh = kmalloc(sizeof(*statfs_gh), GFP_NOFS); + if (!statfs_gh) + return -ENOMEM; + } + + gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, gh); + ret = gfs2_glock_nq(gh); if (ret) goto out_uninit; @@ -894,7 +903,7 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *fro struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); ret = gfs2_glock_nq_init(m_ip->i_gl, LM_ST_EXCLUSIVE, - GL_NOCACHE, &m_ip->i_gh); + GL_NOCACHE, statfs_gh); if (ret) goto out_unlock; } @@ -905,16 +914,15 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *fro if (ret > 0) iocb->ki_pos += ret; - if (inode == sdp->sd_rindex) { - struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); - - gfs2_glock_dq_uninit(&m_ip->i_gh); - } + if (inode == sdp->sd_rindex) + gfs2_glock_dq_uninit(statfs_gh); out_unlock: - gfs2_glock_dq(&ip->i_gh); + gfs2_glock_dq(gh); out_uninit: - gfs2_holder_uninit(&ip->i_gh); + gfs2_holder_uninit(gh); + if (statfs_gh) + kfree(statfs_gh); return ret; } @@ -969,7 +977,7 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) goto out_unlock; iocb->ki_flags |= IOCB_DSYNC; - buffered = gfs2_file_buffered_write(iocb, from); + buffered = gfs2_file_buffered_write(iocb, from, &gh); if (unlikely(buffered <= 0)) { if (!ret) ret = buffered; @@ -990,7 +998,7 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (!ret || ret2 > 0) ret += ret2; } else { - ret = gfs2_file_buffered_write(iocb, from); + ret = gfs2_file_buffered_write(iocb, from, &gh); if (likely(ret > 0)) ret = generic_write_sync(iocb, ret); } diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index 0fe49770166e..3b82fd2e917b 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -386,9 +386,8 @@ struct gfs2_inode { u64 i_generation; u64 i_eattr; unsigned long i_flags; /* GIF_... */ - struct gfs2_glock *i_gl; /* Move into i_gh? */ + struct gfs2_glock *i_gl; struct gfs2_holder i_iopen_gh; - struct gfs2_holder i_gh; /* for prepare/commit_write only */ struct gfs2_qadata *i_qadata; /* quota allocation data */ struct gfs2_holder i_rgd_gh; struct gfs2_blkreserv i_res; /* rgrp multi-block reservation */ From patchwork Tue Apr 12 05:15:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7358EC433F5 for ; Tue, 12 Apr 2022 05:16:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239848AbiDLFTG (ORCPT ); Tue, 12 Apr 2022 01:19:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231492AbiDLFTE (ORCPT ); Tue, 12 Apr 2022 01:19:04 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B38E5344EF; Mon, 11 Apr 2022 22:16:48 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23BMg76K001710; Tue, 12 Apr 2022 05:16:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=dekTfSbvGO0S38A0OuWWp9+8RvU4gBxetFaiOVcOQ/EIgbKiPUPkPne+KWhM2F0wy6je WjCJvttd0kIbmoFJnbiSofNmTU93zYfsM0SrjvaLrINJ2TsIT1RFY05tpcEJdb3lpHbm WjkNzhv5IiaIWKLUIbj37V8Pr2IKtwkLLjNcpJPXERObpMj47c5858fXd/0Mim688ttP fC5S6FxRptBwDSjHOoUpFZo9f50s/TNx1yzHBGy+Z6jQo/7LWFL3N80MBPmJhaqq/PoL T+8lPEbKtIpP9SLCMhckTn8Ev+UWc3ijlkeMqXSYRpgiqxEk7DNRajNqBP7vvU4sEN84 9A== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2dmbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:41 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5GOnN039706; Tue, 12 Apr 2022 05:16:40 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2171.outbound.protection.outlook.com [104.47.57.171]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fck129f0r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mDfIw7j32x3/tmCgTcsmHN0afJzXkcLH//Zy+c41uKbntyYFqxBVwQq6T9PW8kwLr7jaSdSLvqoayETbvRc/1R36CmzEivLrFlSkTXTD4YuGZFOt1vnwSMoOPMFqLOk7e86Mw+Iy4+3Xx0WBfXxL0yhzZoHNvOYj+32IOJshNP/SUJyseyttZG16wvcHPHB0dl1mJeidUo5yTX/pQiiJ1TKq+LMCQnG6ukeiDohBIfc7Dau88lN8QQV2HNoyqd0P2NFdtVvz3vtlCgwF+ONDCrg0moxHvAcH5OlHBoxlmrN+KFFL8FF/K6ypeSPh3SBsn5lmGCaOAQ6hvPGVIN4IWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=dB29fu0IwlTBQHsHirJEVgRrbCp0VoUw+dxhS7puILwUL/2VhT1Fs/66vnTjA0PBBAfZ+8YdfvXL5dPQq0p6oLaRmc8vRRtLtGTqxPFkXkgVgj0ui7BWneHEl9uwqxc002Uuf5Qxlnumc6nEz6idlVuyQLyD6xhLhn/Rig1ntz/9zDw0R+x2Jv1byGypQrPDwnFBbEt8pfg8lFxPn1jtPF4/SmXcMdCJmfYfEHCcb0Z3Vm+SvKm4BY97X5BBUhR4QYMVfkOeeSkCFatTTxgr19r1D1yTz/oBEK3hAUeCvoqtwda32tUCPee0mlHQR6HmTvroi+CEK1hwIB6gr5zyHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=MW57IlOTOINkEWmYXATdIoFcslSrp2W4+ysis+0+ryJkwmJlPrSxTflP5oplPg+KQBJSccU6niXfjjWdoYSA3gbfYl3wHw5lYCDMfQzMpR1nZlX1eWXA6AXHj7VBud9aAgfo2LEzJGaRjVeqIbMrmKlv3v3xufQbKywWd/K2Flo= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:16:38 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:16:38 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , "Darrick J . Wong" , Christoph Hellwig , Anand Jain Subject: [PATCH v2 09/18 stable-5.15.y] iomap: Fix iomap_dio_rw return value for user copies Date: Tue, 12 Apr 2022 13:15:06 +0800 Message-Id: <6407750d70683b7c09596f45e4c11acddb21d53e.1649733186.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SGBP274CA0006.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b0::18) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 59307ad9-83fc-4e68-07b2-08da1c439edd X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: akaFqhGN9xZcdu86H2kO+aCeC3t6upAM9XkxOuTfRtvENP2UNXjdSjsM1Hvob8pt/ReRbUmez2Un8qId+sf8OqHWb3M3qUhiw73qCXwraizDZ9CHNhkOhn/3DQ6BhD3iw8CKjTrO9EJiAwJA7wbu1EY6wsdYaMfwkYePoubAvKVfrf8lzKntwqUpoClWxbTnOkyY3IO7izuxlY5828CY4yq/JVkS/jLl8BwslWfVRaMTCv4P6Q8X/LxG/HSrJDDjmJdpj/1eYlC4+tj5HuIJE4AvIoOMd1bvax7ABKxvLRzLHBRP+iWE1qIA1WqHd3TCSuxQGFo8Z5jC2kkbXKKfrFRabQO7M/6OlsAkMlXLwEDa42TOphnevvu+7Dcj8fwz9Kw4G4g9VMIg1RzaVf7lqVqLAqoS9Ep/or/23INPHJHtHDCR7Gom91pY/rGRm7Ut2Ooo43x8N5AIBLG/LO1F2PmxWrRo2pBsYI1mN/U3mR4w/J1cjwDmBzzkGqiI9EVl2IRFePmkK36fzTk6Sih9sy9O0hpioUUmXQRtfoL10McHmfFQ+7kJhnStR+YB8/5cg2RDXLceBKT4wqBCsVGIgkQATCidnTbsR30sLb2kuavaWfK3dg3arjideGWmQBA5IWrQdYxlucNSggbHdVmDbA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Q0yvE+l3xnISLbU/hAfTJciTo6ciY2/7w2tjgn0wjOx2j69UFw+6wiqgedrcls3HrwefDFJr4tTo1mQriLLzMw8Z3nRcUMXLYt/g8poJ/d3jKQsxXUJXgQSm82uAzgBsdfwjxR8RK8cY4pvLZb4u2trW2L8CD7xp8td7NP3/iZinVyKdf9xcpr6kzzOYP2/hO5ZqDx8JDLjFQSgRQn9qxrcqKKCfn1KamTHRNkc4Hso6ST8/MXjzNHn8JI1YiYOE59NZSgG8fV6lQPaUVT7sZ41LAwmXdktmGlTeu7wBEFHC8ZEamNvb4hqKG7Iw1+Oa2O04wu2JlWEsZaZ2Nm0vWBrvsSaMbiLDenQevKXMXZ5SIDXbdAT11DjXRSvflV3570JLP64aQV0lvnoDux4HdKQaCsbt9Yc2pE0k272IUZ/IdAyay4F/0K4onltcIxvU+YRlw7GHFb2dqwrX47WBGNZ+jbJpkh7/9s9MOBaGSF1eazCoHqPF6JaERM2eFOCeBM3pVHl/4LFLAzenfma+jjmCtO15b3+ZwVa3ACTwvq2CIj6bSgEWX99ms2yBHLLnD8G6jL9hTaVkS75W6NkaVfjehONBtUpoSWBVfuLnngAPrwK22VEC9XAlX6J4jISu5WEatInK5kWm3cxzn40GPzVZ9jUtuubHFqS77g0QWEEi0pB/io/ZUw8j5N/zYdOxeH83yd9d4WAlEkcJ5gl6G+uoQJrgRe7/TcO+O8HRjXV1Nyfj48KOg0iRmGhS4107lHNeL9+P/Tdvc2R4h50Zww5IrIH9hNbvLBGwMzlpdWm42Cc1uN23xuVMZlYuD58cT1Yt4GjoEZ67PtFglw2SUwCRzuHNptdOQBEUOxhQ8cSyrI9wkk6AC6+PEa9rxCff7jb4JK7Na/mIlDcPnK30UcPDRgJPvSzMbx9rZruQhIOrIxetOBCBgmRUAV6DqbQJLxxpK28OKvr6lt/w1BXoaxSfaHFDbsGCNMc++yDWFaZce5PaxEW+wKm56RartXwkZKdf2UZfy3IP8D8Jt81xyWTZqlAz+aT8ujwXc1JetJPMRydtqyWS3T5lzRFPxgJhi+eh4JapfAkzg8tSakpoT6yQ0c84wjuYLBsjmxbyNtTpe+DGQhSgi3v7pQjZZ6tkC4RVrmwfRIn/rM2suZ3v+QlQMd9KM8uv5vmgyninuoodzK+aY+W6q9QiRIwvt/y4siXb6bA/+oVeXHEctIq7qvCkWGndY0CM2sf1LmQW3FznCmKvv0IBFRVLDH8+QabC+g2BKgzw9L1onRerHyW/Znqic2EDFRSNU4pz2L610CY2H6S8M6+8zBPO3tYu0HmPZL6bZ5m6aTy6Qxk/R/nGc+dtyBezVfoHOydjHpNz7uBwEiTwoOZrTcgsZeIP3uwzxSo+nZvOsXy/yHWLMQcnl6WrU6JhSqgiQCNVpu4qFjOiNkqjnti1DBNwEBqh1s07qX7nRf+JpYNVm/g5emg5VLn+ceXStjTMyqhVeSTw/KqSvrTDRHLgnbgbrgfNEcNhxUCkoHBv4vSZ/c0GLk0g3Gyq2Er8kocljitX8PstNm7t4dCSqfHlqY5gjXM7xsXPaxdtSZ9qSGRR6SBbXpjTf7GwF+GW8zyG3keZQFCmfGewua8hdjKT6GK/6zyrBRg+AiZcvGKx476PwfK1sZWgS2xBTox8Q+ONSchiGMjARoUuDCAQCiCyVB+GXMj28vSthG37f7n6c5BPupndERdU7+KXICpqsiX0oKjmAUlrJasqs1yzoGQce4lNM/3qh1Ie X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 59307ad9-83fc-4e68-07b2-08da1c439edd X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:16:38.1112 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Xba9fD41LZT+VHWKgZygJBOH5gQuqq9zAtkz9KTLSH0mn3c2jHsBIn6ZkTfLxCxDJfhIp0qbij9a9OJBRBKS/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-ORIG-GUID: RjrQ2Ur3swc31sU5jluGI3VhtP9Psktq X-Proofpoint-GUID: RjrQ2Ur3swc31sU5jluGI3VhtP9Psktq Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 42c498c18a94eed79896c50871889af52fa0822e upstream When a user copy fails in one of the helpers of iomap_dio_rw, fail with -EFAULT instead of returning 0. This matches what iomap_dio_bio_actor returns when it gets an -EFAULT from bio_iov_iter_get_pages. With these changes, iomap_dio_actor now consistently fails with -EFAULT when a user page cannot be faulted in. Signed-off-by: Andreas Gruenbacher Reviewed-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Signed-off-by: Anand Jain --- fs/iomap/direct-io.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 4ecd255e0511..a2a368e824c0 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -371,6 +371,8 @@ static loff_t iomap_dio_hole_iter(const struct iomap_iter *iter, loff_t length = iov_iter_zero(iomap_length(iter), dio->submit.iter); dio->size += length; + if (!length) + return -EFAULT; return length; } @@ -402,6 +404,8 @@ static loff_t iomap_dio_inline_iter(const struct iomap_iter *iomi, copied = copy_to_iter(inline_data, length, iter); } dio->size += copied; + if (!copied) + return -EFAULT; return copied; } From patchwork Tue Apr 12 05:15:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 748ECC433EF for ; Tue, 12 Apr 2022 05:17:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347764AbiDLFTS (ORCPT ); Tue, 12 Apr 2022 01:19:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344726AbiDLFTQ (ORCPT ); Tue, 12 Apr 2022 01:19:16 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 947D93465E; Mon, 11 Apr 2022 22:17:00 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23C1kKnj012649; Tue, 12 Apr 2022 05:16:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=g9pPeqlZi2yFw+xE/OSM0HtRwaN3sKKJA3lDSwxuZqOCCEC32jgIg4Yr1myYzJMWzq+u lLwth8f9OnokII7vpcBGxJeNvAbwX1GXDiw+0AFgIVzvXgNpCQE4RIpJWiyU6xtCY27w w//06+3x3IuGwy/vhbliLr4C5y0asoAxJVOB5EU6XTLnv8rIGDTMMNQ1xfr4SXHVET2y F9dCw53u1mHzyqvP8Cg63XYAVnpJf8hfeUDn3sEujSQaY+an35toFktmd8Fe/YkHaWaZ qfIE+KgmRcVDSO6TRAaDlq2obmys5Oba0+yIDud87TnMk9L6ffU38vXRsw/X50K2cKuy 3Q== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb2ptwnsw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:55 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5GNcX039690; Tue, 12 Apr 2022 05:16:54 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2171.outbound.protection.outlook.com [104.47.57.171]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fck129f37-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:16:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MOVm7c6/s7Sfq3yd5kpHEdgJmYFX9T56vFRS+6z9pDT4gErxxEEjWK2OtB3RlthqlnTVm20lBo9WeBZQydI7umhPM3/9hOxFkRlxbdhvXbhhsh8rntRLbY0+PixP+ip0cLLBp3B2UUijxj7hHtQqyaRr5YuCwSVyiFNet7+0BxsdrbyGuFRZsjIHPYH80xLg70dmZqHlzhwtA/HgsonRv1QoBu91vhRw81UeNVcXO/sg4aUTqxgtLxxBgIYSwESzTJXV7Md1R6ivtwqhDgJ7kKCTXUIYMhY2cqKq9DfiS5xK/X6qfhefzRStmA74vKHEZsK/G9ZnAx7XiM3TBw946Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=hI/XE78uxCxBorfD2vJpcsYSAadECVBJbXhO/h3M+ULM2Kbps0kJtF+6Es1KJ60qVik0Zyy/B+PLMYw680hZRpp0bpoHE4H63wvAt387mCcOOR8QI7Cpe2Pu9WYNOxxjPXyBoWSP+U/yhzl3NkEl3TO4aCAtY2DFy8yz5nU4fFU05/M4ozoInVqBMHYnkpuX9zt3InCGNlkRkat0Kg0C2jL6EZCG9vwei2W5h9RPvvJkB1+gfuJLfcJL1MACzDGEz9YLsUvcHXlGbKd5hwoyYJmgjudM7g1x15RXqopNe8xjauuzREjlcsLUvaPu3tDzUrkAEbiV1ypRDxrvSHnKdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=pBfDyFRNqdgJ1QRKYa+P9rq3ThE855pvSOX2+Zfd1+syQc6fGjUug+LiZZt7UQHCxjWoSav86AW4sVHx450OPZZ9UYszVGf6zcfnZ9dHaMtf0P5kOwU2rRHuiFeHevX8XsPUTCIN+XdiepNXwN4l3NjyX3beVT99g8UvUZcu6fM= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:16:52 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:16:52 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , "Darrick J . Wong" , Anand Jain Subject: [PATCH v2 11/18 stable-5.15.y] iomap: Add done_before argument to iomap_dio_rw Date: Tue, 12 Apr 2022 13:15:08 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SG2PR02CA0044.apcprd02.prod.outlook.com (2603:1096:3:18::32) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c0b430a6-3ad4-4092-e8ed-08da1c43a72b X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lUSmPPihBaY8qeHjXQJZ6uyv0ENk1APi0LpzVd0Q+Le+Z12fwTZLE8vJhtG0LhjI1h3DjjPBk0rdLclu7bim7tk6aHLRtPv0si/uMntQ1+qQSRSOvpwlRf8eKKiyu9SmxwMc+o0ysjgjxKl36KmL8YzBYUIgXVo9fHsSR7DrqOzbr26ySOo1GH/NTdNSLZ6aftCh8mMufFo0YYWxs3UdG7tUwebdu1YD/Nnlcuqla7JhSvTRVB5J5B+LTYzm0MUTg8wHLKh4eHVKbMXSqP/w/4Wu2rV/1Lb5iSjpe5ISImYk7Vib35yA6BcYnFI+9mlXHPgyGxDiOzzHRtVJAbz+lImym74McakVri3hLtFsP9CvUY0k5hm7nWoqVICqIyqFe2aXXV7Vu/7g5Ymm0v585rFCxTKccKRzd0t6ZXSWnUWcNT4/IR7K6MS4Feo60kncLlgL7vwRTefsT0NI7xtWpAQVTSP4ERNltZE45W5Chxeixp80/s9YodmHhAHSrGBmbzDSsCKZ5GHV1tIJd8k9TVkQfUUyoyFgaoN1LRax4YH6oA66Cs2RXz0CFzVYOpSI55JestgtEhsWdToJEs7kIzDtYmZvxgouCiTlHhV1ngVS8pejU7mh+MxzQMvLZpiIwWJVETfXWCNM6WVy9kDeJQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(83380400001)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FafFrDt+uOq2oE51vmXpqscudFsTpskO3tKfk4E6cRZvVkRuwBza8o/8G36O+noSuJpIkWYqVODiKiHdSptXjtFWxhyDBcGJ8aZA2P6Y3rLslm2cCkCNAAYKmAoYBhJGHhqtbyNJyA3/3knMKhAVeXzf9I+4sgD9ZftlJOJZiIFPO24eYHVJhGk3uZkUx2Gfve0yMM/GE3wBLzlcMWNUHTitKNCVZPlIXX/q4t3bPS1sXFojNxEaVEd/h1w/fMZLovyA1KfE7CvtGVmMexpuqTbDyLRnPFfq84qNdgsCVt+jzSp/LYHChH3VF2G+Vjzabv7OH+H9z8Iz7tBuuXFjH548PLkmE2Io0Kl64mvCW0jHgBNO5JESEg2m8obFHz58a53GQmQ6lT79SnRaUWDKyMLaRKOAjQNJk5OyeUC7hUMtr1AbLyQ7Dj5jOxEuOAN1ICOETLABH3kbkZjex/6wSzSR8lWVQLSfbRwtfB71e/XuJ284E4rXWIWZQzt4QJQ7HkVtVEDLdfG3Q5xQ1vsBFTVz2BAlRLW+uewy2USOFqMFNoIDM0lODF0EgPiJPyyn/ewz3R4FZtwl/ZJ34U3dPZ9v9BN/Q6WNXfXIs5g8N/WT7hXo3WGUpgHP8KJFuDCcVGLrsLuf6O2MaZ9X7BWsosutB2WOiVpHicyVhtKiGapZiPnvWSKA3ObDRq+T9Ly2ad86/qYUEQG+4cAOgZcOFahs4ra66sEmDic8juzW8VrrVOx8W5DTT94003Rlphg0/wQdEtvaGdB3W7nJU0tPgoSj0XPnbPevCcnrWKZFZSCMYQK35nZSDJlz1D0m1qPoksInCzF/FPEBBc/wCdGJmOA2tCtPpbwZuvyGnl49VZ0WM4euAHBWu0MCekVRud1p8a0V9XbVpykg/EOWRqezXiw0m2LmJoMkTzLED0TH8it0l1JJvIQrNl+GGDwZlcd7/mqU7Pawkso7Y3o0fqor/PJBIwzebHzCOG7MbDI07Ywl/C4pxPREed5vwqOLjFJ4kbP4iAR+7z3yhA0dnQH+rE7dxG0wY7w2t7jmQpGatT8gH8nylbPAtfn0jgvxkrlChJlBJAQG3YfDGtAtpi8FCVdn3VW1mbl9UZHS0dFVpQLHm+8xoCtQ33Vx6bzZLBAhVYVInS2wiv4BC9WfV40HvZtuvVcxatCtWnwARz453kLHmH9m32d52Sd6o4aHew5gFsfeTFSztB5RUrSbOyABrYLFK2BwdVgZHUPFE3ICCxdcoEIR60cQxi7K0mIFt9AOZkpKxVNNeGb79bet5vXC2lPGaaK8G4chM2ZKc6xE9qPiQxbXepVCC1AbEKcN6GZobzWc9RDlODFg9CS7voh7RVCfxPMOios7hrIQrTxxzg/aq0CKeuLG5VZ0SqFzhjEm4E45B+9ONj4Pg7v6i6fQiwAm4MY0uiZgRGrVTZyL4aXbdqFOsbE2lhMkrA2+CnLkzMM0Kcrxny+6aVSRqYlw23pr+Bgc+YgDXO31rJHN4xs09rrzh6GNZkQdBAhdhDV/ddGTT90i69kFPzBk3SdpF3M+vFPYWUe/OfmyO7GKae2xr8vrZwDlW5NatRAyltzz0uoo8J5f9pux0/qyJJnu29TbgMru/ZCJoxZ9Qyk836WBaOkPF4tIkTER6C+1waxOy10j7DnGCNAO6SI4ltnz9i6P3nAjeU+Nb1hl/OVZOTlr/Ozyx6qD/ZgfJxqm2pKxfHBAQXwq1EprQenmyheFJ4MpHMK5jGGKe4fxNaAmyhwH+E/SAdrGFbotYEg9ozit X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c0b430a6-3ad4-4092-e8ed-08da1c43a72b X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:16:52.2217 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UMU2hbcugQQdy4fdbBUOFDp0M0BsCp2lwwxO0qp7eVeSc3rqJ8Ws8j99abL21NWOht7J6zV3waZlXbnd8n41nA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-ORIG-GUID: hsLmIBNaBFevmqBXWJnnSx63C8k6A_nv X-Proofpoint-GUID: hsLmIBNaBFevmqBXWJnnSx63C8k6A_nv Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 4fdccaa0d184c202f98d73b24e3ec8eeee88ab8d upstream Add a done_before argument to iomap_dio_rw that indicates how much of the request has already been transferred. When the request succeeds, we report that done_before additional bytes were tranferred. This is useful for finishing a request asynchronously when part of the request has already been completed synchronously. We'll use that to allow iomap_dio_rw to be used with page faults disabled: when a page fault occurs while submitting a request, we synchronously complete the part of the request that has already been submitted. The caller can then take care of the page fault and call iomap_dio_rw again for the rest of the request, passing in the number of bytes already tranferred. Signed-off-by: Andreas Gruenbacher Reviewed-by: Darrick J. Wong Signed-off-by: Anand Jain --- fs/btrfs/file.c | 5 +++-- fs/erofs/data.c | 2 +- fs/ext4/file.c | 5 +++-- fs/gfs2/file.c | 4 ++-- fs/iomap/direct-io.c | 19 ++++++++++++++++--- fs/xfs/xfs_file.c | 6 +++--- fs/zonefs/super.c | 4 ++-- include/linux/iomap.h | 4 ++-- 8 files changed, 32 insertions(+), 17 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 5bf4304366e9..cd4950476366 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1956,7 +1956,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) } dio = __iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - 0); + 0, 0); btrfs_inode_unlock(inode, ilock_flags); @@ -3659,7 +3659,8 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) return 0; btrfs_inode_lock(inode, BTRFS_ILOCK_SHARED); - ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, 0); + ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, + 0, 0); btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); return ret; } diff --git a/fs/erofs/data.c b/fs/erofs/data.c index 9db829715652..16a41d0db55a 100644 --- a/fs/erofs/data.c +++ b/fs/erofs/data.c @@ -287,7 +287,7 @@ static ssize_t erofs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) if (!err) return iomap_dio_rw(iocb, to, &erofs_iomap_ops, - NULL, 0); + NULL, 0, 0); if (err < 0) return err; } diff --git a/fs/ext4/file.c b/fs/ext4/file.c index ac0e11bbb445..b25c1f8f7c4f 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -74,7 +74,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) return generic_file_read_iter(iocb, to); } - ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, 0); inode_unlock_shared(inode); file_accessed(iocb->ki_filp); @@ -566,7 +566,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) if (ilock_shared) iomap_ops = &ext4_iomap_overwrite_ops; ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops, - (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0); + (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0, + 0); if (ret == -ENOTBLK) ret = 0; diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 2d0aa55205ed..81835d34d6f6 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -823,7 +823,7 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, if (ret) goto out_uninit; - ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0, 0); gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); @@ -857,7 +857,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, if (offset + len > i_size_read(&ip->i_inode)) goto out; - ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0, 0); if (ret == -ENOTBLK) ret = 0; out: diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index a434fb7887b2..468dcbba45bc 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -31,6 +31,7 @@ struct iomap_dio { atomic_t ref; unsigned flags; int error; + size_t done_before; bool wait_for_completion; union { @@ -124,6 +125,9 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio) if (ret > 0 && (dio->flags & IOMAP_DIO_NEED_SYNC)) ret = generic_write_sync(iocb, ret); + if (ret > 0) + ret += dio->done_before; + kfree(dio); return ret; @@ -450,13 +454,21 @@ static loff_t iomap_dio_iter(const struct iomap_iter *iter, * may be pure data writes. In that case, we still need to do a full data sync * completion. * + * When page faults are disabled and @dio_flags includes IOMAP_DIO_PARTIAL, + * __iomap_dio_rw can return a partial result if it encounters a non-resident + * page in @iter after preparing a transfer. In that case, the non-resident + * pages can be faulted in and the request resumed with @done_before set to the + * number of bytes previously transferred. The request will then complete with + * the correct total number of bytes transferred; this is essential for + * completing partial requests asynchronously. + * * Returns -ENOTBLK In case of a page invalidation invalidation failure for * writes. The callers needs to fall back to buffered I/O in this case. */ struct iomap_dio * __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags) + unsigned int dio_flags, size_t done_before) { struct address_space *mapping = iocb->ki_filp->f_mapping; struct inode *inode = file_inode(iocb->ki_filp); @@ -486,6 +498,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, dio->dops = dops; dio->error = 0; dio->flags = 0; + dio->done_before = done_before; dio->submit.iter = iter; dio->submit.waiter = current; @@ -652,11 +665,11 @@ EXPORT_SYMBOL_GPL(__iomap_dio_rw); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags) + unsigned int dio_flags, size_t done_before) { struct iomap_dio *dio; - dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags); + dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags, done_before); if (IS_ERR_OR_NULL(dio)) return PTR_ERR_OR_ZERO(dio); return iomap_dio_complete(dio); diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 7aa943edfc02..240eb932c014 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -259,7 +259,7 @@ xfs_file_dio_read( ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); if (ret) return ret; - ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, 0); xfs_iunlock(ip, XFS_IOLOCK_SHARED); return ret; @@ -569,7 +569,7 @@ xfs_file_dio_write_aligned( } trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, 0); + &xfs_dio_write_ops, 0, 0); out_unlock: if (iolock) xfs_iunlock(ip, iolock); @@ -647,7 +647,7 @@ xfs_file_dio_write_unaligned( trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, flags); + &xfs_dio_write_ops, flags, 0); /* * Retry unaligned I/O with exclusive blocking semantics if the DIO diff --git a/fs/zonefs/super.c b/fs/zonefs/super.c index 807f33553a8e..bced33b76bea 100644 --- a/fs/zonefs/super.c +++ b/fs/zonefs/super.c @@ -852,7 +852,7 @@ static ssize_t zonefs_file_dio_write(struct kiocb *iocb, struct iov_iter *from) ret = zonefs_file_dio_append(iocb, from); else ret = iomap_dio_rw(iocb, from, &zonefs_iomap_ops, - &zonefs_write_dio_ops, 0); + &zonefs_write_dio_ops, 0, 0); if (zi->i_ztype == ZONEFS_ZTYPE_SEQ && (ret > 0 || ret == -EIOCBQUEUED)) { if (ret > 0) @@ -987,7 +987,7 @@ static ssize_t zonefs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } file_accessed(iocb->ki_filp); ret = iomap_dio_rw(iocb, to, &zonefs_iomap_ops, - &zonefs_read_dio_ops, 0); + &zonefs_read_dio_ops, 0, 0); } else { ret = generic_file_read_iter(iocb, to); if (ret == -EIO) diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 2a213b0d1e1f..829f2325ecba 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -339,10 +339,10 @@ struct iomap_dio_ops { ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags); + unsigned int dio_flags, size_t done_before); struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags); + unsigned int dio_flags, size_t done_before); ssize_t iomap_dio_complete(struct iomap_dio *dio); int iomap_dio_iopoll(struct kiocb *kiocb, bool spin); From patchwork Tue Apr 12 05:15:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80A84C433EF for ; Tue, 12 Apr 2022 05:17:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231492AbiDLFTa (ORCPT ); Tue, 12 Apr 2022 01:19:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345584AbiDLFT3 (ORCPT ); Tue, 12 Apr 2022 01:19:29 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0EEA434663; Mon, 11 Apr 2022 22:17:13 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23C1kKno012649; Tue, 12 Apr 2022 05:17:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=wFb7yn0UElD5ZwXtq53ZMcrHZz+UjSVIqt0gx9tSaQtiDyGvv7rM7hlAQBEmRqhdBPP8 KBwaArtDnGHn/Ri/+Ij/oovAR8YwTcObpDZFmhxSmCgYSLgPPHCQ2z0TR8LI9pig6+uT aLdMpmWwnCoeIX6g1YrbAE7E9iRYlr8oiqO/0lbI8HkZfEonjZtcT3pfTx7/n9WQ8gLC X2BQ0GAkYIkgTubdAeLsZEQxL3/sKXYFeFIbh+zUWBB9Se0sXOJGxBAkSW+ATrWE44lx EldYz1GO1Pv/gr9IdzXgqqCM/HHsnFRqVVRqRsi1vhvt9pcKlDvGl0pPEuVJmIoYVmOt 5Q== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb2ptwnte-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:10 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5BJs1023480; Tue, 12 Apr 2022 05:17:10 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2175.outbound.protection.outlook.com [104.47.57.175]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k2snv7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VJNAYkPLNqZmw0d6Iyby2wfnjE6cAkGMRLqKINGrzKhPD26gqJtP8XBZbAvpOiY55U85IbE3iCtnnb5UEPmq2mJBv0d1fx1GdlvGz+NNXV2JEo0R7uyiHx4TiOAyIk16qjQYQKIfQ+PEMd3c06frfO0yKywTR84Ko0rHxQcAeVkSMz160TiERjSig+QW1AJe9u6/no2j7J7YQ8cit+EUK1PzWYlKz5jGzxHaeJLOc+kJFu3clwy4sX4anT6HcZw/wZz0yBWVVvHnD59ADr/IhNM+qqpzs6gtpvcf80/yQLrYfvTYQLMRrPBHJZwGZAaOg6BSyAIyxIaXwgUEpADkDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=j4uYljw5WiqRg5NDjy7oNfli6sC/deLmxSJi+y0ptDcrSjJ10bs1wW9J0QmwgSOfVzaYSw424g+DKff26BCbG5f1HVjICpn0cYQAAyqWoYnd0WOAwQMLyXTGL79I6h53ENyj+V/dHvIwA5IaS4wsNqDCOUPOfVlO3VNrkg1Zulu/ECBUxkcES8k7Yl0O6mNneWD4aEL1TIhgNfylqas3xkxkfibYmSZKCCOImZR6k/JLwKjXvhWiZip5A+h7wQkdsg7KLKxwcY5wOaNL18zBMNIgyYQBB6W0Zoesnh5wOaTwblKsokALT7tgL5qoNXgzlkW1rzjPqD0sCcY8C8X7EA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=n2bnVycLfHQwWJuKQMW+7rUnVgTr1l6hhJ06iqoc+JPaoSwlAnjrCMfnv4rELpIHSpMg6vBHCu4PBR/kp8i19cLtDwqU+hJnBFV19yJnM4fO0ET5JPkmrjHVNgMg4ZkLm/PAIpn9Udnv1P+mvtqVK3d/fhbUeWm3IffyWg34kWM= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BN8PR10MB3329.namprd10.prod.outlook.com (2603:10b6:408:cd::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:17:08 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:17:08 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 13/18 stable-5.15.y] iov_iter: Introduce nofault flag to disable page faults Date: Tue, 12 Apr 2022 13:15:10 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SG2PR06CA0241.apcprd06.prod.outlook.com (2603:1096:4:ac::25) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1da42e34-7a81-45b8-ea2c-08da1c43b0d9 X-MS-TrafficTypeDiagnostic: BN8PR10MB3329:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nRDBMsBDX5Az7ad2U6kR4FO3zRvrGOSxXkP/8UsdU71mq66/Picdz6r60ONfnDxGwAhF8vH/TqWDNEUhszGb77ysojgpfrAyan5mnXGDSxp6cclumeZmN7OMzzVMPJLl+zGZbt6eFRMWhOKTORKXy+7MHdaJwvzxcGDGPTXqn3lMtbpuwfoJhUSXAakaQeqRNbHBQLhhWBMtDZ04PeUNPkIcp/nQs0Rxm0P+nNdZo+MCCuuIhowKwLrrNvWdja7eb5bxtpFYv2w/bpfJGDSgLYX9YQugWCu7HzUjrMS3wX8m/UMWVgnI+dJzD8hQCwZgkqZb5BaDgYWHZIVM31/yEnsJhboITzOMVl9a5zNe2aSwX1VxHJ0vGf/laBL5oHa+IhZum/unsB6Ipz61QvJ0XvhKXn8F91rHEWENa1Th7w83hwWy0tWE3famPH7+spiNobT3a5d9ecwFlyusn9sxYHcgBedmUhd/5huq0CTKdRBImdkgS6T/eZ4t71gJSs6K0c5KKdjSB/GPduToI+zySQmPeth4mmo4V0LTgHIBfV4RpIShdPlcWoQWmJnbaMHBRq2s3IJetofoyJWKSvVso3s50NgrPy3EZqsD3EHPJsLlh0WxRsZJuKkxAmIbEyh+L6h3U2++6Hpka4O2INifiA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6506007)(86362001)(38100700002)(54906003)(6916009)(316002)(6486002)(508600001)(6512007)(2906002)(6666004)(5660300002)(107886003)(2616005)(83380400001)(8676002)(66476007)(186003)(66946007)(66556008)(44832011)(8936002)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3/Zt0uhgUuKFbDWoPK+83p5c70wqSui74buM/95aRAngDs7DcXsGvILJA9urNVNO+5Od5QCMJx6G4387341+xVgD5vKgPSnd4vuqIXPnJrE0G+rqaHMXepo0V/1R+itmohPzNMjFa+wMJra2kGs9L/9dpD5CBiGzUuStsiT+2WHnYEtrDa1XE2lEgwaKqbJlfuJQjYjMoC636j80YagqNr9BNPchWb99gI6jrlv/K9Sb+8glxJQv3mM1Zp9RdEfi+0zhGgUH6Az1oX/WiD2ZrQDqm9hKrmJo8njNm46gVw6bpoJBb0TVbKegQfzQ4y3eR8lQSysQ09AigSS39WMJsx99bpB6fD0Kh7BKWtuQV/3mfWC8wMP2j9wOZCdviIedJ3tLf0wQzDR2jOKAl1eMu33BZyl4YB715AzaPp0FANCycFTtjVal4bOlz663yZps/3Cs3+IFPcIJ/03tENyWSMDY+VnEuFqjUQE25QwCqR0+5BkVodQV8IpiYIr9eESR4YpZH6JTberU4eJdLLfd9nf6Oi9oGN/i/PCZ5Iw0MrDqVjSOx3nbg2AXSflWMGDGJAc8zcUjGjQ93tcOZC2C33W2OHLOlshNlX39lnxCI/5HTBmtYhd3nHb9J88+Pq2LTl8Z00mYaqzGTF67FqqpxsE6WIv1A+fDcvrgMyATrWXoQpzJrBtOc9s+Vet59tSsBZrRTNavPQAtGTNU9Pw3soAEhzWYDoWm0OEyQvsqNsoyccnU2FdfnreC1Jth8fBaW41WdnWTO79gSziVvrBqGHWZD2lGbIQQsq+nGyngz41c37lb71HcnkXPHMMOE8a62HNS08Po/c57VJdP/Skn0sBENm0jZMK1H6v1rJc+8AYTveLWS7xK8bcK17v8SNvRHwRgNwmVzjqutDvrlSC+1JABnOFyBCldNevGlUSF8Or4ij3vzIeFm+bEQHAmfoMLgwMbGb/MLDhvRJkWE0F0wUXvsj/Sj4hYqpNyMjvIcEl8bbCVS6nT3w9w3LNS4QEG+g/b9HvCzkLcgc8UxCpwukj7UiTjrC6B5dRUyLFjmw3xKNTESCwVc2HqQE9qxpX1bokSl0+Y9hwYyz2kHyoDZa2L3++KbGXtwkvPihBsuBE0XnhRn/K1q3CvoXeq4XoX2PBA2R0hGUDgH9CQTchHKhoOyg8bFHmVnUpsjSV4p4GuiPv/R3fb1v+RmKVt+Im0oQ4nEU6cYvg+t6AQLwXR3b17AuLpPpyl9BBWl2m9zK1ctDQWfAfg/GmatQcAunPPSrdnPw4Tkap6duuZGzyWX7o1tZSnUseJwF3PWEstP38yVM0V9O2iYhC7HXg+hEWm6wUGxV2bAMumD7J6l2JGl60XpAs+KwkZTyU7RRoyGCkjSxIySmcRohb041bEBwlVtoNROilLfUGriwXnCFogHzH70Bn/YZbrXAsBC6MSa+ikj1aoqJAcdU57R7lPAAV3pcOAIhlctlfGB+moxhSz0uKojFTzZ7FYSNADX8iNp20Hnu4keAKbhPeXzNACAucOPetBzgPDl+8Tng3Tw/qeRcsuF82WkPux26y1np6MZKragdQhdIZeFmgEOCGFR7X0Nn2IltqOKYIPTu8kQ3xxTOtD7eQH0bbijXk5XmWDk9uQw5pPWDBzDzU6zE6Cmqh1oO/4U3SXQwGV3Uzf6sYUL7ar6kvBUzBK9NL2tt97sULf4hQdTPGVtrl5bi6HjM8Er6GxPdvWU2IB0aJe6Ji1NYoljPFE5qYveMZ7CHGITuSuSqRpFb+JDUYfz0eh0bcy X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1da42e34-7a81-45b8-ea2c-08da1c43b0d9 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:17:08.4615 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rHvSZ3rgRXKFS55toB3y88Vuotqfk1W2eAZpZLPqdy1/OY7xFFaY3SADdQxfES0p9rM+01tvlpPJORCHHUlBLQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3329 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120023 X-Proofpoint-ORIG-GUID: JJw9xkaS53jrvKYVbE0jePH2-TxmtJWC X-Proofpoint-GUID: JJw9xkaS53jrvKYVbE0jePH2-TxmtJWC Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 3337ab08d08b1a375f88471d9c8b1cac968cb054 upstream Introduce a new nofault flag to indicate to iov_iter_get_pages not to fault in user pages. This is implemented by passing the FOLL_NOFAULT flag to get_user_pages, which causes get_user_pages to fail when it would otherwise fault in a page. We'll use the ->nofault flag to prevent iomap_dio_rw from faulting in pages when page faults are not allowed. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- include/linux/uio.h | 1 + lib/iov_iter.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 25d1c24fd829..6350354f97e9 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -35,6 +35,7 @@ struct iov_iter_state { struct iov_iter { u8 iter_type; + bool nofault; bool data_source; size_t iov_offset; size_t count; diff --git a/lib/iov_iter.c b/lib/iov_iter.c index b137da9afd7a..6d146f77601d 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -514,6 +514,7 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, WARN_ON(direction & ~(READ | WRITE)); *i = (struct iov_iter) { .iter_type = ITER_IOVEC, + .nofault = false, .data_source = direction, .iov = iov, .nr_segs = nr_segs, @@ -1529,13 +1530,17 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, return 0; if (likely(iter_is_iovec(i))) { + unsigned int gup_flags = 0; unsigned long addr; + if (iov_iter_rw(i) != WRITE) + gup_flags |= FOLL_WRITE; + if (i->nofault) + gup_flags |= FOLL_NOFAULT; + addr = first_iovec_segment(i, &len, start, maxsize, maxpages); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, - iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, - pages); + res = get_user_pages_fast(addr, n, gup_flags, pages); if (unlikely(res <= 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1651,15 +1656,20 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, return 0; if (likely(iter_is_iovec(i))) { + unsigned int gup_flags = 0; unsigned long addr; + if (iov_iter_rw(i) != WRITE) + gup_flags |= FOLL_WRITE; + if (i->nofault) + gup_flags |= FOLL_NOFAULT; + addr = first_iovec_segment(i, &len, start, maxsize, ~0U); n = DIV_ROUND_UP(len, PAGE_SIZE); p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, - iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p); + res = get_user_pages_fast(addr, n, gup_flags, p); if (unlikely(res <= 0)) { kvfree(p); *pages = NULL; From patchwork Tue Apr 12 05:15:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5158BC433F5 for ; Tue, 12 Apr 2022 05:17:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345584AbiDLFTo (ORCPT ); Tue, 12 Apr 2022 01:19:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48932 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232265AbiDLFTn (ORCPT ); Tue, 12 Apr 2022 01:19:43 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8302834659; Mon, 11 Apr 2022 22:17:26 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23C3I1j6008564; Tue, 12 Apr 2022 05:17:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=Dlt4DVzFlV4rj/Ms/7TJfrHF6jPaXGEsuCAgl1GofH4=; b=0GbcugxaHmVzXgvuEGOpV+lA3djOM6YEw2Q1kcKcGNdWCBnNZ1dXzlUqW1GetZMEOYOT x1WMB/eY6EZRg1OTxVyArsuUmDU+QNZdSaQQggDjAk0XME2QLiwREkV1l3MOORAs6h2r RS7c/XP/HqFpZopc9Uat/hqHTEGsNBIgPNbjQ/rKh8M1rYH/M68jaXQQKGy9GI5cvw3d S66do76WjS72t8U00RuI9HRjyATt+CG3Vz3pwCbVpBXQAcf3ELwDdWQ8nTWNEP1t/1Nq 32j9olo/9ScNAUQeU1XoGMq3wxh32lmVT2KNhYR2vj2CJin3LUyHw/+rhS42XRF98a2L zQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2dmc7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:24 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5HMLb016490; Tue, 12 Apr 2022 05:17:23 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2041.outbound.protection.outlook.com [104.47.66.41]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k2cmyj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W1QqPa0F5uyUcZStPWee4RDI0/zurFCAkYKJspiBw5hY/9Wt866qCxZDw4rI/JXDEOg+Mfc0hDHzPpUxuRS0KhJfUzfBBUtlzG/5Ggdw7c9zOm183mVFL94lmqLPEcmKVUu1IA02fd5CZvbPHkAwqi5rEYVV5t83l8rRyLTXNfuji45KEI6aZuW1EtzRhbaf7ZBV6HL6RqgX19AkgfiiANS9+NCisyX8DJFDpIUOK1YyXaxYeDTshXYInUB00d85s0pZuAct0mNpO2t+IrNrJbxaUwjuoKM96TQ/F+HoJ7HA46r63DmSkNPkG0GLIIHtjzknsY3bAPyutfBty2zxkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Dlt4DVzFlV4rj/Ms/7TJfrHF6jPaXGEsuCAgl1GofH4=; b=k18Jq64XQtzNf7XwvnVi6S9E9A4bjTkwUAfUVaMbJiBp/Jajo1I2pavsS/D+7mZkl+c8n5ryw65erG0AzEU1qGP9nzWKS+FdlW2iHmtmdY6LmREENvi0XugMTTveGpB374IxdoCJh36cHGL1Cx3yPsUlqUyhObNDmNH86ipYt5kDoP0zbV02I179HokWl9lW/e6IJSxtrb0RNMJfItsLMZ/ryryEN2gDAjVYX82cHFihtJbZbTu977skjtEic524waQ+x/8Zz1FwTacneXmOpdQdM2FB+QCEg0vjWcboOyOQ2xb73mzFHg0ca3+IQJuHBHGZBpXvR7ALkRKOg0FoQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Dlt4DVzFlV4rj/Ms/7TJfrHF6jPaXGEsuCAgl1GofH4=; b=i/5A0UpZVkPSWkCjx3iB44uHhK04l5QUhJhpkhNQQQw/n04YftEvKm1YTsxcuHRhLq0+MBHXL83BI+61SHUWIt51lcsKdnRFh2M2Kbrdhf+3Ei2G/SL6TfPGdeRdkqKESxvCSd9pcLjyiU9FHQAxipdaQ0GJEO6zrAOLHWN4BIQ= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BYAPR10MB3256.namprd10.prod.outlook.com (2603:10b6:a03:152::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:17:20 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:17:20 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Bob Peterson , Andreas Gruenbacher , Anand Jain Subject: [PATCH v2 15/18 stable-5.15.y] gfs2: Introduce flag for glock holder auto-demotion Date: Tue, 12 Apr 2022 13:15:12 +0800 Message-Id: <8f783c16379c9255434fdebe3cd8aea7e3b8002e.1649733186.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TY2PR02CA0046.apcprd02.prod.outlook.com (2603:1096:404:a6::34) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5b3fa398-2423-4aac-720f-08da1c43b844 X-MS-TrafficTypeDiagnostic: BYAPR10MB3256:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /22aUknSYa6lQxsfWCgZNCmeCqCO9XHU51zihbVTpQUWweIfDWdzwNI6YOOO4aEXOewfPVzIYIRQ6EnpcR2R7CXMtRzdH60Hw7JOxDxZW91qiza4VTNwljhqOMbL+XOSQFJkyAQqMZBKwB71SH+1grnn/Wmz37f0fdQcl+S9/nN8Gio030fH5NNont2X19hXUmczrxwMINwk9PXv0YLWw1JEXvzdPuNoq7Tfmf2UJEGlujKOtkN3HqTlUm4gJrtkYvsrrNGaInu/HjESA7HKNXaog6ezQcPMycLQ6Ax8tjgJZU4RVmlCjTGC9icgYUYKkh1qoIM29r9kin7STYag9zv4J/CCCK3v4VG0QuUM7LFt76vEt2fUYWtoFJ/D0hCUY0VaNYRnMh+hoIF9eeWlIPIKN+Tpfl4EhWQfvpdQr/1HZzDdbD3Ma1+ShZ+OW7nBAO/52weXVhpYzhwdDVlVtVSEZI9oygV/OZbV87yb8+Zub9riAI+O3QOnsFQTJmaj3N5PPitpM0QP5eig4VacQQWS1nV8d23v7YXMsxdkht2VhJc75ZuKydgebRZmbLljMn1ea04vwrN1uWu5dFLBFb90+zLozwXRc4+vNZweSrfgJwupP09krx3zf7i8S/i/jUq+SbTgpCXrY9UzhmRTgg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(38100700002)(8936002)(5660300002)(30864003)(2906002)(83380400001)(6512007)(2616005)(66476007)(44832011)(107886003)(186003)(6506007)(6666004)(508600001)(66556008)(66946007)(6486002)(86362001)(54906003)(36756003)(8676002)(4326008)(6916009)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fVB1VM02emZMo4ScF1QfnZLBJkbYvcYQQFaCkgKaFee6sTfRilSVb40UYmssOtWPMyvlU6QB75EqdRws/Dm2XtikacW6WR3+PfbA/EDYtTOsgys2SD/vgB7AVOMGO8gsXDbJTDmbVhm8tc8U2HuMdIey6ObmaJZitOhR1jVbaLACU2phIp1TpuhV2cEYiKV7gQ3WeY1qpByrLL/5A2C87fG/0FmLgFldLbATbmJut8IymsLuReT5fX8ONOtNdnHoYTPbkx7hHy+McHdeyeoMhDmvX0cwppa0k0/bN38FkCsHy8Uc0SguBcInlUoK953eXuj/S/UPXsPyOS5VyYs0imvQaL/btJEmvAcP49JbYBx2UnzsXHt5NKgl1WSGDs58c/O9k0XHa+3ASuUHJKGjPw5cVyc2P60XeeagHIhTlGigR1+sp5MPUSUtvJZ5AkHbI7W99iR0r4izdnvjkY9sWCtxpIDou8jBjqE3OYl9ptbarCcD213TqsXFLJmZMaHGJGYrsaWr3yrfPXJCCJiH/F6uH+M+mzgcjHUTlNbjKiJPVXvfZLbg6P21umS88v0lO6WxN9i9/pOt83Q7LuoxqLA5eD7dGroCBNjG75RcB6FSwZH2bEbeiTBqDHT7cCQA8Arsp2/ouFLb2QfP4KGi4qOqtWcC6x1PAGPlu6QaR1jnBhP4WaGhJqLf24jo7/P3KcbG4SPJDKBf4ZKj4Xc5zU3+VXw1QwXEe0wfNn4R3EJYdWIiN8ZG7UT3XL2KQA+Lbqcv+HscT+ckMUG+e1P/OUFfeU5vLJ/QbN+FnReXsY6o0Zg1JJqhRxd/de8ByE+zvobW9xxdMrbwI6GeNqnMWbtHkSOlEHOqnAaqjKAIfBVPyYxQw3HggqIh1AUQkBZ7p7d8qsfLusEneCTQL32BHD+LkIe6NHH4z+CTVcu4n7Q8GWOX4zoBZkil8PWU+k/pDRRM1/rFvaNNZefNrcNv/ZU1IWwoNNqP+zU4m2ZEng89ZDJ8h2MijLJVWvrDLejBGYIQw9v7Rvf3ehapoK9AZfv94QptHRHYo0vAv+TsRhFky+T8nSKx/p60UUCN5+HjP1UXBoqc0JQktU+zXI0pHsWoG7D8zq1qBJie8JdPTdW0w1uEeuVNI3W6ih0yGvfZGqN8pmWPEsfzvPcC2oQcqcKKwYnv1tTa9dcErKrWCJyu1x6MsrYzAfQ1TLwb+u1KGlb/R93NGyIVSzZvoACBfwsl1CCELte5Vezx7Hv+bIcem4rILlV3/EzRoN24NYFZSGnXenq5JiXnFueDvN775kBEgGSSYm1m0SIOjf+vy/fHfD/Zpv2pYaxsRR8Ibbu1vnVdZ0tXCmV8l5UXSYAVbDIQWtc3itlGxmUTpoBLLB0fV0OsXkRZ6FlkqaL/Kool/nnJDbVUvjAL6njpQzgP+9QZgSlJ+l6+oA7jBEziyMQ+/oxyQ0MzakOVeMcx0+oFVEb0qLfUrRGDZfWx4ox12GCGLuBgln7udcEdzsPkWKpUOhevvraUsz8U1Zq62z4WrMw5dfgZ1K3uYxuC7iW0Mv3+AJQ32hs97r8tgMq/Kd6Pck3zD2wqumvsoM/2iJmBQnilhMax9uhX/58jfz6//bAdtkl8jiuQtGmgOljF7eEJtBa+XjXQDwNpayFl+b+1hcbGhdrAS9yrguatyzzMX7Vz5J3oeXRny+jUBwHwI3fCAMhRYtmaZo+r/N64LxBqK/uxfcZYpjkWc8EZ2shqCmKjdORJTQF6b5YkJf0/ltNlodTIZqQRhiHTfebBPVpe X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b3fa398-2423-4aac-720f-08da1c43b844 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:17:20.7942 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lNGvLhAquVVcrwaKl7/6x2aTTmlP7NusOPOTs8e8Gs+Kaptveg0263XCL08QP1pXwRonkuLHWhgL5tgb0iPQWQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3256 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-ORIG-GUID: zHqejQSiuO6RNjyJpN_ck2bzHxoo4QVT X-Proofpoint-GUID: zHqejQSiuO6RNjyJpN_ck2bzHxoo4QVT Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Bob Peterson commit dc732906c2450939c319fec6e258aa89ecb5a632 upstream This patch introduces a new HIF_MAY_DEMOTE flag and infrastructure that will allow glocks to be demoted automatically on locking conflicts. When a locking request comes in that isn't compatible with the locking state of an active holder and that holder has the HIF_MAY_DEMOTE flag set, the holder will be demoted before the incoming locking request is granted. Note that this mechanism demotes active holders (with the HIF_HOLDER flag set), while before we were only demoting glocks without any active holders. This allows processes to keep hold of locks that may form a cyclic locking dependency; the core glock logic will then break those dependencies in case a conflicting locking request occurs. We'll use this to avoid giving up the inode glock proactively before faulting in pages. Processes that allow a glock holder to be taken away indicate this by calling gfs2_holder_allow_demote(), which sets the HIF_MAY_DEMOTE flag. Later, they call gfs2_holder_disallow_demote() to clear the flag again, and then they check if their holder is still queued: if it is, they are still holding the glock; if it isn't, they can re-acquire the glock (or abort). Signed-off-by: Bob Peterson Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/glock.c | 215 +++++++++++++++++++++++++++++++++++++++-------- fs/gfs2/glock.h | 20 +++++ fs/gfs2/incore.h | 1 + 3 files changed, 200 insertions(+), 36 deletions(-) diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 8f30ad956270..e85ef6b14777 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -58,6 +58,7 @@ struct gfs2_glock_iter { typedef void (*glock_examiner) (struct gfs2_glock * gl); static void do_xmote(struct gfs2_glock *gl, struct gfs2_holder *gh, unsigned int target); +static void __gfs2_glock_dq(struct gfs2_holder *gh); static struct dentry *gfs2_root; static struct workqueue_struct *glock_workqueue; @@ -197,6 +198,12 @@ static int demote_ok(const struct gfs2_glock *gl) if (gl->gl_state == LM_ST_UNLOCKED) return 0; + /* + * Note that demote_ok is used for the lru process of disposing of + * glocks. For this purpose, we don't care if the glock's holders + * have the HIF_MAY_DEMOTE flag set or not. If someone is using + * them, don't demote. + */ if (!list_empty(&gl->gl_holders)) return 0; if (glops->go_demote_ok) @@ -379,7 +386,7 @@ static void do_error(struct gfs2_glock *gl, const int ret) struct gfs2_holder *gh, *tmp; list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + if (!test_bit(HIF_WAIT, &gh->gh_iflags)) continue; if (ret & LM_OUT_ERROR) gh->gh_error = -EIO; @@ -393,6 +400,40 @@ static void do_error(struct gfs2_glock *gl, const int ret) } } +/** + * demote_incompat_holders - demote incompatible demoteable holders + * @gl: the glock we want to promote + * @new_gh: the new holder to be promoted + */ +static void demote_incompat_holders(struct gfs2_glock *gl, + struct gfs2_holder *new_gh) +{ + struct gfs2_holder *gh; + + /* + * Demote incompatible holders before we make ourselves eligible. + * (This holder may or may not allow auto-demoting, but we don't want + * to demote the new holder before it's even granted.) + */ + list_for_each_entry(gh, &gl->gl_holders, gh_list) { + /* + * Since holders are at the front of the list, we stop when we + * find the first non-holder. + */ + if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) + return; + if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags) && + !may_grant(gl, new_gh, gh)) { + /* + * We should not recurse into do_promote because + * __gfs2_glock_dq only calls handle_callback, + * gfs2_glock_add_to_lru and __gfs2_glock_queue_work. + */ + __gfs2_glock_dq(gh); + } + } +} + /** * find_first_holder - find the first "holder" gh * @gl: the glock @@ -411,6 +452,26 @@ static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) return NULL; } +/** + * find_first_strong_holder - find the first non-demoteable holder + * @gl: the glock + * + * Find the first holder that doesn't have the HIF_MAY_DEMOTE flag set. + */ +static inline struct gfs2_holder * +find_first_strong_holder(struct gfs2_glock *gl) +{ + struct gfs2_holder *gh; + + list_for_each_entry(gh, &gl->gl_holders, gh_list) { + if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) + return NULL; + if (!test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags)) + return gh; + } + return NULL; +} + /** * do_promote - promote as many requests as possible on the current queue * @gl: The glock @@ -425,14 +486,20 @@ __acquires(&gl->gl_lockref.lock) { const struct gfs2_glock_operations *glops = gl->gl_ops; struct gfs2_holder *gh, *tmp, *first_gh; + bool incompat_holders_demoted = false; int ret; restart: - first_gh = find_first_holder(gl); + first_gh = find_first_strong_holder(gl); list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + if (!test_bit(HIF_WAIT, &gh->gh_iflags)) continue; if (may_grant(gl, first_gh, gh)) { + if (!incompat_holders_demoted) { + demote_incompat_holders(gl, first_gh); + incompat_holders_demoted = true; + first_gh = gh; + } if (gh->gh_list.prev == &gl->gl_holders && glops->go_lock) { spin_unlock(&gl->gl_lockref.lock); @@ -458,6 +525,11 @@ __acquires(&gl->gl_lockref.lock) gfs2_holder_wake(gh); continue; } + /* + * If we get here, it means we may not grant this holder for + * some reason. If this holder is the head of the list, it + * means we have a blocked holder at the head, so return 1. + */ if (gh->gh_list.prev == &gl->gl_holders) return 1; do_error(gl, 0); @@ -1372,7 +1444,7 @@ __acquires(&gl->gl_lockref.lock) if (test_bit(GLF_LOCK, &gl->gl_flags)) { struct gfs2_holder *first_gh; - first_gh = find_first_holder(gl); + first_gh = find_first_strong_holder(gl); try_futile = !may_grant(gl, first_gh, gh); } if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags)) @@ -1381,7 +1453,8 @@ __acquires(&gl->gl_lockref.lock) list_for_each_entry(gh2, &gl->gl_holders, gh_list) { if (unlikely(gh2->gh_owner_pid == gh->gh_owner_pid && - (gh->gh_gl->gl_ops->go_type != LM_TYPE_FLOCK))) + (gh->gh_gl->gl_ops->go_type != LM_TYPE_FLOCK) && + !test_bit(HIF_MAY_DEMOTE, &gh2->gh_iflags))) goto trap_recursive; if (try_futile && !(gh2->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB))) { @@ -1477,51 +1550,83 @@ int gfs2_glock_poll(struct gfs2_holder *gh) return test_bit(HIF_WAIT, &gh->gh_iflags) ? 0 : 1; } -/** - * gfs2_glock_dq - dequeue a struct gfs2_holder from a glock (release a glock) - * @gh: the glock holder - * - */ +static inline bool needs_demote(struct gfs2_glock *gl) +{ + return (test_bit(GLF_DEMOTE, &gl->gl_flags) || + test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags)); +} -void gfs2_glock_dq(struct gfs2_holder *gh) +static void __gfs2_glock_dq(struct gfs2_holder *gh) { struct gfs2_glock *gl = gh->gh_gl; struct gfs2_sbd *sdp = gl->gl_name.ln_sbd; unsigned delay = 0; int fast_path = 0; - spin_lock(&gl->gl_lockref.lock); /* - * If we're in the process of file system withdraw, we cannot just - * dequeue any glocks until our journal is recovered, lest we - * introduce file system corruption. We need two exceptions to this - * rule: We need to allow unlocking of nondisk glocks and the glock - * for our own journal that needs recovery. + * This while loop is similar to function demote_incompat_holders: + * If the glock is due to be demoted (which may be from another node + * or even if this holder is GL_NOCACHE), the weak holders are + * demoted as well, allowing the glock to be demoted. */ - if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) && - glock_blocked_by_withdraw(gl) && - gh->gh_gl != sdp->sd_jinode_gl) { - sdp->sd_glock_dqs_held++; - spin_unlock(&gl->gl_lockref.lock); - might_sleep(); - wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY, - TASK_UNINTERRUPTIBLE); - spin_lock(&gl->gl_lockref.lock); - } - if (gh->gh_flags & GL_NOCACHE) - handle_callback(gl, LM_ST_UNLOCKED, 0, false); + while (gh) { + /* + * If we're in the process of file system withdraw, we cannot + * just dequeue any glocks until our journal is recovered, lest + * we introduce file system corruption. We need two exceptions + * to this rule: We need to allow unlocking of nondisk glocks + * and the glock for our own journal that needs recovery. + */ + if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) && + glock_blocked_by_withdraw(gl) && + gh->gh_gl != sdp->sd_jinode_gl) { + sdp->sd_glock_dqs_held++; + spin_unlock(&gl->gl_lockref.lock); + might_sleep(); + wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY, + TASK_UNINTERRUPTIBLE); + spin_lock(&gl->gl_lockref.lock); + } + + /* + * This holder should not be cached, so mark it for demote. + * Note: this should be done before the check for needs_demote + * below. + */ + if (gh->gh_flags & GL_NOCACHE) + handle_callback(gl, LM_ST_UNLOCKED, 0, false); + + list_del_init(&gh->gh_list); + clear_bit(HIF_HOLDER, &gh->gh_iflags); + trace_gfs2_glock_queue(gh, 0); + + /* + * If there hasn't been a demote request we are done. + * (Let the remaining holders, if any, keep holding it.) + */ + if (!needs_demote(gl)) { + if (list_empty(&gl->gl_holders)) + fast_path = 1; + break; + } + /* + * If we have another strong holder (we cannot auto-demote) + * we are done. It keeps holding it until it is done. + */ + if (find_first_strong_holder(gl)) + break; - list_del_init(&gh->gh_list); - clear_bit(HIF_HOLDER, &gh->gh_iflags); - if (list_empty(&gl->gl_holders) && - !test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags) && - !test_bit(GLF_DEMOTE, &gl->gl_flags)) - fast_path = 1; + /* + * If we have a weak holder at the head of the list, it + * (and all others like it) must be auto-demoted. If there + * are no more weak holders, we exit the while loop. + */ + gh = find_first_holder(gl); + } if (!test_bit(GLF_LFLUSH, &gl->gl_flags) && demote_ok(gl)) gfs2_glock_add_to_lru(gl); - trace_gfs2_glock_queue(gh, 0); if (unlikely(!fast_path)) { gl->gl_lockref.count++; if (test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags) && @@ -1530,6 +1635,19 @@ void gfs2_glock_dq(struct gfs2_holder *gh) delay = gl->gl_hold_time; __gfs2_glock_queue_work(gl, delay); } +} + +/** + * gfs2_glock_dq - dequeue a struct gfs2_holder from a glock (release a glock) + * @gh: the glock holder + * + */ +void gfs2_glock_dq(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + __gfs2_glock_dq(gh); spin_unlock(&gl->gl_lockref.lock); } @@ -1692,6 +1810,7 @@ void gfs2_glock_dq_m(unsigned int num_gh, struct gfs2_holder *ghs) void gfs2_glock_cb(struct gfs2_glock *gl, unsigned int state) { + struct gfs2_holder mock_gh = { .gh_gl = gl, .gh_state = state, }; unsigned long delay = 0; unsigned long holdtime; unsigned long now = jiffies; @@ -1706,6 +1825,28 @@ void gfs2_glock_cb(struct gfs2_glock *gl, unsigned int state) if (test_bit(GLF_REPLY_PENDING, &gl->gl_flags)) delay = gl->gl_hold_time; } + /* + * Note 1: We cannot call demote_incompat_holders from handle_callback + * or gfs2_set_demote due to recursion problems like: gfs2_glock_dq -> + * handle_callback -> demote_incompat_holders -> gfs2_glock_dq + * Plus, we only want to demote the holders if the request comes from + * a remote cluster node because local holder conflicts are resolved + * elsewhere. + * + * Note 2: if a remote node wants this glock in EX mode, lock_dlm will + * request that we set our state to UNLOCKED. Here we mock up a holder + * to make it look like someone wants the lock EX locally. Any SH + * and DF requests should be able to share the lock without demoting. + * + * Note 3: We only want to demote the demoteable holders when there + * are no more strong holders. The demoteable holders might as well + * keep the glock until the last strong holder is done with it. + */ + if (!find_first_strong_holder(gl)) { + if (state == LM_ST_UNLOCKED) + mock_gh.gh_state = LM_ST_EXCLUSIVE; + demote_incompat_holders(gl, &mock_gh); + } handle_callback(gl, state, delay, true); __gfs2_glock_queue_work(gl, delay); spin_unlock(&gl->gl_lockref.lock); @@ -2097,6 +2238,8 @@ static const char *hflags2str(char *buf, u16 flags, unsigned long iflags) *p++ = 'H'; if (test_bit(HIF_WAIT, &iflags)) *p++ = 'W'; + if (test_bit(HIF_MAY_DEMOTE, &iflags)) + *p++ = 'D'; *p = 0; return buf; } diff --git a/fs/gfs2/glock.h b/fs/gfs2/glock.h index 31a8f2f649b5..9012487da4c6 100644 --- a/fs/gfs2/glock.h +++ b/fs/gfs2/glock.h @@ -150,6 +150,8 @@ static inline struct gfs2_holder *gfs2_glock_is_locked_by_me(struct gfs2_glock * list_for_each_entry(gh, &gl->gl_holders, gh_list) { if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) break; + if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags)) + continue; if (gh->gh_owner_pid == pid) goto out; } @@ -325,6 +327,24 @@ static inline void glock_clear_object(struct gfs2_glock *gl, void *object) spin_unlock(&gl->gl_lockref.lock); } +static inline void gfs2_holder_allow_demote(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + set_bit(HIF_MAY_DEMOTE, &gh->gh_iflags); + spin_unlock(&gl->gl_lockref.lock); +} + +static inline void gfs2_holder_disallow_demote(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + clear_bit(HIF_MAY_DEMOTE, &gh->gh_iflags); + spin_unlock(&gl->gl_lockref.lock); +} + extern void gfs2_inode_remember_delete(struct gfs2_glock *gl, u64 generation); extern bool gfs2_inode_already_deleted(struct gfs2_glock *gl, u64 generation); diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index 3b82fd2e917b..ca42d310fd4d 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -252,6 +252,7 @@ struct gfs2_lkstats { enum { /* States */ + HIF_MAY_DEMOTE = 1, HIF_HOLDER = 6, /* Set for gh that "holds" the glock */ HIF_WAIT = 10, }; From patchwork Tue Apr 12 05:15:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 560797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27F14C433F5 for ; Tue, 12 Apr 2022 05:17:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347853AbiDLFUA (ORCPT ); Tue, 12 Apr 2022 01:20:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347840AbiDLFT7 (ORCPT ); Tue, 12 Apr 2022 01:19:59 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62FDC34671; Mon, 11 Apr 2022 22:17:43 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23BMmklJ006846; Tue, 12 Apr 2022 05:17:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=C7yqYlH/U412/eBU0f5DtObhR2ocTAS3AAYCVqSkxm9o20sTrOnItf8ag6YFjyAma430 P9eQVdOqQu8Zq97i7nZlXOC84gp+CpWLCwjv8+Qz/M8Tg94CRCzc7QSyZ/bIHFjoZldH poqYV7P4A3kgq5GQnlR24HcwLk0Vxf6pcI9PGAUC5yyGtpQYZIM25odokLQOEdu9X0Tl AqqejrtHqU8ma6XkCMZlEKHToMKcyzZsNV3cGfoZR6klidFz6Jkg/i7Gkes2ziSoztWM 1UseyMwVGDksdWJ6RzHxY7Au/JaqjMPjhEEy2mXKM3mxGVF2UbPEEHOBGoAKOB5leBCJ 4Q== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb1rs5njj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:40 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23C5HIAC034872; Tue, 12 Apr 2022 05:17:40 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2042.outbound.protection.outlook.com [104.47.66.42]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k2sp1w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Apr 2022 05:17:39 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gehihMklmMmiBdzHWK2nW49oFESaR8ua6LrigPM7K6AS9r6w0iqVknSJ/9qAs9tb07NGiCwjkC2oGX5OljgHY3ETIe0A2gJeiYnU4/NmF4xTbf12/StkWyWHfeQfg5405ig7EcwNJReYjFB/33+HEI3nHs5aTTlpIntjzxdU0A8+vcrLxGiElyMLhOsCU+EfNOttL6A24v5rBZwbX2+wCweOrub4p/a4Zrk4hTgq2LGvMmRBwcphQZMDMpyflL5JGqbfxFihxHPFKehrUfc0tGNf+5o/7FCGu+ZU8mqbc9Tn3PYzBC63Zp2MDPudf4nskFvqn6z/wUaf/gp4Hloxqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=kWk2Ql1KNP/0dl91CxOOKxqbpIjKRBuB6AkI3fleJO7GVib8zgxRE1OrH+CrWo7i8GcR31BacRnAYFaVs5O2oxdQuV3fk4HIJJ+I5n5RkWeDe9Huq7jHmbTJ+DIse6Jga8Cv1YhP4+W7RtEdJcShvsVMUMzvXFnj1CLvL/k8bAQBLdoKxTubIPyz5zRcU2mJgoNEDyQm/kvTFG117KjmzLEnsYs0Y8jXzv5M9m9idY0yCofjh/8ZiAaWwk1EK3+Vr90dywV2hJfnrNX+yfltyarc1fDn7uZXi2CAkmOUPUfel7DvDN2qsMvjAKHEYcvGXywVV/HKxvwQkEeWt3L8Sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=DzGotWXOk7pkzPlejUSOW+htU2wB4ekSyDrdtkanOqKM+Px9F6zZppAbouCmIZgHA/T2SkZQX0ohsLwuiVrGCqoRm5uWGzXgxFffykT2Ol4z8zVLgU7ARDlCmMIaJWTHQ4sj283w4BA+R/jxGoxeHwAaaO2j6ZQ9hsXPpIGGB0E= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by BYAPR10MB3256.namprd10.prod.outlook.com (2603:10b6:a03:152::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5144.29; Tue, 12 Apr 2022 05:17:38 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Tue, 12 Apr 2022 05:17:38 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Linus Torvalds , Andreas Gruenbacher , David Hildenbrand , Anand Jain Subject: [PATCH v2 18/18 stable-5.15.y] mm: gup: make fault_in_safe_writeable() use fixup_user_fault() Date: Tue, 12 Apr 2022 13:15:15 +0800 Message-Id: <440141ae160ed87665522c46dc6ebfdc400de198.1649733186.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYAPR01CA0153.jpnprd01.prod.outlook.com (2603:1096:404:7e::21) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2c89b78f-d469-4f91-4a22-08da1c43c2a0 X-MS-TrafficTypeDiagnostic: BYAPR10MB3256:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6diTArpV82DahRgaEsqgRVT6W5ZW+dJEENmIhJSb2uY+Cjymoe7BgxV73CTcz1zOe0wHjtDpHHsKSLLRl+U/Y0D+XHM/fZuqbziLJ8cumq5XhqMqw3Q7ccPbAmu7Z3K0KRnMMgkTydxLNJqK6TWcStIw0x0Ui08XssQYRJ797tFju6mnYVefqChb2ANuoeLPg2EBYUGithyZ3C76agmIYkSA6Svo5xj7mWo+N1hinSH/T1Zd0/zjYPutxKGVzu/ZJGI8k/TRjsWDcTMGSNKGQSc80eb1wXdm0leJ+AO6NPlt8yGIL/U+axtOpqPjd9lxcWKfYsmpF9bgJzPXmRDAci3PyjvMfqpmUThKe3vnKO7Nr+MJdKzEnHJqOLlJFxxWJ6aCPUkFA7Rcaz3JZhpWB/T/v0T0lCbU6O8XAS/Rd1CT2FV2maLwINUu5PZh9KNd6Zo5/qo/NDKmyFHRcwohoZJui0t47Hx3zCQynLcVrqbHtg38XvTVLRlme85A5YonttyX/BGFbczwKtdQ9Md6c/faBuqCrtmhUlaZxgSsEF5ALhbfzQX/9ZbghGH13czGWdGFdxnotyIqrjrAEP9fgnQItfyWZT14lFRN6VYT/QsC++zyWyfJRGADN9cBLPcwBZDz7V49iGLNNCGK1VTtxkijstctQgi7P381pOLVv8QWEQAoJkXwt6VUNhC6vxCZBruPz2TszdA00ciUdiC9ZBLIDsM6lz8obZeYi3qRFhg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(38100700002)(8936002)(5660300002)(2906002)(83380400001)(6512007)(2616005)(66476007)(44832011)(107886003)(186003)(6506007)(6666004)(966005)(508600001)(66556008)(66946007)(6486002)(86362001)(54906003)(36756003)(8676002)(4326008)(6916009)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8O28c0Rht+q9CDpb4zxuQ3toVtSvndwVfdmwG3dVmJkVVD2jIrbc5vUHDyVJ9dIPakGOpGfhoN4IexA9UPd8Gkrg0okrnE4UgVIrBR9SpcYBBOQEUUWacTyNsI12jgUHRynoTqCCl55XiwZCPKROIBmHgpIx8PkfwTk+qH00hEH6/5T+wlL7lvfAQfPWPMQI/SAz59dedjoROcWCJOj4vv4a+XIL/Jf4h9qn7jAUZchiQgP8K4h/VekLA0XepRw9zpX8gFc770KD5htPefOfjJkRbVugjcqvw0NjJ7Mq8D9oo06apwXFpBcnWbFhil09lwUzca1GXzhuUtYSF4RNERE60ai2W+VMzZTpVxI/iAD/I2/2ZIbdagbD7IFQ2PbAj72q+zfv7E9LmghvgMnfpt4in+n0rLbyW7Dwg9G+A8YTCLR9xHIxocaEsOiLid17kaxiY41szg5YuAn6qMtvICXDNyn1J03ma5wCMvvJnQjOer61B5GDiMmwKCrZakp1RZvqP57Wmftap2EGjVXKHN4sGzQ3WiKj/yjxiYkd0SzWh7sIqeKBW+O6XUGCeD97UdkcJJVd4QO8zzIGprS1A1MlJRyETmcQRmn5iJDJ2/m6D//xsRKYe+pIDAiVCnb7QAHvDE9JdesO3PUr0nhsTa4CgCTXca7cA3i1f8Z65mp0rDX5Fbcvy7qGcEKBinBHE5zilgcKzmdLCiaDcKF8yvjAGatwbqqZIO/Eb5NFnkX1IF9jL5OCiZ0ermNY7WPBM5Q1IfG/9A3chaHuG/hKKC64rlBshf5HrD0OhdpycFIVEWUk6WD7yWy2How8HTTydWnteCbOwug6Vv3Elx4hmMR99lyel9BSuQT3PNSQEYDM40L2DOBlGPdFOlwljRb0KUpa3V01wNxf0mjnz9cR5Oj1iO5urxKi/oYeZdzIow4UfqKlTSsI27+fwPND2jyfaLq5fiNLfkiGZuy3nBXq0t6bfk1yqvJMtegMa/lwC6DM7gKVeT44TFfnZOEPdULklPiuvoRxBDM4LV+FVmwm04PLkCIGolDqaGS6nY2Xy88k8svwrQtQz2QavNoOuPr0Ns+EEECj+4FLvRNe/4+ze/dTWsKwjU4FZTGKPij7jWJCyenYI5AG4GUMH3Nrv6zJvWkGGqsw/V02aXol7d3TA2YLEsi29gjsO0FY4iiwXbq+ihkweZSLxa9kxfeTKCAdlsqNCfWGLbDsUN6P9KeUfsapP7jKvNu1Gn7b7DfIYRmg+Kk4t4h7baWb88ST6uQbdG8ODZMJHuomjyybmiOmaY/nSMLLLGtQSnj2fVx/Eht8qLsUFFN+hP+Q/TOLwBfGNWoMN4vZVhQxs+FqJfnbqBrcoUCUlufU6i5D+DfqajUGUBZRkhl1TY6hoVqalRhMDLgyjcewEo8aTXct/OMltEh1X70X3nruwEsJcVCpgbibtP1SpMqUtZZKBL/wgvGN6cj4GKe++6rpK+Z+oW3QfFCkCeuArgJizK5DHViUvL0c3CCnGQ0xOJh7X4qsd0xxoT08aH5cFeriZJe1kHGmFM2KvY682kWfrZvd5wJP0C4afNS6FOUWP48v2hGK7LrjGWwBttastmXJq/ryQEjESj788ITkLVxa16WvG/TO1mCNsFe0wXHetC0x7Tp0Tli4Naz2p1l/2cQaQT9gssWlc+v9VPRh6XDdtfYZiuIwi2PZvgTV3vMk2XbQDbTvGx+Zp1HRsEtQeFscjmKGfJBhAzusJECO1k3STNHkZTmGMUAnrLS60xsPoT5/OvsyFvUl X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2c89b78f-d469-4f91-4a22-08da1c43c2a0 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 05:17:38.2048 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KOx/8vRW4xWb50alxy4OFXeVhDyWp2HrQx5HkFi8pkEtDLFFQGyTev/Htvqulsj395UjDRUNUGJNi1lwSszWgQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3256 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.858 definitions=2022-04-12_01:2022-04-11, 2022-04-12 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204120024 X-Proofpoint-ORIG-GUID: xkzsNb0KTPrrGVbojFK5GUaNL7moOVM5 X-Proofpoint-GUID: xkzsNb0KTPrrGVbojFK5GUaNL7moOVM5 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Linus Torvalds commit fe673d3f5bf1fc50cdc4b754831db91a2ec10126 upstream Instead of using GUP, make fault_in_safe_writeable() actually force a 'handle_mm_fault()' using the same fixup_user_fault() machinery that futexes already use. Using the GUP machinery meant that fault_in_safe_writeable() did not do everything that a real fault would do, ranging from not auto-expanding the stack segment, to not updating accessed or dirty flags in the page tables (GUP sets those flags on the pages themselves). The latter causes problems on architectures (like s390) that do accessed bit handling in software, which meant that fault_in_safe_writeable() didn't actually do all the fault handling it needed to, and trying to access the user address afterwards would still cause faults. Reported-and-tested-by: Andreas Gruenbacher Fixes: cdd591fc86e3 ("iov_iter: Introduce fault_in_iov_iter_writeable") Link: https://lore.kernel.org/all/CAHc6FU5nP+nziNGG0JAF1FUx-GV7kKFvM7aZuU_XD2_1v4vnvg@mail.gmail.com/ Acked-by: David Hildenbrand Signed-off-by: Linus Torvalds Signed-off-by: Anand Jain --- mm/gup.c | 57 +++++++++++++++++++------------------------------------- 1 file changed, 19 insertions(+), 38 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index a4c6affe6df3..ba2ab7a223f8 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1723,11 +1723,11 @@ EXPORT_SYMBOL(fault_in_writeable); * @uaddr: start of address range * @size: length of address range * - * Faults in an address range using get_user_pages, i.e., without triggering - * hardware page faults. This is primarily useful when we already know that - * some or all of the pages in the address range aren't in memory. + * Faults in an address range for writing. This is primarily useful when we + * already know that some or all of the pages in the address range aren't in + * memory. * - * Other than fault_in_writeable(), this function is non-destructive. + * Unlike fault_in_writeable(), this function is non-destructive. * * Note that we don't pin or otherwise hold the pages referenced that we fault * in. There's no guarantee that they'll stay in memory for any duration of @@ -1738,46 +1738,27 @@ EXPORT_SYMBOL(fault_in_writeable); */ size_t fault_in_safe_writeable(const char __user *uaddr, size_t size) { - unsigned long start = (unsigned long)untagged_addr(uaddr); - unsigned long end, nstart, nend; + unsigned long start = (unsigned long)uaddr, end; struct mm_struct *mm = current->mm; - struct vm_area_struct *vma = NULL; - int locked = 0; + bool unlocked = false; - nstart = start & PAGE_MASK; + if (unlikely(size == 0)) + return 0; end = PAGE_ALIGN(start + size); - if (end < nstart) + if (end < start) end = 0; - for (; nstart != end; nstart = nend) { - unsigned long nr_pages; - long ret; - if (!locked) { - locked = 1; - mmap_read_lock(mm); - vma = find_vma(mm, nstart); - } else if (nstart >= vma->vm_end) - vma = vma->vm_next; - if (!vma || vma->vm_start >= end) - break; - nend = end ? min(end, vma->vm_end) : vma->vm_end; - if (vma->vm_flags & (VM_IO | VM_PFNMAP)) - continue; - if (nstart < vma->vm_start) - nstart = vma->vm_start; - nr_pages = (nend - nstart) / PAGE_SIZE; - ret = __get_user_pages_locked(mm, nstart, nr_pages, - NULL, NULL, &locked, - FOLL_TOUCH | FOLL_WRITE); - if (ret <= 0) + mmap_read_lock(mm); + do { + if (fixup_user_fault(mm, start, FAULT_FLAG_WRITE, &unlocked)) break; - nend = nstart + ret * PAGE_SIZE; - } - if (locked) - mmap_read_unlock(mm); - if (nstart == end) - return 0; - return size - min_t(size_t, nstart - start, size); + start = (start + PAGE_SIZE) & PAGE_MASK; + } while (start != end); + mmap_read_unlock(mm); + + if (size > (unsigned long)uaddr - start) + return size - ((unsigned long)uaddr - start); + return 0; } EXPORT_SYMBOL(fault_in_safe_writeable);