From patchwork Thu Apr 14 22:28:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3609C433FE for ; Thu, 14 Apr 2022 22:29:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233860AbiDNWbo (ORCPT ); Thu, 14 Apr 2022 18:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347011AbiDNWbn (ORCPT ); Thu, 14 Apr 2022 18:31:43 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F521C334E; Thu, 14 Apr 2022 15:29:16 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJsTXU008887; Thu, 14 Apr 2022 22:29:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=DwrdZQu0jlDMEJdjPDBxmVFgXoc2G+++c8YQtGyNNXuZBcru9oVl7Mxv48AWSQQq1/7s NzGm6u4+bxUIlaIe7+Z1IGMaDPo1D2n/Ay9NIWtIpI8Jbl8gCjlXogcnFvCrjb6Lu5XM 1Yoc9QTNdw1026TC9lxpaC5O0ZXhIfiapHKNSrFe7KCFUIsmJHDXVo+tiXf88rVq8piT XOaYWPNMEg7qHnel9szsChqob1//t5kANtCT9hP83L23Fb4sei+DLDCfkddI/wE/508D JAxPsNwA0svBC6gSk+nbcFTwyCmfp8gtmLzT7X7/dPxxQaIcPAFm3FYImPh611/Eufop Ew== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2p3ty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:14 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGVIr009355; Thu, 14 Apr 2022 22:29:13 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2108.outbound.protection.outlook.com [104.47.55.108]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fck15d90y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VFcsw5PUF//Mvw0Kx9uEpcens5x1QPB/SoAUnOnLLkezKZAVWc5yZ+w29FCLcElUXodumCf4RaMPncxfe9e5BWtGYXopu2m93l+HsYxlemU0SUU1rqxeHlbprxfGZJRN8avEEV1+zrzjVNCCXrLLPc37CvsS5WeUIt+5+c9WS9hw9WtEuc56nUCcjfvynXP2ZNbQlMCtmJmnjFnSoReFycxn5+N51qTRs0hl4+JTyIOLUfiYQ7+beM8/A/LXjO6sa+UFg7oFitdPl4LXCkr0xThpleT+eSH7bwRIWVg9163c/UtdU1R/C2INti8VKlWh+sAYJTZEzuUQd27FukEZYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=G6byUOtMxnJhyFSSxQ7fZ9fSdAwOhdclOBUnfkWPM+QPyhtu2rnWNK9zPn3WagvLL0GrK1xciamGbe4M3BqIAuJ3tp3+yNohzk1MRXtPL3YDunE3GPB7QVYGkksvAOj+RmWtzlX44v8yhSi7x3nALd9PuCoqMpp3fONhb//o2/c7kiAP9LppXFBm29Q7dQmw4ebTrLEQ56pUNsJrTJ210b2pWxppyGNQvay+HbEhLbgcLBWzsgXj66d0l30zvJ8UKHL3h2llHA7uXRJEiyb95Tct1lI8tDcA6hoeiC45A5+YbAasGe1FqVQYUuEw38yeZQGVFrY0dzY6hs47Mky26w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6bF9vyIk5yV9BuCYF/O3x6dh+jd+7z5G073ajZC+whU=; b=lyj5Bcex6AD9wVp6L+CsjuvXQLBHz3KmBHbcH4gJXHXPKK4rkQlsX6aUW7Gp1LLF1NPfyF1PKN7D+l+xOLuMmofvmFIFPHXN/hhYMiC/ChCJuBiOJ4harrx+GtPbHoMuwkstIV2bg0OCqb/rfWbxgskPQ0ZS95HS+zp4DBgd29Q= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:11 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:11 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 01/18 stable-5.15.y] gup: Turn fault_in_pages_{readable,writeable} into fault_in_{readable,writeable} Date: Fri, 15 Apr 2022 06:28:39 +0800 Message-Id: <92b6e65e73dd2764bef59e0e20b65143ab28914a.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYAPR03CA0020.apcprd03.prod.outlook.com (2603:1096:404:14::32) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3aef494a-dc48-43a8-535e-08da1e66328e X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pVgzDvnPKMPeZvZbcu8r86HUl5P2qCW8WTQYmL9Xwl3cdSi1Df0bX5JK5dFxVj7gTEQZqHsDKJjIHsWPtNRHvHZtdRlpxBKaNgaX2Qc4kj/tKc+/MhQntrydSJoDFRPqKg0gORaMsALPC8TwCcjf+R8H1G0aJ+nk9V2Az2XUHObVgCF/vO3WJMzW96aGTHosZxdOezShLLZtXy+EVgKgx9Kcqe50xdoJB0Tjp0wzV8J/IQ8Sn0CQE/52CIHPBfqwbWl43UkygAlYHWJnj1df1fY8YUyzFo065ct/CC1SZGA2KR9tuLDQgUkYdeVBcLPdnSGptfPI9YPEDerslHNwNz1r7o0hO5X0C9ZL5PvXUI5vAA8Uc2KwnOBtzqmf5fQ6AVXXVLnAwy7kek/S1Tve9/GdccOyeDZgTMSChDfx4xzEpCymComlF87cbnAW0/aaSt1g7F8+k/DJ+LSzA5VnxcIdF4juVr+tVxWkQbrXrknigW1Lsxg05JU4YhaVtXyBL6Fm4dHSa8yK0u61w/bDfx2sAdhmTTis962NhZ6qsVT+Ot1TLk5OlfPLuBN7O8WJ7O7wDKJhqrkPn2BdIHQDTy98j9TZGZlT1hIsvaqgs7Obz3Q8FAkZGdMhf5Cn0vYDEyrqpmAD5itsUBhTiwKGHw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(30864003)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: v1cKlxzwIotKBeRf4bVdGa2G/f+3oAFddP+5qz/ymLGc9SRSCa8EpvrXOKtA3DQu36ZSh5mMw/mos338CaA5jxgBd6armfbRraeoZvpZ7nZ1Jgkl6LqbTzYCwgTbrt93WbogbPDP/LS7/xG/+j7zWecQY5BwxJcO6QIjt3QG+GWCJwFeQMDVDIvuvQz274G8vw7rrlrbfs1dYxWeOHFNCm0Bik5u7DS/uPGsN5Mm9Y3sseFgjVmZLvDjlz27BNAJWurJuJfwz/xyj88HRDNNterRqdX1WQ+PKrMu7jWloU6jF3K3vwy8gUqrxME1CXCLYF/7rLoKTf48HImvv3mkUEYhdmmbymcO2CB7MEy7ESIk4Fk91++Vj+5+t1AwAZp9CbKchgjWhpPgYzcPDfuNUj52Z5sEfyeg68/DhJ+pQdGcPwHDDoHFrn8PgiYa2IA3igaVsE8glwdsCgYrOuNA0qp2JpuWFvJ66cv0KFrsH1VJriLMkXyezs/sBZpEy5+Gk+6yjBNbVf6gLDJzKH0RAhh08BoitYjKazV/ocrzP171TwOeCQFK9gFudQpn6vL/wLcB1Ys4cvcty9CqMJJUJm0haYYsolsnKBUGahKC9cqEfR0A1/b8ASVDrFC92Av1YninDaGSOktrEwoJPtX82AveZjNDVfkW2Pc83trf7PrchgbKMBLU1zFbeyXQACahIDDqCXX0ryZoJr6M9irGEGhbRu9Yiaz6N+mfjgz6JZkPyIIVguW4wJxrf+4pIMMdDMA33Nr9l/wdhcrnNhYC+kgF+Hku0kSHFo7nAIHWzdaFLXoZa10YRdwwjkK4p2WC4ssVqn7AqoJG0llVNmooj4QQBH97zFS61VEB3sCQmilCcBXOl17DFzPzKKrWQLLXQx2YYuVmxvBNv4lByAglNRemgqIuPEsVcpuGK9Sp7MEgJkq4UYXz43XS95SNh7tJGP9glWUDFZY3fIT7RdFix/y50kzxXyDws5tHyG1wMZoG5EMtnrv7KixRIfwc/bRnv1Lk93m+HIWMrB1DterSMIlppUIjRhbXJIYoIJF7XHpdh0UILqBF649l4fl19ZGy2WieScslOZWn/6EEQNGrP3M+yJdfoBTa4QzaWEf1mGEtVoPsjH7MsTqM6cjOIFsl0JMTnhDdG2A+vNtqqelZZOcbEJExbaaZSwjxgwlp0ajxocgsx+y6R8q0GcTuQBBZEet2GiImThGpK9utY42/0QUyF4CukTNh6s1hKv8yIAJ0if0ufnkaJ+iHmqWEtckzOPN8xP+KHGpViqhvZ8U46P/Cnp0Qh4BgUEmwb77XeRPF+jx/ZCotX1QjrMHfUzpbFxvZuX9Z41ufntyylbsSOztifbpam4jTM+8bHZgBgRaCb5OO3yJdXuxvYGvJ+qvVYzwuAeGI0W9L70wGo5PUQ35lRO07MIF/fs5w+iYsCQkmD+6Bm7xqQ8rVhnfF3mahniLMp5oP++AI5nxdYaXz0yWl23CiazWxjRNt3QA2LIIeoyGyYhxwNuwfwrRjaaQrw8pOqUA4b1R8XzEnNZKTcW2xFKD0RF/Q0v9zyzcnDzWKRiQ4PFTSsV6xSg5o8zvinPeySt+qsqbpycflMKl9ph2IVPCwFaPQmiybIrTPTNlQFYJSHHBuzxaz862SWLO4ZXPuIKcDGCalqkLrytqkdvesDKOvk+NYE1TIAADNm1BOzJYz6lD3Odj1h5D8b8rpo7qPjyG2FfWzgLarULw/6ldSGypni0q4wh6jTVxQWBvQw/EmkQhygJ4MOG9ro+E0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3aef494a-dc48-43a8-535e-08da1e66328e X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:11.1438 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: enG1u47Awzw3TPra7/55AiHwalYn+HHDKPiBWSuSm5cM8CpDAaKdM6/O174Qm9NU6hMWyiV66z4C2mdNjDa5rQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: oSSv1eT5uTelYNasziBpfLJqUTEI-Ske X-Proofpoint-GUID: oSSv1eT5uTelYNasziBpfLJqUTEI-Ske Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit bb523b406c849eef8f265a07cd7f320f1f177743 upstream Turn fault_in_pages_{readable,writeable} into versions that return the number of bytes not faulted in, similar to copy_to_user, instead of returning a non-zero value when any of the requested pages couldn't be faulted in. This supports the existing users that require all pages to be faulted in as well as new users that are happy if any pages can be faulted in. Rename the functions to fault_in_{readable,writeable} to make sure this change doesn't silently break things. Neither of these functions is entirely trivial and it doesn't seem useful to inline them, so move them to mm/gup.c. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- arch/powerpc/kernel/kvm.c | 3 +- arch/powerpc/kernel/signal_32.c | 4 +- arch/powerpc/kernel/signal_64.c | 2 +- arch/x86/kernel/fpu/signal.c | 7 ++- drivers/gpu/drm/armada/armada_gem.c | 7 ++- fs/btrfs/ioctl.c | 5 +- include/linux/pagemap.h | 57 ++--------------------- lib/iov_iter.c | 10 ++-- mm/filemap.c | 2 +- mm/gup.c | 72 +++++++++++++++++++++++++++++ 10 files changed, 93 insertions(+), 76 deletions(-) diff --git a/arch/powerpc/kernel/kvm.c b/arch/powerpc/kernel/kvm.c index d89cf802d9aa..6568823cf306 100644 --- a/arch/powerpc/kernel/kvm.c +++ b/arch/powerpc/kernel/kvm.c @@ -669,7 +669,8 @@ static void __init kvm_use_magic_page(void) on_each_cpu(kvm_map_magic_page, &features, 1); /* Quick self-test to see if the mapping works */ - if (fault_in_pages_readable((const char *)KVM_MAGIC_PAGE, sizeof(u32))) { + if (fault_in_readable((const char __user *)KVM_MAGIC_PAGE, + sizeof(u32))) { kvm_patching_worked = false; return; } diff --git a/arch/powerpc/kernel/signal_32.c b/arch/powerpc/kernel/signal_32.c index f2da879264bc..3e053e2fd6b6 100644 --- a/arch/powerpc/kernel/signal_32.c +++ b/arch/powerpc/kernel/signal_32.c @@ -1048,7 +1048,7 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext __user *, old_ctx, if (new_ctx == NULL) return 0; if (!access_ok(new_ctx, ctx_size) || - fault_in_pages_readable((u8 __user *)new_ctx, ctx_size)) + fault_in_readable((char __user *)new_ctx, ctx_size)) return -EFAULT; /* @@ -1239,7 +1239,7 @@ SYSCALL_DEFINE3(debug_setcontext, struct ucontext __user *, ctx, #endif if (!access_ok(ctx, sizeof(*ctx)) || - fault_in_pages_readable((u8 __user *)ctx, sizeof(*ctx))) + fault_in_readable((char __user *)ctx, sizeof(*ctx))) return -EFAULT; /* diff --git a/arch/powerpc/kernel/signal_64.c b/arch/powerpc/kernel/signal_64.c index bb9c077ac132..d1e1fc0acbea 100644 --- a/arch/powerpc/kernel/signal_64.c +++ b/arch/powerpc/kernel/signal_64.c @@ -688,7 +688,7 @@ SYSCALL_DEFINE3(swapcontext, struct ucontext __user *, old_ctx, if (new_ctx == NULL) return 0; if (!access_ok(new_ctx, ctx_size) || - fault_in_pages_readable((u8 __user *)new_ctx, ctx_size)) + fault_in_readable((char __user *)new_ctx, ctx_size)) return -EFAULT; /* diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index 831b25c5e705..7f71bd4dcd0d 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -205,7 +205,7 @@ int copy_fpstate_to_sigframe(void __user *buf, void __user *buf_fx, int size) fpregs_unlock(); if (ret) { - if (!fault_in_pages_writeable(buf_fx, fpu_user_xstate_size)) + if (!fault_in_writeable(buf_fx, fpu_user_xstate_size)) goto retry; return -EFAULT; } @@ -278,10 +278,9 @@ static int restore_fpregs_from_user(void __user *buf, u64 xrestore, if (ret != -EFAULT) return -EINVAL; - ret = fault_in_pages_readable(buf, size); - if (!ret) + if (!fault_in_readable(buf, size)) goto retry; - return ret; + return -EFAULT; } /* diff --git a/drivers/gpu/drm/armada/armada_gem.c b/drivers/gpu/drm/armada/armada_gem.c index 21909642ee4c..8fbb25913327 100644 --- a/drivers/gpu/drm/armada/armada_gem.c +++ b/drivers/gpu/drm/armada/armada_gem.c @@ -336,7 +336,7 @@ int armada_gem_pwrite_ioctl(struct drm_device *dev, void *data, struct drm_armada_gem_pwrite *args = data; struct armada_gem_object *dobj; char __user *ptr; - int ret; + int ret = 0; DRM_DEBUG_DRIVER("handle %u off %u size %u ptr 0x%llx\n", args->handle, args->offset, args->size, args->ptr); @@ -349,9 +349,8 @@ int armada_gem_pwrite_ioctl(struct drm_device *dev, void *data, if (!access_ok(ptr, args->size)) return -EFAULT; - ret = fault_in_pages_readable(ptr, args->size); - if (ret) - return ret; + if (fault_in_readable(ptr, args->size)) + return -EFAULT; dobj = armada_gem_object_lookup(file, args->handle); if (dobj == NULL) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 6a863b3f6de0..bf53af8694f8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2258,9 +2258,8 @@ static noinline int search_ioctl(struct inode *inode, key.offset = sk->min_offset; while (1) { - ret = fault_in_pages_writeable(ubuf + sk_offset, - *buf_size - sk_offset); - if (ret) + ret = -EFAULT; + if (fault_in_writeable(ubuf + sk_offset, *buf_size - sk_offset)) break; ret = btrfs_search_forward(root, &key, path, sk->min_transid); diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 62db6b0176b9..9fe94f7a4f7e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -733,61 +733,10 @@ int wait_on_page_private_2_killable(struct page *page); extern void add_page_wait_queue(struct page *page, wait_queue_entry_t *waiter); /* - * Fault everything in given userspace address range in. + * Fault in userspace address range. */ -static inline int fault_in_pages_writeable(char __user *uaddr, size_t size) -{ - char __user *end = uaddr + size - 1; - - if (unlikely(size == 0)) - return 0; - - if (unlikely(uaddr > end)) - return -EFAULT; - /* - * Writing zeroes into userspace here is OK, because we know that if - * the zero gets there, we'll be overwriting it. - */ - do { - if (unlikely(__put_user(0, uaddr) != 0)) - return -EFAULT; - uaddr += PAGE_SIZE; - } while (uaddr <= end); - - /* Check whether the range spilled into the next page. */ - if (((unsigned long)uaddr & PAGE_MASK) == - ((unsigned long)end & PAGE_MASK)) - return __put_user(0, end); - - return 0; -} - -static inline int fault_in_pages_readable(const char __user *uaddr, size_t size) -{ - volatile char c; - const char __user *end = uaddr + size - 1; - - if (unlikely(size == 0)) - return 0; - - if (unlikely(uaddr > end)) - return -EFAULT; - - do { - if (unlikely(__get_user(c, uaddr) != 0)) - return -EFAULT; - uaddr += PAGE_SIZE; - } while (uaddr <= end); - - /* Check whether the range spilled into the next page. */ - if (((unsigned long)uaddr & PAGE_MASK) == - ((unsigned long)end & PAGE_MASK)) { - return __get_user(c, end); - } - - (void)c; - return 0; -} +size_t fault_in_writeable(char __user *uaddr, size_t size); +size_t fault_in_readable(const char __user *uaddr, size_t size); int add_to_page_cache_locked(struct page *page, struct address_space *mapping, pgoff_t index, gfp_t gfp_mask); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index c5b2f0f4b8a8..2e07a4b083ed 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -191,7 +191,7 @@ static size_t copy_page_to_iter_iovec(struct page *page, size_t offset, size_t b buf = iov->iov_base + skip; copy = min(bytes, iov->iov_len - skip); - if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_pages_writeable(buf, copy)) { + if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_writeable(buf, copy)) { kaddr = kmap_atomic(page); from = kaddr + offset; @@ -275,7 +275,7 @@ static size_t copy_page_from_iter_iovec(struct page *page, size_t offset, size_t buf = iov->iov_base + skip; copy = min(bytes, iov->iov_len - skip); - if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_pages_readable(buf, copy)) { + if (IS_ENABLED(CONFIG_HIGHMEM) && !fault_in_readable(buf, copy)) { kaddr = kmap_atomic(page); to = kaddr + offset; @@ -447,13 +447,11 @@ int iov_iter_fault_in_readable(const struct iov_iter *i, size_t bytes) bytes = i->count; for (p = i->iov, skip = i->iov_offset; bytes; p++, skip = 0) { size_t len = min(bytes, p->iov_len - skip); - int err; if (unlikely(!len)) continue; - err = fault_in_pages_readable(p->iov_base + skip, len); - if (unlikely(err)) - return err; + if (fault_in_readable(p->iov_base + skip, len)) + return -EFAULT; bytes -= len; } } diff --git a/mm/filemap.c b/mm/filemap.c index 1293c3409e42..d697b3446a4a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -90,7 +90,7 @@ * ->lock_page (filemap_fault, access_process_vm) * * ->i_rwsem (generic_perform_write) - * ->mmap_lock (fault_in_pages_readable->do_page_fault) + * ->mmap_lock (fault_in_readable->do_page_fault) * * bdi->wb.list_lock * sb_lock (fs/fs-writeback.c) diff --git a/mm/gup.c b/mm/gup.c index 52f08e3177e9..e063cb2bb187 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1681,6 +1681,78 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, } #endif /* !CONFIG_MMU */ +/** + * fault_in_writeable - fault in userspace address range for writing + * @uaddr: start of address range + * @size: size of address range + * + * Returns the number of bytes not faulted in (like copy_to_user() and + * copy_from_user()). + */ +size_t fault_in_writeable(char __user *uaddr, size_t size) +{ + char __user *start = uaddr, *end; + + if (unlikely(size == 0)) + return 0; + if (!PAGE_ALIGNED(uaddr)) { + if (unlikely(__put_user(0, uaddr) != 0)) + return size; + uaddr = (char __user *)PAGE_ALIGN((unsigned long)uaddr); + } + end = (char __user *)PAGE_ALIGN((unsigned long)start + size); + if (unlikely(end < start)) + end = NULL; + while (uaddr != end) { + if (unlikely(__put_user(0, uaddr) != 0)) + goto out; + uaddr += PAGE_SIZE; + } + +out: + if (size > uaddr - start) + return size - (uaddr - start); + return 0; +} +EXPORT_SYMBOL(fault_in_writeable); + +/** + * fault_in_readable - fault in userspace address range for reading + * @uaddr: start of user address range + * @size: size of user address range + * + * Returns the number of bytes not faulted in (like copy_to_user() and + * copy_from_user()). + */ +size_t fault_in_readable(const char __user *uaddr, size_t size) +{ + const char __user *start = uaddr, *end; + volatile char c; + + if (unlikely(size == 0)) + return 0; + if (!PAGE_ALIGNED(uaddr)) { + if (unlikely(__get_user(c, uaddr) != 0)) + return size; + uaddr = (const char __user *)PAGE_ALIGN((unsigned long)uaddr); + } + end = (const char __user *)PAGE_ALIGN((unsigned long)start + size); + if (unlikely(end < start)) + end = NULL; + while (uaddr != end) { + if (unlikely(__get_user(c, uaddr) != 0)) + goto out; + uaddr += PAGE_SIZE; + } + +out: + (void)c; + if (size > uaddr - start) + return size - (uaddr - start); + return 0; +} +EXPORT_SYMBOL(fault_in_readable); + /** * get_dump_page() - pin user page in memory while writing it to core dump * @addr: user address From patchwork Thu Apr 14 22:28:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3072DC433EF for ; Thu, 14 Apr 2022 22:29:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347031AbiDNWbu (ORCPT ); Thu, 14 Apr 2022 18:31:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347025AbiDNWbt (ORCPT ); Thu, 14 Apr 2022 18:31:49 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 550872BEE; Thu, 14 Apr 2022 15:29:22 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKFdKH031973; Thu, 14 Apr 2022 22:29:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=VXhvYgHCI8teKTC0JpwFjQ0ZxEBrqYZP/5krj2OwKvc=; b=Q36nmzmqI1qxu1Jw41yILPRh1Kq+3ZnXh4Ku9JynwmWxbB6MZaOVyBBsuNfJaFVI4Up4 6MIgFwpTapi+f+cAgb3m/BZqbYX0kKVlhDludReiXxDd3DiH8chehGG40Hz4KAXoPEKu rxfrNGhOZrajlj3+VHg0aJ02kvDGFj9GHDJf1KE/bcBvxZnspIbrPjZA1x96ZM0KY7gi vxcHaL/DYGg2KyqE6SXP5hfBKf8ltFrJCqClow2u3evmguDvs10jsTCmrlKIE4OagY0Y eLEz3swSncxBCs8SYP1kr3VLMoSGU3c3Arlf0p4BhrkwJWtiXyX1F5QeWH4cS10GsG/w rQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jddp1h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:20 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMHW6x014857; Thu, 14 Apr 2022 22:29:19 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2101.outbound.protection.outlook.com [104.47.55.101]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5uc46-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RGQZHDK5zWGY9bellHuyHNLL/+urLn3i+4GIbxBm6u7Ef/V3S2CEILidLazPrGIEc7KKEd9PuF8WXRdDH1wXyfc/Fk/EL3KsaVqVMwHJci8yawtLD6a65GDVdbVATAs5zJ2GPyrrvtWSfYZQqy93DpCemJGn7b/qjd5C1DLGqsdnNqbs92lrUfIJDEpRn3Zb6MDQ2UQZPpjGOJ7aDsq4PvDAKXmdyIsz1coeBUGzMdGqT26gU1VFCwirQRzY81YAaPOpNmnNRBp+Cfi5OeFCk32Za20RAkhmfeDzHKm9bnCfE+tpXJ1ATOWsdpl5e6EyoyFVX0/D+Z5y1oRJzcrXvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VXhvYgHCI8teKTC0JpwFjQ0ZxEBrqYZP/5krj2OwKvc=; b=X7UwtBxkycRJFtWHj2nyeGA+bY4v1W9TJw4+DdeLm+c4mkNhDOcRI82fUt/o89TB3prUX57MVwow8JGVoch1C01qdAOSGi0DKxkGh6KJ4PKx2FLMwdBSlM3iFL9/kqBffLsz1SBikPPMLD5AtZjLXRlLNBSMYoEq+REDK+wFooA76Kn5Xp34lCoH1iIUPDGqc4sGc4dxibZt5+g0+eH4HEQiEDQhVhVMvcLvezOi83tQqtuKzmVN1pHXk8U1DpoSmkF36MDFwitK40XaejC5be8awKzp91hygwjlal/UELdFRwpfS8xqBkPMVyLuOLlF7/LH/Cbo21BdpWlHZbNU9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VXhvYgHCI8teKTC0JpwFjQ0ZxEBrqYZP/5krj2OwKvc=; b=X30shtu4ZyNvlaIY9S/mpfvTd0pWosim+CzgRkeJgjPuaAqHtMcZH7jhoaQxsF3NFj2Bhy4lCCZBA3TQbThmHboAki4gyi9Dx1Vt7Pk1s+McjAX2tABMtxSE48wwyoelYMstupfIX7M3kzzgRMT9jKNjLpCOoUc5E2WJ0AKSn3k= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:17 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:17 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 02/18 stable-5.15.y] iov_iter: Turn iov_iter_fault_in_readable into fault_in_iov_iter_readable Date: Fri, 15 Apr 2022 06:28:40 +0800 Message-Id: <2f18cef5634943c5bcd007b3753c3839feee9bd9.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TY2PR01CA0018.jpnprd01.prod.outlook.com (2603:1096:404:a::30) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1973e846-b578-47ad-cb2b-08da1e66362d X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uEqHMYWi2VxifOcm85KhbJKqGaqxlSMNPzFND8Eb234sBHJGr0lCwRaNoJNA9jWM6Ebpvu/gNyDVhRXVeIQI71qGGYkKKG8OHKcjP3SWEG59aOUxmv4vWgA/meVzTtGJo+xGBxfzUs6/VkCdItMI+gRcrSyFfGaL0Bkb2NAjMriW4hjZRDTRWeEcbzEtpN87dIkIogRkYUSj0TGgW4kbrz136K17dpbK3KhJpz2VHJFPY/HPBFGO1FayCvzSH7JYpOqX8zAWwNsXv++A4jNFtG/afQIjOnHglcxenRaMWTinPT7S8t+zBMNS/MasncQ8p3z4tjY9EuT42srBrYVrFHGHoe2+qkFEF7u4GMp7pp4BHnuDe4rW2LwUGU8l+wHAadi0MwXYBVpTyr+MdhTOsq4HwzLYsbbq4Jaa2EuzwiG8Jr/dpGbBNi3zfZb9RKopar9/eiUNezuyWqnMDlM6z5+o0JMQRn+tysPiUAB27hCoSX13lDxjs7W0L6N9ntfWxHO7E86+1r7xiOgNPTLMk7kox9u5bvL35AFx+AZEZFOKG2Zu0d8NXB9KlyjUH0ADG95jz5gfP+fwyKAttJ73nwG7rEIEUTwnbGwzTmOHLRkkk+x58dpMd4Uuki1/OGtrA0dCpUE2zCcjvz0yXtuHCQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: lqn+tZVBck7wNjXXtQ8xAKysJIjceLZiUNa37o7jOue7fCSlOJFlNMmJh6VmiZ4sNn9fgYLyRtUY7khnoShOMsYmDUvHzI2NWPcV0mRF6bp6ZML7u7kw3YdIhbdrtRLIM43lo4tnsS0Cs4VdAFi2hgqeUEhGUlcOJuExWPHMt9gjJ2Q/1dMKH/3d5qn5SCd5Wo3Meaho4/GPqfse4jeQFFRLtv7tmLM2N8ZKshGYcF5rKu5bAgUZEZrnl+9qP1Up+sylP0FwBed3CMgiOHefucbehH0OuSjwTOBcB4gJ6IyXio/9KIfTY2rWzkKshtA23VRFtEcnSHMAei169ZPQ2Nr9j9352mxDY91px272u/eH2T2xbmmDh6pdfH80cWUy3VG+aChv7hyUzhQebOu32fTs2HyxD16Zfcr+knJpiRpc2WMJcmXz5UNGUoqJCEuIWCC0JVJ+yyAUGenkSvmhmnerI7e7DaUGquFXlUmNEZUbH1mdm4oyBuoZt3MCcGy7moYISjrVXOXeErjXigSRMSOMLSGIV7UTayzwSftkI5spcpLmOvs6vLJfRLNDPdJ5L35ey6vjDlK0n3Vh6EeEKOzBcsS23Av8fU2vVpz1ejcnZEwBBkQLmhZEpLQSTqrchd0jx3BzgyetPK0kIVMSd5MxAtw43YjWJwjNxQzE3XXszgPOKy49vDwhnnDdTARffdd/QDM4euddRg5c1t8eI+f5fold2UHnBxc4gtkb0e2tuF/Xp5sh29D9ULXcWPfVAewdgvxbEq6RLPgCT7HwXablBtOq2kbfqtY+WLfRVecmwafAF6f6zopxayWckBOQJhtIv1kgExE9RjiXc+smw7zxde3Ikt/jXolt+GPzm0MpqgQCantbWHLf0byK9I7TC2MZFbg3QzHDoKDi4DlgLjaTiwzCOXN9eG+p+7vRRa60qGcAXincfaSLxW1WCcIHVbPtWO2FZqL1JTs5joKYNxLNdeT6ELPm8kfi3PemZCq7h8Hagu7l0RJtFIa4BxggHT2+HcTIeLtsgoh3+NDUMxbgdZykGy1hgXmisCBmkAko0f/Z+SA16N2mISwgmNUJqRgAe1UvwpR34BcTtdZSXeVIhljEBTq9rr5VFWbZJhiWD3aCdk3p79htfBILAgbhTQjbo5KgDgc2DDnFoKfSoJHhuLwlA6pXhNYIl0ocmMNKK1u52flvO9a60IAxqyY7FWzPOPX/ls4IBHDtzeOXZh2tz6jCiSwZ5eEta5ahQrs2R9qS1Ii5KrxT+Vx6587qwlFHDLVE5XIpXu4HrqfwjoS9gadNFeaCndD2hFIB3oYEgqQjrm0s2/tip7WFTJ5yI0y3oMZogVffHSZ14cxQghSTFjbK4JHAiEqMTqeK1InCbbbEuI80ElfrB4fyXeb9qwYH6WXdSUpLjGKfSUiZYRHATB0qwqic4GrkjNkf8kDw2345DKHHbhwD2Tj1eryAGc8IZ61kpI+TsDnTzCbqo7s1TFSffTb0oIN4iWWcREcSA+S7ZvgMqqe4tkLTBa/N2Bxrr/ltbtYz2goG15bzLqHPCkJW7w7KsAF1MnAuou9emWNL+7a1DinpwOwkPPs6DyAzwHa+7DuhmVCOwcui2MboAaP6Rsfx9Y1rFskqGuyeolrRA8+jx0iqjrZAxKEm6Sg/3CeiBwyWnjUbCy8iwG/d0rGqfuXdFfYIBmWAEnec4SipifaB52njtmCxwZo3WPB7+zI626/SlhQVLZXEKK6EINYmv/PgrZTkyDBlY5i4Ki0wL7JRAlXn4X0O52Br X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1973e846-b578-47ad-cb2b-08da1e66362d X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:17.2533 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: E6VTCrn3pk3BO9xMNW7bunZHGBnDxYCCcu1UfOSdYOAU783/NFfApLVpCYM16rNtEUd0up5j57Nfg+EO1BsvMQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: DSJ13krJ9h6NiGKMuaM4MKtdcbI0il08 X-Proofpoint-GUID: DSJ13krJ9h6NiGKMuaM4MKtdcbI0il08 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit a6294593e8a1290091d0b078d5d33da5e0cd3dfe upstream Turn iov_iter_fault_in_readable into a function that returns the number of bytes not faulted in, similar to copy_to_user, instead of returning a non-zero value when any of the requested pages couldn't be faulted in. This supports the existing users that require all pages to be faulted in as well as new users that are happy if any pages can be faulted in. Rename iov_iter_fault_in_readable to fault_in_iov_iter_readable to make sure this change doesn't silently break things. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/btrfs/file.c | 2 +- fs/f2fs/file.c | 2 +- fs/fuse/file.c | 2 +- fs/iomap/buffered-io.c | 2 +- fs/ntfs/file.c | 2 +- fs/ntfs3/file.c | 2 +- include/linux/uio.h | 2 +- lib/iov_iter.c | 33 +++++++++++++++++++++------------ mm/filemap.c | 2 +- 9 files changed, 29 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index a1762363f61f..5bf4304366e9 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1709,7 +1709,7 @@ static noinline ssize_t btrfs_buffered_write(struct kiocb *iocb, * Fault pages before locking them in prepare_pages * to avoid recursive lock */ - if (unlikely(iov_iter_fault_in_readable(i, write_bytes))) { + if (unlikely(fault_in_iov_iter_readable(i, write_bytes))) { ret = -EFAULT; break; } diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c index 0e14dc41ed4e..8ef92719c679 100644 --- a/fs/f2fs/file.c +++ b/fs/f2fs/file.c @@ -4279,7 +4279,7 @@ static ssize_t f2fs_file_write_iter(struct kiocb *iocb, struct iov_iter *from) size_t target_size = 0; int err; - if (iov_iter_fault_in_readable(from, iov_iter_count(from))) + if (fault_in_iov_iter_readable(from, iov_iter_count(from))) set_inode_flag(inode, FI_NO_PREALLOC); if ((iocb->ki_flags & IOCB_NOWAIT)) { diff --git a/fs/fuse/file.c b/fs/fuse/file.c index bc50a9fa84a0..71e9e301e569 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -1164,7 +1164,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia, again: err = -EFAULT; - if (iov_iter_fault_in_readable(ii, bytes)) + if (fault_in_iov_iter_readable(ii, bytes)) break; err = -ENOMEM; diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 97119ec3b850..fe10d8a30f6b 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -757,7 +757,7 @@ static loff_t iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i) * same page as we're writing to, without it being marked * up-to-date. */ - if (unlikely(iov_iter_fault_in_readable(i, bytes))) { + if (unlikely(fault_in_iov_iter_readable(i, bytes))) { status = -EFAULT; break; } diff --git a/fs/ntfs/file.c b/fs/ntfs/file.c index ab4f3362466d..a43adeacd930 100644 --- a/fs/ntfs/file.c +++ b/fs/ntfs/file.c @@ -1829,7 +1829,7 @@ static ssize_t ntfs_perform_write(struct file *file, struct iov_iter *i, * pages being swapped out between us bringing them into memory * and doing the actual copying. */ - if (unlikely(iov_iter_fault_in_readable(i, bytes))) { + if (unlikely(fault_in_iov_iter_readable(i, bytes))) { status = -EFAULT; break; } diff --git a/fs/ntfs3/file.c b/fs/ntfs3/file.c index 43b1451bff53..54b9599640ef 100644 --- a/fs/ntfs3/file.c +++ b/fs/ntfs3/file.c @@ -989,7 +989,7 @@ static ssize_t ntfs_compress_write(struct kiocb *iocb, struct iov_iter *from) frame_vbo = pos & ~(frame_size - 1); index = frame_vbo >> PAGE_SHIFT; - if (unlikely(iov_iter_fault_in_readable(from, bytes))) { + if (unlikely(fault_in_iov_iter_readable(from, bytes))) { err = -EFAULT; goto out; } diff --git a/include/linux/uio.h b/include/linux/uio.h index 207101a9c5c3..d18458af6681 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -133,7 +133,7 @@ size_t copy_page_from_iter_atomic(struct page *page, unsigned offset, size_t bytes, struct iov_iter *i); void iov_iter_advance(struct iov_iter *i, size_t bytes); void iov_iter_revert(struct iov_iter *i, size_t bytes); -int iov_iter_fault_in_readable(const struct iov_iter *i, size_t bytes); +size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t bytes); size_t iov_iter_single_seg_count(const struct iov_iter *i); size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, struct iov_iter *i); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 2e07a4b083ed..b8de180420c7 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -431,33 +431,42 @@ static size_t copy_page_to_iter_pipe(struct page *page, size_t offset, size_t by } /* + * fault_in_iov_iter_readable - fault in iov iterator for reading + * @i: iterator + * @size: maximum length + * * Fault in one or more iovecs of the given iov_iter, to a maximum length of - * bytes. For each iovec, fault in each page that constitutes the iovec. + * @size. For each iovec, fault in each page that constitutes the iovec. + * + * Returns the number of bytes not faulted in (like copy_to_user() and + * copy_from_user()). * - * Return 0 on success, or non-zero if the memory could not be accessed (i.e. - * because it is an invalid address). + * Always returns 0 for non-userspace iterators. */ -int iov_iter_fault_in_readable(const struct iov_iter *i, size_t bytes) +size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t size) { if (iter_is_iovec(i)) { + size_t count = min(size, iov_iter_count(i)); const struct iovec *p; size_t skip; - if (bytes > i->count) - bytes = i->count; - for (p = i->iov, skip = i->iov_offset; bytes; p++, skip = 0) { - size_t len = min(bytes, p->iov_len - skip); + size -= count; + for (p = i->iov, skip = i->iov_offset; count; p++, skip = 0) { + size_t len = min(count, p->iov_len - skip); + size_t ret; if (unlikely(!len)) continue; - if (fault_in_readable(p->iov_base + skip, len)) - return -EFAULT; - bytes -= len; + ret = fault_in_readable(p->iov_base + skip, len); + count -= len - ret; + if (ret) + break; } + return count + size; } return 0; } -EXPORT_SYMBOL(iov_iter_fault_in_readable); +EXPORT_SYMBOL(fault_in_iov_iter_readable); void iov_iter_init(struct iov_iter *i, unsigned int direction, const struct iovec *iov, unsigned long nr_segs, diff --git a/mm/filemap.c b/mm/filemap.c index d697b3446a4a..00e391e75880 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3760,7 +3760,7 @@ ssize_t generic_perform_write(struct file *file, * same page as we're writing to, without it being marked * up-to-date. */ - if (unlikely(iov_iter_fault_in_readable(i, bytes))) { + if (unlikely(fault_in_iov_iter_readable(i, bytes))) { status = -EFAULT; break; } From patchwork Thu Apr 14 22:28:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3571FC433EF for ; Thu, 14 Apr 2022 22:29:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347019AbiDNWb6 (ORCPT ); Thu, 14 Apr 2022 18:31:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231996AbiDNWb4 (ORCPT ); Thu, 14 Apr 2022 18:31:56 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93BD86559; Thu, 14 Apr 2022 15:29:28 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EIbRft018418; Thu, 14 Apr 2022 22:29:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=OETOJv78tEwAASDyJQXeLXtJ/S/DjnZ0ROS9H25ZNsuGKjZ7zMGmt6F3GAfmG0vPLutO zW8iC2qrBxLtfCPFpY5BFq8TpqdiS8NEgKXnQzh28x5k9rL43zJ9KZ6BXqoGg6/bgezX jNMORniKAzo2bgXBKARXvy3OqX0rTFAh7ic/wNtsIkrTJtF9pvN/m/7Cu2uafrr6QvcM zUb0i4/GP99Wh+QqYgPXOql18VIEBG2egfCe/Cx2z9NY+pp1hJ0JC+UtOb78WRn45at/ qK7myGvAfPc0r4Y1TnOXYWQdor/n0TyxMnRr5vF41NwhXnzpNbQlqBVcAoTmgT1pbvQw nA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0r1p624-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:26 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMFvg5007784; Thu, 14 Apr 2022 22:29:25 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2046.outbound.protection.outlook.com [104.47.56.46]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k5erqm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TAuTi/cr1voUlHIs1b72rYj5QqMvFJKAl8S5Lm6pqSbUGW0SxxHUYia2fY1kfbW8WyvHkhjzLr4TQiWy7/v2VPGCiOM16vFySrxGAF9GuEebm0/5dI4F0AUY2YhH1pumXjqYDdNci1j+NLgRnvVlcyuciqn/vj4gwjtQaAVGV2nR+NXtWpPRvsTz74FtOx2Z4WDwPDuOQa6G1NIMKhHUPeAAiKdq6CbCV6YHSAbi6Maav1uNwppzcNZtqiNtBj8oB42/kg4r5XesqNkaMuh9C5Um7XCJvNGHkA+C0fKND3xUxGnrG2peVHse8kOo5ag7uk7kxFQaGYrahq3dPXUUbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=jLLR0liy7+eLo18S01Tf1nWIwk+ey9DJnzgiv4X3D0FCAy2WchW6q8vloCSUbqKHui4XToPDBp75t7iJEKcrJDctSm3g+KJWeiUot0G/eufAgs9i4Ui44p3iOto+aEEe3d+nIfYPrQj/F12TBgMBJGirrl7VPXPsnHx7zcBotGjEvSPxuqOij7kroR/+WlN7jvFJj1BNmBdsTxaBmiOOoc0N6oX5IDkfB6Xhq2Qj3k+90QB62+z0hO5RtVFuL4LMVBZrPyNJxbPAlbhlmHX/nwIVS2Aaj4V3nQIGNPvAF+ieK7Dox5zcvo5ELv8xj2l8cYOqpW/lFrVSnDdMir17qg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xxGyidWi2psGJwQqlwpxCh5pl8mDh9nmIKiIz6H+xiU=; b=Ovk3vOdUJ2GZO8EUY6XJr6ztRfChbVgNUUR8X/6CTWi8C1aiTmdu1s3uqmcG9AwpVg6ER91t5Aos2MM7yYDMQQYousbh9MW7Ib46JLziXaFALypBU3AwlttUUxOKUlpBKVjF2FomVFKvSe1dFg5FUDCiIRMii6/6bFnmUol4VcQ= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:29:23 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:23 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 03/18 stable-5.15.y] iov_iter: Introduce fault_in_iov_iter_writeable Date: Fri, 15 Apr 2022 06:28:41 +0800 Message-Id: <8181618a0badc14fd9bbe13e26164bc601c59df9.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYWP286CA0004.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::6) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1dc0b8e7-bd22-40f0-1eee-08da1e663999 X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: f+/ypwbHSxq7Q4uhhjAIN1RilJhes4vAyOjZjJhW+h4K+VKnVs/5iqEfDhNHfKtVe845YRo9wqzaJ0+odegppCGBGSkh++rHVgvZyJXgfMja9zhaFUg2yjnrdjjL9GjuOq0B7CfbWClcPiIdTuwompJAReT+0y0x8HOaWekqex+X1v/vKSehvb0ct09gmfC3QQQ1QFXk/98nKcXIFCklpyUHrrb9F30gDK3O8S1yXgP7hK0sajMYB3vCpY/2ogiNpkhd27VtBJZK81fqOMUxtHGdSYyef0Tyqv8hW8G15A/t10vZ4fSfmJG9PTekMEtQXF6O0zA96BSTYI1Nz3kUQq4HWCMdrJ5MD44mrKznXZPuTKgdcBer5h4DD1W6U87/DWBMsZFoggi5PSnqv4dogn6NW8xqD6S9l1AsTLH/DcbLx5Zu/X9rwTVQ4VzCTkPfKxZHRtzgXay11VNTZU/ZlZlYPZjL6bibQZtyn9Nkk9PmmDulbchfodXDRfGYsdO994B6ep+b3Pc1hwsE9hVvGtCQMGwWaxW/4FsFQprZYaZPDz5gzfsMneRCTkHn8QHgnJB3EFMCXBx1OCo4MDFqiaZl+QqQD5w00muU8wZUBlXvsKpAZdR6N0JtSjz36EqXcduHgf4xQzxTSmoAcgRT3Q== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(107886003)(83380400001)(44832011)(54906003)(2616005)(186003)(6916009)(5660300002)(8936002)(316002)(4326008)(2906002)(36756003)(6666004)(6506007)(38100700002)(86362001)(6512007)(66946007)(508600001)(66476007)(6486002)(8676002)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: N3SxrRmTg+TdyUlU4SvpcjWf0qJcOORZZ1aVxyndBirG4MuKHd04wAXbc+XebcuGpn8y+G+/9CdO/Pa3xcnHjoNNFsfv2a12EjftkdIZEhBnf+6rNZOH8796YA5kvHOEkBfOlrsY1glt0x9zR+jbEvQPharAyZ86uRVmgPGvAYe680N/PICqVtK+Z+5JpoAXwhwuhCEEQpuIizlPp90LjcWRp8GHOh2rtqX0x0PHIEIhpLThCXpKiTBvc4Dkz/KUUxhzEm9LK62u0TUPS/W6B7RJsbIVsbwTq7X83qXpqL87Bk4BIZVghn2ZuEd5JhLKG0iuYX3SYXuTiGT46EFrgg1r15LXLC7jSI5ckf4h15nbfyz/ObwxUjfOrs+FyFi2UEWfHOjCNA8n71jXmXuX4Yvd+mqxDqWlq8g0dVfaAWlF2BQ6OJQCePPKhw00Fkcd5aWodgCT+gaWaB+B12JKkjEPusBDrj7sRZcYwXxO0ZHf4rl3b6SPxOJ6vv6cv8TB1L4lFtVR7VVeFxOEfQCicgMVr0wYFYVzZFHa/I7Z3JYAXKKPC1Eg7P2Fi4J4nbi9RAgck3M924vKDFgUdCRqY8epFByt67yIN9Ml2BkDq0Wsng7z6OgoHxTtjioeOoGBc1La633+nzpiJZBXpAGPgO2UMFk46KSDd0+lHeZoOfgHBxi1aeND2s5TtD15gWqkWv0zciU7XeOeJMkW+zAEvODpFF0f8aSBC2DByuwh6OoZjv+ixWJpnZMdUt8cw4SnrXLXexj8djiTOiowRFMzPVhAM1Zo1oru5XUtkr/pc6Vg30imr6tLfPa2A/z2bXIGv8bxWmiWmXAouvImpFAzMAYLuiXu19Do14Q0MkYMjPrhHdFCIZcyIfllwUlI5ywT6PRRz+BIPfFIlonOpcFymSIB4GuqBKsPQmnmd43Ua4ob0q5o1tQ6bWWU42W41PRAa0hl06VnxS/wpQrFOsxRgYtHBRAjkkf+y6bSxRdKWnBqKVQ0GZ3jvbbFVWAi70DvfnoIyM0IHNinTuSW85on8W3gm6FHijzoZTaYCmcTv9KtJ/QHiVY/vfYDVoyTgnHGkm6lrSqwqbJzS2nQrsqAwlR3iZtfj5RL3LTFMgMC4HxTT7snlTi5DgHdA5N4Uw1Pj3ab/hLT0FK5AfGURhiZP8TQtFDH7VtbloKpMGCfwmbdmwik7X8/nk2U+d3R/XwpV2bq4gY4CXiWva/OZR+uH4cMIcHrRajXUcXLoX6+hftiGFOlcj2BVS4GnoC+73D+P+dBgZIdprRYoRWgHLoxBJMtdzujitEsMPtNDXz72YHS1M4z00u05gsal9kp22IXE6U/+sE5gjvU5CiW+jqRBVhDYAuybvWtQRRKJLohlfXL8YEep0wST9H8opLqiB2Kx4cYaGsnclzo4Du0/oNfFbTuvEFWlCtDiPpUBO6zS1XifyQSB73IG6JpakSbvbAOBEeJTrdDjPvXOGu09Q0wnlnyjVpDhqBeI/W54q7pDjMhIySOtBw+wzs8cpdp/oRFSnHdf/dAU6rli4IH4OBSw5xpJVwD5KtN1xq2DRANB5R7Y2kRYXnPCvpNk71o/omPyFLaBsR9vORwJ+ZJYP9BzuXXMrfA17XBeOzaSIcWOHhwM52Bgj8MtaT935RxBPxKUhVocIL+RXLmqOTYAJo5pHJD7Vllr7vGMp1W4X/pqUAHl84a0AKJkPtgVd/t9E7ii/P01uMC/Jbt7PTWVblRGqN6a8bmxmvCchilXfV5bv8FYdxTV4RFY6vBeB+3bW5h X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1dc0b8e7-bd22-40f0-1eee-08da1e663999 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:22.9898 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: X13WthV9evbZA+Ia64ISASoU/bXlHD/MGE+Oij/P1YyCM36a1JTiCQqgpM5KtGkXk1EbogWG/IqQVbyMuiwX6g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-GUID: oZzrCJXUBF87KRE5gh86ZFSIllY87oOY X-Proofpoint-ORIG-GUID: oZzrCJXUBF87KRE5gh86ZFSIllY87oOY Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit cdd591fc86e38ad3899196066219fbbd845f3162 upstream Introduce a new fault_in_iov_iter_writeable helper for safely faulting in an iterator for writing. Uses get_user_pages() to fault in the pages without actually writing to them, which would be destructive. We'll use fault_in_iov_iter_writeable in gfs2 once we've determined that the iterator passed to .read_iter isn't in memory. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- include/linux/pagemap.h | 1 + include/linux/uio.h | 1 + lib/iov_iter.c | 39 +++++++++++++++++++++++++ mm/gup.c | 63 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 9fe94f7a4f7e..2f7dd14083d9 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -736,6 +736,7 @@ extern void add_page_wait_queue(struct page *page, wait_queue_entry_t *waiter); * Fault in userspace address range. */ size_t fault_in_writeable(char __user *uaddr, size_t size); +size_t fault_in_safe_writeable(const char __user *uaddr, size_t size); size_t fault_in_readable(const char __user *uaddr, size_t size); int add_to_page_cache_locked(struct page *page, struct address_space *mapping, diff --git a/include/linux/uio.h b/include/linux/uio.h index d18458af6681..25d1c24fd829 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -134,6 +134,7 @@ size_t copy_page_from_iter_atomic(struct page *page, unsigned offset, void iov_iter_advance(struct iov_iter *i, size_t bytes); void iov_iter_revert(struct iov_iter *i, size_t bytes); size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t bytes); +size_t fault_in_iov_iter_writeable(const struct iov_iter *i, size_t bytes); size_t iov_iter_single_seg_count(const struct iov_iter *i); size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, struct iov_iter *i); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index b8de180420c7..b137da9afd7a 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -468,6 +468,45 @@ size_t fault_in_iov_iter_readable(const struct iov_iter *i, size_t size) } EXPORT_SYMBOL(fault_in_iov_iter_readable); +/* + * fault_in_iov_iter_writeable - fault in iov iterator for writing + * @i: iterator + * @size: maximum length + * + * Faults in the iterator using get_user_pages(), i.e., without triggering + * hardware page faults. This is primarily useful when we already know that + * some or all of the pages in @i aren't in memory. + * + * Returns the number of bytes not faulted in, like copy_to_user() and + * copy_from_user(). + * + * Always returns 0 for non-user-space iterators. + */ +size_t fault_in_iov_iter_writeable(const struct iov_iter *i, size_t size) +{ + if (iter_is_iovec(i)) { + size_t count = min(size, iov_iter_count(i)); + const struct iovec *p; + size_t skip; + + size -= count; + for (p = i->iov, skip = i->iov_offset; count; p++, skip = 0) { + size_t len = min(count, p->iov_len - skip); + size_t ret; + + if (unlikely(!len)) + continue; + ret = fault_in_safe_writeable(p->iov_base + skip, len); + count -= len - ret; + if (ret) + break; + } + return count + size; + } + return 0; +} +EXPORT_SYMBOL(fault_in_iov_iter_writeable); + void iov_iter_init(struct iov_iter *i, unsigned int direction, const struct iovec *iov, unsigned long nr_segs, size_t count) diff --git a/mm/gup.c b/mm/gup.c index e063cb2bb187..bd53a5bb715d 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1716,6 +1716,69 @@ size_t fault_in_writeable(char __user *uaddr, size_t size) } EXPORT_SYMBOL(fault_in_writeable); +/* + * fault_in_safe_writeable - fault in an address range for writing + * @uaddr: start of address range + * @size: length of address range + * + * Faults in an address range using get_user_pages, i.e., without triggering + * hardware page faults. This is primarily useful when we already know that + * some or all of the pages in the address range aren't in memory. + * + * Other than fault_in_writeable(), this function is non-destructive. + * + * Note that we don't pin or otherwise hold the pages referenced that we fault + * in. There's no guarantee that they'll stay in memory for any duration of + * time. + * + * Returns the number of bytes not faulted in, like copy_to_user() and + * copy_from_user(). + */ +size_t fault_in_safe_writeable(const char __user *uaddr, size_t size) +{ + unsigned long start = (unsigned long)untagged_addr(uaddr); + unsigned long end, nstart, nend; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = NULL; + int locked = 0; + + nstart = start & PAGE_MASK; + end = PAGE_ALIGN(start + size); + if (end < nstart) + end = 0; + for (; nstart != end; nstart = nend) { + unsigned long nr_pages; + long ret; + + if (!locked) { + locked = 1; + mmap_read_lock(mm); + vma = find_vma(mm, nstart); + } else if (nstart >= vma->vm_end) + vma = vma->vm_next; + if (!vma || vma->vm_start >= end) + break; + nend = end ? min(end, vma->vm_end) : vma->vm_end; + if (vma->vm_flags & (VM_IO | VM_PFNMAP)) + continue; + if (nstart < vma->vm_start) + nstart = vma->vm_start; + nr_pages = (nend - nstart) / PAGE_SIZE; + ret = __get_user_pages_locked(mm, nstart, nr_pages, + NULL, NULL, &locked, + FOLL_TOUCH | FOLL_WRITE); + if (ret <= 0) + break; + nend = nstart + ret * PAGE_SIZE; + } + if (locked) + mmap_read_unlock(mm); + if (nstart == end) + return 0; + return size - min_t(size_t, nstart - start, size); +} +EXPORT_SYMBOL(fault_in_safe_writeable); + /** * fault_in_readable - fault in userspace address range for reading * @uaddr: start of user address range From patchwork Thu Apr 14 22:28:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6638FC433F5 for ; Thu, 14 Apr 2022 22:29:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231996AbiDNWcE (ORCPT ); Thu, 14 Apr 2022 18:32:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347048AbiDNWcD (ORCPT ); Thu, 14 Apr 2022 18:32:03 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 58A131C11B; Thu, 14 Apr 2022 15:29:36 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKkvFo006846; Thu, 14 Apr 2022 22:29:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=G9Z9LeAVfs4uOTE51NUhaVWjpTJWUFQ4WyZtXdzwv4s=; b=tTNzMrPrGcgGIs7njcJZ6n6ZraNr4znQLSPBaB0uTgfBafcXr+SB2HP+zie3DQJ7oh5K Xlr+mFfQKN62Ry9LcrvxFl/loDvsqpThLRsyvXyThFhq+tr5HA3P4T8fq0hZQN69y6rf 8z/Gk1nIsY78H5zOtdN67G1H4gC+0TBz7ABJXaaiNgNqr3XhLPxNqmsCf5wg/qpKY6l8 fT/7XasNuunqs3K5eu2faVQJ/Yu8x6ifbIl40iet0n8c/17VQRzIZcuOSH9PocSs4Eah pYxOj97IllJ2y3m7+it/EzEvIiIxseRcbr7mzisej2bZM+nwEP9yUDPZI5dAezkJN5cU lQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb1rse6vk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:33 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGOCp008407; Thu, 14 Apr 2022 22:29:32 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9m6jv0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:31 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I7MVV1sGuwe/Wstsc+f88KM2xWmCMSdcEuOBIT3g2A8UnlFZN+Mo8QHwmKaDJDoNGmBydQqtzuuQ806FFTwocPLMoMTezDgsPNZZ6HxDlLhT4Oa8RnpB6ut6oPYKprHY0D8ufXXqxOv/JsoCRtc7+DfIvBu/3McMR8qvp4p4s7sB5S+OH7ena1RHVGi9vc4ARd9KVFyKsD2QQIwzLWE03eLBKYU3KZT9Rk6F0VnVTdpJDmZJ6IMWCRsm+/iR8VOwT0/0y4s6rNE5FnsfrL+8JepPWv7xPYkSU3YUDr7CsaDFhcZUfBEJCwueJipkmfJegc+BBLmieekFJ0CkgZWC5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=G9Z9LeAVfs4uOTE51NUhaVWjpTJWUFQ4WyZtXdzwv4s=; b=AUJsNVHPkPkZ9SKW2svBXuvYAEdN08xOHAT7CpcmF3ZcQuUOBqWWqStsT0ZMOhLD+BuFza2YZyclzVUj0mQQdeSTOj4MkZHFCayr67/aBl89rz4Tg9ULRpjJbzsirvqEakFu1ErxoKdQVcXjiL0FOwBhnV/r3Ar+BXiEsS9DJe0L6hco5KUyKQXyzxqlwWWK88mzAGd8OhsxrEZ3jQyFflNa+TJGjXII6hfS6DoQ0AdRnEQCMz1+d1pxa9KItnV+B/7rSn0MX82K2S8IV/NgCqfCvR6lBWYMNmw+0cNL1b4OzO9/yasMsu2mojiZ0gJLe2h8XVfN45Hhe9lXtsFAbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=G9Z9LeAVfs4uOTE51NUhaVWjpTJWUFQ4WyZtXdzwv4s=; b=gNZDUffgq3WCpjEvvFh2+bW+i51s/iHwgmJ6ZFMQHTpsi9vy2+kXDqRq6sqDFpI3DZDhNpDwbAg3cau1tzXjAAYXVKnlvzMCYUbUPOHx42SKG72hoyRfGjKPQxzFvv/P+Lw0thWLmA3OLNbqhvs6xpPl/iNykXmEourEKzQbdmI= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:30 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:30 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 04/18 stable-5.15.y] gfs2: Add wrapper for iomap_file_buffered_write Date: Fri, 15 Apr 2022 06:28:42 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYCPR01CA0120.jpnprd01.prod.outlook.com (2603:1096:405:4::36) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e8ce84d0-51fe-4fdc-1fa9-08da1e663dd6 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Uy22X9/Ix0y6/AxAj4/meHFtnpCgnaTdRi9UX5vXqw7pylg8SospIZjGnVz0ezNrTIhdsJMSg01Ixo4rpfPDV/BUGxHotQTRSEBGqlnut492NR8gP3Yja1ED0Em2pKJU8WP75L4BEju6LLJOp/naCMf03PVYqJk1S9uhnlnNhlejZi09W5rqAWWtwmCF46WnxkcrX2QR2LPglDFQG1nGNPqBga9OmuxM44MoEvLnTUyJZnlPreGbFc31LjPUFkFs5HbFmOtnEljhZA6KFAwrvo7G+3oxI1MTNsSry4ZKaCQh4j8zM37MPXn/LSIdt7JNJ7fQieCV7aaR1DhuBKCt6PJB15AOgzKRQQZPVWsOUG89jqLdTdOislWNXd+4vjQHsO7R4wZiIQqcQa4aW7mDjzFMkhVkltp5CG7kujjLM6Fs6qgYqFkDbbK6hLMxZOyoIzqKckIdvmbGV10EL7WH0eKnls9lW3X00wpHbn7TYrIBgfo1FJ+375zCR36OvAQw+DmThIy0EkIgWjAkNBSmCm8aRYBSRotOHWDI5J5rPO3yVICuIx3S9MDaP0sidiL86D+9fFjS5xlecYBl9ehUdqstjjl5gSkd7471NNy/2h7Zvuoy2r+KuPkBx/nzSWR6kiIiXrWaFmhKNe/D1xVs2g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dWnizC+1EqkeeZG8IDL1PWuYUY7dab0jHxs3QaDSUb24neOdtZHt4cQdneYYKa9/bBXr1jnSRaxEVKQTPqhgaPw5/CYN7H44S58C8oNBIA3Kuexez+UEETXfWZyGq0yL9F3Je30h8s73WgrYOeoXC6QZnlqdMwmZcnSp5Es9YClX7FysXjdL+nU7LkSo0rVIZ1OBiRUIrJnk9drSNYw6RhqBx80czrRdzllJzeNIDINLeGHYZpxDjSSR+ZwejIl5eZvDtAIsBjiEnF4FqdIdqv5TcedRIF2ITChVEjLY1ffCoZyZbGpWzkTHkdvM8Sen4uRNM029gl0rFNJy5MD2EE+SMPEm0jjsyQ/zqT3PMrIP/uOiVPvWlu+BpOHzJ/HxdpD197s5zBYEbExuUB+sFV6mZ2avRCLDUt8TS0rH5sD5TflV2RWVFSq0xIhZSXq1Clvq4gYiJfxeu0S2qr3EhCrJucasgieX7BLXXOizPBdj5TRYvo69jKyostwmqfpRenKgVxymEKP3IAa6xfgk1lakITYlRjU5F9S8DB71YuSKZJVbVpBWunmeBL26V0rsOjPHuA1S2kA83IiE/R140j+uwH6iWUfX92scHaRow5rmEOu9y7f0J9wIhdwkMaI7gsH3bLO7Ho1ytgU3/q/MvvbgOGOmWkoQj8LK97f7wbBUo8CehFiMY6FMUry9BfhU2fgYMcS/i4By8UUPS1u+QsTsTfBkDSoXyrUQA9xul3K0ZYQdu1FouZluSFTTwQpsHOJn66s9VSBTlltE0Dh9cW+VdqZ1oqzDT9HSVcpW8A4NjS58P8p1MTzG+hp4PhrxqmTsHGW6oKioXtxYk/qZbuv/sqFx6oH88aZyVO6IHs+eYHFmWLgkVMSMwbt9uMNN59dMn07XoFyVy3OqQiwsBZC9qJXFwesqEqjE05mJfYTa+2SvCuqbYMJA/SoB1KHxERBhp+DTiO4SB9TUqaHPQOtFtd+xNNbmLjFQeH7XM1g/PBKqH0/BFZBnOXoeSzhHtX/F1MXvaCGQbIwaF2cpyMJ7ft5KXwcTFeDPyowJZrpfAKnvG7sARqUYKx3lwhFVRmN1nGazMIGT8wxzoXLUEVEd8cIVmFzzg/NrohCSN/BQJWTLO2wEd/OLRJvdGZUUW8EC8bpkKo+VGFrpeSSgmA8Yis6VaJuqn6q1Jn2rmdturvC88dRb7babvPPBv/a3lRDJYvBgIwlAIfDv+Jf42082rCVhxT6OR3vFzAcR9jJ4TXxoxK5PV/PZIJu47UkgSnFFkuglG+kiaQ7Hd01FaHLhsUW+DfM3DEd7phM6JDQ59QsK6i/+AyrbZ9UYem6loplpR0/uMMyvdCdqo75umjyyJQkAeYPKCItJFPDBwVo7MdgNUn7MWEHCPPXAwvfiHUUqS+U8UtVqIvH/trMS8/tjdLfaTcqi+jLxHsP/eNuKg6Tus8AA3NHRHfwuRcMU2di6zBShcoTAlOOdhY3Nu9imnXVR542tQcPfCP4Pq7XRK0H2Gnu/Dw+/AgUUN52IF9hhv21rpEHUi2ygff3deFbly9+3Iq8LAGrBBpXyjDHO/5F0+uWhIb2Noxkp5Ys4EOKE0jfSfqWWkLGUWCn+U+FGTVQNprayjj0/hnLPuCCpXYeQraVgDZqYTuYCuLerurFLHdPMCYaO1PXdmk1rFsrcab4gG3fN0GMIC13JOwCxzx4/jLcj4wJxXKroU+8oGaY/WgPSvqVdsYHwqcfehz12pzjf2YVEGH2JbU0ZyAQnWanwpy0vFWKiPvnyYYPL X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8ce84d0-51fe-4fdc-1fa9-08da1e663dd6 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:30.1004 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x+8JbYxKzB0SnONjxHkbxeZn+VKpMt3vYXF/IsIsJ2Le7tb5+DgfKyYuNxi7NnClRAdm/g10ka8OIRozYTKt9A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: zMqFA84djWIm1GvUmcabuzZtrF062GXE X-Proofpoint-GUID: zMqFA84djWIm1GvUmcabuzZtrF062GXE Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 2eb7509a05443048fb4df60b782de3f03c6c298b upstream Add a wrapper around iomap_file_buffered_write. We'll add code for when the operation needs to be retried here later. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/file.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 1c8b747072cb..df5504214dd4 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -877,6 +877,20 @@ static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return written ? written : ret; } +static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *from) +{ + struct file *file = iocb->ki_filp; + struct inode *inode = file_inode(file); + ssize_t ret; + + current->backing_dev_info = inode_to_bdi(inode); + ret = iomap_file_buffered_write(iocb, from, &gfs2_iomap_ops); + current->backing_dev_info = NULL; + if (ret > 0) + iocb->ki_pos += ret; + return ret; +} + /** * gfs2_file_write_iter - Perform a write to a file * @iocb: The io context @@ -928,9 +942,7 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) goto out_unlock; iocb->ki_flags |= IOCB_DSYNC; - current->backing_dev_info = inode_to_bdi(inode); - buffered = iomap_file_buffered_write(iocb, from, &gfs2_iomap_ops); - current->backing_dev_info = NULL; + buffered = gfs2_file_buffered_write(iocb, from); if (unlikely(buffered <= 0)) { if (!ret) ret = buffered; @@ -944,7 +956,6 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) * the direct I/O range as we don't know if the buffered pages * made it to disk. */ - iocb->ki_pos += buffered; ret2 = generic_write_sync(iocb, buffered); invalidate_mapping_pages(mapping, (iocb->ki_pos - buffered) >> PAGE_SHIFT, @@ -952,13 +963,9 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (!ret || ret2 > 0) ret += ret2; } else { - current->backing_dev_info = inode_to_bdi(inode); - ret = iomap_file_buffered_write(iocb, from, &gfs2_iomap_ops); - current->backing_dev_info = NULL; - if (likely(ret > 0)) { - iocb->ki_pos += ret; + ret = gfs2_file_buffered_write(iocb, from); + if (likely(ret > 0)) ret = generic_write_sync(iocb, ret); - } } out_unlock: From patchwork Thu Apr 14 22:28:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BD12C433EF for ; Thu, 14 Apr 2022 22:29:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232190AbiDNWcI (ORCPT ); Thu, 14 Apr 2022 18:32:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347047AbiDNWcH (ORCPT ); Thu, 14 Apr 2022 18:32:07 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 305391EC6F; Thu, 14 Apr 2022 15:29:41 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJ07dl018439; Thu, 14 Apr 2022 22:29:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=nYGjQgaKEbVGI7PYBS0b8Si+Y6hA9wC7fIbt1C7iAWheUq08kNv/5fbyXf0vmxlCjvw5 3lMNtwr73Xx8qfld8ZDvvE0D0LRQygimicgCSxjKRgW/r1n01n7I5Znet2ycOHad2sqd Fx19vTU6VN4u8yEJCsq18jfb0PfcJyMYSbDKv42jPnsR8bf3IsMFPcIryWblgxfPMiSW YC4PaiD9wjMh4VZqzfW81DpqIyImNN900/6fVxAITaUENKyNipiewvMxnpaLa5mFt65L 8vkts1uAdXRrS7fxAfSrU5F2FkWoUgCQd0WqgqAWMnoXBimN0ObXaD4p5oZ9ZRcR4Q/+ EA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0r1p62f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:39 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMHhD1014987; Thu, 14 Apr 2022 22:29:38 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2100.outbound.protection.outlook.com [104.47.55.100]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5uc7w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cNaZikEaXtz9t0L7MWwbWFth8ccFCGlLDMz7rwiiPCmuPrFrFsL5WxHWi/v7JCfc0t7WT6Nd6d/8BoNFvul2KvcxP+iSXm/5ln/Ol4AOKQN/TMu+r0Ts5UmHQJnx69Q8GljNSjUt9mtcfExpG3F16ormlNuXFPWpnhcswZRYeDVLQGMMJYUftc4nCYfYCBwqlg5GYaD/y0H0sB+bk7g6AMb/F+iF/0hBYUstSEBVcjv/L7ld4XLEuxgUkejzDaIkqWI1K9LQdNdBwhfUvVQJP6TxLyLnK5wRC+Aq98ueD2e81syD++JrzclHnMcDGAA8k3Vovwq0lJwZR1kxcEYuAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=BX7VFed/DqNn6sfyJb7kPYi+83GyNVaztvIsRVIHZHEfBxwPXY/UqigcR0+fEiZFecEcD73RHIKupvMmDjDSfs1vbfJv9rv583LNOhZlFlA7Yb42DZ4vE+onUJv9BlioL02G3swuCkAS+2NsLLTft7ccoldLV5H5tJI3TCdswAZd0Xr38Ept/zWprJL+BIBuO4KoNVN85nnfbEThH/r2PqseIoVyjvwa6npT74rA21IbeuM655M1jPlVm0ysqOIVxDbhghhJzcqghVORn1n6bp9hIyURHrKxxDym+mnbYXTpKzodYQCagLiyu8+zDALO1Npg2SA/96eJIYU25+YN6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IEndfXy9jiOzWuwTkMw4OXx/3CbSemHQeUkLPmGuOhc=; b=MoihSEWJ2RBGJdzcBMOfYbluCMJAIeck0zs5H5TvlVnlk4e/qkjcBlplPqxwCC8SFSjdRHdJlhGCLByZpNGu62Qy+SI5n6xF10e+MCI8G0/y5Edm04iUj6vyy72F1dZk9Wve0gz9K1Cy4+vEvfnStdftKkmfCRuzFgqjijWXO7M= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:36 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:36 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 05/18 stable-5.15.y] gfs2: Clean up function may_grant Date: Fri, 15 Apr 2022 06:28:43 +0800 Message-Id: <16061e1d0b15ee024905913510b9569e0c5011b4.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYAPR01CA0095.jpnprd01.prod.outlook.com (2603:1096:404:2c::35) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 68555fc4-40eb-45a7-8a7a-08da1e664199 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 19PnDl7cKhOJ9HieWz2wCjVemewqfdB5cFEF6f84Zfzm8jYHYOAmUIi1nhjNFhX7cfh05kZWmlPc9i3TaDLUuCx2LyNF+uyPZCTVwtvLy6N+rxThaDbeX8yELnGGxUVH87+FAYpW+VcM3OlVERjGig1+FUaJMkirrhYRbUv5xYD+3w+kCP+M1V6kzyeWouqwRwCGwPJuzEG0xxmMlHCMpa9hcfn1ANA8F648bns1yF7ktxjcb4bEdmuRU1thx9cYFQlhOyqgSXMcKIWCUsLDzicovOVjAUkBBzGVS35+ivuzgJHyunnxWUtwbN1Z/IeimRDSw6rlhZaIZFzYUlDMDGv8TfYFudmdKBy4dZ4iO0+5VEg2CeOj0WcvqrXuspK81MZl6HVInz4zdjKicVZHQXSZb+VPBOh/UFL3ePsv5oXe7zCnmPC905k2HhwxiOMTtHYbrlvadbkRlyLsONCOm+KpkroORsNIxjKnS54N4jPJ62unErEbNlRdxXAbdu3vvCCh/gnxd2J2ykM/RFbo+ilRRgXVbiE/Klyte5ZfSHMp1c/gGkADYNp12YKP3gfm+danjgNWjmgbVmzw9QnSK0tBrirdNDLVmtCtb2oGZoVotUz4a3xvACdXvs2FppVradxmAqXAGZxocpZNFq8MBQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TG83qMpooXyixME9o7dFQTMby5Bz8rPAX0Spwmo4uodFa2DkKV/1Y4KhZ5KdcBCPqwmOXTqbFSsTY8xFTsFbzLW4MhJsrPswSO4kJLy6orF55jFR/bhczZxJLPQDvRdgtObnkNEqM6TelWitmvdQ58sLt8lfbX9DHbzduAnGdJvShST7PRuE7kJQTfLxnQ4duViYsBHAjIQVjjIX0GTQVxhKLC4qoKvfXPI4NL3qOFCa8cJYhY9Q3TcatlAl9LEqhvHFf3mNsxwXZ0DGYBMeBXOBGeaAMS26r9NzNRpFvjHxrtC/I9VfIRNUak9RTZa72NzMEk6OaBhc86tABVcmJy0rNMAuy1MNLVpBPqa4QFdYPqdDpiW3GARz5yA8B0uDzI4/VWNdILVhTPdJidv8ce1BGy9bVGTnvYU/US4rhMpLG8sMgHi9FVrmnSh2eWEeAL2PiH5Noo9B6wyAgu3DPlMljV8Jjnk/16QvwWxq/himPBw2C3nO7Xr8ShBS0HUxhwdm2fbuADHXAhfA1eNoecLcYN6mlz5V6DiWEceXRvm/656dQJ0q/QQZOuT8u+GksEo4+82Hj58INW/rjv1DeehZtkFUWc3Z14EBka4JhIvcJQ4houu/JvjBZUR1SA6t/XstDN1npbai5raLRKlqd2/Is/peXMuNr0fxEgAQjQlk/9espvlW4JqgsOsXLuW1KrH2Gku5xljt0HvGK+PlQfrraeOo8l6d87zrH52/1cg8pTEwEzT+qF+bJmyw0I20RGfhV4rgn9MlvJtaQ5+K8PTcomhNVXByij8usd0be5djd3DWW9v9sb7AbdGupEbxwJOVnFpe/braB/e+bk6BJ93i6iA2FftzwCZevlLUuenTAnoQPHPSUW4V1uH8JhoiQdlfPg1lNsCCH3YcK+lyg8GaZc2xB/A1P/BAn9fYPNdOnxlwiL5NeP5rf94IXRcd5O6GoIANHWICGMi+kNEcQolnUZ2ZnxQNK5IVkSsLgs4em96PZ607vd5DmJIkRuGzo4ZN28INKz19+CMWO6FxPLAYx8cSnULpNYYhWuC3kpFKX+pfL1JKz4qzevOWZFe196yKg3JzrzeAQdXAtWnp2N+/+kYwYxpIG1XtHU3OYRduHJFlheKP4WYDrtqsTKGhO4o6ezqHRJM5q/vBMCG+s2jhpM0UowMlwrN8tfYFt0Xm+xuwSC7dK3aaEXM7eNtMSNVwZMjLmDK6+z6oNSjxsaacEVINfNSYKdZH8kk1QZZFYdQy+igl1WQCVgAhfaXW4knLQBLVv0e+3+AlFOAyYvPset4u+49bq3q8hCbtWPEOk0Q9Q3PlsOjzQDxIwyTjdSLlCL2+f/vXu1SQRSvSPVuho6fbT3Bw76pCwZ/9R3hU8OZiGhqcbCs5VDQGIAE+n7a1PvC1SywtTTNySwbCA4F3HajXfKw+7XEl3NGjoJW/LJnzyzfuRcPGJeaP40Fdc6kfhZezBcbJGSD14cOmmMECiBFRXkCn9b8eJ5+ry9A7DiJsOmn7nm52fCptidZ5cRmHcrGZYfIAaK+FyFjjCriIg1tG239fAKSPmEzOcpOIDwCTiTtzQ0DwdugFSEUQqMOAroZHNP6ZmhfM9D8KZIFNtQV+quxTnYZyZua8T/d9+nrGVVLoNDEOKmknVK9n6gfTKLiN4WzPzzLTzEVlET2b9T5YJQT/BzkM30NpXa79xiNT6p92gXVfmAZ5Zx4UyFvrDfZZbyuySNq+vFnzF66/aHbgFpNgAGoCfu62olWRChqTRisI6Ld/DeS2Qn1L X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68555fc4-40eb-45a7-8a7a-08da1e664199 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:36.4300 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9nvW1PXqRBeF5lbxOLma+a0F8M3wOqJ6cveJltK9VxGVhOfFWQqH6TSokCEvrYpenHxLxejWGaY40OKo/J8D9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-GUID: kJIJatnOFWpiZiyPm2kIGt_XqVulqQQ4 X-Proofpoint-ORIG-GUID: kJIJatnOFWpiZiyPm2kIGt_XqVulqQQ4 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 6144464937fe1e6135b13a30502a339d549bf093 upstream Pass the first current glock holder into function may_grant and deobfuscate the logic there. While at it, switch from BUG_ON to GLOCK_BUG_ON in may_grant. To make that build cleanly, de-constify the may_grant arguments. We're now using function find_first_holder in do_promote, so move the function's definition above do_promote. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/glock.c | 119 ++++++++++++++++++++++++++++-------------------- 1 file changed, 69 insertions(+), 50 deletions(-) diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 02cd0ae98208..8f30ad956270 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -301,46 +301,59 @@ void gfs2_glock_put(struct gfs2_glock *gl) } /** - * may_grant - check if its ok to grant a new lock + * may_grant - check if it's ok to grant a new lock * @gl: The glock + * @current_gh: One of the current holders of @gl * @gh: The lock request which we wish to grant * - * Returns: true if its ok to grant the lock + * With our current compatibility rules, if a glock has one or more active + * holders (HIF_HOLDER flag set), any of those holders can be passed in as + * @current_gh; they are all the same as far as compatibility with the new @gh + * goes. + * + * Returns true if it's ok to grant the lock. */ -static inline int may_grant(const struct gfs2_glock *gl, const struct gfs2_holder *gh) -{ - const struct gfs2_holder *gh_head = list_first_entry(&gl->gl_holders, const struct gfs2_holder, gh_list); +static inline bool may_grant(struct gfs2_glock *gl, + struct gfs2_holder *current_gh, + struct gfs2_holder *gh) +{ + if (current_gh) { + GLOCK_BUG_ON(gl, !test_bit(HIF_HOLDER, ¤t_gh->gh_iflags)); + + switch(current_gh->gh_state) { + case LM_ST_EXCLUSIVE: + /* + * Here we make a special exception to grant holders + * who agree to share the EX lock with other holders + * who also have the bit set. If the original holder + * has the LM_FLAG_NODE_SCOPE bit set, we grant more + * holders with the bit set. + */ + return gh->gh_state == LM_ST_EXCLUSIVE && + (current_gh->gh_flags & LM_FLAG_NODE_SCOPE) && + (gh->gh_flags & LM_FLAG_NODE_SCOPE); - if (gh != gh_head) { - /** - * Here we make a special exception to grant holders who agree - * to share the EX lock with other holders who also have the - * bit set. If the original holder has the LM_FLAG_NODE_SCOPE bit - * is set, we grant more holders with the bit set. - */ - if (gh_head->gh_state == LM_ST_EXCLUSIVE && - (gh_head->gh_flags & LM_FLAG_NODE_SCOPE) && - gh->gh_state == LM_ST_EXCLUSIVE && - (gh->gh_flags & LM_FLAG_NODE_SCOPE)) - return 1; - if ((gh->gh_state == LM_ST_EXCLUSIVE || - gh_head->gh_state == LM_ST_EXCLUSIVE)) - return 0; + case LM_ST_SHARED: + case LM_ST_DEFERRED: + return gh->gh_state == current_gh->gh_state; + + default: + return false; + } } + if (gl->gl_state == gh->gh_state) - return 1; + return true; if (gh->gh_flags & GL_EXACT) - return 0; + return false; if (gl->gl_state == LM_ST_EXCLUSIVE) { - if (gh->gh_state == LM_ST_SHARED && gh_head->gh_state == LM_ST_SHARED) - return 1; - if (gh->gh_state == LM_ST_DEFERRED && gh_head->gh_state == LM_ST_DEFERRED) - return 1; + return gh->gh_state == LM_ST_SHARED || + gh->gh_state == LM_ST_DEFERRED; } - if (gl->gl_state != LM_ST_UNLOCKED && (gh->gh_flags & LM_FLAG_ANY)) - return 1; - return 0; + if (gh->gh_flags & LM_FLAG_ANY) + return gl->gl_state != LM_ST_UNLOCKED; + return false; } static void gfs2_holder_wake(struct gfs2_holder *gh) @@ -380,6 +393,24 @@ static void do_error(struct gfs2_glock *gl, const int ret) } } +/** + * find_first_holder - find the first "holder" gh + * @gl: the glock + */ + +static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) +{ + struct gfs2_holder *gh; + + if (!list_empty(&gl->gl_holders)) { + gh = list_first_entry(&gl->gl_holders, struct gfs2_holder, + gh_list); + if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + return gh; + } + return NULL; +} + /** * do_promote - promote as many requests as possible on the current queue * @gl: The glock @@ -393,14 +424,15 @@ __releases(&gl->gl_lockref.lock) __acquires(&gl->gl_lockref.lock) { const struct gfs2_glock_operations *glops = gl->gl_ops; - struct gfs2_holder *gh, *tmp; + struct gfs2_holder *gh, *tmp, *first_gh; int ret; restart: + first_gh = find_first_holder(gl); list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { if (test_bit(HIF_HOLDER, &gh->gh_iflags)) continue; - if (may_grant(gl, gh)) { + if (may_grant(gl, first_gh, gh)) { if (gh->gh_list.prev == &gl->gl_holders && glops->go_lock) { spin_unlock(&gl->gl_lockref.lock); @@ -722,23 +754,6 @@ __acquires(&gl->gl_lockref.lock) spin_lock(&gl->gl_lockref.lock); } -/** - * find_first_holder - find the first "holder" gh - * @gl: the glock - */ - -static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) -{ - struct gfs2_holder *gh; - - if (!list_empty(&gl->gl_holders)) { - gh = list_first_entry(&gl->gl_holders, struct gfs2_holder, gh_list); - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) - return gh; - } - return NULL; -} - /** * run_queue - do all outstanding tasks related to a glock * @gl: The glock in question @@ -1354,8 +1369,12 @@ __acquires(&gl->gl_lockref.lock) GLOCK_BUG_ON(gl, true); if (gh->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB)) { - if (test_bit(GLF_LOCK, &gl->gl_flags)) - try_futile = !may_grant(gl, gh); + if (test_bit(GLF_LOCK, &gl->gl_flags)) { + struct gfs2_holder *first_gh; + + first_gh = find_first_holder(gl); + try_futile = !may_grant(gl, first_gh, gh); + } if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags)) goto fail; } From patchwork Thu Apr 14 22:28:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E953C433EF for ; Thu, 14 Apr 2022 22:29:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347047AbiDNWcP (ORCPT ); Thu, 14 Apr 2022 18:32:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347051AbiDNWcO (ORCPT ); Thu, 14 Apr 2022 18:32:14 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC1843CFCF; Thu, 14 Apr 2022 15:29:47 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJKxnN032238; Thu, 14 Apr 2022 22:29:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=Kt0CnLsZZY0ne5qnhahnih47bah4v2LKeMfkhFr3RkI=; b=FPr/O3ocXqftz4PX8OmDBE3ZzWm6BoYQ2PJ5wBf1WehC90k0gOSPQoldaknNssGJMXwc sQB4rAaqN1sYQqVcX4mCRFFhlyIY07JBgFoP+pRTITyAyN+bIuYlumtkuWhZ1AYUmFu8 Jn8c8ZnrES+tRoD3vTT61pIc+rX7ZmFvH6Tklytvehr0uK0J+kRp0ASKAxw1mD+eKKpz InnzwJaf+WCYS5QLd5eEPyCbJzGiAGfAVLeI/KFkEN3P86q6fLAFi64xpH2dbw8S7p3b QArQg/5imxbGlTc5TIybI/HNrI2/t4fq/d/mBM9p5KQuxFVHFAGmg/xtThx+PrbVz022 rg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jddp22-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:45 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMFolS007650; Thu, 14 Apr 2022 22:29:45 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2106.outbound.protection.outlook.com [104.47.55.106]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k5erts-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X4jGGtUgEvG2HkihiCp+f0rjrzU+kQ3F8MELClZiTTEkQLnikp0URjv2votQGvGXJ0PBMQUqkkJq+WBpeS9CP7C2EdcQ6oGqPkA1Nhjh/2DdHKZuR4a2Gult++4wvNASPWpSBqhEGlm2BdwW0evghvAnU06PnmDL6JTDBHznLyTQVDimy1ogL20KvqvuX+bQlTy5j2gzfYMrt4iirpq/XeXpJkgn5rjNy6fa7yaz3pBeXSKQrAPXTNz79jh7lm+riPX+ihNsweYRccm5WOg4SLEI8Vp2AXaQuCMDc0tsRD1bfkwDXyVcVIU9rJ2q5JISluHhPB/SDtxE0wIJ+2VhCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Kt0CnLsZZY0ne5qnhahnih47bah4v2LKeMfkhFr3RkI=; b=VVUuUbHBbCTzlYJKbpQ5v4U55g9SCpcDm8H8lqynJFJZDKqGdWevnljlH6HEl3DlXFygJ3Y8npxkpOWbM9tpjUSbPfVpUmKee4L8Myo/cicjJoUbkVITBYQ17YfyKVuLL580TD+NvvoON71l0LYjtoKyxqPcd0wb1n9ILICpiwvVELNMUAS3FqjPSbrosIkQLr2K9ID2rV86W0m5qm+tJh3rbBOoi0AsB/jOEQs9skjDJkX4MZFxn/Rd4slpNaYFbkp9mIcizhx0zvK36vNmdZ8veMe58lp7UmERTd4Oa5cfAP6TuELhCAaHfTPort5pcsybUKXH/gtUAo8Maqmd9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kt0CnLsZZY0ne5qnhahnih47bah4v2LKeMfkhFr3RkI=; b=PW+F9dXOIgR2VketykJotaIlyAopTkR1fHEsMxFxOBCqraPUtP3ZKSw0Lu4n9VyC31FYp8hWkee+7A6uzJ03OXZFedokUsIPm3aNrZexORfOJVkOlLE32DYH2YE50enpIl1eFsthoqcUdgOfzZFinUVP/kt8ow8vDRtLeKIx2bY= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:42 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:42 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Bob Peterson , Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 06/18 stable-5.15.y] gfs2: Introduce flag for glock holder auto-demotion Date: Fri, 15 Apr 2022 06:28:44 +0800 Message-Id: <51a4309baa83be7f31064db7fad3b9d3649d239d.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYWP286CA0009.JPNP286.PROD.OUTLOOK.COM (2603:1096:400:178::7) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 23a7d680-4b23-4f51-5bd7-08da1e664507 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qBjSZ9QnGWXmAz/IyZwC0ELmchU8Ag9f006ND87JLf8ecsLqWlA0JsUMd2yP6naLGYmlwjQlo844d//1EcJ6Xh5AAf2Ze/igg1iaFUiVEmVTsVXHf13pR4f9GUzvykzrbjNva5+L6fPhnIrf6tv8G0fHFtiy0IyQqI10qbuWSMb3BRseFMGtViEKaMDwwuSTug+luveHXYumpt6rLfCbttG61PzYXUf8Lwr29yVJEUegWdmy26mlTkKeMv/v6w6q0mwa7l1lMdHimuKyj9qlgi1GlUL/SnshEQPXO8W40HO5w0ypRpqa7I68F4hFFGilpvfaeHUJQcE8nObLmFWe+gEJ8z3UYeVAKmMp2rIXO1VRA22e/00pnh5KJmcYJUeLc8+uy7IzJHk0G7ZFaV3SmGZx5FaMnbCbnZ8AE+/Rk0H8SuYXinx2+mbkiQ0WHpRW/PIsDRXqOAyyMMRUdcXl5ZasS3O8yJBAGRK/JeikBrY+Vzm7c+C/UkG+ZoReWs/eL/Ytr1G9+NOm8FBWa3C8Lea+aClbUfwY/hV6nOPSRnVBfVc+CMmwH0F6P6jrSO0kqHHMGd9glO4TMWINk2SuxacAx/xXiDVOoFb4rrD9sSYHdPOq9ViNT9wIATgt9N9L4BZoOKV/FzzuJAs1F394vQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(30864003)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IyzL7ayczvcQBehPXH7iBlhDTOHYJN4OpaRXy30ceyyntX2eo7b6uNTZYP3vxbX1HPhp2XPVSUDX5dJUjp1DrO19ubi1kgW8Nm1v4JrKUCOM6iOA00feH3UvuQ2W8Qzs7O5jYeDHJHX5b5kmMyYI+hyVkQ5VbA1OSYYBzaU+QWnvDMiSN1tZbEihif8wEyZzgxpa6NmAExzzflyeR5JBKbP8HK8ZwnNEx5h6aovzFinKReOylCYi01gMHe3uI0KUtOWAUOUIJY6DAMYMYzYtV0JojhtOny1+Htxlye8Pqc6qGNruMdKrd4aZ1xWTP2GzU8/q1r5XVz7QTwKxTHGz+cQcMPQ+GB5LMptCLC48xKgqU7MhRe3NryF3ItxrEQ4TO0lvMHQ8AhK7gpm2KZMQ6DC/7g35ftOIDI0H2rGrKSpDlGEbTVeoEaulqSLhpR5GZrmXwzx3Bhjm5wrX5KFMwIGuAsdR6w8B6d2KW+CBVOTK5chKFEcCJG+uIA9zeEqN2c5exkCX9jCSdgbzC6mIAWyyx1tsa11K48JEzmu8wLr9i4HG3ZO7qcMEoogcUUogcqgFWmsu6tWonk1OCA2eNZbvBzsWy9i/hFnZDyxJcAhgxPxusri9y8bYFQRiWQWezWo3ORdc5E7TSp2fhHe8LJaUm9oVDJRYPbswpzgWJ4LbRU6B8WKnBKPE4QdSpwbh4PbJPER0m8/R7VgYRvV4LSbFD+H+NaBLM9bkW4VtaNjLktjBgDD8NoFkVgzRIKNFDkJ7drMcqkil7FZR8fBNjPzUTqpCNvNEZ00KG2y3Hgca1kDaV7fyTG8LRD0vD3KSZRyo2hqzn7fFkIrvX2LDTBu99StwAOItUEe8tznvYX5zAWrJdTi2RrMHId0DWqNUqB6scT2rLzIHgR+KcHzWqOSWCGqQRsjcqW3pTPD11/i7pXXJZTXdPqVUA1kgBMQX2kRMcitDxxlV9JKR6JKRrP53VkrB/6QYUwt/OltlptMruuU1iyxw9ipXe1v1NDlN7TsZryf1ULMLz5kkmb/m26eDtqtfd03vvZP0RNvTodQEmpkNq1kqX1uyFEkPn7fukxkcRSp9DobqLYkxC/018Qn9yU0CtkZHduq+lTLSp8LKNgJj6BQmd5VTjXK8ElCGWdqMCqNjwcbetjaVocnp2qpJ20ZURZgjewMMpnO4hlp1XdHMwxiSP41UmuCWBjpaxGxW268fDlcfP72B64T8683F1ZapGS0bcsB9U4ssacPE8fmscyBfz+oR5poJxERxeEksIWgCfN8HobgmB9IOb8sObC7pXZvRvstFHAds1Nzg6Wb/l6s3Jbh1JLObR94RIQ4RT9OZFVcQ8E2KrOG+9cN4EhGwyw5wJR73YhkDHg/+CZfjPaVpWv8SFNVbxD76WmVvhdbubX28zSutMzu5qibcdheX6uZdnplGN2Yh4qtIUgOATR4qzRAVxuGl3kvU9RNYmnBPOtRaWenm0d3XPAaPStTUyzNiT+Dd5IszNwEMe3wzRWneAQw1Y3lqDbWHGlC6Z8zoEPqDGD/l3nfY/IcwqEiPViJmgBAYyXPrI6FP5UiO2OkZzD6RQHmdU13i30GwZjMYQIn27hqBhi1umtBZv0lbAIQvEh5gLI0TLgSVFMizSE9hBfP56+4Wzi6LGqcpI2DQJcARvGYJsEtE3w4HvH0jRW6hq0652qfo7ewM27LvZC3K9xI/4mMuq6vgZw60M94vDrM3MyPCRKtqLZpMjJ8BiHR4TN0g8OIKBJ4hFn5mfNXGYRSt827z05Ph X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 23a7d680-4b23-4f51-5bd7-08da1e664507 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:42.1360 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6TGxSbrg8fD1OZu7UR7X+zsxaxxgRjuXkHTnU1I/sVLm2yJ5ZSsyTjOIifbCb2Br8DOd5gKenYaRz5EmNmUX1g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: 904HYRj0v6FKP4etJwtA2xFXHk_DHJh_ X-Proofpoint-GUID: 904HYRj0v6FKP4etJwtA2xFXHk_DHJh_ Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Bob Peterson commit dc732906c2450939c319fec6e258aa89ecb5a632 upstream This patch introduces a new HIF_MAY_DEMOTE flag and infrastructure that will allow glocks to be demoted automatically on locking conflicts. When a locking request comes in that isn't compatible with the locking state of an active holder and that holder has the HIF_MAY_DEMOTE flag set, the holder will be demoted before the incoming locking request is granted. Note that this mechanism demotes active holders (with the HIF_HOLDER flag set), while before we were only demoting glocks without any active holders. This allows processes to keep hold of locks that may form a cyclic locking dependency; the core glock logic will then break those dependencies in case a conflicting locking request occurs. We'll use this to avoid giving up the inode glock proactively before faulting in pages. Processes that allow a glock holder to be taken away indicate this by calling gfs2_holder_allow_demote(), which sets the HIF_MAY_DEMOTE flag. Later, they call gfs2_holder_disallow_demote() to clear the flag again, and then they check if their holder is still queued: if it is, they are still holding the glock; if it isn't, they can re-acquire the glock (or abort). Signed-off-by: Bob Peterson Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/glock.c | 215 +++++++++++++++++++++++++++++++++++++++-------- fs/gfs2/glock.h | 20 +++++ fs/gfs2/incore.h | 1 + 3 files changed, 200 insertions(+), 36 deletions(-) diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 8f30ad956270..e85ef6b14777 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -58,6 +58,7 @@ struct gfs2_glock_iter { typedef void (*glock_examiner) (struct gfs2_glock * gl); static void do_xmote(struct gfs2_glock *gl, struct gfs2_holder *gh, unsigned int target); +static void __gfs2_glock_dq(struct gfs2_holder *gh); static struct dentry *gfs2_root; static struct workqueue_struct *glock_workqueue; @@ -197,6 +198,12 @@ static int demote_ok(const struct gfs2_glock *gl) if (gl->gl_state == LM_ST_UNLOCKED) return 0; + /* + * Note that demote_ok is used for the lru process of disposing of + * glocks. For this purpose, we don't care if the glock's holders + * have the HIF_MAY_DEMOTE flag set or not. If someone is using + * them, don't demote. + */ if (!list_empty(&gl->gl_holders)) return 0; if (glops->go_demote_ok) @@ -379,7 +386,7 @@ static void do_error(struct gfs2_glock *gl, const int ret) struct gfs2_holder *gh, *tmp; list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + if (!test_bit(HIF_WAIT, &gh->gh_iflags)) continue; if (ret & LM_OUT_ERROR) gh->gh_error = -EIO; @@ -393,6 +400,40 @@ static void do_error(struct gfs2_glock *gl, const int ret) } } +/** + * demote_incompat_holders - demote incompatible demoteable holders + * @gl: the glock we want to promote + * @new_gh: the new holder to be promoted + */ +static void demote_incompat_holders(struct gfs2_glock *gl, + struct gfs2_holder *new_gh) +{ + struct gfs2_holder *gh; + + /* + * Demote incompatible holders before we make ourselves eligible. + * (This holder may or may not allow auto-demoting, but we don't want + * to demote the new holder before it's even granted.) + */ + list_for_each_entry(gh, &gl->gl_holders, gh_list) { + /* + * Since holders are at the front of the list, we stop when we + * find the first non-holder. + */ + if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) + return; + if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags) && + !may_grant(gl, new_gh, gh)) { + /* + * We should not recurse into do_promote because + * __gfs2_glock_dq only calls handle_callback, + * gfs2_glock_add_to_lru and __gfs2_glock_queue_work. + */ + __gfs2_glock_dq(gh); + } + } +} + /** * find_first_holder - find the first "holder" gh * @gl: the glock @@ -411,6 +452,26 @@ static inline struct gfs2_holder *find_first_holder(const struct gfs2_glock *gl) return NULL; } +/** + * find_first_strong_holder - find the first non-demoteable holder + * @gl: the glock + * + * Find the first holder that doesn't have the HIF_MAY_DEMOTE flag set. + */ +static inline struct gfs2_holder * +find_first_strong_holder(struct gfs2_glock *gl) +{ + struct gfs2_holder *gh; + + list_for_each_entry(gh, &gl->gl_holders, gh_list) { + if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) + return NULL; + if (!test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags)) + return gh; + } + return NULL; +} + /** * do_promote - promote as many requests as possible on the current queue * @gl: The glock @@ -425,14 +486,20 @@ __acquires(&gl->gl_lockref.lock) { const struct gfs2_glock_operations *glops = gl->gl_ops; struct gfs2_holder *gh, *tmp, *first_gh; + bool incompat_holders_demoted = false; int ret; restart: - first_gh = find_first_holder(gl); + first_gh = find_first_strong_holder(gl); list_for_each_entry_safe(gh, tmp, &gl->gl_holders, gh_list) { - if (test_bit(HIF_HOLDER, &gh->gh_iflags)) + if (!test_bit(HIF_WAIT, &gh->gh_iflags)) continue; if (may_grant(gl, first_gh, gh)) { + if (!incompat_holders_demoted) { + demote_incompat_holders(gl, first_gh); + incompat_holders_demoted = true; + first_gh = gh; + } if (gh->gh_list.prev == &gl->gl_holders && glops->go_lock) { spin_unlock(&gl->gl_lockref.lock); @@ -458,6 +525,11 @@ __acquires(&gl->gl_lockref.lock) gfs2_holder_wake(gh); continue; } + /* + * If we get here, it means we may not grant this holder for + * some reason. If this holder is the head of the list, it + * means we have a blocked holder at the head, so return 1. + */ if (gh->gh_list.prev == &gl->gl_holders) return 1; do_error(gl, 0); @@ -1372,7 +1444,7 @@ __acquires(&gl->gl_lockref.lock) if (test_bit(GLF_LOCK, &gl->gl_flags)) { struct gfs2_holder *first_gh; - first_gh = find_first_holder(gl); + first_gh = find_first_strong_holder(gl); try_futile = !may_grant(gl, first_gh, gh); } if (test_bit(GLF_INVALIDATE_IN_PROGRESS, &gl->gl_flags)) @@ -1381,7 +1453,8 @@ __acquires(&gl->gl_lockref.lock) list_for_each_entry(gh2, &gl->gl_holders, gh_list) { if (unlikely(gh2->gh_owner_pid == gh->gh_owner_pid && - (gh->gh_gl->gl_ops->go_type != LM_TYPE_FLOCK))) + (gh->gh_gl->gl_ops->go_type != LM_TYPE_FLOCK) && + !test_bit(HIF_MAY_DEMOTE, &gh2->gh_iflags))) goto trap_recursive; if (try_futile && !(gh2->gh_flags & (LM_FLAG_TRY | LM_FLAG_TRY_1CB))) { @@ -1477,51 +1550,83 @@ int gfs2_glock_poll(struct gfs2_holder *gh) return test_bit(HIF_WAIT, &gh->gh_iflags) ? 0 : 1; } -/** - * gfs2_glock_dq - dequeue a struct gfs2_holder from a glock (release a glock) - * @gh: the glock holder - * - */ +static inline bool needs_demote(struct gfs2_glock *gl) +{ + return (test_bit(GLF_DEMOTE, &gl->gl_flags) || + test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags)); +} -void gfs2_glock_dq(struct gfs2_holder *gh) +static void __gfs2_glock_dq(struct gfs2_holder *gh) { struct gfs2_glock *gl = gh->gh_gl; struct gfs2_sbd *sdp = gl->gl_name.ln_sbd; unsigned delay = 0; int fast_path = 0; - spin_lock(&gl->gl_lockref.lock); /* - * If we're in the process of file system withdraw, we cannot just - * dequeue any glocks until our journal is recovered, lest we - * introduce file system corruption. We need two exceptions to this - * rule: We need to allow unlocking of nondisk glocks and the glock - * for our own journal that needs recovery. + * This while loop is similar to function demote_incompat_holders: + * If the glock is due to be demoted (which may be from another node + * or even if this holder is GL_NOCACHE), the weak holders are + * demoted as well, allowing the glock to be demoted. */ - if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) && - glock_blocked_by_withdraw(gl) && - gh->gh_gl != sdp->sd_jinode_gl) { - sdp->sd_glock_dqs_held++; - spin_unlock(&gl->gl_lockref.lock); - might_sleep(); - wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY, - TASK_UNINTERRUPTIBLE); - spin_lock(&gl->gl_lockref.lock); - } - if (gh->gh_flags & GL_NOCACHE) - handle_callback(gl, LM_ST_UNLOCKED, 0, false); + while (gh) { + /* + * If we're in the process of file system withdraw, we cannot + * just dequeue any glocks until our journal is recovered, lest + * we introduce file system corruption. We need two exceptions + * to this rule: We need to allow unlocking of nondisk glocks + * and the glock for our own journal that needs recovery. + */ + if (test_bit(SDF_WITHDRAW_RECOVERY, &sdp->sd_flags) && + glock_blocked_by_withdraw(gl) && + gh->gh_gl != sdp->sd_jinode_gl) { + sdp->sd_glock_dqs_held++; + spin_unlock(&gl->gl_lockref.lock); + might_sleep(); + wait_on_bit(&sdp->sd_flags, SDF_WITHDRAW_RECOVERY, + TASK_UNINTERRUPTIBLE); + spin_lock(&gl->gl_lockref.lock); + } + + /* + * This holder should not be cached, so mark it for demote. + * Note: this should be done before the check for needs_demote + * below. + */ + if (gh->gh_flags & GL_NOCACHE) + handle_callback(gl, LM_ST_UNLOCKED, 0, false); + + list_del_init(&gh->gh_list); + clear_bit(HIF_HOLDER, &gh->gh_iflags); + trace_gfs2_glock_queue(gh, 0); + + /* + * If there hasn't been a demote request we are done. + * (Let the remaining holders, if any, keep holding it.) + */ + if (!needs_demote(gl)) { + if (list_empty(&gl->gl_holders)) + fast_path = 1; + break; + } + /* + * If we have another strong holder (we cannot auto-demote) + * we are done. It keeps holding it until it is done. + */ + if (find_first_strong_holder(gl)) + break; - list_del_init(&gh->gh_list); - clear_bit(HIF_HOLDER, &gh->gh_iflags); - if (list_empty(&gl->gl_holders) && - !test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags) && - !test_bit(GLF_DEMOTE, &gl->gl_flags)) - fast_path = 1; + /* + * If we have a weak holder at the head of the list, it + * (and all others like it) must be auto-demoted. If there + * are no more weak holders, we exit the while loop. + */ + gh = find_first_holder(gl); + } if (!test_bit(GLF_LFLUSH, &gl->gl_flags) && demote_ok(gl)) gfs2_glock_add_to_lru(gl); - trace_gfs2_glock_queue(gh, 0); if (unlikely(!fast_path)) { gl->gl_lockref.count++; if (test_bit(GLF_PENDING_DEMOTE, &gl->gl_flags) && @@ -1530,6 +1635,19 @@ void gfs2_glock_dq(struct gfs2_holder *gh) delay = gl->gl_hold_time; __gfs2_glock_queue_work(gl, delay); } +} + +/** + * gfs2_glock_dq - dequeue a struct gfs2_holder from a glock (release a glock) + * @gh: the glock holder + * + */ +void gfs2_glock_dq(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + __gfs2_glock_dq(gh); spin_unlock(&gl->gl_lockref.lock); } @@ -1692,6 +1810,7 @@ void gfs2_glock_dq_m(unsigned int num_gh, struct gfs2_holder *ghs) void gfs2_glock_cb(struct gfs2_glock *gl, unsigned int state) { + struct gfs2_holder mock_gh = { .gh_gl = gl, .gh_state = state, }; unsigned long delay = 0; unsigned long holdtime; unsigned long now = jiffies; @@ -1706,6 +1825,28 @@ void gfs2_glock_cb(struct gfs2_glock *gl, unsigned int state) if (test_bit(GLF_REPLY_PENDING, &gl->gl_flags)) delay = gl->gl_hold_time; } + /* + * Note 1: We cannot call demote_incompat_holders from handle_callback + * or gfs2_set_demote due to recursion problems like: gfs2_glock_dq -> + * handle_callback -> demote_incompat_holders -> gfs2_glock_dq + * Plus, we only want to demote the holders if the request comes from + * a remote cluster node because local holder conflicts are resolved + * elsewhere. + * + * Note 2: if a remote node wants this glock in EX mode, lock_dlm will + * request that we set our state to UNLOCKED. Here we mock up a holder + * to make it look like someone wants the lock EX locally. Any SH + * and DF requests should be able to share the lock without demoting. + * + * Note 3: We only want to demote the demoteable holders when there + * are no more strong holders. The demoteable holders might as well + * keep the glock until the last strong holder is done with it. + */ + if (!find_first_strong_holder(gl)) { + if (state == LM_ST_UNLOCKED) + mock_gh.gh_state = LM_ST_EXCLUSIVE; + demote_incompat_holders(gl, &mock_gh); + } handle_callback(gl, state, delay, true); __gfs2_glock_queue_work(gl, delay); spin_unlock(&gl->gl_lockref.lock); @@ -2097,6 +2238,8 @@ static const char *hflags2str(char *buf, u16 flags, unsigned long iflags) *p++ = 'H'; if (test_bit(HIF_WAIT, &iflags)) *p++ = 'W'; + if (test_bit(HIF_MAY_DEMOTE, &iflags)) + *p++ = 'D'; *p = 0; return buf; } diff --git a/fs/gfs2/glock.h b/fs/gfs2/glock.h index 31a8f2f649b5..9012487da4c6 100644 --- a/fs/gfs2/glock.h +++ b/fs/gfs2/glock.h @@ -150,6 +150,8 @@ static inline struct gfs2_holder *gfs2_glock_is_locked_by_me(struct gfs2_glock * list_for_each_entry(gh, &gl->gl_holders, gh_list) { if (!test_bit(HIF_HOLDER, &gh->gh_iflags)) break; + if (test_bit(HIF_MAY_DEMOTE, &gh->gh_iflags)) + continue; if (gh->gh_owner_pid == pid) goto out; } @@ -325,6 +327,24 @@ static inline void glock_clear_object(struct gfs2_glock *gl, void *object) spin_unlock(&gl->gl_lockref.lock); } +static inline void gfs2_holder_allow_demote(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + set_bit(HIF_MAY_DEMOTE, &gh->gh_iflags); + spin_unlock(&gl->gl_lockref.lock); +} + +static inline void gfs2_holder_disallow_demote(struct gfs2_holder *gh) +{ + struct gfs2_glock *gl = gh->gh_gl; + + spin_lock(&gl->gl_lockref.lock); + clear_bit(HIF_MAY_DEMOTE, &gh->gh_iflags); + spin_unlock(&gl->gl_lockref.lock); +} + extern void gfs2_inode_remember_delete(struct gfs2_glock *gl, u64 generation); extern bool gfs2_inode_already_deleted(struct gfs2_glock *gl, u64 generation); diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index 0fe49770166e..58b7bac501e4 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -252,6 +252,7 @@ struct gfs2_lkstats { enum { /* States */ + HIF_MAY_DEMOTE = 1, HIF_HOLDER = 6, /* Set for gh that "holds" the glock */ HIF_WAIT = 10, }; From patchwork Thu Apr 14 22:28:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5EC2C433F5 for ; Thu, 14 Apr 2022 22:29:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347056AbiDNWcW (ORCPT ); Thu, 14 Apr 2022 18:32:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347054AbiDNWcU (ORCPT ); Thu, 14 Apr 2022 18:32:20 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10DA083B17; Thu, 14 Apr 2022 15:29:54 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKkvFt006846; Thu, 14 Apr 2022 22:29:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=PTEA+N1XhqybQyFM2JLWgGwf4Cn2MWL8Z3+i5Yv4H5Y=; b=kIMON+/UugBMBphgx5PlvtZHm9X2SdnG7viiYx/vGwWjLL1mfUeMUppsyPpXHtZvv4ib LbL7uCvnPjlwWzWPQhU7Bu7s4uIBOv1ovSg+/MI+7NAIOkAjnRmCD3C47Vkwq7/qCwh4 U/6Bag3WncfSttZ253XjeSYXiL0EqMqfF3pqS+JN4FaDR/rD60mCDRR8O0Bxmr3Fphax E15V0PNtXpQF1o18vfJKHiRJintII6/ZMBAW5r1lYKnwI1GFq1c9u1hPZoUFVZqouVgF E13H2igAuACrseOCo4bWm076jFBtFdDN3auY6cQNTf0SNfNwAUs6DoYQkYtK3tskw60a 9w== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb1rse6w4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:51 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGhsX008736; Thu, 14 Apr 2022 22:29:50 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9m6k17-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jiz0OO8IAVtwKEw5qamLf5bVD4bMtRazN4tX604JKFv8Etloc6yl41Ch0hLf39R52Dgkl1TkP6om2RVJq9gtYl5+N8ZnecnU9rx1imtoP9GtJ4MuJOShpLdpiUWqMhD0lUNqC4AIJhsy3xqFtjW2XPqwBB3GE1edPLDOQk8rkaUVaCMPwwXIssXcKquYA066GqBx1aBChLl/H3MnpTujZwlpsU5JIQ3GUXCueB5TcW5yJvVyRrqviihhbLZtkGGBReiGl8bZmbr6PjcCRy1iHGYXmUmm405rKiYJTcuOyGIukGgb987FFEGhl3PORWgcKzV6kMUO8jsWR9N2ZxZcHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PTEA+N1XhqybQyFM2JLWgGwf4Cn2MWL8Z3+i5Yv4H5Y=; b=dW/FBQ1PgVt2f8CT3oEDUyW//pNfk7l38d5ZIUmVuhVDWGwRy0sQcH7DCXTshIsfJbeCpceIMh6B2rP4cQIPbxKNt6KFmt2rC7isQhLrcOJEmdj1alBfT/me+JhyVHDudhQZrAr1nFeXyU7O0egM6nsWOk/ryHoUqtjVUcDJgOpbcE406CZLuytmGXx5HDVYCzpAMijoubyS7WO0Rr6USeh6GSutoezJLbKt8Wru0dRy8qtXoA3VQ9yasgxw629IQkWu4QEJxJGpUz8exIlr/kqiPSvDwGdozO+TI6ZbqhdXoj/oR2z0qW0+2RFffcDawt21J29O6EI5XVJ4l3FHGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PTEA+N1XhqybQyFM2JLWgGwf4Cn2MWL8Z3+i5Yv4H5Y=; b=jRPUM44LVGmltHwcYvHklRZR8EjuJBDlW0Fz63Bf8bD0tpfBnMsPkD9wgWpxYc7TbSvYlfSEeL435Cco+FgILYHGrJtfaqn1JetLxD4kBh/k56IFW66LyqhEquDRqBqpFeP9tE9ts45MElkuehpLAj//QP2kA2yCWk1r+sLKAto= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:48 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:48 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 07/18 stable-5.15.y] gfs2: Move the inode glock locking to gfs2_file_buffered_write Date: Fri, 15 Apr 2022 06:28:45 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TY2PR06CA0016.apcprd06.prod.outlook.com (2603:1096:404:42::28) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 32fb5859-eb7e-4da8-524f-08da1e6648c8 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1/P9wjviQni1HheWXWm3SkDiFRmLV0S2TLAWoKD22HzyEl/CSLJE8dv6c4VivZwQxUle8RVtda3gblC83dFchUT6DoRw34ICcWrwAmm2oFYZeMxP6Zkyq0EVwd9Jt9nStmiyehgLmjk1378ays8Eevw+Yv8WaWqsXzmLZ7VYadi/Ak1/hBrpNzi/5ub8x8NiPQS9JhHWwmdOdkKvpWQ9MM58QJWQASawSHT4uGb2RgUkzPRZwbN+kUJU5AlSX5fHXMgibpCEv9HHQQk/Ho/mENxsjQB1kKmqGiW8TV/o61fBaK7p0D5ibfliba+aK2tOpmf9fYvPn+92aT1bOJrHdgceccryIdzkWQZ4rLBZV5PGEgOxeeKsMiuuA5AxVyMM1QweuEIPfxJqAydqP0ts0FxR7Nb6BsheVO4gWkeYwB/I10We6ghsT+EpVGrSvJCfEFc2C4KApFkwS/KjL77Vdmf13HyzE4fdFOI9VDb2bPYHQLD9hFDbLcWgXFcR3UjTNKWL4TECP7vFzCJec+W2sbo/EMDTaDI/glquyAEzgunNKMjms5QJchJg5gRYRiedW19bUnEgGKXUFOY5oi7dhYUjNhNwbKRx3JokM1ebN1LgzmRMM2iD8W7qF016hHclyr6PKRBuX2rNoZTsJ3OK0g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: jtfcRVsSyHilX3BWGRFjxvFD7EaNE2mdz8rl67Ol04LQLh+1rv0lCi+0+BF79UAnUFZo9hOoEky7UCM1D63jkPPOuZFNhe2T+oEuw3FETOlWOtqOfDwEjw9yG/rcaGqTWRNyWmUOB5Y83FXDkRTgeVGi1w6VDf/gZfQFAktCw76lE0oRageV3Q9mhLSIKBJJSuw9DtpJqjmfkPxoSN0bE7iMdGO7s6zB9Z7wFIDKnCIJM9Q/MGBoaMxPUTgdtdyiy/Y5ie7WFyTf8FvDzk3Uy9SQW/0RSIRJ/Q1rEMgimj6nBCGzJDQW/CUDQhxvfa+9qISsWCtaZqEvywmcQMl0CtPW8bhAa58Py4ArI5k6a9jkalagoSSnohOQ4QYRhrlj7Ar/X6ix1xi6ppNovUol1Kl4SSJhaURn40k8SNesrbVCdzE/t01++A5wPGjmqR+xr5kxcHJIp4Y86qDiQwKUCAOp17sO/IQHBt+X6sZUTBx1r77SBZ5M0qnMdaTUlkaA1SJKivRcPraJtsqY5T1abOaGMKG2Gheb7aFEmz/n0q5TUhWjSGbZOEi91C9D7hhGj5gIpPPrbkjjhAVXUbFeT4cML9r/aeJY4mc99JuPGENfDcDHwANK3NYTs6M9lBliMkxxseuuk/YNFqlmQdKdAG4y8eH2Kja5wmD2z2Po8Ifw9oVis6c923gNhmqO1b9D/6Mc2cbbuzb5VPi+NOgO/Q8Vi1qWHpvZzeU9GpjwkQEr8OIOzxt7PUzTqDbyMQGe6YvtKRRo+SO5UyvxyXlplVe4yfqZTnfqUW2ZNfU60b0PSmgegAVCj/voeZ8DhT6Jh8Ny/lo2HnbMej/zTGE0OjuFq2VidsoSpA8kqHsAdn3sMTwH1VcniFjtRXyMdeJhxykU6lmDU9QZoKkecPxIB6t/uqqWJeORyl6kmjSGZMURFYfP1v6tL7NhmG8lQTH/+VzmDWKh0NFpri4FIDngnuRw/VzYpCEXsajRhjZn+b+xTG+fMNc9kTT3V1FFbSHCIy6WpcUdxDTf6drVXmwASYzpjbdOhBzEznDctXokp+PHB1ez6EoGsl3KPfmfwl8S3Dq4JunJdknyxOM9i0LThYASfNZqMW1bFwKifq4//H2Mnw2BGSVuZPMMjq5G8HDBqgKMo35I8SFI5xCxwu1AJHLX1qCRhsLWJGZ3q8iWruXBSVRTfy3keyeCFVkCBpLDnyfDAiD4VDrp99GOkvAVpPPlKFYAO1vK8KJJMQAGkxWZXK3JV5zEM2P66BibvJD/7iv0rKiXXFv6agPgHeXHB98FjzRAKBpwYqYdCvUEXHQdvB3VBXYVLjGM6K2oUhY970LAaJ1cGPnO5aoSCKptzI6uBa2PHL7W4Alyu4UArSyR2PbtXvjAQsr4H47F4UhFei7AkmTXH0Be2HmjKsxLWoGzaUafw5Gt3bW01guBTp7blAi4vIQz2JQMq+g8lstk6AzgEE+PUeCK63eD5LMDHsOeZ25ThTqC7DFpgnTlla30X+loytSpa0Li3G8+hiK6XEnVfi0puH0CnnqE+TOOdu8Naqhi9Pntwf7ypheku4vfu4EAHmN3Y9JuDNk9AjqVe9UVrdzfS46+oSwO4UbznPbwLpmrWC/4OCqCYCMspjdm8p4qQB03mGWyYOuN8z6BE23MRNhr8RMIsaNTRGglcbzlg1bDJodAwSAjco4oFk8pIOIV7e01tWFip55d9Fqxv5UiplCt2lyKVWhQcHsWtOicXazgKL4jAVC8seZ1q6FdG9HaxMIbDIQsuomTZwP8 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 32fb5859-eb7e-4da8-524f-08da1e6648c8 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:48.4181 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7avWeaVMdCV+TOt8n+3IlOLg4MFTsv6CdJ+AZUkk8CVSdPAPhcWV3ijTBmyZfm2WONLTUYkLsKx7/83pBSgnaw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: GPc-RXWypI1_XgELGNO7hkC4S45wAHt0 X-Proofpoint-GUID: GPc-RXWypI1_XgELGNO7hkC4S45wAHt0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit b924bdab7445946e2ed364a0e6e249d36f1f1158 upstream So far, for buffered writes, we were taking the inode glock in gfs2_iomap_begin and dropping it in gfs2_iomap_end with the intention of not holding the inode glock while iomap_write_actor faults in user pages. It turns out that iomap_write_actor is called inside iomap_begin ... iomap_end, so the user pages were still faulted in while holding the inode glock and the locking code in iomap_begin / iomap_end was completely pointless. Move the locking into gfs2_file_buffered_write instead. We'll take care of the potential deadlocks due to faulting in user pages while holding a glock in a subsequent patch. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/bmap.c | 60 +------------------------------------------------- fs/gfs2/file.c | 27 +++++++++++++++++++++++ 2 files changed, 28 insertions(+), 59 deletions(-) diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c index bb9014ced702..fbdb7a30470a 100644 --- a/fs/gfs2/bmap.c +++ b/fs/gfs2/bmap.c @@ -961,46 +961,6 @@ static int __gfs2_iomap_get(struct inode *inode, loff_t pos, loff_t length, goto out; } -static int gfs2_write_lock(struct inode *inode) -{ - struct gfs2_inode *ip = GFS2_I(inode); - struct gfs2_sbd *sdp = GFS2_SB(inode); - int error; - - gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &ip->i_gh); - error = gfs2_glock_nq(&ip->i_gh); - if (error) - goto out_uninit; - if (&ip->i_inode == sdp->sd_rindex) { - struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); - - error = gfs2_glock_nq_init(m_ip->i_gl, LM_ST_EXCLUSIVE, - GL_NOCACHE, &m_ip->i_gh); - if (error) - goto out_unlock; - } - return 0; - -out_unlock: - gfs2_glock_dq(&ip->i_gh); -out_uninit: - gfs2_holder_uninit(&ip->i_gh); - return error; -} - -static void gfs2_write_unlock(struct inode *inode) -{ - struct gfs2_inode *ip = GFS2_I(inode); - struct gfs2_sbd *sdp = GFS2_SB(inode); - - if (&ip->i_inode == sdp->sd_rindex) { - struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); - - gfs2_glock_dq_uninit(&m_ip->i_gh); - } - gfs2_glock_dq_uninit(&ip->i_gh); -} - static int gfs2_iomap_page_prepare(struct inode *inode, loff_t pos, unsigned len) { @@ -1118,11 +1078,6 @@ static int gfs2_iomap_begin_write(struct inode *inode, loff_t pos, return ret; } -static inline bool gfs2_iomap_need_write_lock(unsigned flags) -{ - return (flags & IOMAP_WRITE) && !(flags & IOMAP_DIRECT); -} - static int gfs2_iomap_begin(struct inode *inode, loff_t pos, loff_t length, unsigned flags, struct iomap *iomap, struct iomap *srcmap) @@ -1135,12 +1090,6 @@ static int gfs2_iomap_begin(struct inode *inode, loff_t pos, loff_t length, iomap->flags |= IOMAP_F_BUFFER_HEAD; trace_gfs2_iomap_start(ip, pos, length, flags); - if (gfs2_iomap_need_write_lock(flags)) { - ret = gfs2_write_lock(inode); - if (ret) - goto out; - } - ret = __gfs2_iomap_get(inode, pos, length, flags, iomap, &mp); if (ret) goto out_unlock; @@ -1168,10 +1117,7 @@ static int gfs2_iomap_begin(struct inode *inode, loff_t pos, loff_t length, ret = gfs2_iomap_begin_write(inode, pos, length, flags, iomap, &mp); out_unlock: - if (ret && gfs2_iomap_need_write_lock(flags)) - gfs2_write_unlock(inode); release_metapath(&mp); -out: trace_gfs2_iomap_end(ip, iomap, ret); return ret; } @@ -1219,15 +1165,11 @@ static int gfs2_iomap_end(struct inode *inode, loff_t pos, loff_t length, } if (unlikely(!written)) - goto out_unlock; + return 0; if (iomap->flags & IOMAP_F_SIZE_CHANGED) mark_inode_dirty(inode); set_bit(GLF_DIRTY, &ip->i_gl->gl_flags); - -out_unlock: - if (gfs2_iomap_need_write_lock(flags)) - gfs2_write_unlock(inode); return 0; } diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index df5504214dd4..f652688716aa 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -881,13 +881,40 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *fro { struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); + struct gfs2_inode *ip = GFS2_I(inode); + struct gfs2_sbd *sdp = GFS2_SB(inode); ssize_t ret; + gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &ip->i_gh); + ret = gfs2_glock_nq(&ip->i_gh); + if (ret) + goto out_uninit; + + if (inode == sdp->sd_rindex) { + struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); + + ret = gfs2_glock_nq_init(m_ip->i_gl, LM_ST_EXCLUSIVE, + GL_NOCACHE, &m_ip->i_gh); + if (ret) + goto out_unlock; + } + current->backing_dev_info = inode_to_bdi(inode); ret = iomap_file_buffered_write(iocb, from, &gfs2_iomap_ops); current->backing_dev_info = NULL; if (ret > 0) iocb->ki_pos += ret; + + if (inode == sdp->sd_rindex) { + struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); + + gfs2_glock_dq_uninit(&m_ip->i_gh); + } + +out_unlock: + gfs2_glock_dq(&ip->i_gh); +out_uninit: + gfs2_holder_uninit(&ip->i_gh); return ret; } From patchwork Thu Apr 14 22:28:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561506 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6975BC433EF for ; Thu, 14 Apr 2022 22:32:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233442AbiDNWeZ (ORCPT ); Thu, 14 Apr 2022 18:34:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347105AbiDNWcb (ORCPT ); Thu, 14 Apr 2022 18:32:31 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AF4089CFF; Thu, 14 Apr 2022 15:30:03 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23ELRUOX001710; Thu, 14 Apr 2022 22:30:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=vZJ98q+gRGUwVgpHtctUzOdAQvBxJqKuS0h9IqLPB1U=; b=CplwYaEDxdrmBCxSEjeL8cvkFyAh01hxN/fZmlk2ZSVtHvlklLWRKz/Sj6qwSaRSjgGW KIPjYsaGQeUBC3BSZbf07uwBrBhBVts9rTt/3eJtoxEgDZPlIKlWs89p8f2UE618wZpw 4RqQeI2xeNptfPQCAbSw7lZ10FTvFMY/6NQcAlMAU8bdhmX0YpXEVhLQhV6gxeUhJ/sa j93nqjS7KohTrKxI/bZIZzVXLOQyaX0z4SCyGGscKfi8QT3h0L45Hu9YuzONQed3mArl xG5Z5QyBqfGyZDI9LEjpR4Was6o8PqkBONo+Qli/2hrfAhZ1htREWvoFtIiXaXlmItI5 1w== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2p3us-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:00 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMHVFq014845; Thu, 14 Apr 2022 22:29:59 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2108.outbound.protection.outlook.com [104.47.55.108]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5ucbt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:29:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c74ySfUYLDaCZKXvE5Cpol2PyPepeeety9xgZs2Eo6F/Y+0U3LLnHtXaU0o8xc2i0IENEYMr0yIv8th8s8ZqXGigKURzWQM0c3zgC3Xjo0S4wHyDspCxKwaTX97ON2QAadNiRmAk5CLmCfVlP/WUNqiEELeUPLGSm2YAR3KzM1o6PaRrlAnVKUzNvimWSMPctxwx73X8+B0KWtlfqzMG8j/oVZeU5uQZnIdI20boIKC+Mvt65X9fkgm7geHs7nb+UWGiTQxYTQ1tGuvo154BHx1Ges82Gb6n0LNBusYwGZvwoInY31SYldgzohyxs6wSHJp9V7AKr5a/MMhy2PdwWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vZJ98q+gRGUwVgpHtctUzOdAQvBxJqKuS0h9IqLPB1U=; b=KkIRc40KNVgZwDq3sr/oJRjBIrKhdA9ryntawlYlnaoUmnqxenbzFRUu4oJyhJ9o1SlrT1UVcFzqzudmt3S3uOkfPOj5AiKGeFrG8HM9stnGKK3O/v50KxO6BV08rsrsFzf1ZvcFoVQ+EFbhRnS06k8o6f5czkZkZ/l4RBbXFLCB4M7iDE3NKYhcktrojzPBMvtmfnI48dUyFfUfOAWPsdJu9oPjJJV/RFiCfydfu1eJ9L4ZqCzJYhvsBkAi03BTn3kYmHkZbH9ZDK5c4xgfEmjNBhu7Ixxn446P8wDAKgc2L0yFirkmRkhEcWZ+HDmvw6b8ZAUPS6qvhazN8k+sdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vZJ98q+gRGUwVgpHtctUzOdAQvBxJqKuS0h9IqLPB1U=; b=GAxqU+e4uwkmDpD8RW7sO2mB2YQWYN/1e2HL429bf1JHrCRn3GQ/NfGDjlb1swvWoG4ly3K/+naiXz4qScgAMr3n5G9TyxJwMLOi69NSSAM+Y9IvjXg6g/UvJ292fOIwENUGgEui2fLfMTrdQIhS4cWr8C5DqhVHHJk45upAf8Q= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:29:57 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:29:57 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 08/18 stable-5.15.y] gfs2: Eliminate ip->i_gh Date: Fri, 15 Apr 2022 06:28:46 +0800 Message-Id: <844b20e15b0e730c43faa93347d7a65ac4e7b465.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR01CA0032.apcprd01.prod.exchangelabs.com (2603:1096:4:192::18) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: df6f7bd4-c066-40fb-e8dc-08da1e664e10 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jtR0hKu9WGq+X7xm8z6G2OqUJ28/J1YfcIG5ZPz3JMb+WbSlUVhOvBVheaMmXIuUYSfPmHAIdM19GEYSkDvmeTJBhyifanC3uGKv9Vxz993nse+kHecSLHZFR6A5iZ0Rzeh3xE7mETg5QpbE8cFQcsVyCLcDeoLvbs/w4Y6jX6MH74GNu+35pHJ8ggXDH95FUmrImboGeDg3DPHcDWB8ZKf+6Fjk+rdUBT1Ugpf3lDycAiMAEvvAfziydHHeMyyFYfeGO8m4j4ebEBZ8Ckfa7jIGjyp7/xW5Ll65hCRUG8pglwMcNzRKeZCsTicEMp0vRturYnPM8vuvlKaK4IvYpj5pYRPRmcL6KChQ9FtVWpcbzViEujjYWpYIYcdzdeCyHD44iUEqHhKHPniSucXZgyFArUYGHSJ3GmMhpZ6vapezvqrBBdLLzIeKCeOjaWV5iGs/FajhwEkPabj8WtK5Hj4VawWzzL7B3naV6WMeI7tvjF2WWtzz6PMapts91KzcHSX3uVz4VVbRzubA8HAtrDFhaLjMWrDPVYMi992E2edFrBsn2Be4W09qy1DBK6/11M0QVKhJYpPLtgKkhbYuczKs4qXKEwQaPxIf2jUz4+qIqnKMdNQAQflo4H9poOZzkAQs6nbFySSxsHm6Fk2Lhg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6pK7DBExUkkcJ1/ttIJEiM1q9qdjDQDOJ1JrQTPEN26TeUl9wMAKPtnpum3X7FnwcvV3x7qZvKCFgCdnjLcVpo9lObqN0KN71oczoVdjMvfYxp2gk/FLS/RePqOXUy5gSED67RqYwgMhWK//BC85Cc4U+QclwUrPW7ao9JG9+OZddYlwROSyxRyZ5MAVycC3+OwIfPtw0pmV3vbIlQ/yNCGVM3pfMbwNSHZmTT7AL+ljJDDPmYDANU9vFGpGhcQJQrjz1STv2U2WEHnUBlX5nZWYpdADgIObtRXQ52qARneLU745NQV3ykl+hjLWpIum2og//UfLbquecY1GyFXL6oJx52M8JHtSR1wQ3S/si9XR5N0hWw8jcQ1XGTnzlUSGkH1gjzh+alipOq7THU8+NLwWBwn1++3btHC+FHM4k+Tjdyhh3Nl3cfVZtVLt9L/WCEDAtrV2JtKDVFVjtzCwq0OxZU6QqfYU+iZuzthOJUJfd58Ia0PxEsUJGQfngZHbt8F6hd/+shS5CZ/ygMiF3WLD8iUn455z/qFq+Nj7c4c5zn36e7AzAHSzgvk1a67y4eeAM51x7+ER3BedYHbMXfLkWX512d3u4qfCIC6ivObsWsScsQgSom4hS55bs0FNfoVEnJct3wqdWJo3phNBlOJQWOtv1X9kDsA8Io/Kj1SwrBRKR6e9MQie+HjypaBBZm0TYiSUHCCL4VGxwJtJnNxyxydZXJrZTBR/X6QarLzp5t2SkxtFkqI9oU09W66WNhwekpveulPVgMqIdtsDtr+V1wb837glvCCdye4CPXwU5oDsn9iy1+3WeeogkYs88EasLXDW7oe3fuPGeRho+yghDovjy4ArGMcsFDi5xHCyWak4dB0puzX8BzsstV0KcSYNOUdm5kR8Ab25oRSPhRyNsIdgRK1mO6KjXv18/2uegMdsEtrEzON7E8Li4mvHFPyGxrDdeNaxhMwwpktLd6t/uT51P0ybPKpPp4/5ViNpk3lIah+d2hN6Duwvbfey3ndunG4lFJV66n4ge8kAOnoi+WXsDwoZeAzY5JBc7pzOfKQzqgxxiCX0IjO1TaYM4oUFhpFpgHENva9z2Nq+yR6ZCBYSPU/cfAj+JhPfZpp59oZBeFjNjfPHz1JcepTAcq9Crf8o9UODcs4rzqfEbkJKacmKNp+kFK12417tEpb6EowihaBEZ0LXlU1vqt/lA8W18/rLMcFibVdCm8iNuqZYwYsMz2jTKjU8juJ+5ydNYK4hRv4Kh+yYg9QaFOlsBT1wzhp/ToEbfcVH1AX1XOv2Ld4ViKlfS3+SI/7y9xXl8NUibYaLgL+0rW6WrYnrHJLb9Cmd7G/i5scy0wXh1x0p/f8Mj/w/vk6ensUVvfCg8rVGZFAcB35949CAHctWRVjNBqwnMP7fBVKCclRPAY6Q1TiLaPjPftzlePMUgLwGD8QAdz2WjM5jYj4kEXBIa3MKkCxGP20qtG06NSEk8QVRZbQu3Fc0/DulGJOQpqKHZbwSskBaiXR1GG+ls8w5V+U7EXtf1R0eU/gBx/3n9aRdu5ODU71XQvKzUplA/Twrzv40IX31izV50e2Q4q45G2KeehZKgBY3Dlaf1GsJuwBRGe/XMWvIa0OHC5yHH5tzUSFD5nQEOk63HeI1cNTl/NFjC35AvXPeOGtFnSa9VoSsK8eQD2noadfdzsnL36cjygqTmxgula8gAXW8rf0g1g34lSOrTS//7NhvxzWhaP0iZKkLfzUjDNjNowdI9cjmwLpfuktVOIiAbrWMc/Hl X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: df6f7bd4-c066-40fb-e8dc-08da1e664e10 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:29:57.4230 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6eahVbWTBwvDWKBJEt8cfbbewR6YTkylcqXn+aYhZFy/5JJUfSug/0Q5y6WAbhIBMwpGfM6yqjdFptl60BFsYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: Ua-gi_VVGpHaf9QT7SK_gtlGdEKqTmPd X-Proofpoint-GUID: Ua-gi_VVGpHaf9QT7SK_gtlGdEKqTmPd Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 1b223f7065bc7d89c4677c27381817cc95b117a8 upstream Now that gfs2_file_buffered_write is the only remaining user of ip->i_gh, we can move the glock holder to the stack (or rather, use the one we already have on the stack); there is no need for keeping the holder in the inode anymore. This is slightly complicated by the fact that we're using ip->i_gh for the statfs inode in gfs2_file_buffered_write as well. Writing to the statfs inode isn't very common, so allocate the statfs holder dynamically when needed. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/file.c | 34 +++++++++++++++++++++------------- fs/gfs2/incore.h | 3 +-- 2 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index f652688716aa..288a789cb54b 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -877,16 +877,25 @@ static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return written ? written : ret; } -static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *from) +static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, + struct iov_iter *from, + struct gfs2_holder *gh) { struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); struct gfs2_inode *ip = GFS2_I(inode); struct gfs2_sbd *sdp = GFS2_SB(inode); + struct gfs2_holder *statfs_gh = NULL; ssize_t ret; - gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, &ip->i_gh); - ret = gfs2_glock_nq(&ip->i_gh); + if (inode == sdp->sd_rindex) { + statfs_gh = kmalloc(sizeof(*statfs_gh), GFP_NOFS); + if (!statfs_gh) + return -ENOMEM; + } + + gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, gh); + ret = gfs2_glock_nq(gh); if (ret) goto out_uninit; @@ -894,7 +903,7 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *fro struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); ret = gfs2_glock_nq_init(m_ip->i_gl, LM_ST_EXCLUSIVE, - GL_NOCACHE, &m_ip->i_gh); + GL_NOCACHE, statfs_gh); if (ret) goto out_unlock; } @@ -905,16 +914,15 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct iov_iter *fro if (ret > 0) iocb->ki_pos += ret; - if (inode == sdp->sd_rindex) { - struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); - - gfs2_glock_dq_uninit(&m_ip->i_gh); - } + if (inode == sdp->sd_rindex) + gfs2_glock_dq_uninit(statfs_gh); out_unlock: - gfs2_glock_dq(&ip->i_gh); + gfs2_glock_dq(gh); out_uninit: - gfs2_holder_uninit(&ip->i_gh); + gfs2_holder_uninit(gh); + if (statfs_gh) + kfree(statfs_gh); return ret; } @@ -969,7 +977,7 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) goto out_unlock; iocb->ki_flags |= IOCB_DSYNC; - buffered = gfs2_file_buffered_write(iocb, from); + buffered = gfs2_file_buffered_write(iocb, from, &gh); if (unlikely(buffered <= 0)) { if (!ret) ret = buffered; @@ -990,7 +998,7 @@ static ssize_t gfs2_file_write_iter(struct kiocb *iocb, struct iov_iter *from) if (!ret || ret2 > 0) ret += ret2; } else { - ret = gfs2_file_buffered_write(iocb, from); + ret = gfs2_file_buffered_write(iocb, from, &gh); if (likely(ret > 0)) ret = generic_write_sync(iocb, ret); } diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index 58b7bac501e4..ca42d310fd4d 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -387,9 +387,8 @@ struct gfs2_inode { u64 i_generation; u64 i_eattr; unsigned long i_flags; /* GIF_... */ - struct gfs2_glock *i_gl; /* Move into i_gh? */ + struct gfs2_glock *i_gl; struct gfs2_holder i_iopen_gh; - struct gfs2_holder i_gh; /* for prepare/commit_write only */ struct gfs2_qadata *i_qadata; /* quota allocation data */ struct gfs2_holder i_rgd_gh; struct gfs2_blkreserv i_res; /* rgrp multi-block reservation */ From patchwork Thu Apr 14 22:28:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48437C433F5 for ; Thu, 14 Apr 2022 22:30:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347093AbiDNWc5 (ORCPT ); Thu, 14 Apr 2022 18:32:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347127AbiDNWcg (ORCPT ); Thu, 14 Apr 2022 18:32:36 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70F14C4E33; Thu, 14 Apr 2022 15:30:10 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJKxnV032238; Thu, 14 Apr 2022 22:30:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=//lQbjyfxI29D4MI73tnBFtgEjvUoVPymuC7OE/GFAA=; b=Xp4HsEylBjzxFFQTLjZ2VAHGHsUBLTCuk2cy5SBzc42uV1NIbv76kFEIIV3/cIY3JyEM /IZ4CUykCPCHK/G+7kW2vaojiyjBMAXqZL9QlCC9TeQYCfJmKRTrtMkxH+jpe7HCoS7d zYQTyMnxqRWMvi1wdPiZkJ8MftYg+/tHf0GxH0NNfFnCpARr3VPuvFrhm+Ykq0QNsiAD HkMr5AHxkbCMh4HfrVcJfTGCEBVEJVLq/Aji5eZNKfyxei+muuP9tmiYxyrdHlEAx7u4 WTLPclqC/m1r8BJ49AV28+c8+VPwfO76/1bbIwNv1s+c36nzZIomtXjn4AQAOADwM9WL Bw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jddp2q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:08 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMFpMH006379; Thu, 14 Apr 2022 22:30:07 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2107.outbound.protection.outlook.com [104.47.55.107]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5p4we-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mIgKGHh3KarQPWF2FS4PVQltbeFyuxbWSrNCFDO/6PBi/Fi4iAb/qUabiMmuYK896TuricnOpVUHEkQ8y3sYnhPJRCdxurRJae6h1p750hZGTGmwZV8ZyfhvLDJe9us6m+lB5io40NoR6MCUYxrWePCRyFbqEnBAdFLARSOhlPDoGnXZGmEWCuJBMVl1KrlhYCKIJvi+4MICp5v/93GATXQpDkpbN7afGtaCJ1DDfS6BMpJFL5nI0mDXOmgEZvjD/YIrM+3A72jNYjmJBIOT7/aDu16KB2Ijo5CBWW+FXYy1//8H1Xdk0mOVeStxPoUPct92yKl7r5jjt4fTTv38zQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=//lQbjyfxI29D4MI73tnBFtgEjvUoVPymuC7OE/GFAA=; b=PffcBZPegVxXdJzAWRTY4Y/qtAkyRFXitOHHD9bIES4VwXZjrUXpCQbNIlt9U59j5u9EVoViYyTewTiyzO0DcR8Oh//x6vOaGARYjxLF/j6MTYcaVaJ4usspDDCOEOc8oqJ1gCPLrgnSQITC/LtMIpkxElZd2COIveiFstNdi6Q7uZgYGjuVhUkWtTCKRMHx1oEDJzE2L7lsaTrksek5073KZoLhtvS0ZQtHS3uehXZrdNM9RdGXdD62KUcDgxuW66+vxLZxJ94VsGMsfSy5gkxmemt4rwV/++XRnML+zokoa2SxuWtSWJA1JgB+rV2bAcvr0pfgqUmsOPqALy/ysg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=//lQbjyfxI29D4MI73tnBFtgEjvUoVPymuC7OE/GFAA=; b=WWuoIfrBGs4981imloidEgXICTgkMHMQ6krXrxdwvPFucwxH4cAFbFaqzJ87PBXvL34jvW/xoVPWJJnQKHo+bQLSKafX1zHrUknZN96uqZMX6cVxp4/If8Q2CcZmxV2iU/T3r+uEGvO4ZGW0VBt1WOlSIFZOoCGEZq2i25NwDTs= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:30:05 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:05 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 09/18 stable-5.15.y] gfs2: Fix mmap + page fault deadlocks for buffered I/O Date: Fri, 15 Apr 2022 06:28:47 +0800 Message-Id: <087a752bc8848ad8814bee4648d8b9d855c8438c.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SG3P274CA0013.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:be::25) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b1d607b0-949e-4329-ebe6-08da1e6652da X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vLXrb/4D+GtttL4ClOQXMWGSuaYpYKSnpqR/uFfEc1b9XpOxquOpiJG3WbfJbObtylTVNeJUECLWsh49mvnAOeiFZAD9x2U5oWpgq1OGU3NyrXSUWyOCrXL9NiY1ToO8j976bNJzn/uvvRm5g7LbGbVlsTQce/2UAXkaRAmLVizPAM9kdDF3K7Jqc57cVww9c3aTvmCRM37LC7+ETqhS8rYTsfui+CBZK/LRFSEwlOap87hAb7jVYMpuMxZ38vp1GgazEQkIiPbr4hp3xUHvE/BAn9dmUsjdvsIFb0WWtHa6G71yzlrlGrUh/K60j6KuoYMMyvpxQeKcfXevsafwIcnpwf8agIxlFkf5sQDc3abjwg4yENSCqny/3G4ICL0zqV0W4YXqcLzL5v5ya9QIunK26oRzZM0QnVeSLAx/o95g7W0Hf6I39FCsoz5Pc3Ifa15nyFly3ow+AUoLWgrM8MtWjzxk+3RBd3O8LQARLXOKi1V1cEGef7nLRY3snmuUX0J1liAhuXD8Vj72fu5fpXCz8pVinDWUWF5ac42uTvzZrj/Xeg7Z+dAM1e2XqCFvS0CpkondJ9M2VL28IyoLS36jitWIFH9mMlX+RTIM8Q+jGUV10GN6wBNhzqd/Xs4jTvZW54IL1VJclG2ZuhUXiQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(83380400001)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dOANZ2P65sT4xm3CPgZhASTMGT9R+T6w9K6p58MtKQaLOCdGnSCSdE/Fbl2Dv2LB3qu7ZV7FfgUkrwVn+RLaR9u0JfgiZGfgVrl6vXyTsYhE5MJBpD5od5W77z83W0RyPe5SOeNx76VFt7vS8GLlkr5wfZbCHDxnaeINweU8OjOUlpwnlMbUO91AR2kauQDqiPtgqEn4onmlXUEyPvmIc64csDBRjmZBdi4oJP3+1jjQt+45vtzlebD2ws43QIc79KWWSnMndLo4c2c4r3nLSCqkL1rrJkp7FVTKl4z2xEfPPRQ2OLKrgnC47XQd8zvV4qQ6d87isIDmxgYCuWo+tem6tTuX3/jA+I4CssN6q2cegZHNo7GFvbAVZH4REjLVxHrMDOTdiqY50vF3fl/VBihQQuGZoQ3jNBCm0UepbsD4Efwtn8ZNFx7ofRim6FIz4hTmrNbKEIkqKz7GamJfETTcSwntIO/sxZK6m8+yZLM0ZTkYLcCoSXrN31ZTJdn2H/UyiCd1/4SivFFLBLQYZVMspUbYLsQuwO+RjnFLfgd8kxkoSuKarIVBblKm8DBXFcGXkzfx1HXD9VYUSqUX17Uw0cLcArnqZKCOCzveo2JJG18M938s0bSNKV3MuESmfFdyv/f9RfckLyy3LVEzWRWRYcWxKmbi+JyIKSjXh2NqLrW5G9xG/hKAbpDOMoaikysACIGMb/6gcFloiNYXXoPCrZX0/DZmDGDbhUhvJz6A33X5NHq0lgYBG008HljyjUulCgz8g7Gmly9lpJEaB5PhyVvD2lS+1UuuSXu53k8k8VJlS/tq2LJq7FWi5mq3NhCyaHobjbwW3zHGf6d0P2Z1LhiIMPUKoZF3KWHcNcyDIm7rTSkyl5fYwnc+F5efTbmyjOKsv/8p2TxZRlnWMWKy/D5YGjY6rvfgno+jOElJWoNBPiG6UrQdMxrHYcwz6V0mjOUmrZMC6kzf9Q+sjxntiJccbfy0CToHB8o5fF567Lo3FeRdybA3ZM5aw5YoEJ9ebwf/HmSbDYtvVTBB2Z974LFWiamqmNHe2UDEZWFO14UIvPQQn5tbU9BNPjjsqDGcdGWcZoIIMoo6VpBvDYxnPIs3CYZw1UvwDLHQUrIq+/rDFs+KlHAy/oJe3DYc43+RfWdS63oo37wKZBLUy3YBl0c2pEun2O1oeS6iaJtbuaG65lxWFhKZxBIr24rbuHIDAXoiPuQtmvnSDVgh4FKjRrogsPhuAfeSDRoSe0vs/HpLj4LzfOReKGFuJlwL3o4wdp0bTKafgF79zNPW3nBEW9tfSZydqeIFtcZ7wb6gb3aQaBY6jHktSGUpkRWvNFfyO1nSgYZpexf4WXh8Lz66d/sh4qk5Qj/ufERtzAoGuWNDHey37SOjU+xowPM90nUgkZUxAcyrrkic4iVeO9x8vngN2WFfNgaeSP7lZRnfLkw5ZJvYYwkiZ9j7hwaenzSmrtKKyHbzuUP+LRLo130cbK8DliHVwxoCj0h6dWc1SgROeLYTXheP0Do8kUhIewH4F6v5ahnnCga5VviWtBHDR1WK70znc0wOt8IJdlCZJyrvUKWIOZMOog1CBf40HOxv+p8ofhrMKzP5sr6tSxC0ZsHQ2l2PrSZEO+4SBwmi3GbfxVu0/d2/RM1EcWnD2NvscDDil94pk1lWeq4QNfUAe/hr/rxmTgcNVDfOFPlCWl2A6Oyn0feIAshR6gnSqLZcyc+xvJWJQzfHYpw6qmwdUZK3of55xzLRswFT4Ss5a/a5ONfwt01xV/mQcMxP X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b1d607b0-949e-4329-ebe6-08da1e6652da X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:05.4265 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PmRBup4gag/O2uQ0HZYinyvTVHAMpDSO1oF+GOpvgSCTS8fq2Ezx+EQmrSHgLQc+VoyXHCu4SRRYZ2F/U+avwQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: II3naCtbDWodFyu9mKbsJvi18qRz6toP X-Proofpoint-GUID: II3naCtbDWodFyu9mKbsJvi18qRz6toP Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 00bfe02f479688a67a29019d1228f1470e26f014 upstream In the .read_iter and .write_iter file operations, we're accessing user-space memory while holding the inode glock. There is a possibility that the memory is mapped to the same file, in which case we'd recurse on the same glock. We could detect and work around this simple case of recursive locking, but more complex scenarios exist that involve multiple glocks, processes, and cluster nodes, and working around all of those cases isn't practical or even possible. Avoid these kinds of problems by disabling page faults while holding the inode glock. If a page fault would occur, we either end up with a partial read or write or with -EFAULT if nothing could be read or written. In either case, we know that we're not done with the operation, so we indicate that we're willing to give up the inode glock and then we fault in the missing pages. If that made us lose the inode glock, we return a partial read or write. Otherwise, we resume the operation. This locking problem was originally reported by Jan Kara. Linus came up with the idea of disabling page faults. Many thanks to Al Viro and Matthew Wilcox for their feedback. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/file.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 94 insertions(+), 5 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 288a789cb54b..2d0aa55205ed 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -777,6 +777,36 @@ static int gfs2_fsync(struct file *file, loff_t start, loff_t end, return ret ? ret : ret1; } +static inline bool should_fault_in_pages(ssize_t ret, struct iov_iter *i, + size_t *prev_count, + size_t *window_size) +{ + char __user *p = i->iov[0].iov_base + i->iov_offset; + size_t count = iov_iter_count(i); + int pages = 1; + + if (likely(!count)) + return false; + if (ret <= 0 && ret != -EFAULT) + return false; + if (!iter_is_iovec(i)) + return false; + + if (*prev_count != count || !*window_size) { + int pages, nr_dirtied; + + pages = min_t(int, BIO_MAX_VECS, + DIV_ROUND_UP(iov_iter_count(i), PAGE_SIZE)); + nr_dirtied = max(current->nr_dirtied_pause - + current->nr_dirtied, 1); + pages = min(pages, nr_dirtied); + } + + *prev_count = count; + *window_size = (size_t)PAGE_SIZE * pages - offset_in_page(p); + return true; +} + static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, struct gfs2_holder *gh) { @@ -841,9 +871,17 @@ static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct gfs2_inode *ip; struct gfs2_holder gh; + size_t prev_count = 0, window_size = 0; size_t written = 0; ssize_t ret; + /* + * In this function, we disable page faults when we're holding the + * inode glock while doing I/O. If a page fault occurs, we indicate + * that the inode glock may be dropped, fault in the pages manually, + * and retry. + */ + if (iocb->ki_flags & IOCB_DIRECT) { ret = gfs2_file_direct_read(iocb, to, &gh); if (likely(ret != -ENOTBLK)) @@ -865,13 +903,34 @@ static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } ip = GFS2_I(iocb->ki_filp->f_mapping->host); gfs2_holder_init(ip->i_gl, LM_ST_SHARED, 0, &gh); +retry: ret = gfs2_glock_nq(&gh); if (ret) goto out_uninit; +retry_under_glock: + pagefault_disable(); ret = generic_file_read_iter(iocb, to); + pagefault_enable(); if (ret > 0) written += ret; - gfs2_glock_dq(&gh); + + if (should_fault_in_pages(ret, to, &prev_count, &window_size)) { + size_t leftover; + + gfs2_holder_allow_demote(&gh); + leftover = fault_in_iov_iter_writeable(to, window_size); + gfs2_holder_disallow_demote(&gh); + if (leftover != window_size) { + if (!gfs2_holder_queued(&gh)) { + if (written) + goto out_uninit; + goto retry; + } + goto retry_under_glock; + } + } + if (gfs2_holder_queued(&gh)) + gfs2_glock_dq(&gh); out_uninit: gfs2_holder_uninit(&gh); return written ? written : ret; @@ -886,8 +945,17 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, struct gfs2_inode *ip = GFS2_I(inode); struct gfs2_sbd *sdp = GFS2_SB(inode); struct gfs2_holder *statfs_gh = NULL; + size_t prev_count = 0, window_size = 0; + size_t read = 0; ssize_t ret; + /* + * In this function, we disable page faults when we're holding the + * inode glock while doing I/O. If a page fault occurs, we indicate + * that the inode glock may be dropped, fault in the pages manually, + * and retry. + */ + if (inode == sdp->sd_rindex) { statfs_gh = kmalloc(sizeof(*statfs_gh), GFP_NOFS); if (!statfs_gh) @@ -895,10 +963,11 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, } gfs2_holder_init(ip->i_gl, LM_ST_EXCLUSIVE, 0, gh); +retry: ret = gfs2_glock_nq(gh); if (ret) goto out_uninit; - +retry_under_glock: if (inode == sdp->sd_rindex) { struct gfs2_inode *m_ip = GFS2_I(sdp->sd_statfs_inode); @@ -909,21 +978,41 @@ static ssize_t gfs2_file_buffered_write(struct kiocb *iocb, } current->backing_dev_info = inode_to_bdi(inode); + pagefault_disable(); ret = iomap_file_buffered_write(iocb, from, &gfs2_iomap_ops); + pagefault_enable(); current->backing_dev_info = NULL; - if (ret > 0) + if (ret > 0) { iocb->ki_pos += ret; + read += ret; + } if (inode == sdp->sd_rindex) gfs2_glock_dq_uninit(statfs_gh); + if (should_fault_in_pages(ret, from, &prev_count, &window_size)) { + size_t leftover; + + gfs2_holder_allow_demote(gh); + leftover = fault_in_iov_iter_readable(from, window_size); + gfs2_holder_disallow_demote(gh); + if (leftover != window_size) { + if (!gfs2_holder_queued(gh)) { + if (read) + goto out_uninit; + goto retry; + } + goto retry_under_glock; + } + } out_unlock: - gfs2_glock_dq(gh); + if (gfs2_holder_queued(gh)) + gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); if (statfs_gh) kfree(statfs_gh); - return ret; + return read ? read : ret; } /** From patchwork Thu Apr 14 22:28:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BD66C433FE for ; Thu, 14 Apr 2022 22:30:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345494AbiDNWc4 (ORCPT ); Thu, 14 Apr 2022 18:32:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347198AbiDNWcv (ORCPT ); Thu, 14 Apr 2022 18:32:51 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99025C4E33; Thu, 14 Apr 2022 15:30:25 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJT05L028178; Thu, 14 Apr 2022 22:30:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=SuLLAqGzkVEgnC9Uurr+MdSPiuqqhV34kPT76gMnqjWdsp4O11z1yq/sfXsyoWyp5VyQ vj028xS1obzehGDkqm9lIA/Ul00Gf42RdEVKFCHlPflSrQg7o+DJ5DoZ5eV3h4IAuFAb Dqq55RlElzk/k/aNIKQiuSYMYmzW9akHzsx91EBvM1UDX4fQTyADKwKunbOF6IY8qFRN PPcqngN8pORLFQQIun6OSXPIgooKDofibDpQtsDBy+TWd95m52DeC+TAtn21Ovt4WTMm EFKe5bRbV0D7fkPvNNVwadfOw1TruYSglyBYrxaB0K0sqxIKnxt91UeoSkCbuwkNshEJ +w== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb21a5m00-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:16 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGOMX008406; Thu, 14 Apr 2022 22:30:15 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2106.outbound.protection.outlook.com [104.47.55.106]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9m6kew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D+b7d+hVY8h/BpOYpzYs/E8sR+0XaVwbxZnWk5sVRvHi8ch7dtYeJlhr2EPs18rXeocbDcgbKvq87pC9zgsTRMZhXUcOWLiOmucSOEX4pREY+yNhCoqPSNkYg2Oxcsqdy68we2efJS0UIbWEp/oVBqCSwcVdBlOpYG4c+bO74t5+fv0Tyw8dnVYG66feFLKjzA2aH60eRrO8l7mQ6dqhyqqgF5LtXdeI/zolItCo8OBRwQUqHuPfPLdA7ymtrj4xrrpc75F/VAelIca7KGHu9e8e9OiYwONoD300Hh+PXzxFyuC2OEgK7eCb321IqgmYKGfvldZQdfcnoSjWF8HoCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=eiy/eoelalPKuz6A7uU3IUopirF9khWKh5DHLMVGasaUjWnRyRsT16UN+6O+mQv0Xi8u6n5zQA7xY0DZmpXxhR0/r13JCiN3s7z+wy/rArBdZ5DC8iGH0RovcjS4L0f0K0+hfaek1DVUQyYrH3y0oV6wWc7wYGbShgJZXHXEhaAAeyGGRg/IUfavwoGmmqAqImV4f3iel4O0n2Vjtd0X8Q/49ArwdWwPfN8fTvGS0Z0+QfU2iIpUFYZSXH9vEzTO/qa+MzBZqnIe0qnQEicxYKZAz4YObnqABrI6l8+1g+zs74n/nx7ACdXzZzYMtoOgo1+UpJEnvVcaFDLqucf9NQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HuGfyFsPlSPlWrwhw45+9Dc4WjbxgSWTpY2S28+blls=; b=D+ct7rQOimVnvzY6tspf4+1yyorXEW83vHrYCvwy3r2WHaCJgDJhtplBUrFZJchnCyWd2GP1NvftbpFEARwF5H8INTZpYLuUUYLFjNk/LUyZNxrfMdS5MdzYSkdo/IAyeadGMFEOhpt8aLrEf4mP+LDvWy2kAS3c+EorUkltFRU= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by MWHPR1001MB2094.namprd10.prod.outlook.com (2603:10b6:301:2b::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Thu, 14 Apr 2022 22:30:13 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:13 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , "Darrick J . Wong" , Christoph Hellwig , Anand Jain Subject: [PATCH v3 10/18 stable-5.15.y] iomap: Fix iomap_dio_rw return value for user copies Date: Fri, 15 Apr 2022 06:28:48 +0800 Message-Id: <63440885619fdfa1a520a9528e38207311f44f2a.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SGXP274CA0017.SGPP274.PROD.OUTLOOK.COM (2603:1096:4:b8::29) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 99cfbcb6-e2e4-46c8-da33-08da1e6657d6 X-MS-TrafficTypeDiagnostic: MWHPR1001MB2094:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HrSQ03uu+TSpdPKDsaBwvpu446g6bxZFZ78fg3v6yb8ytW4jFwUFSTueJMf+76Ntv8KjPG9OSL35bKCZVY6ghqcSmUa+PjIkeFKSWVK5p4Ju3naHeE4ldKkAJxgU/+MoQJ9rnktK9zA3yVpL/+xWwZSlbb7CUmmC2JBkGvQPFq61p0/pfbWIEk+WA0qPhx5w5xk2xjehTGP4KkiNZD1CGIJs0EGOmjaz1OVRmm75LtgHK1z91VkXmw6TLpUtZmwWqxbhV00DayopplUxtLbZZt/QPiA6xBDEvYFa4FAp4rxBNu87MSgpRflj8rMEVA6wrFQXHE5/4CI4vs+2jASe4eQA5Lg1hTcN/MjHZewtOvWASMH1Sla+1jOhxaFaI0iGUuO5/rjlwkS08r+D3okCFuHii1lbUC+3pLnuhVUPYlh0aKdqFdAn9HBYqvsGldXIhpB2SzWV7k9TvTVcLZ7LCN5DIs3fyegs4KwX1Qd9zf0x0iLllSHHeshuy3l8dZh90syLGrjs3yeaX83Fe3CaJ9JiwTFHB+3cgfkZRU9db5ucdH212TrleYqjycx69vS0hoD0+rN8LmAxkqiCTriHPhEzyTVfESaeWCXvKJchW1wLq9vnU8DjSmgclLEjZqJTjBtG6Pk5jb7fDqJkewdJIw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(36756003)(6486002)(2906002)(86362001)(44832011)(508600001)(6666004)(6506007)(6512007)(5660300002)(8936002)(54906003)(316002)(186003)(6916009)(66476007)(66946007)(66556008)(4326008)(38100700002)(2616005)(107886003)(8676002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hgQBOZYtV8NuBjLhcTQe1ivIZoC5RTDbCm3JaFjhp7wW/ljNLSh3aWtPBveaMRoEVv+hNL2uDicCYxtlFf0wrggAbX1tEgVkeydFp1FVTguOfVWuxSDqZTbqN54pEvT81ohVFG57+fh4e7pEPLoEHLbdGNm+UQfxjXPPWXWJ7/aG0ZTZd/TntLJgaLhjnC4S47WK9Elh56jEEbW5oDdl/7fnaZCXusPNzS0QTh3hKFh1dv8vqYBAf05QaKGywY/EdB2zRjXhUZfErVfC6hXvOSMnnlibX+s1v6hpZNsR4n7E4b0ql/+nq/3j/IDku4JPjDnJaCg0p/FNzPUh0ldbuzkLRvnkUeldn559NWRH1RJBfAL++e5BsWVE8qhPfLYzWCvrtguZdveZ1cM1F7U03glRoWHqluGQR50g52gVJh7Xl+iQYwV9uhRom13NYbT3DcyijnXCYYsGB8SQiuRbFjjUySDpKUOUKDnREYd+5MYaWYDyabadRyfQ33AEzo42CjAJBsjOh/eg7b7HUgBygNK1c2BZ98SWIluACHIwQ5RKHnQ6hJqEAQVMMfYxlXbPXw9cKbAAlkkLPP1qONSpjwpGFLgH8sY0adbyByFJYNhN3bVi0+ldM2ODrfwS/5JdPe7CScosPFW6MnrBpHi7pMGSe9GeK2Xf4/dUUrakWtnRVjzanzv2TUCzwIKRgx+OTY3WNAk0KcvJ3fUnpzPoyVFYOEPYvTv6vjK/qr7xcEr1fmREqKGTsLb7w7xjJz4JeyFaErnV03TxdQqSB6/JMEEDU/830+/hvtidyAaWxnhgSEnKSJEhP1tyHulaclnV/XM92iQxpP41WK5Y4qmgPA0wgworYfm1NTCzprd/W8K+ofYlphBFZjnDZFfB/vBrrT6XXXgOawouUfOECglkkTD8HtV1cdXMCJ4D/DtjdziWP1jXoROgQSUUB+I5nLWmt4jDLUlPhmdiqmrPz1r48W1wiCBAaOR5jTRgsa441urFeyvopd4yjjXuJVTYuGGfeAHuqSMHKx6+FD/vi30Om5vPL+yWoQUiSHZZKhixHQMNvv5j1pgZLer/unQaBYmu3arlX0nyDOvBWVVhfUe2tz9aSjE1nf7UNcVVLQpM3YFSa0Ih6yUeWY6lLkOazGJqW2Ddv010TZvDmkgFgrc18bVhALBey+UBY4Qz3WRoYgbpPNOjTs/LP6bQEFViKRni8E2lGQ+uYpTGrQCpkgWD9JZlZAF2rbq0GwQmJVCNtAETKRZyRG5KHlILzZ/EhRJ8p6S5FDiUii9a6YmV84pGiDIdfSdnucg2tHixQ5BchD84Q6jSCIhhYRrxlap6qjjAOf7MZovRNGUmrc4oxkzuOA+37utnb1/zLCS4jqg22e08Gmtwhx6vg8W0TSPkxBkRgJtWyZRNRr8kCfownQzxq8LkccpZuUknoFhnz1QhLFbbFzpcOinXKkHbZGULrTNgdxaX0fiq5psMKl+bK3O9VaNGzyLnWDz5MiHirDZ7w4wi8PQSTgq8LhKjsQ4vSilv8NGKNXi9AnW+uqu+OGGvmuJA7oQhaOQB5Na2iQ6Cr9p6Nru7NuTc3/XpbhRR5vLFNs55OHD51bSOGIBSvuCuJcbVvrREy+PK84asXLT3m15xlXlSuwk7vv0p9eKqCFYak6M82c4+iK0APDHeWGtR64JuowzeHl4YNzMTVnv49K1Tg+CmQXz0h19d0/zMvt3xMXXViLsruFh2309pM7xfph4XvHu4NjG4bQZfLHD1cfA/GS2yqct+e5GNpebjzDPL X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 99cfbcb6-e2e4-46c8-da33-08da1e6657d6 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:13.8041 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FIycPGLST8T9IOApfQeCUC88JqzJfUonxBlw/mNEdYyXUhrEFVGqibzMbs1h352Tmat9UpKBqnK6UA+lMJL0oA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2094 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-GUID: K2JQ_mdeEdXTXu4XCSp-bRX2LlUsAVXm X-Proofpoint-ORIG-GUID: K2JQ_mdeEdXTXu4XCSp-bRX2LlUsAVXm Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 42c498c18a94eed79896c50871889af52fa0822e upstream When a user copy fails in one of the helpers of iomap_dio_rw, fail with -EFAULT instead of returning 0. This matches what iomap_dio_bio_actor returns when it gets an -EFAULT from bio_iov_iter_get_pages. With these changes, iomap_dio_actor now consistently fails with -EFAULT when a user page cannot be faulted in. Signed-off-by: Andreas Gruenbacher Reviewed-by: Darrick J. Wong Reviewed-by: Christoph Hellwig Signed-off-by: Anand Jain --- fs/iomap/direct-io.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 4ecd255e0511..a2a368e824c0 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -371,6 +371,8 @@ static loff_t iomap_dio_hole_iter(const struct iomap_iter *iter, loff_t length = iov_iter_zero(iomap_length(iter), dio->submit.iter); dio->size += length; + if (!length) + return -EFAULT; return length; } @@ -402,6 +404,8 @@ static loff_t iomap_dio_inline_iter(const struct iomap_iter *iomi, copied = copy_to_iter(inline_data, length, iter); } dio->size += copied; + if (!copied) + return -EFAULT; return copied; } From patchwork Thu Apr 14 22:28:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B994DC4321E for ; Thu, 14 Apr 2022 22:30:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347106AbiDNWc6 (ORCPT ); Thu, 14 Apr 2022 18:32:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347202AbiDNWcy (ORCPT ); Thu, 14 Apr 2022 18:32:54 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EEF1C4E33; Thu, 14 Apr 2022 15:30:28 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EK44qo029741; Thu, 14 Apr 2022 22:30:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=OJHiOrVzOQHcjM9MuaDqOrx6aWEc8l2qZYk1hIKwVoo=; b=AVVagtZxgYpvCZay2tGGWltX0SUr0YNvzWQvpKamM2iZF5i5mlkAMHqrz/PL68U2pyWN Ry2Ckjiv7rT77Jt522hhGz0aiipiBQD1hh/3cuiRD+phSgQ/rHvP2A5W+mWiqqGioLUa iHOTg2M02N8vmvGm7ywk3JlohIp18Y7mHeG3IMa4ZGLRmPlmjQ/ZFKnkZTLu1rFlveh6 9eGlP8Wu0VZB2uj9q7XZikLdhbP53Ap947tDFiX2Ehai9g0pZLl7+OucbXm4wPoJ3Qs+ a5XvYE17nYpB1f8D7MK5IluIwt6iI9BiHEjk6rPef84XIKiprdKQbmAx/u8AZtBq8m7X 6A== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jddp2y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:23 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGUvm009301; Thu, 14 Apr 2022 22:30:22 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2040.outbound.protection.outlook.com [104.47.56.40]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fck15da3s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bJzUB/MN08NWsVb0qjXChQF0jxfq0H30tz/bBS1n9qcjrgj3JDcg6D8H0iVd9WWuGig9OUALpl8QVdCV0e2YtmbsIEhJVWgAe8NrdQbefUIldraH/AUTGBkUbZe3gRxdsHMk8FwCtiE+w4IsU7QV0MkpaXzMh/wY2zwIXZX9GAbPJQUouhJ1PckYydvlDgql8YsjvFTMkAKIFMpH04ltNoonLrW65Zipz8591EQQao/PmWX4i2vZyylnT2ULHbRc3E+wcue9hlAudcJRCJ6qb0mjrmPaqzBmg5lnDcU+qtmvbccIMnprdBVxWaPVU76h11YsoQKHUiKkb0h6OrvkSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OJHiOrVzOQHcjM9MuaDqOrx6aWEc8l2qZYk1hIKwVoo=; b=nSXiiDYJ24heFgLekEBpPdmZQCvPfzLIqflOEWROQo43hwGsqZq7TDAVg6XFIVpf2S140G8QwadySCCJI0F4+gpmIs55D6CPej4SJ3hpupld+ioBDcUK3iAI5FVLi/7C5YZQanTmon0oLzkrkOwQD7dDMx43KHq1ywxaL/8gfWpU4F4nrzUoAF+FhuY6+udYQ6TG1XYa6IyTPES7yJTJKAwviB2cr7MVkSIlexyaiMYF+kHQ0WjsUyVTbTXxYhVdPUZ31TcLkhRDkztf4cHY9g5PnxZd64cLx7gz+dhLGbAWb9ehoIx3fBSLQzjc649hdgaIHvLz2ewyZumq7V7+JA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OJHiOrVzOQHcjM9MuaDqOrx6aWEc8l2qZYk1hIKwVoo=; b=O3aqXKBJSqEcZBA4TsTGOkp2bkh6d1GsE+krTwKSQp2JyPJvPbthZQynQMeY+D/6fc6ABqd512ez1luttTD7xAGtfKQk4Ja1FJUlWdsI666t9WOPC2ogGdYim2fJ7guRxfKVniSPxeJdDoREZLjAuY4ms5PIKlh20OppdNmg6QQ= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:21 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:20 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , "Darrick J . Wong" , Anand Jain Subject: [PATCH v3 11/18 stable-5.15.y] iomap: Support partial direct I/O on user copy failures Date: Fri, 15 Apr 2022 06:28:49 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR02CA0044.apcprd02.prod.outlook.com (2603:1096:4:196::17) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 78e04562-8438-4a44-d9ca-08da1e665c1c X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a71mW2mSl4wuUa3gK/uayXJDQkVFBUtrSmugZ75dFghAyuBGkFhVmnA4E6DxYjBwxTqy7PYrUBL9ppHD+ABBQVrvbqKHquPQlOs8F5IWwc+/LAEQqZNNp8udHVYG0DZxHlkfopyTk31tfaD7Bxr6uiU9dPS1gOZsmUAKQjufvtAZUwD+ygR1/gzgo+Fq7q66Iih6kmgxfy+RSRkmzLjhkdDrwrORQUt0LjKkS3rm8N9+Mj/St+IVuO+1ZB7UdXCRTClueb29KI46vzbI02PFec+iGC4FZ30QvS+MsDj3u8cfm9YlnFAUIjEc6YHWVDUGTVyJB+mzQRDvSqxnXEiJ61IF4IYxwUqyJc582kFuQHkWwd6C5yCXMDn45U7c/zEtFOy6y9niAkDAyiAKFrLcMkH62/1bGqksT9hRIGLLZa+3uXgdc8m8IbXkmNSO/LzBTWjW27Dyvs9hBIEMjybmHzJOpS3lg1LyC0UTtlJ1Y0XXgY17vb1kCpr368I4BCWtYkpvY2M6Fi7B2GWi6HwYUHRsEhG8/UZY1vtXV8SQ6dmV3LMoDmaiGcUkL0Qlozf4PRYCHf2ALvRTLIDrng1oXmG9YHa6yrjQfpCoLsTvUMrF3HdQz8g+78yyPEUv1yYVfoftoa9ddRh/XasMdfY6wQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ustVVGslml6YVDs5x84d8v5Bp6EOxh22JyE5rbsJZkEJ1kh96cFQQFkEoKevNcM8I5u8bDkF0NUv87bpsWt7u0TyfyRYj9/plTinQrT4ecO0DAA3YeXfX4o1o3nZZzSJMv77CYEEGzh+DTw8izLW6DWHVBf2pW/2jM+0qgOFa3B2ncCtMTeymSsFWh1DA/5GoW4zdnnFkbGgHcZm7PeaJHPqoydC7SwxoOQEqAIWFwL3UxzybHo3FoacqAyIM/1h1i9hfmBvIlJ5hYVfIGgitvybu7V8cTCZ30jKHnhbt13W7cFIHaxEJDUNG0P8JgCMft1XNGArQE181dn2tYJvrJ4s5xRM37SquW4DfdecMW4RTX+UKh9YHZU4V8UzyyZ0QuNyqLitr6vxBNLQWqqr1py+iOvkboXKXwcexAA11Xyv0fnq1On0NnyaVel3Yq2DmXnw94H+Z5wrpvDwFjZprXWBuQjVIGfxcP0O1DNrBUVQVSkQEjT3lYTwknaY+3LAP8yeO/+XS8QLdH51pOTl9W55eYQHxrr10hCOTvvXTj9A7cDrEsyZK1h4qvE9c76waB6hIKmTp3kDTX+b1DpqyqoOwhce8u4GnSGbbG/mK/4eLvEl8AK3Oi/CkUcja0jPbJYizG45tx/SDjmjt2kMtJM9zSc9nMgkZCh4PEDIR7UwL01qBfg/ONQVEw0sMchYiPMh5jhUPJac4CVKQaVSwGMt14qsU8ek3K+5zqHvsDM/KnZWRlJ43E0QtSEkxT3jUjoVU+73uM+/oByaf/V8Uc5k0YUpMtI0U/OoBnkQeMC/91lVk08LvGIlheTIke6Pq4EkjgHg0S03aPsQJP8FpcIsCq8wjbf0dvVYYOj73/5V+Plg6Sv/CdZpox5+FU5isFvyXGquSESt7q/8LVYArn69RIobrWbqzxxiLej0hJpdg2YEhumAE0OXNktmPXA3TC5v/tLHWSsoUbO9miNbNXnAopI9lhkPf1jhouNFOO6kTSsotv7PaTQP5om7QLVvqPgZHM1Yr57o5xeTs8+IZa/f7aQ+rcpBjWWOBJLlgcT/P6FeX/qhZQFT9lCcoDjQdE1cQ5FvUg4+2BIBYTO+6DglC0EcK5VZigDkrpbyFe6oebV8PR9mbmFE4+kdLFwo7IGET7wJ+UWkTe2VZXIxhrpZV7u4/ABxNbY/H58ntaFHterjB4aiYjASV1xg8rhW8F3KbRzNFNCgry24TouSiXvZEuLsfr5K6jtp/LWMy21V3Q5YueMwIxdyowylnHAopgfR5i9zmMzEkiqxq/SzeBPSNFGtRDJR18kd3G6QIi7zybB18BPSvp9OIdwGkjGPfVE+jeIGnU7liu6HXwJLCYecTmeZI5qrOU91/gZGz4yCrE5BZUSh2aqG36E6qCEnKbH42mqtKGdXn4AnZefCoCA+cXsL+fyut3UqOT1PbVVhfK5D0zxd6FbBp31znX8NGqtwHiKeiCGzOeajZ3knowc6U75Q+cGffyGcbViKf9Kpxs5IpwlNA6kWLNDG5OeZDlOwkJMHOrvGDi9C3aDjRlQqNdpU+ZSQp1U8eXNKdOHTB1Mz3wqC+mrKoBquTpvJXFPvPTJRQGxzlt42dTmUoyWZ8BzaHuSsLWSBDNqE4xV2Qy1KEp6WolklbDUVl3QAlgnQVl94Ih1lX2P8UjXCAuvnQcLeWHp5SmRD28pMXacPtdaq7XNi17hUckCAmCWcXyevA991Jgu2DvoY/1rASTipu7+LWNDQXPFOTALaeNX37RivoTGNobep8blAbDfn X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 78e04562-8438-4a44-d9ca-08da1e665c1c X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:20.8478 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3TZscD96Vsvo4R8OLiWoAYHODYhsthS63pzw3h/KcKZJyVety5WDv3T7fDedoC/K1B3xLnN6oQJhnVN0BS/dOA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: XxKSxNkpgKfWXNFtoeC3mRvPGiPKm00o X-Proofpoint-GUID: XxKSxNkpgKfWXNFtoeC3mRvPGiPKm00o Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 97308f8b0d867e9ef59528cd97f0db55ffdf5651 upstream In iomap_dio_rw, when iomap_apply returns an -EFAULT error and the IOMAP_DIO_PARTIAL flag is set, complete the request synchronously and return a partial result. This allows the caller to deal with the page fault and retry the remainder of the request. Signed-off-by: Andreas Gruenbacher Reviewed-by: Darrick J. Wong Signed-off-by: Anand Jain --- fs/iomap/direct-io.c | 6 ++++++ include/linux/iomap.h | 7 +++++++ 2 files changed, 13 insertions(+) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index a2a368e824c0..a434fb7887b2 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -581,6 +581,12 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, if (iov_iter_rw(iter) == READ && iomi.pos >= dio->i_size) iov_iter_revert(iter, iomi.pos - dio->i_size); + if (ret == -EFAULT && dio->size && (dio_flags & IOMAP_DIO_PARTIAL)) { + if (!(iocb->ki_flags & IOCB_NOWAIT)) + wait_for_completion = true; + ret = 0; + } + /* magic error code to fall back to buffered I/O */ if (ret == -ENOTBLK) { wait_for_completion = true; diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 24f8489583ca..2a213b0d1e1f 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -330,6 +330,13 @@ struct iomap_dio_ops { */ #define IOMAP_DIO_OVERWRITE_ONLY (1 << 1) +/* + * When a page fault occurs, return a partial synchronous result and allow + * the caller to retry the rest of the operation after dealing with the page + * fault. + */ +#define IOMAP_DIO_PARTIAL (1 << 2) + ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, unsigned int dio_flags); From patchwork Thu Apr 14 22:28:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24BDEC433EF for ; Thu, 14 Apr 2022 22:30:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343983AbiDNWdI (ORCPT ); Thu, 14 Apr 2022 18:33:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347118AbiDNWdC (ORCPT ); Thu, 14 Apr 2022 18:33:02 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C018C5592; Thu, 14 Apr 2022 15:30:35 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKkvG3006846; Thu, 14 Apr 2022 22:30:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=UbS/2mgOvDRVrL6smuNPjlVMKvZ528SrT492wM8M+zRy1rVg/S9X+WIY0JV6rE8Xfk2r NLy+645gdWljmghFeMwL8HU8x8+emekWuATnAknHkcrABnxc80SDXM03Nq0boLU8S70n qoB8L9zFZeodRLcUkI8wE+5pmydrWkpNol83R9vkdOWmzrx59jaaxOs/IQF4nI8ReNTB NPcAL28oELZlLrMeW8Wae0WVScq6P+WvsW+31dIQp/iPxyUlX1VKBh5a3miQ7BHcGNDz OvXYpQpTg4dduOXDmM1rHfFj6Tgg0fpSGqBOvlLqE6f+yXXxwfsW+Alg1lGHmAHt2064 hg== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb1rse6x3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:30 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMHWLV014862; Thu, 14 Apr 2022 22:30:30 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2043.outbound.protection.outlook.com [104.47.56.43]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5ucra-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:29 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mBw3EiXaOvBv/lUHZeHGG+LoVMiU+G4fkJcqbIn9/36FXIl9i1ZFzGzwUT0nOs1RThnI0SDZ9vj3aMUFi0Hw5e811RTTvaewQIairF3s96D2Zf1yIlhXNVToKZ9NQvIt3GRc/QLrst1mWF/rY9nYzf6axJDWzjdLvH5pm7FAqsdvCNGLGMVYUnZ4K+pgv4U9Bym5JZhZErJLJ5pFxRNHNCb60sWoyB9sGix9cFfVq4Y38guRaUUX0q3WIvF56rBqT959LwiDPQeJWRtoxMGNtk9ZawPord4nyIUftNZMypnIEoEDgnmUFo/bwNnW85JCdmQVQ/sT8fIfVJeqpn8PuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=Q+KGN9sxegC/pNMtTTA/VMV+X1wpg6IZ9WTro3qX4usL2sbFF4VNFkhyXFzeGJfctzIQ3wNevoTG5zwe12nN0OROoNqJVzsphATy63hM+sgjhywvzXBp6GeqMBWTNNBob+ZGwmR3Qhw7ggEkDQFHA9rRB7T8Y6QlxopNejwLcWj7UWlHr4IG1bwpEokfBvVWH1nZrIxviUBSssibeY+ayH1OMR26muet08TXQwwZ6PilyVoJQePoyYfbP6tSPeCNpN9sIXDXZvjQ8imNMh+/8f8kInJ3SM7Yp24UAJ6JGI/48wb6MhACXlR2IKmqrVyMwdCjTCxEo7UIu37Py6IK9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fniwVvOEnjVKx9xl+H1/NiS+KSAJHHGhJgiKJNQ7Tk8=; b=p4gGZYwczOUyU2wlDAcdC+21WxYlBtJW0uVqrGksQkiEJ3JHR9PqzgdmLUjhcVA80I5jS/Z6SZUPxSXUGIsuxSlZ0vMDDDNnYOGNoSZdei1wiSI8Ha9Z+yruLUhMrSCb3KLw/B15WTZEZ+iSMdCmlhd4JQy6OamTvNqyOmjp+s8= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:28 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:28 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , "Darrick J . Wong" , Anand Jain Subject: [PATCH v3 12/18 stable-5.15.y] iomap: Add done_before argument to iomap_dio_rw Date: Fri, 15 Apr 2022 06:28:50 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR01CA0005.apcprd01.prod.exchangelabs.com (2603:1096:4:191::23) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 034656de-ff7f-4dc5-8777-08da1e666057 X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BTHWrrT5YXaLiZml9PMyn1Hsv0S3WGCO6wKFu9amBc/97Kb+B/asUMPHCXQKZvUA7NgaWR30Uyx8s5YkamqTKe6mhYyRXb9dBgKnG4UdxBjNHYmduEmrTJQ71lH49hJhsrZKSQEVrwr7diKR5kEDP9lvKwsX2Q/HA+qivXdbr5xf5r16hmQcB67fO/GP99zxw1apk58q2eIY16IpLkdwFoVs/aojyPJx8CYCxiYZM/2DtzWrNBTOc/pzxbGKOgAgdXoGpijEvkow0a+89L9UtCgWggKuMsmUo8vzg0tAKwpyOTv2TTBIpkSIx9Ls/Z6/Z4kZ8JnppyRZpIlAxWT6ADPKyopn3Z8QZmkgQlkp7pByQGKNxGskN60o5fJ7wr0GUmHmbrOzFLRZOUQSt/klella7RQS2ptWBO26W2pQVnKQuQlQ2rD4nmiwa8MK0RXJ4jEXWwsxvPypPR9deTotL34hpixY4orEVwW6NkgItIuivdSsSTnkA4ICIH+UjxkyxIw4dbTpTnGygulPbaLf9aKQR9yuFNE9sy+vD/gVXYWlPH4x1StsbvzeMXXuxoRUne/n032fxv0b2Fhl5qNLpPaoTf+BwCz6KyqU8oNX633K9FshRv4uLRqZ1wYSSN0VfoWZ9m265W1aWTFzU3KUUg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vd7UarMWpLzg+XaAcV6po8euJajA41T8IHzD8Kml49ItpfR9I80SsSXtT+PflZCHnHxkFmBSzn91YRDQTF2aFNyTCqMpuL2gsMMVyhjeKG5/rBSt1k92vopJZfMiTgFX8/TuuW2pwamfjFPTatzialZ1BIf0WtxO/2Cja0WohSiRrIzJ+5yAzXT5LpVgBEq9eb7tzLR7t8LTA6M6Ylv/ho7P5/CTy9e0Ztz74wtUzCHztJdZC66CCYd+fU9wPXwOV9r3e0/nrRqP8G1FV72CzP8BjRCfvIqrsyNuecfNkzBKW1wXxaZCcx97KDgnHwr6MnM3krNs/ltaCx03ggiaPSbWEymYJqOV0BDheezBM6kaFssvobX8B7OQkabrFw6BptsTqJAKzAPa3SUd7TrfMZ6VXy/K24XoQNMxISAXXlQI6bIMOxaT/qleUZLO1txaLyAVHNZ72vCjCNqNkXP79hrOio+TSRtW7LWHGl8EkpuSuj0JFKDoklQ2Pe4p8kc+92/ArZYSwZIB5mPkUSnk8s2d5N4kn1Q85SnM0z0mZHGC4jRsnjs//tZFWcldkSMNrXEG636BecN6/z88v4ymjSJ6PaM5Xrqohb9wwi0ywB5feK7KNTwshYYwjU0s335yDMEoM7Ffy8nrvODYRnmb9UcLC7/6AV6mV/0M4fRmy5WKu1hDhU07Irsh4FU6lWbFSARYvv2nDzxSlm6eYAV/bWqPzt7XVZKlBfbdMsYDy898Hz6isYK9DoMV7XqUOSmFF0mTEMcitrSPohJ4eV+xsrGBQBNFmJ6Q+0J4p3oTkXpD8bws1rpCUKe6O5PVhtn81OW57seM1gauD4X44iojwbD8Ms1QswRt91L3QYQMZxf0GXpXj6yYiKDo0ceN3/woUT97AuEuxXHzel5HhK2rIfgrP5umy8s1TuIfre4TqWIvYtFzwaX8a9txW1cKlFMIgbUE13silWH8+WUgQTWJHmB6yHRcTUn/a7tAWm51nh97btoVz9izEvwsv7N4jdyo2QsJmoZamDaSmgfQ4I7cufa3yR6z4m66OaszUKbc55YBF/hvJNoVrOEBJohzRT5MrYdZWH2McibfoUxaGcpDsGcxJ5TkIQmWWh2sgjjIaKAS8mrpOq3n4Uplz546QqVsg6d8Pks3xCQUEsvlbT8qVO1wWhQCiyoWwUrslJOQBeDs9MlmyH4RstM8qB8rdJUuSJFEo1fMZpDK6Z+iruyzeIVCTPKQlwMWPHhG/Jy4NL1CSk6YVTdHxye0kXAg6zT60BaWp3oax+Tj2fLsxaYUP25CgS2eU4nhI3BSIZFjTPqvckaiGHNpWRsUAtVb5rpwOlpqTM6JwXT6TphrEb+DJ4Wv/LcP2gKY5HkGCyHHqkaJvgopX8udoy8EgAnD4W9vD3CvlfkYZt1ixPgUUbeX246qzulTxlZuTTAlN/lWRtlOUnLtdSf5xpp/xQwG0MDyYyuttizdXc5aiYmsfnf4NXMlKHZ1rrKLqSVSNgxY7hXnyQdQ9DEdMGVNVTpKfAuhexsXPE9lYJf/CkhtXwMiWwGejes2qsP5nBKnGXf0EQtRbO+zUh8ecfkNqmkLm0/ug9Ecjd2iQ+PIx3Mb/euvILhG7LeVqcgL+53l6Xftpp59OeepTPATVIiFiLcwkNpZJpGDHJmtxedsKhmX1tEIJhjODGDw0Vy0tToQBpUVcpEskSzKGHdORRosys2b69CuieQs2BCerL1zrSus/FugSy3dIHVozvk3DXNvDchWqpyyEIlwzsu1ywARndbrpiOM X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 034656de-ff7f-4dc5-8777-08da1e666057 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:27.9442 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wk9Sm6X+IG9qa5DaQv1r0qTUrGv1AyzqaCf6UNUTjkucZGWqILc84huRhj9XG2UqSTPyyYIvT7xWrmFDUugTQA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: WcAyqotsr1UJYogj878wvWIZagMW-l90 X-Proofpoint-GUID: WcAyqotsr1UJYogj878wvWIZagMW-l90 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 4fdccaa0d184c202f98d73b24e3ec8eeee88ab8d upstream Add a done_before argument to iomap_dio_rw that indicates how much of the request has already been transferred. When the request succeeds, we report that done_before additional bytes were tranferred. This is useful for finishing a request asynchronously when part of the request has already been completed synchronously. We'll use that to allow iomap_dio_rw to be used with page faults disabled: when a page fault occurs while submitting a request, we synchronously complete the part of the request that has already been submitted. The caller can then take care of the page fault and call iomap_dio_rw again for the rest of the request, passing in the number of bytes already tranferred. Signed-off-by: Andreas Gruenbacher Reviewed-by: Darrick J. Wong Signed-off-by: Anand Jain --- fs/btrfs/file.c | 5 +++-- fs/erofs/data.c | 2 +- fs/ext4/file.c | 5 +++-- fs/gfs2/file.c | 4 ++-- fs/iomap/direct-io.c | 19 ++++++++++++++++--- fs/xfs/xfs_file.c | 6 +++--- fs/zonefs/super.c | 4 ++-- include/linux/iomap.h | 4 ++-- 8 files changed, 32 insertions(+), 17 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 5bf4304366e9..cd4950476366 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1956,7 +1956,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) } dio = __iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - 0); + 0, 0); btrfs_inode_unlock(inode, ilock_flags); @@ -3659,7 +3659,8 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) return 0; btrfs_inode_lock(inode, BTRFS_ILOCK_SHARED); - ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, 0); + ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, + 0, 0); btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); return ret; } diff --git a/fs/erofs/data.c b/fs/erofs/data.c index 9db829715652..16a41d0db55a 100644 --- a/fs/erofs/data.c +++ b/fs/erofs/data.c @@ -287,7 +287,7 @@ static ssize_t erofs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) if (!err) return iomap_dio_rw(iocb, to, &erofs_iomap_ops, - NULL, 0); + NULL, 0, 0); if (err < 0) return err; } diff --git a/fs/ext4/file.c b/fs/ext4/file.c index ac0e11bbb445..b25c1f8f7c4f 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -74,7 +74,7 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) return generic_file_read_iter(iocb, to); } - ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, 0); inode_unlock_shared(inode); file_accessed(iocb->ki_filp); @@ -566,7 +566,8 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) if (ilock_shared) iomap_ops = &ext4_iomap_overwrite_ops; ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops, - (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0); + (unaligned_io || extend) ? IOMAP_DIO_FORCE_WAIT : 0, + 0); if (ret == -ENOTBLK) ret = 0; diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 2d0aa55205ed..81835d34d6f6 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -823,7 +823,7 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, if (ret) goto out_uninit; - ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0, 0); gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); @@ -857,7 +857,7 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, if (offset + len > i_size_read(&ip->i_inode)) goto out; - ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0, 0); if (ret == -ENOTBLK) ret = 0; out: diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index a434fb7887b2..468dcbba45bc 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -31,6 +31,7 @@ struct iomap_dio { atomic_t ref; unsigned flags; int error; + size_t done_before; bool wait_for_completion; union { @@ -124,6 +125,9 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio) if (ret > 0 && (dio->flags & IOMAP_DIO_NEED_SYNC)) ret = generic_write_sync(iocb, ret); + if (ret > 0) + ret += dio->done_before; + kfree(dio); return ret; @@ -450,13 +454,21 @@ static loff_t iomap_dio_iter(const struct iomap_iter *iter, * may be pure data writes. In that case, we still need to do a full data sync * completion. * + * When page faults are disabled and @dio_flags includes IOMAP_DIO_PARTIAL, + * __iomap_dio_rw can return a partial result if it encounters a non-resident + * page in @iter after preparing a transfer. In that case, the non-resident + * pages can be faulted in and the request resumed with @done_before set to the + * number of bytes previously transferred. The request will then complete with + * the correct total number of bytes transferred; this is essential for + * completing partial requests asynchronously. + * * Returns -ENOTBLK In case of a page invalidation invalidation failure for * writes. The callers needs to fall back to buffered I/O in this case. */ struct iomap_dio * __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags) + unsigned int dio_flags, size_t done_before) { struct address_space *mapping = iocb->ki_filp->f_mapping; struct inode *inode = file_inode(iocb->ki_filp); @@ -486,6 +498,7 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, dio->dops = dops; dio->error = 0; dio->flags = 0; + dio->done_before = done_before; dio->submit.iter = iter; dio->submit.waiter = current; @@ -652,11 +665,11 @@ EXPORT_SYMBOL_GPL(__iomap_dio_rw); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags) + unsigned int dio_flags, size_t done_before) { struct iomap_dio *dio; - dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags); + dio = __iomap_dio_rw(iocb, iter, ops, dops, dio_flags, done_before); if (IS_ERR_OR_NULL(dio)) return PTR_ERR_OR_ZERO(dio); return iomap_dio_complete(dio); diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 7aa943edfc02..240eb932c014 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -259,7 +259,7 @@ xfs_file_dio_read( ret = xfs_ilock_iocb(iocb, XFS_IOLOCK_SHARED); if (ret) return ret; - ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0); + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, 0, 0); xfs_iunlock(ip, XFS_IOLOCK_SHARED); return ret; @@ -569,7 +569,7 @@ xfs_file_dio_write_aligned( } trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, 0); + &xfs_dio_write_ops, 0, 0); out_unlock: if (iolock) xfs_iunlock(ip, iolock); @@ -647,7 +647,7 @@ xfs_file_dio_write_unaligned( trace_xfs_file_direct_write(iocb, from); ret = iomap_dio_rw(iocb, from, &xfs_direct_write_iomap_ops, - &xfs_dio_write_ops, flags); + &xfs_dio_write_ops, flags, 0); /* * Retry unaligned I/O with exclusive blocking semantics if the DIO diff --git a/fs/zonefs/super.c b/fs/zonefs/super.c index 807f33553a8e..bced33b76bea 100644 --- a/fs/zonefs/super.c +++ b/fs/zonefs/super.c @@ -852,7 +852,7 @@ static ssize_t zonefs_file_dio_write(struct kiocb *iocb, struct iov_iter *from) ret = zonefs_file_dio_append(iocb, from); else ret = iomap_dio_rw(iocb, from, &zonefs_iomap_ops, - &zonefs_write_dio_ops, 0); + &zonefs_write_dio_ops, 0, 0); if (zi->i_ztype == ZONEFS_ZTYPE_SEQ && (ret > 0 || ret == -EIOCBQUEUED)) { if (ret > 0) @@ -987,7 +987,7 @@ static ssize_t zonefs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) } file_accessed(iocb->ki_filp); ret = iomap_dio_rw(iocb, to, &zonefs_iomap_ops, - &zonefs_read_dio_ops, 0); + &zonefs_read_dio_ops, 0, 0); } else { ret = generic_file_read_iter(iocb, to); if (ret == -EIO) diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 2a213b0d1e1f..829f2325ecba 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -339,10 +339,10 @@ struct iomap_dio_ops { ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags); + unsigned int dio_flags, size_t done_before); struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, - unsigned int dio_flags); + unsigned int dio_flags, size_t done_before); ssize_t iomap_dio_complete(struct iomap_dio *dio); int iomap_dio_iopoll(struct kiocb *kiocb, bool spin); From patchwork Thu Apr 14 22:28:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09B4DC4332F for ; Thu, 14 Apr 2022 22:30:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347150AbiDNWdJ (ORCPT ); Thu, 14 Apr 2022 18:33:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347164AbiDNWdH (ORCPT ); Thu, 14 Apr 2022 18:33:07 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E9AFC558B; Thu, 14 Apr 2022 15:30:41 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJ4f0n018415; Thu, 14 Apr 2022 22:30:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=rM2OYNDnjOSXgpIxpKd8yTK/tasdQ+W01Fq39WF5u3k=; b=behEU2BLdDYegnjQA1SJyOFSUoyX/y80OLaEr+WPh10ZYYs16caCWuq0FE2YPGvhqXBO cMmaybros75qUZu6h32eTfBCCc8ii+tcIXDNAqzxXRKhrgOE3Dw+3QpYdeQ0sr8KVnbl MTRK9tLU7nmhtVoAIQpq+tmZMLGft7HBgQRjOtIVZ9UZJtHXgSeptDIIDOeSdtdk2QId ufbQqiDwPzevofbZsMReLtA6TvvjBv22KJekG56vTuCYvS7XjpPwWn2KbyZiUCtjk6ih krYfQOe5s+jm5QrX887/V7ug6pBy6byW8kt0IQgKjaYR6aQSacvmS2Yo477gxyhqcSii lA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0r1p63u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:39 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMHPqu020912; Thu, 14 Apr 2022 22:30:37 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2046.outbound.protection.outlook.com [104.47.56.46]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fb0k57g2j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XEOrzr46rfYgfmpTVdDAtn/LdKMe2D4vgiNlsEhSx8xw7nD3KtbYOgap/ad2QDezw0aA7gtmM4rZpnhLMDo9psIMqDB8lzOOBC1b6y0v4VSYZ5arEvhvrrUpBYpu8pI8EFrHh1Q5cVolW3W/J+kAkbpLuvFH7GN917qr9SvkNaa/LHZ9nC/tAZFQyXOzRkFmkrg5hXgZRFWpZJIsJ7vOcJzpMQ0N/gtGyq4n20nHFH2RfmpIW/O/EDm4MRsmk+wjwEbUk10Yjii4FyyesFwjuFhG9lJhxccbqclogCxSD3RPxIjaeaNUo04kzVnAT+iUp655lD+B3+9D5ZJ3U9J2HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rM2OYNDnjOSXgpIxpKd8yTK/tasdQ+W01Fq39WF5u3k=; b=SN+zmnMdAxW0yNYAsKT2Tv7OYZEiPqG0Vek9p2KkNroSyvyoUscxp/9BzDCkctnRnkxWx5GaG9rFOoJELSacoaL5FW2UVWcweyFL1iCEsK58WqGGLwjxMoCEend11MSe8egQMquo4htNacScWrtwva3OVAKnuYcmIdVAy7ay9j/9Un1Vqkf8BrucJA25H5eSJzDwqZwI6vqzu2NpnB5Emm3ku1pPOdip/edOqBrjDjbtfkk1qDk+oYxjlDX25ZThyDyJeWgNrvcyR2SxXRZu7AbIFkIoNv7dxvn6ljNqm5RsI9DovoNVFJ+jsPGKmOLBnx/ZYyqHAa7hm0sRQfWyqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rM2OYNDnjOSXgpIxpKd8yTK/tasdQ+W01Fq39WF5u3k=; b=lrUVsQpHeY35+cXtCFbs1cd3fgiV4tmWCACMgPwUn3qQEMjiC+2fQIh7jyA7AduzYsF4ZHJ6B84aEHA8iQUIiDBZSQTzI9CbmoiycuBuA8JdhXT/W4gInVPyH4Nh12NyupFMBbTjo3l1UhCF2+ImraOIoqCMG+IFz/7kaZDpZqA= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:36 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:36 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 13/18 stable-5.15.y] gup: Introduce FOLL_NOFAULT flag to disable page faults Date: Fri, 15 Apr 2022 06:28:51 +0800 Message-Id: <2ee1e383ae1cca975426b54ab251257f6d4e12c0.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR06CA0018.apcprd06.prod.outlook.com (2603:1096:4:186::8) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ee23dbdd-5a1e-48ca-729c-08da1e666521 X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SIZ06lEf9beV2fRBqOgnanZHn8unUL6UFGY/LQ8ICFvwGEwgx5YB51UFjLQxq4mSN85FiPwnQ8kttvWb5F8GH38SdoZcKdPWcqaWITZoLatS+P54kKplDwF6NZSoYm1P0Rz2sQBj/9KJ10V+X5CrOW+f8PBDqoAZdfmTnErB98H5FNfeNteIqWeDE9B8+oHSlJ2AX3QI/QR6q4TH+/tlF3BDrcOghYDNH8I4abFlKLxFY+0SQSOrgJ3nGZW8RTiSkc0RCnWh7OUekKcLIofKHgRjqroB1EKoYcfROVSK6Twji/K5hTm7wLo+Ho/7rTANgDsxqakAjtlh8qQ/7/1uGOqtfwr8WVhZzX1UUE4ExQZt29xF4e5kPuId2LoHy8AaCjqIO1nLE3BwXvUf6qc9WAsQJxuOSAwnI2I8TgoVLlqxlrrzVDTnFDECLu1P29SdCY05U7vDs+wPOG40+LtNoQiSz4m2BMIjAOc1PRszzci6jjh1k7tEXXnAmvpFUkZTaMptRbf5NQYJrQ4JD64zNjZlGU297QZrkf2agwUXc+d7fJUJR72zm3gjlGQCR7QNusFecPRuHmbHbcyZwKM57xWQ2DX48VY+C//7HInQbfXTXBacTxIO7AXmRGzC5ZCD8Z/BkaH0q39AghFLMQZFiw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mzFpJ9j9KHzindn5ZYICNv6LKvC+2Of5feDoi1WQZIzzX3gw4E1INX8D+WCGcoUmZad/JWJuf/MKEb4KxwHzPVyTAIejh3vPdbfeiY8BXKCBGOjDKbFvZPMuR+yx+dt2BHdE6V33Ldj6kmJheL4z65A7FdFsmdb0iY/rPwEIvTtTikbj1XrLwZcVr06tYBBl4UaBU1T24Km4dF29gaHUrmiR8Dqpcx2NYtf8un3m/JHBaHHXliJIai8VdKyjHf8Cvw9g7guTCdo+P3H9CW8yvUUGjIKTcy2gWanylaE6FTOZggWllMQCyWlJ+dbfV6TQJoOaR/F+xl7FCYDy1ZbgPNGxH5NkJA7yoRwSFPZxL6AYbBV0REBYb22ZxmUUOODSEqDhtD/S3xPCcXPtgBJt4Ff3Fwa5njF4Z7sSRFBNGVD+tKoGOo4jha4nWJSmPl5zYw9QgHn7i5gHuklyL/Mu9NTyuehxQQHYlv6UfA1Rijd8Z6s+p8xTqiEp9WLoMvyUlqYS2+Wx6z1OkwnlzaUNdUAyxTjiTSC1/NAbhPazleBpPrgx6WBgSMhNGnwC6meSM8Ub/ALIIcd81UIuogfrBeXgY3VRsSue35YJqDM9RjxMghctwcPxTmyRqwKwSv/WGCtKUF+s8U3c+ueXEb+9pD0IQrJeBuNQcwqD6IGq0P8MazTupc8bj3ElbVkKYvU5QqaXuEgpUgsUeZm8drXiBMC5tCSOVWC8k7Ef2R0oukH+9oHAJx8GBMLGEsLirYvmPOT4H/TPG7gtWvFrgtUoCFekwGeyVnRYU5TMTKx12XdXbXnOG8WZXrc2OseL9tuZvDvepyzWtM0OiTDyeyqwo5C6TN5E0k01ZFqwv4rsmZNFNeREmt2Mxir2YJqcglf2CQaP+ABshslJevuLKIk8xIHRjH12AWuJQCAWUHM7fuL+sQX7irNAy6hp5ED864BJWRW/Fsu1wD7UvhmwPIMyeu4dMMYvhITTMj1OJr6GZ+4h8ZNOW1LKE8wduk/CIQo689WJ0CYZ1sW86R0kQ2X79GpWgc4RZjOrcqml1ww7X7b7v+Dngluf/wrShaFCoqU/3jmDIbSgi1zdFqhRZyy7mSII8rWiMNDYdyIar4Jjz/+Tm8ba6bTpCyHkg9l9xrMEyfY0WM/ajfTpu1A/q+46yCO6q6mNXhio9ECV86mo0n6b9e+ZnmuBev9f9IPEnu9QHdA5iG7rW5V/fmByOCzzXdy+rE11kOcTDlrWyYd/Yl6efCLoE2LvFTDDh2r7VR8XUXuY7dZyGr2VkCIYKVf4p17uv7y+f9L/AsPFlgjRzuj+HZS3e/h9ffdHtAiXVbQe6HuGN+mBpsRbQGzT3664UlKusPeELlOWU8FiBk4NxaeM7jb/G39oTJUybQxCsAunu4BC1JhjoW/XEq2+0/lcLFybtRJ97x/Bo+TIB/gh0XZCHJMkgnZcWzCgtt05Ta9cfDCDvKXpfwRYZ8Wuq1Fp+SAnqdpNccRxZmgf97WbSApidLz5rzGHyAeQrD/OFEt0BfmY58Nc9j394xad4eVx4mRr+qr+DVp9fciP5OT8HXGDdg0alfrxQWM8A1dRdlb99UcVteD31fBj//icRl7mUI+E0TXdo/ZMjXQQpqiLghtywUKvhHKkeggfoXjfXZ9JJr08lXzNNvo+z0sLfi2tPG8DBaC4XTJvaTo4F5JrSZbpMTtnZ+8T5Vdx6+nHKJXuYbngjWCDlQUpeXov39jg7wSks6Umr9oL5E3mOIjL3+LvgOrGsFaxYaBa3/VLz+Ww X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ee23dbdd-5a1e-48ca-729c-08da1e666521 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:35.9628 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RaWtCsSYVAuEI+fu5MoQ+BBWJJYywu0aXIU710mqwzJUJfZbjuxS1v9xXTGePGD498JD76b5ddrj21R7dYELLg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-GUID: JCfclztuLK2th8LhaJczUXT3XIi7C5It X-Proofpoint-ORIG-GUID: JCfclztuLK2th8LhaJczUXT3XIi7C5It Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 55b8fe703bc51200d4698596c90813453b35ae63 upstream Introduce a new FOLL_NOFAULT flag that causes get_user_pages to return -EFAULT when it would otherwise trigger a page fault. This is roughly similar to FOLL_FAST_ONLY but available on all architectures, and less fragile. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- include/linux/mm.h | 3 ++- mm/gup.c | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 90c2d7f3c7a8..04345ff97f8c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2858,7 +2858,8 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_FORCE 0x10 /* get_user_pages read/write w/o permission */ #define FOLL_NOWAIT 0x20 /* if a disk transfer is needed, start the IO * and return without waiting upon it */ -#define FOLL_POPULATE 0x40 /* fault in page */ +#define FOLL_POPULATE 0x40 /* fault in pages (with FOLL_MLOCK) */ +#define FOLL_NOFAULT 0x80 /* do not fault in pages */ #define FOLL_HWPOISON 0x100 /* check page is hwpoisoned */ #define FOLL_NUMA 0x200 /* force NUMA hinting page fault */ #define FOLL_MIGRATION 0x400 /* wait for page to replace migration entry */ diff --git a/mm/gup.c b/mm/gup.c index bd53a5bb715d..a4c6affe6df3 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -943,6 +943,8 @@ static int faultin_page(struct vm_area_struct *vma, /* mlock all present pages, but do not fault in new pages */ if ((*flags & (FOLL_POPULATE | FOLL_MLOCK)) == FOLL_MLOCK) return -ENOENT; + if (*flags & FOLL_NOFAULT) + return -EFAULT; if (*flags & FOLL_WRITE) fault_flags |= FAULT_FLAG_WRITE; if (*flags & FOLL_REMOTE) @@ -2868,7 +2870,7 @@ static int internal_get_user_pages_fast(unsigned long start, if (WARN_ON_ONCE(gup_flags & ~(FOLL_WRITE | FOLL_LONGTERM | FOLL_FORCE | FOLL_PIN | FOLL_GET | - FOLL_FAST_ONLY))) + FOLL_FAST_ONLY | FOLL_NOFAULT))) return -EINVAL; if (gup_flags & FOLL_PIN) From patchwork Thu Apr 14 22:28:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8056C433F5 for ; Thu, 14 Apr 2022 22:30:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347118AbiDNWdP (ORCPT ); Thu, 14 Apr 2022 18:33:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60878 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239673AbiDNWdO (ORCPT ); Thu, 14 Apr 2022 18:33:14 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79461C4E36; Thu, 14 Apr 2022 15:30:48 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EIrgU7031505; Thu, 14 Apr 2022 22:30:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=DIlRl+J+Ng4MBRFCGYV+2tc36qeXG1xvV9Pmwp/eR4GHnobgWa5ZlPXi9+eicB3rROEZ Jq5i1SVg/dqc4pwHRji4iuEhozOpvVhQS4/4rG5oTZjysU//WB2UpLoxQvv+S/c5XJjM yxxwRuvErpFkvSkdrywvLGVPdJuw2wC4KOdQyMdK/xNj5C3KmT+OZcCdQbkB9aeBo6l2 AmD0O0D9opjnFMEl5ZsyKefEMUNGZMGaBilTQXSXThRUHR0ufh++/LlvThO3oKNJ6iC9 7rVfcEo3otquDXruNbhet4lde5/0GPyvaDLnnFi3Awy5fSI7FlMTRuPBrvz5rhE/gG5v 1A== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb1rse6xb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:46 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGOBa008378; Thu, 14 Apr 2022 22:30:45 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2040.outbound.protection.outlook.com [104.47.56.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9m6ktp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:45 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hqm6VSyVdVRNmTaLLQ4nXPEuE3L+5h07JDRPW+AwxihuWzGw3upJhQklGWqxIM1Mh7DZT3UujNt6g8Vv+KFpmQgMoM7aifFUfiBp3RKdYHaGXaxR+uzGI404d9z1q4xv2lIfsSOi7uRD1YrpS8wMZBf5gfSkxZuqeJiOIp3He0uCvIxcq9B9C17QZsCl3pXR4F3ag91QOZvjU9nFHGFva/jDLut7T2EtLGP9U+dC+CyK3pZ4LYfJCANoI3A5AhgUrYEapJbkuwvY3wwXP0ZN3pWCRv7Cuihp+SlprMqf1CNwjIBlh2ku+zHu4zuKcG0ySqEmQRfCWJ96yFfuFAGUrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=efGFlmuMtYnAYMEnrnQRTt46oAWyr9I+vXoaJUFh2fjkOUqMJGz8AFAg6y902/wQbX9POnO/mH5JZqO4YyVrcRPyASTWU3akQ886fZ0/ruhBOCl1UJ8edCCqgYS5KxYi/o9SX5ZrKAAgGWtRLbCy4G+jnCkLbb4E5i+ScreOVhjS+gPawNcCiQcCuuuWNhNIsNDCrnlQm2skdLdW2HxqFEN0TUGzG+IRPMYU4hxq6dU1n2CJNag1ofEeUJDmGmYGwrYThM8R1NTrB0gVYAoFyhTMMkFPH3ykG6qIrxaV0W0IkE4hFMidOSMtp/E64CPBDN+wiz8ppeDkHrSiYBH94A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9uzLC0RNS5lLuWjzCZuI4dVyBqgt1uGnSMgLjZYLMFE=; b=yqx5RJNENscXPAgq9y4Bc9m9PTydcWW+k3KFDzbma5n6GhC/lGFaCYeCHsc6XvbcUAJAa2PTVjcVsKUtyoOhye1RgWideyaAL9R9LcKoqMR5DepvhiDMxDoQJ75g5CuHXaxrJ0q2DL3RBhL6np4fy3/gePtw47h0iA0CG1P9cNE= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:43 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:43 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 14/18 stable-5.15.y] iov_iter: Introduce nofault flag to disable page faults Date: Fri, 15 Apr 2022 06:28:52 +0800 Message-Id: <56bf354a8e9c5f2d3d9482c90510d4ff0890d996.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: SI2PR04CA0010.apcprd04.prod.outlook.com (2603:1096:4:197::11) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 95032659-72b9-48fb-69c3-08da1e66699a X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5teGwCZcciu9NjAIagH4tp+u8NLU34Qhn4QJNE8qhnNbocPwGO+kxWsz2KQV3OHSIooa8E6+E5gc3TB7gigpqetOEbNNczOTfWR2wZkWVxi0j4YVrpgK6adrZ1iXN2GfRn3LRUKR3SLAHamEn8F7ql5elZCVfIznSaltXE7zyFz4qUyNaYjL/xi6EvtNG6gr65OqZ2AUYApXVipu8QOf4ClRPVoFJivKSn963ZMiX5lFxTaKOX0HpFDshH4ILn7CcrSlOPOf+R0+XuBgiMQxugBsQM7EVH5HFX+ZwAGt2pQPCRIBZ3/y7iO1xpBOEreD1ZF3+Eyokf2oLOORW8csgFpmvSDLuOUqnyQ48Yzc2HbmvI/xM1jUHhEfFIammBlCVyuFbQ02bDMliyTmkgbNH5dyuKujOWm5CAQUAGW4IaAqNcwblyDxYgWy6cmTGrAH1ANrYOUX1Kc+SrhZ7vpfNbua34VovlERBvqK3YR1CzDmjtecIzZcGuFgSRel+IjSvkremwNYlmLJNb7tZkvfQHh9Y5PziwjFGpHmpz1efuePGTF7PxrrJm43ISNABfp5vvlUe+jFoI35vWKRLVjglApvHY6KWgws4L9WSW5eMdrlD7CXDoZ8Ivwx/DhvIdboMXvc9ZO++2YaC91+mGZMRQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IfGQprZlSs/aNh8z2WbNOPcvktDAAfYn2WwUK8c4DrqW9ONgovZ5nAociOOxQSf2kCpYEklYiXoKzMsevNrI5IATNEZJKxzn4rOGXW1lNC3I2IR0pqc3Dqy/4bucij+pavExy0m5QqZLnCp3pl9QrAJn/Z3HTPty+/RjE4RJS1gxATpQGE2I0XbrA0nfb9CQRNvjJNRwKO+/tBLK33HyoRAHU7HVgjpDPZhBmcvqViz/m6Y2kaOqx/HhNruYdRM0Ii0n5qVeRe6AgbUPjWjsMmkoHAmuiojtboWVjoQxj2JUVpddfq2Pcmn6CtG+VYEozEgIlV6NP732AhskQndIDa5cKbRzJR6vUrtfBu8eNtRGdoOuu8d5klDFjJTqPpc+4vFgnV/+zwwNo+WiISmpUQJN0V1BJxA3BI5GCQCmLYAscgXMu6jy7cYfjLkGU4js8f5fmLDWbcnc/8+ePOE+EiAGBzT4iCLpKmrT85lCMr5aTizyue7oc0r+I+O+VmIvbkpkZWPaF/oN31gKj6ceKA8R5tLjyOvsTCLorkmdvPH9jRbBeIGX1PzH0RWSKR5dRTwr4tN7bVLD3GfZErVR7kwbzbCQnXigJ82RGcT1iNRBhD61W/cgyVjw30vtl1wIGIgWLcp6nAeC6SrE9Hjhwh0jjLSaw4RDiKJ8OhJ0mlqY5YOfP8lMRzZwatVTyXG4fIYOlXswoE6mjK9Vdo55U3+Td+XAD4WsHCNwqo8R+WxsmCl0u3WT1SOpE3HORdg6mFIrFWshUwM2fPbjF6q38gIY1Y4je1O+mOH78e6wE7sek/4w9Yw28iIAURnQXiVBToP/8jFeQtTaXv6HLYK1KLbSYKgpv2G0Vswyyr5Naf7IYFkGbgmiGiJMciylotxKv4B1fxLKyCwpMexZMoljfSFTO2BJDCcMcWZze4Nf3MFce8ypxRqcz1OdWwEBZKdqN+RQXI0WtrUOZWESQm4A65sArukNTY5Wu29WfMXFgUj3gXLyLZu9hL8GPVH822LtS9vfd1C6FV41dg1Iixr+aMQorxelHSEtVNggvlMvQMms00wJKOlqqkXhQM1ZyO91d27LyXRouhkBjGL6hVRX1xDlFemC04RzTUhJftHu/pD0JhEUTC5pqGiXpBc/EwQzHOIu5l0e8ajhnRIwU4E2i6V2IxhCaTvJiTs5HblbUc2cb9bx9pI/CCywY1x+4Ug+UtOXkyYRFdLyxHk7WDxpY17cH/iw8Qa5gwz82K5EcxPLWUuhOnvnkr3outo9n42fD8gj8a2MUli2/lajUojijiLvmxSBd1/FBj1b5o1zStaypeBTaSovKccxxTp7wcQvvumaGKHmM6BTRaeoO4OVcwQFKcCMJZQkKRvxPyfaScYh+prpkdckGQby6aV6KtqHGe7BLHRixNVDt6oxkAVwMiBTKqUFElwhrFhKr9hfgPyoqvfQJOEXdtO56whRncGJ8y1zuxQKWArio0pBnzLrYQN9+2oo5D9ylAWZJklBZXR2NNfd8eUgECiqTOWNpl2osyo33tnB9LFWOLoxHyEC5bgzhXJ261vLtLgwwtFO3qBa6FApKB1+r+30eKl5JCO+EKFMkI3YsMu6HbnlintWTwkW6Dmz2ZmvNsnubEvQ8eBIcomTA1Kgh9wx/OD1qZBFxKRrmSVFWCB38sDmY1JAt9s1mnD5tZxUHD9XqesoZrW/7/31bpugcUwHxS8/g8cEg4yVSmp7ygPASmPngU0JU3zVCDExZNn2N/ivFsK9C652Kish7CWGPGVQpNkMXHua X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 95032659-72b9-48fb-69c3-08da1e66699a X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:43.6070 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KwgKEdA2JNG/ngVpFQxUWh3r5Eugz1+WIFZGVd01ZR4YMMueDj0K+elWr+jGRwWAuJUtxu8EjV6nUQYrTA20pA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: idmyAuWGrQ3t12Y4uvMEXcE1Y1KPG0H5 X-Proofpoint-GUID: idmyAuWGrQ3t12Y4uvMEXcE1Y1KPG0H5 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit 3337ab08d08b1a375f88471d9c8b1cac968cb054 upstream Introduce a new nofault flag to indicate to iov_iter_get_pages not to fault in user pages. This is implemented by passing the FOLL_NOFAULT flag to get_user_pages, which causes get_user_pages to fail when it would otherwise fault in a page. We'll use the ->nofault flag to prevent iomap_dio_rw from faulting in pages when page faults are not allowed. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- include/linux/uio.h | 1 + lib/iov_iter.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 25d1c24fd829..6350354f97e9 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -35,6 +35,7 @@ struct iov_iter_state { struct iov_iter { u8 iter_type; + bool nofault; bool data_source; size_t iov_offset; size_t count; diff --git a/lib/iov_iter.c b/lib/iov_iter.c index b137da9afd7a..6d146f77601d 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -514,6 +514,7 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, WARN_ON(direction & ~(READ | WRITE)); *i = (struct iov_iter) { .iter_type = ITER_IOVEC, + .nofault = false, .data_source = direction, .iov = iov, .nr_segs = nr_segs, @@ -1529,13 +1530,17 @@ ssize_t iov_iter_get_pages(struct iov_iter *i, return 0; if (likely(iter_is_iovec(i))) { + unsigned int gup_flags = 0; unsigned long addr; + if (iov_iter_rw(i) != WRITE) + gup_flags |= FOLL_WRITE; + if (i->nofault) + gup_flags |= FOLL_NOFAULT; + addr = first_iovec_segment(i, &len, start, maxsize, maxpages); n = DIV_ROUND_UP(len, PAGE_SIZE); - res = get_user_pages_fast(addr, n, - iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, - pages); + res = get_user_pages_fast(addr, n, gup_flags, pages); if (unlikely(res <= 0)) return res; return (res == n ? len : res * PAGE_SIZE) - *start; @@ -1651,15 +1656,20 @@ ssize_t iov_iter_get_pages_alloc(struct iov_iter *i, return 0; if (likely(iter_is_iovec(i))) { + unsigned int gup_flags = 0; unsigned long addr; + if (iov_iter_rw(i) != WRITE) + gup_flags |= FOLL_WRITE; + if (i->nofault) + gup_flags |= FOLL_NOFAULT; + addr = first_iovec_segment(i, &len, start, maxsize, ~0U); n = DIV_ROUND_UP(len, PAGE_SIZE); p = get_pages_array(n); if (!p) return -ENOMEM; - res = get_user_pages_fast(addr, n, - iov_iter_rw(i) != WRITE ? FOLL_WRITE : 0, p); + res = get_user_pages_fast(addr, n, gup_flags, p); if (unlikely(res <= 0)) { kvfree(p); *pages = NULL; From patchwork Thu Apr 14 22:28:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E6B4C433EF for ; Thu, 14 Apr 2022 22:31:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347204AbiDNWdY (ORCPT ); Thu, 14 Apr 2022 18:33:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347197AbiDNWdV (ORCPT ); Thu, 14 Apr 2022 18:33:21 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07911C559B; Thu, 14 Apr 2022 15:30:55 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EJT05Q028178; Thu, 14 Apr 2022 22:30:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=dXLgYHGX2bcuJ1qIa1hHlApYOsaF6wp685Y8tY54+6o=; b=DDbb1yeSVQ3HYx36SR8fgZ5tmnBsEwTxtUUMvTpwoeYu7pPVCUlylWC94JxpbKFlW33g EYWmeBwECxSnWclMaSgoKKkxTjJJaRV5sgc+zUmu6GEjdOgRCyiOvKo0J6rn7hjFFRC0 2YFdbx+tbJip5BlEu0ocrBY5STYcQqyECdQ9C4yPm8Y+E6cm1Lq64oSGNfgart1AVKNg YqApNiljxlOINy5U4xj/jFrdx4R5bwxDmdMiBsfChoq8w5nryuy9IpvabfpSTHObgnmh 8HC5hVEdGFFxxxIU5Wdh8J2uyNH0q8Ab1xEKH6wl9eZ26yBrfVFBquYvIugieFTTc9cj Sw== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb21a5m0m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:52 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGT7v009294; Thu, 14 Apr 2022 22:30:51 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fck15dac9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MUFHKj+KehihwS/HkqjUmmnIxiyofbL8KxrRgVnOyOddRaRRxGSTl4X+ZWIG7fbX4Il1pXyhF2vI11KffFOsrlYBV7c9SrlLg0lYiiMnXXvjFKEZyMatoTf5e2xUVRgA6NEiLb73k05PFaXYZ4PwinTHNOZ7yoFHBHH518j2+yEjPeS9jjDbrhpcHB/zKfde8ZASU6QQRWUpG0+Z2sM/Oy6SwY8W+1C99DQAxPMHl+dV304Lbe2STPF12FqhzW/nnoaBYev3Cu8feZaSter/+ANFyjxZ2VQ7U9+wxvR2XjHI3ABJzeTyaxAU4WN1al7zUVnbpeDF9q30m1r35RQmCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dXLgYHGX2bcuJ1qIa1hHlApYOsaF6wp685Y8tY54+6o=; b=YuqOJuTlteeujetbLPxQNfPVpApdiBFxVwf0eTUFUJu1AYL86TAQt+zo9FM6DbNzZqLoB0BRMtHzJOm8XbdwmS7OPkIf2/0i8Bik0uYDW01Ud3y9zIUCx+RNwSSpvgg7GjFANswiC2MLotrl36c3ITr+2M8vxlBrfy6Znfcls3YUROO3lJUN/d0CKs7HOla1GppmuldmE3cZ2fHyJJhU8+mMdiuqHxUMEsYcwPg8PxaRIyDbmzr0Xib+h/vsmJ0TACCTsi94untAxlplEPUs9Qy8SCu6Bvhf+Mz8rcC6LIM1kj4eMm11f5irC20OG42lF+OO/bakoxaXZh+GUzkWsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dXLgYHGX2bcuJ1qIa1hHlApYOsaF6wp685Y8tY54+6o=; b=lf3wx0GHGWMWuYuuFGokt3qepqe0GG8E9W40hubNvYKPq/dnUgA6E7oohhumOBNpEWt6jVaQDwR0LoW/88sYYSxAHlodHvmgHl8a8CidTL939aEbIwbPyk2PK54GsRNGGwwdkCkp2/whpI9KkH679BRKkGZWIaWxZvX9ec0KBxM= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:49 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:49 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Andreas Gruenbacher , Anand Jain Subject: [PATCH v3 15/18 stable-5.15.y] gfs2: Fix mmap + page fault deadlocks for direct I/O Date: Fri, 15 Apr 2022 06:28:53 +0800 Message-Id: <02aca00403b19d316add3a4c835d40436a615103.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYCPR01CA0057.jpnprd01.prod.outlook.com (2603:1096:405:2::21) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 165da06e-1e37-422c-7e19-08da1e666d1d X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6YZcoYg/l+ICi5T9/36TxpHfXFPkp5ZNa7dgxq4humM6+II80SJQCMetM0c2wczw6s6coWmmjmz3PgyoF0PLL1ThxAE1wJgzKKNg5qvkO4pM61n8lhrsvg1/zCc5W45wSYMpG/cDVcIDV3d6mEPNQnmhw/Z9BSy4hO98+vV+0m5AoUFMhySuhinEeUGnyxBcvRQr9BgOArRZsEa3VM89AoQ2GLu75+7z8HT5xNH5xL0gk7IIwQrxWSdKuP6RfJ48eI2T3AUTmok47wbFbUtEn26nVDFB8Q6hpgEQEiR8o94x0/5Kt28pTnw780ytT5/J+3AbwiONja0lSTiexVzSNK/rbM5wLfImVpfo/7gg9uSKIUuE5arPtT1+rilWnQMX4gstj61i2BeuXelnEvUDRWvjGFozaO0KF7BRHXEDUvY0JGT7voyH9TGMagPJ+XyahZf0p7Nbewcal0eF8/v2NMXr3SigTN1l9HV6ubY5gmHZXbRKCI5iOhfpVBzKMbvzO+A/ZWrRKaA88PaU1/xOm1xPDgISz0LSuOnF2e3KlmHxaMslS4n2msMpCqPM+LL43MiM2n1x6mCtq/LbJPrMhQ2vNvX1jiuHdxaMX01TWaH5irGAlxhR0RVjTjHVfqiJyfcwnrR/l3Sr99oow/7vkQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qL+7s49vVW36elxmUWB10WSQVUgYYkwadA2wDYHeRvzSmo4/WyXYSEuq3x83VgJNG6xTQOFXS2Hy/lFMoE0SJyixs1Fs8U9nq0DN6D+toBYnIqS6KVi49GwCze6tBsVsIHXC3ekp4aRwLaQWMJrgTZodZiZxaVr6ix7os0EArlJlfARm5ubZ2hD4Z3q7SixpHnxWbu/q4Xn8jDtdKUsXur2Ebuuqv/rGLiBfGCcOk7CloQhwhdNYdDow79FQmdJ+kFZ18YP8aRwE28YGvpXBQUnmSYpl2CPD7cEBDBi5i0nVKMjBuA8x/YLdwllWu3Mm+qkiSbntiPWbOSapXu4FcYgsKttQGG4TPobYhradMxP7fhqvQGfmp7P6t2HkZJZFMD17rzGCl8W8Lc8+5X/Vv9UZVbcure+p0cbnSSnb79LbWYnL/PehjwGz4cCBsJJN75bMHEankBwWL4eYXTVALT9lGtBhkR5GYur3e1AcQERzdgf0h/7PVL4dgepT4CUo9aprdHR5nzLCowNEJGKVsWLaZ6Ujv+NDOCVoY0s2LfzXmuBXGU37Ahk+5MWc6fnJDyHEo6joaqvGqzifY3Fc5N2JYO/1dD5jVxYjVcCEeKISPAIK/85buEWluTUuGohXp1c3mZ0gtX5OICvoeQjScWhPzbJxsbgsyNYgM5Zer6ys/gAMHO+W4d6HEmK4nkhR+PoSX+P1zlo6HktRcXeIqxIQ6sMCT34zTW0BX9i8zf7cLvqwb6kK6souM686oq38kS1xs15ZA4bz238PTUkSqcUMbbmjkRpbD6Wgy8RaC0GJeGwT+lQf/iUstsC/63QJebNjy/9IDG1fyzJIkN/L4KSYBA1DpIw/DptsDBauh/cPMNJXG+NXxl3iDJNyuCCTyiR/avr+tlgGIhaCgRD27YoeVA/CTHFpWVpcpqE2a4MYzGYvhuT+v7Ontuf4/baEfZqdqrcHVQ6Vaxc96GP9Sb1LzEARp9K7ZMbHa+IHLigdhpb98sFLJ2oDIh2lL/iZLtVLGW0bkVLJQ4avxRb8HExAyxlRhQ1iT/WB89oL0uuSPH36p/KPGBrcE5I5blWqoH5yPFk2wnfoMhdqGBMPap1FVVUqX12Dh3eOb8R57fRfaqmirtktVk8PWajqMBSVvmNV3njjy8QE80Bap+QLh2YKcg0s4+kWw7qLq86901ERXGcLARpu3js/V/zKe3A5TT2JkuhxItJIPw3QpiLAo6wi095FGHzc+6e6jVbufBNvttkVDrIMH/aJg4CZNntUziMpf/ry+qD1PPlV9Ku3gDpUJrkbyQr/vxtTDpyc1CjyQITxAcN8VERKD2iR9fDwGr5rHcz0jinA4l5P6eH3p3e1Eyt40rijV2t/KsE/fwTXDMG0pjBQi+47BgIqGCxN3OYWOQ+Mk2O6RqbvF1ANMefqe6C2jJQvUkDJYmvg9kmy3gQT3XVovqDk5F+sOasAPPpO+tnqrExMM166ISAenWaRhzVp0q/MVJmq9GYWnwLzJ2JhtYy8kzy2X+yZcpaCJo8x82A2Qw5CbXfvkmiAyP1ayIroxH7XjX7lN5tD9P0+4yXAbDqM39JAdl6qRxD//IjNOhUJic+4tYxesKHTiHSZ1VyipORE+oB4g/V+nliEyEaEZr5NXgugefFjRNnh4hbEwkVF2Sj3tGfvSJhBxinpQFK23UeO2kEl426gwrmGDgO1fcPziyQ1VZXShVKlZbQuuvcWYKGUG1MAGtaVirZEDkYjr2oIOOjjYKWg0kUA579soxEd1GeGoJg+vs4o X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 165da06e-1e37-422c-7e19-08da1e666d1d X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:49.4211 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TpUbKEvga7h8SkIx8RPl/5vHAFRO3TAYz9g6cEKJkW8k28HEyQ2KJsIMAQG0oFH6U8T6gVKEi5y1KhGeLRHPhQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 adultscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-GUID: 0GgPr6T3PgpEQfCXX9VDRzFckatUStbK X-Proofpoint-ORIG-GUID: 0GgPr6T3PgpEQfCXX9VDRzFckatUStbK Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andreas Gruenbacher commit b01b2d72da25c000aeb124bc78daf3fb998be2b6 upstream Also disable page faults during direct I/O requests and implement a similar kind of retry logic as in the buffered I/O case. The retry logic in the direct I/O case differs from the buffered I/O case in the following way: direct I/O doesn't provide the kinds of consistency guarantees between concurrent reads and writes that buffered I/O provides, so once we lose the inode glock while faulting in user pages, we always resume the operation. We never need to return a partial read or write. This locking problem was originally reported by Jan Kara. Linus came up with the idea of disabling page faults. Many thanks to Al Viro and Matthew Wilcox for their feedback. Signed-off-by: Andreas Gruenbacher Signed-off-by: Anand Jain --- fs/gfs2/file.c | 99 ++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 87 insertions(+), 12 deletions(-) diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c index 81835d34d6f6..247b8d95b5ef 100644 --- a/fs/gfs2/file.c +++ b/fs/gfs2/file.c @@ -812,22 +812,64 @@ static ssize_t gfs2_file_direct_read(struct kiocb *iocb, struct iov_iter *to, { struct file *file = iocb->ki_filp; struct gfs2_inode *ip = GFS2_I(file->f_mapping->host); - size_t count = iov_iter_count(to); + size_t prev_count = 0, window_size = 0; + size_t written = 0; ssize_t ret; - if (!count) + /* + * In this function, we disable page faults when we're holding the + * inode glock while doing I/O. If a page fault occurs, we indicate + * that the inode glock may be dropped, fault in the pages manually, + * and retry. + * + * Unlike generic_file_read_iter, for reads, iomap_dio_rw can trigger + * physical as well as manual page faults, and we need to disable both + * kinds. + * + * For direct I/O, gfs2 takes the inode glock in deferred mode. This + * locking mode is compatible with other deferred holders, so multiple + * processes and nodes can do direct I/O to a file at the same time. + * There's no guarantee that reads or writes will be atomic. Any + * coordination among readers and writers needs to happen externally. + */ + + if (!iov_iter_count(to)) return 0; /* skip atime */ gfs2_holder_init(ip->i_gl, LM_ST_DEFERRED, 0, gh); +retry: ret = gfs2_glock_nq(gh); if (ret) goto out_uninit; +retry_under_glock: + pagefault_disable(); + to->nofault = true; + ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, + IOMAP_DIO_PARTIAL, written); + to->nofault = false; + pagefault_enable(); + if (ret > 0) + written = ret; - ret = iomap_dio_rw(iocb, to, &gfs2_iomap_ops, NULL, 0, 0); - gfs2_glock_dq(gh); + if (should_fault_in_pages(ret, to, &prev_count, &window_size)) { + size_t leftover; + + gfs2_holder_allow_demote(gh); + leftover = fault_in_iov_iter_writeable(to, window_size); + gfs2_holder_disallow_demote(gh); + if (leftover != window_size) { + if (!gfs2_holder_queued(gh)) + goto retry; + goto retry_under_glock; + } + } + if (gfs2_holder_queued(gh)) + gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); - return ret; + if (ret < 0) + return ret; + return written; } static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, @@ -836,10 +878,20 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, struct file *file = iocb->ki_filp; struct inode *inode = file->f_mapping->host; struct gfs2_inode *ip = GFS2_I(inode); - size_t len = iov_iter_count(from); - loff_t offset = iocb->ki_pos; + size_t prev_count = 0, window_size = 0; + size_t read = 0; ssize_t ret; + /* + * In this function, we disable page faults when we're holding the + * inode glock while doing I/O. If a page fault occurs, we indicate + * that the inode glock may be dropped, fault in the pages manually, + * and retry. + * + * For writes, iomap_dio_rw only triggers manual page faults, so we + * don't need to disable physical ones. + */ + /* * Deferred lock, even if its a write, since we do no allocation on * this path. All we need to change is the atime, and this lock mode @@ -849,22 +901,45 @@ static ssize_t gfs2_file_direct_write(struct kiocb *iocb, struct iov_iter *from, * VFS does. */ gfs2_holder_init(ip->i_gl, LM_ST_DEFERRED, 0, gh); +retry: ret = gfs2_glock_nq(gh); if (ret) goto out_uninit; - +retry_under_glock: /* Silently fall back to buffered I/O when writing beyond EOF */ - if (offset + len > i_size_read(&ip->i_inode)) + if (iocb->ki_pos + iov_iter_count(from) > i_size_read(&ip->i_inode)) goto out; - ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, 0, 0); + from->nofault = true; + ret = iomap_dio_rw(iocb, from, &gfs2_iomap_ops, NULL, + IOMAP_DIO_PARTIAL, read); + from->nofault = false; + if (ret == -ENOTBLK) ret = 0; + if (ret > 0) + read = ret; + + if (should_fault_in_pages(ret, from, &prev_count, &window_size)) { + size_t leftover; + + gfs2_holder_allow_demote(gh); + leftover = fault_in_iov_iter_readable(from, window_size); + gfs2_holder_disallow_demote(gh); + if (leftover != window_size) { + if (!gfs2_holder_queued(gh)) + goto retry; + goto retry_under_glock; + } + } out: - gfs2_glock_dq(gh); + if (gfs2_holder_queued(gh)) + gfs2_glock_dq(gh); out_uninit: gfs2_holder_uninit(gh); - return ret; + if (ret < 0) + return ret; + return read; } static ssize_t gfs2_file_read_iter(struct kiocb *iocb, struct iov_iter *to) From patchwork Thu Apr 14 22:28:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78F07C433F5 for ; Thu, 14 Apr 2022 22:31:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347211AbiDNWda (ORCPT ); Thu, 14 Apr 2022 18:33:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347197AbiDNWd2 (ORCPT ); Thu, 14 Apr 2022 18:33:28 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B959C55A5; Thu, 14 Apr 2022 15:31:01 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKX5sh008564; Thu, 14 Apr 2022 22:30:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=8Ct0seMaWqqoEdaNRFLwwYNLam8EXfh3cGOja5tWS8Q=; b=iukYm+sGpreU7G1lK8sCZZQ6reesM4EAAwugHfsDR1nl0+wW6wLHELYaHSTEzlTf3RWa pQ2HpgOeUikqnvIWLqJhZgwyBGVCCNSqwTeBXOV1jPph8YS62AakKxTisXZp+oKjM0MZ P6tlquY4vdCnYKXuEI/MNWyOXDCuw7V1eS5l3URPAyZ54vuMRBfDgsv8fQRAd7IcoLya eDlOUWJ0LsOr2v8I0OugQInh1zrOXkxvsoDfMUayQT9zcQwgnSsjXGmPwrPsg84KPnO0 CVjK7D/09IhzjgfP1AwRcdEniTdBK1lhZsACHE/reh3u4mG7BYVOD6kkkmeJPA0ihZbQ rg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2p3w1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:57 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMFowJ006346; Thu, 14 Apr 2022 22:30:56 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2047.outbound.protection.outlook.com [104.47.56.47]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5p5n5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:30:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kE2evXRAoPyUgZowN3pAURGl0LFV9F4CprqiiFeMmHrjnqP+/ELL4NYsbzFKzAwAikD+9xFGHxliFWxTrMgcKPcfpxujCFfeA7ky1AsHf0T+9eegdfpxQD5Dqc7yivRlu63mz0Lw0lVgba3e/kAlkzyAHtmKmJo1Fq1gX9W+kUs2hGeNPqOtZR+qGiLhylmsyWvUYAZjTNeSIVJemA5d6z6N9Nane82BI913fyEC4UxNWzq9XvGy+oTPHOKe8FFtHq+Io4vs/qQtSGbIVb5Hkee4JceEIG+L8HqXhaQT18btd0z2yz6Kr41jLkdU8DxQKwWuQI+TRRSr0+R9bg/C/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8Ct0seMaWqqoEdaNRFLwwYNLam8EXfh3cGOja5tWS8Q=; b=UJuWzo/Ghyl1zVucu2mHttCFuK6yZYW8+NBPmPgKDF4ZMZXBUwbQFBC4MvQ13upQ00JjN6fWR5jUZ68CyAoI4VgDT3GUkmFSe1hOMyaSekIYOXUtH8z45ZvAg+rB+O0SOKPl6qHADHg27CT1YWdJfqKEYBQwA9z0Y4ZcMy1ROT+j3A8nkgBrNEJbd1sv/zmVlpxCAhZAMvv3jTyB0+AQPkOxaSn4Omn42Eb3oNJrpmdub49yqyWdgiv9jwLnxlqkOn3rR2eDcOFwov3ITUwWXD0P90Rj2gGl8r3Gy1GlGd+wraDt0lA4HV0RU1sZPdnTonrVa91T0itcH4jGQ6tZlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Ct0seMaWqqoEdaNRFLwwYNLam8EXfh3cGOja5tWS8Q=; b=xLsi7eQQNA7uXdZitlChnUVlxRtG15yt2M8ScHyB7sY6J0Bf9SDH2M41Jx28G4/vDmRg7HWl9B3zshlumb7R1zc2OR0zTsIcCOxMF6XoyAOXh5Q3XiaVd4pk2T88plvTjTVZkFjZS+RuadhYpL2Bvs0/9k78flmWzkEoJBcsMFw= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:30:54 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:30:54 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana , Josef Bacik , David Sterba , Anand Jain Subject: [PATCH v3 16/18 stable-5.15.y] btrfs: fix deadlock due to page faults during direct IO reads and writes Date: Fri, 15 Apr 2022 06:28:54 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYCPR01CA0057.jpnprd01.prod.outlook.com (2603:1096:405:2::21) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40c51a68-596b-4e51-c5b7-08da1e66700c X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TI3sZyRMKyev6VDK31z2YoAqe+TXgncBuXJSGJKBAW+ooo1jfABNxI1crhRaTHxH/d5ulA/9nJp2JAvLh6hOWC7/U4jRI0m6gtfVOzNONyQjC9He3O3Bh5bXL8rOdw9ml+5mRfUrqLMHd/px6KlAswr37VH+X99D1D0gT9eb2DWvQwvPZSeAx3fAQ1WuHTGB1njRNXZ/LaeR7+hyM1jBZ5KDRd8QOCrBTo5u2/4/e1AT2H+ywvV+Jqp0O/K8jy8n17Vh+JtrM6tfcBmuoHmOKIKR+9unqig7kfpFxl/lbJ1trfTWSjfsUr0tShklqMZgtrNU50kkDV8AAqekVKALHBRF0Pc8fTn88TTlO0okdKmJivLKR/lbpBokx12kPYJrbgX7DDpHUTqCWVvXDm43zzxd65gznGSOCDH1Wc1b/g1qc0iA04T62QU8mRTUzKqfidoSQoO77n+E1Y96pUCFeRwRTMyoR+/Rt2eOxpKQZox1kEUcRsqWtLEe5256dCl4O7Fk3y2zVRUiUGvLqEya316IxE3S6RKvW3iPY4G1/6tOIYebPQV6hfnDQ1PnAq2m6WQD3x5u2YXyjkZtTJybxKVliwSRMfHrHPPA33wWD54QNub0OEWYQz11aJU4e9PcJFmr1wfGlJh7MmPQUAe/VA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(30864003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tnbXDicVn1eFPJjtc+/CHHDUXhUq329/Y2zmWpHAj0joTVtP6dPT39eAiiW54wZ7QblQ3rO7VC8fvxExa5/IZr1mQAZ+bVqZ+f+xZqKzpRd4bEjYvhHc95VCTqdWyhX7WELRq3OXyBwE5qa8AkjZaD90cUk8swk2MXYTdCDfolCXUhzW9RY24tWls9hfSVZFjePRru+h7wnrmcnG7l+vmCaMRK1GVtKBcxPdfYmeJR5D6NkLFwDjerpEJHa5fREFkC/I7wanc4F7vB5U97KYiorjmKjmOZijzIc8TmYDL8tnzuvZ7sAMStN3sH3fxIkU/9/5dLBJLokpt6S2sTRM1xFW0qFVWtO+0vAwUBhlfGCeLt+DEED5OjMYPc0+tSAXJr0tPUHWx6B+XLrVuYu5pW8ZtC1HQs7PgimdGJPpIItj8f24HraocEWu0l3Nc/ZWI4iTJjix9lzPRZeEnImmmvlHlz590rQXRKXAUvxUCQ+xf2Cc0PF+eF9K22ZtUZrlhhlMG9inZLG14NWItN7cJSHXfXFTWzQGJKxqoJisnREVOufidvs9rNpVNRZFQ/OeC09Jt/MV15G8yWXvZVmiNXkavL1jpdkMdkJ5E6WaWvT/aEoE+/MXkIuiaw8H3/IG3g+8UFUP8pRmreP1vSyiTxW6vluC6D2G7nr0mBOKjl5HRPDLwd6HDcGUFheoLX0g61ogaxwy3Dph0Ac4r5Kl7f0VVuj7eG3k2LbvVbFBmdFF+OUrmfl91NfQYkOWllwvn6mRZBDr28usVM32kA/LECCxtPw+4VMXzvssOxZSiJXUlYybrA6FsiGkYlXNUfQZEkrI3N6sAWuiqsJaJrPApLqTSQ8T42/OinOTOSBba1FpvwJsA3iGXKdge7Jx2YiXC6x1care4bjIe+xLsL2RR/bAvqyFNRj8+ISwl6kb3Q+QxKMhk+XfjUYNYea8QaYMgcnFD2Df79ooveWS/NYfSRIHZoI+d30mx5o+DhDnqyn3bNPVwfGn77L9/lpecjZJ/fmtQqwZMxKtPVSVlqlI5sTYs7OYQojRf0cfbmTsw6+hVkOqPcuGOkUWn5N3XNfYQNoDu5Ba3AODNAT2zIa/qcSGuhLochqTb6pocoj2YlUawNnhF56wOYQAseFJsN0vcrIcEjP3WkSiS15I4JLn3KYW2Qex1KenjtkliNIm2NNA4G8i2+0W/PeaqOGOWEqjaDV20l6slCiZsiKlWmwOpTeGxS9jEYSqE5FaFbJEnmujviMT304NaK6oVRNd/79VqDv/e8jhEBHU4PqyjWqXoXMUUhidXlpUGpxnAZpGDr37gEKt04ZTR2oG/iaUCRxehH/UUHme0xphslMipUVum2E97UN3kaw6IaspWB/EaZO6k/dbPO2qFL24mCHv4VNvdc5DQLpghN7yMJlyEW3N7iUP0j276fMN6sp9fbb3FPp45uUD+NfxoEELdP5JZKiiA0QC/Wv2uSOTyXKEOLE6fZ5xho7mlE7Z2PWg5mymuf+GbNDQF165tQGfIYoXTa2hyRHy8rOaTKF02zGs+aJx5WuYgaMdYBEKUPg87UfceGLT4IorL7edKky5N46Ls908bU9FMU5dxT0N7pQ8sG3pvtyeLvxOsFP16M0NILZ/Zkjuikp4+lahzYEa087T/Y0otBL3kA5dLMC4K6DowQwiu+RfcgYKKQSQgEQENL7l7lz6xFTxORslDIYAZsMlNff6BNlXug82gwUT29BKIBelFgQOKfeZJKRKbJ2faSbr6oH6fjj7mf7n9XVWxH/h6aSF X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 40c51a68-596b-4e51-c5b7-08da1e66700c X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:30:54.3134 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: u1U7EgH25XspZ7Z4ziJiUX6O0/dPrBomqtrL5mumh+rp8Hd0tsdeQYyqBsUAJ2g50Pbx3fJ9VhGSiNF/oTy1zA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: j8yxx6EPSSgrTdfMLy8KiM3q3uVbQDck X-Proofpoint-GUID: j8yxx6EPSSgrTdfMLy8KiM3q3uVbQDck Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Filipe Manana commit 51bd9563b6783de8315f38f7baed949e77c42311 upstream If we do a direct IO read or write when the buffer given by the user is memory mapped to the file range we are going to do IO, we end up ending in a deadlock. This is triggered by the new test case generic/647 from fstests. For a direct IO read we get a trace like this: [967.872718] INFO: task mmap-rw-fault:12176 blocked for more than 120 seconds. [967.874161] Not tainted 5.14.0-rc7-btrfs-next-95 #1 [967.874909] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [967.875983] task:mmap-rw-fault state:D stack: 0 pid:12176 ppid: 11884 flags:0x00000000 [967.875992] Call Trace: [967.875999] __schedule+0x3ca/0xe10 [967.876015] schedule+0x43/0xe0 [967.876020] wait_extent_bit.constprop.0+0x1eb/0x260 [btrfs] [967.876109] ? do_wait_intr_irq+0xb0/0xb0 [967.876118] lock_extent_bits+0x37/0x90 [btrfs] [967.876150] btrfs_lock_and_flush_ordered_range+0xa9/0x120 [btrfs] [967.876184] ? extent_readahead+0xa7/0x530 [btrfs] [967.876214] extent_readahead+0x32d/0x530 [btrfs] [967.876253] ? lru_cache_add+0x104/0x220 [967.876255] ? kvm_sched_clock_read+0x14/0x40 [967.876258] ? sched_clock_cpu+0xd/0x110 [967.876263] ? lock_release+0x155/0x4a0 [967.876271] read_pages+0x86/0x270 [967.876274] ? lru_cache_add+0x125/0x220 [967.876281] page_cache_ra_unbounded+0x1a3/0x220 [967.876291] filemap_fault+0x626/0xa20 [967.876303] __do_fault+0x36/0xf0 [967.876308] __handle_mm_fault+0x83f/0x15f0 [967.876322] handle_mm_fault+0x9e/0x260 [967.876327] __get_user_pages+0x204/0x620 [967.876332] ? get_user_pages_unlocked+0x69/0x340 [967.876340] get_user_pages_unlocked+0xd3/0x340 [967.876349] internal_get_user_pages_fast+0xbca/0xdc0 [967.876366] iov_iter_get_pages+0x8d/0x3a0 [967.876374] bio_iov_iter_get_pages+0x82/0x4a0 [967.876379] ? lock_release+0x155/0x4a0 [967.876387] iomap_dio_bio_actor+0x232/0x410 [967.876396] iomap_apply+0x12a/0x4a0 [967.876398] ? iomap_dio_rw+0x30/0x30 [967.876414] __iomap_dio_rw+0x29f/0x5e0 [967.876415] ? iomap_dio_rw+0x30/0x30 [967.876420] ? lock_acquired+0xf3/0x420 [967.876429] iomap_dio_rw+0xa/0x30 [967.876431] btrfs_file_read_iter+0x10b/0x140 [btrfs] [967.876460] new_sync_read+0x118/0x1a0 [967.876472] vfs_read+0x128/0x1b0 [967.876477] __x64_sys_pread64+0x90/0xc0 [967.876483] do_syscall_64+0x3b/0xc0 [967.876487] entry_SYSCALL_64_after_hwframe+0x44/0xae [967.876490] RIP: 0033:0x7fb6f2c038d6 [967.876493] RSP: 002b:00007fffddf586b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000011 [967.876496] RAX: ffffffffffffffda RBX: 0000000000001000 RCX: 00007fb6f2c038d6 [967.876498] RDX: 0000000000001000 RSI: 00007fb6f2c17000 RDI: 0000000000000003 [967.876499] RBP: 0000000000001000 R08: 0000000000000003 R09: 0000000000000000 [967.876501] R10: 0000000000001000 R11: 0000000000000246 R12: 0000000000000003 [967.876502] R13: 0000000000000000 R14: 00007fb6f2c17000 R15: 0000000000000000 This happens because at btrfs_dio_iomap_begin() we lock the extent range and return with it locked - we only unlock in the endio callback, at end_bio_extent_readpage() -> endio_readpage_release_extent(). Then after iomap called the btrfs_dio_iomap_begin() callback, it triggers the page faults that resulting in reading the pages, through the readahead callback btrfs_readahead(), and through there we end to attempt to lock again the same extent range (or a subrange of what we locked before), resulting in the deadlock. For a direct IO write, the scenario is a bit different, and it results in trace like this: [1132.442520] run fstests generic/647 at 2021-08-31 18:53:35 [1330.349355] INFO: task mmap-rw-fault:184017 blocked for more than 120 seconds. [1330.350540] Not tainted 5.14.0-rc7-btrfs-next-95 #1 [1330.351158] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [1330.351900] task:mmap-rw-fault state:D stack: 0 pid:184017 ppid:183725 flags:0x00000000 [1330.351906] Call Trace: [1330.351913] __schedule+0x3ca/0xe10 [1330.351930] schedule+0x43/0xe0 [1330.351935] btrfs_start_ordered_extent+0x108/0x1c0 [btrfs] [1330.352020] ? do_wait_intr_irq+0xb0/0xb0 [1330.352028] btrfs_lock_and_flush_ordered_range+0x8c/0x120 [btrfs] [1330.352064] ? extent_readahead+0xa7/0x530 [btrfs] [1330.352094] extent_readahead+0x32d/0x530 [btrfs] [1330.352133] ? lru_cache_add+0x104/0x220 [1330.352135] ? kvm_sched_clock_read+0x14/0x40 [1330.352138] ? sched_clock_cpu+0xd/0x110 [1330.352143] ? lock_release+0x155/0x4a0 [1330.352151] read_pages+0x86/0x270 [1330.352155] ? lru_cache_add+0x125/0x220 [1330.352162] page_cache_ra_unbounded+0x1a3/0x220 [1330.352172] filemap_fault+0x626/0xa20 [1330.352176] ? filemap_map_pages+0x18b/0x660 [1330.352184] __do_fault+0x36/0xf0 [1330.352189] __handle_mm_fault+0x1253/0x15f0 [1330.352203] handle_mm_fault+0x9e/0x260 [1330.352208] __get_user_pages+0x204/0x620 [1330.352212] ? get_user_pages_unlocked+0x69/0x340 [1330.352220] get_user_pages_unlocked+0xd3/0x340 [1330.352229] internal_get_user_pages_fast+0xbca/0xdc0 [1330.352246] iov_iter_get_pages+0x8d/0x3a0 [1330.352254] bio_iov_iter_get_pages+0x82/0x4a0 [1330.352259] ? lock_release+0x155/0x4a0 [1330.352266] iomap_dio_bio_actor+0x232/0x410 [1330.352275] iomap_apply+0x12a/0x4a0 [1330.352278] ? iomap_dio_rw+0x30/0x30 [1330.352292] __iomap_dio_rw+0x29f/0x5e0 [1330.352294] ? iomap_dio_rw+0x30/0x30 [1330.352306] btrfs_file_write_iter+0x238/0x480 [btrfs] [1330.352339] new_sync_write+0x11f/0x1b0 [1330.352344] ? NF_HOOK_LIST.constprop.0.cold+0x31/0x3e [1330.352354] vfs_write+0x292/0x3c0 [1330.352359] __x64_sys_pwrite64+0x90/0xc0 [1330.352365] do_syscall_64+0x3b/0xc0 [1330.352369] entry_SYSCALL_64_after_hwframe+0x44/0xae [1330.352372] RIP: 0033:0x7f4b0a580986 [1330.352379] RSP: 002b:00007ffd34d75418 EFLAGS: 00000246 ORIG_RAX: 0000000000000012 [1330.352382] RAX: ffffffffffffffda RBX: 0000000000001000 RCX: 00007f4b0a580986 [1330.352383] RDX: 0000000000001000 RSI: 00007f4b0a3a4000 RDI: 0000000000000003 [1330.352385] RBP: 00007f4b0a3a4000 R08: 0000000000000003 R09: 0000000000000000 [1330.352386] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000003 [1330.352387] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 Unlike for reads, at btrfs_dio_iomap_begin() we return with the extent range unlocked, but later when the page faults are triggered and we try to read the extents, we end up btrfs_lock_and_flush_ordered_range() where we find the ordered extent for our write, created by the iomap callback btrfs_dio_iomap_begin(), and we wait for it to complete, which makes us deadlock since we can't complete the ordered extent without reading the pages (the iomap code only submits the bio after the pages are faulted in). Fix this by setting the nofault attribute of the given iov_iter and retry the direct IO read/write if we get an -EFAULT error returned from iomap. For reads, also disable page faults completely, this is because when we read from a hole or a prealloc extent, we can still trigger page faults due to the call to iov_iter_zero() done by iomap - at the moment, it is oblivious to the value of the ->nofault attribute of an iov_iter. We also need to keep track of the number of bytes written or read, and pass it to iomap_dio_rw(), as well as use the new flag IOMAP_DIO_PARTIAL. This depends on the iov_iter and iomap changes introduced in commit c03098d4b9ad ("Merge tag 'gfs2-v5.15-rc5-mmap-fault' of git://git.kernel.org/pub/scm/linux/kernel/git/gfs2/linux-gfs2"). Reviewed-by: Josef Bacik Signed-off-by: Filipe Manana Signed-off-by: David Sterba Signed-off-by: Anand Jain --- fs/btrfs/file.c | 139 ++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 123 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index cd4950476366..5ac6ec80b970 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -1903,16 +1903,17 @@ static ssize_t check_direct_IO(struct btrfs_fs_info *fs_info, static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) { + const bool is_sync_write = (iocb->ki_flags & IOCB_DSYNC); struct file *file = iocb->ki_filp; struct inode *inode = file_inode(file); struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); loff_t pos; ssize_t written = 0; ssize_t written_buffered; + size_t prev_left = 0; loff_t endbyte; ssize_t err; unsigned int ilock_flags = 0; - struct iomap_dio *dio = NULL; if (iocb->ki_flags & IOCB_NOWAIT) ilock_flags |= BTRFS_ILOCK_TRY; @@ -1955,23 +1956,80 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) goto buffered; } - dio = __iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - 0, 0); + /* + * We remove IOCB_DSYNC so that we don't deadlock when iomap_dio_rw() + * calls generic_write_sync() (through iomap_dio_complete()), because + * that results in calling fsync (btrfs_sync_file()) which will try to + * lock the inode in exclusive/write mode. + */ + if (is_sync_write) + iocb->ki_flags &= ~IOCB_DSYNC; - btrfs_inode_unlock(inode, ilock_flags); + /* + * The iov_iter can be mapped to the same file range we are writing to. + * If that's the case, then we will deadlock in the iomap code, because + * it first calls our callback btrfs_dio_iomap_begin(), which will create + * an ordered extent, and after that it will fault in the pages that the + * iov_iter refers to. During the fault in we end up in the readahead + * pages code (starting at btrfs_readahead()), which will lock the range, + * find that ordered extent and then wait for it to complete (at + * btrfs_lock_and_flush_ordered_range()), resulting in a deadlock since + * obviously the ordered extent can never complete as we didn't submit + * yet the respective bio(s). This always happens when the buffer is + * memory mapped to the same file range, since the iomap DIO code always + * invalidates pages in the target file range (after starting and waiting + * for any writeback). + * + * So here we disable page faults in the iov_iter and then retry if we + * got -EFAULT, faulting in the pages before the retry. + */ +again: + from->nofault = true; + err = iomap_dio_rw(iocb, from, &btrfs_dio_iomap_ops, &btrfs_dio_ops, + IOMAP_DIO_PARTIAL, written); + from->nofault = false; - if (IS_ERR_OR_NULL(dio)) { - err = PTR_ERR_OR_ZERO(dio); - if (err < 0 && err != -ENOTBLK) - goto out; - } else { - written = iomap_dio_complete(dio); + /* No increment (+=) because iomap returns a cumulative value. */ + if (err > 0) + written = err; + + if (iov_iter_count(from) > 0 && (err == -EFAULT || err > 0)) { + const size_t left = iov_iter_count(from); + /* + * We have more data left to write. Try to fault in as many as + * possible of the remainder pages and retry. We do this without + * releasing and locking again the inode, to prevent races with + * truncate. + * + * Also, in case the iov refers to pages in the file range of the + * file we want to write to (due to a mmap), we could enter an + * infinite loop if we retry after faulting the pages in, since + * iomap will invalidate any pages in the range early on, before + * it tries to fault in the pages of the iov. So we keep track of + * how much was left of iov in the previous EFAULT and fallback + * to buffered IO in case we haven't made any progress. + */ + if (left == prev_left) { + err = -ENOTBLK; + } else { + fault_in_iov_iter_readable(from, left); + prev_left = left; + goto again; + } } - if (written < 0 || !iov_iter_count(from)) { - err = written; + btrfs_inode_unlock(inode, ilock_flags); + + /* + * Add back IOCB_DSYNC. Our caller, btrfs_file_write_iter(), will do + * the fsync (call generic_write_sync()). + */ + if (is_sync_write) + iocb->ki_flags |= IOCB_DSYNC; + + /* If 'err' is -ENOTBLK then it means we must fallback to buffered IO. */ + if ((err < 0 && err != -ENOTBLK) || !iov_iter_count(from)) goto out; - } buffered: pos = iocb->ki_pos; @@ -1996,7 +2054,7 @@ static ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from) invalidate_mapping_pages(file->f_mapping, pos >> PAGE_SHIFT, endbyte >> PAGE_SHIFT); out: - return written ? written : err; + return err < 0 ? err : written; } static ssize_t btrfs_file_write_iter(struct kiocb *iocb, @@ -3650,6 +3708,8 @@ static int check_direct_read(struct btrfs_fs_info *fs_info, static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) { struct inode *inode = file_inode(iocb->ki_filp); + size_t prev_left = 0; + ssize_t read = 0; ssize_t ret; if (fsverity_active(inode)) @@ -3659,10 +3719,57 @@ static ssize_t btrfs_direct_read(struct kiocb *iocb, struct iov_iter *to) return 0; btrfs_inode_lock(inode, BTRFS_ILOCK_SHARED); +again: + /* + * This is similar to what we do for direct IO writes, see the comment + * at btrfs_direct_write(), but we also disable page faults in addition + * to disabling them only at the iov_iter level. This is because when + * reading from a hole or prealloc extent, iomap calls iov_iter_zero(), + * which can still trigger page fault ins despite having set ->nofault + * to true of our 'to' iov_iter. + * + * The difference to direct IO writes is that we deadlock when trying + * to lock the extent range in the inode's tree during he page reads + * triggered by the fault in (while for writes it is due to waiting for + * our own ordered extent). This is because for direct IO reads, + * btrfs_dio_iomap_begin() returns with the extent range locked, which + * is only unlocked in the endio callback (end_bio_extent_readpage()). + */ + pagefault_disable(); + to->nofault = true; ret = iomap_dio_rw(iocb, to, &btrfs_dio_iomap_ops, &btrfs_dio_ops, - 0, 0); + IOMAP_DIO_PARTIAL, read); + to->nofault = false; + pagefault_enable(); + + /* No increment (+=) because iomap returns a cumulative value. */ + if (ret > 0) + read = ret; + + if (iov_iter_count(to) > 0 && (ret == -EFAULT || ret > 0)) { + const size_t left = iov_iter_count(to); + + if (left == prev_left) { + /* + * We didn't make any progress since the last attempt, + * fallback to a buffered read for the remainder of the + * range. This is just to avoid any possibility of looping + * for too long. + */ + ret = read; + } else { + /* + * We made some progress since the last retry or this is + * the first time we are retrying. Fault in as many pages + * as possible and retry. + */ + fault_in_iov_iter_writeable(to, left); + prev_left = left; + goto again; + } + } btrfs_inode_unlock(inode, BTRFS_ILOCK_SHARED); - return ret; + return ret < 0 ? ret : read; } static ssize_t btrfs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) From patchwork Thu Apr 14 22:28:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 561507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71D59C433F5 for ; Thu, 14 Apr 2022 22:31:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347164AbiDNWdg (ORCPT ); Thu, 14 Apr 2022 18:33:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240406AbiDNWdf (ORCPT ); Thu, 14 Apr 2022 18:33:35 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2215CC55AA; Thu, 14 Apr 2022 15:31:09 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23EKFdKT031973; Thu, 14 Apr 2022 22:31:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=i5na+iiepRR7J2nHb//fiHfdQUlvPFKc3XueqaxAQbM=; b=tcCkxkwdFIOdVBcpJ4A5d264fWgX2imgwQw3wQcNYDwMZZ5tlOIbHZ41nwCntKHt5hD1 PWVJRwUynFI3tDuSh9x3BNZPdCexOmToWdya65CqHmFG3nYx8aDwnL4zxQve88IsADTf UHwmDee0LsZtw6eOLA1QSpMxT6eASoJ0c8H/z44W6C2GVGMGPvROg8zyE2AODRb889Je bx23sbDIT+Y24yQY3t/Hx5J9jkI7zbufFgADOk9x8lHGhXM7MZwbwWqcyWVFsPQ6ey0t oHRlnV02rD4Ng0WOyBdeNNl/gBaKhomi6+uMdrYy6OZ330Lw1LuOsgSclyzrabf3ES6T IA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0jddp3s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:31:05 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMGOBr008408; Thu, 14 Apr 2022 22:31:04 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2046.outbound.protection.outlook.com [104.47.56.46]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fcg9m6m0h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:31:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I1BAwHzfshobXVpDeLSbFew6Kl4FYVFpmaPTqJT9/GX5/bOJr80EPCyYDPdydkmxy/l+6DShFnemUZSIkVkoXt3utO7/YwQYuKmc4H/QVXmzCMB3pMBZwq2prd3rmxNiSO3v7k2FfbjrZMxQ3rcpIw5KcVG6sovBMNvH8A5xU4ru+xAThDsjkS0S3UOV4C8K1WjvrjLNkj1rPbcg1KpCq/bNsesqkIVnx4pSFexeNc784p37iVP+MaIU5bunyEuUzcZ+Nx/do4zsmWTno3tRkTXERl1f3K1B8MakFJ5o0LH02LkRYMZT1moLQsZr2iL4bfDXoBfX0aAYyMMlQqnCnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i5na+iiepRR7J2nHb//fiHfdQUlvPFKc3XueqaxAQbM=; b=TXV/hS1v86042XdOG4xulsxIxNnnVWvb0DPnhAZEwT4WY6XjHnGshl5VeEzzxMNasPSwsQACJhqRPU66U4kS4Rt3w/T4MT0MlM/6BLNNKYMY9iSanzTruGd/j/5onqqP/QJj7XgwA+c8REVE2NJaza6ly8yhrJBfsQ3UyNGTcRkMMXjIlPx725NmiC8J5ucMRBXzU1MkGrmzf8hBJaBuR7jkFJ8/MnHcPCtFnec0C/thFY9a5FZxpNWkgXuo1F7v/XXGWKFJdbyF30jsRRaG5qRNFT5t/sJmp1q7PN01ehZifjTDRz/wtkf//gcKyqHBl9tAIPVXZVeP9ffz1bLxSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i5na+iiepRR7J2nHb//fiHfdQUlvPFKc3XueqaxAQbM=; b=E6Qe+8+TnTyhcPOzA+rtSvI81FbnGHWKYjjMhVxsJqjmgUS/PxyS30SUyxPiaM4YCeAhoMUArVttCuRh7gHdf+qgJF9XjFHIPuNay7ttPgHzraaVwhIWRS9RLmMfYfGXTixZ7SuU1sd28f7bv/njhB0Jy2ua5L508eTyrrYfMnM= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:31:01 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:31:01 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana , Josef Bacik , David Sterba , Anand Jain Subject: [PATCH v3 17/18 stable-5.15.y] btrfs: fallback to blocking mode when doing async dio over multiple extents Date: Fri, 15 Apr 2022 06:28:55 +0800 Message-Id: <9127cbbcd2bf2f8efd46298d8799e36282e1a311.1649951733.git.anand.jain@oracle.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYBP286CA0016.JPNP286.PROD.OUTLOOK.COM (2603:1096:404:ce::28) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9621f74f-cce8-4835-6d74-08da1e667445 X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ckCWRK//sp2FQRmy78Xc3TtDwjXPgUVJGQ61v0rl03Pv+r3psX2rwE8THJoWOXSovd7qbyOxI9X0qsU0j8ssM3b2lT+z6HgitJ3Rwp3FNCscnl8Sd/J7ZZjsVkdrurgBHtJonKLqEACOy23Q1KVUr7mewRTnXYl8ASbXE1CtFJX2sbj9mpwnbzPuEMKOPsc0C24KIisSDLE26MUaCXYnlNAsP9rVdlw287tnhz4+Y2R2TCTCjYjfHB6A2vT6aJ3+5RYNb99trjydvD8A1beTQyz+5AVzJ0AlOyVxCf3POEEZSWrvtxHBWjnlAzESEAdYXBNSGvlvYGNQHZGbgWRPxBlw3NitBW0q3ArHQQfq9qyxW7WB+ejx2Exe9C3pSSl4P36wlfRSIb2GMgsfYnUO5kgx5C0yrgFq0g/Kn5PC15/jJRS/6ly9fqpWQbpWIAnD+jh+boJuYHY9ifd58iKKyBqzqqwDkAkcaTRN5SHwdVTrxrXknYSnjFuvjNkOTsKQ+zoMAQVeeij7kM6QZ2LDjX53vjrqT+UMn7xY48S42+GmiXaSkiu6NVOOeM1JahNSpKhN1li3H/BqsoojmF9YGuFs/G9JpFUvbQYEItpl8J8TsNxQ84MnG7fzDauY4YmVgxy7t3uMu3E77TS0twt/BcNOZGVzPKl046Ae0J3HLd7xnCxYngXEsjDL9T2yFr94u8Zu92jzzkz5hRPg+rO4uLN+1X0UOi21MeU0oNCmEAza68oz6vZiF3LPLeinVj6B7wVZuxfdOR+IY3yA2m5tzw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(966005)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(30864003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HuW+sQWOCDCXZANwx9utjrtNHQEjNhQd+aCfZDnAANNjWzreWKyGSlp26+3uNClCNUT4DFFil+2cy0rbL0M80X26Gb+AQWiPF2rOOllEufz8Rr8n3WvOFb7koZ9qF197oMJ1a4ZMX02DVOoipK0+ezomd0khBdhXYu9Xmg9CfXvZSOKdZfG1rPtX4qSmJmtjmV/J6bHt76nDNh4KPvYqcqFMVZK02D2PaHY0iqZPUApUvYlkUWr5WtPV1hbVeM30I1GaR/NSAU9sGTcGm5VPe3HmIxNpr59fOTLx9iYHGc+xIwLNKn/SLSWR/jag8DW6zC/MLgpcvdzgcRUwrYeL1wGoeJ4qCD99JqwDC0wMJjDzy1Fe33hY1+w1qEjq984IA6qwlIuko0C0HkdB+pGk65cs7VUPN2r/B9sRZSPgF6/bq/cyTvoF5ytIwHTizoV2To3wmwHb6XzqrVGTpdW9aZHGo9EXVtQ7zkbvzp63zexKiu6bJ/gCQDEmct+96nX9iqy+4xLPlM491uxUkcRWU4lz9WZFgJe8ulOR2siQUa2jy8b/XvXHeztOPEi+ZAGZHcN5SOoitTTNKC4xqouO2NUrxWSiCPdGJgVKCDkgLnwFhkUp9nAGjF4IBFvBgbFyBz1jD2m6Ecobb4PCfWpquys16EQkFn8LvNAELa2Best+C+zRL8hdIpkw7taVobVVhEerI4atniHtx+9IJs2+6ln3UUfW7RbvjE6CQ+w1rQozOGG6WSQZVZ40GgtCUczMEI6aicaINuEb7PW0GxZQFWC3HS399MY/ZFD5wBUgxr6XSLmlWG+0MQj4ylWphCJeclPWFlqIaw7iG05AvajD9x28TWOxNLEaECBC7XUX5Uy4ZMfvg3ZCX8Z+DvmBpfeVyjtjjoVmGPyIo9seTvPu/gpaxaoXmKWMqz1qvjk0O+b6ngCb6RUpAhBvP2FwpdDwwiHyDRd1BxIu7U03W8m27FenxE4CQJdYyjbrsSnkQplw2v7oB/ESm5ZkIMKkITDuTKR+rf0gOWFnJVeZzvkmharg00ZkuZla9xKbz3xuwM6CDCUNq1IjaVZtQUz3q2c971xoFsa7tfmelzLHuA8i1c3QxGUKJXs25TMd0pfPGaXyS9slud8dXb4acIBopnoJUSIuqwyA1O0S25OKnCzY91/RNK0rMQQHg6XBqSlyvSWS0V1Wj4gOcGbVvdprFVGB20LfpPqJ0qQyl+c7pvZKTKOLtge+YdacAMZnrPd7GAH53UCV0+AD/cOnd2nLj4KZGdzM9IvFrXvcByxfnYcxK+DQiMPG6vN2T2YqOhsud6BG0/DybYodiCugovjKfp9/C1FrvLFFVss554/aXGvgj7ztPhOlAxt8WJAcLdI1+VSIsHQe6UbQrKsY5wwe1Gzuaf13Ho50iUOht33hHD9RltZ2WlxfZdKosceg+VHm602ODG3mi+8lqZV8EwjMNuC1GL0bdD+AYEM263thDq3y5Der2/46G9zWuEPrqmXml9Ft1tdwDzTfcT7xo4mHSgXy1jRMk6TKFYWHnmxJICWYpwcjMTDd7872rJgm5zcQkcw2HzNAh4Ke0aeeTHV4Jn2sRkUhjp2MwaLcAIPOnv48x6lau9kuXpM0SxxkIEwpls1XSvQ53a9WEPXg6mfUuWqj+cP3ePV1EnGvlvUT3+AKTq8037IHMpUykn79OUCkbIBkfjcx1yqfmOv9vPmFgOJQLYgK/pUOYrK7/BvRLf/keDUOCAMGrQ+EUYQqJ3OMR15GoSDM6brCsuEKVr0fsv4B X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9621f74f-cce8-4835-6d74-08da1e667445 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:31:01.3957 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4bE0Gq89KbMrnxErUJdOZ5WEvtolqhps2+3mU0UMsOPH8LZbmh6xAUWEt17bubFssDb64S6PNen2NmnUaEhECw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: _kB-nOiXFBzAYPXOkNW-TarH2vzJzK_5 X-Proofpoint-GUID: _kB-nOiXFBzAYPXOkNW-TarH2vzJzK_5 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Filipe Manana commit ca93e44bfb5fd7996b76f0f544999171f647f93b upstream Some users recently reported that MariaDB was getting a read corruption when using io_uring on top of btrfs. This started to happen in 5.16, after commit 51bd9563b6783d ("btrfs: fix deadlock due to page faults during direct IO reads and writes"). That changed btrfs to use the new iomap flag IOMAP_DIO_PARTIAL and to disable page faults before calling iomap_dio_rw(). This was necessary to fix deadlocks when the iovector corresponds to a memory mapped file region. That type of scenario is exercised by test case generic/647 from fstests. For this MariaDB scenario, we attempt to read 16K from file offset X using IOCB_NOWAIT and io_uring. In that range we have 4 extents, each with a size of 4K, and what happens is the following: 1) btrfs_direct_read() disables page faults and calls iomap_dio_rw(); 2) iomap creates a struct iomap_dio object, its reference count is initialized to 1 and its ->size field is initialized to 0; 3) iomap calls btrfs_dio_iomap_begin() with file offset X, which finds the first 4K extent, and setups an iomap for this extent consisting of a single page; 4) At iomap_dio_bio_iter(), we are able to access the first page of the buffer (struct iov_iter) with bio_iov_iter_get_pages() without triggering a page fault; 5) iomap submits a bio for this 4K extent (iomap_dio_submit_bio() -> btrfs_submit_direct()) and increments the refcount on the struct iomap_dio object to 2; The ->size field of the struct iomap_dio object is incremented to 4K; 6) iomap calls btrfs_iomap_begin() again, this time with a file offset of X + 4K. There we setup an iomap for the next extent that also has a size of 4K; 7) Then at iomap_dio_bio_iter() we call bio_iov_iter_get_pages(), which tries to access the next page (2nd page) of the buffer. This triggers a page fault and returns -EFAULT; 8) At __iomap_dio_rw() we see the -EFAULT, but we reset the error to 0 because we passed the flag IOMAP_DIO_PARTIAL to iomap and the struct iomap_dio object has a ->size value of 4K (we submitted a bio for an extent already). The 'wait_for_completion' variable is not set to true, because our iocb has IOCB_NOWAIT set; 9) At the bottom of __iomap_dio_rw(), we decrement the reference count of the struct iomap_dio object from 2 to 1. Because we were not the only ones holding a reference on it and 'wait_for_completion' is set to false, -EIOCBQUEUED is returned to btrfs_direct_read(), which just returns it up the callchain, up to io_uring; 10) The bio submitted for the first extent (step 5) completes and its bio endio function, iomap_dio_bio_end_io(), decrements the last reference on the struct iomap_dio object, resulting in calling iomap_dio_complete_work() -> iomap_dio_complete(). 11) At iomap_dio_complete() we adjust the iocb->ki_pos from X to X + 4K and return 4K (the amount of io done) to iomap_dio_complete_work(); 12) iomap_dio_complete_work() calls the iocb completion callback, iocb->ki_complete() with a second argument value of 4K (total io done) and the iocb with the adjust ki_pos of X + 4K. This results in completing the read request for io_uring, leaving it with a result of 4K bytes read, and only the first page of the buffer filled in, while the remaining 3 pages, corresponding to the other 3 extents, were not filled; 13) For the application, the result is unexpected because if we ask to read N bytes, it expects to get N bytes read as long as those N bytes don't cross the EOF (i_size). MariaDB reports this as an error, as it's not expecting a short read, since it knows it's asking for read operations fully within the i_size boundary. This is typical in many applications, but it may also be questionable if they should react to such short reads by issuing more read calls to get the remaining data. Nevertheless, the short read happened due to a change in btrfs regarding how it deals with page faults while in the middle of a read operation, and there's no reason why btrfs can't have the previous behaviour of returning the whole data that was requested by the application. The problem can also be triggered with the following simple program: /* Get O_DIRECT */ #ifndef _GNU_SOURCE #define _GNU_SOURCE #endif #include #include #include #include #include #include #include int main(int argc, char *argv[]) { char *foo_path; struct io_uring ring; struct io_uring_sqe *sqe; struct io_uring_cqe *cqe; struct iovec iovec; int fd; long pagesize; void *write_buf; void *read_buf; ssize_t ret; int i; if (argc != 2) { fprintf(stderr, "Use: %s \n", argv[0]); return 1; } foo_path = malloc(strlen(argv[1]) + 5); if (!foo_path) { fprintf(stderr, "Failed to allocate memory for file path\n"); return 1; } strcpy(foo_path, argv[1]); strcat(foo_path, "/foo"); /* * Create file foo with 2 extents, each with a size matching * the page size. Then allocate a buffer to read both extents * with io_uring, using O_DIRECT and IOCB_NOWAIT. Before doing * the read with io_uring, access the first page of the buffer * to fault it in, so that during the read we only trigger a * page fault when accessing the second page of the buffer. */ fd = open(foo_path, O_CREAT | O_TRUNC | O_WRONLY | O_DIRECT, 0666); if (fd == -1) { fprintf(stderr, "Failed to create file 'foo': %s (errno %d)", strerror(errno), errno); return 1; } pagesize = sysconf(_SC_PAGE_SIZE); ret = posix_memalign(&write_buf, pagesize, 2 * pagesize); if (ret) { fprintf(stderr, "Failed to allocate write buffer\n"); return 1; } memset(write_buf, 0xab, pagesize); memset(write_buf + pagesize, 0xcd, pagesize); /* Create 2 extents, each with a size matching page size. */ for (i = 0; i < 2; i++) { ret = pwrite(fd, write_buf + i * pagesize, pagesize, i * pagesize); if (ret != pagesize) { fprintf(stderr, "Failed to write to file, ret = %ld errno %d (%s)\n", ret, errno, strerror(errno)); return 1; } ret = fsync(fd); if (ret != 0) { fprintf(stderr, "Failed to fsync file\n"); return 1; } } close(fd); fd = open(foo_path, O_RDONLY | O_DIRECT); if (fd == -1) { fprintf(stderr, "Failed to open file 'foo': %s (errno %d)", strerror(errno), errno); return 1; } ret = posix_memalign(&read_buf, pagesize, 2 * pagesize); if (ret) { fprintf(stderr, "Failed to allocate read buffer\n"); return 1; } /* * Fault in only the first page of the read buffer. * We want to trigger a page fault for the 2nd page of the * read buffer during the read operation with io_uring * (O_DIRECT and IOCB_NOWAIT). */ memset(read_buf, 0, 1); ret = io_uring_queue_init(1, &ring, 0); if (ret != 0) { fprintf(stderr, "Failed to create io_uring queue\n"); return 1; } sqe = io_uring_get_sqe(&ring); if (!sqe) { fprintf(stderr, "Failed to get io_uring sqe\n"); return 1; } iovec.iov_base = read_buf; iovec.iov_len = 2 * pagesize; io_uring_prep_readv(sqe, fd, &iovec, 1, 0); ret = io_uring_submit_and_wait(&ring, 1); if (ret != 1) { fprintf(stderr, "Failed at io_uring_submit_and_wait()\n"); return 1; } ret = io_uring_wait_cqe(&ring, &cqe); if (ret < 0) { fprintf(stderr, "Failed at io_uring_wait_cqe()\n"); return 1; } printf("io_uring read result for file foo:\n\n"); printf(" cqe->res == %d (expected %d)\n", cqe->res, 2 * pagesize); printf(" memcmp(read_buf, write_buf) == %d (expected 0)\n", memcmp(read_buf, write_buf, 2 * pagesize)); io_uring_cqe_seen(&ring, cqe); io_uring_queue_exit(&ring); return 0; } When running it on an unpatched kernel: $ gcc io_uring_test.c -luring $ mkfs.btrfs -f /dev/sda $ mount /dev/sda /mnt/sda $ ./a.out /mnt/sda io_uring read result for file foo: cqe->res == 4096 (expected 8192) memcmp(read_buf, write_buf) == -205 (expected 0) After this patch, the read always returns 8192 bytes, with the buffer filled with the correct data. Although that reproducer always triggers the bug in my test vms, it's possible that it will not be so reliable on other environments, as that can happen if the bio for the first extent completes and decrements the reference on the struct iomap_dio object before we do the atomic_dec_and_test() on the reference at __iomap_dio_rw(). Fix this in btrfs by having btrfs_dio_iomap_begin() return -EAGAIN whenever we try to satisfy a non blocking IO request (IOMAP_NOWAIT flag set) over a range that spans multiple extents (or a mix of extents and holes). This avoids returning success to the caller when we only did partial IO, which is not optimal for writes and for reads it's actually incorrect, as the caller doesn't expect to get less bytes read than it has requested (unless EOF is crossed), as previously mentioned. This is also the type of behaviour that xfs follows (xfs_direct_write_iomap_begin()), even though it doesn't use IOMAP_DIO_PARTIAL. A test case for fstests will follow soon. Link: https://lore.kernel.org/linux-btrfs/CABVffEM0eEWho+206m470rtM0d9J8ue85TtR-A_oVTuGLWFicA@mail.gmail.com/ Link: https://lore.kernel.org/linux-btrfs/CAHF2GV6U32gmqSjLe=XKgfcZAmLCiH26cJ2OnHGp5x=VAH4OHQ@mail.gmail.com/ CC: stable@vger.kernel.org # 5.16+ Reviewed-by: Josef Bacik Signed-off-by: Filipe Manana Signed-off-by: David Sterba Signed-off-by: Anand Jain --- fs/btrfs/inode.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 58053b5f0ce1..af1819bdb609 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7955,6 +7955,34 @@ static int btrfs_dio_iomap_begin(struct inode *inode, loff_t start, } len = min(len, em->len - (start - em->start)); + + /* + * If we have a NOWAIT request and the range contains multiple extents + * (or a mix of extents and holes), then we return -EAGAIN to make the + * caller fallback to a context where it can do a blocking (without + * NOWAIT) request. This way we avoid doing partial IO and returning + * success to the caller, which is not optimal for writes and for reads + * it can result in unexpected behaviour for an application. + * + * When doing a read, because we use IOMAP_DIO_PARTIAL when calling + * iomap_dio_rw(), we can end up returning less data then what the caller + * asked for, resulting in an unexpected, and incorrect, short read. + * That is, the caller asked to read N bytes and we return less than that, + * which is wrong unless we are crossing EOF. This happens if we get a + * page fault error when trying to fault in pages for the buffer that is + * associated to the struct iov_iter passed to iomap_dio_rw(), and we + * have previously submitted bios for other extents in the range, in + * which case iomap_dio_rw() may return us EIOCBQUEUED if not all of + * those bios have completed by the time we get the page fault error, + * which we return back to our caller - we should only return EIOCBQUEUED + * after we have submitted bios for all the extents in the range. + */ + if ((flags & IOMAP_NOWAIT) && len < length) { + free_extent_map(em); + ret = -EAGAIN; + goto unlock_err; + } + if (write) { ret = btrfs_get_blocks_direct_write(&em, inode, dio_data, start, len); From patchwork Thu Apr 14 22:28:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 562191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFBA3C433EF for ; Thu, 14 Apr 2022 22:31:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240406AbiDNWdn (ORCPT ); Thu, 14 Apr 2022 18:33:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347214AbiDNWdj (ORCPT ); Thu, 14 Apr 2022 18:33:39 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFFE5C55A9; Thu, 14 Apr 2022 15:31:13 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 23ELRUOi001710; Thu, 14 Apr 2022 22:31:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=xGm141X5ORNOWlahahcMKFwVYYj6JXXRT2iVDZoGil95YaAyvruFNkWfTiUS2HTp6jr/ haO961PbMJaD9EeQMfCUhYIi68b7GYlglQmeBEShdh22VTetNmWHLpT1G9XeP3cOPUTT dbmPqa0tjGxtE9SuW89MmF5b8uvDxSuQI900MNuvkbuY1URNp3E/ywLnGK5jR4+VRUdF rkuxOsnaIrsZwrXQzdTqK3GYwRJ7s+dNChrbwihiX5cV9krG5pdKaqj3U6yRyLJpHHqk lLoLHlaPeMHfXuQCfKe4pD39vklgRJnZG5ymAVnxWaTk8+9duqUPcBSkNR4T82dVGxJv pg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com with ESMTP id 3fb0x2p3wc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:31:11 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 23EMFrdO006467; Thu, 14 Apr 2022 22:31:10 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2045.outbound.protection.outlook.com [104.47.56.45]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com with ESMTP id 3fb0k5p5tc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 14 Apr 2022 22:31:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AWhvZQM5WHxCn2xMBF4A2d/dQSjBtZ53jGGt2neTstkQ9iPqbgXCciRBCHkaL7qCJF62uiM8J0AkWk48LwHkmF0JUt1oq1BmmPb17iCr8Ys0wWtH9dJzN8I0sz7HVWhfFSIpn/GwAMfIqvV9a1XxY7fFIvKNPsE06D8y30Aa56O1BKSptVftVJawBQYNHZPRx7wZpKKHySV6i43UYNssBy7Httz8P9Q7zjh0McQZDofX8/ZraQIwRG9GNBdxNHX2CtcRl4vFgfjxmFSu79yqzUxetI9dKmGh3bSWtAG1c7RX5jvZdoD8XBDzT6ulTZAtUEeu09akwwgOI64VyEgStg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=gGkl0p3b9KXfQLwBpgegiiLokY0DXzi4uW67tGbd9Q0+3FRhcVH7qNuhPucQ+c7VmL5phILekNXftAEsTi/xoPFODkM3cX/dTPFiZMjRbbiCjrFX2F6lsrVGDQMTYrd9KmRJC2B/uZxwMX8n+3R1mxBmqZf431p9GjMNI+t2o+Kazom8TpxTICsUrSR0JgsATVMVahTh5llI2tO0zsqLXLGfVX9qLSS434TIV/2TqJPjSgMJdYY8B+Ag0r1QfXy5ZhLSTrPnD0ek0ytaWLZ0uIqZdtMW6KM2Sqjd8Ew/Ih4Gxx4j4YKdNttf8Ptb6XFuu8LqEAQOw9WoDaVPyITOgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3zfILGXASlYPIa/w4OJAu7wqk9O+xKx2AATdByT/LZg=; b=yNq9qLDjUzAHvkBxDJijbb9Q++zFvktU+vnGQxEV5WyVfOmpGvVP02ZzQQGMjI+ycHHYmHMRt/MYZ/CeOl86uwqqJ+ZvLGEVN/VTBxbHh+FEryc5RYhg65N3RvllNqP9Bs7KnfI2HCOcSOb7SqYYty4YXephlWB6tlA5rOdbu7c= Received: from PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) by CO6PR10MB5572.namprd10.prod.outlook.com (2603:10b6:303:147::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 22:31:07 +0000 Received: from PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306]) by PH0PR10MB5706.namprd10.prod.outlook.com ([fe80::d414:7654:e1b8:4306%7]) with mapi id 15.20.5144.029; Thu, 14 Apr 2022 22:31:07 +0000 From: Anand Jain To: stable@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Linus Torvalds , Andreas Gruenbacher , David Hildenbrand , Anand Jain Subject: [PATCH v3 18/18 stable-5.15.y] mm: gup: make fault_in_safe_writeable() use fixup_user_fault() Date: Fri, 15 Apr 2022 06:28:56 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: X-ClientProxiedBy: TYAPR03CA0007.apcprd03.prod.outlook.com (2603:1096:404:14::19) To PH0PR10MB5706.namprd10.prod.outlook.com (2603:10b6:510:148::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e144686a-0e39-4213-362b-08da1e667817 X-MS-TrafficTypeDiagnostic: CO6PR10MB5572:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lhec0CZDk0DOomx1GzZFTv1fiVo333IFOW8badi/XfAkS4HaisrRnVXPvkHGVJuCQ4zDhRzapkiRIFDBiGk37F2mg2GA1aLAfGSXTS1Qp9QEPni1pU1DS3U5LA4mVJ6Ti80l0YnCrwLVSqwY1YZhA9Ubh8HZ41KHY3FuenMCEAwlbQ2Z/HOjyLdBDLF8atXIi1jPWX7QCpgrU70gHu9A5i/MvHTIWF0KboMF4IfRrrb07jz6Jrjed/qzPm7ovX7G/jE6C89UgJ9hmLN35m9jbWPGCrkttoIzL5ZkvDaTvBAStQAwXmFvES0QrabSSWwmXqeGx133wbfruuNqu/fS6Lo/KJByZgjq55Ygqa7IBd5bUVFTOS/iV0jmuCdUD1kavTvHpYSRIhHkDyrV/cDpAYLzGmD0caYPUUAbdYolkBwyq8/PZQ2cQK8jBa2ECjTKOEKv7qQPu2+L51H55YtWqbmbG5OSAyeZO3lsSQViPfD4lyIpZsa/NQeQpEz7zt3AtSGB+val9jkVsnWl132dB+pRI0zcCxvhuIbXE4Up48imr+C2sFBtg9aJ0II+vuCfdx6pDdZ824tOZxQIOc8Zt4TQDCoqlKfQRsyqWwDaoll/9JcWPzawlL1xy5WKZBV+Mcx4b3bRWpaZVTJGbdCNxNnGvW8oIazR1ClE+pvR4ni4wB/r3o6ORTPvq5az8P387zCCcSBiw0JS/dpsgrkf4YOuMDmjuzc9crgIA7xD7io= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR10MB5706.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(6666004)(38100700002)(86362001)(6512007)(6506007)(2906002)(4326008)(36756003)(66556008)(66476007)(66946007)(508600001)(8676002)(6486002)(966005)(44832011)(83380400001)(107886003)(8936002)(6916009)(5660300002)(316002)(2616005)(54906003)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YmNFKm3GvpYvom9n/PCUh++09Zps7ARfksqJ3MTn9ig6J9AXrNzew6zQDD035TdfjnWrMqKV07TCjv0fiiEatNWDEutzYVGhtcw207NLu06iEuISf+CNdC8CprXK0cW36tyhD25wZbdeT13zLhlx4CO2OBoigvO/nisrgpWfS9N+NqDbbA+2+PSLS4/3hZdFaZfR2BP6xgyptFQeVGHZoo/6udljU32zti6sg+Nuntj/HmHSCZEMQl4eLhMBeAu/XWKf17JUt6szGr9u3NnXXzqwJ9PflR93Y7RNYJOjhpnWCSvAkrDwR6EsqvPwB+BSJeKUcnHzRinaAPjkdzAn0LvhkdJ1cpDy3IZY5sntIPtE/1ctRwyjzluALhO9MPcwhLgwFC+KSoqVUfIXcjOC/vrr//bwyPMjiEbcdvuKANOl/6rhXqVZBrgmGgy5mDXqoaj7H0BdG0ZmkvVpt43Jy5qvZNQS4gwAKRBnSyoZsJQBIqNcYcY0xBRjHL3/By4a9b+18sGEZoZVLC7mLiaAWSQgR9fL+gppqqKYzCdA0S6D3FyCp7s1fSRy+oLDgO+dgNlEizzZnk2nBhM2fbEQV1YJMrTw0Pon7dF0Tu079Ae8Nr0GGgA7Rjg+yrGArkFkh6ckNzopvQmM2KlCTS2PFM1C1dkK8VeaUUJyZZ44Yp10YDy/1/UeOJf2W1qUGoJeTKgpWFKmRzL/hzbBHWL01gxJ2706X9fDnFDKotLuaJ5i+7GkxIZJHFJHAnLvK3vTLDK6ZLV6+AT5whHnPY4tYSuYAOO3lGeLgV8a/65GNB3jprBlEHunr/4dF7gONKkjv5RqP5Zy+x8guKJQL0Da7F5cD9uKnh1ZUsgQ8hF36Xij7oagishxCOexw0EwTRgsy9ez2XM9v78EFz6jiQb15A6l+8xD7++ZpXmKrcqNw4vdDiZH9wS2j54COKTlvzvgRywa8IYxzPMaqAV/iaF7J0kcbR3eypq5CEcELTniX3jHCgrxIFQuzwZGGAHBJz6h/7LVnZrGu2GNpJmty5mCXyCEWZahaQiybu+3wAJRD0zA0k7bj7uSm/PrB0iryW1awQ0Kjlxf6MTo2xgnlZ0fcuK0T5f9TNh1DopOGJ2STpsS6+NTvoiLsI8xLA7jb80dkSEWkWjBIeDy4j72ORClI33wGoHTIlFXVZUvo4z20KN2CbiSgPhuyC93c5+QsKw9G83sjM8HuJQlyWyu2nS+yQKdLByGMYDv+UaqrALGAT3X+T9FkpOOCoX1+X3TpEPBK3Du6esSfZRQexTCJTLSeOUB7rWmxBL6g0C9N9+DOUqKY8bfgvyVHAUtm1I8bfD5mDeQsAobjkG01caSpeMCc6SpYHdBI6KGq3u9hYgnHZRc3yRQnbC8McC7wYj5HaUGtEB6oaH/Hd6WWgLM/lg3A+tFJ5EJPpG4tOYRvZFwALF9o1RNLfXHNs6YVffoL0MVUopT7fX4h/sdhwra/JKXMoLjGGcDjPvfuwib38kg75i/AIM2l8aCEUaywLsESJldIC8Y/O9jeLs4u9/JfAFnQWZO6gbb5qiXNh4GiKbhHtz4hbtvL+pA9oLNHfn3/PDKBrwMAluu13WGclEl/kH9eeEMM4+BuJT5QR1ohtCbkE4y2aSNU9RMCtwnfZ/cw/l6grN2iFRGwXQFxzmso3KsA+R6TyUQDbTGOssamX4IjcI0mstCyMp1INDc2QB1shMq1+gT6LEJJbH2QHwNT1Oq/8hWWrl3DYgN/6ZKM8DRwV7ogmtDQNsWUEC9FH9cVvb+ X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e144686a-0e39-4213-362b-08da1e667817 X-MS-Exchange-CrossTenant-AuthSource: PH0PR10MB5706.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 22:31:07.7724 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YdZ1ugUpRyWtfTVdD5HBLyERyADDfM4pWBwhUsa4lE6MPodRCCPnEbYfEnT9lcVpTBwmTUv6qhWaR/6wcjspSg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5572 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486, 18.0.858 definitions=2022-04-14_07:2022-04-14, 2022-04-14 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2204140116 X-Proofpoint-ORIG-GUID: D9IzGa18VGcwBKRFnz00RdpxB0TcXx8S X-Proofpoint-GUID: D9IzGa18VGcwBKRFnz00RdpxB0TcXx8S Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Linus Torvalds commit fe673d3f5bf1fc50cdc4b754831db91a2ec10126 upstream Instead of using GUP, make fault_in_safe_writeable() actually force a 'handle_mm_fault()' using the same fixup_user_fault() machinery that futexes already use. Using the GUP machinery meant that fault_in_safe_writeable() did not do everything that a real fault would do, ranging from not auto-expanding the stack segment, to not updating accessed or dirty flags in the page tables (GUP sets those flags on the pages themselves). The latter causes problems on architectures (like s390) that do accessed bit handling in software, which meant that fault_in_safe_writeable() didn't actually do all the fault handling it needed to, and trying to access the user address afterwards would still cause faults. Reported-and-tested-by: Andreas Gruenbacher Fixes: cdd591fc86e3 ("iov_iter: Introduce fault_in_iov_iter_writeable") Link: https://lore.kernel.org/all/CAHc6FU5nP+nziNGG0JAF1FUx-GV7kKFvM7aZuU_XD2_1v4vnvg@mail.gmail.com/ Acked-by: David Hildenbrand Signed-off-by: Linus Torvalds Signed-off-by: Anand Jain --- mm/gup.c | 57 +++++++++++++++++++------------------------------------- 1 file changed, 19 insertions(+), 38 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index a4c6affe6df3..ba2ab7a223f8 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1723,11 +1723,11 @@ EXPORT_SYMBOL(fault_in_writeable); * @uaddr: start of address range * @size: length of address range * - * Faults in an address range using get_user_pages, i.e., without triggering - * hardware page faults. This is primarily useful when we already know that - * some or all of the pages in the address range aren't in memory. + * Faults in an address range for writing. This is primarily useful when we + * already know that some or all of the pages in the address range aren't in + * memory. * - * Other than fault_in_writeable(), this function is non-destructive. + * Unlike fault_in_writeable(), this function is non-destructive. * * Note that we don't pin or otherwise hold the pages referenced that we fault * in. There's no guarantee that they'll stay in memory for any duration of @@ -1738,46 +1738,27 @@ EXPORT_SYMBOL(fault_in_writeable); */ size_t fault_in_safe_writeable(const char __user *uaddr, size_t size) { - unsigned long start = (unsigned long)untagged_addr(uaddr); - unsigned long end, nstart, nend; + unsigned long start = (unsigned long)uaddr, end; struct mm_struct *mm = current->mm; - struct vm_area_struct *vma = NULL; - int locked = 0; + bool unlocked = false; - nstart = start & PAGE_MASK; + if (unlikely(size == 0)) + return 0; end = PAGE_ALIGN(start + size); - if (end < nstart) + if (end < start) end = 0; - for (; nstart != end; nstart = nend) { - unsigned long nr_pages; - long ret; - if (!locked) { - locked = 1; - mmap_read_lock(mm); - vma = find_vma(mm, nstart); - } else if (nstart >= vma->vm_end) - vma = vma->vm_next; - if (!vma || vma->vm_start >= end) - break; - nend = end ? min(end, vma->vm_end) : vma->vm_end; - if (vma->vm_flags & (VM_IO | VM_PFNMAP)) - continue; - if (nstart < vma->vm_start) - nstart = vma->vm_start; - nr_pages = (nend - nstart) / PAGE_SIZE; - ret = __get_user_pages_locked(mm, nstart, nr_pages, - NULL, NULL, &locked, - FOLL_TOUCH | FOLL_WRITE); - if (ret <= 0) + mmap_read_lock(mm); + do { + if (fixup_user_fault(mm, start, FAULT_FLAG_WRITE, &unlocked)) break; - nend = nstart + ret * PAGE_SIZE; - } - if (locked) - mmap_read_unlock(mm); - if (nstart == end) - return 0; - return size - min_t(size_t, nstart - start, size); + start = (start + PAGE_SIZE) & PAGE_MASK; + } while (start != end); + mmap_read_unlock(mm); + + if (size > (unsigned long)uaddr - start) + return size - ((unsigned long)uaddr - start); + return 0; } EXPORT_SYMBOL(fault_in_safe_writeable);