From patchwork Tue Feb 13 12:58:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 128226 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp4225133ljc; Tue, 13 Feb 2018 04:58:52 -0800 (PST) X-Google-Smtp-Source: AH8x226eVA1iaRtiIoJOI31OdKr2RoDzkJ23rGXQsKfD+usf7LlK7NxreHXYo7PazgTWMl1bLcq3 X-Received: by 10.98.48.2 with SMTP id w2mr1176649pfw.162.1518526732037; Tue, 13 Feb 2018 04:58:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518526732; cv=none; d=google.com; s=arc-20160816; b=fbhiQGyprmh3Ayq9hKppsYL/g/69iUimifIJMIagLNPjadWY8KmVIPgLm2GTKrLaEk HucWwk8OfbhGrpmTYc4B0BTQxe59slNR3EAAKwl8j/QYp0J1yn9HKCTi7edi4Zxd44mO gfogIchV6sdPNa/5rKZi9TUqoa04WgllhIx5L0p7fAQ93UgKzksFQQ6DOMINuXMqocO0 KhTjmPa4CKyHiBMNHhGdcELTAXq7Z7KKHP9dZ1Tt0CcMfvYHJbow/LS9xYN50+FHoC1t sbu11nN1X8/6NQT6z+AZpKfYy9g5eBFbvNZWcQAw1nBQjdmcC5ejFb1Jmjlj+GViHWhc Jjaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :arc-authentication-results; bh=B/bYN4jqxiFq7ckxTpjDf+IQFj4V/K77TbN5LZotyII=; b=v6U3t4zM7Xyv9rY2Vhiqh69HnoGORpt2j2N32pmyNCiZcq5HOK/FtdPWk2BBoCW5pY yIF3HIGAZsSB9OtyOoropgiIGEQbyqMa1GiaiZeHhZjxzn4Z2mJKYJ5J/tfH8T/Yc7fT xOmv8c2L7g1bnGcSTboJX8nwMCvSW+2fzcWxyNYwUpBzA5Xp0jeSHekoVWa/Ya2APXfc OKrdaky2wiyxA9xkQCqJUeTjVTCt8mb6kJ/T/RxIGLBhvb7X7CClIN0uDHLqky5o2EXg wh0J/BjQuxu3abHojs2Rj182+BZHhzQvLk3TgE098MHox87BC3x4ivdyeggAKixeWSV0 eGkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o21si973439pgv.679.2018.02.13.04.58.51; Tue, 13 Feb 2018 04:58:52 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964812AbeBMM6s (ORCPT + 28 others); Tue, 13 Feb 2018 07:58:48 -0500 Received: from foss.arm.com ([217.140.101.70]:57150 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935077AbeBMM6q (ORCPT ); Tue, 13 Feb 2018 07:58:46 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1FFE71529; Tue, 13 Feb 2018 04:58:46 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E60523F487; Tue, 13 Feb 2018 04:58:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 920F81AE545C; Tue, 13 Feb 2018 12:58:52 +0000 (GMT) From: Will Deacon To: linux-fsdevel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Will Deacon , Peter Zijlstra , Al Viro Subject: [PATCH] fs: dcache: Avoid livelock between d_alloc_parallel and __d_add Date: Tue, 13 Feb 2018 12:58:51 +0000 Message-Id: <1518526731-26546-1-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org If d_alloc_parallel runs concurrently with __d_add, it is possible for d_alloc_parallel to continuously retry whilst i_dir_seq has been incremented to an odd value by __d_add: CPU0: __d_add n = start_dir_add(dir); cmpxchg(&dir->i_dir_seq, n, n + 1) == n CPU1: d_alloc_parallel retry: seq = smp_load_acquire(&parent->d_inode->i_dir_seq) & ~1; hlist_bl_lock(b); bit_spin_lock(0, (unsigned long *)b); // Always succeeds CPU0: __d_lookup_done(dentry) hlist_bl_lock bit_spin_lock(0, (unsigned long *)b); // Never succeeds CPU1: if (unlikely(parent->d_inode->i_dir_seq != seq)) { hlist_bl_unlock(b); goto retry; } Since the simple bit_spin_lock used to implement hlist_bl_lock does not provide any fairness guarantees, then CPU1 can starve CPU0 of the lock and prevent it from reaching end_dir_add(dir), therefore CPU1 cannot exit its retry loop because the sequence number always has the bottom bit set. This patch resolves the livelock by not taking hlist_bl_lock in d_alloc_parallel if the sequence counter is odd, since any subsequent masked comparison with i_dir_seq will fail anyway. Cc: Peter Zijlstra Cc: Al Viro Signed-off-by: Will Deacon --- fs/dcache.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) -- 2.1.4 Acked-by: Peter Zijlstra (Intel) Reviewed-by: Matthew Wilcox diff --git a/fs/dcache.c b/fs/dcache.c index 7c38f39958bc..b243deec298c 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -2474,7 +2474,7 @@ struct dentry *d_alloc_parallel(struct dentry *parent, retry: rcu_read_lock(); - seq = smp_load_acquire(&parent->d_inode->i_dir_seq) & ~1; + seq = smp_load_acquire(&parent->d_inode->i_dir_seq); r_seq = read_seqbegin(&rename_lock); dentry = __d_lookup_rcu(parent, name, &d_seq); if (unlikely(dentry)) { @@ -2495,6 +2495,12 @@ struct dentry *d_alloc_parallel(struct dentry *parent, rcu_read_unlock(); goto retry; } + + if (unlikely(seq & 1)) { + rcu_read_unlock(); + goto retry; + } + hlist_bl_lock(b); if (unlikely(parent->d_inode->i_dir_seq != seq)) { hlist_bl_unlock(b);