From patchwork Sat Jan 15 00:59:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jaegeuk Kim X-Patchwork-Id: 532563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D200C43217 for ; Sat, 15 Jan 2022 01:01:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229533AbiAOBBw (ORCPT ); Fri, 14 Jan 2022 20:01:52 -0500 Received: from sin.source.kernel.org ([145.40.73.55]:34450 "EHLO sin.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231296AbiAOBBv (ORCPT ); Fri, 14 Jan 2022 20:01:51 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 29620CE24BA for ; Sat, 15 Jan 2022 01:01:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 83FECC36AE9; Sat, 15 Jan 2022 01:01:48 +0000 (UTC) From: Jaegeuk Kim To: stable@vger.kernel.org Cc: Peter Zijlstra , Jaegeuk Kim Subject: [PATCH 1/7] locking/rwsem: Better collate rwsem_read_trylock() Date: Fri, 14 Jan 2022 16:59:40 -0800 Message-Id: <20220115005945.2125174-2-jaegeuk@google.com> X-Mailer: git-send-email 2.34.1.703.g22d0c6ccf7-goog In-Reply-To: <20220115005945.2125174-1-jaegeuk@google.com> References: <20220115005945.2125174-1-jaegeuk@google.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Peter Zijlstra commit 3379116a0ca965b00e6522c7ea3f16c9dbd8f9f9 upstream. All users of rwsem_read_trylock() do rwsem_set_reader_owned(sem) on success, move it into rwsem_read_trylock() proper. Cc: # 5.10 Signed-off-by: Peter Zijlstra (Intel) Link: https://lkml.kernel.org/r/20201207090243.GE3040@hirez.programming.kicks-ass.net Signed-off-by: Jaegeuk Kim --- kernel/locking/rwsem.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c index a163542d178e..5c0dc7ebace9 100644 --- a/kernel/locking/rwsem.c +++ b/kernel/locking/rwsem.c @@ -273,9 +273,16 @@ static inline void rwsem_set_nonspinnable(struct rw_semaphore *sem) static inline bool rwsem_read_trylock(struct rw_semaphore *sem) { long cnt = atomic_long_add_return_acquire(RWSEM_READER_BIAS, &sem->count); + if (WARN_ON_ONCE(cnt < 0)) rwsem_set_nonspinnable(sem); - return !(cnt & RWSEM_READ_FAILED_MASK); + + if (!(cnt & RWSEM_READ_FAILED_MASK)) { + rwsem_set_reader_owned(sem); + return true; + } + + return false; } /* @@ -1340,8 +1347,6 @@ static inline void __down_read(struct rw_semaphore *sem) if (!rwsem_read_trylock(sem)) { rwsem_down_read_slowpath(sem, TASK_UNINTERRUPTIBLE); DEBUG_RWSEMS_WARN_ON(!is_rwsem_reader_owned(sem), sem); - } else { - rwsem_set_reader_owned(sem); } } @@ -1351,8 +1356,6 @@ static inline int __down_read_interruptible(struct rw_semaphore *sem) if (IS_ERR(rwsem_down_read_slowpath(sem, TASK_INTERRUPTIBLE))) return -EINTR; DEBUG_RWSEMS_WARN_ON(!is_rwsem_reader_owned(sem), sem); - } else { - rwsem_set_reader_owned(sem); } return 0; } @@ -1363,8 +1366,6 @@ static inline int __down_read_killable(struct rw_semaphore *sem) if (IS_ERR(rwsem_down_read_slowpath(sem, TASK_KILLABLE))) return -EINTR; DEBUG_RWSEMS_WARN_ON(!is_rwsem_reader_owned(sem), sem); - } else { - rwsem_set_reader_owned(sem); } return 0; }