From patchwork Thu Sep 16 15:56:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 513970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA167C433F5 for ; Thu, 16 Sep 2021 16:32:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1C42613CF for ; Thu, 16 Sep 2021 16:32:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240819AbhIPQd1 (ORCPT ); Thu, 16 Sep 2021 12:33:27 -0400 Received: from mail.kernel.org ([198.145.29.99]:44660 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243386AbhIPQbY (ORCPT ); Thu, 16 Sep 2021 12:31:24 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 118C16138E; Thu, 16 Sep 2021 16:19:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1631809162; bh=t36PD241Vt04inE54Cxt7bIoDCPhYYCSm2Egul+soOs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aKlhDTt1O3/23o2YJ0GxrVXMZgxpsSdu5jGcCJvjxnWoCH/VaPJ/mQFVBTtjBjnYG Ugce8qoypS6rFoLFyVxLMdYhgvi3i2+INSgCjng4kubgSN23k7kNoTTe+Wfi1WSXWx W4Y5lnDUU1/fMyLmV3NFk9JIGMAyCZO7dFXcNMbU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Andres Freund , Jens Axboe Subject: [PATCH 5.13 054/380] io-wq: fix wakeup race when adding new work Date: Thu, 16 Sep 2021 17:56:51 +0200 Message-Id: <20210916155805.820023413@linuxfoundation.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210916155803.966362085@linuxfoundation.org> References: <20210916155803.966362085@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Jens Axboe commit 87df7fb922d18e96992aa5e824aa34b2065fef59 upstream. When new work is added, io_wqe_enqueue() checks if we need to wake or create a new worker. But that check is done outside the lock that otherwise synchronizes us with a worker going to sleep, so we can end up in the following situation: CPU0 CPU1 lock insert work unlock atomic_read(nr_running) != 0 lock atomic_dec(nr_running) no wakeup needed Hold the wqe lock around the "need to wakeup" check. Then we can also get rid of the temporary work_flags variable, as we know the work will remain valid as long as we hold the lock. Cc: stable@vger.kernel.org Reported-by: Andres Freund Signed-off-by: Jens Axboe Signed-off-by: Greg Kroah-Hartman --- fs/io-wq.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/fs/io-wq.c +++ b/fs/io-wq.c @@ -798,7 +798,7 @@ append: static void io_wqe_enqueue(struct io_wqe *wqe, struct io_wq_work *work) { struct io_wqe_acct *acct = io_work_get_acct(wqe, work); - int work_flags; + bool do_wake; unsigned long flags; /* @@ -811,14 +811,14 @@ static void io_wqe_enqueue(struct io_wqe return; } - work_flags = work->flags; raw_spin_lock_irqsave(&wqe->lock, flags); io_wqe_insert_work(wqe, work); wqe->flags &= ~IO_WQE_FLAG_STALLED; + do_wake = (work->flags & IO_WQ_WORK_CONCURRENT) || + !atomic_read(&acct->nr_running); raw_spin_unlock_irqrestore(&wqe->lock, flags); - if ((work_flags & IO_WQ_WORK_CONCURRENT) || - !atomic_read(&acct->nr_running)) + if (do_wake) io_wqe_wake_worker(wqe, acct); }