From patchwork Sat Dec 11 00:28:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 524239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 087CAC433F5 for ; Sat, 11 Dec 2021 00:28:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241531AbhLKAcV (ORCPT ); Fri, 10 Dec 2021 19:32:21 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:54476 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241482AbhLKAcV (ORCPT ); Fri, 10 Dec 2021 19:32:21 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0B678B821CD; Sat, 11 Dec 2021 00:28:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B42B7C341C6; Sat, 11 Dec 2021 00:28:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639182523; bh=GXdn75I8saqXkrtakHw/QnPNWQzQUBkIRGRreApXKUI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=N3bV0e75JPvpTMggl8vPZ+FsDR9/Frm+MT2trZpFE9ppSGnQfzqenogcfE/kcGLXT a8c1pkeG/KLgOb9OwxtptL99NSzHN/mZelCm8WnlH6QTa7Bqk9RFst9uo4p9yflWLC 1wV5//1MkCaZtFiwCuQKiFj6zcjL3Wssangvk8bdEpcXiiU2nqID/4enn56E3eDFAy Z3ejSmlWz3aC8iuUUl6AEn2f9+TecSqimZS63Vcx5YBLLKnEegQJu6+2fhs8L7ueDy dHlmU7BGmcXY5M0GmhvTsoCMLjr0ByPSRtMKEdD5BfHHMloEj0FAANxE9Xyfw/OJvl jrnBND67+FQ/A== From: Eric Biggers To: stable@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Linus Torvalds Subject: [PATCH 4.4,4.9 1/3] wait: add wake_up_pollfree() Date: Fri, 10 Dec 2021 16:28:30 -0800 Message-Id: <20211211002832.153742-2-ebiggers@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20211211002832.153742-1-ebiggers@kernel.org> References: <20211211002832.153742-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Eric Biggers commit 42288cb44c4b5fff7653bc392b583a2b8bd6a8c0 upstream. Several ->poll() implementations are special in that they use a waitqueue whose lifetime is the current task, rather than the struct file as is normally the case. This is okay for blocking polls, since a blocking poll occurs within one task; however, non-blocking polls require another solution. This solution is for the queue to be cleared before it is freed, using 'wake_up_poll(wq, EPOLLHUP | POLLFREE);'. However, that has a bug: wake_up_poll() calls __wake_up() with nr_exclusive=1. Therefore, if there are multiple "exclusive" waiters, and the wakeup function for the first one returns a positive value, only that one will be called. That's *not* what's needed for POLLFREE; POLLFREE is special in that it really needs to wake up everyone. Considering the three non-blocking poll systems: - io_uring poll doesn't handle POLLFREE at all, so it is broken anyway. - aio poll is unaffected, since it doesn't support exclusive waits. However, that's fragile, as someone could add this feature later. - epoll doesn't appear to be broken by this, since its wakeup function returns 0 when it sees POLLFREE. But this is fragile. Although there is a workaround (see epoll), it's better to define a function which always sends POLLFREE to all waiters. Add such a function. Also make it verify that the queue really becomes empty after all waiters have been woken up. Reported-by: Linus Torvalds Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20211209010455.42744-2-ebiggers@kernel.org Signed-off-by: Eric Biggers --- include/linux/wait.h | 26 ++++++++++++++++++++++++++ kernel/sched/wait.c | 8 ++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 2408e8d5c05cc..e022ee95bd45c 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -202,6 +202,7 @@ void __wake_up_locked_key(wait_queue_head_t *q, unsigned int mode, void *key); void __wake_up_sync_key(wait_queue_head_t *q, unsigned int mode, int nr, void *key); void __wake_up_locked(wait_queue_head_t *q, unsigned int mode, int nr); void __wake_up_sync(wait_queue_head_t *q, unsigned int mode, int nr); +void __wake_up_pollfree(wait_queue_head_t *wq_head); void __wake_up_bit(wait_queue_head_t *, void *, int); int __wait_on_bit(wait_queue_head_t *, struct wait_bit_queue *, wait_bit_action_f *, unsigned); int __wait_on_bit_lock(wait_queue_head_t *, struct wait_bit_queue *, wait_bit_action_f *, unsigned); @@ -236,6 +237,31 @@ wait_queue_head_t *bit_waitqueue(void *, int); #define wake_up_interruptible_sync_poll(x, m) \ __wake_up_sync_key((x), TASK_INTERRUPTIBLE, 1, (void *) (m)) +/** + * wake_up_pollfree - signal that a polled waitqueue is going away + * @wq_head: the wait queue head + * + * In the very rare cases where a ->poll() implementation uses a waitqueue whose + * lifetime is tied to a task rather than to the 'struct file' being polled, + * this function must be called before the waitqueue is freed so that + * non-blocking polls (e.g. epoll) are notified that the queue is going away. + * + * The caller must also RCU-delay the freeing of the wait_queue_head, e.g. via + * an explicit synchronize_rcu() or call_rcu(), or via SLAB_DESTROY_BY_RCU. + */ +static inline void wake_up_pollfree(wait_queue_head_t *wq_head) +{ + /* + * For performance reasons, we don't always take the queue lock here. + * Therefore, we might race with someone removing the last entry from + * the queue, and proceed while they still hold the queue lock. + * However, rcu_read_lock() is required to be held in such cases, so we + * can safely proceed with an RCU-delayed free. + */ + if (waitqueue_active(wq_head)) + __wake_up_pollfree(wq_head); +} + #define ___wait_cond_timeout(condition) \ ({ \ bool __cond = (condition); \ diff --git a/kernel/sched/wait.c b/kernel/sched/wait.c index 9453efe9b25a6..133afaf05c3f4 100644 --- a/kernel/sched/wait.c +++ b/kernel/sched/wait.c @@ -10,6 +10,7 @@ #include #include #include +#include void __init_waitqueue_head(wait_queue_head_t *q, const char *name, struct lock_class_key *key) { @@ -156,6 +157,13 @@ void __wake_up_sync(wait_queue_head_t *q, unsigned int mode, int nr_exclusive) } EXPORT_SYMBOL_GPL(__wake_up_sync); /* For internal use only */ +void __wake_up_pollfree(wait_queue_head_t *wq_head) +{ + __wake_up(wq_head, TASK_NORMAL, 0, (void *)(POLLHUP | POLLFREE)); + /* POLLFREE must have cleared the queue. */ + WARN_ON_ONCE(waitqueue_active(wq_head)); +} + /* * Note: we use "set_current_state()" _after_ the wait-queue add, * because we need a memory barrier there on SMP, so that any From patchwork Sat Dec 11 00:28:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 523372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B46EC433EF for ; Sat, 11 Dec 2021 00:28:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345473AbhLKAcY (ORCPT ); Fri, 10 Dec 2021 19:32:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345465AbhLKAcX (ORCPT ); Fri, 10 Dec 2021 19:32:23 -0500 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93190C061746; Fri, 10 Dec 2021 16:28:47 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id DFA46CE2DDB; Sat, 11 Dec 2021 00:28:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F143BC341C8; Sat, 11 Dec 2021 00:28:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639182524; bh=qpi9l/PqhmH2sxL4+/VUW06naq+hTMA57sGIqqM7cOk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WJvA63xxl6Z0qb5/SDWRrVYBY3jb1S5IC7KYh3HecXZSZmbHa7spkpVl88DMquDtM h+ZWI7XZF7ekzVhDfLYUPNSEpw4SkSMK79Wb0h3XIayBV5kJu18l+2StO75ala1P0J DfDhWO1qccurq9IIommOPwPuntdquNl66OQAecX2vqz5pDXOFElHH4sgHspuV/GLsP +rtWhWjVYXUjAQsWROpwIqOWMJYxPAP5UlsVd5j5ujfEbWbsawEKTLUwoYbhdJVKiN 1Qyl4V1PcTAdx71tBvcD7MTnJQD/kxXaiNSPUfSUV1CnmogdqGauhe1uxknemSayKv V89/XCmQ+nzhg== From: Eric Biggers To: stable@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Linus Torvalds Subject: [PATCH 4.4,4.9 2/3] binder: use wake_up_pollfree() Date: Fri, 10 Dec 2021 16:28:31 -0800 Message-Id: <20211211002832.153742-3-ebiggers@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20211211002832.153742-1-ebiggers@kernel.org> References: <20211211002832.153742-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Eric Biggers commit a880b28a71e39013e357fd3adccd1d8a31bc69a8 upstream. wake_up_poll() uses nr_exclusive=1, so it's not guaranteed to wake up all exclusive waiters. Yet, POLLFREE *must* wake up all waiters. epoll and aio poll are fortunately not affected by this, but it's very fragile. Thus, the new function wake_up_pollfree() has been introduced. Convert binder to use wake_up_pollfree(). Reported-by: Linus Torvalds Fixes: f5cb779ba163 ("ANDROID: binder: remove waitqueue when thread exits.") Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20211209010455.42744-3-ebiggers@kernel.org Signed-off-by: Eric Biggers --- drivers/android/binder.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/drivers/android/binder.c b/drivers/android/binder.c index bf047f16fce9e..31a204ebfa6c5 100644 --- a/drivers/android/binder.c +++ b/drivers/android/binder.c @@ -2641,21 +2641,18 @@ static int binder_free_thread(struct binder_proc *proc, } /* - * If this thread used poll, make sure we remove the waitqueue - * from any epoll data structures holding it with POLLFREE. - * waitqueue_active() is safe to use here because we're holding - * the global lock. + * If this thread used poll, make sure we remove the waitqueue from any + * poll data structures holding it. */ - if ((thread->looper & BINDER_LOOPER_STATE_POLL) && - waitqueue_active(&thread->wait)) { - wake_up_poll(&thread->wait, POLLHUP | POLLFREE); - } + if (thread->looper & BINDER_LOOPER_STATE_POLL) + wake_up_pollfree(&thread->wait); /* - * This is needed to avoid races between wake_up_poll() above and - * and ep_remove_waitqueue() called for other reasons (eg the epoll file - * descriptor being closed); ep_remove_waitqueue() holds an RCU read - * lock, so we can be sure it's done after calling synchronize_rcu(). + * This is needed to avoid races between wake_up_pollfree() above and + * someone else removing the last entry from the queue for other reasons + * (e.g. ep_remove_wait_queue() being called due to an epoll file + * descriptor being closed). Such other users hold an RCU read lock, so + * we can be sure they're done after we call synchronize_rcu(). */ if (thread->looper & BINDER_LOOPER_STATE_POLL) synchronize_rcu(); From patchwork Sat Dec 11 00:28:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 523373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A7E4C433FE for ; Sat, 11 Dec 2021 00:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241482AbhLKAcW (ORCPT ); Fri, 10 Dec 2021 19:32:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244279AbhLKAcW (ORCPT ); Fri, 10 Dec 2021 19:32:22 -0500 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B49E5C061714; Fri, 10 Dec 2021 16:28:46 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 80C0AB82A4A; Sat, 11 Dec 2021 00:28:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A114C341CA; Sat, 11 Dec 2021 00:28:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1639182524; bh=XuaZ1fZ6TwT9fwMbUAdcVm8nv57KY1gx1t0F0FZ5++o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TDEJfKUPBqkEOTsZ9D35lUwCEG8DDVjBINAjB/pDg89ibdvTEkM4cZGDxUKY1lo86 8ShIaWg63ZuTczE3pekMNneJ/hLcghUF2yJOYgx5/nas3RNfsjVhC14+Kkj9HCUIzi 340DIgASHmULinIHKjgw9RBpa3ETlG6TExDtQR0xZ3lbHEzVVsPMydTpEfVndk1Vpn 4BM78qwYih9FL341priu19QTNYkRcoouyXPV25cTOpeIJCwKbH8+DblGukji2Idjsv WpGpq2XbVN3b1ETRM1QuOUgr8Ydbb48TnurP5QDO0cin8yBaA5YkwabKtdCq4o8p2z d4ucKqUpgn6nA== From: Eric Biggers To: stable@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Linus Torvalds Subject: [PATCH 4.4,4.9 3/3] signalfd: use wake_up_pollfree() Date: Fri, 10 Dec 2021 16:28:32 -0800 Message-Id: <20211211002832.153742-4-ebiggers@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20211211002832.153742-1-ebiggers@kernel.org> References: <20211211002832.153742-1-ebiggers@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Eric Biggers commit 9537bae0da1f8d1e2361ab6d0479e8af7824e160 upstream. wake_up_poll() uses nr_exclusive=1, so it's not guaranteed to wake up all exclusive waiters. Yet, POLLFREE *must* wake up all waiters. epoll and aio poll are fortunately not affected by this, but it's very fragile. Thus, the new function wake_up_pollfree() has been introduced. Convert signalfd to use wake_up_pollfree(). Reported-by: Linus Torvalds Fixes: d80e731ecab4 ("epoll: introduce POLLFREE to flush ->signalfd_wqh before kfree()") Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20211209010455.42744-4-ebiggers@kernel.org Signed-off-by: Eric Biggers --- fs/signalfd.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/fs/signalfd.c b/fs/signalfd.c index 270221fcef42c..9c5fa0ab5e0fe 100644 --- a/fs/signalfd.c +++ b/fs/signalfd.c @@ -34,17 +34,7 @@ void signalfd_cleanup(struct sighand_struct *sighand) { - wait_queue_head_t *wqh = &sighand->signalfd_wqh; - /* - * The lockless check can race with remove_wait_queue() in progress, - * but in this case its caller should run under rcu_read_lock() and - * sighand_cachep is SLAB_DESTROY_BY_RCU, we can safely return. - */ - if (likely(!waitqueue_active(wqh))) - return; - - /* wait_queue_t->func(POLLFREE) should do remove_wait_queue() */ - wake_up_poll(wqh, POLLHUP | POLLFREE); + wake_up_pollfree(&sighand->signalfd_wqh); } struct signalfd_ctx {