From patchwork Wed Oct 7 15:36:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 268534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6381C41604 for ; Wed, 7 Oct 2020 15:36:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DB602168B for ; Wed, 7 Oct 2020 15:36:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="rhF3yt9+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727909AbgJGPg0 (ORCPT ); Wed, 7 Oct 2020 11:36:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726849AbgJGPg0 (ORCPT ); Wed, 7 Oct 2020 11:36:26 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F012C061755 for ; Wed, 7 Oct 2020 08:36:26 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id n6so2759018ioc.12 for ; Wed, 07 Oct 2020 08:36:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version :content-language; bh=AagYrRGSBMaFGJNP3Dq9S8sQ4o72k/yjns7ayG+bjjc=; b=rhF3yt9+SgVMoWXkQs0N/GWU5SV7DUh+j61FVE0QOTxpV2RZ5IoBllk9MJjWBwJ5zM lfO0zsgNAgi0pyBysXUzpw0JDBIsTglU1MMkF3g90IPJk/mq43ZlBdUHvpija9Nnfb2h 05R4wwpjqIMnXIbBz0IbHC9Jq61ZrPN8tdUHTOD5lcJllsxnXcqw3c1jBwC61Y8DWfnK M/QzAnFEFSw9GN4m3UQNMzE5O7HLD3NIhCPi1vxHLVch8RbCZbg/vlaxwBlAMwXd2Dgf tKeVR6spvQgg2cDGnBdgc9Sgbfa8ulXqCH0DDka2Q38wcgfqoJuLlRwJAxrbY4LVKMv8 WOWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language; bh=AagYrRGSBMaFGJNP3Dq9S8sQ4o72k/yjns7ayG+bjjc=; b=KbcWcu2gPVb36jJUCz2ysKZrSqKLyIfAviiJi0sggzEZ3YFMLmGvhKEfGcLIw9PmvC 5ItZbWQyVrHKqFcaVhYr2QpQ2r2R7AP4CCPL40ZJQNqKb7wUHophmTCEXzUtE3FfvEyV al81a6QvDp0MZaGXhEnVWmGzShzNFr1HKKQjmRznbHBDNdxWiVTL7U492hQLszxR5rjv 7bkGSpRZO2nHVK8lfFnqLLbpFwffEaEk0sK+GIfHMcmEuF7CSGXFtDSAkxQo5j2uqTuq zAPMnzp/SkeVU1541hwxK2P7NcteItbLlgoqVONVDmkWjWsHBaUQSvYW32U984pTCqJR YJhA== X-Gm-Message-State: AOAM531PIubBCzHG6HgeWt9mFvd8myx3+r4VS+DfTrfwGv8jz2hgpiI1 SmixB4/e/TdrxipLGzrcw+ndK0QBb2xizw== X-Google-Smtp-Source: ABdhPJw+HWeblEMavp5SO9gwi41wL4FoSnsLxSVQrYytLF9iJYur9sS4bTyq9njSIgIHTTdvHt7Qsg== X-Received: by 2002:a05:6638:58e:: with SMTP id a14mr3371746jar.81.1602084985295; Wed, 07 Oct 2020 08:36:25 -0700 (PDT) Received: from [192.168.1.30] ([65.144.74.34]) by smtp.gmail.com with ESMTPSA id s17sm1240842ilb.24.2020.10.07.08.36.24 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 07 Oct 2020 08:36:24 -0700 (PDT) To: stable@vger.kernel.org From: Jens Axboe Subject: Request for inclusion for 5.4-stable Message-ID: <690b1e06-742d-6b1f-2f09-83edcc562d95@kernel.dk> Date: Wed, 7 Oct 2020 09:36:23 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org Hi, Can you queue up this series for 5.4-stable? It fixes some issues with an earlier patch that was queued up for 5.4-stable. Thanks! Signed-off-by: Yinyin Zhu Signed-off-by: Jens Axboe diff --git a/fs/io_uring.c b/fs/io_uring.c index 454cef93a39e8..2f46def7f5832 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -339,7 +339,7 @@ struct io_kiocb { u64 user_data; u32 result; u32 sequence; - struct task_struct *task; + struct files_struct *files; struct fs_struct *fs; @@ -513,7 +513,7 @@ static inline void io_queue_async_work(struct io_ring_ctx *ctx, } } - req->task = current; + req->files = current->files; spin_lock_irqsave(&ctx->task_lock, flags); list_add(&req->task_list, &ctx->task_list); @@ -2387,6 +2387,8 @@ static bool io_add_to_prev_work(struct async_list *list, struct io_kiocb *req) if (ret) { struct io_ring_ctx *ctx = req->ctx; + req->files = current->files; + spin_lock_irq(&ctx->task_lock); list_add(&req->task_list, &ctx->task_list); req->work_task = NULL; @@ -3717,7 +3719,7 @@ static int io_uring_fasync(int fd, struct file *file, int on) } static void io_cancel_async_work(struct io_ring_ctx *ctx, - struct task_struct *task) + struct files_struct *files) { if (list_empty(&ctx->task_list)) return; @@ -3729,7 +3731,7 @@ static void io_cancel_async_work(struct io_ring_ctx *ctx, req = list_first_entry(&ctx->task_list, struct io_kiocb, task_list); list_del_init(&req->task_list); req->flags |= REQ_F_CANCEL; - if (req->work_task && (!task || req->task == task)) + if (req->work_task && (!files || req->files == files)) send_sig(SIGINT, req->work_task, 1); } spin_unlock_irq(&ctx->task_lock); @@ -3754,7 +3756,7 @@ static int io_uring_flush(struct file *file, void *data) struct io_ring_ctx *ctx = file->private_data; if (fatal_signal_pending(current) || (current->flags & PF_EXITING)) - io_cancel_async_work(ctx, current); + io_cancel_async_work(ctx, data); return 0; } -- 2.11.0 >From MAILER-DAEMON Wed Oct 7 14:39:43 2020 From: Muchun Song To: axboe@kernel.dk, viro@zeniv.linux.org.uk Cc: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhuyinyin@bytedance.com, Muchun Song Subject: [PATCH v3 2/4] io_uring: Fix missing smp_mb() in io_cancel_async_work() Date: Wed, 07 Oct 2020 11:16:33 +0800 Message-Id: <20201007031635.65295-3-songmuchun@bytedance.com> In-Reply-To: <20201007031635.65295-1-songmuchun@bytedance.com> References: <20201007031635.65295-1-songmuchun@bytedance.com> List-ID: X-Mailing-List: linux-block@vger.kernel.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit The store to req->flags and load req->work_task should not be reordering in io_cancel_async_work(). We should make sure that either we store REQ_F_CANCE flag to req->flags or we see the req->work_task setted in io_sq_wq_submit_work(). Fixes: 1c4404efcf2c ("io_uring: make sure async workqueue is canceled on exit") Signed-off-by: Muchun Song --- fs/io_uring.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 2f46def7f5832..5d9583e3d0d25 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2252,6 +2252,12 @@ static void io_sq_wq_submit_work(struct work_struct *work) if (!ret) { req->work_task = current; + + /* + * Pairs with the smp_store_mb() (B) in + * io_cancel_async_work(). + */ + smp_mb(); /* A */ if (req->flags & REQ_F_CANCEL) { ret = -ECANCELED; goto end_req; @@ -3730,7 +3736,15 @@ static void io_cancel_async_work(struct io_ring_ctx *ctx, req = list_first_entry(&ctx->task_list, struct io_kiocb, task_list); list_del_init(&req->task_list); - req->flags |= REQ_F_CANCEL; + + /* + * The below executes an smp_mb(), which matches with the + * smp_mb() (A) in io_sq_wq_submit_work() such that either + * we store REQ_F_CANCEL flag to req->flags or we see the + * req->work_task setted in io_sq_wq_submit_work(). + */ + smp_store_mb(req->flags, req->flags | REQ_F_CANCEL); /* B */ + if (req->work_task && (!files || req->files == files)) send_sig(SIGINT, req->work_task, 1); } -- 2.11.0 >From MAILER-DAEMON Wed Oct 7 14:39:43 2020 From: Muchun Song To: axboe@kernel.dk, viro@zeniv.linux.org.uk Cc: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhuyinyin@bytedance.com, Muchun Song Subject: [PATCH v3 3/4] io_uring: Fix remove irrelevant req from the task_list Date: Wed, 07 Oct 2020 11:16:34 +0800 Message-Id: <20201007031635.65295-4-songmuchun@bytedance.com> In-Reply-To: <20201007031635.65295-1-songmuchun@bytedance.com> References: <20201007031635.65295-1-songmuchun@bytedance.com> List-ID: X-Mailing-List: linux-block@vger.kernel.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit If the process 0 has been initialized io_uring is complete, and then fork process 1. If process 1 exits and it leads to delete all reqs from the task_list. If we kill process 0. We will not send SIGINT signal to the kworker. So we can not remove the req from the task_list. The io_sq_wq_submit_work() can do that for us. Fixes: 1c4404efcf2c ("io_uring: make sure async workqueue is canceled on exit") Signed-off-by: Muchun Song --- fs/io_uring.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 5d9583e3d0d25..c65f78f395655 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2277,13 +2277,11 @@ static void io_sq_wq_submit_work(struct work_struct *work) break; cond_resched(); } while (1); -end_req: - if (!list_empty(&req->task_list)) { - spin_lock_irq(&ctx->task_lock); - list_del_init(&req->task_list); - spin_unlock_irq(&ctx->task_lock); - } } +end_req: + spin_lock_irq(&ctx->task_lock); + list_del_init(&req->task_list); + spin_unlock_irq(&ctx->task_lock); /* drop submission reference */ io_put_req(req); @@ -3727,15 +3725,16 @@ static int io_uring_fasync(int fd, struct file *file, int on) static void io_cancel_async_work(struct io_ring_ctx *ctx, struct files_struct *files) { + struct io_kiocb *req; + if (list_empty(&ctx->task_list)) return; spin_lock_irq(&ctx->task_lock); - while (!list_empty(&ctx->task_list)) { - struct io_kiocb *req; - req = list_first_entry(&ctx->task_list, struct io_kiocb, task_list); - list_del_init(&req->task_list); + list_for_each_entry(req, &ctx->task_list, task_list) { + if (files && req->files != files) + continue; /* * The below executes an smp_mb(), which matches with the @@ -3745,7 +3744,7 @@ static void io_cancel_async_work(struct io_ring_ctx *ctx, */ smp_store_mb(req->flags, req->flags | REQ_F_CANCEL); /* B */ - if (req->work_task && (!files || req->files == files)) + if (req->work_task) send_sig(SIGINT, req->work_task, 1); } spin_unlock_irq(&ctx->task_lock); -- 2.11.0 >From MAILER-DAEMON Wed Oct 7 14:39:43 2020 From: Muchun Song To: axboe@kernel.dk, viro@zeniv.linux.org.uk Cc: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, zhuyinyin@bytedance.com, Muchun Song , Jiachen Zhang Subject: [PATCH v3 4/4] io_uring: Fix double list add in io_queue_async_work() Date: Wed, 07 Oct 2020 11:16:35 +0800 Message-Id: <20201007031635.65295-5-songmuchun@bytedance.com> In-Reply-To: <20201007031635.65295-1-songmuchun@bytedance.com> References: <20201007031635.65295-1-songmuchun@bytedance.com> List-ID: X-Mailing-List: linux-block@vger.kernel.org MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit If we queue work in io_poll_wake(), it will leads to list double add. So we should add the list when the callback func is the io_sq_wq_submit_work. The following oops was seen: list_add double add: new=ffff9ca6a8f1b0e0, prev=ffff9ca62001cee8, next=ffff9ca6a8f1b0e0. ------------[ cut here ]------------ kernel BUG at lib/list_debug.c:31! Call Trace: io_poll_wake+0xf3/0x230 __wake_up_common+0x91/0x170 __wake_up_common_lock+0x7a/0xc0 io_commit_cqring+0xea/0x280 ? blkcg_iolatency_done_bio+0x2b/0x610 io_cqring_add_event+0x3e/0x60 io_complete_rw+0x58/0x80 dio_complete+0x106/0x250 blk_update_request+0xa0/0x3b0 blk_mq_end_request+0x1a/0x110 blk_mq_complete_request+0xd0/0xe0 nvme_irq+0x129/0x270 [nvme] __handle_irq_event_percpu+0x7b/0x190 handle_irq_event_percpu+0x30/0x80 handle_irq_event+0x3c/0x60 handle_edge_irq+0x91/0x1e0 do_IRQ+0x4d/0xd0 common_interrupt+0xf/0xf Fixes: 1c4404efcf2c ("io_uring: make sure async workqueue is canceled on exit") Reported-by: Jiachen Zhang Signed-off-by: Muchun Song --- fs/io_uring.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index c65f78f395655..a7cfe976480d8 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -513,12 +513,14 @@ static inline void io_queue_async_work(struct io_ring_ctx *ctx, } } - req->files = current->files; + if (req->work.func == io_sq_wq_submit_work) { + req->files = current->files; - spin_lock_irqsave(&ctx->task_lock, flags); - list_add(&req->task_list, &ctx->task_list); - req->work_task = NULL; - spin_unlock_irqrestore(&ctx->task_lock, flags); + spin_lock_irqsave(&ctx->task_lock, flags); + list_add(&req->task_list, &ctx->task_list); + req->work_task = NULL; + spin_unlock_irqrestore(&ctx->task_lock, flags); + } queue_work(ctx->sqo_wq[rw], &req->work); } @@ -667,6 +669,7 @@ static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx, state->cur_req++; } + INIT_LIST_HEAD(&req->task_list); req->file = NULL; req->ctx = ctx; req->flags = 0;