From patchwork Sat Sep 14 00:19:08 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 20300 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-yh0-f72.google.com (mail-yh0-f72.google.com [209.85.213.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id EF75A20286 for ; Sat, 14 Sep 2013 00:19:20 +0000 (UTC) Received: by mail-yh0-f72.google.com with SMTP id z20sf2137111yhz.3 for ; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=Vo/dSQMr8Ym6T9enR8EYcyu/3dvJz8S71ZL4udJP6rA=; b=i7P4tB3JfQh5ssdonQdeU6xfOlKshGcvK5fe5wIV3vM/HbksAflSckejfEA00rmt05 T8Dj0cmEYGvah4tcVP7Xh/X1gGJVmkKElirU30G4Vn1+iMFQDgdY+n0bmV3ml5pjgyUL 9xWURNnuCaHFn+e7nbWn8AEqCzUA4WYCUC7XkdxyHJR7t/q9W0shJ0v1stEIDG7YbURc k3hlg24AIQXzB4d6cH+OO072ukcUHAdZ/BbKB1LzlZeA5mBC8wqFYnnPFXN+BzNEkDuw w0ZyQwnl7r7vG+H8ejAnNPhx6J2eDj6Ek1pmJd7Bz1sboAYxqfk9YJ5+q6+FEJABl1GI fpXQ== X-Gm-Message-State: ALoCoQmxeTHiLIv0Ok4hNlySpgAMp3ywhwftqDU8fdNuGX6elgdq/7o8QSujWDzb8hGujkdkXkZi X-Received: by 10.236.116.135 with SMTP id g7mr1627139yhh.53.1379117959712; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.64.68 with SMTP id m4ls318433qes.5.gmail; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) X-Received: by 10.58.146.71 with SMTP id ta7mr9511296veb.23.1379117959610; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) Received: from mail-vb0-f53.google.com (mail-vb0-f53.google.com [209.85.212.53]) by mx.google.com with ESMTPS id d14si3047963veh.39.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 13 Sep 2013 17:19:19 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.212.53 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.53; Received: by mail-vb0-f53.google.com with SMTP id i3so1461387vbh.40 for ; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) X-Received: by 10.220.10.194 with SMTP id q2mr14295855vcq.2.1379117959509; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp108395vcz; Fri, 13 Sep 2013 17:19:19 -0700 (PDT) X-Received: by 10.66.242.17 with SMTP id wm17mr18377504pac.102.1379117958051; Fri, 13 Sep 2013 17:19:18 -0700 (PDT) Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) by mx.google.com with ESMTPS id hk5si12978052pac.328.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 13 Sep 2013 17:19:18 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.192.175 is neither permitted nor denied by best guess record for domain of john.stultz@linaro.org) client-ip=209.85.192.175; Received: by mail-pd0-f175.google.com with SMTP id q10so1874898pdj.20 for ; Fri, 13 Sep 2013 17:19:17 -0700 (PDT) X-Received: by 10.68.197.3 with SMTP id iq3mr15912504pbc.113.1379117957645; Fri, 13 Sep 2013 17:19:17 -0700 (PDT) Received: from localhost.localdomain (c-67-170-153-23.hsd1.or.comcast.net. [67.170.153.23]) by mx.google.com with ESMTPSA id mr3sm14493935pbb.27.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 13 Sep 2013 17:19:17 -0700 (PDT) From: John Stultz To: lkml Cc: John Stultz , Li Zefan , Mathieu Desnoyers , Steven Rostedt , Peter Zijlstra , Ingo Molnar , Thomas Gleixner Subject: [PATCH 2/2] [RFC] cpuset: Fix potential deadlock w/ set_mems_allowed Date: Fri, 13 Sep 2013 17:19:08 -0700 Message-Id: <1379117948-7385-2-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1379117948-7385-1-git-send-email-john.stultz@linaro.org> References: <1379117948-7385-1-git-send-email-john.stultz@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: john.stultz@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.53 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , After adding lockdep support to seqlock/seqcount structures, I started seeing the following warning: [ 1.070907] ====================================================== [ 1.072015] [ INFO: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected ] [ 1.073181] 3.11.0+ #67 Not tainted [ 1.073801] ------------------------------------------------------ [ 1.074882] kworker/u4:2/708 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: [ 1.076088] (&p->mems_allowed_seq){+.+...}, at: [] new_slab+0x5f/0x280 [ 1.077572] [ 1.077572] and this task is already holding: [ 1.078593] (&(&q->__queue_lock)->rlock){..-...}, at: [] blk_execute_rq_nowait+0x53/0xf0 [ 1.080042] which would create a new lock dependency: [ 1.080042] (&(&q->__queue_lock)->rlock){..-...} -> (&p->mems_allowed_seq){+.+...} [ 1.080042] [ 1.080042] but this new dependency connects a SOFTIRQ-irq-safe lock: [ 1.080042] (&(&q->__queue_lock)->rlock){..-...} [ 1.080042] ... which became SOFTIRQ-irq-safe at: [ 1.080042] [] __lock_acquire+0x5b9/0x1db0 [ 1.080042] [] lock_acquire+0x95/0x130 [ 1.080042] [] _raw_spin_lock+0x41/0x80 [ 1.080042] [] scsi_device_unbusy+0x7e/0xd0 [ 1.080042] [] scsi_finish_command+0x32/0xf0 [ 1.080042] [] scsi_softirq_done+0xa1/0x130 [ 1.080042] [] blk_done_softirq+0x73/0x90 [ 1.080042] [] __do_softirq+0x110/0x2f0 [ 1.080042] [] run_ksoftirqd+0x2d/0x60 [ 1.080042] [] smpboot_thread_fn+0x156/0x1e0 [ 1.080042] [] kthread+0xd6/0xe0 [ 1.080042] [] ret_from_fork+0x7c/0xb0 [ 1.080042] [ 1.080042] to a SOFTIRQ-irq-unsafe lock: [ 1.080042] (&p->mems_allowed_seq){+.+...} [ 1.080042] ... which became SOFTIRQ-irq-unsafe at: [ 1.080042] ... [] __lock_acquire+0x613/0x1db0 [ 1.080042] [] lock_acquire+0x95/0x130 [ 1.080042] [] kthreadd+0x82/0x180 [ 1.080042] [] ret_from_fork+0x7c/0xb0 [ 1.080042] [ 1.080042] other info that might help us debug this: [ 1.080042] [ 1.080042] Possible interrupt unsafe locking scenario: [ 1.080042] [ 1.080042] CPU0 CPU1 [ 1.080042] ---- ---- [ 1.080042] lock(&p->mems_allowed_seq); [ 1.080042] local_irq_disable(); [ 1.080042] lock(&(&q->__queue_lock)->rlock); [ 1.080042] lock(&p->mems_allowed_seq); [ 1.080042] [ 1.080042] lock(&(&q->__queue_lock)->rlock); [ 1.080042] [ 1.080042] *** DEADLOCK *** The issue stems from the kthreadd() function calling set_mems_allowed with irqs enabled. While its possibly unlikely for the actual deadlock to trigger, a fix is fairly simple: disable irqs before taking the mems_allowed_seq lock. Let me know if you have any other suggestions or alternative fixes you'd prefer. Cc: Li Zefan Cc: Mathieu Desnoyers Cc: Steven Rostedt Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Thomas Gleixner Signed-off-by: John Stultz --- include/linux/cpuset.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h index cc1b01c..3fe661f 100644 --- a/include/linux/cpuset.h +++ b/include/linux/cpuset.h @@ -110,10 +110,14 @@ static inline bool put_mems_allowed(unsigned int seq) static inline void set_mems_allowed(nodemask_t nodemask) { + unsigned long flags; + task_lock(current); + local_irq_save(flags); write_seqcount_begin(¤t->mems_allowed_seq); current->mems_allowed = nodemask; write_seqcount_end(¤t->mems_allowed_seq); + local_irq_restore(flags); task_unlock(current); }