From patchwork Mon Jul 10 07:06:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wangyijing X-Patchwork-Id: 107280 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp3039718qge; Mon, 10 Jul 2017 00:04:43 -0700 (PDT) X-Received: by 10.84.169.67 with SMTP id g61mr16326746plb.155.1499670283061; Mon, 10 Jul 2017 00:04:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499670283; cv=none; d=google.com; s=arc-20160816; b=CvojzbhIfXSDBp7M+kxEcTjVvtdOs24UopcwICl99x5AuHAFJD14cshcrBos9atwlr 2NgmnZwNURnVp2XgIN+H7gIX/NEAOo6EScyYjpaZimjhDzHVJAzSYrg59b6ccNmmsqyJ Ud58V+6hGeHwtBD5HmbNU0NE3KDImjLTY6IFnY4d8qwm2i6Xd98ne9uJ77/Mt4q0+rmK KHBYCpTnteVSF8pmTo9Wx/xOwbjpXHOU431bYxv+SPW8hj1VmX7c0F3BXF2EQL8CHrUf CtFaKFgSDPRkviCRiCVpuA5cjbb96WPrWejgEPZ4YNuuAoYxgT45WzJQogtv7VUhruO2 7UtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=V0MvnF74yA51nfxBagAVz3G3mMaNgrMpn4nUBqGC2RE=; b=ejihHOmJTQZFCaN65PRV5GNQyNH4ayEp2uamufTjCkbr2pFNewREDXEZgn64gU/FfL SV93B1kAgkIQ67TjP2TxTzMOTOlLZLg5jA2qcGxXIWr8SH8Vrh7n3dW/HMAhJ4XV2kyb 4sj54PfY4tuvlYQZEGX9RES0FR6c+xnXrc8lBymBfET3T/WOfmrBHQt4xk8h6+eunglf KGQe9rPDP0tR3l6G3st6+hk4lc/Plxs9PDGCIQ0MPB8Nq29LsHQ6WeeJcpGOXPi6Atc/ ZcNIfU7TVJ5K7ccJgfV6nM/JqRZ9QGWfhJudvP1TvBqAbXjsp2Qjb5OcK1ACNCF+gkpH KT2w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n189si4499168pgn.110.2017.07.10.00.04.42; Mon, 10 Jul 2017 00:04:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753525AbdGJHDr (ORCPT + 25 others); Mon, 10 Jul 2017 03:03:47 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:9296 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751045AbdGJHCU (ORCPT ); Mon, 10 Jul 2017 03:02:20 -0400 Received: from 172.30.72.54 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.54]) by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id ARU37600; Mon, 10 Jul 2017 15:02:11 +0800 (CST) Received: from 138.huawei.com (10.175.124.28) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 10 Jul 2017 15:02:02 +0800 From: Yijing Wang To: , CC: , , , , , , , , , , , , , , , , , , , Yijing Wang , "Johannes Thumshirn" Subject: [PATCH v3 1/7] libsas: Use static sas event pool to appease sas event lost Date: Mon, 10 Jul 2017 15:06:03 +0800 Message-ID: <1499670369-44143-2-git-send-email-wangyijing@huawei.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> References: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.28] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020205.59632675.0041, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: c76184ec645123ba136c7402f2eba364 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now libsas hotplug work is static, every sas event type has its own static work, LLDD driver queue the hotplug work into shost->work_q. If LLDD driver burst post lots hotplug events to libsas, the hotplug events may pending in the workqueue like shost->work_q new work[PORTE_BYTES_DMAED] --> |[PHYE_LOSS_OF_SIGNAL][PORTE_BYTES_DMAED] -> processing |<-------wait worker to process-------->| In this case, a new PORTE_BYTES_DMAED event coming, libsas try to queue it to shost->work_q, but this work is already pending, so it would be lost. Finally, libsas delete the related sas port and sas devices, but LLDD driver expect libsas add the sas port and devices(last sas event). This patch and use static sas event work pool to appease this issue, since it's static work pool, it won't make memory exhaust. Signed-off-by: Yijing Wang CC: John Garry CC: Johannes Thumshirn CC: Ewan Milne CC: Christoph Hellwig CC: Tomas Henzl CC: Dan Williams --- drivers/scsi/libsas/sas_event.c | 208 ++++++++++++++++++++++++++++++++----- drivers/scsi/libsas/sas_init.c | 6 -- drivers/scsi/libsas/sas_internal.h | 3 + drivers/scsi/libsas/sas_phy.c | 48 +++------ drivers/scsi/libsas/sas_port.c | 18 ++-- include/scsi/libsas.h | 16 +-- 6 files changed, 216 insertions(+), 83 deletions(-) -- 2.5.0 diff --git a/drivers/scsi/libsas/sas_event.c b/drivers/scsi/libsas/sas_event.c index c0d0d97..a1370bd 100644 --- a/drivers/scsi/libsas/sas_event.c +++ b/drivers/scsi/libsas/sas_event.c @@ -27,13 +27,20 @@ #include "sas_internal.h" #include "sas_dump.h" +static DEFINE_SPINLOCK(sas_event_lock); + +static const work_func_t sas_ha_event_fns[HA_NUM_EVENTS] = { + [HAE_RESET] = sas_hae_reset, +}; + int sas_queue_work(struct sas_ha_struct *ha, struct sas_work *sw) { int rc = 0; if (!test_bit(SAS_HA_REGISTERED, &ha->state)) - return 0; + return rc; + rc = 1; if (test_bit(SAS_HA_DRAINING, &ha->state)) { /* add it to the defer list, if not already pending */ if (list_empty(&sw->drain_node)) @@ -44,19 +51,15 @@ int sas_queue_work(struct sas_ha_struct *ha, struct sas_work *sw) return rc; } -static int sas_queue_event(int event, unsigned long *pending, - struct sas_work *work, +static int sas_queue_event(int event, struct sas_work *work, struct sas_ha_struct *ha) { int rc = 0; + unsigned long flags; - if (!test_and_set_bit(event, pending)) { - unsigned long flags; - - spin_lock_irqsave(&ha->lock, flags); - rc = sas_queue_work(ha, work); - spin_unlock_irqrestore(&ha->lock, flags); - } + spin_lock_irqsave(&ha->lock, flags); + rc = sas_queue_work(ha, work); + spin_unlock_irqrestore(&ha->lock, flags); return rc; } @@ -64,6 +67,8 @@ static int sas_queue_event(int event, unsigned long *pending, void __sas_drain_work(struct sas_ha_struct *ha) { + int ret; + unsigned long flags; struct workqueue_struct *wq = ha->core.shost->work_q; struct sas_work *sw, *_sw; @@ -78,7 +83,12 @@ void __sas_drain_work(struct sas_ha_struct *ha) clear_bit(SAS_HA_DRAINING, &ha->state); list_for_each_entry_safe(sw, _sw, &ha->defer_q, drain_node) { list_del_init(&sw->drain_node); - sas_queue_work(ha, sw); + ret = sas_queue_work(ha, sw); + if (ret != 1) { + spin_lock_irqsave(&sas_event_lock, flags); + sw->used = false; + spin_unlock_irqrestore(&sas_event_lock, flags); + } } spin_unlock_irq(&ha->lock); } @@ -119,51 +129,197 @@ void sas_enable_revalidation(struct sas_ha_struct *ha) if (!test_and_clear_bit(ev, &d->pending)) continue; - sas_queue_event(ev, &d->pending, &d->disc_work[ev].work, ha); + sas_queue_event(ev, &d->disc_work[ev].work, ha); } mutex_unlock(&ha->disco_mutex); } +static void sas_free_ha_event(struct sas_ha_event *event) +{ + unsigned long flags; + spin_lock_irqsave(&sas_event_lock, flags); + event->work.used = false; + spin_unlock_irqrestore(&sas_event_lock, flags); +} + +static void sas_free_port_event(struct asd_sas_event *event) +{ + unsigned long flags; + spin_lock_irqsave(&sas_event_lock, flags); + event->work.used = false; + spin_unlock_irqrestore(&sas_event_lock, flags); +} + +static void sas_free_phy_event(struct asd_sas_event *event) +{ + unsigned long flags; + spin_lock_irqsave(&sas_event_lock, flags); + event->work.used = false; + spin_unlock_irqrestore(&sas_event_lock, flags); +} + +static void sas_ha_event_worker(struct work_struct *work) +{ + struct sas_ha_event *ev = to_sas_ha_event(work); + + sas_ha_event_fns[ev->type](work); + sas_free_ha_event(ev); +} + +static void sas_port_event_worker(struct work_struct *work) +{ + struct asd_sas_event *ev = to_asd_sas_event(work); + + sas_port_event_fns[ev->type](work); + sas_free_port_event(ev); +} + +static void sas_phy_event_worker(struct work_struct *work) +{ + struct asd_sas_event *ev = to_asd_sas_event(work); + + sas_phy_event_fns[ev->type](work); + sas_free_phy_event(ev); +} + +static struct sas_ha_event *sas_alloc_ha_event(struct sas_ha_struct *sas_ha) +{ + int i; + unsigned long flags; + + spin_lock_irqsave(&sas_event_lock, flags); + for (i = 0; i < HA_NUM_EVENTS; i++) + if (!sas_ha->ha_events[i].work.used) + break; + + if (i == HA_NUM_EVENTS) { + spin_unlock_irqrestore(&sas_event_lock, flags); + return NULL; + } + + sas_ha->ha_events[i].work.used = true; + spin_unlock_irqrestore(&sas_event_lock, flags); + return &sas_ha->ha_events[i]; +} + static int notify_ha_event(struct sas_ha_struct *sas_ha, enum ha_event event) { + int ret; + struct sas_ha_event *ev; + BUG_ON(event >= HA_NUM_EVENTS); - return sas_queue_event(event, &sas_ha->pending, - &sas_ha->ha_events[event].work, sas_ha); + ev = sas_alloc_ha_event(sas_ha); + if (!ev) { + pr_err("%s: alloc sas ha event fail!\n", __func__); + return 0; + } + + INIT_SAS_WORK(&ev->work, sas_ha_event_worker); + ev->ha = sas_ha; + ev->type = event; + ret = sas_queue_event(event, &ev->work, sas_ha); + if (ret != 1) + sas_free_ha_event(ev); + + return ret; +} + +struct asd_sas_event *sas_alloc_port_event(struct asd_sas_phy *phy) +{ + int i; + unsigned long flags; + + spin_lock_irqsave(&sas_event_lock, flags); + for (i = 0; i < PORT_POOL_SIZE; i++) + { + if (!phy->port_events[i].work.used) + break; + } + + if (i == PORT_POOL_SIZE) { + spin_unlock_irqrestore(&sas_event_lock, flags); + return NULL; + } + + phy->port_events[i].work.used = true; + spin_unlock_irqrestore(&sas_event_lock, flags); + return &phy->port_events[i]; } static int notify_port_event(struct asd_sas_phy *phy, enum port_event event) { + int ret; + struct asd_sas_event *ev; struct sas_ha_struct *ha = phy->ha; BUG_ON(event >= PORT_NUM_EVENTS); - return sas_queue_event(event, &phy->port_events_pending, - &phy->port_events[event].work, ha); + ev = sas_alloc_port_event(phy); + if (!ev) { + pr_err("%s: alloc sas port event fail!\n", __func__); + return 0; + } + + INIT_SAS_WORK(&ev->work, sas_port_event_worker); + ev->phy = phy; + ev->type = event; + ret = sas_queue_event(event, &ev->work, ha); + if (ret != 1) + sas_free_port_event(ev); + + return ret; } +struct asd_sas_event *sas_alloc_phy_event(struct asd_sas_phy *phy) +{ + int i; + unsigned long flags; + + spin_lock_irqsave(&sas_event_lock, flags); + for (i = 0; i < PHY_POOL_SIZE; i++) + if (!phy->phy_events[i].work.used) + break; + + if (i == PHY_POOL_SIZE) { + spin_unlock_irqrestore(&sas_event_lock, flags); + return NULL; + } + + phy->phy_events[i].work.used = true; + spin_unlock_irqrestore(&sas_event_lock, flags); + return &phy->phy_events[i]; +} int sas_notify_phy_event(struct asd_sas_phy *phy, enum phy_event event) { + int ret; + struct asd_sas_event *ev; struct sas_ha_struct *ha = phy->ha; BUG_ON(event >= PHY_NUM_EVENTS); - return sas_queue_event(event, &phy->phy_events_pending, - &phy->phy_events[event].work, ha); + ev = sas_alloc_phy_event(phy); + if (!ev) { + pr_err("%s: alloc sas phy event fail!\n", __func__); + return 0; + } + + INIT_SAS_WORK(&ev->work, sas_phy_event_worker); + ev->phy = phy; + ev->type = event; + ret = sas_queue_event(event, &ev->work, ha); + if (ret != 1) + sas_free_phy_event(ev); + + return ret; } int sas_init_events(struct sas_ha_struct *sas_ha) { - static const work_func_t sas_ha_event_fns[HA_NUM_EVENTS] = { - [HAE_RESET] = sas_hae_reset, - }; - int i; - for (i = 0; i < HA_NUM_EVENTS; i++) { - INIT_SAS_WORK(&sas_ha->ha_events[i].work, sas_ha_event_fns[i]); - sas_ha->ha_events[i].ha = sas_ha; - } + for (i = 0; i < HA_NUM_EVENTS; i++) + sas_ha->ha_events[i].work.used = false; sas_ha->notify_ha_event = notify_ha_event; sas_ha->notify_port_event = notify_port_event; diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c index 64e9cdd..c227a8b 100644 --- a/drivers/scsi/libsas/sas_init.c +++ b/drivers/scsi/libsas/sas_init.c @@ -111,10 +111,6 @@ void sas_hash_addr(u8 *hashed, const u8 *sas_addr) void sas_hae_reset(struct work_struct *work) { - struct sas_ha_event *ev = to_sas_ha_event(work); - struct sas_ha_struct *ha = ev->ha; - - clear_bit(HAE_RESET, &ha->pending); } int sas_register_ha(struct sas_ha_struct *sas_ha) @@ -375,8 +371,6 @@ void sas_prep_resume_ha(struct sas_ha_struct *ha) struct asd_sas_phy *phy = ha->sas_phy[i]; memset(phy->attached_sas_addr, 0, SAS_ADDR_SIZE); - phy->port_events_pending = 0; - phy->phy_events_pending = 0; phy->frame_rcvd_size = 0; } } diff --git a/drivers/scsi/libsas/sas_internal.h b/drivers/scsi/libsas/sas_internal.h index a216c95..f03ce64 100644 --- a/drivers/scsi/libsas/sas_internal.h +++ b/drivers/scsi/libsas/sas_internal.h @@ -97,6 +97,9 @@ void sas_hae_reset(struct work_struct *work); void sas_free_device(struct kref *kref); +extern const work_func_t sas_phy_event_fns[PHY_NUM_EVENTS]; +extern const work_func_t sas_port_event_fns[PORT_NUM_EVENTS]; + #ifdef CONFIG_SCSI_SAS_HOST_SMP extern int sas_smp_host_handler(struct Scsi_Host *shost, struct request *req, struct request *rsp); diff --git a/drivers/scsi/libsas/sas_phy.c b/drivers/scsi/libsas/sas_phy.c index cdee446c..07766ad 100644 --- a/drivers/scsi/libsas/sas_phy.c +++ b/drivers/scsi/libsas/sas_phy.c @@ -35,7 +35,6 @@ static void sas_phye_loss_of_signal(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PHYE_LOSS_OF_SIGNAL, &phy->phy_events_pending); phy->error = 0; sas_deform_port(phy, 1); } @@ -45,7 +44,6 @@ static void sas_phye_oob_done(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PHYE_OOB_DONE, &phy->phy_events_pending); phy->error = 0; } @@ -58,8 +56,6 @@ static void sas_phye_oob_error(struct work_struct *work) struct sas_internal *i = to_sas_internal(sas_ha->core.shost->transportt); - clear_bit(PHYE_OOB_ERROR, &phy->phy_events_pending); - sas_deform_port(phy, 1); if (!port && phy->enabled && i->dft->lldd_control_phy) { @@ -88,8 +84,6 @@ static void sas_phye_spinup_hold(struct work_struct *work) struct sas_internal *i = to_sas_internal(sas_ha->core.shost->transportt); - clear_bit(PHYE_SPINUP_HOLD, &phy->phy_events_pending); - phy->error = 0; i->dft->lldd_control_phy(phy, PHY_FUNC_RELEASE_SPINUP_HOLD, NULL); } @@ -99,8 +93,6 @@ static void sas_phye_resume_timeout(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PHYE_RESUME_TIMEOUT, &phy->phy_events_pending); - /* phew, lldd got the phy back in the nick of time */ if (!phy->suspended) { dev_info(&phy->phy->dev, "resume timeout cancelled\n"); @@ -112,30 +104,12 @@ static void sas_phye_resume_timeout(struct work_struct *work) sas_deform_port(phy, 1); } - /* ---------- Phy class registration ---------- */ int sas_register_phys(struct sas_ha_struct *sas_ha) { int i; - static const work_func_t sas_phy_event_fns[PHY_NUM_EVENTS] = { - [PHYE_LOSS_OF_SIGNAL] = sas_phye_loss_of_signal, - [PHYE_OOB_DONE] = sas_phye_oob_done, - [PHYE_OOB_ERROR] = sas_phye_oob_error, - [PHYE_SPINUP_HOLD] = sas_phye_spinup_hold, - [PHYE_RESUME_TIMEOUT] = sas_phye_resume_timeout, - - }; - - static const work_func_t sas_port_event_fns[PORT_NUM_EVENTS] = { - [PORTE_BYTES_DMAED] = sas_porte_bytes_dmaed, - [PORTE_BROADCAST_RCVD] = sas_porte_broadcast_rcvd, - [PORTE_LINK_RESET_ERR] = sas_porte_link_reset_err, - [PORTE_TIMER_EVENT] = sas_porte_timer_event, - [PORTE_HARD_RESET] = sas_porte_hard_reset, - }; - /* Now register the phys. */ for (i = 0; i < sas_ha->num_phys; i++) { int k; @@ -143,15 +117,12 @@ int sas_register_phys(struct sas_ha_struct *sas_ha) phy->error = 0; INIT_LIST_HEAD(&phy->port_phy_el); - for (k = 0; k < PORT_NUM_EVENTS; k++) { - INIT_SAS_WORK(&phy->port_events[k].work, sas_port_event_fns[k]); - phy->port_events[k].phy = phy; - } - for (k = 0; k < PHY_NUM_EVENTS; k++) { - INIT_SAS_WORK(&phy->phy_events[k].work, sas_phy_event_fns[k]); - phy->phy_events[k].phy = phy; - } + for (k = 0; k < PORT_POOL_SIZE; k++) + phy->port_events[k].work.used = false; + + for (k = 0; k < PHY_POOL_SIZE; k++) + phy->phy_events[k].work.used = false; phy->port = NULL; phy->ha = sas_ha; @@ -179,3 +150,12 @@ int sas_register_phys(struct sas_ha_struct *sas_ha) return 0; } + +const work_func_t sas_phy_event_fns[PHY_NUM_EVENTS] = { + [PHYE_LOSS_OF_SIGNAL] = sas_phye_loss_of_signal, + [PHYE_OOB_DONE] = sas_phye_oob_done, + [PHYE_OOB_ERROR] = sas_phye_oob_error, + [PHYE_SPINUP_HOLD] = sas_phye_spinup_hold, + [PHYE_RESUME_TIMEOUT] = sas_phye_resume_timeout, + +}; diff --git a/drivers/scsi/libsas/sas_port.c b/drivers/scsi/libsas/sas_port.c index d3c5297..9326628 100644 --- a/drivers/scsi/libsas/sas_port.c +++ b/drivers/scsi/libsas/sas_port.c @@ -261,8 +261,6 @@ void sas_porte_bytes_dmaed(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PORTE_BYTES_DMAED, &phy->port_events_pending); - sas_form_port(phy); } @@ -273,8 +271,6 @@ void sas_porte_broadcast_rcvd(struct work_struct *work) unsigned long flags; u32 prim; - clear_bit(PORTE_BROADCAST_RCVD, &phy->port_events_pending); - spin_lock_irqsave(&phy->sas_prim_lock, flags); prim = phy->sas_prim; spin_unlock_irqrestore(&phy->sas_prim_lock, flags); @@ -288,8 +284,6 @@ void sas_porte_link_reset_err(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PORTE_LINK_RESET_ERR, &phy->port_events_pending); - sas_deform_port(phy, 1); } @@ -298,8 +292,6 @@ void sas_porte_timer_event(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PORTE_TIMER_EVENT, &phy->port_events_pending); - sas_deform_port(phy, 1); } @@ -308,8 +300,6 @@ void sas_porte_hard_reset(struct work_struct *work) struct asd_sas_event *ev = to_asd_sas_event(work); struct asd_sas_phy *phy = ev->phy; - clear_bit(PORTE_HARD_RESET, &phy->port_events_pending); - sas_deform_port(phy, 1); } @@ -353,3 +343,11 @@ void sas_unregister_ports(struct sas_ha_struct *sas_ha) sas_deform_port(sas_ha->sas_phy[i], 0); } + +const work_func_t sas_port_event_fns[PORT_NUM_EVENTS] = { + [PORTE_BYTES_DMAED] = sas_porte_bytes_dmaed, + [PORTE_BROADCAST_RCVD] = sas_porte_broadcast_rcvd, + [PORTE_LINK_RESET_ERR] = sas_porte_link_reset_err, + [PORTE_TIMER_EVENT] = sas_porte_timer_event, + [PORTE_HARD_RESET] = sas_porte_hard_reset, +}; diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h index cfaeed2..c41328d 100644 --- a/include/scsi/libsas.h +++ b/include/scsi/libsas.h @@ -229,6 +229,8 @@ struct domain_device { struct sas_work { struct list_head drain_node; struct work_struct work; + struct list_head list; + bool used; }; static inline void INIT_SAS_WORK(struct sas_work *sw, void (*fn)(struct work_struct *)) @@ -300,6 +302,7 @@ struct asd_sas_port { struct asd_sas_event { struct sas_work work; struct asd_sas_phy *phy; + int type; }; static inline struct asd_sas_event *to_asd_sas_event(struct work_struct *work) @@ -309,16 +312,16 @@ static inline struct asd_sas_event *to_asd_sas_event(struct work_struct *work) return ev; } +#define PORT_POOL_SIZE (PORT_NUM_EVENTS * 5) +#define PHY_POOL_SIZE (PHY_NUM_EVENTS * 5) + /* The phy pretty much is controlled by the LLDD. * The class only reads those fields. */ struct asd_sas_phy { /* private: */ - struct asd_sas_event port_events[PORT_NUM_EVENTS]; - struct asd_sas_event phy_events[PHY_NUM_EVENTS]; - - unsigned long port_events_pending; - unsigned long phy_events_pending; + struct asd_sas_event port_events[PORT_POOL_SIZE]; + struct asd_sas_event phy_events[PHY_POOL_SIZE]; int error; int suspended; @@ -365,6 +368,7 @@ struct scsi_core { struct sas_ha_event { struct sas_work work; struct sas_ha_struct *ha; + int type; }; static inline struct sas_ha_event *to_sas_ha_event(struct work_struct *work) @@ -384,8 +388,6 @@ enum sas_ha_state { struct sas_ha_struct { /* private: */ struct sas_ha_event ha_events[HA_NUM_EVENTS]; - unsigned long pending; - struct list_head defer_q; /* work queued while draining */ struct mutex drain_mutex; unsigned long state; From patchwork Mon Jul 10 07:06:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wangyijing X-Patchwork-Id: 107277 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp3038459qge; Mon, 10 Jul 2017 00:03:14 -0700 (PDT) X-Received: by 10.84.233.204 with SMTP id m12mr16935335pln.292.1499670194676; Mon, 10 Jul 2017 00:03:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499670194; cv=none; d=google.com; s=arc-20160816; b=hQJxO3MgtnctFTCf+FcuRyoFB84BYiz+h/H96zSnpCrE5j5f30XrXFUhsg2icd2rLT 7rLc9WqnG68pI+6TheudAyllqZaP3p8BvDJBrz0B8oZgLUqqQd3G07osyuagdub6PN4m Ohucw6Z5hoAgDl1p8A+a4e5ihcmyfE90rgJ7NEx16YUircnwp0cCEBXXjIhIiqSOflRY q1JAEDxxrgnHEmY4dEmywY4/CvKATfDPb9INfOKvSghI2+ptrlY4MGvpI0zMJ8W6AAJn AmeA5KTAImnlW0KtLWjMJ/vQZQcPmcEh+DAtSg5VVMkyNx2jwWNSBm6zoT6ROWADCv8u +tiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=+SOAvezOufk/ckMy+AfYCoEU/NEnBXJbkIjFoV05w1I=; b=nhun6fISYCmuIiDHnf8Fg+489shoa/xAKy0oSvqTVMMU88/paRt4HP33qVE9mZ2D7y SwgnXJ85bYocoimujmdjLHj4BFniDfq/YsMkPij4onXBQvpdHOEgUQBtRt8D8ujkrHO+ Qu7XRjy0kz9jBGOw4uhNa76KiCdERN+vzEG2T4p+9jVtWheHZmllVUKeFQKLkXv6uCFd csAF6OvyLEBejmdmKccTXTcobYWNHAQgQWID3odwsvouu//TxlBzd/NmTu3mvt80HT7R RRmhni+vFzDxwxJOXj4ZLGQuGMvrh6cWntvCyZ/CpxJgJdXvGW+HvkmtZbZSUNuR67M8 ZqSQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d184si7436669pgc.158.2017.07.10.00.03.14; Mon, 10 Jul 2017 00:03:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752892AbdGJHCZ (ORCPT + 25 others); Mon, 10 Jul 2017 03:02:25 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:9297 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751161AbdGJHCU (ORCPT ); Mon, 10 Jul 2017 03:02:20 -0400 Received: from 172.30.72.54 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.54]) by dggrg01-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id ARU37599; Mon, 10 Jul 2017 15:02:11 +0800 (CST) Received: from 138.huawei.com (10.175.124.28) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 10 Jul 2017 15:02:04 +0800 From: Yijing Wang To: , CC: , , , , , , , , , , , , , , , , , , , Yijing Wang , "Johannes Thumshirn" Subject: [PATCH v3 3/7] libsas: Use new workqueue to run sas event Date: Mon, 10 Jul 2017 15:06:05 +0800 Message-ID: <1499670369-44143-4-git-send-email-wangyijing@huawei.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> References: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.28] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020204.59632674.003A, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 2e3100d947df2923e6be90f2336e7895 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now all libsas works are queued to scsi host workqueue, include sas event work post by LLDD and sas discovery work, and a sas hotplug flow may be divided into several works, e.g libsas receive a PORTE_BYTES_DMAED event, now we process it as following steps: sas_form_port --- run in work in shot workq sas_discover_domain --- run in another work in shost workq ... sas_probe_devices --- run in new work in shost workq We found during hot-add a device, libsas may need run several works in same workqueue to add device in system, the process is not atomic, it may interrupt by other sas event works, like PHYE_LOSS_OF_SIGNAL. Finally, we would found lots unexpected errors. This patch is preparation of execute libsas sas event in sync. Signed-off-by: Yijing Wang CC: John Garry CC: Johannes Thumshirn CC: Ewan Milne CC: Christoph Hellwig CC: Tomas Henzl CC: Dan Williams --- drivers/scsi/libsas/sas_event.c | 4 ++-- drivers/scsi/libsas/sas_init.c | 7 +++++++ include/scsi/libsas.h | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) -- 2.5.0 Reviewed-by: Hannes Reinecke diff --git a/drivers/scsi/libsas/sas_event.c b/drivers/scsi/libsas/sas_event.c index a1370bd..a72a089 100644 --- a/drivers/scsi/libsas/sas_event.c +++ b/drivers/scsi/libsas/sas_event.c @@ -46,7 +46,7 @@ int sas_queue_work(struct sas_ha_struct *ha, struct sas_work *sw) if (list_empty(&sw->drain_node)) list_add(&sw->drain_node, &ha->defer_q); } else - rc = scsi_queue_work(ha->core.shost, &sw->work); + rc = queue_work(ha->event_q, &sw->work); return rc; } @@ -69,7 +69,7 @@ void __sas_drain_work(struct sas_ha_struct *ha) { int ret; unsigned long flags; - struct workqueue_struct *wq = ha->core.shost->work_q; + struct workqueue_struct *wq = ha->event_q; struct sas_work *sw, *_sw; set_bit(SAS_HA_DRAINING, &ha->state); diff --git a/drivers/scsi/libsas/sas_init.c b/drivers/scsi/libsas/sas_init.c index c227a8b..2f3b736 100644 --- a/drivers/scsi/libsas/sas_init.c +++ b/drivers/scsi/libsas/sas_init.c @@ -115,6 +115,7 @@ void sas_hae_reset(struct work_struct *work) int sas_register_ha(struct sas_ha_struct *sas_ha) { + char name[64]; int error = 0; mutex_init(&sas_ha->disco_mutex); @@ -146,6 +147,11 @@ int sas_register_ha(struct sas_ha_struct *sas_ha) goto Undo_ports; } + snprintf(name, 64, "%s_event_q", dev_name(sas_ha->dev)); + sas_ha->event_q = create_singlethread_workqueue(name); + if (!sas_ha->event_q) + goto Undo_ports; + INIT_LIST_HEAD(&sas_ha->eh_done_q); INIT_LIST_HEAD(&sas_ha->eh_ata_q); @@ -180,6 +186,7 @@ int sas_unregister_ha(struct sas_ha_struct *sas_ha) mutex_lock(&sas_ha->drain_mutex); __sas_drain_work(sas_ha); mutex_unlock(&sas_ha->drain_mutex); + destroy_workqueue(sas_ha->event_q); return 0; } diff --git a/include/scsi/libsas.h b/include/scsi/libsas.h index 628f48b..a01ca42 100644 --- a/include/scsi/libsas.h +++ b/include/scsi/libsas.h @@ -402,6 +402,7 @@ struct sas_ha_struct { char *sas_ha_name; struct device *dev; /* should be set */ struct module *lldd_module; /* should be set */ + struct workqueue_struct *event_q; u8 *sas_addr; /* must be set */ u8 hashed_sas_addr[HASHED_SAS_ADDR_SIZE]; From patchwork Mon Jul 10 07:06:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wangyijing X-Patchwork-Id: 107276 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp3038239qge; Mon, 10 Jul 2017 00:02:59 -0700 (PDT) X-Received: by 10.99.166.17 with SMTP id t17mr13543706pge.60.1499670179355; Mon, 10 Jul 2017 00:02:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499670179; cv=none; d=google.com; s=arc-20160816; b=G+iND9jlFIsD4NPKiDYoLrCS4Z2xcDJ7mXI7k9+qD/GwKC2+nPou+T1RMlmKgvmMBx bvVYGONHuNQ/J4M2QLg1kh5ykx5Xz7epuPbC7a9OEdw2HnZaLAWKvDUlq/oiPVV9Ah4T iwPFOvKHW2pMW2EQ1FNtrZAD75LbK72e4pgrhv0+pWulWSQaLt6LbqorlN7E1vKXwCXn bDNlJ6zgfIsuSL6tGM57wR1M0gxkCYhyvvwY5VJ+PSgbwYlaMt9QsDNCe5+PkfOXIa67 Ia/dzBffgZz/AwURFh2QI3nuxD2jt1ffGwGMP9njePJ14erp2y079d1+W89wRDGE45qs DM2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=RmJgRcEl+VahwCr5Aq4xncZkx+5PuiYnNtaoiq1323E=; b=STq6VB2CNpGOuXOxtTFSKkXZFgWkzcolq0H72fqFPGGaADe+1qturmDQjc7z2B1IRq rJ737EfDhixwo0W89azFh+BOaW/xS1mQOLVmCVauViE/kWQhtUo7BW+K/luib7Nl/Uoq djQ8L+JXypPP/rsx9gs/e79uvRH0jpMUnblW/ln+O+B+OrpuEyPmqxxpjeCjPqZjS6WN YCHeWgz23Ir0FK2ySYUF7BeZEZxSyV3rj1teJUK5FlQ3LuAAncVYJkXGtSKJRsai4W74 XW1daaNKFXYvCxuxfx1fNuwkuB1nMZmA4PBOowTYs4zLx7u4REC70YdROxrYLD5KNw4c b2nA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f5si7528255pgk.495.2017.07.10.00.02.59; Mon, 10 Jul 2017 00:02:59 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753446AbdGJHCc (ORCPT + 25 others); Mon, 10 Jul 2017 03:02:32 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:9814 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753413AbdGJHCa (ORCPT ); Mon, 10 Jul 2017 03:02:30 -0400 Received: from 172.30.72.53 (EHLO dggeml406-hub.china.huawei.com) ([172.30.72.53]) by dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued) with ESMTP id AQU38388; Mon, 10 Jul 2017 15:02:16 +0800 (CST) Received: from 138.huawei.com (10.175.124.28) by dggeml406-hub.china.huawei.com (10.3.17.50) with Microsoft SMTP Server id 14.3.301.0; Mon, 10 Jul 2017 15:02:08 +0800 From: Yijing Wang To: , CC: , , , , , , , , , , , , , , , , , , , Yijing Wang , "Johannes Thumshirn" Subject: [PATCH v3 7/7] libsas: release disco mutex during waiting in sas_ex_discover_end_dev Date: Mon, 10 Jul 2017 15:06:09 +0800 Message-ID: <1499670369-44143-8-git-send-email-wangyijing@huawei.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> References: <1499670369-44143-1-git-send-email-wangyijing@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.28] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A0B0204.59632679.0093, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: aab5a385a1f8cfa8ed9ea75cbe812aac Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Disco mutex was introudced to prevent domain rediscovery competing with ata error handling(87c8331). If we have already hold the lock in sas_revalidate_domain and sync executing probe, deadlock caused, because, sas_probe_sata() also need hold disco_mutex. Since disco mutex use to prevent revalidata domain happen during ata error handler, it should be safe to release disco mutex when sync probe, because no new revalidate domain event would be process until the sync return, and the current sas revalidate domain finish. Signed-off-by: Yijing Wang CC: John Garry CC: Johannes Thumshirn CC: Ewan Milne CC: Christoph Hellwig CC: Tomas Henzl CC: Dan Williams --- drivers/scsi/libsas/sas_expander.c | 10 ++++++++++ 1 file changed, 10 insertions(+) -- 2.5.0 Reviewed-by: Hannes Reinecke diff --git a/drivers/scsi/libsas/sas_expander.c b/drivers/scsi/libsas/sas_expander.c index 9d26c28..077024e 100644 --- a/drivers/scsi/libsas/sas_expander.c +++ b/drivers/scsi/libsas/sas_expander.c @@ -776,6 +776,7 @@ static struct domain_device *sas_ex_discover_end_dev( struct ex_phy *phy = &parent_ex->ex_phy[phy_id]; struct domain_device *child = NULL; struct sas_rphy *rphy; + bool prev_lock; int res; if (phy->attached_sata_host || phy->attached_sata_ps) @@ -803,6 +804,7 @@ static struct domain_device *sas_ex_discover_end_dev( sas_ex_get_linkrate(parent, child, phy); sas_device_set_phy(child, phy->port); + prev_lock = mutex_is_locked(&child->port->ha->disco_mutex); #ifdef CONFIG_SCSI_SAS_ATA if ((phy->attached_tproto & SAS_PROTOCOL_STP) || phy->attached_sata_dev) { res = sas_get_ata_info(child, phy); @@ -832,7 +834,11 @@ static struct domain_device *sas_ex_discover_end_dev( SAS_ADDR(parent->sas_addr), phy_id, res); goto out_list_del; } + if (prev_lock) + mutex_unlock(&child->port->ha->disco_mutex); sas_disc_wait_completion(child->port, DISCE_PROBE); + if (prev_lock) + mutex_lock(&child->port->ha->disco_mutex); } else #endif @@ -861,7 +867,11 @@ static struct domain_device *sas_ex_discover_end_dev( SAS_ADDR(parent->sas_addr), phy_id, res); goto out_list_del; } + if (prev_lock) + mutex_unlock(&child->port->ha->disco_mutex); sas_disc_wait_completion(child->port, DISCE_PROBE); + if (prev_lock) + mutex_lock(&child->port->ha->disco_mutex); } else { SAS_DPRINTK("target proto 0x%x at %016llx:0x%x not handled\n", phy->attached_tproto, SAS_ADDR(parent->sas_addr),