From patchwork Fri Jul 19 07:39:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 813611 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E9EB4D8A7 for ; Fri, 19 Jul 2024 07:39:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721374756; cv=none; b=EScp2QVM+i74tSTlhWwfUryFa9a8In17c7TAqe2lV7pfQ1iE1ue5PiW0wHJw2KB7D3zzoYTwYnC+sjFsmzT2u6BJgu4ttuJJoPYaOfL5wk3ynqtmFIZzpVsUdSlkljlEJBqdNwftCaYX9/IZUS6qf4CbgX2VZ6M5PX4c0Dlt7Hk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721374756; c=relaxed/simple; bh=WHhKfyC9miUT0pQix7K4cvCqHqfLQgZZnNq/08fxrl0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QdVb1t9cP7tNA6LAu3Qatpmg121j+mdgv0DoMNtfuzw8paUBUiNnZjBljL3aL5X01WuoR3gKYTxTf3XOYZcJ/x2Uux36+1uJldIKfWN/V+v10ukNt0vbUEJ0Q1nTwVo5ObfQUhTZYDiBlkk0wlFxyTYV3ko1yq2MYdLTj+8Bp38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=p163UGer; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="p163UGer" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5A17FC32782; Fri, 19 Jul 2024 07:39:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721374756; bh=WHhKfyC9miUT0pQix7K4cvCqHqfLQgZZnNq/08fxrl0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p163UGer6OFizhx2FPz3PlKWbcgK0SUchp9tCXsDIqG25XpYis8UUqtYg97xhRGe5 Y1HSjGM7dFTuLTX91YbrYNPA9hiIw+sak5lmEYzonlbBE/wsK8wlBqTlOk8tvm+qMg 46eH3N7hlCw8GO9X0Fv7WKuRQptTW5R2pvXUiYWiOV+bR9nqSzwkYb3IZolnD+a0bt YSYeLxWn0ERlOPAXUgjfPQdeaIJ7flwsNNdzXOcW3hg+VMeTsxaZgwMm7fTR3ASLDK JxG3a42COfkO9NxqeWIqr8t2e9iFT4KDmLZvhURfIgOZc6GXLOUGEiSssHbB3kqdUb F4pNDTEPV7HQA== From: Damien Le Moal To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Christoph Hellwig , Sathya Prakash , Sreekanth Reddy , Suganath Prabu Subramani Subject: [PATCH 1/2] scsi: mpi3mr: Avoid IOMMU page faults on report zones Date: Fri, 19 Jul 2024 16:39:11 +0900 Message-ID: <20240719073913.179559-2-dlemoal@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240719073913.179559-1-dlemoal@kernel.org> References: <20240719073913.179559-1-dlemoal@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some firmware versions of the 9600 series SAS HBA byte-swap the report zone command reply buffer from ATA-ZAC devices by directly accessing the buffer in the host memory. This does not respect the default command DMA direction and causes IOMMU page faults on architectures with an IOMMU enforcing write-only mappings for DMA_FROM_DEVICE DMA direction (e.g. AMD hosts), leading to the device capacity to be dropped to 0: scsi 18:0:58:0: Direct-Access-ZBC ATA WDC WSH722626AL W930 PQ: 0 ANSI: 7 scsi 18:0:58:0: Power-on or device reset occurred sd 18:0:58:0: Attached scsi generic sg9 type 20 sd 18:0:58:0: [sdj] Host-managed zoned block device mpi3mr 0000:c1:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0001 address=0xfec0c400 flags=0x0050] mpi3mr 0000:c1:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0001 address=0xfec0c500 flags=0x0050] sd 18:0:58:0: [sdj] REPORT ZONES start lba 0 failed sd 18:0:58:0: [sdj] REPORT ZONES: Result: hostbyte=DID_SOFT_ERROR driverbyte=DRIVER_OK sd 18:0:58:0: [sdj] 0 4096-byte logical blocks: (0 B/0 B) sd 18:0:58:0: [sdj] Write Protect is off sd 18:0:58:0: [sdj] Mode Sense: 6b 00 10 08 sd 18:0:58:0: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA sd 18:0:58:0: [sdj] Attached SCSI disk Avoid this issue by always mapping the buffer of report zones commands using DMA_BIDIRECTIONAL, that is, using a read-write IOMMU mapping. Suggested-by: Christoph Hellwig Fixes: 023ab2a9b4ed ("scsi: mpi3mr: Add support for queue command processing") Cc: stable@vger.kernel.org Signed-off-by: Damien Le Moal --- drivers/scsi/mpi3mr/mpi3mr_os.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi3mr_os.c b/drivers/scsi/mpi3mr/mpi3mr_os.c index bce639a6cca1..3a2ec4b87440 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_os.c +++ b/drivers/scsi/mpi3mr/mpi3mr_os.c @@ -3453,6 +3453,17 @@ static int mpi3mr_prepare_sg_scmd(struct mpi3mr_ioc *mrioc, scmd->sc_data_direction); priv->meta_sg_valid = 1; /* To unmap meta sg DMA */ } else { + /* + * Some firmware versions byte-swap the report zone command + * reply from ATA-ZAC devices by directly accessing in the host + * buffer. This does not respect the default command DMA + * direction and causes IOMMU page faults on some architectures + * with an IOMMU enforcing write mappings (e.g. AMD hosts). + * Avoid such issue by making the report zones buffer mapping + * bi-directional. + */ + if (scmd->cmnd[0] == ZBC_IN && scmd->cmnd[1] == ZI_REPORT_ZONES) + scmd->sc_data_direction = DMA_BIDIRECTIONAL; sg_scmd = scsi_sglist(scmd); sges_left = scsi_dma_map(scmd); } From patchwork Fri Jul 19 07:39:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Le Moal X-Patchwork-Id: 813525 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 936184D8A7 for ; Fri, 19 Jul 2024 07:39:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721374757; cv=none; b=SfGiaODzbvqrGySG0DeR/UZKIMc1GbFD5IKskoJjq7WhdGyF6V7YcyEP1MQTPTOTa5kOoMdObzYq6YgBeR6XhZsDuwVniEHYodQd4wNlFzLmasXpZNIKo8pNs7VD2r4iwnLl8xapda03K426rhwW93AmavfDvlquHqyoCgBvLII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721374757; c=relaxed/simple; bh=30Scoh1HtQifE4Nibzw22PN19pCDQTM6s9kautXJPHM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sjXiYHHVAUa7VOLHkrOV6WAV0FzvHLUMTit8KUfTIv6s/Gr2AIC3K43We9Nk47laUmF6pYN9+mMm9XdXi2jAoYhgzg/jfWAOu5dAvTizzCBZ2C8gJ8lRjexpjL0qabZ/pPuRhK1wismjsb7cVHagx8Bj6Gcy809HrnMGH2Trjsk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bo+OaNYx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bo+OaNYx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6384AC4AF0C; Fri, 19 Jul 2024 07:39:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721374757; bh=30Scoh1HtQifE4Nibzw22PN19pCDQTM6s9kautXJPHM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bo+OaNYxFdphTvgh2nkQnygMLsjo637T5IsaWBkm0/LbAbJHs8dOuKm7zqcjVN8TO x9UsfjbLd5Oo6v+3eyXVxeQAlO+FepR20++wR4jAE68m8tWfbVJ/7kE/g2PS2l40IY 2apiVqD0sd8eIVHGuG6b23hRq0drF5Jorfbky2x3OLCqMkmumqcueTb/KcnaLO+vfV Cja0YHPzbCSnvT6fFqYiOCnUieh1mhKSltZXE+LhnqU+mtgu8TPiZQjGE4Sbxl+m9W v1tY6v3skEBUghH0E3SGbIib38TcWRuhxeff1XgEjJ6L1Z8/A1QQYUMnKupMnUWr07 M7fUAOjQIl7QQ== From: Damien Le Moal To: "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Christoph Hellwig , Sathya Prakash , Sreekanth Reddy , Suganath Prabu Subramani Subject: [PATCH 2/2] scsi: mpt3sas: Avoid IOMMU page faults on report zones Date: Fri, 19 Jul 2024 16:39:12 +0900 Message-ID: <20240719073913.179559-3-dlemoal@kernel.org> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240719073913.179559-1-dlemoal@kernel.org> References: <20240719073913.179559-1-dlemoal@kernel.org> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some firmware versions of the 9600 series SAS HBA byte-swap the report zone command reply buffer from ATA-ZAC devices by directly accessing the buffer in the host memory. This does not respect the default command DMA direction and causes IOMMU page faults on architectures with an IOMMU enforcing write-only mappings for DMA_FROM_DEVICE DMA driection (e.g. AMD hosts). scsi 18:0:0:0: Direct-Access-ZBC ATA WDC WSH722020AL W870 PQ: 0 ANSI: 6 scsi 18:0:0:0: SATA: handle(0x0027), sas_addr(0x300062b2083e7c40), phy(0), device_name(0x5000cca29dc35e11) scsi 18:0:0:0: enclosure logical id (0x300062b208097c40), slot(0) scsi 18:0:0:0: enclosure level(0x0000), connector name( C0.0) scsi 18:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) scsi 18:0:0:0: qdepth(32), tagged(1), scsi_level(7), cmd_que(1) sd 18:0:0:0: Attached scsi generic sg2 type 20 sd 18:0:0:0: [sdc] Host-managed zoned block device mpt3sas 0000:41:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0xfff9b200 flags=0x0050] mpt3sas 0000:41:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0xfff9b300 flags=0x0050] mpt3sas_cm0: mpt3sas_ctl_pre_reset_handler: Releasing the trace buffer due to adapter reset. mpt3sas_cm0 fault info from func: mpt3sas_base_make_ioc_ready mpt3sas_cm0: fault_state(0x2666)! mpt3sas_cm0: sending diag reset !! mpt3sas_cm0: diag reset: SUCCESS sd 18:0:0:0: [sdc] REPORT ZONES start lba 0 failed sd 18:0:0:0: [sdc] REPORT ZONES: Result: hostbyte=DID_RESET driverbyte=DRIVER_OK sd 18:0:0:0: [sdc] 0 4096-byte logical blocks: (0 B/0 B) Avoid such issue by always mapping the buffer of report zones commands using DMA_BIDIRECTIONAL (read+write IOMMU mapping). This is done by introducing the helper function _base_scsi_dma_map() and using this helper in _base_build_sg_scmd() and _base_build_sg_scmd_ieee() instead of calling directly scsi_dma_map(). Fixes: 471ef9d4e498 ("mpt3sas: Build MPI SGL LIST on GEN2 HBAs and IEEE SGL LIST on GEN3 HBAs") Cc: stable@vger.kernel.org Signed-off-by: Damien Le Moal --- drivers/scsi/mpt3sas/mpt3sas_base.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index b2bcf4a27ddc..6e103875d672 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -2671,6 +2671,22 @@ _base_build_zero_len_sge_ieee(struct MPT3SAS_ADAPTER *ioc, void *paddr) _base_add_sg_single_ieee(paddr, sgl_flags, 0, 0, -1); } +static inline int _base_scsi_dma_map(struct scsi_cmnd *cmd) +{ + /* + * Some firmware versions byte-swap the report zone command reply from + * ATA-ZAC devices by directly accessing in the host buffer. This does + * not respect the default command DMA direction and causes IOMMU page + * faults on some architectures with an IOMMU enforcing write mappings + * (e.g. AMD hosts). Avoid such issue by making the report zones buffer + * mapping bi-directional. + */ + if (cmd->cmnd[0] == ZBC_IN && cmd->cmnd[1] == ZI_REPORT_ZONES) + cmd->sc_data_direction = DMA_BIDIRECTIONAL; + + return scsi_dma_map(cmd); +} + /** * _base_build_sg_scmd - main sg creation routine * pcie_device is unused here! @@ -2717,7 +2733,7 @@ _base_build_sg_scmd(struct MPT3SAS_ADAPTER *ioc, sgl_flags = sgl_flags << MPI2_SGE_FLAGS_SHIFT; sg_scmd = scsi_sglist(scmd); - sges_left = scsi_dma_map(scmd); + sges_left = _base_scsi_dma_map(scmd); if (sges_left < 0) return -ENOMEM; @@ -2861,7 +2877,7 @@ _base_build_sg_scmd_ieee(struct MPT3SAS_ADAPTER *ioc, } sg_scmd = scsi_sglist(scmd); - sges_left = scsi_dma_map(scmd); + sges_left = _base_scsi_dma_map(scmd); if (sges_left < 0) return -ENOMEM;