From patchwork Fri Mar 19 13:25:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Garry X-Patchwork-Id: 404793 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp1377661jai; Fri, 19 Mar 2021 06:31:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyeHvrQgkoEexx/L2bWn8Jd1nVwrERKuV8XEaHSzRm7wNoYUv9JaLidPfad7qVG2kAnNCri X-Received: by 2002:a05:6402:c:: with SMTP id d12mr9354196edu.100.1616160684980; Fri, 19 Mar 2021 06:31:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1616160684; cv=none; d=google.com; s=arc-20160816; b=wzwCXjm7CDWwZMiO91P4Q1gGSvtIdLFbRb+F7q1oorMKETssJDvj4M93Rsoj+sRm/X luyPLGiVtVj6GR9//2PHpmBgQbRDrSi+Wu4nbCra9qkLOBXAMTzQq7HCmiS+C54HouOc 2+s+mzrVYECvJSZQ/6b0kRq0zN6/EnFZtyEqSvwkr5otkLYLmTBVFZFMimke044Jos75 eB73GemNZYzBe1stS2cHZRAI8s1YE8XLURtvF52dTYuNRS1nINOuuTdh6XjuoKlsWPan Ii6OjYymRYYvnChdU245GDovm1qZOq/VIlkYBLdyLPWidgmdYKchOB1M4LKRlhczZOSI 10kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:subject:cc:to:from; bh=kOQCtt0D1C/8FjUlDyAWgCiVD3W52E659PQeU25XR4s=; b=pURwTe7pWaXlIgqzFl5A/tl21dV/ZQCCp5YOx/1P14zxtHRTASkQ+RiS10bX7PdlyB 4ZlJWstz0VdvZ1lCVaT5bEkfMnsNO6DArfCtXAfAkEz8Li0Lsa2aKTYln/Pe/vul23fE 55KaskL9n3GL9WPbd2XDJWEK1DwwuIA9WrcRGT0HPUl9Av4O2/5IlgjEhMMi3FSqaFTD NLZCiL8u1Q5o9Xzgu3v3NMs9OilU9alUQoZLUirIhLaPezYi8CKEcZkieaL3CdtdCiPY 2MyadQVQX9DC0vzKmjb7nU+a6pW+uNn1V0iIzgushcMpyXhfEFYo0tIdGi3hMoAIVJLc IKbg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=huawei.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m21si4292630ejx.725.2021.03.19.06.31.24 for ; Fri, 19 Mar 2021 06:31:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-scsi-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-scsi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=huawei.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230296AbhCSNar (ORCPT ); Fri, 19 Mar 2021 09:30:47 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:14022 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbhCSNaX (ORCPT ); Fri, 19 Mar 2021 09:30:23 -0400 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F24T25Y6CzPkQv; Fri, 19 Mar 2021 21:27:50 +0800 (CST) Received: from localhost.localdomain (10.69.192.58) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Fri, 19 Mar 2021 21:30:06 +0800 From: John Garry To: , , , , , , CC: , , , , John Garry Subject: [PATCH 0/6] dma mapping/iommu: Allow IOMMU IOVA rcache range to be configured Date: Fri, 19 Mar 2021 21:25:42 +0800 Message-ID: <1616160348-29451-1-git-send-email-john.garry@huawei.com> X-Mailer: git-send-email 2.8.1 MIME-Version: 1.0 X-Originating-IP: [10.69.192.58] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org For streaming DMA mappings involving an IOMMU and whose IOVA len regularly exceeds the IOVA rcache upper limit (meaning that they are not cached), performance can be reduced. This is much more pronounced from commit 4e89dce72521 ("iommu/iova: Retry from last rb tree node if iova search fails"), as discussed at [0]. IOVAs which cannot be cached are highly involved in the IOVA aging issue, as discussed at [1]. This series attempts to allow the device driver hint what upper limit its DMA mapping IOVA lengths would be, so that the caching range may be increased. Some figures on storage scenario: v5.12-rc3 baseline: 600K IOPS With series: 1300K IOPS With reverting 4e89dce72521: 1250K IOPS All above are for IOMMU strict mode. Non-strict mode gives ~1750K IOPS in all scenarios. I will say that APIs and their semantics are a bit ropey - any better ideas welcome... [0] https://lore.kernel.org/linux-iommu/20210129092120.1482-1-thunder.leizhen@huawei.com/ [1] https://lore.kernel.org/linux-iommu/1607538189-237944-1-git-send-email-john.garry@huawei.com/ John Garry (6): iommu: Move IOVA power-of-2 roundup into allocator iova: Add a per-domain count of reserved nodes iova: Allow rcache range upper limit to be configurable iommu: Add iommu_dma_set_opt_size() dma-mapping/iommu: Add dma_set_max_opt_size() scsi: hisi_sas: Set max optimal DMA size for v3 hw drivers/iommu/dma-iommu.c | 23 ++++--- drivers/iommu/iova.c | 88 ++++++++++++++++++++------ drivers/scsi/hisi_sas/hisi_sas_v3_hw.c | 2 + include/linux/dma-map-ops.h | 1 + include/linux/dma-mapping.h | 5 ++ include/linux/iova.h | 12 +++- kernel/dma/mapping.c | 11 ++++ 7 files changed, 115 insertions(+), 27 deletions(-) -- 2.26.2