From patchwork Fri Mar 5 10:28:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suganath Prabu S X-Patchwork-Id: 394042 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84A28C4332B for ; Fri, 5 Mar 2021 10:30:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47F9364F76 for ; Fri, 5 Mar 2021 10:30:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229779AbhCEK3y (ORCPT ); Fri, 5 Mar 2021 05:29:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229792AbhCEK3m (ORCPT ); Fri, 5 Mar 2021 05:29:42 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6DE4C061574 for ; Fri, 5 Mar 2021 02:29:41 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id d8so1180059plg.10 for ; Fri, 05 Mar 2021 02:29:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=BPn4Mm+1FoD0Ay2cClEFu2WWoNdGrvapH+NAw6xIrfQ=; b=CQ86P4knEHXVSeFWGOkxxGbjtad2XUbKrmmz21I2g5IUiH68M59WUc9+UQPAs10WOl tCWnBKsIiCg0Y9MNLec34sqQM9isNXNNiK2VUm1kTDAzzdXgHz/FrnhR3YwzbU0Xay79 bbaTs+mDkyxacfrNYBDOHYIxkYC8CIF/RdpdI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=BPn4Mm+1FoD0Ay2cClEFu2WWoNdGrvapH+NAw6xIrfQ=; b=NbJ0pfaTQYNTTez2L04pBuAM61pE8L6vm6qsE60Y1APtyXau7gcuwVWzebQy0IS/Rn rYeO22lg2pEcT7um9xFxEEqWoVVH/rBaQ4FAh+4pBXONLb4y5bsDlZDLnUC7tL8in4Qu fYokYCeFE39irTIk3BZLU74lUIrgtVb+U1UIfjDNjKd7yHCn4WtI+a6L/HN8hxDNI5x9 C+0l8a/tEb5Bz6WWr0i91tbFGMI05E2d3o1v5Qoy3AgcP/6Jgts4oKVb31E4QjOHJIj6 VkELbZb3k723339RejTU+hKnLSSzKsO/7X/jqyA5+BK6sMjgh/NCjeW+N5HyAjKBvG9g vY7Q== X-Gm-Message-State: AOAM533KkkJsKr3bnprJZGQlphzo9vvvQbBuKr0GNu+7XhU13xgrVsbv uMY0JN0TH24ejBN7OQqbALfL9dDG2QQb2zXZUdUplquA2zvxfQLGn8wTbmf25fhl2WLqd/qczIR PkFq4/uHKTaa7PHUUJ59Ew9iy+ElEYSrE7VkEnhWwqBqsHKOL8Jf9+mOROnPRHKpEDjf68v+opY BiTJFtIn4ekFdJbQ/BlHY9 X-Google-Smtp-Source: ABdhPJwWiYrK+QifrhB9j2Q37cXVSOCK5124PjIkxrlfHnE1s9TL/8ZX4+fIpbLNydSUGKbsBZFOQg== X-Received: by 2002:a17:902:c14c:b029:e5:cd82:a0b with SMTP id 12-20020a170902c14cb02900e5cd820a0bmr8161487plj.34.1614940180801; Fri, 05 Mar 2021 02:29:40 -0800 (PST) Received: from dhcp-10-123-20-76.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id v15sm2015983pgl.44.2021.03.05.02.29.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Mar 2021 02:29:40 -0800 (PST) From: Suganath Prabu S To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: Sathya.Prakash@broadcom.com, sreekanth.reddy@broadcom.com, Suganath Prabu S Subject: [PATCH 2/7] mpt3sas: Handle chain buffer DMA allocations in same 4G region Date: Fri, 5 Mar 2021 15:58:59 +0530 Message-Id: <20210305102904.7560-3-suganath-prabu.subramani@broadcom.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210305102904.7560-1-suganath-prabu.subramani@broadcom.com> References: <20210305102904.7560-1-suganath-prabu.subramani@broadcom.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org According to MPI Specification Chain buffers should not cross 4GB boundary. So while allocating Chain buffers, if any buffer crosses the 4GB boundary then, * Release the already allocated memory pools and * Reallocate them by changing the DMA coherent mask to 32 bit. Signed-off-by: Suganath Prabu S --- drivers/scsi/mpt3sas/mpt3sas_base.c | 83 ++++++++++++++++++++--------- 1 file changed, 57 insertions(+), 26 deletions(-) diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c index f9e6f8e..7542f7a 100644 --- a/drivers/scsi/mpt3sas/mpt3sas_base.c +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c @@ -5481,6 +5481,52 @@ _base_allocate_pcie_sgl_pool(struct MPT3SAS_ADAPTER *ioc, u32 sz) return 0; } +/** + * _base_allocate_chain_dma_pool - Allocating DMA'able memory + * for chain dma pool. + * @ioc: Adapter object + * @sz: DMA Pool size + * @ctr: Chain tracker + * Return: 0 for success, non-zero for failure. + */ +static int +_base_allocate_chain_dma_pool(struct MPT3SAS_ADAPTER *ioc, u32 sz) +{ + int i = 0, j = 0; + struct chain_tracker *ctr; + + ioc->chain_dma_pool = dma_pool_create("chain pool", &ioc->pdev->dev, + ioc->chain_segment_sz, 16, 0); + if (!ioc->chain_dma_pool) + return -ENOMEM; + + for (i = 0; i < ioc->scsiio_depth; i++) { + for (j = ioc->chains_per_prp_buffer; + j < ioc->chains_needed_per_io; j++) { + ctr = &ioc->chain_lookup[i].chains_per_smid[j]; + ctr->chain_buffer = dma_pool_alloc(ioc->chain_dma_pool, + GFP_KERNEL, &ctr->chain_buffer_dma); + if (!ctr->chain_buffer) + return -EAGAIN; + if (!mpt3sas_check_same_4gb_region((long) + ctr->chain_buffer, ioc->chain_segment_sz)) { + ioc_err(ioc, + "Chain buffers are not in same 4G !!! Chain buff (0x%p) dma = (0x%llx)\n", + ctr->chain_buffer, + (unsigned long long)ctr->chain_buffer_dma); + ioc->use_32bit_dma = true; + return -EAGAIN; + } + } + } + dinitprintk(ioc, ioc_info(ioc, + "chain_lookup depth (%d), frame_size(%d), pool_size(%d kB)\n", + ioc->scsiio_depth, ioc->chain_segment_sz, ((ioc->scsiio_depth * + (ioc->chains_needed_per_io - ioc->chains_per_prp_buffer) * + ioc->chain_segment_sz))/1024)); + return 0; +} + /** * base_alloc_rdpq_dma_pool - Allocating DMA'able memory * for reply queues. @@ -5578,9 +5624,8 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc) u16 max_request_credit, nvme_blocks_needed; unsigned short sg_tablesize; u16 sge_size; - int i, j; + int i; int ret = 0, rc = 0; - struct chain_tracker *ct; dinitprintk(ioc, ioc_info(ioc, "%s\n", __func__)); @@ -5907,31 +5952,17 @@ _base_allocate_memory_pools(struct MPT3SAS_ADAPTER *ioc) total_sz += sz * ioc->scsiio_depth; } - ioc->chain_dma_pool = dma_pool_create("chain pool", &ioc->pdev->dev, - ioc->chain_segment_sz, 16, 0); - if (!ioc->chain_dma_pool) { - ioc_err(ioc, "chain_dma_pool: dma_pool_create failed\n"); - goto out; - } - for (i = 0; i < ioc->scsiio_depth; i++) { - for (j = ioc->chains_per_prp_buffer; - j < ioc->chains_needed_per_io; j++) { - ct = &ioc->chain_lookup[i].chains_per_smid[j]; - ct->chain_buffer = dma_pool_alloc( - ioc->chain_dma_pool, GFP_KERNEL, - &ct->chain_buffer_dma); - if (!ct->chain_buffer) { - ioc_err(ioc, "chain_lookup: pci_pool_alloc failed\n"); - goto out; - } - } - total_sz += ioc->chain_segment_sz; - } - + rc = _base_allocate_chain_dma_pool(ioc, ioc->chain_segment_sz); + if (rc == -ENOMEM) + return -ENOMEM; + else if (rc == -EAGAIN) + goto try_32bit_dma; + total_sz += ioc->chain_segment_sz * ((ioc->chains_needed_per_io - + ioc->chains_per_prp_buffer) * ioc->scsiio_depth); dinitprintk(ioc, - ioc_info(ioc, "chain pool depth(%d), frame_size(%d), pool_size(%d kB)\n", - ioc->chain_depth, ioc->chain_segment_sz, - (ioc->chain_depth * ioc->chain_segment_sz) / 1024)); + ioc_info(ioc, "chain pool depth(%d), frame_size(%d), pool_size(%d kB)\n", + ioc->chain_depth, ioc->chain_segment_sz, + (ioc->chain_depth * ioc->chain_segment_sz) / 1024)); /* sense buffers, 4 byte align */ sz = ioc->scsiio_depth * SCSI_SENSE_BUFFERSIZE;