From patchwork Thu Jan 4 20:10:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 123451 Delivered-To: patch@linaro.org Received: by 10.80.135.92 with SMTP id 28csp7052edv; Thu, 4 Jan 2018 12:13:00 -0800 (PST) X-Google-Smtp-Source: ACJfBovj6UAgKVAxoF/3O1YipoZKhZwb/STCm8DxQzoRTCcLDcSmOzELLr/7EtzQ9WN4nYJ2uEgW X-Received: by 10.159.249.7 with SMTP id bf7mr683500plb.162.1515096780592; Thu, 04 Jan 2018 12:13:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515096780; cv=none; d=google.com; s=arc-20160816; b=lXrTXRehZU6DEB8SRo5izzqoeieSGhP6NIcYGObzdyTGScCvrbqjroFEV1MaSeTNLN 9X6y57b7++kD9/H/oXiIRPHtGDWUcY7xZf7d9ywbl6nwUgJXt0rllwQOfEuHUbOOKT2b kMwEenQrnvrt5uemM2g0i40NPjhRc1e1FZmTOLMYpctzQRMWno+FFjEeGiS3WAtqz3cQ UgqyICFmTr4IfG8y4KF3jcwbYo4RYBumsV4CECH9XLLIKL71QapZODIbgkF3i0q+6vZ5 A3dkvpoS/S3Nqu7nPet0qOHWAV8oeEetie7qWs159qBJob1sEeYkzJ2t+2WZEo5tt5Y1 +JOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=nzp7o30dL7OH60u8CymeIAmY18wE4P41lU4dfZM8bug=; b=af1nYvlvDovM4+OzekJi8NF0wnc2nmiBLwZjaGZoxyKN9KC/FsBZXU1CvI2pFtXUWM ZircOAkPhsFbNCavp84uZnXZyiyX3U5w9+0yI1aLmCaoK6yfI5LJXjVf0xKwFtEEqxKd 3PdQqoDX4y64DuE50uAI2QPNmDbWo0oI4NePHIde0luWex6tivZCcILffdBCc7ubP6je rSFmGhSj9Y6/9qrrfYyAxej7zUpZzwhIxProvUbYxup43WPpm+A9jsdlE6s/PqbsoN0I JwyC0f2haOucaqB3yhiJ0vx1zAUfbOPt+bNiq1GiR/8aTv6foMgc3Zr2V6bqRBZOebZl /VVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=MOeoGULt; spf=pass (google.com: best guess record for domain of linux-mmc-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-mmc-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n126si2470932pga.216.2018.01.04.12.13.00; Thu, 04 Jan 2018 12:13:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-mmc-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=MOeoGULt; spf=pass (google.com: best guess record for domain of linux-mmc-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-mmc-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752552AbeADUM7 (ORCPT + 6 others); Thu, 4 Jan 2018 15:12:59 -0500 Received: from mail-lf0-f67.google.com ([209.85.215.67]:40616 "EHLO mail-lf0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751946AbeADUM6 (ORCPT ); Thu, 4 Jan 2018 15:12:58 -0500 Received: by mail-lf0-f67.google.com with SMTP id u84so2990817lff.7 for ; Thu, 04 Jan 2018 12:12:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=msHpUMdlU2bD5nmeH1juHpb0E12kZ4laZq8qz+Ax6+Y=; b=MOeoGULtmQe4UGrncPbiyL4nrtTkL/Mne5u/gD52OwtILWHI+hvWWvFBUxF6/ldBSm eCV5qbJIhJoG4mee+ePqFRN7zbVN1xufLc8at9zBIXD9QyEmO+OyvsyKyaZzfhtKxzM4 i06S5CHeUxjJvMkD9IXF+qEn/MltL/gzhdNh4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=msHpUMdlU2bD5nmeH1juHpb0E12kZ4laZq8qz+Ax6+Y=; b=bsPBLGZjuf4OcINK/k92gT9vf8Rv4WcrJx3gSYAS2fgp6LcSyi2dygbYySv0Cpq9xg hb3ZRDKqTtZOC5Nd7gfsOXZj/P7+aS/u9F4/f84ILJ7oVjLgOxpcDkFEsZktQEAFc1xn R2QVh/xf8ipmShH26BMLsZuSFHTRo5W15GSnX/4dbI2ee5l1oyA9gH/WjvTfbP+Yfx1j fDptvr0l+bi0hxD17qY57rOUkEK3qNrmynEFfq80bv2xV0iMCrEFFuvwJrg9KWaaU7Ng cGgA/ey7FPILALiRhg7ZOZdbCZa2hBJnoXJEGwLc6bY6jharHuKl2JaKjryuX/HzVMdn 0Ovg== X-Gm-Message-State: AKGB3mJfb2GXjq17lZga/ZiddyUFaqhIKl8JQzGqMJDsTcqR4fEhr67f Th0x3IW8Hp+F0VmRbGPCdhTojFBZS94= X-Received: by 10.46.66.18 with SMTP id p18mr403454lja.74.1515096776914; Thu, 04 Jan 2018 12:12:56 -0800 (PST) Received: from localhost.localdomain (c-cb7471d5.014-348-6c756e10.cust.bredbandsbolaget.se. [213.113.116.203]) by smtp.gmail.com with ESMTPSA id g29sm738863lfl.85.2018.01.04.12.12.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 04 Jan 2018 12:12:56 -0800 (PST) From: Linus Walleij To: linux-mmc@vger.kernel.org, Ulf Hansson , Benjamin Beckmeyer , Adrian Hunter Cc: Linus Walleij Subject: [PATCH] RFT: mmc: sdhci: Implement an SDHCI-specific bounce buffer Date: Thu, 4 Jan 2018 21:10:45 +0100 Message-Id: <20180104201045.31139-1-linus.walleij@linaro.org> X-Mailer: git-send-email 2.14.3 Sender: linux-mmc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mmc@vger.kernel.org The bounce buffer is gone from the MMC core, and now we found out that there are some (crippled) i.MX boards out there that have broken ADMA (cannot do scatter-gather), and broken PIO so they must use SDMA. SDMA sets down the number of segments to one, so that each segment gets turned into a singular request that ping-pongs to the block layer before the next request/segment is issued. These devices can see major benefits from a bounce buffer, as a fragmented read or write buffer may come in even though the sectors we will be reading or writing to the MMC/SD-card are consecutive. This patch accumulates those fragmented scatterlists in a physically contigous bounce buffer so that we can issue bigger DMA data chunks to/from the card. It is possible to achieve even better speed-ups by adding a second bounce buffer so that the ->pre_req() hook in the driver can do the buffer copying and DMA mapping/flushing while the request is in flight. We save this optimization for later. Signed-off-by: Linus Walleij --- I would be surprised if this works, which is why i sprinkled a few extra prints all over the place so we can see what goes wrong. If it would happen to work, try to comment out the noisy prints ("bouncing read/write data") so they don't disturb test results. --- drivers/mmc/host/sdhci.c | 118 +++++++++++++++++++++++++++++++++++++++++++---- drivers/mmc/host/sdhci.h | 3 ++ 2 files changed, 113 insertions(+), 8 deletions(-) -- 2.14.3 -- To unsubscribe from this list: send the line "unsubscribe linux-mmc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c index e9290a3439d5..d35e25effe7d 100644 --- a/drivers/mmc/host/sdhci.c +++ b/drivers/mmc/host/sdhci.c @@ -502,8 +502,28 @@ static int sdhci_pre_dma_transfer(struct sdhci_host *host, if (data->host_cookie == COOKIE_PRE_MAPPED) return data->sg_count; - sg_count = dma_map_sg(mmc_dev(host->mmc), data->sg, data->sg_len, - mmc_get_dma_dir(data)); + /* Bounce write requests to the bounce buffer */ + if (host->bounce_buffer) { + if (mmc_get_dma_dir(data) == DMA_TO_DEVICE) { + pr_info("bouncing write data\n"); + + /* Copy the data to the bounce buffer */ + sg_copy_to_buffer(data->sg, data->sg_len, + host->bounce_buffer, host->bounce_buffer_size); + } + + /* Map the bounce buffer to the device for both reads and writes */ + sg_count = dma_map_sg(mmc_dev(host->mmc), host->bounce_sg, + 1, DMA_TO_DEVICE); + + /* Overzealous check */ + if (sg_count != 1) + pr_err("mapped bounce buffer more than one SG entry\n"); + } else { + /* Just access the data directly from memory */ + sg_count = dma_map_sg(mmc_dev(host->mmc), data->sg, data->sg_len, + mmc_get_dma_dir(data)); + } if (sg_count == 0) return -ENOSPC; @@ -858,8 +878,13 @@ static void sdhci_prepare_data(struct sdhci_host *host, struct mmc_command *cmd) SDHCI_ADMA_ADDRESS_HI); } else { WARN_ON(sg_cnt != 1); - sdhci_writel(host, sg_dma_address(data->sg), - SDHCI_DMA_ADDRESS); + /* Bounce buffer goes to work */ + if (host->bounce_buffer) + sdhci_writel(host, sg_dma_address(host->bounce_sg), + SDHCI_DMA_ADDRESS); + else + sdhci_writel(host, sg_dma_address(data->sg), + SDHCI_DMA_ADDRESS); } } @@ -2248,7 +2273,12 @@ static void sdhci_pre_req(struct mmc_host *mmc, struct mmc_request *mrq) mrq->data->host_cookie = COOKIE_UNMAPPED; - if (host->flags & SDHCI_REQ_USE_DMA) + /* + * No pre-mapping in the pre hook if we're using the bounce buffer, + * for that we would need two bounce buffers since one buffer is + * in flight when this is getting called. + */ + if (host->flags & SDHCI_REQ_USE_DMA && !host->bounce_buffer) sdhci_pre_dma_transfer(host, mrq->data, COOKIE_PRE_MAPPED); } @@ -2352,8 +2382,25 @@ static bool sdhci_request_done(struct sdhci_host *host) struct mmc_data *data = mrq->data; if (data && data->host_cookie == COOKIE_MAPPED) { - dma_unmap_sg(mmc_dev(host->mmc), data->sg, data->sg_len, - mmc_get_dma_dir(data)); + if (host->bounce_buffer) { + /* Unmap the bounce buffer */ + dma_unmap_sg(mmc_dev(host->mmc), host->bounce_sg, + 1, mmc_get_dma_dir(data)); + + /* On reads, copy the data back to the sglist */ + if (mmc_get_dma_dir(data) == DMA_FROM_DEVICE) { + pr_info("bouncing read data\n"); + + sg_copy_from_buffer(data->sg, data->sg_len, + host->bounce_buffer, + host->bounce_buffer_size); + } + } else { + /* Unmap the raw data */ + dma_unmap_sg(mmc_dev(host->mmc), data->sg, + data->sg_len, + mmc_get_dma_dir(data)); + } data->host_cookie = COOKIE_UNMAPPED; } } @@ -2636,7 +2683,12 @@ static void sdhci_data_irq(struct sdhci_host *host, u32 intmask) */ if (intmask & SDHCI_INT_DMA_END) { u32 dmastart, dmanow; - dmastart = sg_dma_address(host->data->sg); + + if (host->bounce_buffer) + dmastart = sg_dma_address(host->bounce_sg); + else + dmastart = sg_dma_address(host->data->sg); + dmanow = dmastart + host->data->bytes_xfered; /* * Force update to the next DMA block boundary. @@ -3713,6 +3765,50 @@ int sdhci_setup_host(struct sdhci_host *host) */ mmc->max_blk_count = (host->quirks & SDHCI_QUIRK_NO_MULTIBLOCK) ? 1 : 65535; + if (mmc->max_segs == 1) { + unsigned int max_blocks; + unsigned int max_seg_size; + + max_seg_size = mmc->max_req_size; + max_blocks = max_seg_size / 512; + + /* + * When we just support one segment, we can get significant speedups + * by the help of a bounce buffer to group scattered reads/writes + * together. + * + * TODO: is this too big? Stealing too much memory? The old bounce + * buffer is max 64K. This should be the 512K that SDMA can handle + * if I read the code above right. Anyways let's try this. + * FIXME: use devm_* + */ + host->bounce_buffer = kmalloc(max_seg_size, GFP_KERNEL); + if (!host->bounce_buffer) { + pr_err("failed to allocate %u bytes for bounce buffer\n", + max_seg_size); + return -ENOMEM; + } + host->bounce_buffer_size = max_seg_size; + + /* + * Initialize single-entry sglist to point at the bounce buffer + * FIXME: use devm_* + */ + host->bounce_sg = kmalloc(sizeof(*host->bounce_sg), GFP_KERNEL); + if (!host->bounce_sg) { + pr_err("failed to allocate bounce buffer sglist\n"); + return -ENOMEM; + } + sg_init_one(host->bounce_sg, host->bounce_buffer, max_seg_size); + + /* Lie about this since we're bouncing */ + mmc->max_segs = max_blocks; + mmc->max_seg_size = max_seg_size; + + pr_info("SDMA with bounce buffer: bounce up to %u segments into one, max segment size %u bytes\n", + max_blocks, max_seg_size); + } + return 0; unreg: @@ -3743,6 +3839,12 @@ void sdhci_cleanup_host(struct sdhci_host *host) host->align_addr); host->adma_table = NULL; host->align_buffer = NULL; + /* kfree() on NULL is fine */ + kfree(host->bounce_sg); + kfree(host->bounce_buffer); + /* Overzealous but following the style above */ + host->bounce_sg = NULL; + host->bounce_buffer = NULL; } EXPORT_SYMBOL_GPL(sdhci_cleanup_host); diff --git a/drivers/mmc/host/sdhci.h b/drivers/mmc/host/sdhci.h index 54bc444c317f..31fc2302454a 100644 --- a/drivers/mmc/host/sdhci.h +++ b/drivers/mmc/host/sdhci.h @@ -440,6 +440,9 @@ struct sdhci_host { int irq; /* Device IRQ */ void __iomem *ioaddr; /* Mapped address */ + char *bounce_buffer; /* For packing SDMA reads/writes */ + size_t bounce_buffer_size; + struct scatterlist *bounce_sg; const struct sdhci_ops *ops; /* Low level hw interface */