From patchwork Wed Aug 1 10:22:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kenneth Lee X-Patchwork-Id: 143266 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp680559ljj; Wed, 1 Aug 2018 03:24:00 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdhWofYw+/NP/sIm0dmQ9eulRHgeP6niFFAx85eztg4HnhTElpAv7IScaGuUoU9sFwQkCcC X-Received: by 2002:a62:a649:: with SMTP id t70-v6mr26230814pfe.149.1533119040179; Wed, 01 Aug 2018 03:24:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533119040; cv=none; d=google.com; s=arc-20160816; b=LJVqL1f741CEsbXZKKq5F1aBeFbuR36Vw86qTPctEQ8cRtpqDtwOXi3X0jTmtJaSRi O5fEADBef8AoUJrMXF+y5WJgm+H65Eb3YBTJJHyKt1Oj3wcH7y+WuY4nUqmUOJZw0vpg wZZy6rWSINH7gHyU+fHTsKneKRAmTV1IMLFRU9Oc7dXytGU1ecdLuN99NCzjW0BtTXCX XOtJJLWSj43RyscU7WePsb6G7C3PWBrEaqcdKLIhjPAYgZOLF6l0/TMVjCn5EUm7ww62 QP6GNmk0YdnFy4kwRI3gktohHSwPcRlSDzcUzE/0iAC+sjOrcpOmi232tB+o10ERN/Sd aoDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=OSd5cwh5ucpAZl5U/0Zl3XcUmV4KNvQXG2PCIRbmgao=; b=TAR8lpREv96J1/0nEYVPVIbDSfuRJ9U6NehrEjrTnKBI0b04sZTqU8mQU9FqhB7IfR PKHCLxaas521cwhZ9CRvXxfOewEhlZM48ZS4rIQkfw95Bjid9lMh6sm93BQO9i4wInCq 2+wc5rO2RmDPlcFlO+13PuLId5Hj411gmWiYtOIteeaJVNFMeeLZb+f/EeT0AoYDmeyx tbqUfuiRhVjGQ8Qa477wzBi6Mp5WpfsSlYk/SNI3KwAUwcn4wF2VEohlFrOKdOdrsZ84 Pm0BAOUJY7AqphT2hsKqYpFjvT/tXczExkIkzDC74Kx34qBpFe1rZZjzid3i5tSPMdUa GUTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ofbTgfRR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s2-v6si15981035pgl.140.2018.08.01.03.23.59; Wed, 01 Aug 2018 03:24:00 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ofbTgfRR; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387605AbeHAMJD (ORCPT + 1 other); Wed, 1 Aug 2018 08:09:03 -0400 Received: from mail-qt0-f193.google.com ([209.85.216.193]:35829 "EHLO mail-qt0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387528AbeHAMJD (ORCPT ); Wed, 1 Aug 2018 08:09:03 -0400 Received: by mail-qt0-f193.google.com with SMTP id a5-v6so19238412qtp.2; Wed, 01 Aug 2018 03:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=OSd5cwh5ucpAZl5U/0Zl3XcUmV4KNvQXG2PCIRbmgao=; b=ofbTgfRRvR+jrKb21pmAfP+V8q4XzTq339i3vcVVVFd8nRGGhfiJzaT2l5Ew1ScjaX iInbv1+2yN28s6a4t0XB6m7/D6ZVk5puQlJa9mWed4WwJkfGdpKPShtcOLfeMHAyUcwQ 4JOSpmuGrX5L16U3BK5yRnnfV0t1afOkfCAOSjMVjJ3JZk8pNjdGtftnHcf5zm6Nf3mx N9/86x6moimxWOt8DhxHHBXr8q0sinDaKepzBLcz/5SeAtwI5g8UV/jVLzvYWWqNtAJ3 w3yDZ8BCoGeUSmlc5ywNM0J53fjjFRi++n/hUFGpF650yKEbcudgOoqaeWZKmqzZBv7e fQYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OSd5cwh5ucpAZl5U/0Zl3XcUmV4KNvQXG2PCIRbmgao=; b=glShtoAtH7Vp5m+cWahLpSwEf5SBEZmL9Mo9QfenO1uAbKjSLASzUUa6FjawCfRCE0 8oH6LqZWEnhCf0O8GMRkH4T+igkOrwePQRtslT6z3sEHd5GqUtkp17orXaUT5/WB3Drl qBzIb/Wd45XT/LKUlMdY20ikUAqhV3oAbz4ZMzIAj/kjqendGAEKim5DAlbw0Am4UH6J ND+pwdlVjcgMqJqx7Aeaj+K0h+iGxvj3TiQ2h9u4hM24LkXDcow/hc6MpmGBegf2uzR1 SQJ/6Rb40hultdIWdycOcxrelCVa8JLNCO70GinHo6EoxDELf00lKriwTBNDNI/JYXP+ zNNA== X-Gm-Message-State: AOUpUlFdBpvsTOZ8D7zxuMrSTMcz+x/KatrKhbFt0oe11CTgqtPDzJDB aXJ7+m5yZpG6d0KftMSmKvw= X-Received: by 2002:ac8:d4a:: with SMTP id r10-v6mr24246915qti.271.1533119035552; Wed, 01 Aug 2018 03:23:55 -0700 (PDT) Received: from localhost.localdomain ([104.237.86.144]) by smtp.gmail.com with ESMTPSA id s19-v6sm11176890qtj.61.2018.08.01.03.23.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Aug 2018 03:23:54 -0700 (PDT) From: Kenneth Lee To: Jonathan Corbet , Herbert Xu , "David S . Miller" , Joerg Roedel , Alex Williamson , Kenneth Lee , Hao Fang , Zhou Wang , Zaibo Xu , Philippe Ombredanne , Greg Kroah-Hartman , Thomas Gleixner , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-accelerators@lists.ozlabs.org, Lu Baolu , Sanjay Kumar Cc: linuxarm@huawei.com Subject: [RFC PATCH 1/7] vfio/spimdev: Add documents for WarpDrive framework Date: Wed, 1 Aug 2018 18:22:15 +0800 Message-Id: <20180801102221.5308-2-nek.in.cn@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180801102221.5308-1-nek.in.cn@gmail.com> References: <20180801102221.5308-1-nek.in.cn@gmail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Kenneth Lee WarpDrive is a common user space accelerator framework. Its main component in Kernel is called spimdev, Share Parent IOMMU Mediated Device. It exposes the hardware capabilities to the user space via vfio-mdev. So processes in user land can obtain a "queue" by open the device and direct access the hardware MMIO space or do DMA operation via VFIO interface. WarpDrive is intended to be used with Jean Philippe Brucker's SVA patchset (it is still in RFC stage) to support multi-process. But This is not a must. Without the SVA patches, WarpDrive can still work for one process for every hardware device. This patch add detail documents for the framework. Signed-off-by: Kenneth Lee --- Documentation/00-INDEX | 2 + Documentation/warpdrive/warpdrive.rst | 153 ++++++ Documentation/warpdrive/wd-arch.svg | 732 ++++++++++++++++++++++++++ Documentation/warpdrive/wd.svg | 526 ++++++++++++++++++ 4 files changed, 1413 insertions(+) create mode 100644 Documentation/warpdrive/warpdrive.rst create mode 100644 Documentation/warpdrive/wd-arch.svg create mode 100644 Documentation/warpdrive/wd.svg -- 2.17.1 diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX index 2754fe83f0d4..9959affab599 100644 --- a/Documentation/00-INDEX +++ b/Documentation/00-INDEX @@ -410,6 +410,8 @@ vm/ - directory with info on the Linux vm code. w1/ - directory with documents regarding the 1-wire (w1) subsystem. +warpdrive/ + - directory with documents about WarpDrive accelerator framework. watchdog/ - how to auto-reboot Linux if it has "fallen and can't get up". ;-) wimax/ diff --git a/Documentation/warpdrive/warpdrive.rst b/Documentation/warpdrive/warpdrive.rst new file mode 100644 index 000000000000..3792b2780ea6 --- /dev/null +++ b/Documentation/warpdrive/warpdrive.rst @@ -0,0 +1,153 @@ +Introduction of WarpDrive +========================= + +*WarpDrive* is a general accelerator framework built on top of vfio. +It can be taken as a light weight virtual function, which you can use without +*SR-IOV* like facility and can be shared among multiple processes. + +It can be used as the quick channel for accelerators, network adaptors or +other hardware in user space. It can make some implementation simpler. E.g. +you can reuse most of the *netdev* driver and just share some ring buffer to +the user space driver for *DPDK* or *ODP*. Or you can combine the RSA +accelerator with the *netdev* in the user space as a Web reversed proxy, etc. + +The name *WarpDrive* is simply a cool and general name meaning the framework +makes the application faster. In kernel, the framework is called SPIMDEV, +namely "Share Parent IOMMU Mediated Device". + + +How does it work +================ + +*WarpDrive* takes the Hardware Accelerator as a heterogeneous processor which +can share some load for the CPU: + +.. image:: wd.svg + :alt: This is a .svg image, if your browser cannot show it, + try to download and view it locally + +So it provides the capability to the user application to: + +1. Send request to the hardware +2. Share memory with the application and other accelerators + +These requirements can be fulfilled by VFIO if the accelerator can serve each +application with a separated Virtual Function. But a *SR-IOV* like VF (we will +call it *HVF* hereinafter) design is too heavy for the accelerator which +service thousands of processes. + +And the *HVF* is not good for the scenario that a device keep most of its +resource but share partial of the function to the user space. E.g. a *NIC* +works as a *netdev* but share some hardware queues to the user application to +send packets direct to the hardware. + +*VFIO-mdev* can solve some of the problem here. But *VFIO-mdev* has two problem: + +1. it cannot make use of its parent device's IOMMU. +2. it is assumed to be openned only once. + +So it will need some add-on for better resource control and let the VFIO +driver be aware of this. + + +Architecture +------------ + +The full *WarpDrive* architecture is represented in the following class +diagram: + +.. image:: wd-arch.svg + :alt: This is a .svg image, if your browser cannot show it, + try to download and view it locally + +The idea is: when a device is probed, it can be registered to the general +framework, e.g. *netdev* or *crypto*, and the *SPIMDEV* at the same time. + +If *SPIMDEV* is registered. A *mdev* creation interface is created. Then the +system administrator can create a *mdev* in the user space and set its +parameters via its sysfs interfacev. But not like the other mdev +implementation, hardware resource will not be allocated until it is opened by +an application. + +With this strategy, the hardware resource can be easily scheduled among +multiple processes. + + +The user API +------------ + +We adopt a polling style interface in the user space: :: + + int wd_request_queue(int container, struct wd_queue *q, + const char *mdev) + void wd_release_queue(struct wd_queue *q); + + int wd_send(struct wd_queue *q, void *req); + int wd_recv(struct wd_queue *q, void **req); + int wd_recv_sync(struct wd_queue *q, void **req); + +the ..._sync() interface is a wrapper to the non sync version. They wait on the +device until the queue become available. + +Memory can be done by VFIO DMA API. Or the following helper function can be +adopted: :: + + int wd_mem_share(struct wd_queue *q, const void *addr, + size_t size, int flags); + void wd_mem_unshare(struct wd_queue *q, const void *addr, size_t size); + +Todo: if the IOMMU support *ATS* or *SMMU* stall mode. mem share is not +necessary. This can be check with SPImdev sysfs interface. + +The user API is not mandatory. It is simply a suggestion and hint what the +kernel interface is supposed to support. + + +The user driver +--------------- + +*WarpDrive* expose the hardware IO space to the user process (via *mmap*). So +it will require user driver for implementing the user API. The following API +is suggested for a user driver: :: + + int open(struct wd_queue *q); + int close(struct wd_queue *q); + int send(struct wd_queue *q, void *req); + int recv(struct wd_queue *q, void **req); + +These callback enable the communication between the user application and the +device. You will still need the hardware-depend algorithm driver to access the +algorithm functionality of the accelerator itself. + + +Multiple processes support +========================== + +In the latest mainline kernel (4.18) when this document is written. +Multi-process is not supported in VFIO yet. + +*JPB* has a patchset to enable this[2]_. We have tested it with our hardware +(which is known as *D06*). It works well. *WarpDrive* rely on them to support +multiple processes. If it is not enabled, *WarpDrive* can still work, but it +support only one process, which will share the same io map table with kernel +(but the user application cannot access the kernel address, So it is not going +to be a security problem) + + +Legacy Mode Support +=================== +For the hardware on which IOMMU is not support, WarpDrive can run on *NOIOMMU* +mode. + + +References +========== +.. [1] Accroding to the comment in in mm/gup.c, The *gup* is only safe within + a syscall. Because it can only keep the physical memory in place + without making sure the VMA will always point to it. Maybe we should + raise the VM_PINNED patchset (see + https://lists.gt.net/linux/kernel/1931993) again to solve this problem. +.. [2] https://patchwork.kernel.org/patch/10394851/ +.. [3] https://zhuanlan.zhihu.com/p/35489035 + +.. vim: tw=78 diff --git a/Documentation/warpdrive/wd-arch.svg b/Documentation/warpdrive/wd-arch.svg new file mode 100644 index 000000000000..1b3d1817c4ba --- /dev/null +++ b/Documentation/warpdrive/wd-arch.svg @@ -0,0 +1,732 @@ + + + + + + + + + + + + + + + + + + + + + + + + generation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + WarpDrive + + + user_driver + + + + spimdev + + + + + Device Driver + * + 1 + + <<vfio>>resource management + 1 + * + + + other standard framework(crypto/nic/others) + + <<lkm>> + register as mdev with "share parent iommu" attribute + register to other subsystem + <<user_lib>> + <<vfio>>Hardware Accessing + wd user api + + + + Device(Hardware) + + Share Parent's IOMMU mdev + + diff --git a/Documentation/warpdrive/wd.svg b/Documentation/warpdrive/wd.svg new file mode 100644 index 000000000000..87ab92ebfbc6 --- /dev/null +++ b/Documentation/warpdrive/wd.svg @@ -0,0 +1,526 @@ + + + + + + + + + + + + + + + + generation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + user application (running by the CPU + + + + + MMU + + + Memory + + + IOMMU + + + Hardware Accelerator + + + + From patchwork Wed Aug 1 10:22:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kenneth Lee X-Patchwork-Id: 143268 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp680819ljj; Wed, 1 Aug 2018 03:24:16 -0700 (PDT) X-Google-Smtp-Source: AAOMgpefhafw8fv8Lls3r3S60h+LomQLQIBrg6/NQj3rnRe4QeUnpQh0wPbJ7XK4fzI6sO1gePjo X-Received: by 2002:a65:4107:: with SMTP id w7-v6mr23097569pgp.302.1533119056602; Wed, 01 Aug 2018 03:24:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533119056; cv=none; d=google.com; s=arc-20160816; b=rbf3rUyn7M0wvdJAi9WCuIKc6l34eAW63bvMQfLJ/5qF7nvb2apsoBkc6TRnZ1TqRx SoydjIU39bu2nL9MuJJW9nKxv5eyIcKIYOqdlvHn2xYsVObtZ2cmDuvn3ozW9EDKZ61R CItZH5YTxUU+D//ZeGlJ2fW2vHcwYSP3OV/kPPJvn/Jn4HlgNbAM0q9kIplKFT1I3Skj PFMOWoHAPF3txvG0Ya4FCgeCjafjEcfxcSecWULTFluV3pqt1XD/wwhIgvBQ2rbsJrIe QscZd2RpcWJbmoaO7J1V45x1lrUkGGiv/h2PwHGSmrPtFnzrxzvsysrdCloS4W8+BE9b qMRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Lpt+samP8CQqQ79TXRngQuvM7gOQ3mc3raE4OeDPodM=; b=KZisR8eMhwiH/WLjlJsZkw4GzmpoubeIEtA96g4mcsFJCCG1gAmawcD6rOj0iAUZTz znq0SScF4yKguB/vM6RdSPKFRi+uNRdyVHnHskFBoGhVUzN2FK+13nLxBkz+nyJTjgD1 IRzshVGtdpCcglCw38X5JXsyDuHpTpa9rBgeaos9sRtjxNjQZo/JdhWH1j+EowBYpV1J t2YXLIeyINItjx1ypyyParhgUiecEuY/xPcXvOwxj2skRGyUpJdR00SbuW/3lIbUUeMi 4KRUuo1bsfjSM6RGdIrv+qgWOzoFqQ4NsOfM3+ssJNUvCoSS1t1RX1Esh6tc7FiwPKWd sCIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=MlLX4P99; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n8-v6si15447494pgl.101.2018.08.01.03.24.16; Wed, 01 Aug 2018 03:24:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=MlLX4P99; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387528AbeHAMJU (ORCPT + 1 other); Wed, 1 Aug 2018 08:09:20 -0400 Received: from mail-qk0-f193.google.com ([209.85.220.193]:46407 "EHLO mail-qk0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387723AbeHAMJT (ORCPT ); Wed, 1 Aug 2018 08:09:19 -0400 Received: by mail-qk0-f193.google.com with SMTP id 191-v6so6324000qki.13; Wed, 01 Aug 2018 03:24:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Lpt+samP8CQqQ79TXRngQuvM7gOQ3mc3raE4OeDPodM=; b=MlLX4P9941JVGEFAS72ot4s4njBHa4NwMk0XSZV5wEUyB+Vy6TnoTn2MDdNxJvy6+F 2JYlp49GF/Z07hEdBzgdoaKYwVpVZm393xIwLXN5LpGZ//Myj8QZrHGNjPdTb27UaZ6V X+iF/I7kgXixN1lVxaF1wjAkV/dP82JL60HWysNtY79HIIOq2aXM7Q0dlsDD8gdvtm51 3WVKyIl3aufd+ZIPgU02nWSkkPGeqcUSvqRLKY6loAAxa6nwUV869R9aQVj413XDOqua 6RrN2NYxV0idZzDRstvh/0Wn0Y56HQOoYHE0Cf/f3fTIF2C9d7AtYmxEzgJpBnBoMNww +mVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Lpt+samP8CQqQ79TXRngQuvM7gOQ3mc3raE4OeDPodM=; b=XIoX9USIX2vSdooV/EkkPpl33w73iWRvmCU1nHtalJ8KLRAJv/CnMow7+0yvnNbAGs 7l91PItR/HUtk2SixLYFTkYm+We46cM5DhNfqAwd631RBOuRydeMZuV1nEK962HorB+N /LNcVouG4b7X8l6DVI6hbB2t49/ZIzfm98SLFMcoQP99TaT8juo8idxwguT2YQJpkJ4V 3PHUpsc6x+CQ0DmU6UcfD/UsFa+XoaAXjHbHsoqvWqddbaz8kTzA3L/1a4plQANSk9Vk tCxhZUY+M492maFhXyE6VN27SS5UvR0qWg9QjvQwQBb3PwnW8qZSTo4ArTa8yEcuCHKh rMdw== X-Gm-Message-State: AOUpUlEAc51dqSk6EFVId/tH1yfsOQSsoxsxiM3wAc0hxVYPahJcUADi 1iR1XQ2Zy8CyWPzW2YEvtFg= X-Received: by 2002:a37:5b83:: with SMTP id p125-v6mr23348358qkb.196.1533119053182; Wed, 01 Aug 2018 03:24:13 -0700 (PDT) Received: from localhost.localdomain ([104.237.86.144]) by smtp.gmail.com with ESMTPSA id s19-v6sm11176890qtj.61.2018.08.01.03.24.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Aug 2018 03:24:12 -0700 (PDT) From: Kenneth Lee To: Jonathan Corbet , Herbert Xu , "David S . Miller" , Joerg Roedel , Alex Williamson , Kenneth Lee , Hao Fang , Zhou Wang , Zaibo Xu , Philippe Ombredanne , Greg Kroah-Hartman , Thomas Gleixner , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-accelerators@lists.ozlabs.org, Lu Baolu , Sanjay Kumar Cc: linuxarm@huawei.com Subject: [RFC PATCH 3/7] vfio: add spimdev support Date: Wed, 1 Aug 2018 18:22:17 +0800 Message-Id: <20180801102221.5308-4-nek.in.cn@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180801102221.5308-1-nek.in.cn@gmail.com> References: <20180801102221.5308-1-nek.in.cn@gmail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Kenneth Lee SPIMDEV is "Share Parent IOMMU Mdev". It is a vfio-mdev. But differ from the general vfio-mdev: 1. It shares its parent's IOMMU. 2. There is no hardware resource attached to the mdev is created. The hardware resource (A `queue') is allocated only when the mdev is opened. Currently only the vfio type-1 driver is updated to make it to be aware of. Signed-off-by: Kenneth Lee Signed-off-by: Zaibo Xu Signed-off-by: Zhou Wang --- drivers/vfio/Kconfig | 1 + drivers/vfio/Makefile | 1 + drivers/vfio/spimdev/Kconfig | 10 + drivers/vfio/spimdev/Makefile | 3 + drivers/vfio/spimdev/vfio_spimdev.c | 421 ++++++++++++++++++++++++++++ drivers/vfio/vfio_iommu_type1.c | 136 ++++++++- include/linux/vfio_spimdev.h | 95 +++++++ include/uapi/linux/vfio_spimdev.h | 28 ++ 8 files changed, 689 insertions(+), 6 deletions(-) create mode 100644 drivers/vfio/spimdev/Kconfig create mode 100644 drivers/vfio/spimdev/Makefile create mode 100644 drivers/vfio/spimdev/vfio_spimdev.c create mode 100644 include/linux/vfio_spimdev.h create mode 100644 include/uapi/linux/vfio_spimdev.h -- 2.17.1 diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig index c84333eb5eb5..3719eba72ef1 100644 --- a/drivers/vfio/Kconfig +++ b/drivers/vfio/Kconfig @@ -47,4 +47,5 @@ menuconfig VFIO_NOIOMMU source "drivers/vfio/pci/Kconfig" source "drivers/vfio/platform/Kconfig" source "drivers/vfio/mdev/Kconfig" +source "drivers/vfio/spimdev/Kconfig" source "virt/lib/Kconfig" diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile index de67c4725cce..28f3ef0cdce1 100644 --- a/drivers/vfio/Makefile +++ b/drivers/vfio/Makefile @@ -9,3 +9,4 @@ obj-$(CONFIG_VFIO_SPAPR_EEH) += vfio_spapr_eeh.o obj-$(CONFIG_VFIO_PCI) += pci/ obj-$(CONFIG_VFIO_PLATFORM) += platform/ obj-$(CONFIG_VFIO_MDEV) += mdev/ +obj-$(CONFIG_VFIO_SPIMDEV) += spimdev/ diff --git a/drivers/vfio/spimdev/Kconfig b/drivers/vfio/spimdev/Kconfig new file mode 100644 index 000000000000..1226301f9d0e --- /dev/null +++ b/drivers/vfio/spimdev/Kconfig @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0 +config VFIO_SPIMDEV + tristate "Support for Share Parent IOMMU MDEV" + depends on VFIO_MDEV_DEVICE + help + Support for VFIO Share Parent IOMMU MDEV, which enable the kernel to + support for the light weight hardware accelerator framework, WrapDrive. + + To compile this as a module, choose M here: the module will be called + spimdev. diff --git a/drivers/vfio/spimdev/Makefile b/drivers/vfio/spimdev/Makefile new file mode 100644 index 000000000000..d02fb69c37e4 --- /dev/null +++ b/drivers/vfio/spimdev/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +spimdev-y := spimdev.o +obj-$(CONFIG_VFIO_SPIMDEV) += vfio_spimdev.o diff --git a/drivers/vfio/spimdev/vfio_spimdev.c b/drivers/vfio/spimdev/vfio_spimdev.c new file mode 100644 index 000000000000..1b6910c9d27d --- /dev/null +++ b/drivers/vfio/spimdev/vfio_spimdev.c @@ -0,0 +1,421 @@ +// SPDX-License-Identifier: GPL-2.0+ +#include +#include +#include +#include +#include + +struct spimdev_mdev_state { + struct vfio_spimdev *spimdev; +}; + +static struct class *spimdev_class; +static DEFINE_IDR(spimdev_idr); + +static int vfio_spimdev_dev_exist(struct device *dev, void *data) +{ + return !strcmp(dev_name(dev), dev_name((struct device *)data)); +} + +#ifdef CONFIG_IOMMU_SVA +static bool vfio_spimdev_is_valid_pasid(int pasid) +{ + struct mm_struct *mm; + + mm = iommu_sva_find(pasid); + if (mm) { + mmput(mm); + return mm == current->mm; + } + + return false; +} +#endif + +/* Check if the device is a mediated device belongs to vfio_spimdev */ +int vfio_spimdev_is_spimdev(struct device *dev) +{ + struct mdev_device *mdev; + struct device *pdev; + + mdev = mdev_from_dev(dev); + if (!mdev) + return 0; + + pdev = mdev_parent_dev(mdev); + if (!pdev) + return 0; + + return class_for_each_device(spimdev_class, NULL, pdev, + vfio_spimdev_dev_exist); +} +EXPORT_SYMBOL_GPL(vfio_spimdev_is_spimdev); + +struct vfio_spimdev *vfio_spimdev_pdev_spimdev(struct device *dev) +{ + struct device *class_dev; + + if (!dev) + return ERR_PTR(-EINVAL); + + class_dev = class_find_device(spimdev_class, NULL, dev, + (int(*)(struct device *, const void *))vfio_spimdev_dev_exist); + if (!class_dev) + return ERR_PTR(-ENODEV); + + return container_of(class_dev, struct vfio_spimdev, cls_dev); +} +EXPORT_SYMBOL_GPL(vfio_spimdev_pdev_spimdev); + +struct vfio_spimdev *mdev_spimdev(struct mdev_device *mdev) +{ + struct device *pdev = mdev_parent_dev(mdev); + + return vfio_spimdev_pdev_spimdev(pdev); +} +EXPORT_SYMBOL_GPL(mdev_spimdev); + +static ssize_t iommu_type_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct vfio_spimdev *spimdev = vfio_spimdev_pdev_spimdev(dev); + + if (!spimdev) + return -ENODEV; + + return sprintf(buf, "%d\n", spimdev->iommu_type); +} + +static DEVICE_ATTR_RO(iommu_type); + +static ssize_t dma_flag_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct vfio_spimdev *spimdev = vfio_spimdev_pdev_spimdev(dev); + + if (!spimdev) + return -ENODEV; + + return sprintf(buf, "%d\n", spimdev->dma_flag); +} + +static DEVICE_ATTR_RO(dma_flag); + +/* mdev->dev_attr_groups */ +static struct attribute *vfio_spimdev_attrs[] = { + &dev_attr_iommu_type.attr, + &dev_attr_dma_flag.attr, + NULL, +}; +static const struct attribute_group vfio_spimdev_group = { + .name = VFIO_SPIMDEV_PDEV_ATTRS_GRP_NAME, + .attrs = vfio_spimdev_attrs, +}; +const struct attribute_group *vfio_spimdev_groups[] = { + &vfio_spimdev_group, + NULL, +}; + +/* default attributes for mdev->supported_type_groups, used by registerer*/ +#define MDEV_TYPE_ATTR_RO_EXPORT(name) \ + MDEV_TYPE_ATTR_RO(name); \ + EXPORT_SYMBOL_GPL(mdev_type_attr_##name); + +#define DEF_SIMPLE_SPIMDEV_ATTR(_name, spimdev_member, format) \ +static ssize_t _name##_show(struct kobject *kobj, struct device *dev, \ + char *buf) \ +{ \ + struct vfio_spimdev *spimdev = vfio_spimdev_pdev_spimdev(dev); \ + if (!spimdev) \ + return -ENODEV; \ + return sprintf(buf, format, spimdev->spimdev_member); \ +} \ +MDEV_TYPE_ATTR_RO_EXPORT(_name) + +DEF_SIMPLE_SPIMDEV_ATTR(flags, flags, "%d"); +DEF_SIMPLE_SPIMDEV_ATTR(name, name, "%s"); /* this should be algorithm name, */ + /* but you would not care if you have only one algorithm */ +DEF_SIMPLE_SPIMDEV_ATTR(device_api, api_ver, "%s"); + +/* this return total queue left, not mdev left */ +static ssize_t +available_instances_show(struct kobject *kobj, struct device *dev, char *buf) +{ + struct vfio_spimdev *spimdev = vfio_spimdev_pdev_spimdev(dev); + + return sprintf(buf, "%d", + spimdev->ops->get_available_instances(spimdev)); +} +MDEV_TYPE_ATTR_RO_EXPORT(available_instances); + +static int vfio_spimdev_mdev_create(struct kobject *kobj, + struct mdev_device *mdev) +{ + struct device *dev = mdev_dev(mdev); + struct device *pdev = mdev_parent_dev(mdev); + struct spimdev_mdev_state *mdev_state; + struct vfio_spimdev *spimdev = mdev_spimdev(mdev); + + if (!spimdev->ops->get_queue) + return -ENODEV; + + mdev_state = devm_kzalloc(dev, sizeof(struct spimdev_mdev_state), + GFP_KERNEL); + if (!mdev_state) + return -ENOMEM; + mdev_set_drvdata(mdev, mdev_state); + mdev_state->spimdev = spimdev; + dev->iommu_fwspec = pdev->iommu_fwspec; + get_device(pdev); + __module_get(spimdev->owner); + + return 0; +} + +static int vfio_spimdev_mdev_remove(struct mdev_device *mdev) +{ + struct device *dev = mdev_dev(mdev); + struct device *pdev = mdev_parent_dev(mdev); + struct vfio_spimdev *spimdev = mdev_spimdev(mdev); + + put_device(pdev); + module_put(spimdev->owner); + dev->iommu_fwspec = NULL; + mdev_set_drvdata(mdev, NULL); + + return 0; +} + +/* Wake up the process who is waiting this queue */ +void vfio_spimdev_wake_up(struct vfio_spimdev_queue *q) +{ + wake_up(&q->wait); +} +EXPORT_SYMBOL_GPL(vfio_spimdev_wake_up); + +static int vfio_spimdev_q_file_open(struct inode *inode, struct file *file) +{ + return 0; +} + +static int vfio_spimdev_q_file_release(struct inode *inode, struct file *file) +{ + struct vfio_spimdev_queue *q = + (struct vfio_spimdev_queue *)file->private_data; + struct vfio_spimdev *spimdev = q->spimdev; + int ret; + + ret = spimdev->ops->put_queue(q); + if (ret) { + dev_err(spimdev->dev, "drv put queue fail (%d)!\n", ret); + return ret; + } + + put_device(mdev_dev(q->mdev)); + + return 0; +} + +static long vfio_spimdev_q_file_ioctl(struct file *file, unsigned int cmd, + unsigned long arg) +{ + struct vfio_spimdev_queue *q = + (struct vfio_spimdev_queue *)file->private_data; + struct vfio_spimdev *spimdev = q->spimdev; + + if (spimdev->ops->ioctl) + return spimdev->ops->ioctl(q, cmd, arg); + + dev_err(spimdev->dev, "ioctl cmd (%d) is not supported!\n", cmd); + + return -EINVAL; +} + +static int vfio_spimdev_q_file_mmap(struct file *file, + struct vm_area_struct *vma) +{ + struct vfio_spimdev_queue *q = + (struct vfio_spimdev_queue *)file->private_data; + struct vfio_spimdev *spimdev = q->spimdev; + + if (spimdev->ops->mmap) + return spimdev->ops->mmap(q, vma); + + dev_err(spimdev->dev, "no driver mmap!\n"); + return -EINVAL; +} + +static __poll_t vfio_spimdev_q_file_poll(struct file *file, poll_table *wait) +{ + struct vfio_spimdev_queue *q = + (struct vfio_spimdev_queue *)file->private_data; + struct vfio_spimdev *spimdev = q->spimdev; + + poll_wait(file, &q->wait, wait); + if (spimdev->ops->is_q_updated(q)) + return EPOLLIN | EPOLLRDNORM; + + return 0; +} + +static const struct file_operations spimdev_q_file_ops = { + .owner = THIS_MODULE, + .open = vfio_spimdev_q_file_open, + .unlocked_ioctl = vfio_spimdev_q_file_ioctl, + .release = vfio_spimdev_q_file_release, + .poll = vfio_spimdev_q_file_poll, + .mmap = vfio_spimdev_q_file_mmap, +}; + +static long vfio_spimdev_mdev_get_queue(struct mdev_device *mdev, + struct vfio_spimdev *spimdev, unsigned long arg) +{ + struct vfio_spimdev_queue *q; + int ret; + +#ifdef CONFIG_IOMMU_SVA + int pasid = arg; + + if (!vfio_spimdev_is_valid_pasid(pasid)) + return -EINVAL; +#endif + + if (!spimdev->ops->get_queue) + return -EINVAL; + + ret = spimdev->ops->get_queue(spimdev, arg, &q); + if (ret < 0) { + dev_err(spimdev->dev, "get_queue failed\n"); + return -ENODEV; + } + + ret = anon_inode_getfd("spimdev_q", &spimdev_q_file_ops, + q, O_CLOEXEC | O_RDWR); + if (ret < 0) { + dev_err(spimdev->dev, "getfd fail %d\n", ret); + goto err_with_queue; + } + + q->fd = ret; + q->spimdev = spimdev; + q->mdev = mdev; + q->container = arg; + init_waitqueue_head(&q->wait); + get_device(mdev_dev(mdev)); + + return ret; + +err_with_queue: + spimdev->ops->put_queue(q); + return ret; +} + +static long vfio_spimdev_mdev_ioctl(struct mdev_device *mdev, unsigned int cmd, + unsigned long arg) +{ + struct spimdev_mdev_state *mdev_state; + struct vfio_spimdev *spimdev; + + if (!mdev) + return -ENODEV; + + mdev_state = mdev_get_drvdata(mdev); + if (!mdev_state) + return -ENODEV; + + spimdev = mdev_state->spimdev; + if (!spimdev) + return -ENODEV; + + if (cmd == VFIO_SPIMDEV_CMD_GET_Q) + return vfio_spimdev_mdev_get_queue(mdev, spimdev, arg); + + dev_err(spimdev->dev, + "%s, ioctl cmd (0x%x) is not supported!\n", __func__, cmd); + return -EINVAL; +} + +static void vfio_spimdev_release(struct device *dev) { } +static void vfio_spimdev_mdev_release(struct mdev_device *mdev) { } +static int vfio_spimdev_mdev_open(struct mdev_device *mdev) { return 0; } + +/** + * vfio_spimdev_register - register a spimdev + * @spimdev: device structure + */ +int vfio_spimdev_register(struct vfio_spimdev *spimdev) +{ + int ret; + const char *drv_name; + + if (!spimdev->dev) + return -ENODEV; + + drv_name = dev_driver_string(spimdev->dev); + if (strstr(drv_name, "-")) { + pr_err("spimdev: parent driver name cannot include '-'!\n"); + return -EINVAL; + } + + spimdev->dev_id = idr_alloc(&spimdev_idr, spimdev, 0, 0, GFP_KERNEL); + if (spimdev->dev_id < 0) + return spimdev->dev_id; + + atomic_set(&spimdev->ref, 0); + spimdev->cls_dev.parent = spimdev->dev; + spimdev->cls_dev.class = spimdev_class; + spimdev->cls_dev.release = vfio_spimdev_release; + dev_set_name(&spimdev->cls_dev, "%s", dev_name(spimdev->dev)); + ret = device_register(&spimdev->cls_dev); + if (ret) + return ret; + + spimdev->mdev_fops.owner = spimdev->owner; + spimdev->mdev_fops.dev_attr_groups = vfio_spimdev_groups; + WARN_ON(!spimdev->mdev_fops.supported_type_groups); + spimdev->mdev_fops.create = vfio_spimdev_mdev_create; + spimdev->mdev_fops.remove = vfio_spimdev_mdev_remove; + spimdev->mdev_fops.ioctl = vfio_spimdev_mdev_ioctl; + spimdev->mdev_fops.open = vfio_spimdev_mdev_open; + spimdev->mdev_fops.release = vfio_spimdev_mdev_release; + + ret = mdev_register_device(spimdev->dev, &spimdev->mdev_fops); + if (ret) + device_unregister(&spimdev->cls_dev); + + return ret; +} +EXPORT_SYMBOL_GPL(vfio_spimdev_register); + +/** + * vfio_spimdev_unregister - unregisters a spimdev + * @spimdev: device to unregister + * + * Unregister a miscellaneous device that wat previously successully registered + * with vfio_spimdev_register(). + */ +void vfio_spimdev_unregister(struct vfio_spimdev *spimdev) +{ + mdev_unregister_device(spimdev->dev); + device_unregister(&spimdev->cls_dev); +} +EXPORT_SYMBOL_GPL(vfio_spimdev_unregister); + +static int __init vfio_spimdev_init(void) +{ + spimdev_class = class_create(THIS_MODULE, VFIO_SPIMDEV_CLASS_NAME); + return PTR_ERR_OR_ZERO(spimdev_class); +} + +static __exit void vfio_spimdev_exit(void) +{ + class_destroy(spimdev_class); + idr_destroy(&spimdev_idr); +} + +module_init(vfio_spimdev_init); +module_exit(vfio_spimdev_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Hisilicon Tech. Co., Ltd."); +MODULE_DESCRIPTION("VFIO Share Parent's IOMMU Mediated Device"); diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 3e5b17710a4f..0ec38a17c98c 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -41,6 +41,7 @@ #include #include #include +#include #define DRIVER_VERSION "0.2" #define DRIVER_AUTHOR "Alex Williamson " @@ -89,6 +90,8 @@ struct vfio_dma { }; struct vfio_group { + /* iommu_group of mdev's parent device */ + struct iommu_group *parent_group; struct iommu_group *iommu_group; struct list_head next; }; @@ -1327,6 +1330,109 @@ static bool vfio_iommu_has_sw_msi(struct iommu_group *group, phys_addr_t *base) return ret; } +/* return 0 if the device is not spimdev. + * return 1 if the device is spimdev, the data will be updated with parent + * device's group. + * return -errno if other error. + */ +static int vfio_spimdev_type(struct device *dev, void *data) +{ + struct iommu_group **group = data; + struct iommu_group *pgroup; + int (*spimdev_mdev)(struct device *dev); + struct device *pdev; + int ret = 1; + + /* vfio_spimdev module is not configurated */ + spimdev_mdev = symbol_get(vfio_spimdev_is_spimdev); + if (!spimdev_mdev) + return 0; + + /* check if it belongs to vfio_spimdev device */ + if (!spimdev_mdev(dev)) { + ret = 0; + goto get_exit; + } + + pdev = dev->parent; + pgroup = iommu_group_get(pdev); + if (!pgroup) { + ret = -ENODEV; + goto get_exit; + } + + if (group) { + /* check if all parent devices is the same */ + if (*group && *group != pgroup) + ret = -ENODEV; + else + *group = pgroup; + } + + iommu_group_put(pgroup); + +get_exit: + symbol_put(vfio_spimdev_is_spimdev); + + return ret; +} + +/* return 0 or -errno */ +static int vfio_spimdev_bus(struct device *dev, void *data) +{ + struct bus_type **bus = data; + + if (!dev->bus) + return -ENODEV; + + /* ensure all devices has the same bus_type */ + if (*bus && *bus != dev->bus) + return -EINVAL; + + *bus = dev->bus; + return 0; +} + +/* return 0 means it is not spi group, 1 means it is, or -EXXX for error */ +static int vfio_iommu_type1_attach_spigroup(struct vfio_domain *domain, + struct vfio_group *group, + struct iommu_group *iommu_group) +{ + int ret; + struct bus_type *pbus = NULL; + struct iommu_group *pgroup = NULL; + + ret = iommu_group_for_each_dev(iommu_group, &pgroup, + vfio_spimdev_type); + if (ret < 0) + goto out; + else if (ret > 0) { + domain->domain = iommu_group_share_domain(pgroup); + if (IS_ERR(domain->domain)) + goto out; + ret = iommu_group_for_each_dev(pgroup, &pbus, + vfio_spimdev_bus); + if (ret < 0) + goto err_with_share_domain; + + if (pbus && iommu_capable(pbus, IOMMU_CAP_CACHE_COHERENCY)) + domain->prot |= IOMMU_CACHE; + + group->parent_group = pgroup; + INIT_LIST_HEAD(&domain->group_list); + list_add(&group->next, &domain->group_list); + + return 1; + } + + return 0; + +err_with_share_domain: + iommu_group_unshare_domain(pgroup); +out: + return ret; +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group) { @@ -1335,8 +1441,8 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, struct vfio_domain *domain, *d; struct bus_type *bus = NULL, *mdev_bus; int ret; - bool resv_msi, msi_remap; - phys_addr_t resv_msi_base; + bool resv_msi = false, msi_remap; + phys_addr_t resv_msi_base = 0; mutex_lock(&iommu->lock); @@ -1373,6 +1479,14 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, if (mdev_bus) { if ((bus == mdev_bus) && !iommu_present(bus)) { symbol_put(mdev_bus_type); + + ret = vfio_iommu_type1_attach_spigroup(domain, group, + iommu_group); + if (ret < 0) + goto out_free; + else if (ret > 0) + goto replay_check; + if (!iommu->external_domain) { INIT_LIST_HEAD(&domain->group_list); iommu->external_domain = domain; @@ -1451,12 +1565,13 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, vfio_test_domain_fgsp(domain); +replay_check: /* replay mappings on new domains */ ret = vfio_iommu_replay(iommu, domain); if (ret) goto out_detach; - if (resv_msi) { + if (!group->parent_group && resv_msi) { ret = iommu_get_msi_cookie(domain->domain, resv_msi_base); if (ret) goto out_detach; @@ -1471,7 +1586,10 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, out_detach: iommu_detach_group(domain->domain, iommu_group); out_domain: - iommu_domain_free(domain->domain); + if (group->parent_group) + iommu_group_unshare_domain(group->parent_group); + else + iommu_domain_free(domain->domain); out_free: kfree(domain); kfree(group); @@ -1533,6 +1651,7 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, struct vfio_iommu *iommu = iommu_data; struct vfio_domain *domain; struct vfio_group *group; + int ret; mutex_lock(&iommu->lock); @@ -1560,7 +1679,11 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, if (!group) continue; - iommu_detach_group(domain->domain, iommu_group); + if (group->parent_group) + iommu_group_unshare_domain(group->parent_group); + else + iommu_detach_group(domain->domain, iommu_group); + list_del(&group->next); kfree(group); /* @@ -1577,7 +1700,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, else vfio_iommu_unmap_unpin_reaccount(iommu); } - iommu_domain_free(domain->domain); + if (!ret) + iommu_domain_free(domain->domain); list_del(&domain->next); kfree(domain); } diff --git a/include/linux/vfio_spimdev.h b/include/linux/vfio_spimdev.h new file mode 100644 index 000000000000..f7e7d90013e1 --- /dev/null +++ b/include/linux/vfio_spimdev.h @@ -0,0 +1,95 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +#ifndef __VFIO_SPIMDEV_H +#define __VFIO_SPIMDEV_H + +#include +#include +#include +#include +#include + +struct vfio_spimdev_queue; +struct vfio_spimdev; + +/** + * struct vfio_spimdev_ops - WD device operations + * @get_queue: get a queue from the device according to algorithm + * @put_queue: free a queue to the device + * @is_q_updated: check whether the task is finished + * @mask_notify: mask the task irq of queue + * @mmap: mmap addresses of queue to user space + * @reset: reset the WD device + * @reset_queue: reset the queue + * @ioctl: ioctl for user space users of the queue + * @get_available_instances: get numbers of the queue remained + */ +struct vfio_spimdev_ops { + int (*get_queue)(struct vfio_spimdev *spimdev, unsigned long arg, + struct vfio_spimdev_queue **q); + int (*put_queue)(struct vfio_spimdev_queue *q); + int (*is_q_updated)(struct vfio_spimdev_queue *q); + void (*mask_notify)(struct vfio_spimdev_queue *q, int event_mask); + int (*mmap)(struct vfio_spimdev_queue *q, struct vm_area_struct *vma); + int (*reset)(struct vfio_spimdev *spimdev); + int (*reset_queue)(struct vfio_spimdev_queue *q); + long (*ioctl)(struct vfio_spimdev_queue *q, unsigned int cmd, + unsigned long arg); + int (*get_available_instances)(struct vfio_spimdev *spimdev); +}; + +struct vfio_spimdev_queue { + struct mutex mutex; + struct vfio_spimdev *spimdev; + int qid; + __u32 flags; + void *priv; + wait_queue_head_t wait; + struct mdev_device *mdev; + int fd; + int container; +#ifdef CONFIG_IOMMU_SVA + int pasid; +#endif +}; + +struct vfio_spimdev { + const char *name; + int status; + atomic_t ref; + struct module *owner; + const struct vfio_spimdev_ops *ops; + struct device *dev; + struct device cls_dev; + bool is_vf; + u32 iommu_type; + u32 dma_flag; + u32 dev_id; + void *priv; + int flags; + const char *api_ver; + struct mdev_parent_ops mdev_fops; +}; + +int vfio_spimdev_register(struct vfio_spimdev *spimdev); +void vfio_spimdev_unregister(struct vfio_spimdev *spimdev); +void vfio_spimdev_wake_up(struct vfio_spimdev_queue *q); +int vfio_spimdev_is_spimdev(struct device *dev); +struct vfio_spimdev *vfio_spimdev_pdev_spimdev(struct device *dev); +int vfio_spimdev_pasid_pri_check(int pasid); +int vfio_spimdev_get(struct device *dev); +int vfio_spimdev_put(struct device *dev); +struct vfio_spimdev *mdev_spimdev(struct mdev_device *mdev); + +extern struct mdev_type_attribute mdev_type_attr_flags; +extern struct mdev_type_attribute mdev_type_attr_name; +extern struct mdev_type_attribute mdev_type_attr_device_api; +extern struct mdev_type_attribute mdev_type_attr_available_instances; +#define VFIO_SPIMDEV_DEFAULT_MDEV_TYPE_ATTRS \ + &mdev_type_attr_name.attr, \ + &mdev_type_attr_device_api.attr, \ + &mdev_type_attr_available_instances.attr, \ + &mdev_type_attr_flags.attr + +#define _VFIO_SPIMDEV_REGION(vm_pgoff) (vm_pgoff & 0xf) + +#endif diff --git a/include/uapi/linux/vfio_spimdev.h b/include/uapi/linux/vfio_spimdev.h new file mode 100644 index 000000000000..3435e5c345b4 --- /dev/null +++ b/include/uapi/linux/vfio_spimdev.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +#ifndef _UAPIVFIO_SPIMDEV_H +#define _UAPIVFIO_SPIMDEV_H + +#include + +#define VFIO_SPIMDEV_CLASS_NAME "spimdev" + +/* Device ATTRs in parent dev SYSFS DIR */ +#define VFIO_SPIMDEV_PDEV_ATTRS_GRP_NAME "params" + +/* Parent device attributes */ +#define SPIMDEV_IOMMU_TYPE "iommu_type" +#define SPIMDEV_DMA_FLAG "dma_flag" + +/* Maximum length of algorithm name string */ +#define VFIO_SPIMDEV_ALG_NAME_SIZE 64 + +/* the bits used in SPIMDEV_DMA_FLAG attributes */ +#define VFIO_SPIMDEV_DMA_INVALID 0 +#define VFIO_SPIMDEV_DMA_SINGLE_PROC_MAP 1 +#define VFIO_SPIMDEV_DMA_MULTI_PROC_MAP 2 +#define VFIO_SPIMDEV_DMA_SVM 4 +#define VFIO_SPIMDEV_DMA_SVM_NO_FAULT 8 +#define VFIO_SPIMDEV_DMA_PHY 16 + +#define VFIO_SPIMDEV_CMD_GET_Q _IO('W', 1) +#endif From patchwork Wed Aug 1 10:22:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kenneth Lee X-Patchwork-Id: 143272 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp681400ljj; Wed, 1 Aug 2018 03:24:57 -0700 (PDT) X-Google-Smtp-Source: AAOMgpd5aBl+9VUhEph2jwXlMiXFcyl/jUS48qwJokuTMZ+S/q+1tKw576gMzy/WRZSxAguynQs4 X-Received: by 2002:a17:902:1682:: with SMTP id h2-v6mr24049501plh.327.1533119097672; Wed, 01 Aug 2018 03:24:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533119097; cv=none; d=google.com; s=arc-20160816; b=hzG07ssn48kEAlpJcQlMLEaoGdbxBRXHVIhcDuggmq/favzkm5XaO5VeJFvMJYANxg Cu4nSzCI8yVqXfqoVnV3hM22lT5vuoSj8qieU1cNQSBDEoZS7zN3MEwAeBWqI9SA/M79 JpH4A+OrPONHfR6/s2l2gsPmCRLktXACVuuXNQPcMxUgN6w+ywzRMSK4sxbXwK9XCAsc NT3qnhLzZi0+ocBJn3E10VViZrfTQ+9oxiKXtYc2jNr4mV2zHPkcfCoyycoxLDwS+XX7 mnzafhQGCmSkyN8TfsUyF88UcM5DtSid2FmheQQmzEH3NOYx+9O49BtxPJyGLE16b8Rr RGYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=jFA837p5UT4krKtQk6EEUkP0P4jx/vNRa9KSnuCmnFE=; b=E05w1E2AqW94oSPfvjpuwwQQbWEc+0ef1tyaQ9BulHzrGO73dQYwh5ESLiGfKSmG53 EFMGeo5UMLCuYygaEeWw6qtMq0duD3SwddAeq8By7o2jaG3/lX0849rkp4oCdvCq3CPF 4XGJoXa2YEFiJpFLdVgNISMABkTUF+Y3xgCa9NlHwpnsSaVWghRBHuCQwq83pY+3gY08 XRI1AAKJRvEeCRkFL4NeqLCps+Z1mektjjRUN+xNOy8Kj5ZCah7zwQvjWnPU8H9rUi7T p9CKs1DKixKWA4TWU2jxAc+fLPghWXEhOO2s4zORmOavvcQtXLnNJK4VuydSvSTDEFuk cviw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BA2eUJCY; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e14-v6si15734644pgj.413.2018.08.01.03.24.57; Wed, 01 Aug 2018 03:24:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BA2eUJCY; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733104AbeHAMKB (ORCPT + 1 other); Wed, 1 Aug 2018 08:10:01 -0400 Received: from mail-qt0-f195.google.com ([209.85.216.195]:39680 "EHLO mail-qt0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387559AbeHAMKA (ORCPT ); Wed, 1 Aug 2018 08:10:00 -0400 Received: by mail-qt0-f195.google.com with SMTP id q12-v6so19227058qtp.6; Wed, 01 Aug 2018 03:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=jFA837p5UT4krKtQk6EEUkP0P4jx/vNRa9KSnuCmnFE=; b=BA2eUJCYXPZVI924pTAoyt5DUbJRO5+ZpYrpDiIcONvoyj0+GspscbY95ynGBdjxYp r1DI2rQHH1hczR1jIU8nzW85eDZe8sxpLaKT36n/oTgCdK6Jt6r7LOr13yJ05NdCHXHr dKRl2gSdmFhUzZkDsm5a3Y0w3Mj5pgKIYo3go8YQZK3ywWoqkqLioYeJUaz5UXmeZrMM 2JLmqdUDvpgnx1fHAMG3yOECOrlXypleE+mMRpwqNKexNTYVvqccet1/sWW81RymX7zi Lq/3Jem9uiX9fLaKlpS3qACxE4I8Fh1SC2h7ndyDQSSvtAOoUgBVZ81Ow41B6ZBQhh58 xx4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=jFA837p5UT4krKtQk6EEUkP0P4jx/vNRa9KSnuCmnFE=; b=ME5LloNuh4nJPkkogrPL7r61hrB3EqeTXoUyr81xarZrLn9wJGqqn9rqYibbvkdAMF x5Wpq44mEDLZdCEMThWYdIhHgZEMiIquVG/LVjizp9XOdXwd8hK7OdMxm5vH93jJ3kT0 Opph3cYjXQLId+JaDL0gL9IOcYY23n6IWbesSgBSygkIfad0dv2NS5GHb3TM8y0JguNI 9DYouPy0NiKkU+PyOu+WmspYKNXEMvE3Vz6VbtOJkr8gtulq51O/fvmy2beRuDlFsYBU Upg8e44khPurXvTgVhiuuaFvRpdVNznNg/2ATrUqsLFPkBGTu7JNMPq2eovbxpm+225F gAvA== X-Gm-Message-State: AOUpUlG72sYtfUwOSn8THEyWn8n/yvIcov6c5B/07sD1z46WwuVsxcoY 8nAlKFYq9nH3lzrgNm9Prco= X-Received: by 2002:ac8:2ad8:: with SMTP id c24-v6mr24836477qta.117.1533119093103; Wed, 01 Aug 2018 03:24:53 -0700 (PDT) Received: from localhost.localdomain ([104.237.86.144]) by smtp.gmail.com with ESMTPSA id s19-v6sm11176890qtj.61.2018.08.01.03.24.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Aug 2018 03:24:52 -0700 (PDT) From: Kenneth Lee To: Jonathan Corbet , Herbert Xu , "David S . Miller" , Joerg Roedel , Alex Williamson , Kenneth Lee , Hao Fang , Zhou Wang , Zaibo Xu , Philippe Ombredanne , Greg Kroah-Hartman , Thomas Gleixner , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-accelerators@lists.ozlabs.org, Lu Baolu , Sanjay Kumar Cc: linuxarm@huawei.com Subject: [RFC PATCH 7/7] vfio/spimdev: add user sample for spimdev Date: Wed, 1 Aug 2018 18:22:21 +0800 Message-Id: <20180801102221.5308-8-nek.in.cn@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180801102221.5308-1-nek.in.cn@gmail.com> References: <20180801102221.5308-1-nek.in.cn@gmail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Kenneth Lee This is the sample code to demostrate how WrapDrive user application should be. It contains: 1. wd.[ch], the common library to provide WrapDrive interface. 2. drv/*, the user driver to access the hardware upon spimdev 3. test/*, the test application to use WrapDrive interface to access the hardware queue(s) of the accelerator. The Hisilicon HIP08 ZIP accelerator is used in this sample. Signed-off-by: Zaibo Xu Signed-off-by: Kenneth Lee Signed-off-by: Hao Fang Signed-off-by: Zhou Wang --- samples/warpdrive/AUTHORS | 2 + samples/warpdrive/ChangeLog | 1 + samples/warpdrive/Makefile.am | 9 + samples/warpdrive/NEWS | 1 + samples/warpdrive/README | 32 +++ samples/warpdrive/autogen.sh | 3 + samples/warpdrive/cleanup.sh | 13 + samples/warpdrive/configure.ac | 52 ++++ samples/warpdrive/drv/hisi_qm_udrv.c | 223 +++++++++++++++++ samples/warpdrive/drv/hisi_qm_udrv.h | 53 ++++ samples/warpdrive/test/Makefile.am | 7 + samples/warpdrive/test/comp_hw.h | 23 ++ samples/warpdrive/test/test_hisi_zip.c | 204 ++++++++++++++++ samples/warpdrive/wd.c | 325 +++++++++++++++++++++++++ samples/warpdrive/wd.h | 153 ++++++++++++ samples/warpdrive/wd_adapter.c | 74 ++++++ samples/warpdrive/wd_adapter.h | 43 ++++ 17 files changed, 1218 insertions(+) create mode 100644 samples/warpdrive/AUTHORS create mode 100644 samples/warpdrive/ChangeLog create mode 100644 samples/warpdrive/Makefile.am create mode 100644 samples/warpdrive/NEWS create mode 100644 samples/warpdrive/README create mode 100755 samples/warpdrive/autogen.sh create mode 100755 samples/warpdrive/cleanup.sh create mode 100644 samples/warpdrive/configure.ac create mode 100644 samples/warpdrive/drv/hisi_qm_udrv.c create mode 100644 samples/warpdrive/drv/hisi_qm_udrv.h create mode 100644 samples/warpdrive/test/Makefile.am create mode 100644 samples/warpdrive/test/comp_hw.h create mode 100644 samples/warpdrive/test/test_hisi_zip.c create mode 100644 samples/warpdrive/wd.c create mode 100644 samples/warpdrive/wd.h create mode 100644 samples/warpdrive/wd_adapter.c create mode 100644 samples/warpdrive/wd_adapter.h -- 2.17.1 diff --git a/samples/warpdrive/AUTHORS b/samples/warpdrive/AUTHORS new file mode 100644 index 000000000000..fe7dc2413b0d --- /dev/null +++ b/samples/warpdrive/AUTHORS @@ -0,0 +1,2 @@ +Kenneth Lee +Zaibo Xu diff --git a/samples/warpdrive/ChangeLog b/samples/warpdrive/ChangeLog new file mode 100644 index 000000000000..b1b716105590 --- /dev/null +++ b/samples/warpdrive/ChangeLog @@ -0,0 +1 @@ +init diff --git a/samples/warpdrive/Makefile.am b/samples/warpdrive/Makefile.am new file mode 100644 index 000000000000..41154a880a97 --- /dev/null +++ b/samples/warpdrive/Makefile.am @@ -0,0 +1,9 @@ +ACLOCAL_AMFLAGS = -I m4 +AUTOMAKE_OPTIONS = foreign subdir-objects +AM_CFLAGS=-Wall -O0 -fno-strict-aliasing + +lib_LTLIBRARIES=libwd.la +libwd_la_SOURCES=wd.c wd_adapter.c wd.h wd_adapter.h \ + drv/hisi_qm_udrv.c drv/hisi_qm_udrv.h + +SUBDIRS=. test diff --git a/samples/warpdrive/NEWS b/samples/warpdrive/NEWS new file mode 100644 index 000000000000..b1b716105590 --- /dev/null +++ b/samples/warpdrive/NEWS @@ -0,0 +1 @@ +init diff --git a/samples/warpdrive/README b/samples/warpdrive/README new file mode 100644 index 000000000000..3adf66b112fc --- /dev/null +++ b/samples/warpdrive/README @@ -0,0 +1,32 @@ +WD User Land Demonstration +========================== + +This directory contains some applications and libraries to demonstrate how a + +WrapDrive application can be constructed. + + +As a demo, we try to make it simple and clear for understanding. It is not + +supposed to be used in business scenario. + + +The directory contains the following elements: + +wd.[ch] + A demonstration WrapDrive fundamental library which wraps the basic + operations to the WrapDrive-ed device. + +wd_adapter.[ch] + User driver adaptor for wd.[ch] + +wd_utils.[ch] + Some utitlities function used by WD and its drivers + +drv/* + User drivers. It helps to fulfill the semantic of wd.[ch] for + particular hardware + +test/* + Test applications to use the wrapdrive library + diff --git a/samples/warpdrive/autogen.sh b/samples/warpdrive/autogen.sh new file mode 100755 index 000000000000..58deaf49de2a --- /dev/null +++ b/samples/warpdrive/autogen.sh @@ -0,0 +1,3 @@ +#!/bin/sh -x + +autoreconf -i -f -v diff --git a/samples/warpdrive/cleanup.sh b/samples/warpdrive/cleanup.sh new file mode 100755 index 000000000000..c5f3d21e5dc1 --- /dev/null +++ b/samples/warpdrive/cleanup.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +if [ -r Makefile ]; then + make distclean +fi + +FILES="aclocal.m4 autom4te.cache compile config.guess config.h.in config.log \ + config.status config.sub configure cscope.out depcomp install-sh \ + libsrc/Makefile libsrc/Makefile.in libtool ltmain.sh Makefile \ + ar-lib m4 \ + Makefile.in missing src/Makefile src/Makefile.in test/Makefile.in" + +rm -vRf $FILES diff --git a/samples/warpdrive/configure.ac b/samples/warpdrive/configure.ac new file mode 100644 index 000000000000..53262f3197c2 --- /dev/null +++ b/samples/warpdrive/configure.ac @@ -0,0 +1,52 @@ +AC_PREREQ([2.69]) +AC_INIT([wrapdrive], [0.1], [liguozhu@hisilicon.com]) +AC_CONFIG_SRCDIR([wd.c]) +AM_INIT_AUTOMAKE([1.10 no-define]) + +AC_CONFIG_MACRO_DIR([m4]) +AC_CONFIG_HEADERS([config.h]) + +# Checks for programs. +AC_PROG_CXX +AC_PROG_AWK +AC_PROG_CC +AC_PROG_CPP +AC_PROG_INSTALL +AC_PROG_LN_S +AC_PROG_MAKE_SET +AC_PROG_RANLIB + +AM_PROG_AR +AC_PROG_LIBTOOL +AM_PROG_LIBTOOL +LT_INIT +AM_PROG_CC_C_O + +AC_DEFINE([HAVE_SVA], [0], [enable SVA support]) +AC_ARG_ENABLE([sva], + [ --enable-sva enable to support sva feature], + AC_DEFINE([HAVE_SVA], [1])) + +# Checks for libraries. + +# Checks for header files. +AC_CHECK_HEADERS([fcntl.h stdint.h stdlib.h string.h sys/ioctl.h sys/time.h unistd.h]) + +# Checks for typedefs, structures, and compiler characteristics. +AC_CHECK_HEADER_STDBOOL +AC_C_INLINE +AC_TYPE_OFF_T +AC_TYPE_SIZE_T +AC_TYPE_UINT16_T +AC_TYPE_UINT32_T +AC_TYPE_UINT64_T +AC_TYPE_UINT8_T + +# Checks for library functions. +AC_FUNC_MALLOC +AC_FUNC_MMAP +AC_CHECK_FUNCS([memset munmap]) + +AC_CONFIG_FILES([Makefile + test/Makefile]) +AC_OUTPUT diff --git a/samples/warpdrive/drv/hisi_qm_udrv.c b/samples/warpdrive/drv/hisi_qm_udrv.c new file mode 100644 index 000000000000..1213dff8daae --- /dev/null +++ b/samples/warpdrive/drv/hisi_qm_udrv.c @@ -0,0 +1,223 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "hisi_qm_udrv.h" + +#if __AARCH64EL__ == 1 +#define mb() {asm volatile("dsb sy" : : : "memory"); } +#else +#warning "this file need to be used on AARCH64EL mode" +#define mb() +#endif + +#define QM_SQE_SIZE 128 +#define QM_CQE_SIZE 16 +#define QM_EQ_DEPTH 1024 + +/* cqe shift */ +#define CQE_PHASE(cq) (((*((__u32 *)(cq) + 3)) >> 16) & 0x1) +#define CQE_SQ_NUM(cq) ((*((__u32 *)(cq) + 2)) >> 16) +#define CQE_SQ_HEAD_INDEX(cq) ((*((__u32 *)(cq) + 2)) & 0xffff) + +#define QM_IOMEM_SIZE 4096 + +#define QM_DOORBELL_OFFSET 0x340 + +struct cqe { + __le32 rsvd0; + __le16 cmd_id; + __le16 rsvd1; + __le16 sq_head; + __le16 sq_num; + __le16 rsvd2; + __le16 w7; /* phase, status */ +}; + +struct hisi_qm_queue_info { + void *sq_base; + void *cq_base; + void *doorbell_base; + __u16 sq_tail_index; + __u16 sq_head_index; + __u16 cq_head_index; + __u16 sqn; + bool cqc_phase; + void *req_cache[QM_EQ_DEPTH]; + int is_sq_full; +}; + +int hacc_db(struct hisi_qm_queue_info *q, __u8 cmd, __u16 index, __u8 priority) +{ + void *base = q->doorbell_base; + __u16 sqn = q->sqn; + __u64 doorbell = 0; + + doorbell = (__u64)sqn | ((__u64)cmd << 16); + doorbell |= ((__u64)index | ((__u64)priority << 16)) << 32; + + *((__u64 *)base) = doorbell; + + return 0; +} + +static int hisi_qm_fill_sqe(void *msg, struct hisi_qm_queue_info *info, __u16 i) +{ + struct hisi_qm_msg *sqe = (struct hisi_qm_msg *)info->sq_base + i; + memcpy((void *)sqe, msg, sizeof(struct hisi_qm_msg)); + assert(!info->req_cache[i]); + info->req_cache[i] = msg; + + return 0; +} + +static int hisi_qm_recv_sqe(struct hisi_qm_msg *sqe, struct hisi_qm_queue_info *info, __u16 i) +{ + __u32 status = sqe->dw3 & 0xff; + __u32 type = sqe->dw9 & 0xff; + + if (status != 0 && status != 0x0d) { + fprintf(stderr, "bad status (s=%d, t=%d)\n", status, type); + return -EIO; + } + + assert(info->req_cache[i]); + memcpy((void *)info->req_cache[i], sqe, sizeof(struct hisi_qm_msg)); + return 0; +} + +int hisi_qm_set_queue_dio(struct wd_queue *q) +{ + struct hisi_qm_queue_info *info; + void *vaddr; + int ret; + + alloc_obj(info); + if (!info) + return -1; + + q->priv = info; + + vaddr = mmap(NULL, + QM_SQE_SIZE * QM_EQ_DEPTH + QM_CQE_SIZE * QM_EQ_DEPTH, + PROT_READ | PROT_WRITE, MAP_SHARED, q->fd, 4096); + if (vaddr <= 0) { + ret = (intptr_t)vaddr; + goto err_with_info; + } + info->sq_base = vaddr; + info->cq_base = vaddr + QM_SQE_SIZE * QM_EQ_DEPTH; + + vaddr = mmap(NULL, QM_IOMEM_SIZE, + PROT_READ | PROT_WRITE, MAP_SHARED, q->fd, 0); + if (vaddr <= 0) { + ret = (intptr_t)vaddr; + goto err_with_scq; + } + info->doorbell_base = vaddr + QM_DOORBELL_OFFSET; + info->sq_tail_index = 0; + info->sq_head_index = 0; + info->cq_head_index = 0; + info->cqc_phase = 1; + + info->is_sq_full = 0; + + return 0; + +err_with_scq: + munmap(info->sq_base, + QM_SQE_SIZE * QM_EQ_DEPTH + QM_CQE_SIZE * QM_EQ_DEPTH); +err_with_info: + free(info); + return ret; +} + +void hisi_qm_unset_queue_dio(struct wd_queue *q) +{ + struct hisi_qm_queue_info *info = (struct hisi_qm_queue_info *)q->priv; + + munmap(info->doorbell_base - QM_DOORBELL_OFFSET, QM_IOMEM_SIZE); + munmap(info->cq_base, QM_CQE_SIZE * QM_EQ_DEPTH); + munmap(info->sq_base, QM_SQE_SIZE * QM_EQ_DEPTH); + free(info); + q->priv = NULL; +} + +int hisi_qm_add_to_dio_q(struct wd_queue *q, void *req) +{ + struct hisi_qm_queue_info *info = (struct hisi_qm_queue_info *)q->priv; + __u16 i; + + if (info->is_sq_full) + return -EBUSY; + + i = info->sq_tail_index; + + hisi_qm_fill_sqe(req, q->priv, i); + + mb(); + + if (i == (QM_EQ_DEPTH - 1)) + i = 0; + else + i++; + + hacc_db(info, DOORBELL_CMD_SQ, i, 0); + + info->sq_tail_index = i; + + if (i == info->sq_head_index) + info->is_sq_full = 1; + + return 0; +} + +int hisi_qm_get_from_dio_q(struct wd_queue *q, void **resp) +{ + struct hisi_qm_queue_info *info = (struct hisi_qm_queue_info *)q->priv; + __u16 i = info->cq_head_index; + struct cqe *cq_base = info->cq_base; + struct hisi_qm_msg *sq_base = info->sq_base; + struct cqe *cqe = cq_base + i; + struct hisi_qm_msg *sqe; + int ret; + + if (info->cqc_phase == CQE_PHASE(cqe)) { + sqe = sq_base + CQE_SQ_HEAD_INDEX(cqe); + ret = hisi_qm_recv_sqe(sqe, info, i); + if (ret < 0) + return -EIO; + + if (info->is_sq_full) + info->is_sq_full = 0; + } else { + return -EAGAIN; + } + + *resp = info->req_cache[i]; + info->req_cache[i] = NULL; + + if (i == (QM_EQ_DEPTH - 1)) { + info->cqc_phase = !(info->cqc_phase); + i = 0; + } else + i++; + + hacc_db(info, DOORBELL_CMD_CQ, i, 0); + + info->cq_head_index = i; + info->sq_head_index = i; + + + return ret; +} diff --git a/samples/warpdrive/drv/hisi_qm_udrv.h b/samples/warpdrive/drv/hisi_qm_udrv.h new file mode 100644 index 000000000000..6a7a06a089c9 --- /dev/null +++ b/samples/warpdrive/drv/hisi_qm_udrv.h @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef __HZIP_DRV_H__ +#define __HZIP_DRV_H__ + +#include +#include "../wd.h" + +/* this is unnecessary big, the hardware should optimize it */ +struct hisi_qm_msg { + __u32 consumed; + __u32 produced; + __u32 comp_date_length; + __u32 dw3; + __u32 input_date_length; + __u32 lba_l; + __u32 lba_h; + __u32 dw7; /* ... */ + __u32 dw8; /* ... */ + __u32 dw9; /* ... */ + __u32 dw10; /* ... */ + __u32 priv_info; + __u32 dw12; /* ... */ + __u32 tag; + __u32 dest_avail_out; + __u32 rsvd0; + __u32 comp_head_addr_l; + __u32 comp_head_addr_h; + __u32 source_addr_l; + __u32 source_addr_h; + __u32 dest_addr_l; + __u32 dest_addr_h; + __u32 stream_ctx_addr_l; + __u32 stream_ctx_addr_h; + __u32 cipher_key1_addr_l; + __u32 cipher_key1_addr_h; + __u32 cipher_key2_addr_l; + __u32 cipher_key2_addr_h; + __u32 rsvd1[4]; +}; + +struct hisi_acc_qm_sqc { + __u16 sqn; +}; + +#define DOORBELL_CMD_SQ 0 +#define DOORBELL_CMD_CQ 1 + +int hisi_qm_set_queue_dio(struct wd_queue *q); +void hisi_qm_unset_queue_dio(struct wd_queue *q); +int hisi_qm_add_to_dio_q(struct wd_queue *q, void *req); +int hisi_qm_get_from_dio_q(struct wd_queue *q, void **resp); + +#endif diff --git a/samples/warpdrive/test/Makefile.am b/samples/warpdrive/test/Makefile.am new file mode 100644 index 000000000000..ad80e80a47d7 --- /dev/null +++ b/samples/warpdrive/test/Makefile.am @@ -0,0 +1,7 @@ +AM_CFLAGS=-Wall -O0 -fno-strict-aliasing + +bin_PROGRAMS=test_hisi_zip + +test_hisi_zip_SOURCES=test_hisi_zip.c + +test_hisi_zip_LDADD=../.libs/libwd.a diff --git a/samples/warpdrive/test/comp_hw.h b/samples/warpdrive/test/comp_hw.h new file mode 100644 index 000000000000..79328fd0c1a0 --- /dev/null +++ b/samples/warpdrive/test/comp_hw.h @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +/** + * This file is shared bewteen user and kernel space Wrapdrive which is + * including algorithm attibutions that both user and driver are caring for + */ + +#ifndef __VFIO_WDEV_COMP_H +#define __VFIO_WDEV_COMP_H + +/* De-compressing algorithms' parameters */ +struct vfio_wdev_comp_param { + __u32 window_size; + __u32 comp_level; + __u32 mode; + __u32 alg; +}; + +/* WD defines all the De-compressing algorithm names here */ +#define VFIO_WDEV_ZLIB "zlib" +#define VFIO_WDEV_GZIP "gzip" +#define VFIO_WDEV_LZ4 "lz4" + +#endif diff --git a/samples/warpdrive/test/test_hisi_zip.c b/samples/warpdrive/test/test_hisi_zip.c new file mode 100644 index 000000000000..72dc2c712649 --- /dev/null +++ b/samples/warpdrive/test/test_hisi_zip.c @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: GPL-2.0+ +#include +#include +#include +#include +#include +#include "../wd.h" +#include "comp_hw.h" +#include "../drv/hisi_qm_udrv.h" + +#if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(__CYGWIN__) +# include +# include +# define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY) +#else +# define SET_BINARY_MODE(file) +#endif + +#define PAGE_SHIFT 12 +#define PAGE_SIZE (1 << PAGE_SHIFT) + +#define ASIZE (8*512*4096) /*16MB*/ + +#define SYS_ERR_COND(cond, msg) \ +do { \ + if (cond) { \ + perror(msg); \ + exit(EXIT_FAILURE); \ + } \ +} while (0) + +#define ZLIB 0 +#define GZIP 1 + +#define CHUNK 65535 + + +int hizip_deflate(FILE *source, FILE *dest, int type) +{ + __u64 in, out; + struct wd_queue q; + struct hisi_qm_msg *msg, *recv_msg; + void *a, *b; + char *src, *dst; + int ret, total_len; + int output_num; + int fd, file_msize; + + q.container = -1; + q.mdev_name = "22e09922-7a82-11e8-9cf6-d74cffa9e87b"; + q.vfio_group_path = "/dev/vfio/7"; //fixme to the right path + q.iommu_ext_path = "/sys/class/spimdev/0000:75:00.0/device/params/iommu_type"; + q.dmaflag_ext_path = "/sys/class/spimdev/0000:75:00.0/device/params/dma_flag"; + q.device_api_path = "/sys/class/spimdev/0000:75:00.0/device/mdev_supported_types/hisi_zip-hisi_zip/device_api"; + ret = wd_request_queue(&q); + SYS_ERR_COND(ret, "wd_request_queue"); + + fprintf(stderr, "pasid=%d, dma_flag=%d\n", q.pasid, q.dma_flag); + fd = fileno(source); + struct stat s; + + if (fstat(fd, &s) < 0) { + close(fd); + perror("fd error\n"); + return -1; + } + total_len = s.st_size; + + if (!total_len) { + ret = -EINVAL; + SYS_ERR_COND(ret, "input file length zero"); + } + if (total_len > 16*1024*1024) { + fputs("error, input file size too large(<16MB)!\n", stderr); + goto release_q; + } + file_msize = !(total_len%PAGE_SIZE) ? total_len : + (total_len/PAGE_SIZE+1)*PAGE_SIZE; + /* mmap file and DMA mapping */ + a = mmap((void *)0x0, file_msize, PROT_READ | PROT_WRITE, + MAP_PRIVATE, fd, 0); + if (!a) { + fputs("mmap file fail!\n", stderr); + goto release_q; + } + ret = wd_mem_share(&q, a, file_msize, 0); + if (ret) { + fprintf(stderr, "wd_mem_share dma a buf fail!err=%d\n", -errno); + goto unmap_file; + } + /* Allocate some space and setup a DMA mapping */ + b = mmap((void *)0x0, ASIZE, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, 0, 0); + if (!b) { + fputs("mmap b fail!\n", stderr); + goto unshare_file; + } + memset(b, 0, ASIZE); + ret = wd_mem_share(&q, b, ASIZE, 0); + if (ret) { + fputs("wd_mem_share dma b buf fail!\n", stderr); + goto unmap_mem; + } + src = (char *)a; + dst = (char *)b; + + msg = malloc(sizeof(*msg)); + if (!msg) { + fputs("alloc msg fail!\n", stderr); + goto alloc_msg_fail; + } + memset((void *)msg, 0, sizeof(*msg)); + msg->input_date_length = total_len; + if (type == ZLIB) + msg->dw9 = 2; + else + msg->dw9 = 3; + msg->dest_avail_out = 0x800000; + + in = (__u64)src; + out = (__u64)dst; + + msg->source_addr_l = in & 0xffffffff; + msg->source_addr_h = in >> 32; + msg->dest_addr_l = out & 0xffffffff; + msg->dest_addr_h = out >> 32; + + ret = wd_send(&q, msg); + if (ret == -EBUSY) { + usleep(1); + goto recv_again; + } + SYS_ERR_COND(ret, "send fail!\n"); +recv_again: + ret = wd_recv(&q, (void **)&recv_msg); + if (ret == -EIO) { + fputs(" wd_recv fail!\n", stderr); + goto alloc_msg_fail; + /* synchronous mode, if get none, then get again */ + } else if (ret == -EAGAIN) + goto recv_again; + + output_num = recv_msg->produced; + /* add zlib compress head and write head + compressed date to a file */ + char zip_head[2] = {0x78, 0x9c}; + + fwrite(zip_head, 1, 2, dest); + fwrite((char *)out, 1, output_num, dest); + fclose(dest); + + free(msg); +alloc_msg_fail: + wd_mem_unshare(&q, b, ASIZE); +unmap_mem: + munmap(b, ASIZE); +unshare_file: + wd_mem_unshare(&q, a, file_msize); +unmap_file: + munmap(a, file_msize); +release_q: + wd_release_queue(&q); + + return ret; +} + +int main(int argc, char *argv[]) +{ + int alg_type = 0; + + /* avoid end-of-line conversions */ + SET_BINARY_MODE(stdin); + SET_BINARY_MODE(stdout); + + if (!argv[1]) { + fputs("<>\n", stderr); + goto EXIT; + } + + if (!strcmp(argv[1], "-z")) + alg_type = ZLIB; + else if (!strcmp(argv[1], "-g")) { + alg_type = GZIP; + } else if (!strcmp(argv[1], "-h")) { + fputs("[version]:1.0.2\n", stderr); + fputs("[usage]: ./test_hisi_zip [type] dest_file\n", + stderr); + fputs(" [type]:\n", stderr); + fputs(" -z = zlib\n", stderr); + fputs(" -g = gzip\n", stderr); + fputs(" -h = usage\n", stderr); + fputs("Example:\n", stderr); + fputs("./test_hisi_zip -z < test.data > out.data\n", stderr); + goto EXIT; + } else { + fputs("Unknow option\n", stderr); + fputs("<>\n", + stderr); + goto EXIT; + } + + hizip_deflate(stdin, stdout, alg_type); +EXIT: + return EXIT_SUCCESS; +} diff --git a/samples/warpdrive/wd.c b/samples/warpdrive/wd.c new file mode 100644 index 000000000000..e7855652374e --- /dev/null +++ b/samples/warpdrive/wd.c @@ -0,0 +1,325 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "config.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "wd.h" +#include "wd_adapter.h" + +#if (defined(HAVE_SVA) & HAVE_SVA) +static int _wd_bind_process(struct wd_queue *q) +{ + struct bind_data { + struct vfio_iommu_type1_bind bind; + struct vfio_iommu_type1_bind_process data; + } wd_bind; + int ret; + __u32 flags = 0; + + if (q->dma_flag & VFIO_SPIMDEV_DMA_MULTI_PROC_MAP) + flags = VFIO_IOMMU_BIND_PRIV; + else if (q->dma_flag & VFIO_SPIMDEV_DMA_SVM_NO_FAULT) + flags = VFIO_IOMMU_BIND_NOPF; + + wd_bind.bind.flags = VFIO_IOMMU_BIND_PROCESS; + wd_bind.bind.argsz = sizeof(wd_bind); + wd_bind.data.flags = flags; + ret = ioctl(q->container, VFIO_IOMMU_BIND, &wd_bind); + if (ret) + return ret; + q->pasid = wd_bind.data.pasid; + return ret; +} + +static int _wd_unbind_process(struct wd_queue *q) +{ + struct bind_data { + struct vfio_iommu_type1_bind bind; + struct vfio_iommu_type1_bind_process data; + } wd_bind; + __u32 flags = 0; + + if (q->dma_flag & VFIO_SPIMDEV_DMA_MULTI_PROC_MAP) + flags = VFIO_IOMMU_BIND_PRIV; + else if (q->dma_flag & VFIO_SPIMDEV_DMA_SVM_NO_FAULT) + flags = VFIO_IOMMU_BIND_NOPF; + + wd_bind.bind.flags = VFIO_IOMMU_BIND_PROCESS; + wd_bind.data.pasid = q->pasid; + wd_bind.data.flags = flags; + wd_bind.bind.argsz = sizeof(wd_bind); + + return ioctl(q->container, VFIO_IOMMU_UNBIND, &wd_bind); +} +#endif + +int wd_request_queue(struct wd_queue *q) +{ + struct vfio_group_status group_status = { + .argsz = sizeof(group_status) }; + int iommu_ext; + int ret; + + if (!q->vfio_group_path || + !q->device_api_path || + !q->iommu_ext_path) { + WD_ERR("please set vfio_group_path," + "device_api_path,and iommu_ext_path before call %s", __func__); + return -EINVAL; + } + + q->hw_type_id = 0; /* this can be set according to the device api_version in the future */ + + q->group = open(q->vfio_group_path, O_RDWR); + if (q->group < 0) { + WD_ERR("open vfio group(%s) fail, errno=%d\n", + q->vfio_group_path, errno); + return -errno; + } + + if (q->container <= 0) { + q->container = open("/dev/vfio/vfio", O_RDWR); + if (q->container < 0) { + WD_ERR("Create VFIO container fail!\n"); + ret = -ENODEV; + goto err_with_group; + } + } + + if (ioctl(q->container, VFIO_GET_API_VERSION) != VFIO_API_VERSION) { + WD_ERR("VFIO version check fail!\n"); + ret = -EINVAL; + goto err_with_container; + } + + q->dma_flag = _get_attr_int(q->dmaflag_ext_path); + if (q->dma_flag < 0) { + ret = q->dma_flag; + goto err_with_container; + } + + iommu_ext = _get_attr_int(q->iommu_ext_path); + if (iommu_ext < 0) { + ret = iommu_ext; + goto err_with_container; + } + ret = ioctl(q->container, VFIO_CHECK_EXTENSION, iommu_ext); + if (!ret) { + WD_ERR("VFIO iommu check (%d) fail (%d)!\n", iommu_ext, ret); + goto err_with_container; + } + + ret = _get_attr_str(q->device_api_path, q->hw_type); + if (ret) + goto err_with_container; + + ret = ioctl(q->group, VFIO_GROUP_GET_STATUS, &group_status); + if (!(group_status.flags & VFIO_GROUP_FLAGS_VIABLE)) { + WD_ERR("VFIO group is not viable\n"); + goto err_with_container; + } + + ret = ioctl(q->group, VFIO_GROUP_SET_CONTAINER, &q->container); + if (ret) { + WD_ERR("VFIO group fail on VFIO_GROUP_SET_CONTAINER\n"); + goto err_with_container; + } + + ret = ioctl(q->container, VFIO_SET_IOMMU, iommu_ext); + if (ret) { + WD_ERR("VFIO fail on VFIO_SET_IOMMU(%d)\n", iommu_ext); + goto err_with_container; + } + + q->mdev = ioctl(q->group, VFIO_GROUP_GET_DEVICE_FD, q->mdev_name); + if (q->mdev < 0) { + WD_ERR("VFIO fail on VFIO_GROUP_GET_DEVICE_FD (%d)\n", q->mdev); + ret = q->mdev; + goto err_with_container; + } +#if (defined(HAVE_SVA) & HAVE_SVA) + if (!(q->dma_flag & (VFIO_SPIMDEV_DMA_PHY | VFIO_SPIMDEV_DMA_SINGLE_PROC_MAP))) { + ret = _wd_bind_process(q); + if (ret) { + close(q->mdev); + WD_ERR("VFIO fails to bind process!\n"); + goto err_with_container; + + } + } +#endif + ret = q->fd = ioctl(q->mdev, VFIO_SPIMDEV_CMD_GET_Q, (unsigned long)q->pasid); + if (ret < 0) { + WD_ERR("get queue fail,ret=%d\n", errno); + goto err_with_mdev; + } + + ret = drv_open(q); + if (ret) + goto err_with_queue; + + return 0; + +err_with_queue: + close(q->fd); +err_with_mdev: + close(q->mdev); +err_with_container: + close(q->container); +err_with_group: + close(q->group); + return ret; +} + +void wd_release_queue(struct wd_queue *q) +{ + drv_close(q); + close(q->fd); +#if (defined(HAVE_SVA) & HAVE_SVA) + if (!(q->dma_flag & (VFIO_SPIMDEV_DMA_PHY | VFIO_SPIMDEV_DMA_SINGLE_PROC_MAP))) { + if (q->pasid <= 0) { + WD_ERR("Wd queue pasid ! pasid=%d\n", q->pasid); + return; + } + if (_wd_unbind_process(q)) { + WD_ERR("VFIO fails to unbind process!\n"); + return; + } + } +#endif + close(q->mdev); + close(q->container); + close(q->group); +} + +int wd_send(struct wd_queue *q, void *req) +{ + return drv_send(q, req); +} + +int wd_recv(struct wd_queue *q, void **resp) +{ + return drv_recv(q, resp); +} + +static int wd_flush_and_wait(struct wd_queue *q, __u16 ms) +{ + struct pollfd fds[1]; + + wd_flush(q); + fds[0].fd = q->fd; + fds[0].events = POLLIN; + return poll(fds, 1, ms); +} + +int wd_send_sync(struct wd_queue *q, void *req, __u16 ms) +{ + int ret; + + while (1) { + ret = wd_send(q, req); + if (ret == -EBUSY) { + ret = wd_flush_and_wait(q, ms); + if (ret) + return ret; + } else + return ret; + } +} + +int wd_recv_sync(struct wd_queue *q, void **resp, __u16 ms) +{ + int ret; + + while (1) { + ret = wd_recv(q, resp); + if (ret == -EBUSY) { + ret = wd_flush_and_wait(q, ms); + if (ret) + return ret; + } else + return ret; + } +} + +void wd_flush(struct wd_queue *q) +{ + drv_flush(q); +} + +static int _wd_mem_share_type1(struct wd_queue *q, const void *addr, + size_t size, int flags) +{ + struct vfio_iommu_type1_dma_map dma_map; + + if (q->dma_flag & VFIO_SPIMDEV_DMA_SVM_NO_FAULT) + return mlock(addr, size); + +#if (defined(HAVE_SVA) & HAVE_SVA) + else if ((q->dma_flag & VFIO_SPIMDEV_DMA_MULTI_PROC_MAP) && + (q->pasid > 0)) + dma_map.pasid = q->pasid; +#endif + else if ((q->dma_flag & VFIO_SPIMDEV_DMA_SINGLE_PROC_MAP)) + ; //todo + else + return -1; + + dma_map.vaddr = (__u64)addr; + dma_map.size = size; + dma_map.iova = (__u64)addr; + dma_map.flags = + VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE | flags; + dma_map.argsz = sizeof(dma_map); + + return ioctl(q->container, VFIO_IOMMU_MAP_DMA, &dma_map); +} + +static void _wd_mem_unshare_type1(struct wd_queue *q, const void *addr, + size_t size) +{ +#if (defined(HAVE_SVA) & HAVE_SVA) + struct vfio_iommu_type1_dma_unmap dma_unmap; +#endif + + if (q->dma_flag & VFIO_SPIMDEV_DMA_SVM_NO_FAULT) { + (void)munlock(addr, size); + return; + } + +#if (defined(HAVE_SVA) & HAVE_SVA) + dma_unmap.iova = (__u64)addr; + if ((q->dma_flag & VFIO_SPIMDEV_DMA_MULTI_PROC_MAP) && (q->pasid > 0)) + dma_unmap.flags = 0; + dma_unmap.size = size; + dma_unmap.argsz = sizeof(dma_unmap); + ioctl(q->container, VFIO_IOMMU_UNMAP_DMA, &dma_unmap); +#endif +} + +int wd_mem_share(struct wd_queue *q, const void *addr, size_t size, int flags) +{ + if (drv_can_do_mem_share(q)) + return drv_share(q, addr, size, flags); + else + return _wd_mem_share_type1(q, addr, size, flags); +} + +void wd_mem_unshare(struct wd_queue *q, const void *addr, size_t size) +{ + if (drv_can_do_mem_share(q)) + drv_unshare(q, addr, size); + else + _wd_mem_unshare_type1(q, addr, size); +} + diff --git a/samples/warpdrive/wd.h b/samples/warpdrive/wd.h new file mode 100644 index 000000000000..d9f3b2fa6c73 --- /dev/null +++ b/samples/warpdrive/wd.h @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef __WD_H +#define __WD_H +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "../../include/uapi/linux/vfio.h" +#include "../../include/uapi/linux/vfio_spimdev.h" + +#define SYS_VAL_SIZE 16 +#define PATH_STR_SIZE 256 +#define WD_NAME_SIZE 64 +#define WD_MAX_MEMLIST_SZ 128 + + +#ifndef dma_addr_t +#define dma_addr_t __u64 +#endif + +typedef int bool; + +#ifndef true +#define true 1 +#endif + +#ifndef false +#define false 0 +#endif + +/* the flags used by wd_capa->flags, the high 16bits are for algorithm + * and the low 16bits are for Framework + */ +#define WD_FLAGS_FW_PREFER_LOCAL_ZONE 1 + +#define WD_FLAGS_FW_MASK 0x0000FFFF +#ifndef WD_ERR +#define WD_ERR(format, args...) fprintf(stderr, format, ##args) +#endif + +/* Default page size should be 4k size */ +#define WDQ_MAP_REGION(region_index) ((region_index << 12) & 0xf000) +#define WDQ_MAP_Q(q_index) ((q_index << 16) & 0xffff0000) + +static inline void wd_reg_write(void *reg_addr, uint32_t value) +{ + *((volatile uint32_t *)reg_addr) = value; +} + +static inline uint32_t wd_reg_read(void *reg_addr) +{ + uint32_t temp; + + temp = *((volatile uint32_t *)reg_addr); + + return temp; +} + +static inline int _get_attr_str(const char *path, char value[PATH_STR_SIZE]) +{ + int fd, ret; + + fd = open(path, O_RDONLY); + if (fd < 0) { + WD_ERR("open %s fail\n", path); + return fd; + } + memset(value, 0, PATH_STR_SIZE); + ret = read(fd, value, PATH_STR_SIZE); + if (ret > 0) { + close(fd); + return 0; + } + close(fd); + + WD_ERR("read nothing from %s\n", path); + return -EINVAL; +} + +static inline int _get_attr_int(const char *path) +{ + char value[PATH_STR_SIZE]; + _get_attr_str(path, value); + return atoi(value); +} + +/* Memory in accelerating message can be different */ +enum wd_addr_flags { + WD_AATTR_INVALID = 0, + + /* Common user virtual memory */ + _WD_AATTR_COM_VIRT = 1, + + /* Physical address*/ + _WD_AATTR_PHYS = 2, + + /* I/O virtual address*/ + _WD_AATTR_IOVA = 4, + + /* SGL, user cares for */ + WD_AATTR_SGL = 8, +}; + +#define WD_CAPA_PRIV_DATA_SIZE 64 + +#define alloc_obj(objp) do { \ + objp = malloc(sizeof(*objp)); \ + memset(objp, 0, sizeof(*objp)); \ +}while(0) +#define free_obj(objp) if (objp)free(objp) + +struct wd_queue { + const char *mdev_name; + char hw_type[PATH_STR_SIZE]; + int hw_type_id; + int dma_flag; + void *priv; /* private data used by the drv layer */ + int container; + int group; + int mdev; + int fd; + int pasid; + int iommu_type; + char *vfio_group_path; + char *iommu_ext_path; + char *dmaflag_ext_path; + char *device_api_path; +}; + +extern int wd_request_queue(struct wd_queue *q); +extern void wd_release_queue(struct wd_queue *q); +extern int wd_send(struct wd_queue *q, void *req); +extern int wd_recv(struct wd_queue *q, void **resp); +extern void wd_flush(struct wd_queue *q); +extern int wd_send_sync(struct wd_queue *q, void *req, __u16 ms); +extern int wd_recv_sync(struct wd_queue *q, void **resp, __u16 ms); +extern int wd_mem_share(struct wd_queue *q, const void *addr, + size_t size, int flags); +extern void wd_mem_unshare(struct wd_queue *q, const void *addr, size_t size); + +/* for debug only */ +extern int wd_dump_all_algos(void); + +/* this is only for drv used */ +extern int wd_set_queue_attr(struct wd_queue *q, const char *name, + char *value); +extern int __iommu_type(struct wd_queue *q); +#endif diff --git a/samples/warpdrive/wd_adapter.c b/samples/warpdrive/wd_adapter.c new file mode 100644 index 000000000000..d8d55b75e99a --- /dev/null +++ b/samples/warpdrive/wd_adapter.c @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + + +#include "wd_adapter.h" +#include "./drv/hisi_qm_udrv.h" + +static struct wd_drv_dio_if hw_dio_tbl[] = { { + .hw_type = "hisi_qm_v1", + .open = hisi_qm_set_queue_dio, + .close = hisi_qm_unset_queue_dio, + .send = hisi_qm_add_to_dio_q, + .recv = hisi_qm_get_from_dio_q, + }, + /* Add other drivers direct IO operations here */ +}; + +/* todo: there should be some stable way to match the device and the driver */ +#define MAX_HW_TYPE (sizeof(hw_dio_tbl) / sizeof(hw_dio_tbl[0])) + +int drv_open(struct wd_queue *q) +{ + int i; + + //todo: try to find another dev if the user driver is not avaliable + for (i = 0; i < MAX_HW_TYPE; i++) { + if (!strcmp(q->hw_type, + hw_dio_tbl[i].hw_type)) { + q->hw_type_id = i; + return hw_dio_tbl[q->hw_type_id].open(q); + } + } + WD_ERR("No matching driver to use!\n"); + errno = ENODEV; + return -ENODEV; +} + +void drv_close(struct wd_queue *q) +{ + hw_dio_tbl[q->hw_type_id].close(q); +} + +int drv_send(struct wd_queue *q, void *req) +{ + return hw_dio_tbl[q->hw_type_id].send(q, req); +} + +int drv_recv(struct wd_queue *q, void **req) +{ + return hw_dio_tbl[q->hw_type_id].recv(q, req); +} + +int drv_share(struct wd_queue *q, const void *addr, size_t size, int flags) +{ + return hw_dio_tbl[q->hw_type_id].share(q, addr, size, flags); +} + +void drv_unshare(struct wd_queue *q, const void *addr, size_t size) +{ + hw_dio_tbl[q->hw_type_id].unshare(q, addr, size); +} + +bool drv_can_do_mem_share(struct wd_queue *q) +{ + return hw_dio_tbl[q->hw_type_id].share != NULL; +} + +void drv_flush(struct wd_queue *q) +{ + if (hw_dio_tbl[q->hw_type_id].flush) + hw_dio_tbl[q->hw_type_id].flush(q); +} diff --git a/samples/warpdrive/wd_adapter.h b/samples/warpdrive/wd_adapter.h new file mode 100644 index 000000000000..bb3ab3ec112a --- /dev/null +++ b/samples/warpdrive/wd_adapter.h @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0 +/* the common drv header define the unified interface for wd */ +#ifndef __WD_ADAPTER_H__ +#define __WD_ADAPTER_H__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +#include "wd.h" + +struct wd_drv_dio_if { + char *hw_type; + int (*open)(struct wd_queue *q); + void (*close)(struct wd_queue *q); + int (*set_pasid)(struct wd_queue *q); + int (*unset_pasid)(struct wd_queue *q); + int (*send)(struct wd_queue *q, void *req); + int (*recv)(struct wd_queue *q, void **req); + void (*flush)(struct wd_queue *q); + int (*share)(struct wd_queue *q, const void *addr, + size_t size, int flags); + int (*unshare)(struct wd_queue *q, const void *addr, size_t size); +}; + +extern int drv_open(struct wd_queue *q); +extern void drv_close(struct wd_queue *q); +extern int drv_send(struct wd_queue *q, void *req); +extern int drv_recv(struct wd_queue *q, void **req); +extern void drv_flush(struct wd_queue *q); +extern int drv_share(struct wd_queue *q, const void *addr, + size_t size, int flags); +extern void drv_unshare(struct wd_queue *q, const void *addr, size_t size); +extern bool drv_can_do_mem_share(struct wd_queue *q); + +#endif