From patchwork Wed Feb 26 02:37:52 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: kim-phillips X-Patchwork-Id: 25339 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ob0-f198.google.com (mail-ob0-f198.google.com [209.85.214.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 7454920143 for ; Wed, 26 Feb 2014 02:40:13 +0000 (UTC) Received: by mail-ob0-f198.google.com with SMTP id vb8sf600148obc.5 for ; Tue, 25 Feb 2014 18:40:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:date :message-id:in-reply-to:references:cc:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=4KukmSV2k34wWp/SZQGLcDXSC6RaOJuUU6kTfWUbbOg=; b=k13ZvzuiIPuP5vRcoWtuOAaG14Gzt+V/YI86nMNnztlNhdws3eD12RrtLPwBG6FQIM aFpL5GtYmmXJAnk485SgVV4D4N8JRIfSWzPjUxu++fIgMM0rcrtF/yhQT/72BzR0s4LH UPa+DgTMqLg+1x4OEu9iZUJsLGQheHysIJQtwbdi/OAKIqEbwe+sedjZPDnE1wLZop7+ gAdEU41lXsUT04f4hzP0ImNKkAnnsI9wknpl0JJrC8kWJKOznd2cCps7eg+BfTT8s8xP iLq4AyQDrfBxmZFW7d/IlhP3RSPGGRF9fx6sO6p7JD6PdDVdB4tRLALLiiYESAfy6ahW aACw== X-Gm-Message-State: ALoCoQmf6Vrca2CeQiHdpnRZZQEnBNWY8ioS0J6fxWPnjm6pVknijoJZtBY3RA45aLcSpXzgU6Xh X-Received: by 10.182.60.195 with SMTP id j3mr263562obr.17.1393382412984; Tue, 25 Feb 2014 18:40:12 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.25.33 with SMTP id 30ls42926qgs.22.gmail; Tue, 25 Feb 2014 18:40:12 -0800 (PST) X-Received: by 10.52.65.171 with SMTP id y11mr24155vds.52.1393382412815; Tue, 25 Feb 2014 18:40:12 -0800 (PST) Received: from mail-ve0-f182.google.com (mail-ve0-f182.google.com [209.85.128.182]) by mx.google.com with ESMTPS id v7si7338202vet.28.2014.02.25.18.40.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Feb 2014 18:40:12 -0800 (PST) Received-SPF: neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.182; Received: by mail-ve0-f182.google.com with SMTP id jy13so1495027veb.41 for ; Tue, 25 Feb 2014 18:40:12 -0800 (PST) X-Received: by 10.58.190.99 with SMTP id gp3mr244448vec.32.1393382412725; Tue, 25 Feb 2014 18:40:12 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp183976vcz; Tue, 25 Feb 2014 18:40:12 -0800 (PST) X-Received: by 10.140.92.65 with SMTP id a59mr4488398qge.34.1393382412292; Tue, 25 Feb 2014 18:40:12 -0800 (PST) Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id f2si1310200qaf.114.2014.02.25.18.40.12 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 25 Feb 2014 18:40:12 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Received: from localhost ([::1]:37908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIUPr-0001mQ-TS for patch@linaro.org; Tue, 25 Feb 2014 21:40:11 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59994) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIUOU-0007jQ-Ib for qemu-devel@nongnu.org; Tue, 25 Feb 2014 21:38:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WIUOL-0001J6-Q6 for qemu-devel@nongnu.org; Tue, 25 Feb 2014 21:38:46 -0500 Received: from mail-ob0-f180.google.com ([209.85.214.180]:49376) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WIUOL-0001J2-Jy for qemu-devel@nongnu.org; Tue, 25 Feb 2014 21:38:37 -0500 Received: by mail-ob0-f180.google.com with SMTP id vb8so155164obc.25 for ; Tue, 25 Feb 2014 18:38:37 -0800 (PST) X-Received: by 10.182.107.232 with SMTP id hf8mr445786obb.75.1393382317033; Tue, 25 Feb 2014 18:38:37 -0800 (PST) Received: from ntel.linksys (cpe-70-112-130-68.austin.res.rr.com. [70.112.130.68]) by mx.google.com with ESMTPSA id kn10sm145842968oeb.0.2014.02.25.18.38.35 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 25 Feb 2014 18:38:36 -0800 (PST) From: Kim Phillips To: qemu-devel@nongnu.org Date: Tue, 25 Feb 2014 20:37:52 -0600 Message-Id: <1393382272-29021-3-git-send-email-kim.phillips@linaro.org> X-Mailer: git-send-email 1.9.0 In-Reply-To: <1393382272-29021-1-git-send-email-kim.phillips@linaro.org> References: <1393382272-29021-1-git-send-email-kim.phillips@linaro.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.214.180 Cc: Peter Maydell , kim.phillips@freescale.com, eric.auger@linaro.org, kim.phillips@linaro.org, agraf@suse.de, stuart.yoder@freescale.com, alex.williamson@redhat.com, Antonios Motakis , kvmarm@lists.cs.columbia.edu, christoffer.dall@linaro.org Subject: [Qemu-devel] [RFC 2/2] hw/misc/vfio: add vfio-platform support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: kim.phillips@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 We basically add support for the SysBusDevice type in addition to the existing PCIDevice support. This involves taking common code from the existing vfio_initfn(), and putting it under a new vfio_find_get_group(), that both vfio_initfn() and the new vfio_platform_realize() call. Since realize() returns void, unlike PCIDevice's initfn(), error codes are moved into the error message text with %m. Some exceptions to the PCI path are added for platform devices, mostly in the form of early returns, since less setup is needed. I chose to reuse VFIODevice's config_size to determine whether the device was a PCI device or a platform device, which might need to change. Currently only MMIO access is supported at this time. It works because of qemu's stage 1 translation, but needs to be in stage 2 in case other entities than the guest OS want to access it. A KVM patch to address this is in the works. The perceived path for future QEMU development is: - support allocating a variable number of regions - VFIODevice's bars[] become dynamically allocated *regions - VFIOBAR's device fd replaced with parent VFIODevice ptr, to facilitate BAR r/w ops calling vfio_eoi() - add support for interrupts - verify and test platform dev unmap path - test existing PCI path for any regressions - add support for creating platform devices on the qemu command line - currently device address specification is hardcoded for test development on Calxeda Midway's fff51000.ethernet device - reset is not supported and registration of reset functions is bypassed for platform devices. - there is no standard means of resetting a platform device, unsure if it suffices to be handled at device--VFIO binding time Signed-off-by: Kim Phillips --- hw/misc/vfio.c | 180 ++++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 145 insertions(+), 35 deletions(-) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index 8db182f..eed24db 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -32,6 +32,7 @@ #include "hw/pci/msi.h" #include "hw/pci/msix.h" #include "hw/pci/pci.h" +#include "hw/sysbus.h" #include "qemu-common.h" #include "qemu/error-report.h" #include "qemu/event_notifier.h" @@ -166,7 +167,10 @@ typedef struct VFIOMSIXInfo { } VFIOMSIXInfo; typedef struct VFIODevice { - PCIDevice pdev; + union { + PCIDevice pdev; + SysBusDevice sbdev; + }; int fd; VFIOINTx intx; unsigned int config_size; @@ -180,6 +184,8 @@ typedef struct VFIODevice { VFIOMSIXInfo *msix; int nr_vectors; /* Number of MSI/MSIX vectors currently in use */ int interrupt; /* Current interrupt type */ + char *name; /* platform device name, e.g., fff51000.ethernet */ + int nr_regions; /* platform devices' number of regions */ VFIOBAR bars[PCI_NUM_REGIONS - 1]; /* No ROM */ VFIOVGA vga; /* 0xa0000, 0x3b0, 0x3c0 */ PCIHostDeviceAddress host; @@ -2497,8 +2503,6 @@ empty_region: memory_region_init(submem, OBJECT(vdev), name, 0); } - memory_region_add_subregion(mem, offset, submem); - return ret; } @@ -2552,6 +2556,7 @@ static void vfio_map_bar(VFIODevice *vdev, int nr) &bar->mmap_mem, &bar->mmap, size, 0, name)) { error_report("%s unsupported. Performance may be slow", name); } + memory_region_add_subregion(&bar->mem, 0, &bar->mmap_mem); if (vdev->msix && vdev->msix->table_bar == nr) { unsigned start; @@ -2566,15 +2571,38 @@ static void vfio_map_bar(VFIODevice *vdev, int nr) &vdev->msix->mmap, size, start, name)) { error_report("%s unsupported. Performance may be slow", name); } + memory_region_add_subregion(&bar->mem, start, &vdev->msix->mmap_mem); } vfio_bar_quirk_setup(vdev, nr); } +static void vfio_map_region(VFIODevice *vdev, int nr) +{ + VFIOBAR *bar = &vdev->bars[nr]; + unsigned size = bar->size; + char name[64]; + + snprintf(name, sizeof(name), "VFIO %s region %d mmap", vdev->name, nr); + + if (vfio_mmap_bar(vdev, bar, &bar->mem, + &bar->mmap_mem, &bar->mmap, size, 0, name)) { + error_report("error mmapping %s: %m", name); + } +} + static void vfio_map_bars(VFIODevice *vdev) { int i; + if (!vdev->config_size) { + /* platform device */ + for (i = 0; i < vdev->nr_regions; i++) { + vfio_map_region(vdev, i); + } + return; + } + for (i = 0; i < PCI_ROM_SLOT; i++) { vfio_map_bar(vdev, i); } @@ -3144,7 +3172,8 @@ static void vfio_pci_reset_handler(void *opaque) QLIST_FOREACH(group, &group_list, next) { QLIST_FOREACH(vdev, &group->device_list, next) { - if (vdev->needs_reset) { + /* HACK: restrict reset to PCI devices (have config_size) for now */ + if (vdev->config_size && vdev->needs_reset) { vfio_pci_hot_reset_multi(vdev); } } @@ -3418,25 +3447,18 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev) DPRINTF("Device %s flags: %u, regions: %u, irgs: %u\n", name, dev_info.flags, dev_info.num_regions, dev_info.num_irqs); - if (!(dev_info.flags & VFIO_DEVICE_FLAGS_PCI)) { - error_report("vfio: Um, this isn't a PCI device"); - goto error; - } - + vdev->nr_regions = dev_info.num_regions; vdev->reset_works = !!(dev_info.flags & VFIO_DEVICE_FLAGS_RESET); - if (dev_info.num_regions < VFIO_PCI_CONFIG_REGION_INDEX + 1) { + if (dev_info.num_regions > PCI_NUM_REGIONS) || + ((dev_info.flags & VFIO_DEVICE_FLAGS_PCI) && + (dev_info.num_regions < VFIO_PCI_CONFIG_REGION_INDEX + 1)) { error_report("vfio: unexpected number of io regions %u", dev_info.num_regions); goto error; } - if (dev_info.num_irqs < VFIO_PCI_MSIX_IRQ_INDEX + 1) { - error_report("vfio: unexpected number of irqs %u", dev_info.num_irqs); - goto error; - } - - for (i = VFIO_PCI_BAR0_REGION_INDEX; i < VFIO_PCI_ROM_REGION_INDEX; i++) { + for (i = 0; i < dev_info.num_regions; i++) { reg_info.index = i; ret = ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, ®_info); @@ -3458,6 +3480,15 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev) QLIST_INIT(&vdev->bars[i].quirks); } + /* following is for PCI devices, at least for now */ + if (!(dev_info.flags & VFIO_DEVICE_FLAGS_PCI)) + return 0; + + if (dev_info.num_irqs < VFIO_PCI_MSIX_IRQ_INDEX + 1) { + error_report("vfio: unexpected number of irqs %u", dev_info.num_irqs); + goto error; + } + reg_info.index = VFIO_PCI_CONFIG_REGION_INDEX; ret = ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, ®_info); @@ -3659,32 +3690,25 @@ static void vfio_unregister_err_notifier(VFIODevice *vdev) event_notifier_cleanup(&vdev->err_notifier); } -static int vfio_initfn(PCIDevice *pdev) +static VFIOGroup *vfio_find_get_group(char *path) { - VFIODevice *pvdev, *vdev = DO_UPCAST(VFIODevice, pdev, pdev); VFIOGroup *group; - char path[PATH_MAX], iommu_group_path[PATH_MAX], *group_name; + char iommu_group_path[PATH_MAX], *group_name; ssize_t len; struct stat st; int groupid; - int ret; - /* Check that the host device exists */ - snprintf(path, sizeof(path), - "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/", - vdev->host.domain, vdev->host.bus, vdev->host.slot, - vdev->host.function); if (stat(path, &st) < 0) { - error_report("vfio: error: no such host device: %s", path); - return -errno; + error_report("vfio: error: no such host device: %s: %m", path); + return NULL; } strncat(path, "iommu_group", sizeof(path) - strlen(path) - 1); len = readlink(path, iommu_group_path, PATH_MAX); if (len <= 0) { - error_report("vfio: error no iommu_group for device"); - return -errno; + error_report("vfio: error no iommu_group for device: %m"); + return NULL; } iommu_group_path[len] = 0; @@ -3692,18 +3716,37 @@ static int vfio_initfn(PCIDevice *pdev) if (sscanf(group_name, "%d", &groupid) != 1) { error_report("vfio: error reading %s: %m", path); - return -errno; + return NULL; } - DPRINTF("%s(%04x:%02x:%02x.%x) group %d\n", __func__, vdev->host.domain, - vdev->host.bus, vdev->host.slot, vdev->host.function, groupid); - group = vfio_get_group(groupid); if (!group) { - error_report("vfio: failed to get group %d", groupid); - return -ENOENT; + error_report("vfio: failed to get group %d: %m", groupid); + return NULL; } + return group; +} + +static int vfio_initfn(PCIDevice *pdev) +{ + VFIODevice *pvdev, *vdev = DO_UPCAST(VFIODevice, pdev, pdev); + VFIOGroup *group; + char path[PATH_MAX]; + int ret; + + /* Check that the host device exists */ + snprintf(path, sizeof(path), + "/sys/bus/pci/devices/%04x:%02x:%02x.%01x/", + vdev->host.domain, vdev->host.bus, vdev->host.slot, + vdev->host.function); + + group = vfio_find_get_group(path); + + DPRINTF("%s(%04x:%02x:%02x.%x) group %d\n", __func__, vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function, + group->groupid); + snprintf(path, sizeof(path), "%04x:%02x:%02x.%01x", vdev->host.domain, vdev->host.bus, vdev->host.slot, vdev->host.function); @@ -3916,3 +3959,70 @@ static void register_vfio_pci_dev_type(void) } type_init(register_vfio_pci_dev_type) + + +/* + * VFIO platform devices + */ +static void vfio_platform_realize(DeviceState *dev, Error **errp) +{ + SysBusDevice *sbdev = SYS_BUS_DEVICE(dev); + VFIODevice *vdev = DO_UPCAST(VFIODevice, sbdev, sbdev); + VFIOGroup *group; + char path[PATH_MAX]; + int ret; + + vdev->name = malloc(PATH_MAX); + strcpy(vdev->name, "fff51000.ethernet"); + + /* Check that the host device exists */ + snprintf(path, sizeof(path), + "/sys/bus/platform/devices/%s/", vdev->name); + + group = vfio_find_get_group(path); + if (!group) { + error_report("vfio: failed to get group for device %s", path); + return; + } + + strcpy(path, vdev->name); + ret = vfio_get_device(group, path, vdev); + if (ret) { + error_report("vfio: failed to get device %s", path); + vfio_put_group(group); + return; + } + + vfio_map_bars(vdev); + + sysbus_init_mmio(sbdev, &vdev->bars[0].mmap_mem); +} + +static const VMStateDescription vfio_platform_vmstate = { + .name = "vfio-platform", + .unmigratable = 1, +}; + +static void vfio_platform_dev_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + dc->realize = vfio_platform_realize; + dc->vmsd = &vfio_platform_vmstate; + dc->desc = "VFIO-based platform device assignment"; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +} + +static const TypeInfo vfio_platform_dev_info = { + .name = "vfio-platform", + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(VFIODevice), + .class_init = vfio_platform_dev_class_init, +}; + +static void register_vfio_platform_dev_type(void) +{ + type_register_static(&vfio_platform_dev_info); +} + +type_init(register_vfio_platform_dev_type)