From patchwork Tue Jan 13 14:25:29 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 43017 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ee0-f69.google.com (mail-ee0-f69.google.com [74.125.83.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id E198320DE8 for ; Tue, 13 Jan 2015 14:36:26 +0000 (UTC) Received: by mail-ee0-f69.google.com with SMTP id d49sf2112137eek.0 for ; Tue, 13 Jan 2015 06:36:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe:list-post :list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list:list-archive; bh=TI8+DSI5rebOU/rfUvgaTpm5P9zDIoN0Z2oqNByk7IU=; b=bX1Omhtgmapm0Sr7eyKVVjMeVPP26f6G1n9h3GEL1GtS2Je45UKWkc6NB5crKeMtMO Gsbj/T7c4DNhZa9sRzUoG9djpC3Zut4V3kBmwcCqtiHW1PUC8SgN0pyn3nGxpxEfj9B7 giROHX7he6Ku2ucyAlJ6tMCclK7uBWiL0/Lo/x1I5XhSKyDykX91clQHKve+Pbt23qag 010Pt/LgmTSqmjFbukCyULP2mE0d2zDXnhZQdvQWR1GYu4rip3HuSvBTuVJBvYYvy+g7 ITCpetdYLbdd1GfSWDyV98o76EkEhuSPxRmbvkeP1aaUp7i557uTJ+LSAdraavJ5Ui1B s3ig== X-Gm-Message-State: ALoCoQmhum+1i7KemOE8eqvVJD38vMYGd8Oj6RPhw7zOnZbRd4cQyA5Dc3gS3S9ZpxM55Jyu0i/x X-Received: by 10.152.6.202 with SMTP id d10mr10910laa.10.1421159786048; Tue, 13 Jan 2015 06:36:26 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.28.67 with SMTP id z3ls5875lag.2.gmail; Tue, 13 Jan 2015 06:36:25 -0800 (PST) X-Received: by 10.112.167.228 with SMTP id zr4mr42639166lbb.20.1421159785906; Tue, 13 Jan 2015 06:36:25 -0800 (PST) Received: from mail-lb0-f179.google.com (mail-lb0-f179.google.com. [209.85.217.179]) by mx.google.com with ESMTPS id v3si3019080lal.96.2015.01.13.06.36.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 13 Jan 2015 06:36:25 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) client-ip=209.85.217.179; Received: by mail-lb0-f179.google.com with SMTP id z11so2856268lbi.10 for ; Tue, 13 Jan 2015 06:36:25 -0800 (PST) X-Received: by 10.112.84.225 with SMTP id c1mr42347519lbz.22.1421159785708; Tue, 13 Jan 2015 06:36:25 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.9.200 with SMTP id c8csp1401119lbb; Tue, 13 Jan 2015 06:36:24 -0800 (PST) X-Received: by 10.229.240.130 with SMTP id la2mr5231124qcb.9.1421159295195; Tue, 13 Jan 2015 06:28:15 -0800 (PST) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id d5si26957190qcm.48.2015.01.13.06.28.14 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 13 Jan 2015 06:28:15 -0800 (PST) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YB2Qs-0001jH-TR; Tue, 13 Jan 2015 14:26:58 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YB2Qq-0001d3-2L for xen-devel@lists.xenproject.org; Tue, 13 Jan 2015 14:26:56 +0000 Received: from [85.158.137.68] by server-13.bemta-3.messagelabs.com id 16/66-27623-F2B25B45; Tue, 13 Jan 2015 14:26:55 +0000 X-Env-Sender: julien.grall@linaro.org X-Msg-Ref: server-3.tower-31.messagelabs.com!1421159214!19302073!1 X-Originating-IP: [74.125.82.54] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 6.12.5; banners=-,-,- X-VirusChecked: Checked Received: (qmail 13964 invoked from network); 13 Jan 2015 14:26:54 -0000 Received: from mail-wg0-f54.google.com (HELO mail-wg0-f54.google.com) (74.125.82.54) by server-3.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 13 Jan 2015 14:26:54 -0000 Received: by mail-wg0-f54.google.com with SMTP id z12so3228663wgg.13 for ; Tue, 13 Jan 2015 06:26:54 -0800 (PST) X-Received: by 10.194.142.234 with SMTP id rz10mr2093724wjb.118.1421159213959; Tue, 13 Jan 2015 06:26:53 -0800 (PST) Received: from chilopoda.uk.xensource.com. ([185.25.64.249]) by mx.google.com with ESMTPSA id ni15sm14513482wic.18.2015.01.13.06.26.52 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jan 2015 06:26:53 -0800 (PST) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 13 Jan 2015 14:25:29 +0000 Message-Id: <1421159133-31526-21-git-send-email-julien.grall@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1421159133-31526-1-git-send-email-julien.grall@linaro.org> References: <1421159133-31526-1-git-send-email-julien.grall@linaro.org> Cc: Wei Liu , ian.campbell@citrix.com, tim@xen.org, Julien Grall , Ian Jackson , stefano.stabellini@citrix.com, Jan Beulich Subject: [Xen-devel] [PATCH v3 20/24] xen/passthrough: Extend XEN_DOMCTL_assign_device to support DT device X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: julien.grall@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: TODO: Update the commit message A device node is described by a path. It will be used to retrieved the node in the device tree and assign the related device to the domain. Only device protected by an IOMMU can be assigned to a guest. Signed-off-by: Julien Grall Cc: Ian Jackson Cc: Wei Liu Cc: Jan Beulich --- Changes in v2: - Use a different number for XEN_DOMCTL_assign_dt_device --- tools/libxc/include/xenctrl.h | 10 ++++ tools/libxc/xc_domain.c | 95 ++++++++++++++++++++++++++++++++-- xen/drivers/passthrough/device_tree.c | 97 +++++++++++++++++++++++++++++++++-- xen/drivers/passthrough/iommu.c | 7 +++ xen/drivers/passthrough/pci.c | 43 +++++++++++----- xen/include/public/domctl.h | 15 +++++- xen/include/xen/iommu.h | 3 ++ 7 files changed, 249 insertions(+), 21 deletions(-) diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h index d66571f..db45475 100644 --- a/tools/libxc/include/xenctrl.h +++ b/tools/libxc/include/xenctrl.h @@ -2055,6 +2055,16 @@ int xc_deassign_device(xc_interface *xch, uint32_t domid, uint32_t machine_bdf); +int xc_assign_dt_device(xc_interface *xch, + uint32_t domid, + char *path); +int xc_test_assign_dt_device(xc_interface *xch, + uint32_t domid, + char *path); +int xc_deassign_dt_device(xc_interface *xch, + uint32_t domid, + char *path); + int xc_domain_memory_mapping(xc_interface *xch, uint32_t domid, unsigned long first_gfn, diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c index eb066cf..bca3aee 100644 --- a/tools/libxc/xc_domain.c +++ b/tools/libxc/xc_domain.c @@ -1637,7 +1637,8 @@ int xc_assign_device( domctl.cmd = XEN_DOMCTL_assign_device; domctl.domain = domid; - domctl.u.assign_device.machine_sbdf = machine_sbdf; + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_PCI; + domctl.u.assign_device.u.pci.machine_sbdf = machine_sbdf; return do_domctl(xch, &domctl); } @@ -1686,7 +1687,8 @@ int xc_test_assign_device( domctl.cmd = XEN_DOMCTL_test_assign_device; domctl.domain = domid; - domctl.u.assign_device.machine_sbdf = machine_sbdf; + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_PCI; + domctl.u.assign_device.u.pci.machine_sbdf = machine_sbdf; return do_domctl(xch, &domctl); } @@ -1700,11 +1702,96 @@ int xc_deassign_device( domctl.cmd = XEN_DOMCTL_deassign_device; domctl.domain = domid; - domctl.u.assign_device.machine_sbdf = machine_sbdf; - + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_PCI; + domctl.u.assign_device.u.pci.machine_sbdf = machine_sbdf; + return do_domctl(xch, &domctl); } +int xc_assign_dt_device( + xc_interface *xch, + uint32_t domid, + char *path) +{ + int rc; + size_t size = strlen(path); + DECLARE_DOMCTL; + DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN); + + if ( xc_hypercall_bounce_pre(xch, path) ) + return -1; + + domctl.cmd = XEN_DOMCTL_assign_device; + domctl.domain = (domid_t)domid; + + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_DT; + domctl.u.assign_device.u.dt.size = size; + set_xen_guest_handle(domctl.u.assign_device.u.dt.path, path); + + rc = do_domctl(xch, &domctl); + + xc_hypercall_bounce_post(xch, path); + + return rc; +} + +int xc_test_assign_dt_device( + xc_interface *xch, + uint32_t domid, + char *path) +{ + int rc; + size_t size = strlen(path); + DECLARE_DOMCTL; + DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN); + + if ( xc_hypercall_bounce_pre(xch, path) ) + return -1; + + domctl.cmd = XEN_DOMCTL_test_assign_device; + domctl.domain = (domid_t)domid; + + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_DT; + domctl.u.assign_device.u.dt.size = size; + set_xen_guest_handle(domctl.u.assign_device.u.dt.path, path); + + rc = do_domctl(xch, &domctl); + + xc_hypercall_bounce_post(xch, path); + + return rc; +} + +int xc_deassign_dt_device( + xc_interface *xch, + uint32_t domid, + char *path) +{ + int rc; + size_t size = strlen(path); + DECLARE_DOMCTL; + DECLARE_HYPERCALL_BOUNCE(path, size, XC_HYPERCALL_BUFFER_BOUNCE_IN); + + if ( xc_hypercall_bounce_pre(xch, path) ) + return -1; + + domctl.cmd = XEN_DOMCTL_deassign_device; + domctl.domain = (domid_t)domid; + + domctl.u.assign_device.dev = XEN_DOMCTL_DEV_DT; + domctl.u.assign_device.u.dt.size = size; + set_xen_guest_handle(domctl.u.assign_device.u.dt.path, path); + + rc = do_domctl(xch, &domctl); + + xc_hypercall_bounce_post(xch, path); + + return rc; +} + + + + int xc_domain_update_msi_irq( xc_interface *xch, uint32_t domid, diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c index d9b486e..11deb1d 100644 --- a/xen/drivers/passthrough/device_tree.c +++ b/xen/drivers/passthrough/device_tree.c @@ -1,9 +1,6 @@ /* * Code to passthrough a device tree node to a guest * - * TODO: This contains only the necessary code to protected device passed to - * dom0. It will need some updates when device passthrough will is added. - * * Julien Grall * Copyright (c) 2014 Linaro Limited. * @@ -20,6 +17,7 @@ #include #include +#include #include #include @@ -85,6 +83,20 @@ fail: return rc; } +static bool_t iommu_dt_device_is_assigned(const struct dt_device_node *dev) +{ + bool_t assigned = 0; + + if ( !dt_device_is_protected(dev) ) + return 1; + + spin_lock(&dtdevs_lock); + assigned = !list_empty(&dev->domain_list); + spin_unlock(&dtdevs_lock); + + return assigned; +} + int iommu_dt_domain_init(struct domain *d) { struct hvm_iommu *hd = domain_hvm_iommu(d); @@ -111,3 +123,82 @@ int iommu_release_dt_devices(struct domain *d) return 0; } + +int iommu_do_dt_domctl(struct xen_domctl *domctl, struct domain *d, + XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) +{ + int ret; + struct dt_device_node *dev; + + /* TODO: How to deal with XSM? */ + /* TODO: Do we need to check is_dying? Mostly to protect against + * hypercall trying to passthrough a device while we are + * dying. + */ + + switch ( domctl->cmd ) + { + case XEN_DOMCTL_assign_device: + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT ) + break; + + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path, + domctl->u.assign_device.u.dt.size, + &dev); + if ( ret ) + break; + + ret = iommu_assign_dt_device(d, dev); + + if ( ret ) + printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\"" + " to dom%u failed (%d)\n", + dt_node_full_name(dev), d->domain_id, ret); + break; + + case XEN_DOMCTL_deassign_device: + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT ) + break; + + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path, + domctl->u.assign_device.u.dt.size, + &dev); + if ( ret ) + break; + + ret = iommu_deassign_dt_device(d, dev); + + if ( ret ) + printk(XENLOG_G_ERR "XEN_DOMCTL_assign_dt_device: assign \"%s\"" + " to dom%u failed (%d)\n", + dt_node_full_name(dev), d->domain_id, ret); + break; + + case XEN_DOMCTL_test_assign_device: + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_DT ) + break; + + ret = dt_find_node_by_gpath(domctl->u.assign_device.u.dt.path, + domctl->u.assign_device.u.dt.size, + &dev); + if ( ret ) + break; + + if ( iommu_dt_device_is_assigned(dev) ) + { + printk(XENLOG_G_ERR "%s already assigned, or not protected\n", + dt_node_full_name(dev)); + ret = -EINVAL; + } + break; + + default: + ret = -ENOSYS; + break; + } + + return ret; +} diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c index 8915244..02d5ec1 100644 --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -337,6 +337,13 @@ int iommu_do_domctl( ret = iommu_do_pci_domctl(domctl, d, u_domctl); #endif + if ( ret != -ENOSYS ) + return ret; + +#ifdef HAS_DEVICE_TREE + ret = iommu_do_dt_domctl(domctl, d, u_domctl); +#endif + return ret; } diff --git a/xen/drivers/passthrough/pci.c b/xen/drivers/passthrough/pci.c index 9a47a37..ecc0cd1 100644 --- a/xen/drivers/passthrough/pci.c +++ b/xen/drivers/passthrough/pci.c @@ -1497,6 +1497,7 @@ int iommu_do_pci_domctl( u16 seg; u8 bus, devfn; int ret = 0; + uint32_t machine_sbdf; switch ( domctl->cmd ) { @@ -1533,13 +1534,19 @@ int iommu_do_pci_domctl( break; case XEN_DOMCTL_test_assign_device: - ret = xsm_test_assign_device(XSM_HOOK, domctl->u.assign_device.machine_sbdf); + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI ) + break; + + machine_sbdf = domctl->u.assign_device.u.pci.machine_sbdf; + + ret = xsm_test_assign_device(XSM_HOOK, machine_sbdf); if ( ret ) break; - seg = domctl->u.assign_device.machine_sbdf >> 16; - bus = (domctl->u.assign_device.machine_sbdf >> 8) & 0xff; - devfn = domctl->u.assign_device.machine_sbdf & 0xff; + seg = machine_sbdf >> 16; + bus = (machine_sbdf >> 8) & 0xff; + devfn = machine_sbdf & 0xff; if ( device_assigned(seg, bus, devfn) ) { @@ -1551,19 +1558,25 @@ int iommu_do_pci_domctl( break; case XEN_DOMCTL_assign_device: + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI ) + break; + if ( unlikely(d->is_dying) ) { ret = -EINVAL; break; } - ret = xsm_assign_device(XSM_HOOK, d, domctl->u.assign_device.machine_sbdf); + machine_sbdf = domctl->u.assign_device.u.pci.machine_sbdf; + + ret = xsm_assign_device(XSM_HOOK, d, machine_sbdf); if ( ret ) break; - seg = domctl->u.assign_device.machine_sbdf >> 16; - bus = (domctl->u.assign_device.machine_sbdf >> 8) & 0xff; - devfn = domctl->u.assign_device.machine_sbdf & 0xff; + seg = machine_sbdf >> 16; + bus = (machine_sbdf >> 8) & 0xff; + devfn = machine_sbdf & 0xff; ret = device_assigned(seg, bus, devfn) ?: assign_device(d, seg, bus, devfn); @@ -1579,13 +1592,19 @@ int iommu_do_pci_domctl( break; case XEN_DOMCTL_deassign_device: - ret = xsm_deassign_device(XSM_HOOK, d, domctl->u.assign_device.machine_sbdf); + ret = -ENOSYS; + if ( domctl->u.assign_device.dev != XEN_DOMCTL_DEV_PCI ) + break; + + machine_sbdf = domctl->u.assign_device.u.pci.machine_sbdf; + + ret = xsm_deassign_device(XSM_HOOK, d, machine_sbdf); if ( ret ) break; - seg = domctl->u.assign_device.machine_sbdf >> 16; - bus = (domctl->u.assign_device.machine_sbdf >> 8) & 0xff; - devfn = domctl->u.assign_device.machine_sbdf & 0xff; + seg = machine_sbdf >> 16; + bus = (machine_sbdf >> 8) & 0xff; + devfn = machine_sbdf & 0xff; spin_lock(&pcidevs_lock); ret = deassign_device(d, seg, bus, devfn); diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index b742b23..d905ab0 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -475,12 +475,23 @@ typedef struct xen_domctl_sendtrigger xen_domctl_sendtrigger_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_sendtrigger_t); -/* Assign PCI device to HVM guest. Sets up IOMMU structures. */ +/* Assign a device to a guest. Sets up IOMMU structures. */ /* XEN_DOMCTL_assign_device */ /* XEN_DOMCTL_test_assign_device */ /* XEN_DOMCTL_deassign_device */ +#define XEN_DOMCTL_DEV_PCI 0 +#define XEN_DOMCTL_DEV_DT 1 struct xen_domctl_assign_device { - uint32_t machine_sbdf; /* machine PCI ID of assigned device */ + uint32_t dev; /* XEN_DOMCTL_DEV_* */ + union { + struct { + uint32_t machine_sbdf; /* machine PCI ID of assigned device */ + } pci; + struct { + uint32_t size; /* Length of the path */ + XEN_GUEST_HANDLE_64(char) path; /* path to the device tree node */ + } dt; + } u; }; typedef struct xen_domctl_assign_device xen_domctl_assign_device_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_assign_device_t); diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index d03df14..d261277 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -119,6 +119,9 @@ int iommu_deassign_dt_device(struct domain *d, struct dt_device_node *dev); int iommu_dt_domain_init(struct domain *d); int iommu_release_dt_devices(struct domain *d); +int iommu_do_dt_domctl(struct xen_domctl *, struct domain *, + XEN_GUEST_HANDLE_PARAM(xen_domctl_t)); + #endif /* HAS_DEVICE_TREE */ struct page_info;