From patchwork Mon Jul 7 12:27:21 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Auger Eric X-Patchwork-Id: 33154 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pa0-f69.google.com (mail-pa0-f69.google.com [209.85.220.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 98AB420C88 for ; Mon, 7 Jul 2014 12:31:11 +0000 (UTC) Received: by mail-pa0-f69.google.com with SMTP id kq14sf32260952pab.0 for ; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=9yInX6SNON+AeroRlrpGCTQxS4qmBuB5xvlB9KtD26k=; b=PiZHrb2ey+0z3ojTAOm6s+7GK+BwJSJj4fqTIcmjddk970cHJDwU4L38Qr4+RL5RT0 lZ6tyfYOkZGYjEvMgTUyT5RQ+HaqBPPlVB7wvu1yS1YT0+j3eA6guZNbf2gbjSGbhERr j72ayTsAoKs5Lg/WWNX05a3vb7kN30NdLRDjOn3g3QX7+haR82K3p+fyqjmoYEhWsHzR da+0fIjg4wAU/al2wNR3W2+6QbGpn1F5KQfhD1JfIxyHUY5qLYtjDD7HD49BLlISBZTC 5ho5xD0YLInu2qulwLW9zGTnAueH1fSPFBIGng3n4luBU+/mIiCgIwHO4bWisdCu13XY uI7A== X-Gm-Message-State: ALoCoQmKdtH69xjvhHRr3ptbUGho/KhIoXf3QMUizH8F3V3p+IK7op2ZFFr5nFU9wxy/9FJW9WiA X-Received: by 10.66.230.226 with SMTP id tb2mr12672742pac.41.1404736270705; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.26.241 with SMTP id 104ls1598275qgv.21.gmail; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) X-Received: by 10.52.165.98 with SMTP id yx2mr93122vdb.80.1404736270579; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) Received: from mail-ve0-f172.google.com (mail-ve0-f172.google.com [209.85.128.172]) by mx.google.com with ESMTPS id tb9si12633045vcb.32.2014.07.07.05.31.10 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 05:31:10 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.172 as permitted sender) client-ip=209.85.128.172; Received: by mail-ve0-f172.google.com with SMTP id jz11so3972272veb.31 for ; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) X-Received: by 10.220.203.134 with SMTP id fi6mr27701119vcb.18.1404736270511; Mon, 07 Jul 2014 05:31:10 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp662473vcb; Mon, 7 Jul 2014 05:31:10 -0700 (PDT) X-Received: by 10.180.93.9 with SMTP id cq9mr35872460wib.27.1404736269622; Mon, 07 Jul 2014 05:31:09 -0700 (PDT) Received: from mail-we0-f180.google.com (mail-we0-f180.google.com [74.125.82.180]) by mx.google.com with ESMTPS id gg4si49736670wjb.47.2014.07.07.05.31.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 05:31:09 -0700 (PDT) Received-SPF: pass (google.com: domain of eric.auger@linaro.org designates 74.125.82.180 as permitted sender) client-ip=74.125.82.180; Received: by mail-we0-f180.google.com with SMTP id x48so4321875wes.25 for ; Mon, 07 Jul 2014 05:31:09 -0700 (PDT) X-Received: by 10.180.14.33 with SMTP id m1mr52160328wic.50.1404736269053; Mon, 07 Jul 2014 05:31:09 -0700 (PDT) Received: from midway01-04-00.lavalab ([88.98.47.97]) by mx.google.com with ESMTPSA id ec8sm95805182wic.10.2014.07.07.05.31.08 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 05:31:08 -0700 (PDT) From: Eric Auger To: eric.auger@st.com, christoffer.dall@linaro.org, qemu-devel@nongnu.org, kim.phillips@freescale.com, a.rigo@virtualopensystems.com Cc: eric.auger@linaro.org, will.deacon@arm.com, kvmarm@lists.cs.columbia.edu, alex.williamson@redhat.com, Bharat.Bhushan@freescale.com, agraf@suse.de, peter.maydell@linaro.org, stuart.yoder@freescale.com, a.motakis@virtualopensystems.com, patches@linaro.org Subject: [RFC v4 11/13] hw/vfio/platform: Add irqfd support Date: Mon, 7 Jul 2014 13:27:21 +0100 Message-Id: <1404736043-22900-12-git-send-email-eric.auger@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1404736043-22900-1-git-send-email-eric.auger@linaro.org> References: <1404736043-22900-1-git-send-email-eric.auger@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: eric.auger@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.128.172 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , This patch aims at optimizing IRQ handling using irqfd framework. Instead of handling the eventfds on user-side they are handled on kernel side using - the KVM irqfd framework, - the VFIO driver virqfd framework. the virtual IRQ completion is trapped at interrupt controller instead of on guest 1st access to any region after IRQ hit. This removes the need for fast/slow path swap. Overall this brings significant performance improvements. It depends on host kernel KVM irqfd/GSI routing capability. Signed-off-by: Alvise Rigo Signed-off-by: Eric Auger --- v3 -> v4: [Alvise Rigo] Use of VFIO Platform driver v6 unmask/virqfd feature and removal of resamplefd handler. Physical IRQ unmasking is now done in VFIO driver. v3: [Eric Auger] initial support with resamplefd handled on QEMU side since the unmask was not supported on VFIO platform driver v5. --- hw/vfio/platform.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index a5fc22b..fb0f7c9 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -381,6 +381,101 @@ static int vfio_populate_interrupts(VFIODevice *vbasedev) return 0; } +static void vfio_enable_intp_kvm(VFIOINTp *intp) +{ +#ifdef CONFIG_KVM + struct kvm_irqfd irqfd = { + .fd = event_notifier_get_fd(&intp->interrupt), + .gsi = intp->virtualID, + .flags = KVM_IRQFD_FLAG_RESAMPLE, + }; + + struct vfio_irq_set *irq_set; + int ret, argsz; + int32_t *pfd; + VFIODevice *vbasedev = &intp->vdev->vbasedev; + + if (!kvm_irqfds_enabled() || + !kvm_check_extension(kvm_state, KVM_CAP_IRQFD_RESAMPLE)) { + return; + } + + /* Get to a known interrupt state */ + qemu_set_fd_handler(irqfd.fd, NULL, NULL, NULL); + vfio_mask_irqindex(vbasedev, intp->pin); + intp->state = VFIO_IRQ_INACTIVE; + qemu_set_irq(intp->qemuirq, 0); + + /* Get an eventfd for resample/unmask */ + if (event_notifier_init(&intp->unmask, 0)) { + error_report("vfio: Error: event_notifier_init failed eoi"); + goto fail; + } + + /* KVM triggers it, VFIO listens for it */ + irqfd.resamplefd = event_notifier_get_fd(&intp->unmask); + + if (kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd)) { + error_report("vfio: Error: Failed to setup resample irqfd: %m"); + goto fail_irqfd; + } + + argsz = sizeof(*irq_set) + sizeof(*pfd); + + irq_set = g_malloc0(argsz); + irq_set->argsz = argsz; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_UNMASK; + irq_set->index = intp->pin; + irq_set->start = 0; + irq_set->count = 1; + pfd = (int32_t *)&irq_set->data; + + *pfd = irqfd.resamplefd; + + ret = ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irq_set); + g_free(irq_set); + if (ret) { + error_report("vfio: Error: Failed to setup INTx unmask fd: %m"); + goto fail_vfio; + } + + /* Let'em rip */ + vfio_unmask_irqindex(vbasedev, intp->pin); + + intp->kvm_accel = true; + + DPRINTF("%s irqfd pin=%d to virtID = %d fd=%d, resamplefd=%d)\n", + __func__, intp->pin, intp->virtualID, + irqfd.fd, irqfd.resamplefd); + + return; + +fail_vfio: + irqfd.flags = KVM_IRQFD_FLAG_DEASSIGN; + kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd); +fail_irqfd: + event_notifier_cleanup(&intp->unmask); +fail: + qemu_set_fd_handler(irqfd.fd, vfio_intp_interrupt, NULL, intp); + vfio_unmask_irqindex(vbasedev, intp->pin); +#endif +} + +void vfio_setup_irqfd(SysBusDevice *s, int index, int virq) +{ + VFIOPlatformDevice *vdev = container_of(s, VFIOPlatformDevice, sbdev); + VFIOINTp *intp; + + QLIST_FOREACH(intp, &vdev->intp_list, next) { + if (intp->pin == index) { + intp->virtualID = virq; + DPRINTF("enable irqfd for irq index %d (virtual IRQ %d)\n", + index, virq); + vfio_enable_intp_kvm(intp); + } + } +} + static VFIODeviceOps vfio_platform_ops = { .vfio_compute_needs_reset = vfio_platform_compute_needs_reset, .vfio_hot_reset_multi = vfio_platform_hot_reset_multi,