From patchwork Sat Aug 9 14:25:48 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Auger Eric X-Patchwork-Id: 35191 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qc0-f200.google.com (mail-qc0-f200.google.com [209.85.216.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id ACCCE2136C for ; Sat, 9 Aug 2014 14:26:26 +0000 (UTC) Received: by mail-qc0-f200.google.com with SMTP id w7sf1444512qcr.11 for ; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=D+LFlfQm34EwVIgofOmupH3lkjhKCBwOZmamiPqi7RA=; b=Z4ZjX4SQ2Rp8io2PL8DHW8j/5MsUhPaNvCzjs19qT+rPcIPytKM/vAXb5nMvYvg3+/ HFYwLLv4VR1mI3pwYEVgxXo6aeLZL82QKbPlZEus6JSx7LmKbbnr80YzoQL7xfsCHf+n wmsAkY4kiIIkqy3PGxTKRWsm3fcp9Tk8QyQx7XxqMi4w+jnACA32H1ax7s6F5pAj4OKq riy8tDqEbOHKsDQvVv288YL9k0mCbQFwIyxjNuzx9dOEJ2yO7uSEOErA7mx6ee59rO5t AFD6dDVMCEM5VJ7trxSFsfymoWSiVSyrzk1PhftfQTpqF9nIvET9pMREj3CqWImbQmYJ 6igQ== X-Gm-Message-State: ALoCoQmx4Y1u/VyJ9drtLVZXnb6l0+YxL+wqqAPWPspll4ZZXWLECrYJPEGZAdSqgOxX8Yc/h9Zx X-Received: by 10.236.101.148 with SMTP id b20mr673618yhg.46.1407594386525; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.43.116 with SMTP id d107ls693271qga.67.gmail; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) X-Received: by 10.52.9.35 with SMTP id w3mr11329032vda.12.1407594386438; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com [209.85.220.181]) by mx.google.com with ESMTPS id mg6si3902276vec.35.2014.08.09.07.26.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 09 Aug 2014 07:26:26 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) client-ip=209.85.220.181; Received: by mail-vc0-f181.google.com with SMTP id lf12so9572672vcb.40 for ; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) X-Received: by 10.52.83.227 with SMTP id t3mr11410240vdy.20.1407594386335; Sat, 09 Aug 2014 07:26:26 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp34398vcb; Sat, 9 Aug 2014 07:26:25 -0700 (PDT) X-Received: by 10.194.206.67 with SMTP id lm3mr40055211wjc.70.1407594385093; Sat, 09 Aug 2014 07:26:25 -0700 (PDT) Received: from mail-we0-f174.google.com (mail-we0-f174.google.com [74.125.82.174]) by mx.google.com with ESMTPS id ga3si7886012wib.18.2014.08.09.07.26.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 09 Aug 2014 07:26:25 -0700 (PDT) Received-SPF: pass (google.com: domain of eric.auger@linaro.org designates 74.125.82.174 as permitted sender) client-ip=74.125.82.174; Received: by mail-we0-f174.google.com with SMTP id x48so6803013wes.33 for ; Sat, 09 Aug 2014 07:26:24 -0700 (PDT) X-Received: by 10.180.210.172 with SMTP id mv12mr4481816wic.45.1407594384491; Sat, 09 Aug 2014 07:26:24 -0700 (PDT) Received: from midway01-04-00.lavalab ([88.98.47.97]) by mx.google.com with ESMTPSA id f3sm19591736wiz.0.2014.08.09.07.26.23 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 09 Aug 2014 07:26:24 -0700 (PDT) From: Eric Auger To: eric.auger@st.com, christoffer.dall@linaro.org, qemu-devel@nongnu.org, kim.phillips@freescale.com, a.rigo@virtualopensystems.com Cc: eric.auger@linaro.org, will.deacon@arm.com, kvmarm@lists.cs.columbia.edu, alex.williamson@redhat.com, Bharat.Bhushan@freescale.com, agraf@suse.de, peter.maydell@linaro.org, stuart.yoder@freescale.com, a.motakis@virtualopensystems.com, patches@linaro.org, joel.schopp@amd.com Subject: [PATCH v5 09/10] hw/vfio/platform: Add irqfd support Date: Sat, 9 Aug 2014 15:25:48 +0100 Message-Id: <1407594349-9291-10-git-send-email-eric.auger@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1407594349-9291-1-git-send-email-eric.auger@linaro.org> References: <1407594349-9291-1-git-send-email-eric.auger@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: eric.auger@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.181 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , This patch aims at optimizing IRQ handling using irqfd framework. Instead of handling the eventfds on user-side they are handled on kernel side using - the KVM irqfd framework, - the VFIO driver virqfd framework. the virtual IRQ completion is trapped at interrupt controller instead of on guest 1st access to any region after IRQ hit. This removes the need for fast/slow path swap. Overall this brings significant performance improvements. It depends on host kernel KVM irqfd/GSI routing capability. Signed-off-by: Alvise Rigo Signed-off-by: Eric Auger --- v4 -> v5: - addition of sysemu/kvm.h header v3 -> v4: [Alvise Rigo] Use of VFIO Platform driver v6 unmask/virqfd feature and removal of resamplefd handler. Physical IRQ unmasking is now done in VFIO driver. v3: [Eric Auger] initial support with resamplefd handled on QEMU side since the unmask was not supported on VFIO platform driver v5. --- hw/vfio/platform.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index f1a1b55..e5c652c 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -23,6 +23,7 @@ #include "exec/memory.h" #include "qemu/queue.h" #include "hw/sysbus.h" +#include "sysemu/kvm.h" extern const MemoryRegionOps vfio_region_ops; extern const MemoryListener vfio_memory_listener; @@ -367,6 +368,99 @@ static int vfio_populate_interrupts(VFIODevice *vbasedev) return 0; } +static void vfio_enable_intp_kvm(VFIOINTp *intp) +{ +#ifdef CONFIG_KVM + struct kvm_irqfd irqfd = { + .fd = event_notifier_get_fd(&intp->interrupt), + .gsi = intp->virtualID, + .flags = KVM_IRQFD_FLAG_RESAMPLE, + }; + + struct vfio_irq_set *irq_set; + int ret, argsz; + int32_t *pfd; + VFIODevice *vbasedev = &intp->vdev->vbasedev; + + if (!kvm_irqfds_enabled() || + !kvm_check_extension(kvm_state, KVM_CAP_IRQFD_RESAMPLE)) { + return; + } + + /* Get to a known interrupt state */ + qemu_set_fd_handler(irqfd.fd, NULL, NULL, NULL); + vfio_mask_irqindex(vbasedev, intp->pin); + intp->state = VFIO_IRQ_INACTIVE; + qemu_set_irq(intp->qemuirq, 0); + + /* Get an eventfd for resample/unmask */ + if (event_notifier_init(&intp->unmask, 0)) { + error_report("vfio: Error: event_notifier_init failed eoi"); + goto fail; + } + + /* KVM triggers it, VFIO listens for it */ + irqfd.resamplefd = event_notifier_get_fd(&intp->unmask); + + if (kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd)) { + error_report("vfio: Error: Failed to setup resample irqfd: %m"); + goto fail_irqfd; + } + + argsz = sizeof(*irq_set) + sizeof(*pfd); + + irq_set = g_malloc0(argsz); + irq_set->argsz = argsz; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_UNMASK; + irq_set->index = intp->pin; + irq_set->start = 0; + irq_set->count = 1; + pfd = (int32_t *)&irq_set->data; + + *pfd = irqfd.resamplefd; + + ret = ioctl(vbasedev->fd, VFIO_DEVICE_SET_IRQS, irq_set); + g_free(irq_set); + if (ret) { + error_report("vfio: Error: Failed to setup IRQ unmask fd: %m"); + goto fail_vfio; + } + + vfio_unmask_irqindex(vbasedev, intp->pin); + + intp->kvm_accel = true; + + DPRINTF("%s irqfd pin=%d to virtID = %d fd=%d, resamplefd=%d)\n", + __func__, intp->pin, intp->virtualID, + irqfd.fd, irqfd.resamplefd); + return; + +fail_vfio: + irqfd.flags = KVM_IRQFD_FLAG_DEASSIGN; + kvm_vm_ioctl(kvm_state, KVM_IRQFD, &irqfd); +fail_irqfd: + event_notifier_cleanup(&intp->unmask); +fail: + qemu_set_fd_handler(irqfd.fd, vfio_intp_interrupt, NULL, intp); + vfio_unmask_irqindex(vbasedev, intp->pin); +#endif +} + +void vfio_setup_irqfd(SysBusDevice *s, int index, int virq) +{ + VFIOPlatformDevice *vdev = container_of(s, VFIOPlatformDevice, sbdev); + VFIOINTp *intp; + + QLIST_FOREACH(intp, &vdev->intp_list, next) { + if (intp->pin == index) { + intp->virtualID = virq; + DPRINTF("enable irqfd for irq index %d (virtual IRQ %d)\n", + index, virq); + vfio_enable_intp_kvm(intp); + } + } +} + static VFIODeviceOps vfio_platform_ops = { .vfio_compute_needs_reset = vfio_platform_compute_needs_reset, .vfio_hot_reset_multi = vfio_platform_hot_reset_multi,