From patchwork Fri Feb 9 14:38:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Przywara X-Patchwork-Id: 127847 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp679921ljc; Fri, 9 Feb 2018 06:42:17 -0800 (PST) X-Google-Smtp-Source: AH8x225SrZecYmevdou3fHPCRl76WXpQikBloWzQdku8IeiqPYdwN+lKwfxZ6/50iZx+gkrB7M8G X-Received: by 10.107.192.2 with SMTP id q2mr3474927iof.112.1518187337671; Fri, 09 Feb 2018 06:42:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518187337; cv=none; d=google.com; s=arc-20160816; b=NjdXc1ywbDe0MYC85wCZbVFpCVQ+PbmI0BY1mZZk+vzgnMuUXF8WvGiV3DzRyWk4k4 FweEKT0rcq6LWvX1gRIG4oABw0onrgUoDoOX/0a/Uo1Lf+4aYnciKTPOav671gc0VuA1 LUnmf0JV+DY4OP9VHzQdn5urxzdmVskcZQLHp/V3RRswy/RoRygDpfaQyKCBLhOJ6Es3 N1Cfq0UU6F6T8ouUoFjTSZ9JXDcfsgsAegkkC2OG15WJBPSk5Watl5DpLoGletk7dPgr V77hWA8x3NrIsdO86PYs0zTjzvntXXl6ZVoXmpxX2/8jAeAMgwgY8mI9xbyUAoCKP5Lk 7KKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:message-id:date:to:from:dkim-signature :arc-authentication-results; bh=9QbUSMIpi2qVSEcKkwBWYSdrxT2dtEVTnnXU5JzNBZE=; b=W77b9XjxKoqnVY7hSP+cT1CXjV8TfN9uWkQ3UU246KFlUroRI39/8foG4F/OFWYeem DQd2VV2PTksR1tLzhpKJCJ7hjXAE9QHJAzjfQcP/iPT9Ko7rba6Mje2eYnm/oZoxIFiN YEj+0STtSEyBd7GloII9Yb6LM4JdFmMGvibkG5CkQCXXW7IKxM2Q0bPdaj5KnaLqYUTV MMKBM+AcU0Gy66VdnsW0pSgyoYnVy0wKfp397aeZ4eU8fq81/keyPIlz74+hy52+FGiF CpREI/razsoeG/iGEjeOfEcYzXHOf/LXRiHh36bSiprg2uvaXj7FJdzyO6QBNRLfZ1IV z4kg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=Ni3+FgU6; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id g1si1627346itd.56.2018.02.09.06.42.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Feb 2018 06:42:17 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=Ni3+FgU6; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ek9q9-0007xf-SH; Fri, 09 Feb 2018 14:39:49 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ek9q8-0007xD-CB for xen-devel@lists.xenproject.org; Fri, 09 Feb 2018 14:39:48 +0000 X-Inumbo-ID: 01abed21-0da7-11e8-ba59-bc764e045a96 Received: from mail-wr0-x22b.google.com (unknown [2a00:1450:400c:c0c::22b]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 01abed21-0da7-11e8-ba59-bc764e045a96; Fri, 09 Feb 2018 15:39:18 +0100 (CET) Received: by mail-wr0-x22b.google.com with SMTP id h9so8470185wre.12 for ; Fri, 09 Feb 2018 06:39:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id; bh=nXSputoyC6PO+IU27KbSWqTEvJNUiIbHi02fBhAx8IU=; b=Ni3+FgU6ZO0HwNfIhQ3xJ1pdWAWafRXEl+f1aaBGEyU+oKhcOdbCavCZzs2t3yr84o OPoGgnezaKWd9I8yojXxMjCzuOzdgAsyIb0V9hnCKYWcVUxURiMJDXyHCNp5htBAUID7 61zPQwZufd2g7B/GcVjBCOF15Ws1dBKpFE4sQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=nXSputoyC6PO+IU27KbSWqTEvJNUiIbHi02fBhAx8IU=; b=NP+lgpdOZJoao7UQrjufDVYTA851VhedBQRtqIh4yVRPROTQACYR7v/bKFNuLvGlIE b/C8uPBIPPznSaQGOrhgstH/CB+NMlqcEdvFrZWvx7aSgS/f4dKx2+Vo8aY8ErwEakE7 FZT87IDCXTQHr/Ba/CT3rb6PXfAIswalqGsKkNjKFmNoSepnP5EfCP3x5Gw9sXt2fV3t CZ51PDnKCcsdqW+eoHOxtqtaFJsU2rSEGfuXMDDuaG1PF60E+59XA4W7AYAIddpBmylD Wzi7lSpt0Qj92ggvh3m9GSURD8eYlA6ABhzVMtxCEA8Fo5/lAMBicMLviYWW1amkfwgg zb5g== X-Gm-Message-State: APf1xPBY1f+XmtpvuELQhYaqg2q7VonXzKKil07r5pLBXfJkV2DGD4dZ iQTHz/QtoWCnnfPdqenibedAHQ== X-Received: by 10.223.186.198 with SMTP id w6mr2504733wrg.172.1518187184990; Fri, 09 Feb 2018 06:39:44 -0800 (PST) Received: from e104803-lin.lan (mail.andrep.de. [217.160.17.100]) by smtp.gmail.com with ESMTPSA id b35sm2552229wra.13.2018.02.09.06.39.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 09 Feb 2018 06:39:44 -0800 (PST) From: Andre Przywara To: Stefano Stabellini , Julien Grall , xen-devel@lists.xenproject.org Date: Fri, 9 Feb 2018 14:38:48 +0000 Message-Id: <20180209143937.28866-1-andre.przywara@linaro.org> X-Mailer: git-send-email 2.14.1 Subject: [Xen-devel] [RFC PATCH 00/49] New VGIC(-v2) implementation X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" tl;dr: More preparatory patches from patch 07, actual new VGIC starting at patch 20. ============= During development of the Dom0 ITS MSI support last year we realised that the existing GIC interrupt controller emulation has some shortcomings. After some tries to fix those in the existing code, it was agreed upon that the problems are fundamental and a new implementation based on the "new VGIC" in KVM is the best choice. This is the first drop of this new VGIC implementation. It lives in the xen/arch/arm/vgic/ directory and is written to be a compile time option, so people can choose whether to use the new VGIC or the existing implementation. This is just for a transitional period, the old VGIC is expected to be removed after confidence in the new implementation has grown. This series starts with some GICv3 redistributor cleanup, which I posted before. I need to incorporate the comments from the list, but for now I left those patches as it from the previous post. Starting with patch 07 there are some more cleanups and preparations for the existing VGIC/GIC code. A big part of those patches are preparations to properly support level triggered interrupts. This is one of the biggest problems in the existing VGIC, which only correctly emulates edge triggered IRQs. This affects both arch code and some users like the timer and the event channel. Starting with patch 20 we plumb in the new VGIC then. This is done in a new directory, with all the files actually not wired into the build system until the very last patch. The idea is to split the series into reviewable chunks without resorting to nasty hacks to keep bisectability. The code was forked from Linux' virt/kvm/arm/vgic/, as of 4.14-rc7, plus some recent changes to improve support for level triggered and hardware mapped interrupts, which is what we use heavily in Dom0. The code was heavily adapted to fit into Xen, starting with using the Xen coding style and using Xen structure and variable names (struct domain instead of struct kvm, for instance). Where interfacing functions were similar enough, they were changed over to the existing Xen name and prototypes (for instance kvm_vgic_create() was renamed to domain_vgic_register()). As far as possible the code layout and split was re-used from KVM, so patches in Linux should be relatively easy to port into Xen. Due to the mentioned changes this can not be done easily in an automatic way, but it should be not too complicated to extract the gist of the patch and re-apply this to our code base. The actual VGIC code splits into several parts: - The core is the struct vgic_irq, which holds every information about a virtual IRQ, including a per-IRQ lock. Also there is on (ordered) per-VCPU list (ap_list), which links the interrupts to be considered by a VCPU. There are functions to deal with queuing and removing IRQs from those lists safely, obeying the locking order. (patches 20-23) - There are functions to push vIRQs on a VCPU list to the list registers, and handle their state changes. (patches 24-26) - The distributor MMIO emulation is using separate functions per register, also having read and write split. (patches 27-37) - There are functions to deal with Xen specialities. (patches 38-43) - The data structures and the wiring of the emulation into the hypervisor and the guests are done in vgic-init.c. (patches 44-47) - Finally patch 49 enables the build of the new VGIC. This requires to increase the size limit for struct vcpu in patch 48. Please consider this series the first post that it is. It was briefly tested on a Juno and a Midway (with and without CONFIG_NEW_VGIC). It does not cover GICv3 yet, though supporting this should be relatively straightforward. In fact I have most of the code here already, but I wanted to wait for initial feedback before applying similar transformations to the *-v3.c files. Also this does not include ITS support, though the code itself is more ready for that than the old VGIC ever was. I did some semi-automatic translation of coding style, so I am sure there are quite some leftovers that slipped through, including identifiers with kvm_ in their name ;-) I would appreciate if somehow could have a look at those patches, patch 20 and following are surely the most interesting ones to have a look at. Cheers, Andre Andre Przywara (49): tools: ARM: vGICv3: avoid inserting optional DT properties ARM: vGICv3: drop GUEST_GICV3_RDIST_REGIONS symbol ARM: GICv3: use hardware GICv3 redistributor regions for Dom0 ARM: GICv3: simplify GICv3 redistributor stride handling ARM: vGICv3: always use architected redist stride ARM: vGICv3: remove rdist_stride from VGIC structure ARM: VGIC: move gic_remove_from_lr_pending() prototype ARM: VGIC: move max_vcpus VGIC limit to struct arch_domain ARM: VGIC: change to level-IRQ compatible IRQ injection interface ARM: VGIC: carve out struct vgic_cpu and struct vgic_dist ARM: VGIC: reorder prototypes in vgic.h ARM: VGIC: introduce gic_get_nr_lrs() ARM: VGIC: Add hypervisor base address to vgic_v2_setup_hw() ARM: VGIC: extend GIC CPU interface definitions ARM: GIC: Allow tweaking the active state of an IRQ ARM: GIC: allow reading pending state of a hardware IRQ ARM: timer: Handle level triggered IRQs correctly ARM: evtchn: Handle level triggered IRQs correctly ARM: vPL011: Use the VGIC's level triggered IRQs handling if available ARM: new VGIC: Add data structure definitions ARM: new VGIC: Add acccessor to new struct vgic_irq instance ARM: new VGIC: Implement virtual IRQ injection ARM: new VGIC: Add IRQ sorting ARM: new VGIC: Add IRQ sync/flush framework ARM: new VGIC: Add GICv2 world switch backend ARM: new VGIC: Implement vgic_vcpu_pending_irq ARM: new VGIC: Add MMIO handling framework ARM: new VGIC: Add GICv2 MMIO handling framework ARM: new VGIC: Add CTLR, TYPER and IIDR handlers ARM: new VGIC: Add ENABLE registers handlers ARM: new VGIC: Add PENDING registers handlers ARM: new VGIC: Add ACTIVE registers handlers ARM: new VGIC: Add PRIORITY registers handlers ARM: new VGIC: Add CONFIG registers handlers ARM: new VGIC: Add TARGET registers handlers ARM: new VGIC: Add SGIR register handler ARM: new VGIC: Add SGIPENDR register handlers ARM: new VGIC: handle hardware mapped IRQs ARM: new VGIC: Add event channel IRQ handling ARM: new VGIC: Handle virtual IRQ allocation/reservation ARM: new VGIC: dump virtual IRQ info ARM: new VGIC: provide system register emulation stub ARM: new VGIC: Add preliminary stub implementations ARM: new VGIC: vgic-init: register VGIC ARM: new VGIC: vgic-init: implement vgic_init ARM: new VGIC: vgic-init: implement map_resources ARM: new VGIC: Add vgic_v2_enable ARM: allocate two pages for struct vcpu ARM: VGIC: wire new VGIC(-v2) files into Xen build system tools/libxl/libxl_arm.c | 8 - xen/arch/arm/Kconfig | 6 +- xen/arch/arm/Makefile | 10 +- xen/arch/arm/domain.c | 25 +- xen/arch/arm/gic-v2.c | 20 +- xen/arch/arm/gic-v3-lpi.c | 2 +- xen/arch/arm/gic-v3.c | 72 ++-- xen/arch/arm/gic-vgic.c | 10 +- xen/arch/arm/gic.c | 15 + xen/arch/arm/irq.c | 2 +- xen/arch/arm/time.c | 36 +- xen/arch/arm/traps.c | 2 + xen/arch/arm/vgic-v2.c | 3 +- xen/arch/arm/vgic-v3.c | 45 +- xen/arch/arm/vgic.c | 46 +- xen/arch/arm/vgic/vgic-init.c | 259 ++++++++++++ xen/arch/arm/vgic/vgic-mmio-v2.c | 297 +++++++++++++ xen/arch/arm/vgic/vgic-mmio.c | 589 ++++++++++++++++++++++++++ xen/arch/arm/vgic/vgic-mmio.h | 194 +++++++++ xen/arch/arm/vgic/vgic-v2.c | 339 +++++++++++++++ xen/arch/arm/vgic/vgic.c | 871 ++++++++++++++++++++++++++++++++++++++ xen/arch/arm/vgic/vgic.h | 69 +++ xen/arch/arm/vpl011.c | 6 +- xen/arch/arm/vtimer.c | 4 +- xen/common/Makefile | 1 + xen/common/list_sort.c | 170 ++++++++ xen/include/asm-arm/arm_vgic.h | 269 ++++++++++++ xen/include/asm-arm/domain.h | 92 +--- xen/include/asm-arm/event.h | 1 + xen/include/asm-arm/gic.h | 31 +- xen/include/asm-arm/gic_v3_defs.h | 5 + xen/include/asm-arm/vgic.h | 170 ++++++-- xen/include/public/arch-arm.h | 3 - xen/include/xen/list_sort.h | 11 + xen/include/xen/timer.h | 2 + 35 files changed, 3456 insertions(+), 229 deletions(-) create mode 100644 xen/arch/arm/vgic/vgic-init.c create mode 100644 xen/arch/arm/vgic/vgic-mmio-v2.c create mode 100644 xen/arch/arm/vgic/vgic-mmio.c create mode 100644 xen/arch/arm/vgic/vgic-mmio.h create mode 100644 xen/arch/arm/vgic/vgic-v2.c create mode 100644 xen/arch/arm/vgic/vgic.c create mode 100644 xen/arch/arm/vgic/vgic.h create mode 100644 xen/common/list_sort.c create mode 100644 xen/include/asm-arm/arm_vgic.h create mode 100644 xen/include/xen/list_sort.h