From patchwork Mon Mar 5 16:03:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andre Przywara X-Patchwork-Id: 130658 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp2853600lja; Mon, 5 Mar 2018 08:07:01 -0800 (PST) X-Google-Smtp-Source: AG47ELuQ5SxFRDVXLBqAwmM53J+AUnCG42t+TpuQL5sLlBboo9lxgyvDfSZY/f+520urLfGDF1ct X-Received: by 10.36.39.86 with SMTP id g83mr15142264ita.28.1520266021735; Mon, 05 Mar 2018 08:07:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1520266021; cv=none; d=google.com; s=arc-20160816; b=vrpCUD87xHThtB8qowmRcPWOlAhY0//+Ckqu2I3WnGN33bYqdFpWoovXtrNw73JiaE b0YQp9C3vW/vs5eunmJJ5he10gipMAJD0ajfa2FDimxCxACBKK+yrjcsBuYluSG8LiY5 4kPRwJFM1+rTBz6k6D4PTLNqJGs34qPrNPMqP7AN6e6FojDOXlc0dHGiV8qAfv5RK272 OjaMaQhXAEzffSeajJl03PtKILB6QQiQ+GCAgO81ZiK/OmeN7KX2toVum3yzn/1t/ArI PQzc1PcG50z3TM14TR6A8JPDY4an3NIAi9V94w+KgHLs01p25hnwPJQnH1LJeogIiF0K vOkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=cv2Xb1VRk3StI5YXS57eqQnQYsV7vBhcPUbzsWK5eUM=; b=mv084XN4or1h0MRMmi6RJPB8snkKYolAE4UMvehlT5NUsyXiSY6K8RYhfcq+jLw0/S n43aT8X7XF/N1RRCi371dyxPN0By2xBO+U6fFdVeFD0awf/Hy5tMZ70oKuPRHtCmsMm4 Nu8q6VZekF7eTwSs1LGjdHpwWkZgI1um7fqz9iAScBR1jCgm8QGocrllL8+e9epuxtX4 wSe3SE5t24Gsnm/IDRYVlyqKqSNgzIkTI2ctbzWhtOuXH40ADIa1IVksmQSFxGJOOvtr GDlLLQTJykrvh5CtSiINnrBuZbypxneyIA9qgBX50A9rV1Nt06w5HSjeqi86eAHSU2Nw RQcQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kMqEfKDE; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id f203si5869819itb.0.2018.03.05.08.07.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Mar 2018 08:07:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kMqEfKDE; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1essbf-0007xK-4k; Mon, 05 Mar 2018 16:04:55 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1essbe-0007wc-UP for xen-devel@lists.xenproject.org; Mon, 05 Mar 2018 16:04:54 +0000 X-Inumbo-ID: c9ab76fb-208e-11e8-ba59-bc764e045a96 Received: from mail-wr0-x242.google.com (unknown [2a00:1450:400c:c0c::242]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id c9ab76fb-208e-11e8-ba59-bc764e045a96; Mon, 05 Mar 2018 17:03:48 +0100 (CET) Received: by mail-wr0-x242.google.com with SMTP id v65so17818036wrc.11 for ; Mon, 05 Mar 2018 08:04:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pRKoe//6FsKFOPzCykYK1wkQdgkYCobMexw0flyLj3A=; b=kMqEfKDE30Ukwl0vQa8BNQho1sRdjFFnXdTM5yrcP6BhRsqECfMwqTzBMYHaGr8LOs FeCdLnGTkQD4uxg09O/vhLBlSjD8sZqGWvJHBzAUon1R9AYDatMU//1LfP55QZ+M1ZPN yjhlkhRYpqNt1q1L40oqBnUcvOOrGhJYuKA6g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pRKoe//6FsKFOPzCykYK1wkQdgkYCobMexw0flyLj3A=; b=AQPFmgCm+Ob9ohtevKjocJj7dnlCCKR1fyHCELdDwx9iOZOCvtrMaC9qJpLHk5LbT0 Bw8MA+M2mwzRCVNkBED4m/JIAqbegXEWvekrHzPMBeEsUv/w/BG24m33a/mfo43QjO/d 8gC/iZXXgHfAQma+u+bpQCkmtE6e3MIX0jS0ahoWVIW7cf089KAga4LqdKifiRMMHG+J Z7GoCt9q95aCo0B5ST5iyH87EbZjLCWuBqOEp01Nm6I2kL79qrCBiiIc/vumHbiH4vKg fj+JvHXms12Bk1DSSmm8thT7m5pJa0t4TxGVlSE2fEIcRn4HDKJq9DL2yEzpv4e3vtPm QPsA== X-Gm-Message-State: APf1xPDSgF+9/MD2e1Fsjqm+dvmfBUWit6a/+HoVFKqXiAM2dbfqTK9M KVZcHSXRz16FMGztcHfP+S6n/w== X-Received: by 10.223.136.164 with SMTP id f33mr12893546wrf.77.1520265892635; Mon, 05 Mar 2018 08:04:52 -0800 (PST) Received: from e104803-lin.lan (mail.andrep.de. [217.160.17.100]) by smtp.gmail.com with ESMTPSA id y6sm6574381wmy.14.2018.03.05.08.04.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 05 Mar 2018 08:04:52 -0800 (PST) From: Andre Przywara To: Julien Grall , Stefano Stabellini Date: Mon, 5 Mar 2018 16:03:48 +0000 Message-Id: <20180305160415.16760-31-andre.przywara@linaro.org> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180305160415.16760-1-andre.przywara@linaro.org> References: <20180305160415.16760-1-andre.przywara@linaro.org> Cc: xen-devel@lists.xenproject.org Subject: [Xen-devel] [PATCH 30/57] ARM: new VGIC: Add IRQ sorting X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Adds the sorting function to cover the case where you have more IRQs to consider than you have LRs. We consider their priorities. This pulls in Linux' list_sort.c, which is a merge sort implementation for linked lists. Apart from adding a full featured license header and adjusting the #include file, nothing has been changed in this code. This is based on Linux commit 8e4447457965, written by Christoffer Dall. Signed-off-by: Andre Przywara --- Changelog RFC ... v1: - use Linux coding style for list_sort.c xen/arch/arm/vgic/vgic.c | 59 +++++++++++++++++ xen/common/list_sort.c | 157 ++++++++++++++++++++++++++++++++++++++++++++ xen/include/xen/list_sort.h | 11 ++++ 3 files changed, 227 insertions(+) create mode 100644 xen/common/list_sort.c create mode 100644 xen/include/xen/list_sort.h diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c index ae922815bd..efa6c67cb7 100644 --- a/xen/arch/arm/vgic/vgic.c +++ b/xen/arch/arm/vgic/vgic.c @@ -17,6 +17,7 @@ #include #include +#include #include #include "vgic.h" @@ -182,6 +183,64 @@ static struct vcpu *vgic_target_oracle(struct vgic_irq *irq) return NULL; } +/* + * The order of items in the ap_lists defines how we'll pack things in LRs as + * well, the first items in the list being the first things populated in the + * LRs. + * + * A hard rule is that active interrupts can never be pushed out of the LRs + * (and therefore take priority) since we cannot reliably trap on deactivation + * of IRQs and therefore they have to be present in the LRs. + * + * Otherwise things should be sorted by the priority field and the GIC + * hardware support will take care of preemption of priority groups etc. + * + * Return negative if "a" sorts before "b", 0 to preserve order, and positive + * to sort "b" before "a". + */ +static int vgic_irq_cmp(void *priv, struct list_head *a, struct list_head *b) +{ + struct vgic_irq *irqa = container_of(a, struct vgic_irq, ap_list); + struct vgic_irq *irqb = container_of(b, struct vgic_irq, ap_list); + bool penda, pendb; + int ret; + + spin_lock(&irqa->irq_lock); + spin_lock(&irqb->irq_lock); + + if ( irqa->active || irqb->active ) + { + ret = (int)irqb->active - (int)irqa->active; + goto out; + } + + penda = irqa->enabled && irq_is_pending(irqa); + pendb = irqb->enabled && irq_is_pending(irqb); + + if ( !penda || !pendb ) + { + ret = (int)pendb - (int)penda; + goto out; + } + + /* Both pending and enabled, sort by priority */ + ret = irqa->priority - irqb->priority; +out: + spin_unlock(&irqb->irq_lock); + spin_unlock(&irqa->irq_lock); + return ret; +} + +/* Must be called with the ap_list_lock held */ +static void vgic_sort_ap_list(struct vcpu *vcpu) +{ + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic; + + ASSERT(spin_is_locked(&vgic_cpu->ap_list_lock)); + + list_sort(NULL, &vgic_cpu->ap_list_head, vgic_irq_cmp); +} + /* * Only valid injection if changing level for level-triggered IRQs or for a * rising edge. diff --git a/xen/common/list_sort.c b/xen/common/list_sort.c new file mode 100644 index 0000000000..af2b2f6519 --- /dev/null +++ b/xen/common/list_sort.c @@ -0,0 +1,157 @@ +/* + * list_sort.c: merge sort implementation for linked lists + * Copied from the Linux kernel (lib/list_sort.c) + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; If not, see . + */ + +#include +#include + +#define MAX_LIST_LENGTH_BITS 20 + +/* + * Returns a list organized in an intermediate format suited + * to chaining of merge() calls: null-terminated, no reserved or + * sentinel head node, "prev" links not maintained. + */ +static struct list_head *merge(void *priv, + int (*cmp)(void *priv, struct list_head *a, + struct list_head *b), + struct list_head *a, struct list_head *b) +{ + struct list_head head, *tail = &head; + + while (a && b) { + /* if equal, take 'a' -- important for sort stability */ + if ((*cmp)(priv, a, b) <= 0) { + tail->next = a; + a = a->next; + } else { + tail->next = b; + b = b->next; + } + tail = tail->next; + } + tail->next = a?:b; + return head.next; +} + +/* + * Combine final list merge with restoration of standard doubly-linked + * list structure. This approach duplicates code from merge(), but + * runs faster than the tidier alternatives of either a separate final + * prev-link restoration pass, or maintaining the prev links + * throughout. + */ +static void merge_and_restore_back_links(void *priv, + int (*cmp)(void *priv, struct list_head *a, + struct list_head *b), + struct list_head *head, + struct list_head *a, struct list_head *b) +{ + struct list_head *tail = head; + u8 count = 0; + + while (a && b) { + /* if equal, take 'a' -- important for sort stability */ + if ((*cmp)(priv, a, b) <= 0) { + tail->next = a; + a->prev = tail; + a = a->next; + } else { + tail->next = b; + b->prev = tail; + b = b->next; + } + tail = tail->next; + } + tail->next = a ? : b; + + do { + /* + * In worst cases this loop may run many iterations. + * Continue callbacks to the client even though no + * element comparison is needed, so the client's cmp() + * routine can invoke cond_resched() periodically. + */ + if (unlikely(!(++count))) + (*cmp)(priv, tail->next, tail->next); + + tail->next->prev = tail; + tail = tail->next; + } while (tail->next); + + tail->next = head; + head->prev = tail; +} + +/** + * list_sort - sort a list + * @priv: private data, opaque to list_sort(), passed to @cmp + * @head: the list to sort + * @cmp: the elements comparison function + * + * This function implements "merge sort", which has O(nlog(n)) + * complexity. + * + * The comparison function @cmp must return a negative value if @a + * should sort before @b, and a positive value if @a should sort after + * @b. If @a and @b are equivalent, and their original relative + * ordering is to be preserved, @cmp must return 0. + */ +void list_sort(void *priv, struct list_head *head, + int (*cmp)(void *priv, struct list_head *a, + struct list_head *b)) +{ + struct list_head *part[MAX_LIST_LENGTH_BITS+1]; /* sorted partial lists + -- last slot is a sentinel */ + int lev; /* index into part[] */ + int max_lev = 0; + struct list_head *list; + + if (list_empty(head)) + return; + + memset(part, 0, sizeof(part)); + + head->prev->next = NULL; + list = head->next; + + while (list) { + struct list_head *cur = list; + list = list->next; + cur->next = NULL; + + for (lev = 0; part[lev]; lev++) { + cur = merge(priv, cmp, part[lev], cur); + part[lev] = NULL; + } + if (lev > max_lev) { + if (unlikely(lev >= ARRAY_SIZE(part)-1)) { + dprintk(XENLOG_DEBUG, + "list too long for efficiency\n"); + lev--; + } + max_lev = lev; + } + part[lev] = cur; + } + + for (lev = 0; lev < max_lev; lev++) + if (part[lev]) + list = merge(priv, cmp, part[lev], list); + + merge_and_restore_back_links(priv, cmp, head, part[max_lev], list); +} +EXPORT_SYMBOL(list_sort); diff --git a/xen/include/xen/list_sort.h b/xen/include/xen/list_sort.h new file mode 100644 index 0000000000..13ce0a55ec --- /dev/null +++ b/xen/include/xen/list_sort.h @@ -0,0 +1,11 @@ +#ifndef _LINUX_LIST_SORT_H +#define _LINUX_LIST_SORT_H + +#include + +struct list_head; + +void list_sort(void *priv, struct list_head *head, + int (*cmp)(void *priv, struct list_head *a, + struct list_head *b)); +#endif