From patchwork Thu Dec 8 17:50:30 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 87328 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp975808qgi; Thu, 8 Dec 2016 10:03:08 -0800 (PST) X-Received: by 10.55.220.3 with SMTP id v3mr63789865qki.37.1481220188689; Thu, 08 Dec 2016 10:03:08 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [208.118.235.17]) by mx.google.com with ESMTPS id o91si6020654qte.134.2016.12.08.10.03.08 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 08 Dec 2016 10:03:08 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:48071 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cF32C-0007L0-8K for patch@linaro.org; Thu, 08 Dec 2016 13:03:08 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60013) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cF2qd-0004SO-Vj for qemu-devel@nongnu.org; Thu, 08 Dec 2016 12:51:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cF2qc-0004JC-GK for qemu-devel@nongnu.org; Thu, 08 Dec 2016 12:51:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:54106) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cF2qU-0004H1-SB; Thu, 08 Dec 2016 12:51:03 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E725E285A5; Thu, 8 Dec 2016 17:51:01 +0000 (UTC) Received: from kamzik.brq.redhat.com (kamzik.brq.redhat.com [10.34.1.143]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uB8HoWnB022068; Thu, 8 Dec 2016 12:50:59 -0500 From: Andrew Jones To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Thu, 8 Dec 2016 18:50:30 +0100 Message-Id: <20161208175030.12269-11-drjones@redhat.com> In-Reply-To: <20161208175030.12269-1-drjones@redhat.com> References: <20161208175030.12269-1-drjones@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Thu, 08 Dec 2016 17:51:02 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH kvm-unit-tests v8 10/10] arm/arm64: gic: don't just use zero X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, marc.zyngier@arm.com, andre.przywara@arm.com, eric.auger@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org, christoffer.dall@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Instead of using cpu0 and irq=0 for the IPI test, use something a bit more interesting. To make sure we can still run the test with the minimal number of cpus (2), we need to use sender=1. As the irq in the test (an SGI) can only be 0 to 15 and there's really no difference, then use irq=1. sender=1 is a bit limited for the long term, as we may want to test IPIs across host sockets with pinned vcpus. But we can improve on this later, possibly with the introduction of a command line parameter. Reviewed-by: Eric Auger Signed-off-by: Andrew Jones --- v8: - remove crufy command line parsing. No need for sender or irq to be input right now. v7: - cleanup cmdline parsing and add complain on bad args [Eric] v6: - make sender/irq names more future-proof [drew] - sanity check inputs [drew] - introduce check_sender/irq and bad_sender/irq to more cleanly do checks [drew] - default sender and irq to 1, instead of still zero [drew] v4: improve structure and make sure spurious checking is done even when the sender isn't cpu0 v2: actually check that the irq received was the irq sent, and (for gicv2) that the sender is the expected one. --- arm/gic.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 73 insertions(+), 23 deletions(-) -- 2.9.3 diff --git a/arm/gic.c b/arm/gic.c index d0d3be0fa36e..3054d45f7387 100644 --- a/arm/gic.c +++ b/arm/gic.c @@ -19,6 +19,9 @@ #include #include +#define IPI_SENDER 1 +#define IPI_IRQ 1 + struct gic { struct { void (*send_self)(void); @@ -28,6 +31,7 @@ struct gic { static struct gic *gic; static int acked[NR_CPUS], spurious[NR_CPUS]; +static int bad_sender[NR_CPUS], bad_irq[NR_CPUS]; static cpumask_t ready; static void nr_cpu_check(int nr) @@ -43,10 +47,23 @@ static void wait_on_ready(void) cpu_relax(); } +static void stats_reset(void) +{ + int i; + + for (i = 0; i < nr_cpus; ++i) { + acked[i] = 0; + bad_sender[i] = -1; + bad_irq[i] = -1; + } + smp_wmb(); +} + static void check_acked(cpumask_t *mask) { int missing = 0, extra = 0, unexpected = 0; int nr_pass, cpu, i; + bool bad = false; /* Wait up to 5s for all interrupts to be delivered */ for (i = 0; i < 50; ++i) { @@ -56,9 +73,21 @@ static void check_acked(cpumask_t *mask) smp_rmb(); nr_pass += cpumask_test_cpu(cpu, mask) ? acked[cpu] == 1 : acked[cpu] == 0; + + if (bad_sender[cpu] != -1) { + printf("cpu%d received IPI from wrong sender %d\n", + cpu, bad_sender[cpu]); + bad = true; + } + + if (bad_irq[cpu] != -1) { + printf("cpu%d received wrong irq %d\n", + cpu, bad_irq[cpu]); + bad = true; + } } if (nr_pass == nr_cpus) { - report("Completed in %d ms", true, ++i * 100); + report("Completed in %d ms", !bad, ++i * 100); return; } } @@ -91,6 +120,22 @@ static void check_spurious(void) } } +static void check_ipi_sender(u32 irqstat) +{ + if (gic_version() == 2) { + int src = (irqstat >> 10) & 7; + + if (src != IPI_SENDER) + bad_sender[smp_processor_id()] = src; + } +} + +static void check_irqnr(u32 irqnr) +{ + if (irqnr != IPI_IRQ) + bad_irq[smp_processor_id()] = irqnr; +} + static void ipi_handler(struct pt_regs *regs __unused) { u32 irqstat = gic_read_iar(); @@ -98,8 +143,10 @@ static void ipi_handler(struct pt_regs *regs __unused) if (irqnr != GICC_INT_SPURIOUS) { gic_write_eoir(irqstat); - smp_rmb(); /* pairs with wmb in ipi_test functions */ + smp_rmb(); /* pairs with wmb in stats_reset */ ++acked[smp_processor_id()]; + check_ipi_sender(irqstat); + check_irqnr(irqnr); smp_wmb(); /* pairs with rmb in check_acked */ } else { ++spurious[smp_processor_id()]; @@ -109,22 +156,22 @@ static void ipi_handler(struct pt_regs *regs __unused) static void gicv2_ipi_send_self(void) { - writel(2 << 24, gicv2_dist_base() + GICD_SGIR); + writel(2 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR); } static void gicv2_ipi_send_broadcast(void) { - writel(1 << 24, gicv2_dist_base() + GICD_SGIR); + writel(1 << 24 | IPI_IRQ, gicv2_dist_base() + GICD_SGIR); } static void gicv3_ipi_send_self(void) { - gic_ipi_send_single(0, smp_processor_id()); + gic_ipi_send_single(IPI_IRQ, smp_processor_id()); } static void gicv3_ipi_send_broadcast(void) { - gicv3_write_sgi1r(1ULL << 40); + gicv3_write_sgi1r(1ULL << 40 | IPI_IRQ << 24); isb(); } @@ -133,10 +180,9 @@ static void ipi_test_self(void) cpumask_t mask; report_prefix_push("self"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_clear(&mask); - cpumask_set_cpu(0, &mask); + cpumask_set_cpu(smp_processor_id(), &mask); gic->ipi.send_self(); check_acked(&mask); report_prefix_pop(); @@ -148,20 +194,18 @@ static void ipi_test_smp(void) int i; report_prefix_push("target-list"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_copy(&mask, &cpu_present_mask); - for (i = 0; i < nr_cpus; i += 2) + for (i = smp_processor_id() & 1; i < nr_cpus; i += 2) cpumask_clear_cpu(i, &mask); - gic_ipi_send_mask(0, &mask); + gic_ipi_send_mask(IPI_IRQ, &mask); check_acked(&mask); report_prefix_pop(); report_prefix_push("broadcast"); - memset(acked, 0, sizeof(acked)); - smp_wmb(); + stats_reset(); cpumask_copy(&mask, &cpu_present_mask); - cpumask_clear_cpu(0, &mask); + cpumask_clear_cpu(smp_processor_id(), &mask); gic->ipi.send_broadcast(); check_acked(&mask); report_prefix_pop(); @@ -178,6 +222,16 @@ static void ipi_enable(void) local_irq_enable(); } +static void ipi_send(void) +{ + ipi_enable(); + wait_on_ready(); + ipi_test_self(); + ipi_test_smp(); + check_spurious(); + exit(report_summary()); +} + static void ipi_recv(void) { ipi_enable(); @@ -232,14 +286,10 @@ int main(int argc, char **argv) for_each_present_cpu(cpu) { if (cpu == 0) continue; - smp_boot_secondary(cpu, ipi_recv); + smp_boot_secondary(cpu, + cpu == IPI_SENDER ? ipi_send : ipi_recv); } - ipi_enable(); - wait_on_ready(); - ipi_test_self(); - ipi_test_smp(); - check_spurious(); - report_prefix_pop(); + ipi_recv(); } else { report_abort("Unknown subtest '%s'", argv[1]); }