From patchwork Mon Oct 5 16:51:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 303693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC8D1C4363A for ; Mon, 5 Oct 2020 16:59:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BF40207BC for ; Mon, 5 Oct 2020 16:59:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BF40207BC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44164 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTpJ-0001OQ-7v for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:59:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46594) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTii-0003IQ-Kh; Mon, 05 Oct 2020 12:52:16 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:13754 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTig-0001dc-RN; Mon, 05 Oct 2020 12:52:16 -0400 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095GhEV9124946; Mon, 5 Oct 2020 12:52:02 -0400 Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0b-001b2d01.pphosted.com with ESMTP id 34078er857-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:02 -0400 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095GasDl013719; Mon, 5 Oct 2020 16:52:00 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma01fra.de.ibm.com with ESMTP id 33xgx7s7h5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:00 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095GpwuH19988746 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:51:58 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5CAE04203F; Mon, 5 Oct 2020 16:51:58 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F29CE42047; Mon, 5 Oct 2020 16:51:57 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by d06av24.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:51:57 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id 49B6D220155; Mon, 5 Oct 2020 18:51:57 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 1/6] spapr/xive: Introduce a StoreEOI capability Date: Mon, 5 Oct 2020 18:51:42 +0200 Message-Id: <20201005165147.526426-2-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 mlxlogscore=842 impostorscore=0 phishscore=0 mlxscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 clxscore=1034 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.158.5; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:05 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When an interrupt has been handled, the OS notifies the interrupt controller with an EOI sequence. On the XIVE interrupt controller (POWER9 and POWER10), this can be done with a load or a store operation on the ESB interrupt management page of the interrupt. The StoreEOI operation has less latency and improves interrupt handling performance but it was deactivated during the POWER9 DD2.0 time-frame because of ordering issues. POWER9 systems use the LoadEOI instead. POWER10 has fixed the issue with a special load command which enforces Load-after-Store ordering and StoreEOI can be safely used. The new StoreEOI capability adds StoreEOI support to the flags returned by the hcall H_INT_GET_SOURCE_INFO. When the machine is using an emulated interrupt controller, TCG or without kernel IRQ chip, there are no limitations and activating StoreEOI is not an issue. However, when running with a kernel IRQ chip, some verification needs to be done on the host. This is done through the DT, which tells us that firmware has configured the HW for StoreEOI, but a new KVM capability would be cleaner. Signed-off-by: Cédric Le Goater --- include/hw/ppc/spapr.h | 4 +++- target/ppc/kvm_ppc.h | 6 ++++++ hw/ppc/spapr.c | 1 + hw/ppc/spapr_caps.c | 30 ++++++++++++++++++++++++++++++ target/ppc/kvm.c | 18 ++++++++++++++++++ 5 files changed, 58 insertions(+), 1 deletion(-) diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index bba8736269f4..b701c14b4e09 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -74,8 +74,10 @@ typedef enum { #define SPAPR_CAP_CCF_ASSIST 0x09 /* Implements PAPR FWNMI option */ #define SPAPR_CAP_FWNMI 0x0A +/* Implements XIVE StoreEOI feature */ +#define SPAPR_CAP_STOREEOI 0x0B /* Num Caps */ -#define SPAPR_CAP_NUM (SPAPR_CAP_FWNMI + 1) +#define SPAPR_CAP_NUM (SPAPR_CAP_STOREEOI + 1) /* * Capability Values diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h index 72e05f1cd2fc..c5a487dbba13 100644 --- a/target/ppc/kvm_ppc.h +++ b/target/ppc/kvm_ppc.h @@ -64,6 +64,7 @@ bool kvmppc_has_cap_htm(void); bool kvmppc_has_cap_mmu_radix(void); bool kvmppc_has_cap_mmu_hash_v3(void); bool kvmppc_has_cap_xive(void); +bool kvmppc_has_cap_xive_storeeoi(void); int kvmppc_get_cap_safe_cache(void); int kvmppc_get_cap_safe_bounds_check(void); int kvmppc_get_cap_safe_indirect_branch(void); @@ -346,6 +347,11 @@ static inline bool kvmppc_has_cap_xive(void) return false; } +static inline bool kvmppc_has_cap_xive_storeeoi(void) +{ + return false; +} + static inline int kvmppc_get_cap_safe_cache(void) { return 0; diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 4256794f3bed..e83de0580142 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -4447,6 +4447,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_ON; smc->default_caps.caps[SPAPR_CAP_FWNMI] = SPAPR_CAP_ON; + smc->default_caps.caps[SPAPR_CAP_STOREEOI] = SPAPR_CAP_OFF; spapr_caps_add_properties(smc); smc->irq = &spapr_irq_dual; smc->dr_phb_enabled = true; diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c index 9341e9782a3f..57c62c22e4cc 100644 --- a/hw/ppc/spapr_caps.c +++ b/hw/ppc/spapr_caps.c @@ -524,6 +524,26 @@ static void cap_fwnmi_apply(SpaprMachineState *spapr, uint8_t val, } } +static void cap_storeeoi_apply(SpaprMachineState *spapr, uint8_t val, + Error **errp) +{ + ERRP_GUARD(); + MachineState *machine = MACHINE(spapr); + bool kvm_storeeoi = kvmppc_has_cap_xive_storeeoi(); + + if (!val) { + return; /* Disabled by default */ + } + + /* Check host support when the KVM device is in use */ + if (kvm_irqchip_in_kernel()) { + if (!kvm_storeeoi) { + error_setg(errp, "StoreEOI not supported by KVM"); + return; + } + } +} + SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = { [SPAPR_CAP_HTM] = { .name = "htm", @@ -632,6 +652,15 @@ SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = { .type = "bool", .apply = cap_fwnmi_apply, }, + [SPAPR_CAP_STOREEOI] = { + .name = "storeeoi", + .description = "Implements XIVE StoreEOI feature", + .index = SPAPR_CAP_STOREEOI, + .get = spapr_cap_get_bool, + .set = spapr_cap_set_bool, + .type = "bool", + .apply = cap_storeeoi_apply, + }, }; static SpaprCapabilities default_caps_with_cpu(SpaprMachineState *spapr, @@ -772,6 +801,7 @@ SPAPR_CAP_MIG_STATE(nested_kvm_hv, SPAPR_CAP_NESTED_KVM_HV); SPAPR_CAP_MIG_STATE(large_decr, SPAPR_CAP_LARGE_DECREMENTER); SPAPR_CAP_MIG_STATE(ccf_assist, SPAPR_CAP_CCF_ASSIST); SPAPR_CAP_MIG_STATE(fwnmi, SPAPR_CAP_FWNMI); +SPAPR_CAP_MIG_STATE(storeeoi, SPAPR_CAP_STOREEOI); void spapr_caps_init(SpaprMachineState *spapr) { diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index d85ba8ffe00b..9ad637151070 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -2448,6 +2448,24 @@ bool kvmppc_has_cap_xive(void) return cap_xive; } +/* + * TODO: Introduce a new KVM capability + */ +bool kvmppc_has_cap_xive_storeeoi(void) +{ + static const char *compat = "ibm,opal-xive-pe"; + void *host_fdt; + int xive_node; + + host_fdt = load_device_tree_from_sysfs(); + xive_node = fdt_node_offset_by_compatible(host_fdt, -1, compat); + if (xive_node < 0) { + return false; + } + + return !!fdt_getprop(host_fdt, xive_node, "store-eoi-support", NULL); +} + static void kvmppc_get_cpu_characteristics(KVMState *s) { struct kvm_ppc_cpu_char c; From patchwork Mon Oct 5 16:51:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 303695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9407AC4363A for ; Mon, 5 Oct 2020 16:54:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1FB8C207BC for ; Mon, 5 Oct 2020 16:54:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FB8C207BC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35538 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTl6-0006Ar-6Y for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:54:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTif-0003BK-MW; Mon, 05 Oct 2020 12:52:13 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:3130 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTie-0001eQ-61; Mon, 05 Oct 2020 12:52:13 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095GX9A1009575; Mon, 5 Oct 2020 12:52:03 -0400 Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com [149.81.74.108]) by mx0b-001b2d01.pphosted.com with ESMTP id 34061casn2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:03 -0400 Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1]) by ppma05fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095GcOst017991; Mon, 5 Oct 2020 16:52:01 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma05fra.de.ibm.com with ESMTP id 33xgx817re-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:01 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095Gpxru11731374 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:51:59 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 149A9AE058; Mon, 5 Oct 2020 16:51:59 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AE4B8AE055; Mon, 5 Oct 2020 16:51:58 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:51:58 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id DEAB22201BC; Mon, 5 Oct 2020 18:51:57 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 2/6] spapr/xive: Add a warning when StoreEOI is activated on POWER8 CPUs Date: Mon, 5 Oct 2020 18:51:43 +0200 Message-Id: <20201005165147.526426-3-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 priorityscore=1501 phishscore=0 mlxlogscore=667 impostorscore=0 lowpriorityscore=0 spamscore=0 clxscore=1034 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.158.5; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:05 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Cédric Le Goater --- hw/ppc/spapr_caps.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c index 57c62c22e4cc..b0a9d0227db2 100644 --- a/hw/ppc/spapr_caps.c +++ b/hw/ppc/spapr_caps.c @@ -535,6 +535,14 @@ static void cap_storeeoi_apply(SpaprMachineState *spapr, uint8_t val, return; /* Disabled by default */ } + /* For POWER8 CPUs, setting StoreEOI is useless as XIVE is not used */ + if (!ppc_type_check_compat(machine->cpu_type, CPU_POWERPC_LOGICAL_3_00, 0, + spapr->max_compat_pvr)) { + warn_report("StoreEOI is for the XIVE interrupt mode " + "(POWER9 and above)"); + return; + } + /* Check host support when the KVM device is in use */ if (kvm_irqchip_in_kernel()) { if (!kvm_storeeoi) { From patchwork Mon Oct 5 16:51:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 303696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29D6AC47095 for ; Mon, 5 Oct 2020 16:53:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D2AF3207F7 for ; Mon, 5 Oct 2020 16:53:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2AF3207F7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59060 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTjk-0004Fa-Lz for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:53:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTie-00039L-Ln; Mon, 05 Oct 2020 12:52:12 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:52548) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTic-0001e3-Vm; Mon, 05 Oct 2020 12:52:12 -0400 Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095GXEFZ035795; Mon, 5 Oct 2020 12:52:05 -0400 Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com with ESMTP id 3406ths78f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:04 -0400 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095GbHql014170; Mon, 5 Oct 2020 16:52:02 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma01fra.de.ibm.com with ESMTP id 33xgx7s7h6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:02 +0000 Received: from b06wcsmtp001.portsmouth.uk.ibm.com (b06wcsmtp001.portsmouth.uk.ibm.com [9.149.105.160]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095GpxVL25952584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:51:59 GMT Received: from b06wcsmtp001.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A5466A405F; Mon, 5 Oct 2020 16:51:59 +0000 (GMT) Received: from b06wcsmtp001.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63479A405C; Mon, 5 Oct 2020 16:51:59 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by b06wcsmtp001.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:51:59 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id 95A85220155; Mon, 5 Oct 2020 18:51:58 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 3/6] spapr/xive: Add a warning when StoreEOI is activated on POWER9 CPUs Date: Mon, 5 Oct 2020 18:51:44 +0200 Message-Id: <20201005165147.526426-4-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=731 malwarescore=0 lowpriorityscore=0 adultscore=0 spamscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 mlxscore=0 clxscore=1034 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.156.1; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:09 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" StoreEOI on POWER9 CPUs is racy because load-after-store ordering is not enforced. Signed-off-by: Cédric Le Goater --- hw/ppc/spapr_caps.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c index b0a9d0227db2..9251badbdc27 100644 --- a/hw/ppc/spapr_caps.c +++ b/hw/ppc/spapr_caps.c @@ -549,6 +549,15 @@ static void cap_storeeoi_apply(SpaprMachineState *spapr, uint8_t val, error_setg(errp, "StoreEOI not supported by KVM"); return; } + + /* + * load-after-store ordering is not enforced on POWER9 CPUs + * and StoreEOI can be racy. + */ + if (!ppc_type_check_compat(machine->cpu_type, CPU_POWERPC_LOGICAL_3_10, + 0, spapr->max_compat_pvr)) { + warn_report("StoreEOI on a POWER9 CPU is unsafe on KVM."); + } } } From patchwork Mon Oct 5 16:51:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 272060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FDACC4363A for ; Mon, 5 Oct 2020 16:57:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C2120207BC for ; Mon, 5 Oct 2020 16:57:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2120207BC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTo5-0000QX-Vp for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:57:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46558) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTif-00039l-2y; Mon, 05 Oct 2020 12:52:13 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:63782) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTid-0001e2-6M; Mon, 05 Oct 2020 12:52:12 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095GWFH0001044; Mon, 5 Oct 2020 12:52:05 -0400 Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com with ESMTP id 34072ggq7n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:04 -0400 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095GbdmT019531; Mon, 5 Oct 2020 16:52:02 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma04fra.de.ibm.com with ESMTP id 33xgx817d0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:02 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095Gq0T532571842 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:52:00 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6A4FBAE04D; Mon, 5 Oct 2020 16:52:00 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 14EA0AE053; Mon, 5 Oct 2020 16:52:00 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:52:00 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id 494572201BC; Mon, 5 Oct 2020 18:51:59 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 4/6] spapr/xive: Enforce load-after-store ordering Date: Mon, 5 Oct 2020 18:51:45 +0200 Message-Id: <20201005165147.526426-5-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1034 bulkscore=0 mlxscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 suspectscore=0 phishscore=0 spamscore=0 mlxlogscore=801 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.156.1; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:09 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The XIVE_ESB_SET_PQ_10 load operation is used to disable temporarily an interrupt source. If StoreEOI is active, a source could be left enabled if the load and store operations come out of order. QEMU makes use of this offset to quiesce the sources before a migration. Enforce the load-after-store ordering always when doing so without querying the characteristics of the sources on the host. The performance penalty will be very small for QEMU. Signed-off-by: Cédric Le Goater Reviewed-by: Greg Kurz --- include/hw/ppc/xive.h | 8 ++++++++ hw/intc/spapr_xive_kvm.c | 12 ++++++++++++ hw/intc/xive.c | 6 ++++++ 3 files changed, 26 insertions(+) diff --git a/include/hw/ppc/xive.h b/include/hw/ppc/xive.h index 445eccfe6b73..39cd273f86d5 100644 --- a/include/hw/ppc/xive.h +++ b/include/hw/ppc/xive.h @@ -279,6 +279,14 @@ static inline hwaddr xive_source_esb_mgmt(XiveSource *xsrc, int srcno) #define XIVE_ESB_SET_PQ_10 0xe00 /* Load */ #define XIVE_ESB_SET_PQ_11 0xf00 /* Load */ +/* + * Load-after-store ordering + * + * Adding this offset to the load address will enforce + * load-after-store ordering. This is required to use with StoreEOI. + */ +#define XIVE_ESB_LD_ST_MO 0x40 /* Load-after-store ordering */ + uint8_t xive_source_esb_get(XiveSource *xsrc, uint32_t srcno); uint8_t xive_source_esb_set(XiveSource *xsrc, uint32_t srcno, uint8_t pq); diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c index 66bf4c06fe55..d428422a7b72 100644 --- a/hw/intc/spapr_xive_kvm.c +++ b/hw/intc/spapr_xive_kvm.c @@ -357,6 +357,18 @@ static uint64_t xive_esb_rw(XiveSource *xsrc, int srcno, uint32_t offset, static uint8_t xive_esb_read(XiveSource *xsrc, int srcno, uint32_t offset) { + /* + * The XIVE_ESB_SET_PQ_10 load operation is used to disable + * temporarily an interrupt source. If StoreEOI is active, a + * source could be left enabled if the load and store operations + * come out of order. + * + * Enforce the load-after-store ordering always. + */ + if (offset == XIVE_ESB_SET_PQ_10) { + offset |= XIVE_ESB_LD_ST_MO; + } + return xive_esb_rw(xsrc, srcno, offset, 0, 0) & 0x3; } diff --git a/hw/intc/xive.c b/hw/intc/xive.c index 489e6256ef70..b710ba2df095 100644 --- a/hw/intc/xive.c +++ b/hw/intc/xive.c @@ -998,6 +998,12 @@ static uint64_t xive_source_esb_read(void *opaque, hwaddr addr, unsigned size) case XIVE_ESB_SET_PQ_01 ... XIVE_ESB_SET_PQ_01 + 0x0FF: case XIVE_ESB_SET_PQ_10 ... XIVE_ESB_SET_PQ_10 + 0x0FF: case XIVE_ESB_SET_PQ_11 ... XIVE_ESB_SET_PQ_11 + 0x0FF: + if (offset == XIVE_ESB_SET_PQ_10 && + xsrc->esb_flags & XIVE_SRC_STORE_EOI) { + qemu_log_mask(LOG_GUEST_ERROR, "XIVE: load-after-store ordering " + "not enforced with Store EOI active for IRQ %d\n", + srcno); + } ret = xive_source_esb_set(xsrc, srcno, (offset >> 8) & 0x3); break; default: From patchwork Mon Oct 5 16:51:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 303694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43F0DC4363A for ; Mon, 5 Oct 2020 16:56:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E6A77204EA for ; Mon, 5 Oct 2020 16:56:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6A77204EA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTmr-0007wZ-LR for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:56:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTie-00039e-W8; Mon, 05 Oct 2020 12:52:13 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:61154) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTid-0001e4-5H; Mon, 05 Oct 2020 12:52:12 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095Gjmax105076; Mon, 5 Oct 2020 12:52:06 -0400 Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 34079w03wa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:05 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095Gq3ts006808; Mon, 5 Oct 2020 16:52:03 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma03ams.nl.ibm.com with ESMTP id 33xgx8ack8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:03 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095Gq1Hh32440652 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:52:01 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 18D2D4C058; Mon, 5 Oct 2020 16:52:01 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B90994C044; Mon, 5 Oct 2020 16:52:00 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by d06av22.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:52:00 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id F2EE2220155; Mon, 5 Oct 2020 18:51:59 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 5/6] spapr/xive: Activate StoreEOI at the source level Date: Mon, 5 Oct 2020 18:51:46 +0200 Message-Id: <20201005165147.526426-6-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 priorityscore=1501 impostorscore=0 clxscore=1034 bulkscore=0 adultscore=0 suspectscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.156.1; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:09 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When the StoreEOI capability is "on", the H_INT_GET_SOURCE_INFO will set the StoreEOI flag for all sources. This could be an issue if StoreEOI is not supported on a specific source, of a passthrough device for instance. In that case, we could either introduce a new KVM ioctl to query the characteristics of the source at the HW level or deactivate StoreEOI on the machine. This is theoretically unsafe on a POWER9 host but it still runs. Signed-off-by: Cédric Le Goater --- hw/intc/spapr_xive.c | 1 + hw/ppc/spapr_irq.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 1fa09f287ac0..41f2719ff93a 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -280,6 +280,7 @@ static void spapr_xive_instance_init(Object *obj) SpaprXive *xive = SPAPR_XIVE(obj); object_initialize_child(obj, "source", &xive->source, TYPE_XIVE_SOURCE); + object_property_add_alias(obj, "flags", OBJECT(&xive->source), "flags"); object_initialize_child(obj, "end_source", &xive->end_source, TYPE_XIVE_END_SOURCE); diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index f59960339ec3..cdf9f9df4173 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -325,9 +325,14 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **errp) if (spapr->irq->xive) { uint32_t nr_servers = spapr_max_server_number(spapr); + uint64_t flags = 0; DeviceState *dev; int i; + if (spapr_get_cap(spapr, SPAPR_CAP_STOREEOI) == SPAPR_CAP_ON) { + flags |= XIVE_SRC_STORE_EOI; + } + dev = qdev_new(TYPE_SPAPR_XIVE); qdev_prop_set_uint32(dev, "nr-irqs", smc->nr_xirqs + SPAPR_XIRQ_BASE); /* @@ -337,6 +342,7 @@ void spapr_irq_init(SpaprMachineState *spapr, Error **errp) qdev_prop_set_uint32(dev, "nr-ends", nr_servers << 3); object_property_set_link(OBJECT(dev), "xive-fabric", OBJECT(spapr), &error_abort); + object_property_set_int(OBJECT(dev), "flags", flags, &error_abort); sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); spapr->xive = SPAPR_XIVE(dev); From patchwork Mon Oct 5 16:51:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 272062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9275C4363A for ; Mon, 5 Oct 2020 16:53:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25972207F7 for ; Mon, 5 Oct 2020 16:53:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25972207F7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPTjy-0004bI-2z for qemu-devel@archiver.kernel.org; Mon, 05 Oct 2020 12:53:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTid-00038z-Is; Mon, 05 Oct 2020 12:52:11 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:39284 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPTib-0001du-OK; Mon, 05 Oct 2020 12:52:11 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 095GX9A4009575; Mon, 5 Oct 2020 12:52:07 -0400 Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0b-001b2d01.pphosted.com with ESMTP id 34061casqf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 12:52:06 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 095GcqOY013442; Mon, 5 Oct 2020 16:52:05 GMT Received: from b06cxnps3074.portsmouth.uk.ibm.com (d06relay09.portsmouth.uk.ibm.com [9.149.109.194]) by ppma06ams.nl.ibm.com with ESMTP id 33xgjh2cj6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 05 Oct 2020 16:52:05 +0000 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 095Gq2oV21889418 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 5 Oct 2020 16:52:03 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B4EE1AE051; Mon, 5 Oct 2020 16:52:01 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 520F0AE057; Mon, 5 Oct 2020 16:52:01 +0000 (GMT) Received: from smtp.tlslab.ibm.com (unknown [9.101.4.1]) by d06av26.portsmouth.uk.ibm.com (Postfix) with SMTP; Mon, 5 Oct 2020 16:52:01 +0000 (GMT) Received: from yukon.ibmuc.com (sig-9-145-2-207.uk.ibm.com [9.145.2.207]) by smtp.tlslab.ibm.com (Postfix) with ESMTP id 9B3872201BC; Mon, 5 Oct 2020 18:52:00 +0200 (CEST) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: David Gibson Subject: [PATCH v2 6/6] spapr/xive: Introduce a new CAS value for the StoreEOI capability Date: Mon, 5 Oct 2020 18:51:47 +0200 Message-Id: <20201005165147.526426-7-clg@kaod.org> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20201005165147.526426-1-clg@kaod.org> References: <20201005165147.526426-1-clg@kaod.org> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-10-05_11:2020-10-05, 2020-10-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 priorityscore=1501 phishscore=0 mlxlogscore=740 impostorscore=0 lowpriorityscore=0 spamscore=0 clxscore=1034 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010050117 Received-SPF: softfail client-ip=148.163.158.5; envelope-from=clg@kaod.org; helo=mx0a-001b2d01.pphosted.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/05 12:52:05 X-ACL-Warn: Detected OS = Linux 3.x [generic] [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-devel@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-ppc@nongnu.org, Greg Kurz , Gustavo Romero Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When the StoreEOI capability is set to "cas", let CAS decide when StoreEOI should be advertised. StoreEOI is safe to use with a P10 compat machine because the OS enforces load-after-store ordering but not with P9 compat. The question now is : should we make "cas" the default at the machine level ? Signed-off-by: Cédric Le Goater --- include/hw/ppc/spapr.h | 1 + include/hw/ppc/spapr_xive.h | 1 + hw/intc/spapr_xive.c | 9 +++++++++ hw/ppc/spapr_caps.c | 21 +++++++++++++++++---- hw/ppc/spapr_hcall.c | 7 +++++++ 5 files changed, 35 insertions(+), 4 deletions(-) diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index b701c14b4e09..17e7d873e8dc 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -87,6 +87,7 @@ typedef enum { #define SPAPR_CAP_ON 0x01 /* Custom Caps */ +#define SPAPR_CAP_CAS 0x02 /* Generic */ #define SPAPR_CAP_BROKEN 0x00 diff --git a/include/hw/ppc/spapr_xive.h b/include/hw/ppc/spapr_xive.h index 26c8d90d7196..8b8aa586e44f 100644 --- a/include/hw/ppc/spapr_xive.h +++ b/include/hw/ppc/spapr_xive.h @@ -75,6 +75,7 @@ void spapr_xive_map_mmio(SpaprXive *xive); int spapr_xive_end_to_target(uint8_t end_blk, uint32_t end_idx, uint32_t *out_server, uint8_t *out_prio); +void spapr_xive_enable_store_eoi(SpaprXive *xive, bool enable); /* * KVM XIVE device helpers diff --git a/hw/intc/spapr_xive.c b/hw/intc/spapr_xive.c index 41f2719ff93a..f57a2681dd91 100644 --- a/hw/intc/spapr_xive.c +++ b/hw/intc/spapr_xive.c @@ -1802,3 +1802,12 @@ void spapr_xive_hcall_init(SpaprMachineState *spapr) spapr_register_hypercall(H_INT_SYNC, h_int_sync); spapr_register_hypercall(H_INT_RESET, h_int_reset); } + +void spapr_xive_enable_store_eoi(SpaprXive *xive, bool enable) +{ + if (enable) { + xive->source.esb_flags |= XIVE_SRC_STORE_EOI; + } else { + xive->source.esb_flags &= ~XIVE_SRC_STORE_EOI; + } +} diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c index 9251badbdc27..c55e1fccb9bc 100644 --- a/hw/ppc/spapr_caps.c +++ b/hw/ppc/spapr_caps.c @@ -524,6 +524,13 @@ static void cap_fwnmi_apply(SpaprMachineState *spapr, uint8_t val, } } +SpaprCapPossible cap_storeeoi_possible = { + .num = 3, + .vals = { "off", "on", "cas" }, + .help = "off - no StoreEOI, on - StoreEOI, " + "cas - negotiated at CAS (POWER10 compat only)", +}; + static void cap_storeeoi_apply(SpaprMachineState *spapr, uint8_t val, Error **errp) { @@ -550,6 +557,11 @@ static void cap_storeeoi_apply(SpaprMachineState *spapr, uint8_t val, return; } + /* CAS will decide to advertise StoreEOI (P10 compat kernels only) */ + if (val == SPAPR_CAP_CAS) { + return; + } + /* * load-after-store ordering is not enforced on POWER9 CPUs * and StoreEOI can be racy. @@ -671,11 +683,12 @@ SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = { }, [SPAPR_CAP_STOREEOI] = { .name = "storeeoi", - .description = "Implements XIVE StoreEOI feature", + .description = "Implements XIVE StoreEOI feature (off, on, cas)", .index = SPAPR_CAP_STOREEOI, - .get = spapr_cap_get_bool, - .set = spapr_cap_set_bool, - .type = "bool", + .get = spapr_cap_get_string, + .set = spapr_cap_set_string, + .type = "string", + .possible = &cap_storeeoi_possible, .apply = cap_storeeoi_apply, }, }; diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 607740150fa2..158b122b9192 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -1804,6 +1804,13 @@ target_ulong do_client_architecture_support(PowerPCCPU *cpu, "Guest requested unavailable interrupt mode (XIVE), try the ic-mode=xive or ic-mode=dual machine property"); exit(EXIT_FAILURE); } + + /* Advertise StoreEOI for a P10 compat guest. */ + if (spapr_get_cap(spapr, SPAPR_CAP_STOREEOI) == SPAPR_CAP_CAS) { + bool enable = ppc_check_compat(cpu, CPU_POWERPC_LOGICAL_3_10, 0, + cpu->compat_pvr); + spapr_xive_enable_store_eoi(spapr->xive, enable); + } } else { if (!spapr->irq->xics) { error_report(