From patchwork Thu Sep 3 22:06:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 306236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 989DFC43461 for ; Thu, 3 Sep 2020 22:09:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4409320716 for ; Thu, 3 Sep 2020 22:09:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZNunQ9kR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4409320716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxQ7-0002LC-G7 for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:09:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45026) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNf-0006I5-TL; Thu, 03 Sep 2020 18:06:55 -0400 Received: from mail-qt1-x844.google.com ([2607:f8b0:4864:20::844]:39748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNd-0007RC-GM; Thu, 03 Sep 2020 18:06:55 -0400 Received: by mail-qt1-x844.google.com with SMTP id 92so3194939qtb.6; Thu, 03 Sep 2020 15:06:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1+upsevs/BTkj+6JCHslmbD/e7PT+189Qq4R2MNrZUc=; b=ZNunQ9kRH/B6XP2S6bovBw67NmMoai14BjWUD9iL8klsBFpfBhLJnQ2YF1Z0H34hp1 /dfyH+ueLWRrEK7GPKy7dD0qCaixqajs2L4Q08tQcJ8RDGE2reRTKQp0DrxPpbvM2R/X uSqbN3ftDRvU1n+5HzNcxem/AQKmreDu6Hm5AjU/B2h2iMfiykIfjwfLmm5b/0hEKa4e 6VpNbwuVkSoJ5SSJx0CfNRJu3w/cKmY/Pl5GjBdX98hgt0zOTtUcFdCF+petmpLtY0ZD zD6EpYoXsJo+9RUDmphrhHZMnOIoGfWcBdjRheH4hF9zpjL5yXz7+LSbTwfFkEI5mxIN L8BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1+upsevs/BTkj+6JCHslmbD/e7PT+189Qq4R2MNrZUc=; b=C/TgkjO7VW+ooBUJjogGTftCuaRaa/9VcIxJnxpqJGrn3N8sCyw8bo+7j9XpAbLOKG I7BksGhouO47kWxJybh9sfjSufBJmjEo4Tq2zWtk1m/OU7mUfosSc54P09ogZmrbsRXl PP99hqqqAMaWwAVPOCAZIsZLszZ5HBdbJWrILy/BDsUWG4jotQ0c7u1mtX9BRpcrekAi rY4mADAWleiyLjvdje43gsf3yZ1k/H35knVCHfAoGCodMQ/PAznwbCMWxPjzT9a8LLo0 owkS2mGKoQCqzWt67nxG3VDM2P3Lt5JMCzzj2V2NxLSA9cxORtQv9iXHfmmbJAL7W4rN O/Dg== X-Gm-Message-State: AOAM533WaH/6kDxzq8BZ1acUl3OSY0dZRWgk+urmucNyUV0MQhBObd3D pTlObnVqKI7o29ihlBU5xQOpyFlFmc4UDg== X-Google-Smtp-Source: ABdhPJxLqIlAuzqTwC5ad/awWRrTPxtwjCDSuDmj+MvSosJKI2GnqWxZ6Kvq0sdpYvRLPx10Zknr5Q== X-Received: by 2002:aed:2986:: with SMTP id o6mr5768245qtd.269.1599170811098; Thu, 03 Sep 2020 15:06:51 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:50 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 1/7] spapr: introduce SpaprMachineState::numa_assoc_array Date: Thu, 3 Sep 2020 19:06:33 -0300 Message-Id: <20200903220639.563090-2-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::844; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x844.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The next step to centralize all NUMA/associativity handling in the spapr machine is to create a 'one stop place' for all things ibm,associativity. This patch introduces numa_assoc_array, a 2 dimensional array that will store all ibm,associativity arrays of all NUMA nodes. This array is initialized in a new spapr_numa_associativity_init() function, called in spapr_machine_init(). It is being initialized with the same values used in other ibm,associativity properties around spapr files (i.e. all zeros, last value is node_id). The idea is to remove all hardcoded definitions and FDT writes of ibm,associativity arrays, doing instead a call to the new helper spapr_numa_write_associativity_dt() helper, that will be able to write the DT with the correct values. We'll start small, handling the trivial cases first. The remaining instances of ibm,associativity will be handled next. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 23 ++++++++++------------- hw/ppc/spapr_numa.c | 30 ++++++++++++++++++++++++++++++ hw/ppc/spapr_nvdimm.c | 19 +++++++------------ hw/ppc/spapr_pci.c | 9 ++------- include/hw/ppc/spapr.h | 12 ++++++++++++ include/hw/ppc/spapr_numa.h | 11 +++++++++++ include/hw/ppc/spapr_nvdimm.h | 2 +- 7 files changed, 73 insertions(+), 33 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index a45912acac..1ad6f59863 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -314,14 +314,9 @@ static void add_str(GString *s, const gchar *s1) g_string_append_len(s, s1, strlen(s1) + 1); } -static int spapr_dt_memory_node(void *fdt, int nodeid, hwaddr start, - hwaddr size) +static int spapr_dt_memory_node(SpaprMachineState *spapr, void *fdt, int nodeid, + hwaddr start, hwaddr size) { - uint32_t associativity[] = { - cpu_to_be32(0x4), /* length */ - cpu_to_be32(0x0), cpu_to_be32(0x0), - cpu_to_be32(0x0), cpu_to_be32(nodeid) - }; char mem_name[32]; uint64_t mem_reg_property[2]; int off; @@ -335,8 +330,7 @@ static int spapr_dt_memory_node(void *fdt, int nodeid, hwaddr start, _FDT((fdt_setprop_string(fdt, off, "device_type", "memory"))); _FDT((fdt_setprop(fdt, off, "reg", mem_reg_property, sizeof(mem_reg_property)))); - _FDT((fdt_setprop(fdt, off, "ibm,associativity", associativity, - sizeof(associativity)))); + spapr_numa_write_associativity_dt(spapr, fdt, off, nodeid); return off; } @@ -649,7 +643,7 @@ static int spapr_dt_memory(SpaprMachineState *spapr, void *fdt) if (!mem_start) { /* spapr_machine_init() checks for rma_size <= node0_size * already */ - spapr_dt_memory_node(fdt, i, 0, spapr->rma_size); + spapr_dt_memory_node(spapr, fdt, i, 0, spapr->rma_size); mem_start += spapr->rma_size; node_size -= spapr->rma_size; } @@ -661,7 +655,7 @@ static int spapr_dt_memory(SpaprMachineState *spapr, void *fdt) sizetmp = 1ULL << ctzl(mem_start); } - spapr_dt_memory_node(fdt, i, mem_start, sizetmp); + spapr_dt_memory_node(spapr, fdt, i, mem_start, sizetmp); node_size -= sizetmp; mem_start += sizetmp; } @@ -1275,7 +1269,7 @@ void *spapr_build_fdt(SpaprMachineState *spapr, bool reset, size_t space) /* NVDIMM devices */ if (mc->nvdimm_supported) { - spapr_dt_persistent_memory(fdt); + spapr_dt_persistent_memory(spapr, fdt); } return fdt; @@ -2810,6 +2804,9 @@ static void spapr_machine_init(MachineState *machine) */ spapr->gpu_numa_id = MAX(1, machine->numa_state->num_nodes); + /* Init numa_assoc_array */ + spapr_numa_associativity_init(spapr, machine); + if ((!kvm_enabled() || kvmppc_has_cap_mmu_radix()) && ppc_type_check_compat(machine->cpu_type, CPU_POWERPC_LOGICAL_3_00, 0, spapr->max_compat_pvr)) { @@ -3394,7 +3391,7 @@ int spapr_lmb_dt_populate(SpaprDrc *drc, SpaprMachineState *spapr, addr = spapr_drc_index(drc) * SPAPR_MEMORY_BLOCK_SIZE; node = object_property_get_uint(OBJECT(drc->dev), PC_DIMM_NODE_PROP, &error_abort); - *fdt_start_offset = spapr_dt_memory_node(fdt, node, addr, + *fdt_start_offset = spapr_dt_memory_node(spapr, fdt, node, addr, SPAPR_MEMORY_BLOCK_SIZE); return 0; } diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index cdf3288cbd..f6b6fe648f 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -15,6 +15,36 @@ #include "hw/ppc/spapr_numa.h" #include "hw/ppc/fdt.h" + +void spapr_numa_associativity_init(SpaprMachineState *spapr, + MachineState *machine) +{ + int nb_numa_nodes = machine->numa_state->num_nodes; + int i; + + /* + * For all associativity arrays: first position is the size, + * position MAX_DISTANCE_REF_POINTS is always the numa_id, + * represented by the index 'i'. + * + * This will break on sparse NUMA setups, when/if QEMU starts + * to support it, because there will be no more guarantee that + * 'i' will be a valid node_id set by the user. + */ + for (i = 0; i < nb_numa_nodes; i++) { + spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS); + spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i); + } +} + +void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, + int offset, int nodeid) +{ + _FDT((fdt_setprop(fdt, offset, "ibm,associativity", + spapr->numa_assoc_array[nodeid], + sizeof(spapr->numa_assoc_array[nodeid])))); +} + /* * Helper that writes ibm,associativity-reference-points and * max-associativity-domains in the RTAS pointed by @rtas diff --git a/hw/ppc/spapr_nvdimm.c b/hw/ppc/spapr_nvdimm.c index 5188e2f503..63872054f3 100644 --- a/hw/ppc/spapr_nvdimm.c +++ b/hw/ppc/spapr_nvdimm.c @@ -31,6 +31,7 @@ #include "hw/ppc/fdt.h" #include "qemu/range.h" #include "sysemu/sysemu.h" +#include "hw/ppc/spapr_numa.h" void spapr_nvdimm_validate(HotplugHandler *hotplug_dev, NVDIMMDevice *nvdimm, uint64_t size, Error **errp) @@ -117,8 +118,8 @@ void spapr_create_nvdimm_dr_connectors(SpaprMachineState *spapr) } -static int spapr_dt_nvdimm(void *fdt, int parent_offset, - NVDIMMDevice *nvdimm) +static int spapr_dt_nvdimm(SpaprMachineState *spapr, void *fdt, + int parent_offset, NVDIMMDevice *nvdimm) { int child_offset; char *buf; @@ -128,11 +129,6 @@ static int spapr_dt_nvdimm(void *fdt, int parent_offset, &error_abort); uint64_t slot = object_property_get_uint(OBJECT(nvdimm), PC_DIMM_SLOT_PROP, &error_abort); - uint32_t associativity[] = { - cpu_to_be32(0x4), /* length */ - cpu_to_be32(0x0), cpu_to_be32(0x0), - cpu_to_be32(0x0), cpu_to_be32(node) - }; uint64_t lsize = nvdimm->label_size; uint64_t size = object_property_get_int(OBJECT(nvdimm), PC_DIMM_SIZE_PROP, NULL); @@ -152,8 +148,7 @@ static int spapr_dt_nvdimm(void *fdt, int parent_offset, _FDT((fdt_setprop_string(fdt, child_offset, "compatible", "ibm,pmemory"))); _FDT((fdt_setprop_string(fdt, child_offset, "device_type", "ibm,pmemory"))); - _FDT((fdt_setprop(fdt, child_offset, "ibm,associativity", associativity, - sizeof(associativity)))); + spapr_numa_write_associativity_dt(spapr, fdt, child_offset, node); buf = qemu_uuid_unparse_strdup(&nvdimm->uuid); _FDT((fdt_setprop_string(fdt, child_offset, "ibm,unit-guid", buf))); @@ -179,12 +174,12 @@ int spapr_pmem_dt_populate(SpaprDrc *drc, SpaprMachineState *spapr, { NVDIMMDevice *nvdimm = NVDIMM(drc->dev); - *fdt_start_offset = spapr_dt_nvdimm(fdt, 0, nvdimm); + *fdt_start_offset = spapr_dt_nvdimm(spapr, fdt, 0, nvdimm); return 0; } -void spapr_dt_persistent_memory(void *fdt) +void spapr_dt_persistent_memory(SpaprMachineState *spapr, void *fdt) { int offset = fdt_subnode_offset(fdt, 0, "persistent-memory"); GSList *iter, *nvdimms = nvdimm_get_device_list(); @@ -202,7 +197,7 @@ void spapr_dt_persistent_memory(void *fdt) for (iter = nvdimms; iter; iter = iter->next) { NVDIMMDevice *nvdimm = iter->data; - spapr_dt_nvdimm(fdt, offset, nvdimm); + spapr_dt_nvdimm(spapr, fdt, offset, nvdimm); } g_slist_free(nvdimms); diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index 0a418f1e67..4d97ff6c70 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -52,6 +52,7 @@ #include "sysemu/kvm.h" #include "sysemu/hostmem.h" #include "sysemu/numa.h" +#include "hw/ppc/spapr_numa.h" /* Copied from the kernel arch/powerpc/platforms/pseries/msi.c */ #define RTAS_QUERY_FN 0 @@ -2321,11 +2322,6 @@ int spapr_dt_phb(SpaprMachineState *spapr, SpaprPhbState *phb, cpu_to_be32(1), cpu_to_be32(RTAS_IBM_RESET_PE_DMA_WINDOW) }; - uint32_t associativity[] = {cpu_to_be32(0x4), - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(phb->numa_node)}; SpaprTceTable *tcet; SpaprDrc *drc; Error *err = NULL; @@ -2358,8 +2354,7 @@ int spapr_dt_phb(SpaprMachineState *spapr, SpaprPhbState *phb, /* Advertise NUMA via ibm,associativity */ if (phb->numa_node != -1) { - _FDT(fdt_setprop(fdt, bus_off, "ibm,associativity", associativity, - sizeof(associativity))); + spapr_numa_write_associativity_dt(spapr, fdt, bus_off, phb->numa_node); } /* Build the interrupt-map, this must matches what is done diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index a1e230ad39..9a63380801 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -105,6 +105,16 @@ typedef enum { #define FDT_MAX_SIZE 0x100000 +/* + * NUMA related macros. MAX_DISTANCE_REF_POINTS was taken + * from Taken from Linux kernel arch/powerpc/mm/numa.h. + * + * NUMA_ASSOC_SIZE is the base array size of an ibm,associativity + * array for any non-CPU resource. + */ +#define MAX_DISTANCE_REF_POINTS 4 +#define NUMA_ASSOC_SIZE (MAX_DISTANCE_REF_POINTS + 1) + typedef struct SpaprCapabilities SpaprCapabilities; struct SpaprCapabilities { uint8_t caps[SPAPR_CAP_NUM]; @@ -231,6 +241,8 @@ struct SpaprMachineState { unsigned gpu_numa_id; SpaprTpmProxy *tpm_proxy; + uint32_t numa_assoc_array[MAX_NODES][NUMA_ASSOC_SIZE]; + Error *fwnmi_migration_blocker; }; diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index 7a370a8768..a2a4df55f7 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -13,8 +13,19 @@ #ifndef HW_SPAPR_NUMA_H #define HW_SPAPR_NUMA_H +#include "hw/boards.h" #include "hw/ppc/spapr.h" +/* + * Having both SpaprMachineState and MachineState as arguments + * feels odd, but it will spare a MACHINE() call inside the + * function. spapr_machine_init() is the only caller for it, and + * it has both pointers resolved already. + */ +void spapr_numa_associativity_init(SpaprMachineState *spapr, + MachineState *machine); void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas); +void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, + int offset, int nodeid); #endif /* HW_SPAPR_NUMA_H */ diff --git a/include/hw/ppc/spapr_nvdimm.h b/include/hw/ppc/spapr_nvdimm.h index 10a6d9dbbc..3eb344e8e9 100644 --- a/include/hw/ppc/spapr_nvdimm.h +++ b/include/hw/ppc/spapr_nvdimm.h @@ -27,7 +27,7 @@ QEMU_BUILD_BUG_ON(SPAPR_MINIMUM_SCM_BLOCK_SIZE % SPAPR_MEMORY_BLOCK_SIZE); int spapr_pmem_dt_populate(SpaprDrc *drc, SpaprMachineState *spapr, void *fdt, int *fdt_start_offset, Error **errp); -void spapr_dt_persistent_memory(void *fdt); +void spapr_dt_persistent_memory(SpaprMachineState *spapr, void *fdt); void spapr_nvdimm_validate(HotplugHandler *hotplug_dev, NVDIMMDevice *nvdimm, uint64_t size, Error **errp); void spapr_add_nvdimm(DeviceState *dev, uint64_t slot, Error **errp); From patchwork Thu Sep 3 22:06:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 306237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D6E3C433E2 for ; Thu, 3 Sep 2020 22:09:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3C5E820722 for ; Thu, 3 Sep 2020 22:09:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OvgCv+mf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3C5E820722 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:58908 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxQ0-00026J-DB for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:09:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNg-0006IX-4B; Thu, 03 Sep 2020 18:06:57 -0400 Received: from mail-qt1-x842.google.com ([2607:f8b0:4864:20::842]:39747) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNe-0007RO-AK; Thu, 03 Sep 2020 18:06:55 -0400 Received: by mail-qt1-x842.google.com with SMTP id 92so3194987qtb.6; Thu, 03 Sep 2020 15:06:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mwvH00LlvAqHSvHjVx15zL8Ar9OIaFAfTOZGCWwV8uo=; b=OvgCv+mfFhWXhaIF9ZyTWN8GqzrDVNUXZqrq/amP0jAZY+UDqXIw0I72MsiEuJmtvR uazHfwSLdkmNoqzMJa11M0vy5fX+w6lkBxMzliwkw95idgurxWCc08cSqJIuvugk4xLX OecoJlENXWNRYeJ3bi7Zm5QSGTytuYe/q+gBZoO93DOc1ryw6I9MpNt4PdwkVK85khU3 hgyMn4/Lq0LZKSInpKUerIMMQ1QoZIxDTmNobOeGhktmAJRcCB5LrBJMlkY8YhHmENOZ KBji2/D4TrwUqeUhngfve42DyPNqau/0pGN5h70KqO82SFvC+ampr6arkY6W7EHt6cy5 vUlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mwvH00LlvAqHSvHjVx15zL8Ar9OIaFAfTOZGCWwV8uo=; b=EJwb5d3fN9EEe75FJTAO/iQT9tsbJQ2A3TBV+2WPsC7DuEI8+0Vq66mYlVXY7ik2qQ 0A5a5t5FEI8dI1L1kQDXQYp+QPICeFEmMgfVdbpdP6lHbk0K7yCGLK2urSX8Kp8Mo1OT xhcGeZv3psp2sUeRUcGIYQJXIFMqgATBtXKc91s12a43RML+1JzwX8may8sTBtnFKLBV v6w8Nd3LsGVS4FKtMGGXbW0sKNxL8cm1jXm/jSn11SDSg9xzQmPV03ZPblhNXqbKXKJ4 7WmkC8GsGyio1/IJOq0ajV0+7geBcCnXI5p3ZzIKpZaVsoPZN3DtakCufPFcjiZtGq2Q uniQ== X-Gm-Message-State: AOAM5314BrbUoxRXEWQPLRTjikOx/lyY7fqpsuK7O2ij04SuEdcX0eEt tjiyJYZVrclNDMdi+DpfAoEYdH9Wn4OJmg== X-Google-Smtp-Source: ABdhPJy6kng+9K4VILrM0jDC/ESJ7uwdaZd23Uvt7Qbx+GfiZ16L50jPEbsbDg1NIFz//0jzXCGBcA== X-Received: by 2002:ac8:2612:: with SMTP id u18mr5952963qtu.363.1599170812740; Thu, 03 Sep 2020 15:06:52 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:52 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 2/7] spapr, spapr_numa: handle vcpu ibm,associativity Date: Thu, 3 Sep 2020 19:06:34 -0300 Message-Id: <20200903220639.563090-3-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::842; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x842.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Vcpus have an additional paramenter to be appended, vcpu_id. This also changes the size of the of property itself, which is being represented in index 0 of numa_assoc_array[cpu->node_id], and defaults to MAX_DISTANCE_REF_POINTS for all cases but vcpus. All this logic makes more sense in spapr_numa.c, where we handle everything NUMA and associativity. A new helper spapr_numa_fixup_cpu_dt() was added, and spapr.c uses it the same way as it was using the former spapr_fixup_cpu_numa_dt(). Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 17 +---------------- hw/ppc/spapr_numa.c | 27 +++++++++++++++++++++++++++ include/hw/ppc/spapr_numa.h | 2 ++ 3 files changed, 30 insertions(+), 16 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 1ad6f59863..badfa86319 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -202,21 +202,6 @@ static int spapr_fixup_cpu_smt_dt(void *fdt, int offset, PowerPCCPU *cpu, return ret; } -static int spapr_fixup_cpu_numa_dt(void *fdt, int offset, PowerPCCPU *cpu) -{ - int index = spapr_get_vcpu_id(cpu); - uint32_t associativity[] = {cpu_to_be32(0x5), - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(cpu->node_id), - cpu_to_be32(index)}; - - /* Advertise NUMA via ibm,associativity */ - return fdt_setprop(fdt, offset, "ibm,associativity", associativity, - sizeof(associativity)); -} - static void spapr_dt_pa_features(SpaprMachineState *spapr, PowerPCCPU *cpu, void *fdt, int offset) @@ -785,7 +770,7 @@ static void spapr_dt_cpu(CPUState *cs, void *fdt, int offset, pft_size_prop, sizeof(pft_size_prop)))); if (ms->numa_state->num_nodes > 1) { - _FDT(spapr_fixup_cpu_numa_dt(fdt, offset, cpu)); + _FDT(spapr_numa_fixup_cpu_dt(spapr, fdt, offset, cpu)); } _FDT(spapr_fixup_cpu_smt_dt(fdt, offset, cpu, compat_smt)); diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index f6b6fe648f..1a1ec8bcff 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -45,6 +45,33 @@ void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, sizeof(spapr->numa_assoc_array[nodeid])))); } +int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, + int offset, PowerPCCPU *cpu) +{ + uint vcpu_assoc_size = NUMA_ASSOC_SIZE + 1; + uint32_t vcpu_assoc[vcpu_assoc_size]; + int index = spapr_get_vcpu_id(cpu); + int i; + + /* + * VCPUs have an extra 'cpu_id' value in ibm,associativity + * compared to other resources. Increment the size at index + * 0, copy all associativity domains already set, then put + * cpu_id last. + */ + vcpu_assoc[0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS + 1); + + for (i = 1; i <= MAX_DISTANCE_REF_POINTS; i++) { + vcpu_assoc[i] = spapr->numa_assoc_array[cpu->node_id][i]; + } + + vcpu_assoc[vcpu_assoc_size - 1] = cpu_to_be32(index); + + /* Advertise NUMA via ibm,associativity */ + return fdt_setprop(fdt, offset, "ibm,associativity", + vcpu_assoc, sizeof(vcpu_assoc)); +} + /* * Helper that writes ibm,associativity-reference-points and * max-associativity-domains in the RTAS pointed by @rtas diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index a2a4df55f7..43c6a16fe3 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -27,5 +27,7 @@ void spapr_numa_associativity_init(SpaprMachineState *spapr, void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas); void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, int offset, int nodeid); +int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, + int offset, PowerPCCPU *cpu); #endif /* HW_SPAPR_NUMA_H */ From patchwork Thu Sep 3 22:06:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 306233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73F5CC433E2 for ; Thu, 3 Sep 2020 22:21:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3295620716 for ; Thu, 3 Sep 2020 22:21:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G0OVBXWg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3295620716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxbr-0006Za-7m for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:21:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45072) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNj-0006Kn-MM; Thu, 03 Sep 2020 18:07:00 -0400 Received: from mail-qt1-x842.google.com ([2607:f8b0:4864:20::842]:42514) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNf-0007Ri-Tm; Thu, 03 Sep 2020 18:06:59 -0400 Received: by mail-qt1-x842.google.com with SMTP id 60so3175829qtc.9; Thu, 03 Sep 2020 15:06:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nAIWifC+pb4N38iKkEPwaKDP1cXqlraF9U9iDbNdAqM=; b=G0OVBXWgMZFjFHo0IyBrZjMbmwWWuXJlo/layXbmsBS+andsr4Txv9O9+rrbNKijT8 T43vUhqC4qNYr2TXdMIUez77KMEHTwVrEFRTWjmLy0KRQbXa2uvJC5xrtHTjuAHucfkq 9ZmfCM4ZI3qnr03LnjiEiPia6UbggAdcdrwXWb5gH7lnki31xEtFbWU5+mJ4/IX9l/O5 VykwwaSXtWf1UDYEd136WZ/OCc9OQwCgjSnBjJEiN3WXyulGFE992jmhTtOmK+4hJld8 dKFn8TWpPwu5jcqdwnFOE9LxzjxKzZ4+xWkbV+7BUVD9yS9Gz17u//Yhw3ZOQW4+93OA U0+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nAIWifC+pb4N38iKkEPwaKDP1cXqlraF9U9iDbNdAqM=; b=UUJbQneniZ+1Xs3If2lQR5rJVBuwLTmLnUBewPJOAHSb3s0oQk/l+BEI0pkn50XIQD OYQ9kRzdOAMMEBl0tFauy8f19DplTCDf/XbcHpx0E/8yaOW10Ud9fOgZ0p75ijpcQFHK CpGZ/OIbeuIRR9ShB8uKv6xqbD6KqKflwD5B0dJbNmxzAaH7owiqrGSJckpVbCKj6Y6S hpIc8QAGMX6kFYZ5zSruKj7R5O+Ljx2nGekCmLqj4PRr40ORm6Tptl3lrLpsOsjVB2Ip zoeqh7YAtWJdbCKWJXaVcZPdx0ZF90TYkgiA7XJgbPquslfrPfHrSTzv48N8b39wKaXd 6hEQ== X-Gm-Message-State: AOAM530WV2zp9KKvBwjkEafVVVdWbsmGDufTww/TSZoBgBMYQIYisqHL DIL5bInp0QxQY9/6CIKUEgz3uHzQGoP/Jw== X-Google-Smtp-Source: ABdhPJyeEovHg6+5ZkFJHkYgb1Yx6QE0fA2gHOcegYn6zFHwfHJ+6GZGCdvlLXjwnfgS+7gWopXxwQ== X-Received: by 2002:ac8:72d3:: with SMTP id o19mr5741226qtp.190.1599170814327; Thu, 03 Sep 2020 15:06:54 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:53 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 3/7] spapr, spapr_numa: move lookup-arrays handling to spapr_numa.c Date: Thu, 3 Sep 2020 19:06:35 -0300 Message-Id: <20200903220639.563090-4-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::842; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x842.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" In a similar fashion as the previous patch, let's move the handling of ibm,associativity-lookup-arrays from spapr.c to spapr_numa.c. A spapr_numa_write_assoc_lookup_arrays() helper was created, and spapr_dt_dynamic_reconfiguration_memory() can now use it to advertise the lookup-arrays. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr.c | 25 ++----------------------- hw/ppc/spapr_numa.c | 34 ++++++++++++++++++++++++++++++++++ include/hw/ppc/spapr_numa.h | 2 ++ 3 files changed, 38 insertions(+), 23 deletions(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index badfa86319..9bce1892b5 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -535,13 +535,10 @@ static int spapr_dt_dynamic_reconfiguration_memory(SpaprMachineState *spapr, void *fdt) { MachineState *machine = MACHINE(spapr); - int nb_numa_nodes = machine->numa_state->num_nodes; - int ret, i, offset; + int ret, offset; uint64_t lmb_size = SPAPR_MEMORY_BLOCK_SIZE; uint32_t prop_lmb_size[] = {cpu_to_be32(lmb_size >> 32), cpu_to_be32(lmb_size & 0xffffffff)}; - uint32_t *int_buf, *cur_index, buf_len; - int nr_nodes = nb_numa_nodes ? nb_numa_nodes : 1; MemoryDeviceInfoList *dimms = NULL; /* @@ -582,25 +579,7 @@ static int spapr_dt_dynamic_reconfiguration_memory(SpaprMachineState *spapr, return ret; } - /* ibm,associativity-lookup-arrays */ - buf_len = (nr_nodes * 4 + 2) * sizeof(uint32_t); - cur_index = int_buf = g_malloc0(buf_len); - int_buf[0] = cpu_to_be32(nr_nodes); - int_buf[1] = cpu_to_be32(4); /* Number of entries per associativity list */ - cur_index += 2; - for (i = 0; i < nr_nodes; i++) { - uint32_t associativity[] = { - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(0x0), - cpu_to_be32(i) - }; - memcpy(cur_index, associativity, sizeof(associativity)); - cur_index += 4; - } - ret = fdt_setprop(fdt, offset, "ibm,associativity-lookup-arrays", int_buf, - (cur_index - int_buf) * sizeof(uint32_t)); - g_free(int_buf); + ret = spapr_numa_write_assoc_lookup_arrays(spapr, fdt, offset); return ret; } diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 1a1ec8bcff..5a82a84438 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -72,6 +72,40 @@ int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, vcpu_assoc, sizeof(vcpu_assoc)); } + +int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState *spapr, void *fdt, + int offset) +{ + MachineState *machine = MACHINE(spapr); + int nb_numa_nodes = machine->numa_state->num_nodes; + int nr_nodes = nb_numa_nodes ? nb_numa_nodes : 1; + uint32_t *int_buf, *cur_index, buf_len; + int ret, i; + + /* ibm,associativity-lookup-arrays */ + buf_len = (nr_nodes * MAX_DISTANCE_REF_POINTS + 2) * sizeof(uint32_t); + cur_index = int_buf = g_malloc0(buf_len); + int_buf[0] = cpu_to_be32(nr_nodes); + /* Number of entries per associativity list */ + int_buf[1] = cpu_to_be32(MAX_DISTANCE_REF_POINTS); + cur_index += 2; + for (i = 0; i < nr_nodes; i++) { + /* + * For the lookup-array we use the ibm,associativity array, + * from numa_assoc_array. without the first element (size). + */ + uint32_t *associativity = spapr->numa_assoc_array[i]; + memcpy(cur_index, ++associativity, + sizeof(uint32_t) * MAX_DISTANCE_REF_POINTS); + cur_index += MAX_DISTANCE_REF_POINTS; + } + ret = fdt_setprop(fdt, offset, "ibm,associativity-lookup-arrays", int_buf, + (cur_index - int_buf) * sizeof(uint32_t)); + g_free(int_buf); + + return ret; +} + /* * Helper that writes ibm,associativity-reference-points and * max-associativity-domains in the RTAS pointed by @rtas diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index 43c6a16fe3..b3fd950634 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -29,5 +29,7 @@ void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, int offset, int nodeid); int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, int offset, PowerPCCPU *cpu); +int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState *spapr, void *fdt, + int offset); #endif /* HW_SPAPR_NUMA_H */ From patchwork Thu Sep 3 22:06:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 274567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7457DC43461 for ; Thu, 3 Sep 2020 22:23:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 31A7E20716 for ; Thu, 3 Sep 2020 22:23:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EKLe54Ac" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31A7E20716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxdI-0007wI-Dw for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:23:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45088) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNl-0006MY-GN; Thu, 03 Sep 2020 18:07:03 -0400 Received: from mail-qt1-x841.google.com ([2607:f8b0:4864:20::841]:39747) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNh-0007Sk-T5; Thu, 03 Sep 2020 18:07:01 -0400 Received: by mail-qt1-x841.google.com with SMTP id 92so3195145qtb.6; Thu, 03 Sep 2020 15:06:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GkJH56YyrTfgDLU7Hl4gNM2JIPN54D3aOTKmprseOmg=; b=EKLe54Ac4dVxyljo/zf+UD+rArPFZEbTEVDzsDmgH6DerAb7pgCGkHTt3ArbeO0Fxu NeMOej5/IiJwgETJC3+rF/3T1N9q5M1ulcOfRWRBaaiKeEezVGL97Q2DgBkKg7ImH2+c d9hzKWPM8uyPugsa1vJKS3yT9aUmOG+RSuRTUvaBAW5r4HI0W/l49SC3WiSiRgepD/I6 LRUrRTHdRX7fUMXvRPag6NRBMEkHOMVl9r8O2JQSYMBqji5tQ3xqWcas68+qAbilLJgu 47krljJTf1bq797S3QpOKRh+qnVatPLlQhNeeS3Om6OlvARQZ0NVeOsYAbfBI3F9Zuya 0rYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GkJH56YyrTfgDLU7Hl4gNM2JIPN54D3aOTKmprseOmg=; b=Uh8w/tKhyI3+vna5gU+UVQGN/gLT6iGmiVLZnqcjC1EYFTpu2n1/G1ib2QvL/vtb5v POjC9hQTv6ooYqiiwYWc+IZO8ySyefgQXxEBwpgojNLNvjsCPtuRgPQ9vhspqhHNtjY/ /5h6aje+793+UB8VZKDCwnoY18+IhMuXkZBVC/zQ9ztnslCTety5D1SGLIbJUPGO5fRe D6eW2xgDgrEmCE6R4bh5YqM6RjXyBhXG/Sp37r3u4/XpHupTjSMwbnKtCEbEGDKyokqn 94byr3gUjQzyl0+N9JCLkm2cpr3R5y263Kcgu4jLpn7G5RyO5kE5q/elbG4ljaIve5ZG 2UZw== X-Gm-Message-State: AOAM533i4eUduW6cPjEvcBGu0lWAwJ3GarZB6VfMDe6YJukSVBZ+0jFn HkF/4yTnltNzLExKtkmV4DCB3GN+AlUGtQ== X-Google-Smtp-Source: ABdhPJzcs9YJyXi20OA5LV4KCyVQtowoHX/KLm4aKeqbjrGgRqpbt5hTGrcbl2mAg05znMja+uAJEg== X-Received: by 2002:ac8:4245:: with SMTP id r5mr5762577qtm.52.1599170816101; Thu, 03 Sep 2020 15:06:56 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:55 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 4/7] spapr_numa: move NVLink2 associativity handling to spapr_numa.c Date: Thu, 3 Sep 2020 19:06:36 -0300 Message-Id: <20200903220639.563090-5-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::841; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x841.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The NVLink2 GPUs works like a regular NUMA node with its own associativity values, regardless of user input. This can be handled inside spapr_numa_associativity_init(), initializing NVGPU_MAX_NUM associativity arrays that can be used by the GPUs. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_numa.c | 28 +++++++++++++++++++++++++++- hw/ppc/spapr_pci_nvlink2.c | 20 +++----------------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 5a82a84438..93a000b729 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -13,14 +13,18 @@ #include "qemu/osdep.h" #include "qemu-common.h" #include "hw/ppc/spapr_numa.h" +#include "hw/pci-host/spapr.h" #include "hw/ppc/fdt.h" +/* Moved from hw/ppc/spapr_pci_nvlink2.c */ +#define SPAPR_GPU_NUMA_ID (cpu_to_be32(1)) void spapr_numa_associativity_init(SpaprMachineState *spapr, MachineState *machine) { + SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr); int nb_numa_nodes = machine->numa_state->num_nodes; - int i; + int i, j, max_nodes_with_gpus; /* * For all associativity arrays: first position is the size, @@ -35,6 +39,28 @@ void spapr_numa_associativity_init(SpaprMachineState *spapr, spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS); spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i); } + + /* + * Initialize NVLink GPU associativity arrays. We know that + * the first GPU will take the first available NUMA id, and + * we'll have a maximum of NVGPU_MAX_NUM GPUs in the machine. + * At this point we're not sure if there are GPUs or not, but + * let's initialize the associativity arrays and allow NVLink + * GPUs to be handled like regular NUMA nodes later on. + */ + max_nodes_with_gpus = nb_numa_nodes + NVGPU_MAX_NUM; + + for (i = nb_numa_nodes; i < max_nodes_with_gpus; i++) { + spapr->numa_assoc_array[i][0] = cpu_to_be32(MAX_DISTANCE_REF_POINTS); + + for (j = 1; j < MAX_DISTANCE_REF_POINTS; j++) { + uint32_t gpu_assoc = smc->pre_5_1_assoc_refpoints ? + SPAPR_GPU_NUMA_ID : cpu_to_be32(i); + spapr->numa_assoc_array[i][j] = gpu_assoc; + } + + spapr->numa_assoc_array[i][MAX_DISTANCE_REF_POINTS] = cpu_to_be32(i); + } } void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, diff --git a/hw/ppc/spapr_pci_nvlink2.c b/hw/ppc/spapr_pci_nvlink2.c index 76ae77ebc8..8ef9b40a18 100644 --- a/hw/ppc/spapr_pci_nvlink2.c +++ b/hw/ppc/spapr_pci_nvlink2.c @@ -26,6 +26,7 @@ #include "qemu-common.h" #include "hw/pci/pci.h" #include "hw/pci-host/spapr.h" +#include "hw/ppc/spapr_numa.h" #include "qemu/error-report.h" #include "hw/ppc/fdt.h" #include "hw/pci/pci_bridge.h" @@ -37,8 +38,6 @@ #define PHANDLE_NVLINK(phb, gn, nn) (0x00130000 | (((phb)->index) << 8) | \ ((gn) << 4) | (nn)) -#define SPAPR_GPU_NUMA_ID (cpu_to_be32(1)) - typedef struct SpaprPhbPciNvGpuSlot { uint64_t tgt; uint64_t gpa; @@ -360,13 +359,6 @@ void spapr_phb_nvgpu_ram_populate_dt(SpaprPhbState *sphb, void *fdt) Object *nv_mrobj = object_property_get_link(OBJECT(nvslot->gpdev), "nvlink2-mr[0]", &error_abort); - uint32_t associativity[] = { - cpu_to_be32(0x4), - cpu_to_be32(nvslot->numa_id), - cpu_to_be32(nvslot->numa_id), - cpu_to_be32(nvslot->numa_id), - cpu_to_be32(nvslot->numa_id) - }; uint64_t size = object_property_get_uint(nv_mrobj, "size", NULL); uint64_t mem_reg[2] = { cpu_to_be64(nvslot->gpa), cpu_to_be64(size) }; char *mem_name = g_strdup_printf("memory@%"PRIx64, nvslot->gpa); @@ -376,14 +368,8 @@ void spapr_phb_nvgpu_ram_populate_dt(SpaprPhbState *sphb, void *fdt) _FDT((fdt_setprop_string(fdt, off, "device_type", "memory"))); _FDT((fdt_setprop(fdt, off, "reg", mem_reg, sizeof(mem_reg)))); - if (sphb->pre_5_1_assoc) { - associativity[1] = SPAPR_GPU_NUMA_ID; - associativity[2] = SPAPR_GPU_NUMA_ID; - associativity[3] = SPAPR_GPU_NUMA_ID; - } - - _FDT((fdt_setprop(fdt, off, "ibm,associativity", associativity, - sizeof(associativity)))); + spapr_numa_write_associativity_dt(SPAPR_MACHINE(qdev_get_machine()), + fdt, off, nvslot->numa_id); _FDT((fdt_setprop_string(fdt, off, "compatible", "ibm,coherent-device-memory"))); From patchwork Thu Sep 3 22:06:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 306235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECB13C43461 for ; Thu, 3 Sep 2020 22:11:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB32320716 for ; Thu, 3 Sep 2020 22:11:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b7msyaBz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB32320716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40988 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxSQ-0006KR-U1 for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:11:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNn-0006NZ-H4; Thu, 03 Sep 2020 18:07:03 -0400 Received: from mail-qt1-x843.google.com ([2607:f8b0:4864:20::843]:37087) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNj-0007T9-N5; Thu, 03 Sep 2020 18:07:03 -0400 Received: by mail-qt1-x843.google.com with SMTP id k25so2249816qtu.4; Thu, 03 Sep 2020 15:06:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rkIDipTQJyp7b+9IYufFx9hAJCjuKNzbTlrb+coLwpE=; b=b7msyaBzZPfCMg+6ek1fTFsbGkN3+bI34jB7b2kUNRkNxM3oftErId6D7WLIcyURXS IbuAfGNbN22zDBS3p97oxW+y4tpi40huq8LeJlWISRFlHkHJSEoFNtNmRpHfettVkO9g qQqJwC3q0RAIl1pYy34xeIHaq5mgWVlynvQWNMmmVZBStwLgL/NfL0e3xSrdi1FRs1oT zoECyMzvpmr6IjvwurjBI6C23tNGwsNegfUrd5XhbmWkoDT7ejaKqTdISu2prIxUWLjK iaYiwmLKEhYlGoRp1vyP9rRt4TjrAE0c4Dh1HGbgnykv7PlxTxelUXnYseTx3S7Gi5Sw qRCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rkIDipTQJyp7b+9IYufFx9hAJCjuKNzbTlrb+coLwpE=; b=qIAs840TnRAYQKxkSgWL1hxokOv7BA1k4hOelZ+5k6fM6DgWI1ehvoc0kuePt7JqZt tIaTn8virGKknupX1damXuIcwyDtOnzBa6OEba6DiRx06hx6rJh5ZAGrmeqvwvx/9UOY //wYuQsk2zphKKm7Ok1eZDma8fuBFPmKhxzi340S87TupJsT7Hj2OS+3ksOpBZ/pHJtA lGhDcn0eFgQAIIK8AD9VwAbPzYdyOguWXlv7/O43xeSRkzHcBVTunaugAlycTLl5TJiO /GEkwVX/wFIpLPpplkuhP+wkoZDPDZz8r/KQPDmxr3JYvppnrQ0Pb1e10eGTF4uKM+di o84g== X-Gm-Message-State: AOAM530SK3Fv6ruLcrmg77BDxHsnrjRMEN9HZu1AsqN0KCxLs9zdHryS O/Fgeu3cM5i2aYtocqmp5ZdB7FJ6whhCSw== X-Google-Smtp-Source: ABdhPJy7ebtJHK/a/vk0+/QWaFqGfiFUMkT/BFFKsg14mAKCU6Rbh74TOJgVdNN59mKzKCJnJE37RQ== X-Received: by 2002:ac8:44b5:: with SMTP id a21mr5806713qto.36.1599170817999; Thu, 03 Sep 2020 15:06:57 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:57 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 5/7] spapr: move h_home_node_associativity to spapr_numa.c Date: Thu, 3 Sep 2020 19:06:37 -0300 Message-Id: <20200903220639.563090-6-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::843; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x843.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The implementation of this hypercall will be modified to use spapr->numa_assoc_arrays input. Moving it to spapr_numa.c makes make more sense. Signed-off-by: Daniel Henrique Barboza --- hw/ppc/spapr_hcall.c | 37 +------------------------------------ hw/ppc/spapr_numa.c | 36 ++++++++++++++++++++++++++++++++++++ include/hw/ppc/spapr_numa.h | 4 ++++ 3 files changed, 41 insertions(+), 36 deletions(-) diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index c1d01228c6..9e9b959bbd 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -18,6 +18,7 @@ #include "kvm_ppc.h" #include "hw/ppc/fdt.h" #include "hw/ppc/spapr_ovec.h" +#include "hw/ppc/spapr_numa.h" #include "mmu-book3s-v3.h" #include "hw/mem/memory-device.h" @@ -1873,42 +1874,6 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu, return ret; } -static target_ulong h_home_node_associativity(PowerPCCPU *cpu, - SpaprMachineState *spapr, - target_ulong opcode, - target_ulong *args) -{ - target_ulong flags = args[0]; - target_ulong procno = args[1]; - PowerPCCPU *tcpu; - int idx; - - /* only support procno from H_REGISTER_VPA */ - if (flags != 0x1) { - return H_FUNCTION; - } - - tcpu = spapr_find_cpu(procno); - if (tcpu == NULL) { - return H_P2; - } - - /* sequence is the same as in the "ibm,associativity" property */ - - idx = 0; -#define ASSOCIATIVITY(a, b) (((uint64_t)(a) << 32) | \ - ((uint64_t)(b) & 0xffffffff)) - args[idx++] = ASSOCIATIVITY(0, 0); - args[idx++] = ASSOCIATIVITY(0, tcpu->node_id); - args[idx++] = ASSOCIATIVITY(procno, -1); - for ( ; idx < 6; idx++) { - args[idx] = -1; - } -#undef ASSOCIATIVITY - - return H_SUCCESS; -} - static target_ulong h_get_cpu_characteristics(PowerPCCPU *cpu, SpaprMachineState *spapr, target_ulong opcode, diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index 93a000b729..d4dca57321 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -165,3 +165,39 @@ void spapr_numa_write_rtas_dt(SpaprMachineState *spapr, void *fdt, int rtas) _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains", maxdomains, sizeof(maxdomains))); } + +target_ulong h_home_node_associativity(PowerPCCPU *cpu, + SpaprMachineState *spapr, + target_ulong opcode, + target_ulong *args) +{ + target_ulong flags = args[0]; + target_ulong procno = args[1]; + PowerPCCPU *tcpu; + int idx; + + /* only support procno from H_REGISTER_VPA */ + if (flags != 0x1) { + return H_FUNCTION; + } + + tcpu = spapr_find_cpu(procno); + if (tcpu == NULL) { + return H_P2; + } + + /* sequence is the same as in the "ibm,associativity" property */ + + idx = 0; +#define ASSOCIATIVITY(a, b) (((uint64_t)(a) << 32) | \ + ((uint64_t)(b) & 0xffffffff)) + args[idx++] = ASSOCIATIVITY(0, 0); + args[idx++] = ASSOCIATIVITY(0, tcpu->node_id); + args[idx++] = ASSOCIATIVITY(procno, -1); + for ( ; idx < 6; idx++) { + args[idx] = -1; + } +#undef ASSOCIATIVITY + + return H_SUCCESS; +} diff --git a/include/hw/ppc/spapr_numa.h b/include/hw/ppc/spapr_numa.h index b3fd950634..5b4d165c06 100644 --- a/include/hw/ppc/spapr_numa.h +++ b/include/hw/ppc/spapr_numa.h @@ -31,5 +31,9 @@ int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, int offset, PowerPCCPU *cpu); int spapr_numa_write_assoc_lookup_arrays(SpaprMachineState *spapr, void *fdt, int offset); +target_ulong h_home_node_associativity(PowerPCCPU *cpu, + SpaprMachineState *spapr, + target_ulong opcode, + target_ulong *args); #endif /* HW_SPAPR_NUMA_H */ From patchwork Thu Sep 3 22:06:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 274570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_INVALID, DKIM_SIGNED, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2324C433E2 for ; Thu, 3 Sep 2020 22:11:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2A22F20716 for ; Thu, 3 Sep 2020 22:11:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="oq3BcToO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A22F20716 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36838 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kDxRq-0004fZ-BC for qemu-devel@archiver.kernel.org; Thu, 03 Sep 2020 18:11:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45112) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kDxNn-0006Ow-UB; Thu, 03 Sep 2020 18:07:03 -0400 Received: from mail-qt1-x842.google.com ([2607:f8b0:4864:20::842]:39748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kDxNl-0007TP-Hv; Thu, 03 Sep 2020 18:07:03 -0400 Received: by mail-qt1-x842.google.com with SMTP id 92so3195262qtb.6; Thu, 03 Sep 2020 15:07:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8AcS+2MgC3Xn7XSXkC9a9dEuvG+oK6tDqMnM4qvC+dU=; b=oq3BcToOSYmqJDEIQMixJsfzZQhp/uVchSleg8Q9DLnh+icgpIPtsk2bLc3bbJqLv0 hx3oxrnQuAIMjKVbZaNufnELf7uoCHxj2dDybHFSmTlF9uA4VqHehq4v5AVWcDuvrMfg F3v4MTbmWXNBxPHd0dc/wEmC1tIHFfGKKLK6vPqYb/2uYaBKYBpM36NhLFJtzVJRmDLT 7JlZOE8K2KmfRIMbgsRBf4JaDnSXUN2Mb+MdUB7UTeorbC6J9ec/MTmoiYrPzLjiZNaI Za2hS2zpakwNzOxGFn3dC7fm3XndgVuAw8RlUM6iugyiTG2d0YCF+bcJ+2rjbhZCzEor cqGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8AcS+2MgC3Xn7XSXkC9a9dEuvG+oK6tDqMnM4qvC+dU=; b=dMYldKm7BQv5a8UTr7WZl8oTrWMZj36f8K6VW6n5/SvDteinw9OAcaPAp+zgpdi3Mn RNBpN5EsEKO2ZTZh5MBr4JEGRzP6d7akHx1akzGbVAVCf9/6xSxeAsOSI1S58A2jU4ZP QQDR3kXfu1u8QN257JioBxQb88T88VaFTIg1MQYjS744w7pa+E+315wWV3KgoMNE5mEO FsFrFRJVcpXZM/2G09D66tKBrrtN1UjXg3NcZwvQH6WHTjGZ1aXKkYzuDe5v1VflBL6r chzh+CbZN4pexI7+fRslO+Ox7O1GKa+ssSoV76+Aohjlkq3SdZRf96rZzpbyuu76Pu3J 2YVA== X-Gm-Message-State: AOAM532j89EgmegP+ErPzzJbSKMyLIwYcTWljulcjVKD/5jwREplM/Kx wOilyJBzPcpPJSt9k+O9rssbSoKwSMKD6A== X-Google-Smtp-Source: ABdhPJxUo882Mu856Qcn/cxeo4RwfJ70o+Ud42Z8WPgg8dX9vLZxwhFCicyZDGCF/CYrTLbOKfIJBw== X-Received: by 2002:ac8:48ca:: with SMTP id l10mr5692348qtr.385.1599170819866; Thu, 03 Sep 2020 15:06:59 -0700 (PDT) Received: from rekt.ibmuc.com ([2804:431:c7c6:1dc:ffbd:d3fb:97ff:aaca]) by smtp.gmail.com with ESMTPSA id j1sm1798757qtk.91.2020.09.03.15.06.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 03 Sep 2020 15:06:59 -0700 (PDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Subject: [PATCH v3 6/7] spapr_numa: create a vcpu associativity helper Date: Thu, 3 Sep 2020 19:06:38 -0300 Message-Id: <20200903220639.563090-7-danielhb413@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200903220639.563090-1-danielhb413@gmail.com> References: <20200903220639.563090-1-danielhb413@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::842; envelope-from=danielhb413@gmail.com; helo=mail-qt1-x842.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Henrique Barboza , qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The work to be done in h_home_node_associativity() intersects with what is already done in spapr_numa_fixup_cpu_dt(). This patch creates a new helper, spapr_numa_get_vcpu_assoc(), to be used for both spapr_numa_fixup_cpu_dt() and h_home_node_associativity(). Signed-off-by: Daniel Henrique Barboza Reviewed-by: David Gibson --- hw/ppc/spapr_numa.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/hw/ppc/spapr_numa.c b/hw/ppc/spapr_numa.c index d4dca57321..abc7361921 100644 --- a/hw/ppc/spapr_numa.c +++ b/hw/ppc/spapr_numa.c @@ -71,14 +71,17 @@ void spapr_numa_write_associativity_dt(SpaprMachineState *spapr, void *fdt, sizeof(spapr->numa_assoc_array[nodeid])))); } -int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, - int offset, PowerPCCPU *cpu) +static uint32_t *spapr_numa_get_vcpu_assoc(SpaprMachineState *spapr, + PowerPCCPU *cpu, + uint *vcpu_assoc_size) { - uint vcpu_assoc_size = NUMA_ASSOC_SIZE + 1; - uint32_t vcpu_assoc[vcpu_assoc_size]; + uint32_t *vcpu_assoc = NULL; int index = spapr_get_vcpu_id(cpu); int i; + *vcpu_assoc_size = (NUMA_ASSOC_SIZE + 1) * sizeof(uint32_t); + vcpu_assoc = g_malloc(*vcpu_assoc_size); + /* * VCPUs have an extra 'cpu_id' value in ibm,associativity * compared to other resources. Increment the size at index @@ -91,11 +94,22 @@ int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, vcpu_assoc[i] = spapr->numa_assoc_array[cpu->node_id][i]; } - vcpu_assoc[vcpu_assoc_size - 1] = cpu_to_be32(index); + vcpu_assoc[MAX_DISTANCE_REF_POINTS + 1] = cpu_to_be32(index); + + return vcpu_assoc; +} + +int spapr_numa_fixup_cpu_dt(SpaprMachineState *spapr, void *fdt, + int offset, PowerPCCPU *cpu) +{ + g_autofree uint32_t *vcpu_assoc = NULL; + uint vcpu_assoc_size; + + vcpu_assoc = spapr_numa_get_vcpu_assoc(spapr, cpu, &vcpu_assoc_size); /* Advertise NUMA via ibm,associativity */ return fdt_setprop(fdt, offset, "ibm,associativity", - vcpu_assoc, sizeof(vcpu_assoc)); + vcpu_assoc, vcpu_assoc_size); }