From patchwork Mon Dec 16 15:38:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 181732 Delivered-To: patch@linaro.org Received: by 2002:ac9:44c4:0:0:0:0:0 with SMTP id t4csp4456849och; Mon, 16 Dec 2019 07:39:10 -0800 (PST) X-Google-Smtp-Source: APXvYqxux44GamoQN96Y+7QnqR/a38JYE2uNrCRLD7iktrGSrvSOr8+N0RM5GdeUKdXGdEa+KAy1 X-Received: by 2002:a9d:7e99:: with SMTP id m25mr3054800otp.212.1576510750313; Mon, 16 Dec 2019 07:39:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1576510750; cv=none; d=google.com; s=arc-20160816; b=yGuyTbjf44mvyMdpmVyEPFUwbH+xggj0Gh30PPla1lfq6+rchQe6uybiE6E8yoNdtY HE4qJu5lio8GzEM7xEpTPSBtA5q1GSwhxYL924FNgLn9etxZRXMjszCerfJoR8XuLzv2 qb71VxvF5ybZhp18734/2AFrORWkO+eDkamjvCfDO/aBszaHDuU0Yt42NaQeaUpoD5Ta WAVKLK/a3dPerDyshfA4s6cRRoZZURT6+jt5FSO9sJpebyKY56zLYVQLrRfFMvp6uS3B m2p+xbENBLkjJZAs4IHZLw1dWwsFQ8S97eF8omd5e+k2gGB0+0MlsdY9guqVxF1f4ayt E8vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=njb6DQZ7gaEzx4PKISLGbxSFVYqBkbyxzkqmU/A7QbY=; b=t0aER7NT5HEy1O9SStSW2yfucC99DPM6Cra6sekabu0cvpXay3kYI+uJwEhAym56b3 +F0x/ueBUC3aVPok1wKiS+qS9Om4HEzFBzhfkx7H5uhp+kY5lMeXMneHA+js8AH/hRA/ IDzAtiqcxBrQsTNE0OxAopJgNbFyyDTsfEdlXBqLm/+Ca9+4VAnw+ycyIyXY/cGkki2G ujrMq6/3Y3o5xqEzpU+Oc4+4YJrzRabViCIDCnt3wwcFfM6W6PLe6h8H5TF9L96DNago CooTY9+nd9sgaXLqmI76epKzxupEpCKJMFmlEETJscqbE8R5g3lz1E3satRzbKtUmC0P dTsg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c10si11102645oto.293.2019.12.16.07.39.10; Mon, 16 Dec 2019 07:39:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728502AbfLPPjJ (ORCPT + 27 others); Mon, 16 Dec 2019 10:39:09 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:40648 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728337AbfLPPjI (ORCPT ); Mon, 16 Dec 2019 10:39:08 -0500 Received: from DGGEMS405-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 5B48DCA6A2C002DBDA19; Mon, 16 Dec 2019 23:39:03 +0800 (CST) Received: from lhrphicprd00229.huawei.com (10.123.41.22) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.439.0; Mon, 16 Dec 2019 23:38:57 +0800 From: Jonathan Cameron To: , , , , CC: Keith Busch , , "Rafael J . Wysocki" , , Andrew Morton , Dan Williams , Tao Xu , Brice Goglin , Lorenzo Pieralisi , Hanjun Guo , Sudeep Holla , Jonathan Cameron Subject: [PATCH V6 2/7] arm64: Support Generic Initiator only domains Date: Mon, 16 Dec 2019 23:38:04 +0800 Message-ID: <20191216153809.105463-3-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> References: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.123.41.22] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The one thing that currently needs doing from an architecture point of view is associating the GI domain with its nearest memory domain. This allows all the standard NUMA aware code to get a 'reasonable' answer. A clever driver might elect to do load balancing etc if there are multiple host / memory domains nearby, but that's a decision for the driver. Signed-off-by: Jonathan Cameron --- arch/arm64/kernel/smp.c | 8 ++++++++ 1 file changed, 8 insertions(+) -- 2.19.1 diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index d4ed9a19d8fe..eb5ef84fe7b9 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -716,6 +716,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus) { int err; unsigned int cpu; + unsigned int node; unsigned int this_cpu; init_cpu_topology(); @@ -754,6 +755,13 @@ void __init smp_prepare_cpus(unsigned int max_cpus) set_cpu_present(cpu, true); numa_store_cpu_info(cpu); } + + /* + * Walk the numa domains and set the node to numa memory reference + * for any that are Generic Initiator Only. + */ + for_each_node_state(node, N_GENERIC_INITIATOR) + set_gi_numa_mem(node, local_memory_node(node)); } void (*__smp_cross_call)(const struct cpumask *, unsigned int); From patchwork Mon Dec 16 15:38:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 181733 Delivered-To: patch@linaro.org Received: by 2002:ac9:44c4:0:0:0:0:0 with SMTP id t4csp4456901och; Mon, 16 Dec 2019 07:39:14 -0800 (PST) X-Google-Smtp-Source: APXvYqx2TKifLprs/0cmS6Nz4bQaqzr/EGzNaQRBJJVMQgbnOuuDILkZPQUaojtX7Eibj/o/QKDq X-Received: by 2002:aca:f1d4:: with SMTP id p203mr9565055oih.116.1576510754122; Mon, 16 Dec 2019 07:39:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1576510754; cv=none; d=google.com; s=arc-20160816; b=JyWVrz74oi14Mm7RqsLW+yqBgDYeXQBJU7D/wIBn1IoMF5+ap48DVU0j42Eze7c4Zz 5ZTtrkfF+X+57iBNtat1EEUyr2uM1uGfnnQ1C0T40gx7I47W8nrDpGg520HV5TyoybuL YoVCH8aGaAlRmxRicZpHikA7X2KZm4Q0cNqk2l1XrXpG+43f4fsLmcB1TnGwSCT/C/B7 J+UPndela+q6PpLDdn+fend6juBrCSlqMjZOkP53lQcYKp3/9cNoTd4GA8EUksGruD9N bVb/mQLlUhO1K//wdhG7S54GyaUvWvWucodzg15TQpY68dPsZ2rA7BBWMTpkxE1pv84k H1PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=qP8xRfhkWQXV+pfJFYFyaC1byBz/FCkuypyjMcqO1fY=; b=ygsRcLd6GdnIzcrAvy2/FL21klF3fiPxyvbsR6HEzhNCx3EEtaqcTu2OWgpLJlmWb1 qkx0Q5lDP0iVRt0bcXJAIjc90PsdD6cbXEojeAuySUg7ENQxmocSEJDyf6RDbw2eRwXz v+qfYUv660nl+6tRDccUzWRWx2mY9V4IX/6zEC0YDrRYxD+HL3ELlIZawE+CoAK16OEM PDl5gToZsHFuiyAbFl98BqzyTSLyp4XqR8yELSVjBYmjP2frA+ordS7pNif8XcheHAcY QI+HulDlGGtFyaHTvBQjR2AM2D1e5eRoApONNVrRgq791CqJqNcL+5/a76yCxjMbisZJ 3dtw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a197si10796356oii.199.2019.12.16.07.39.13; Mon, 16 Dec 2019 07:39:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728248AbfLPPjM (ORCPT + 27 others); Mon, 16 Dec 2019 10:39:12 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:40762 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728337AbfLPPjL (ORCPT ); Mon, 16 Dec 2019 10:39:11 -0500 Received: from DGGEMS405-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 6AB2D10314FD91876B55; Mon, 16 Dec 2019 23:39:08 +0800 (CST) Received: from lhrphicprd00229.huawei.com (10.123.41.22) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.439.0; Mon, 16 Dec 2019 23:39:01 +0800 From: Jonathan Cameron To: , , , , CC: Keith Busch , , "Rafael J . Wysocki" , , Andrew Morton , Dan Williams , Tao Xu , Brice Goglin , Lorenzo Pieralisi , Hanjun Guo , Sudeep Holla , Jonathan Cameron Subject: [PATCH V6 3/7] x86: Support Generic Initiator only proximity domains Date: Mon, 16 Dec 2019 23:38:05 +0800 Message-ID: <20191216153809.105463-4-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> References: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.123.41.22] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Done in a somewhat different fashion to arm64. Here the infrastructure for memoryless domains was already in place. That infrastruture applies just as well to domains that also don't have a CPU, hence it works for Generic Initiator Domains. In common with memoryless domains we only register GI domains if the proximity node is not online. If a domain is already a memory containing domain, or a memoryless domain there is nothing to do just because it also contains a Generic Initiator. Signed-off-by: Jonathan Cameron --- arch/x86/include/asm/numa.h | 2 ++ arch/x86/kernel/setup.c | 1 + arch/x86/mm/numa.c | 14 ++++++++++++++ 3 files changed, 17 insertions(+) -- 2.19.1 diff --git a/arch/x86/include/asm/numa.h b/arch/x86/include/asm/numa.h index bbfde3d2662f..f631467272a3 100644 --- a/arch/x86/include/asm/numa.h +++ b/arch/x86/include/asm/numa.h @@ -62,12 +62,14 @@ extern void numa_clear_node(int cpu); extern void __init init_cpu_to_node(void); extern void numa_add_cpu(int cpu); extern void numa_remove_cpu(int cpu); +extern void init_gi_nodes(void); #else /* CONFIG_NUMA */ static inline void numa_set_node(int cpu, int node) { } static inline void numa_clear_node(int cpu) { } static inline void init_cpu_to_node(void) { } static inline void numa_add_cpu(int cpu) { } static inline void numa_remove_cpu(int cpu) { } +static inline void init_gi_nodes(void) { } #endif /* CONFIG_NUMA */ #ifdef CONFIG_DEBUG_PER_CPU_MAPS diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index cedfe2077a69..c21fc5b9f729 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1279,6 +1279,7 @@ void __init setup_arch(char **cmdline_p) prefill_possible_map(); init_cpu_to_node(); + init_gi_nodes(); io_apic_init_mappings(); diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c index 99f7a68738f0..53ac09f6138c 100644 --- a/arch/x86/mm/numa.c +++ b/arch/x86/mm/numa.c @@ -733,6 +733,20 @@ static void __init init_memory_less_node(int nid) */ } +/* + * Generic Initiator Nodes may have neither CPU nor Memory. + * At this stage if either of the others were present we would + * already be online. + */ +void __init init_gi_nodes(void) +{ + int nid; + + for_each_node_state(nid, N_GENERIC_INITIATOR) + if (!node_online(nid)) + init_memory_less_node(nid); +} + /* * Setup early cpu_to_node. * From patchwork Mon Dec 16 15:38:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 181736 Delivered-To: patch@linaro.org Received: by 2002:ac9:44c4:0:0:0:0:0 with SMTP id t4csp4457221och; Mon, 16 Dec 2019 07:39:29 -0800 (PST) X-Google-Smtp-Source: APXvYqwurS4R68DaeDy25UjAnU6O1QAZa/a/bSCYIK7K59oHQf+P433S/KSh/lrISKL0lj7QhN7J X-Received: by 2002:aca:cd92:: with SMTP id d140mr9517701oig.68.1576510769250; Mon, 16 Dec 2019 07:39:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1576510769; cv=none; d=google.com; s=arc-20160816; b=ddUqNBV/fCZ6OvlqQe8HXBhl2wuxptxS8PH4EkHZSpY5JK3bnukz5YCXkEr6J68hvQ X6Lz5AkkswKznCan59qCvThIyLN9zCppqlNt/HJVG+ZO39Xu+h5a7jp8hF16IYSLde6E 6ehFsv9/mobP5VrQ7RBJbT9Dw2i/2w16KqnA4C6oTnsCkpmdhIrzWJYimCl4xMTbWLw0 KhKlaBy8IPpQxuO81mfP35kX7VB3auEVMa6Oxergl6maoRSlMzAoTlQh6GUdeiAwPBvz N9MbmiiE5yutqu0T75hCeEsRLEvw/bKU0Sol4dUyUWDrFV7aKq/P90MzepzOnmagmEiu 5lTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=s0P99RqkkmFyhsiY/LIZW5TZoJ4P6h7EySEZE5gL1JI=; b=0Hc2eRH6oMnucuFuj0l96/1fE+bifCBzrEwy7k39iEc2oGhBXDpmQW1JVTQuEzrDpA 5npe9snf3Z/CKc8bpVRFX2Zr+6XiROtn43t6DXeo9Ze2lkoemR6XGhLtdapj2k7EeTmX ys8VY4iJxc4IjjumRyg6gqd9TRGB/NKWTdsCgmzwUkytbmTQ2zJJlCtZMixACmbPsEtJ O9+bcn9nPdlOKnpBvHtW8NqRVS2tOGZEU0gKnZRQ+tdxT93VITTLIplHxxLRplQASeFw LYQIPeYDIDlsacIRti+gsr+qBluKVy/iPKvtIpCp77OystzANIH01pBZ/VQ7EaW/RpJ/ xb9g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z3si10004094oib.164.2019.12.16.07.39.28; Mon, 16 Dec 2019 07:39:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728555AbfLPPj1 (ORCPT + 27 others); Mon, 16 Dec 2019 10:39:27 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:7690 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728527AbfLPPjZ (ORCPT ); Mon, 16 Dec 2019 10:39:25 -0500 Received: from DGGEMS405-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 9D3173CB701614B8F7A8; Mon, 16 Dec 2019 23:39:23 +0800 (CST) Received: from lhrphicprd00229.huawei.com (10.123.41.22) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.439.0; Mon, 16 Dec 2019 23:39:14 +0800 From: Jonathan Cameron To: , , , , CC: Keith Busch , , "Rafael J . Wysocki" , , Andrew Morton , Dan Williams , Tao Xu , Brice Goglin , Lorenzo Pieralisi , Hanjun Guo , Sudeep Holla , Jonathan Cameron Subject: [PATCH V6 6/7] node: Add access1 class to represent CPU to memory characteristics Date: Mon, 16 Dec 2019 23:38:08 +0800 Message-ID: <20191216153809.105463-7-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> References: <20191216153809.105463-1-Jonathan.Cameron@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.123.41.22] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org New access1 class is nearly the same as access0, but always provides characteristics for CPUs to memory. The existing access0 class provides characteristics to nearest or direct connnect initiator which may be a Generic Initiator such as a GPU or network adapter. This new class allows thread placement on CPUs to be performed so as to give optimal access characteristics to memory, even if that memory is for example attached to a GPU or similar and only accessible to the CPU via an appropriate bus. Suggested-by: Dan Willaims Signed-off-by: Jonathan Cameron --- Note that this code could have been shorter by copying the bitmap and factoring out the generic parts of access0 and access1. Personally I felt that reduced readability but happy to change that if people prefer. drivers/acpi/numa/hmat.c | 87 +++++++++++++++++++++++++++++++--------- 1 file changed, 68 insertions(+), 19 deletions(-) -- 2.19.1 diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 07cfe50136e0..00b4cdbefb5e 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -56,7 +56,7 @@ struct memory_target { unsigned int memory_pxm; unsigned int processor_pxm; struct resource memregions; - struct node_hmem_attrs hmem_attrs; + struct node_hmem_attrs hmem_attrs[2]; struct list_head caches; struct node_cache_attrs cache_attrs; bool registered; @@ -65,6 +65,7 @@ struct memory_target { struct memory_initiator { struct list_head node; unsigned int processor_pxm; + bool has_cpu; }; struct memory_locality { @@ -108,6 +109,7 @@ static __init void alloc_memory_initiator(unsigned int cpu_pxm) return; initiator->processor_pxm = cpu_pxm; + initiator->has_cpu = node_state(pxm_to_node(cpu_pxm), N_CPU); list_add_tail(&initiator->node, &initiators); } @@ -215,28 +217,28 @@ static u32 hmat_normalize(u16 entry, u64 base, u8 type) } static void hmat_update_target_access(struct memory_target *target, - u8 type, u32 value) + u8 type, u32 value, int access) { switch (type) { case ACPI_HMAT_ACCESS_LATENCY: - target->hmem_attrs.read_latency = value; - target->hmem_attrs.write_latency = value; + target->hmem_attrs[access].read_latency = value; + target->hmem_attrs[access].write_latency = value; break; case ACPI_HMAT_READ_LATENCY: - target->hmem_attrs.read_latency = value; + target->hmem_attrs[access].read_latency = value; break; case ACPI_HMAT_WRITE_LATENCY: - target->hmem_attrs.write_latency = value; + target->hmem_attrs[access].write_latency = value; break; case ACPI_HMAT_ACCESS_BANDWIDTH: - target->hmem_attrs.read_bandwidth = value; - target->hmem_attrs.write_bandwidth = value; + target->hmem_attrs[access].read_bandwidth = value; + target->hmem_attrs[access].write_bandwidth = value; break; case ACPI_HMAT_READ_BANDWIDTH: - target->hmem_attrs.read_bandwidth = value; + target->hmem_attrs[access].read_bandwidth = value; break; case ACPI_HMAT_WRITE_BANDWIDTH: - target->hmem_attrs.write_bandwidth = value; + target->hmem_attrs[access].write_bandwidth = value; break; default: break; @@ -329,8 +331,12 @@ static __init int hmat_parse_locality(union acpi_subtable_headers *header, if (mem_hier == ACPI_HMAT_MEMORY) { target = find_mem_target(targs[targ]); - if (target && target->processor_pxm == inits[init]) - hmat_update_target_access(target, type, value); + if (target && target->processor_pxm == inits[init]) { + hmat_update_target_access(target, type, value, 0); + /* If the node has a CPU, update access 1*/ + if (node_state(pxm_to_node(inits[init]), N_CPU)) + hmat_update_target_access(target, type, value, 1); + } } } } @@ -566,6 +572,7 @@ static void hmat_register_target_initiators(struct memory_target *target) unsigned int mem_nid, cpu_nid; struct memory_locality *loc = NULL; u32 best = 0; + bool access0done = false; int i; mem_nid = pxm_to_node(target->memory_pxm); @@ -577,7 +584,11 @@ static void hmat_register_target_initiators(struct memory_target *target) if (target->processor_pxm != PXM_INVAL) { cpu_nid = pxm_to_node(target->processor_pxm); register_memory_node_under_compute_node(mem_nid, cpu_nid, 0); - return; + access0done = true; + if (node_state(cpu_nid, N_CPU)) { + register_memory_node_under_compute_node(mem_nid, cpu_nid, 1); + return; + } } if (list_empty(&localities)) @@ -591,6 +602,40 @@ static void hmat_register_target_initiators(struct memory_target *target) */ bitmap_zero(p_nodes, MAX_NUMNODES); list_sort(p_nodes, &initiators, initiator_cmp); + if (!access0done) { + for (i = WRITE_LATENCY; i <= READ_BANDWIDTH; i++) { + loc = localities_types[i]; + if (!loc) + continue; + + best = 0; + list_for_each_entry(initiator, &initiators, node) { + u32 value; + + if (!test_bit(initiator->processor_pxm, p_nodes)) + continue; + + value = hmat_initiator_perf(target, initiator, + loc->hmat_loc); + if (hmat_update_best(loc->hmat_loc->data_type, value, &best)) + bitmap_clear(p_nodes, 0, initiator->processor_pxm); + if (value != best) + clear_bit(initiator->processor_pxm, p_nodes); + } + if (best) + hmat_update_target_access(target, loc->hmat_loc->data_type, best, 0); + } + + for_each_set_bit(i, p_nodes, MAX_NUMNODES) { + cpu_nid = pxm_to_node(i); + register_memory_node_under_compute_node(mem_nid, cpu_nid, 0); + } + } + + /* Access 1 ignores Generic Initiators */ + bitmap_zero(p_nodes, MAX_NUMNODES); + list_sort(p_nodes, &initiators, initiator_cmp); + best = 0; for (i = WRITE_LATENCY; i <= READ_BANDWIDTH; i++) { loc = localities_types[i]; if (!loc) @@ -600,6 +645,10 @@ static void hmat_register_target_initiators(struct memory_target *target) list_for_each_entry(initiator, &initiators, node) { u32 value; + if (!initiator->has_cpu) { + clear_bit(initiator->processor_pxm, p_nodes); + continue; + } if (!test_bit(initiator->processor_pxm, p_nodes)) continue; @@ -610,12 +659,11 @@ static void hmat_register_target_initiators(struct memory_target *target) clear_bit(initiator->processor_pxm, p_nodes); } if (best) - hmat_update_target_access(target, loc->hmat_loc->data_type, best); + hmat_update_target_access(target, loc->hmat_loc->data_type, best, 1); } - for_each_set_bit(i, p_nodes, MAX_NUMNODES) { cpu_nid = pxm_to_node(i); - register_memory_node_under_compute_node(mem_nid, cpu_nid, 0); + register_memory_node_under_compute_node(mem_nid, cpu_nid, 1); } } @@ -628,10 +676,10 @@ static void hmat_register_target_cache(struct memory_target *target) node_add_cache(mem_nid, &tcache->cache_attrs); } -static void hmat_register_target_perf(struct memory_target *target) +static void hmat_register_target_perf(struct memory_target *target, int access) { unsigned mem_nid = pxm_to_node(target->memory_pxm); - node_set_perf_attrs(mem_nid, &target->hmem_attrs, 0); + node_set_perf_attrs(mem_nid, &target->hmem_attrs[access], access); } static void hmat_register_target_device(struct memory_target *target, @@ -733,7 +781,8 @@ static void hmat_register_target(struct memory_target *target) if (!target->registered) { hmat_register_target_initiators(target); hmat_register_target_cache(target); - hmat_register_target_perf(target); + hmat_register_target_perf(target, 0); + hmat_register_target_perf(target, 1); target->registered = true; } mutex_unlock(&target_lock);