From patchwork Wed May 29 08:44:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Richter X-Patchwork-Id: 165337 Delivered-To: patch@linaro.org Received: by 2002:a92:9e1a:0:0:0:0:0 with SMTP id q26csp9580152ili; Wed, 29 May 2019 01:44:57 -0700 (PDT) X-Google-Smtp-Source: APXvYqyO+51P6n0ox6TYfjInnxdCT7OMgZ8nzqClzjL+sel59QbD5tUEKywP67VCLDCxNEFMVMTs X-Received: by 2002:a65:638a:: with SMTP id h10mr69569911pgv.64.1559119496959; Wed, 29 May 2019 01:44:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559119496; cv=none; d=google.com; s=arc-20160816; b=VgaO5LNKqp2GYEQEdldyM5YQGbzYx2hpdFcDgwmrlw0LEtgkxeuo0FTk857NBZ96u4 myfyXE5IDDgJkag+gFuysk1N6rL6iAr0ryvxrhunq3vX3XIeZS87WYbXejRQ5dbSoG1V N8wonTAxH7e55KoeSrHh56dVJEV7homh49NIQVJQGCwhWdqVQUn43K7/P2EvQsHvevNY 4TOGVhlSp47l88CNgJiquUVRT6yet1t/4ttUZSnF1PbLAsAcafDb18GcEWaM1uR2Zhgv zO12DHGQUFRVVOzlMi2Gp24V6WGfdD6U9Ws3Uf64ZyYeVOpE/1RFbRNnbJ3qbJufKOpO hGWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:content-transfer-encoding :content-language:accept-language:in-reply-to:references:message-id :date:thread-index:thread-topic:subject:cc:to:from:dkim-signature :dkim-signature; bh=u9alLq7IezxnE8uRDb+x5yZz/pbYz8zbGCqaefQpu/A=; b=yVr3ooIDp2/ClqdMe8eDFAf3AESBWw3FeMFmF7Cm/qIvSJgo/S1ENSN9jPFMQ+1u/B OmZSCVFX/PybHMQMEEl3bY45CqcudKdq4YwugvKYGHS0seFFTfboJ4H4Z94/ir5aQiO4 7fyKAbQguDq3lRWgRwhdZ+NHXjBHhSjbEb8I8pBUPYvSWlnKnecuhoEeZi25AYRSlKm7 Zbxrp611I6gCo+1E+Gnh3GA/IhsZ9C6nEU5MAVPzEtASnY+rlGhDNKUdPAdLOFqz0Ki1 aAWikx3nIh+T5mNAUXfvSUWrKVb9KiY5CcrmO7/biVlJjzz8Ywf8MbMHDHZ/TsCSFVoN SMyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@marvell.com header.s=pfpt0818 header.b=mLhsNSPA; dkim=pass header.i=@marvell.onmicrosoft.com header.s=selector2-marvell-onmicrosoft-com header.b=wWFJSX8u; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=marvell.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n7si24246780pgq.459.2019.05.29.01.44.56; Wed, 29 May 2019 01:44:56 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@marvell.com header.s=pfpt0818 header.b=mLhsNSPA; dkim=pass header.i=@marvell.onmicrosoft.com header.s=selector2-marvell-onmicrosoft-com header.b=wWFJSX8u; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=marvell.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726911AbfE2Ioz (ORCPT + 30 others); Wed, 29 May 2019 04:44:55 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:49578 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726798AbfE2Ioq (ORCPT ); Wed, 29 May 2019 04:44:46 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x4T8do5I017769; Wed, 29 May 2019 01:44:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=u9alLq7IezxnE8uRDb+x5yZz/pbYz8zbGCqaefQpu/A=; b=mLhsNSPAVtAIOC2aVTNyV9WGnElMhrqICSX19QLcXLJi1hqAzqf9n4RS/pQOBcbctnBQ x6fjfC3qbkgY35B7Di2Z+uWi+uuIVCqJL0VzYqazj0lDt1HRuyMQUb7E8MU2SlT12WeK tdWP7VVPZMqlBlmX3ed7x45r40thJXduLnl43YoK3P1QCuFgbvy53caO4nXPQwHBOMkC zbFSBdVF3NnehKuHYoXj9N0t0yYmgpv/z/VHQ/4hYrbdK4KzN1TivAzlDWO0BW3IzT1s qPmW7+WMtIL8xy2KYJ8758fyfqpRGyi072JEZztRzSLsTwAeVQ0wLD1blBXxqLA5Mzm1 eQ== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2sskp88p6g-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 29 May 2019 01:44:38 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 29 May 2019 01:44:37 -0700 Received: from NAM03-CO1-obe.outbound.protection.outlook.com (104.47.40.53) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Wed, 29 May 2019 01:44:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u9alLq7IezxnE8uRDb+x5yZz/pbYz8zbGCqaefQpu/A=; b=wWFJSX8uN2Me5hmId/PnSgA4S3cTumVTmjJv6keiNY17EJM7p1zqCKFsKySfcrhM5Gp2LFsjwJ5IYw21Ryu9x8BvBuHHcg0wlTRprKv92b/MO5zlPIIlkgXmWDvHHJo3M+jXCyv5QzCAKOQh8TnMtUbHDfReityKHlTM6ct9L0k= Received: from MN2PR18MB3408.namprd18.prod.outlook.com (10.255.238.217) by MN2PR18MB3437.namprd18.prod.outlook.com (10.255.239.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1922.20; Wed, 29 May 2019 08:44:35 +0000 Received: from MN2PR18MB3408.namprd18.prod.outlook.com ([fe80::7c9a:f3bf:fe2e:fe4a]) by MN2PR18MB3408.namprd18.prod.outlook.com ([fe80::7c9a:f3bf:fe2e:fe4a%4]) with mapi id 15.20.1922.021; Wed, 29 May 2019 08:44:35 +0000 From: Robert Richter To: Borislav Petkov , Tony Luck , "James Morse" , Mauro Carvalho Chehab CC: "linux-edac@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Robert Richter Subject: [PATCH 14/21] EDAC, ghes: Extract numa node information for each dimm Thread-Topic: [PATCH 14/21] EDAC, ghes: Extract numa node information for each dimm Thread-Index: AQHVFfq9sPfT7Zt2PE+R3bvTGIYDig== Date: Wed, 29 May 2019 08:44:35 +0000 Message-ID: <20190529084344.28562-15-rrichter@marvell.com> References: <20190529084344.28562-1-rrichter@marvell.com> In-Reply-To: <20190529084344.28562-1-rrichter@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: AM6PR01CA0046.eurprd01.prod.exchangelabs.com (2603:10a6:20b:e0::23) To MN2PR18MB3408.namprd18.prod.outlook.com (2603:10b6:208:16c::25) x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.20.1 x-originating-ip: [78.54.13.57] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ac202288-64fa-4567-8f08-08d6e411e036 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:MN2PR18MB3437; x-ms-traffictypediagnostic: MN2PR18MB3437: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2887; x-forefront-prvs: 0052308DC6 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(396003)(376002)(39860400002)(136003)(189003)(199004)(8936002)(4326008)(53936002)(50226002)(110136005)(107886003)(68736007)(36756003)(5660300002)(54906003)(11346002)(26005)(186003)(52116002)(2906002)(2616005)(476003)(446003)(6506007)(14454004)(66476007)(25786009)(478600001)(386003)(86362001)(486006)(81166006)(81156014)(8676002)(305945005)(316002)(7736002)(14444005)(256004)(99286004)(76176011)(6512007)(6436002)(73956011)(64756008)(66446008)(66556008)(66066001)(3846002)(66946007)(1076003)(6486002)(102836004)(71190400001)(71200400001)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3437; H:MN2PR18MB3408.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: oCcwaQG/Xey/h8j1hSSXI/fQ8xAcXr2cpgAnOoNpFKvtN3pgu8FcvsH7YFIdXo0W1zgTCm7eYB+bXPsqV5lnqkDFfqTlAFOv119nG6qpScL7OIMS/QE4Ku0ITLf3avziQ8ITVfOkIR6D0U/Mp7bYLEr7QkOUheDUOBzxJMTMPYyezuFHZweW2ejBhwY/ywWf9bGnkbAwh9hG2wykC6d1GPDPLCN3w4Fw/s0UxA+NWk/1rH9CfM7mrSCawMKZ3HchXm+tQvJ1a0cZaTJ+vE4Tb8JpzyTpmSVBAyIjXoOlg9aa3RB4FJo/zZVHyUuM7juwN2xf25Y4crRhIabVN6eYeqXrvmAx8y3w+oasD7AjyPmDC6VdRpnj//OZyh+C4abObZuVJ0yPMcVuU85N8kTlzZGRbdyRjLpR8oGmYxAhkIA= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: ac202288-64fa-4567-8f08-08d6e411e036 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2019 08:44:35.0325 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rrichter@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3437 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-05-29_05:, , signatures=0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In a later patch we want to have one mc device per node. This patch extracts the numa node information for each dimm. This is done by collecting the physical address ranges from the DMI table (Memory Array Mapped Address - Type 19 of SMBIOS spec). The node information for a physical address is already know to a numa aware system (e.g. by using the ACPI _PXM method or the ACPI SRAT table), so based on the PA we can assign the node id to the dimms. A fallback that disables numa is implemented in case the node information is inconsistent. E.g., on a ThunderX2 system the following node mappings are found based on the DMI table: EDAC DEBUG: mem_info_setup: DIMM0: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM1: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM2: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM3: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM4: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM5: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM6: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM7: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM8: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM9: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM10: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM11: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM12: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM13: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM14: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM15: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 Signed-off-by: Robert Richter --- drivers/edac/ghes_edac.c | 104 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 103 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/drivers/edac/ghes_edac.c b/drivers/edac/ghes_edac.c index 50f4ee36b755..083452a48b42 100644 --- a/drivers/edac/ghes_edac.c +++ b/drivers/edac/ghes_edac.c @@ -67,14 +67,34 @@ struct memdev_dmi_entry { u16 conf_mem_clk_speed; } __attribute__((__packed__)); +/* Memory Array Mapped Address - Type 19 of SMBIOS spec */ +struct memarr_dmi_entry { + u8 type; + u8 length; + u16 handle; + u32 start; + u32 end; + u16 phys_mem_array_handle; + u8 partition_width; + u64 ext_start; + u64 ext_end; +} __attribute__((__packed__)); + struct ghes_dimm_info { struct dimm_info dimm_info; int idx; + int numa_node; + phys_addr_t start; + phys_addr_t end; + u16 phys_handle; }; struct ghes_mem_info { - int num_dimm; + int num_dimm; struct ghes_dimm_info *dimms; + int num_nodes; + int num_per_node[MAX_NUMNODES]; + bool enable_numa; }; struct ghes_mem_info mem_info; @@ -97,10 +117,50 @@ static void ghes_dimm_info_init(void) for_each_dimm(dimm) { dimm->idx = idx; + dimm->numa_node = NUMA_NO_NODE; idx++; } } +static void ghes_edac_set_nid(const struct dmi_header *dh, void *arg) +{ + struct memarr_dmi_entry *entry = (struct memarr_dmi_entry *)dh; + struct ghes_dimm_info *dimm; + phys_addr_t start, end; + int nid; + + if (dh->type != DMI_ENTRY_MEM_ARRAY_MAPPED_ADDR) + return; + + /* only support SMBIOS 2.7+ */ + if (entry->length < sizeof(*entry)) + return; + + if (entry->start == 0xffffffff) + start = entry->ext_start; + else + start = entry->start; + if (entry->end == 0xffffffff) + end = entry->ext_end; + else + end = entry->end; + + if (!pfn_valid(PHYS_PFN(start))) + return; + + nid = pfn_to_nid(PHYS_PFN(start)); + if (nid < 0 || nid >= MAX_NUMNODES || !node_possible(nid)) + nid = NUMA_NO_NODE; + + for_each_dimm(dimm) { + if (entry->phys_mem_array_handle == dimm->phys_handle) { + dimm->numa_node = nid; + dimm->start = start; + dimm->end = end; + } + } +} + static int get_dimm_smbios_index(u16 handle) { struct mem_ctl_info *mci = ghes_pvt->mci; @@ -213,8 +273,25 @@ static void ghes_edac_dmidecode(const struct dmi_header *dh, void *arg) } } +static void mem_info_disable_numa(void) +{ + struct ghes_dimm_info *dimm; + + for_each_dimm(dimm) { + if (dimm->numa_node != NUMA_NO_NODE) + mem_info.num_per_node[dimm->numa_node] = 0; + dimm->numa_node = 0; + } + + mem_info.num_per_node[0] = mem_info.num_dimm; + mem_info.num_nodes = 1; + mem_info.enable_numa = false; +} + static int mem_info_setup(void) { + struct ghes_dimm_info *dimm; + bool enable_numa = true; int idx = 0; memset(&mem_info, 0, sizeof(mem_info)); @@ -231,6 +308,29 @@ static int mem_info_setup(void) ghes_dimm_info_init(); dmi_walk(ghes_edac_dmidecode, &idx); + dmi_walk(ghes_edac_set_nid, NULL); + + for_each_dimm(dimm) { + if (dimm->numa_node == NUMA_NO_NODE) { + enable_numa = false; + } else { + if (!mem_info.num_per_node[dimm->numa_node]) + mem_info.num_nodes++; + mem_info.num_per_node[dimm->numa_node]++; + } + + edac_dbg(1, "DIMM%i: Found mem range [%pa-%pa] on node %d\n", + dimm->idx, &dimm->start, &dimm->end, dimm->numa_node); + } + + mem_info.enable_numa = enable_numa; + if (enable_numa) + return 0; + + /* something went wrong, disable numa */ + if (num_possible_nodes() > 1) + pr_warn("Can't get numa info, disabling numa\n"); + mem_info_disable_numa(); return 0; } @@ -258,6 +358,8 @@ static int mem_info_setup_fake(void) dimm->dtype = DEV_UNKNOWN; dimm->edac_mode = EDAC_SECDED; + mem_info_disable_numa(); + return 0; }