From patchwork Thu Dec 27 06:22:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hemant Agrawal X-Patchwork-Id: 154548 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp6604370ljp; Wed, 26 Dec 2018 22:23:22 -0800 (PST) X-Google-Smtp-Source: ALg8bN4PKm8bMnN9n3YR7OkJH7cjbCCJcfB6TA3RNyAfChXCRuhbD6/OqSjI/+TM/Nu3BtlNKhtO X-Received: by 2002:adf:fd87:: with SMTP id d7mr20076613wrr.74.1545891802589; Wed, 26 Dec 2018 22:23:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1545891802; cv=none; d=google.com; s=arc-20160816; b=Jp84HL+5WkZ3ECdwoTVJ+xil+MgaeG7ogpVPY9J4ZbcFuQsTg0GSI5oDz8DHe0PEzn 6H22Mjuuh00vSa2sVhD82FiZHNtO7fKkxDds5pgNaUchRLsHt0IrbBYFLHkhhwC7K2+r XrcQVMA7MJATmlI0nK2Qv/H2CCVqS0IWfslXuefRfSVwgJLLyLJ8ZDx/cM1/rHpR8rgj PJPV3ZRQ4bvccscNQONlBf1Z7OaOV7L4eCcn81+ikCqxmW4mxG8YFh5VDn3AtCl+HWpb CZJHuKtMEnfYQyNzznQ4yoPbD9eEdReyTAlvYAlhA/r1CWemoJOBg83pK9aDW/cZLMZw 7w6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version :content-transfer-encoding:spamdiagnosticmetadata :spamdiagnosticoutput:content-language:accept-language:in-reply-to :references:message-id:date:thread-index:thread-topic:cc:to:from :dkim-signature; bh=KB6Z6l9znzTi2fMkUHnDMQSkuZ0Jvn5aDoH89/6zV4g=; b=iAvXAp87pXVXDseXkV2iRE3xBBHZmXUWvMOn6h02+wCFyLroH9IKMYMjFPHOTzzr5L rx8klZuGOAB8EWGf61yOlbWpj3naL/Yn6OoLMT+PfewcMdN+m3mGrcIjiC4U9TUoHLQP 4iazbwKAuUHvU4zkU9i7qujmpEI9TF/WT6o/AVqwgrBzst23aAEhYpxzH2lMLI31aJPk mNJnaHPH7FDkEZUvfS00AqtNF5JK7vFZjRvfhsx/QVmMNqhYxWe4vuvvEogdbaTa6Hju soTcHh5AbfTAh0mj/Z0REmDAFpxI3PXNWGGXjSetW7E3bd/mR1JHroYfddpr5wSE3DnZ WLyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@nxp.com header.s=selector1 header.b=drak5m5Y; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id d63si17000477wmf.51.2018.12.26.22.23.22; Wed, 26 Dec 2018 22:23:22 -0800 (PST) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; dkim=fail header.i=@nxp.com header.s=selector1 header.b=drak5m5Y; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4ACEE4C99; Thu, 27 Dec 2018 07:23:00 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140058.outbound.protection.outlook.com [40.107.14.58]) by dpdk.org (Postfix) with ESMTP id 030E54C6C; Thu, 27 Dec 2018 07:22:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KB6Z6l9znzTi2fMkUHnDMQSkuZ0Jvn5aDoH89/6zV4g=; b=drak5m5YGjd6sRIw9r8kCBtv2f9VSoqO7uV26aksodtf2LIkbL8KpoFOdcc2o0W0+frUyExizpWt0MPG8xmz/1DlynmKMu88//oORFVJbT4dcL9g5QwOsRNEp1p4YAu0KWqYmfhXCmZoJlDtjyZ4tL90Qss6CUV8CWdWZAhINkA= Received: from VI1PR0401MB2541.eurprd04.prod.outlook.com (10.168.65.19) by VI1PR0401MB2382.eurprd04.prod.outlook.com (10.169.134.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1471.20; Thu, 27 Dec 2018 06:22:55 +0000 Received: from VI1PR0401MB2541.eurprd04.prod.outlook.com ([fe80::555e:164:f8f0:dbb4]) by VI1PR0401MB2541.eurprd04.prod.outlook.com ([fe80::555e:164:f8f0:dbb4%2]) with mapi id 15.20.1471.019; Thu, 27 Dec 2018 06:22:55 +0000 From: Hemant Agrawal To: "dev@dpdk.org" CC: "ferruh.yigit@intel.com" , Shreyansh Jain , "stable@dpdk.org" Thread-Topic: [PATCH 03/20] bus/fslmc: fix to use correct physical core for logical core Thread-Index: AQHUnaya6tXM27Eib0evvOZM5Aq7IA== Date: Thu, 27 Dec 2018 06:22:55 +0000 Message-ID: <20181227062233.30781-4-hemant.agrawal@nxp.com> References: <20181227062233.30781-1-hemant.agrawal@nxp.com> In-Reply-To: <20181227062233.30781-1-hemant.agrawal@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [14.143.30.134] x-mailer: git-send-email 2.17.1 x-clientproxiedby: BM1PR01CA0120.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:40::14) To VI1PR0401MB2541.eurprd04.prod.outlook.com (2603:10a6:800:56::19) authentication-results: spf=none (sender IP is ) smtp.mailfrom=hemant.agrawal@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0401MB2382; 6:eSbNQzB8YSPJfVQ1YaMwLK9qu2kf+TUx6ZbJzurfD/N71e/C8eITGL/szMU4ypST67xLC+jvLml8rJlqmvJcFco+DleZz7euxKrZ7ZlnbfVNaPCQJL7F4+TVx6yTA6kjIrFKaO6oJFK76AqONvxaXnMuG9vIkgzFTVP3U6qe2aui2zA/hDONV4xq9gStO07EDec2N2i/z/6GtQScy6yNNAILhF8pcDFckOhZCBQ8Hp6RbweqftcXXBhk9TSysyQv+BMayo3PIlJL8a1eUinzZ0gzllQhHRyXl+wNa5OxWrlW2asztiYdHd9Bk9dunIeH56cDRRHTdQTuEVEZrINnbGW18Kp407JpSoSoRNbS7C4iGIMntHOqncH54Xm79rwNPc7GZtR/pGdzn4khDLwICEMPEbogABY8bZaKtWF3vyM2XGp7ExYFaa2dNVKNZj5YxnR8rGu+ukKDHB3DZMvRXw==; 5:fHIf3GG8mKOdX9MootC3Hx/lDyCOOcMG/sDEAHRhigyYyGvZ3OMmgSgGIZJT9HYRbYVHTKCeTn9KHZnWITwrMKybnNv9A5sW/174mB/OYiF84+2byNVIdNRGQYpKoXt+bzRpsLETMUVy4Zo0dwxwKD4dsHbk62pduo3SUAcRK/g=; 7:IykoCms8KQEltY56zw1JfsI6+ukgoPHK4+0Vfxs8McGxDUDgAZFBcE9yfYgjYz/fXpk9UhSqb6TUlfBxBXyheISv3bJOB3x7Tll+Z0aHvPRH9ib2k1kQH5Hg3ecbTZjxNvEsysh/ByFD/Pdru0Cgzw== x-ms-office365-filtering-correlation-id: 97a621da-399c-4dad-ac96-08d66bc3bd0b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0401MB2382; x-ms-traffictypediagnostic: VI1PR0401MB2382: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(2401047)(8121501046)(10201501046)(3002001)(93006095)(93001095)(3231475)(944501520)(52105112)(6055026)(6041310)(20161123560045)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:VI1PR0401MB2382; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0401MB2382; x-forefront-prvs: 0899B47777 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(396003)(366004)(376002)(39860400002)(346002)(199004)(189003)(5660300001)(1076003)(26005)(76176011)(186003)(71200400001)(256004)(25786009)(105586002)(4326008)(6436002)(305945005)(11346002)(106356001)(446003)(44832011)(476003)(7736002)(55236004)(102836004)(78486014)(486006)(386003)(68736007)(52116002)(86362001)(6506007)(2616005)(71190400001)(66066001)(316002)(54906003)(6116002)(3846002)(2906002)(14454004)(36756003)(99286004)(50226002)(8936002)(1730700003)(81156014)(81166006)(8676002)(6512007)(53936002)(478600001)(97736004)(6486002)(6916009)(5640700003)(2351001)(2501003); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0401MB2382; H:VI1PR0401MB2541.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: s/H+tZAii537c26GTGS1NbjybhbdulP0TXpQKua/LF6UgkuI7tCARWmcyzjnAEeg7i72f+bJg6SYOBlJOBF/Vzi2X4rUpxgsyHuBJBEbGA3QBeZH4S3eCR8qH5dUvFWeyBgYlYjQArhAj99nPCHemJVQNt2WAdlfcP6mpF8b2bIcRzk+TpjcNqhWcuvpPo+/hWYiqFBUZYZ2qPs+OC0by/5pbPJi3Y1vMD9fjPuPs2LV8QIMXd9S2JkVuw4EbjMAqloRW326Ej9TCJhp2PWNQ6VYtHz1GuOfZFd+dN9tF9ZHTSg8rK/0O+05k5iXAxS/ spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97a621da-399c-4dad-ac96-08d66bc3bd0b X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Dec 2018 06:22:55.8019 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0401MB2382 Subject: [dpdk-dev] [PATCH 03/20] bus/fslmc: fix to use correct physical core for logical core X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Existing code is using the lcore id as the physical core id. Add code to get the right physical id. Also, dpaa2 can not support one lcore mapping to multiple cpus, print err on such cases. Fixes: ce9efbf5bb09 ("bus/fslmc: support dynamic logging") Cc: stable@dpdk.org Signed-off-by: Hemant Agrawal --- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 76 ++++++++++++++++++++---- 1 file changed, 63 insertions(+), 13 deletions(-) -- 2.17.1 diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c index 4fc6efec5..ba2e28ce1 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -53,6 +53,10 @@ static uint32_t io_space_count; /* Variable to store DPAA2 platform type */ uint32_t dpaa2_svr_family; +/* Physical core id for lcores running on dpaa2. */ +/* DPAA2 only support 1 lcore to 1 phy cpu mapping */ +static unsigned int dpaa2_cpu[RTE_MAX_LCORE]; + /* Variable to store DPAA2 DQRR size */ uint8_t dpaa2_dqrr_size; /* Variable to store DPAA2 EQCR size */ @@ -92,7 +96,8 @@ dpaa2_core_cluster_sdest(int cpu_id) } #ifdef RTE_LIBRTE_PMD_DPAA2_EVENTDEV -static void dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id) +static void +dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id, int lcoreid) { #define STRING_LEN 28 #define COMMAND_LEN 50 @@ -125,7 +130,7 @@ static void dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id) return; } - cpu_mask = cpu_mask << rte_lcore_id(); + cpu_mask = cpu_mask << dpaa2_cpu[lcoreid]; snprintf(command, COMMAND_LEN, "echo %X > /proc/irq/%s/smp_affinity", cpu_mask, token); ret = system(command); @@ -139,7 +144,7 @@ static void dpaa2_affine_dpio_intr_to_respective_core(int32_t dpio_id) fclose(file); } -static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev) +static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev, int lcoreid) { struct epoll_event epoll_ev; int eventfd, dpio_epoll_fd, ret; @@ -176,32 +181,36 @@ static int dpaa2_dpio_intr_init(struct dpaa2_dpio_dev *dpio_dev) } dpio_dev->epoll_fd = dpio_epoll_fd; - dpaa2_affine_dpio_intr_to_respective_core(dpio_dev->hw_id); + dpaa2_affine_dpio_intr_to_respective_core(dpio_dev->hw_id, lcoreid); return 0; } #endif static int -dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id) +dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int lcoreid) { int sdest, ret; + int cpu_id; /* Set the Stashing Destination */ - if (cpu_id < 0) { - cpu_id = rte_get_master_lcore(); - if (cpu_id < 0) { + if (lcoreid < 0) { + lcoreid = rte_get_master_lcore(); + if (lcoreid < 0) { DPAA2_BUS_ERR("Getting CPU Index failed"); return -1; } } + + cpu_id = dpaa2_cpu[lcoreid]; + /* Set the STASH Destination depending on Current CPU ID. * Valid values of SDEST are 4,5,6,7. Where, */ sdest = dpaa2_core_cluster_sdest(cpu_id); - DPAA2_BUS_DEBUG("Portal= %d CPU= %u SDEST= %d", - dpio_dev->index, cpu_id, sdest); + DPAA2_BUS_DEBUG("Portal= %d CPU= %u lcore id =%u SDEST= %d", + dpio_dev->index, cpu_id, lcoreid, sdest); ret = dpio_set_stashing_destination(dpio_dev->dpio, CMD_PRI_LOW, dpio_dev->token, sdest); @@ -211,7 +220,7 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id) } #ifdef RTE_LIBRTE_PMD_DPAA2_EVENTDEV - if (dpaa2_dpio_intr_init(dpio_dev)) { + if (dpaa2_dpio_intr_init(dpio_dev, lcoreid)) { DPAA2_BUS_ERR("Interrupt registration failed for dpio"); return -1; } @@ -220,7 +229,7 @@ dpaa2_configure_stashing(struct dpaa2_dpio_dev *dpio_dev, int cpu_id) return 0; } -struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(int cpu_id) +struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(int lcoreid) { struct dpaa2_dpio_dev *dpio_dev = NULL; int ret; @@ -236,7 +245,7 @@ struct dpaa2_dpio_dev *dpaa2_get_qbman_swp(int cpu_id) DPAA2_BUS_DEBUG("New Portal %p (%d) affined thread - %lu", dpio_dev, dpio_dev->index, syscall(SYS_gettid)); - ret = dpaa2_configure_stashing(dpio_dev, cpu_id); + ret = dpaa2_configure_stashing(dpio_dev, lcoreid); if (ret) DPAA2_BUS_ERR("dpaa2_configure_stashing failed"); @@ -340,6 +349,39 @@ dpaa2_affine_qbman_ethrx_swp(void) } } +/* + * This checks for not supported lcore mappings as well as get the physical + * cpuid for the lcore. + * one lcore can only map to 1 cpu i.e. 1@10-14 not supported. + * one cpu can be mapped to more than one lcores. + */ +static int +dpaa2_check_lcore_cpuset(void) +{ + unsigned int lcore_id, i; + int ret = 0; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) + dpaa2_cpu[lcore_id] = 0xffffffff; + + for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) { + for (i = 0; i < RTE_MAX_LCORE; i++) { + if (CPU_ISSET(i, &lcore_config[lcore_id].cpuset)) { + RTE_LOG(DEBUG, EAL, "lcore id = %u cpu=%u\n", + lcore_id, i); + if (dpaa2_cpu[lcore_id] != 0xffffffff) { + DPAA2_BUS_ERR( + "ERR:lcore map to multi-cpu not supported"); + ret = -1; + } else { + dpaa2_cpu[lcore_id] = i; + } + } + } + } + return ret; +} + static int dpaa2_create_dpio_device(int vdev_fd, struct vfio_device_info *obj_info, @@ -349,6 +391,7 @@ dpaa2_create_dpio_device(int vdev_fd, struct vfio_region_info reg_info = { .argsz = sizeof(reg_info)}; struct qbman_swp_desc p_des; struct dpio_attr attr; + static int check_lcore_cpuset; if (obj_info->num_regions < NUM_DPIO_REGIONS) { DPAA2_BUS_ERR("Not sufficient number of DPIO regions"); @@ -368,6 +411,13 @@ dpaa2_create_dpio_device(int vdev_fd, /* Using single portal for all devices */ dpio_dev->mc_portal = rte_mcp_ptr_list[MC_PORTAL_INDEX]; + if (!check_lcore_cpuset) { + check_lcore_cpuset = 1; + + if (dpaa2_check_lcore_cpuset() < 0) + goto err; + } + dpio_dev->dpio = malloc(sizeof(struct fsl_mc_io)); memset(dpio_dev->dpio, 0, sizeof(struct fsl_mc_io));