From patchwork Thu Apr 23 16:31:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 185726 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp2256189ilf; Thu, 23 Apr 2020 09:32:10 -0700 (PDT) X-Google-Smtp-Source: APiQypJ2CEo3HWVDaV/+ZD99GlFmRfY8jH5p7V0TV/9gFMeMxYupjrjXvZCiUEqLXlgRFAcHxx55 X-Received: by 2002:a17:907:a89:: with SMTP id by9mr3232625ejc.289.1587659530767; Thu, 23 Apr 2020 09:32:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587659530; cv=none; d=google.com; s=arc-20160816; b=O41RTVphTwWRvaS6llCZvx1gBIFgFdNe/1nExzD0q2js/kJdNg44tDYWhaGT4A5mkJ HlLCdK/To8h/bcEGu15GxqrU35VaTITlrQ82pQ7mml9WPNvcMhY3OQCCe+LgvOak8e/K 5nhG9MSiwoLfpQ2ruEEga7YLTBGWcqgc99Hn0fN2mUTWGl+AgdPNbZl8gwrYRBXN03Vw KrT98VibyU+w60oObxStuqWbJ+QiYDLy5m2GGyXGx+3JB3UzghNvUz8RYsdwPZijr8Cc c5zpZOa/B0LQEx7py7RvZ4AKM5+QOtIMnKMIl2R3ZxVQ0IKKsueurEm64ZjL4Kxw5z7c XzhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=WkPtePAVAXarcZKboZCqxgnfN+MrlkCxUbIGhEXcwqU=; b=sE/6PmLRcgmodnO4O3A+GzDQtaJyyHymnJK3E1O2TREDrWQDex9A64FiFSBQ4VKGZK MHYaGoW6+ZSsvKhuJvap2dxmsLSdH/UZikajarSeUm76DStthwG7D1CazZGQ1huvlQFO wEVWoIbR1oKB9+fSrRYJubRewZaD6Gp4L+fAnxolxjitolhk3QWleoadIycWoHru2qIT 5HWkopRLUGRoel4if3dutttvOCiM93h8ftWRY1HrkKKoOT5u9sWnEqdTAyOOBKoY1TAc lRj3qzHRvrsmEHYUHTNgdMuYeQOafStiFa/v4IY3U44ZRaeM1bnp05ndfW8TqtFmt9RL aGGA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id e5si1538122edc.160.2020.04.23.09.32.10; Thu, 23 Apr 2020 09:32:10 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 35EE71D161; Thu, 23 Apr 2020 18:32:01 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 84B211D152; Thu, 23 Apr 2020 18:31:59 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0E67730E; Thu, 23 Apr 2020 09:31:59 -0700 (PDT) Received: from phil-VirtualBox.arm.com (A010647.Arm.com [10.170.243.60]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E1BD33F68F; Thu, 23 Apr 2020 09:31:55 -0700 (PDT) From: Phil Yang To: harry.van.haaren@intel.com, dev@dpdk.org Cc: thomas@monjalon.net, david.marchand@redhat.com, konstantin.ananyev@intel.com, jerinj@marvell.com, hemant.agrawal@nxp.com, Honnappa.Nagarahalli@arm.com, gavin.hu@arm.com, nd@arm.com, Honnappa Nagarahalli , stable@dpdk.org Date: Fri, 24 Apr 2020 00:31:17 +0800 Message-Id: <1587659482-27133-2-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587659482-27133-1-git-send-email-phil.yang@arm.com> References: <1584407863-774-8-git-send-email-phil.yang@arm.com> <1587659482-27133-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v2 1/6] service: fix race condition for MT unsafe service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Honnappa Nagarahalli The MT unsafe service might get configured to run on another core while the service is running currently. This might result in the MT unsafe service running on multiple cores simultaneously. Use 'execute_lock' always when the service is MT unsafe. Fixes: e9139a32f6e8 ("service: add function to run on app lcore") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Phil Yang --- lib/librte_eal/common/rte_service.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/lib/librte_eal/common/rte_service.c b/lib/librte_eal/common/rte_service.c index 70d17a5..b8c465e 100644 --- a/lib/librte_eal/common/rte_service.c +++ b/lib/librte_eal/common/rte_service.c @@ -50,6 +50,10 @@ struct rte_service_spec_impl { uint8_t internal_flags; /* per service statistics */ + /* Indicates how many cores the service is mapped to run on. + * It does not indicate the number of cores the service is running + * on currently. + */ rte_atomic32_t num_mapped_cores; uint64_t calls; uint64_t cycles_spent; @@ -370,12 +374,7 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, cs->service_active_on_lcore[i] = 1; - /* check do we need cmpset, if MT safe or <= 1 core - * mapped, atomic ops are not required. - */ - const int use_atomics = (service_mt_safe(s) == 0) && - (rte_atomic32_read(&s->num_mapped_cores) > 1); - if (use_atomics) { + if (service_mt_safe(s) == 0) { if (!rte_atomic32_cmpset((uint32_t *)&s->execute_lock, 0, 1)) return -EBUSY; From patchwork Thu Apr 23 16:31:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Phil Yang X-Patchwork-Id: 185727 Delivered-To: patch@linaro.org Received: by 2002:a92:3d9a:0:0:0:0:0 with SMTP id k26csp2256438ilf; Thu, 23 Apr 2020 09:32:22 -0700 (PDT) X-Google-Smtp-Source: APiQypKMMheErvGvmGYJFBJEN2XbzZyVUIbyFW73letP27afQvGbecodGOpx0ExLr61XndVVb9Wp X-Received: by 2002:a17:906:da1b:: with SMTP id fi27mr3346837ejb.194.1587659542114; Thu, 23 Apr 2020 09:32:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587659542; cv=none; d=google.com; s=arc-20160816; b=KlF1MYai0B3NbT3GAsHqTYt4u3s1vhZJSH3UMMFudlgACi9mCQuUTndktybAn36vl1 FooMH/TVLiY/rYxM+qvdt/VXDZogqvOcWG6EwrEcqB/W+XviO3wdIQjcO5vhO6aH1Qr3 y+wU26/A0zRC3RJvIeSYKZHNOzjEiTAJFPO0v4JUzULJDBY4T4YkfOywiTNgKcES8pD4 mIk4870WFk7k42QyHHYdDOTOlh4bDTDIxlmzZ9hkezlXKM1xKo1Q1rlkEcFQzEYKO20O vr3B+TpV8Ln+uZclr0YqLiNilZzKEHs+IK/PLY4atqS4o5azl7Iv+f+y2BBycM3SFkgx vewA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=npHOVHUy1XSj5DIIgseRxtbiYAWzofRFI6e1mWd9p8k=; b=P73s2jdtfKnlmXN6MCw2npDigP5J7Nd/5sHyLqvIcbC7/MsA5dRDaM7Y6pxjlu21GT wQ0A47SVZcrK0ROx05h0ofZQcLIWBB77MO2AX6sJSu3SPfB+QP+W/dYRFVpEmIDxmnkw PP2aTjswPvwAreV/Zmr0ZuWO4qAGbCpjKtwaFoahBYT3S77cYlvQ8TfPO16tedYLU49O swfcsbZsWGPDOOkVMEs9ORHoscDpXr9YMY+2UrhBGNW3jyLi224kRB7Ns3GqnfBboHdb bHi3EryTKY7w5NX+soOqf5Y27wEwnmdN4gRYp7plTsvuCFcEZHPQ/FABIudTwfg7XTBS xuqw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id p17si1559056ejd.260.2020.04.23.09.32.21; Thu, 23 Apr 2020 09:32:22 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B52311D41F; Thu, 23 Apr 2020 18:32:06 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 84D8B1C2F5; Thu, 23 Apr 2020 18:32:05 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0B88730E; Thu, 23 Apr 2020 09:32:05 -0700 (PDT) Received: from phil-VirtualBox.arm.com (A010647.Arm.com [10.170.243.60]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C77A93F68F; Thu, 23 Apr 2020 09:32:01 -0700 (PDT) From: Phil Yang To: harry.van.haaren@intel.com, dev@dpdk.org Cc: thomas@monjalon.net, david.marchand@redhat.com, konstantin.ananyev@intel.com, jerinj@marvell.com, hemant.agrawal@nxp.com, Honnappa.Nagarahalli@arm.com, gavin.hu@arm.com, nd@arm.com, Honnappa Nagarahalli , stable@dpdk.org Date: Fri, 24 Apr 2020 00:31:18 +0800 Message-Id: <1587659482-27133-3-git-send-email-phil.yang@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587659482-27133-1-git-send-email-phil.yang@arm.com> References: <1584407863-774-8-git-send-email-phil.yang@arm.com> <1587659482-27133-1-git-send-email-phil.yang@arm.com> Subject: [dpdk-dev] [PATCH v2 2/6] service: identify service running on another core correctly X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Honnappa Nagarahalli The logic to identify if the MT unsafe service is running on another core can return -EBUSY spuriously. In such cases, running the service becomes costlier than using atomic operations. Assume that the application passes the right parameters and reduces the number of instructions for all cases. Cc: stable@dpdk.org Fixes: 8d39d3e237c2 ("service: fix race in service on app lcore function") Signed-off-by: Honnappa Nagarahalli Reviewed-by: Phil Yang --- lib/librte_eal/common/rte_service.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) -- 2.7.4 diff --git a/lib/librte_eal/common/rte_service.c b/lib/librte_eal/common/rte_service.c index b8c465e..c89472b 100644 --- a/lib/librte_eal/common/rte_service.c +++ b/lib/librte_eal/common/rte_service.c @@ -360,7 +360,7 @@ rte_service_runner_do_callback(struct rte_service_spec_impl *s, /* Expects the service 's' is valid. */ static int32_t service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, - struct rte_service_spec_impl *s) + struct rte_service_spec_impl *s, uint32_t serialize_mt_unsafe) { if (!s) return -EINVAL; @@ -374,7 +374,7 @@ service_run(uint32_t i, struct core_state *cs, uint64_t service_mask, cs->service_active_on_lcore[i] = 1; - if (service_mt_safe(s) == 0) { + if ((service_mt_safe(s) == 0) && (serialize_mt_unsafe == 1)) { if (!rte_atomic32_cmpset((uint32_t *)&s->execute_lock, 0, 1)) return -EBUSY; @@ -412,24 +412,14 @@ rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); - /* Atomically add this core to the mapped cores first, then examine if - * we can run the service. This avoids a race condition between - * checking the value, and atomically adding to the mapped count. + /* Increment num_mapped_cores to indicate that the service + * is running on a core. */ - if (serialize_mt_unsafe) - rte_atomic32_inc(&s->num_mapped_cores); + rte_atomic32_inc(&s->num_mapped_cores); - if (service_mt_safe(s) == 0 && - rte_atomic32_read(&s->num_mapped_cores) > 1) { - if (serialize_mt_unsafe) - rte_atomic32_dec(&s->num_mapped_cores); - return -EBUSY; - } - - int ret = service_run(id, cs, UINT64_MAX, s); + int ret = service_run(id, cs, UINT64_MAX, s, serialize_mt_unsafe); - if (serialize_mt_unsafe) - rte_atomic32_dec(&s->num_mapped_cores); + rte_atomic32_dec(&s->num_mapped_cores); return ret; } @@ -449,7 +439,7 @@ rte_service_runner_func(void *arg) if (!service_valid(i)) continue; /* return value ignored as no change to code flow */ - service_run(i, cs, service_mask, service_get(i)); + service_run(i, cs, service_mask, service_get(i), 1); } cs->loops++;