From patchwork Fri May 20 08:48:03 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi He X-Patchwork-Id: 68228 Delivered-To: patch@linaro.org Received: by 10.140.92.199 with SMTP id b65csp3652635qge; Fri, 20 May 2016 01:52:19 -0700 (PDT) X-Received: by 10.55.119.66 with SMTP id s63mr1838883qkc.169.1463734339085; Fri, 20 May 2016 01:52:19 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id c128si16978926qkd.12.2016.05.20.01.52.18; Fri, 20 May 2016 01:52:19 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B8FFF616DA; Fri, 20 May 2016 08:52:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 752D0616AB; Fri, 20 May 2016 08:52:13 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 92A5A616D9; Fri, 20 May 2016 08:52:07 +0000 (UTC) Received: from mail-pf0-f170.google.com (mail-pf0-f170.google.com [209.85.192.170]) by lists.linaro.org (Postfix) with ESMTPS id EC81F616DE for ; Fri, 20 May 2016 08:49:02 +0000 (UTC) Received: by mail-pf0-f170.google.com with SMTP id b66so18535323pfb.2 for ; Fri, 20 May 2016 01:49:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9CeF9PAvnI1Nqwbj6fstuD43k64OxosC2kDio5/bLhA=; b=XYaW0cFvtkrZLjCsSfBL3MBjs+SxfkUitvIGCc2QY0DzXV+Nswn8pdh93/oX/bxRRG 5OKvqZUGj8K+0fuzTVzAHKbyfUQ2UB5By9cp2pzJFAtWhTwQLEVSNYh6WBm7733aeBDP ZMDPZ6UOxNvux4tx8L+ueojhhYZjDwURAA+jX5L8/c+aPYxGLohsBBFKY4MOKV5a/zjw P1dR4PNqQbBh+To+ZqeQjcXKZDGr5d/2UA1sw2ffocFw7Gf91esfwfdR6EEOKmoCcMM7 R4FkNRyxEFTMdggVZxw33DDdaBx2KdWdCrEHsplUSso2a160/ER7fand+6Oc8zbvs01Z PYbQ== X-Gm-Message-State: AOPr4FXqZ0RJ57UH0rIpCM97liDkY93ZqOaVp87eSRK5BjGkJcLN3atDgnmVCktRPhkRhhUgc5k= X-Received: by 10.98.100.132 with SMTP id y126mr2898369pfb.101.1463734142098; Fri, 20 May 2016 01:49:02 -0700 (PDT) Received: from ubuntu.heyii.co (ubuntu.heyii.co. [45.32.66.203]) by smtp.googlemail.com with ESMTPSA id yp9sm25391988pab.42.2016.05.20.01.49.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 20 May 2016 01:49:01 -0700 (PDT) From: Yi He To: lng-odp@lists.linaro.org Date: Fri, 20 May 2016 08:48:03 +0000 Message-Id: <1463734083-18267-2-git-send-email-yi.he@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1463734083-18267-1-git-send-email-yi.he@linaro.org> References: <1463734083-18267-1-git-send-email-yi.he@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCH] helper: linux: thread and process cpu affinity APIs X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Set affinity to 1st available control cpu for all odp validation programs in odp_cunit_common library. Signed-off-by: Yi He --- helper/include/odp/helper/linux.h | 47 +++++++++++++++++++ helper/linux.c | 32 +++++++++++++ helper/test/thread.c | 76 +++++++++++++++++++++++++++++-- test/validation/common/odp_cunit_common.c | 15 ++++-- 4 files changed, 164 insertions(+), 6 deletions(-) diff --git a/helper/include/odp/helper/linux.h b/helper/include/odp/helper/linux.h index e2dca35..fa815e1 100644 --- a/helper/include/odp/helper/linux.h +++ b/helper/include/odp/helper/linux.h @@ -84,6 +84,29 @@ int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, */ void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); +/** + * Set CPU affinity of the current thread + * + * CPU affinity determines the set of CPUs on which the thread is + * eligible to run. + * + * @param cpuset A bitmask lists the affinity CPU cores + * + * @return 0 on success, -1 on failure + */ +int odph_linux_pthread_setaffinity(const odp_cpumask_t *cpuset); + +/** + * Get CPU affinity of the current thread + * + * CPU affinity determines the set of CPUs on which the thread is + * eligible to run. + * + * @param cpuset[out] A bitmask lists the affinity CPU cores + * + * @return 0 on success, -1 on failure + */ +int odph_linux_pthread_getaffinity(odp_cpumask_t *cpuset); /** * Fork a process @@ -134,6 +157,30 @@ int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); /** + * Set CPU affinity of the current process + * + * CPU affinity determines the set of CPUs on which the process is + * eligible to run. + * + * @param cpuset A bitmask lists the affinity CPU cores + * + * @return 0 on success, -1 on failure + */ +int odph_linux_process_setaffinity(const odp_cpumask_t *cpuset); + +/** + * Get CPU affinity of the current process + * + * CPU affinity determines the set of CPUs on which the process is + * eligible to run. + * + * @param cpuset[out] A bitmask lists the affinity CPU cores + * + * @return 0 on success, -1 on failure + */ +int odph_linux_process_getaffinity(odp_cpumask_t *cpuset); + +/** * @} */ diff --git a/helper/linux.c b/helper/linux.c index 24e243b..6ce7e7d 100644 --- a/helper/linux.c +++ b/helper/linux.c @@ -114,6 +114,22 @@ void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) } } +int odph_linux_pthread_setaffinity(const odp_cpumask_t *cpuset) +{ + const cpu_set_t *_cpuset = &cpuset->set; + + return (0 == pthread_setaffinity_np(pthread_self(), + sizeof(cpu_set_t), _cpuset)) ? 0 : -1; +} + +int odph_linux_pthread_getaffinity(odp_cpumask_t *cpuset) +{ + cpu_set_t *_cpuset = &cpuset->set; + + return (0 == pthread_getaffinity_np(pthread_self(), + sizeof(cpu_set_t), _cpuset)) ? 0 : -1; +} + int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, const odp_cpumask_t *mask, const odph_linux_thr_params_t *thr_params) @@ -236,3 +252,19 @@ int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) return 0; } + +int odph_linux_process_setaffinity(const odp_cpumask_t *cpuset) +{ + const cpu_set_t *_cpuset = &cpuset->set; + + return (0 == sched_setaffinity(0, /* pid zero means calling process */ + sizeof(cpu_set_t), _cpuset)) ? 0 : -1; +} + +int odph_linux_process_getaffinity(odp_cpumask_t *cpuset) +{ + cpu_set_t *_cpuset = &cpuset->set; + + return (0 == sched_getaffinity(0, /* pid zero means calling process */ + sizeof(cpu_set_t), _cpuset)) ? 0 : -1; +} diff --git a/helper/test/thread.c b/helper/test/thread.c index b290753..97b4331 100644 --- a/helper/test/thread.c +++ b/helper/test/thread.c @@ -4,19 +4,51 @@ * SPDX-License-Identifier: BSD-3-Clause */ +#include + #include #include #include #define NUMBER_WORKERS 16 + +/* delayed assertion after threads collection */ +static int worker_results[NUMBER_WORKERS]; + static void *worker_fn(void *arg TEST_UNUSED) { - /* depend on the odp helper to call odp_init_local */ + odp_cpumask_t workers, affinity; + /* save the thread result for delayed assertion */ + int *result = &worker_results[odp_cpu_id() % NUMBER_WORKERS]; + /* depend on the odp helper to call odp_init_local */ printf("Worker thread on CPU %d\n", odp_cpu_id()); - /* depend on the odp helper to call odp_term_local */ + odp_cpumask_zero(&workers); + odp_cpumask_zero(&affinity); + + odp_cpumask_default_worker(&workers, NUMBER_WORKERS); + + /* verify affinity works */ + if (odph_linux_pthread_getaffinity(&affinity) != 0) { + printf("Read worker thread affinity failed %d.\n", + odp_cpu_id()); + *result = -1; + } else if (!odp_cpumask_isset(&workers, + odp_cpumask_first(&affinity))) { + printf("Verify worker thread affinity failed %d.\n", + odp_cpu_id()); + *result = -1; + } + + /* verify API is not broken */ + if (odph_linux_pthread_setaffinity(&affinity) != 0) { + printf("Re-configure worker thread affinity failed %d.\n", + odp_cpu_id()); + *result = -1; + } + /* depend on the odp helper to call odp_term_local */ return 0; } @@ -24,7 +56,7 @@ static void *worker_fn(void *arg TEST_UNUSED) int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED) { odph_linux_pthread_t thread_tbl[NUMBER_WORKERS]; - odp_cpumask_t cpu_mask; + odp_cpumask_t cpu_mask, cpuset; int num_workers; int cpu; char cpumaskstr[ODP_CPUMASK_STR_SIZE]; @@ -41,6 +73,36 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED) exit(EXIT_FAILURE); } + /* reset all worker thread results to success */ + memset(worker_results, 0, sizeof(worker_results)); + odp_cpumask_zero(&cpu_mask); + odp_cpumask_zero(&cpuset); + + /* allocate the 1st available control cpu to main process */ + if (odp_cpumask_default_control(&cpu_mask, 1) != 1) { + LOG_ERR("Allocate main process affinity failed.\n"); + exit(EXIT_FAILURE); + } + if (odph_linux_process_setaffinity(&cpu_mask) != 0) { + LOG_ERR("Set main process affinify (%d) failed.\n", + odp_cpumask_first(&cpu_mask)); + exit(EXIT_FAILURE); + } + /* read back affinity to verify */ + if ((odph_linux_process_getaffinity(&cpuset) != 0) || + !odp_cpumask_equal(&cpu_mask, &cpuset)) { + odp_cpumask_to_str(&cpuset, + cpumaskstr, sizeof(cpumaskstr)); + + LOG_ERR("Verify main process affinity failed: " + "set(%d) read(%s).\n", + odp_cpumask_first(&cpu_mask), cpumaskstr); + exit(EXIT_FAILURE); + } + + odp_cpumask_zero(&cpuset); + odp_cpumask_zero(&cpu_mask); + /* discover how many threads this system can support */ num_workers = odp_cpumask_default_worker(&cpu_mask, NUMBER_WORKERS); if (num_workers < NUMBER_WORKERS) { @@ -73,6 +135,14 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED) odph_linux_pthread_create(&thread_tbl[0], &cpu_mask, &thr_params); odph_linux_pthread_join(thread_tbl, num_workers); + /* assert all worker thread results */ + for (cpu = 0; cpu < num_workers; cpu++) { + if (worker_results[cpu] < 0) { + LOG_ERR("Worker thread %d failed.\n", cpu); + exit(EXIT_FAILURE); + } + } + if (odp_term_local()) { LOG_ERR("Error: ODP local term failed.\n"); exit(EXIT_FAILURE); diff --git a/test/validation/common/odp_cunit_common.c b/test/validation/common/odp_cunit_common.c index 2712abe..535a9f9 100644 --- a/test/validation/common/odp_cunit_common.c +++ b/test/validation/common/odp_cunit_common.c @@ -329,9 +329,18 @@ int odp_cunit_update(odp_suiteinfo_t testsuites[]) int odp_cunit_register(odp_suiteinfo_t testsuites[]) { /* call test executable init hook, if any */ - if (global_init_term.global_init_ptr && - ((*global_init_term.global_init_ptr)(&instance) != 0)) - return -1; + if (global_init_term.global_init_ptr) { + if ((*global_init_term.global_init_ptr)(&instance) == 0) { + odp_cpumask_t cpuset; + + odp_cpumask_zero(&cpuset); + /* set main process affinity after ODP initialization */ + if (1 == odp_cpumask_default_control(&cpuset, 1)) + odph_linux_process_setaffinity(&cpuset); + } else { + return -1; + } + } CU_set_error_action(CUEA_ABORT);