From patchwork Thu Oct 2 12:24:41 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petri Savolainen X-Patchwork-Id: 38293 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 79DA120549 for ; Thu, 2 Oct 2014 12:25:19 +0000 (UTC) Received: by mail-la0-f69.google.com with SMTP id q1sf1112900lam.0 for ; Thu, 02 Oct 2014 05:25:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:subject :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:errors-to:sender :x-original-sender:x-original-authentication-results:mailing-list :content-type:content-transfer-encoding; bh=c2qvwgxbOgOIj644Pn9t7f/hAtha4av/PeNfe8DCPsA=; b=VAToIdB1Yz84u6PUeI9YDQgj4CxEaBT7DlIcJYXSkmkRCf41F9KlDSSlfDsVi71umB HQbrWmCyEtUeOKKmNiavzCcZ1aoHz516pE/8OACsucGadSzb9wMnTRaQZt5mv1G0b5tZ VKyxUa/MCJLOd/sdnOERlS9cIQ2kFgb6UA1Bds8TkjAwM4jEUElYV2QKp2RHyW8L+ibb gCFaJvPUvaV9a5oct+SD0LGQbgbwQiF4NvbpiUUuA8I9O3H5MJY4YAkIJqaGuSj6nJbe V0QaIKB3rT4MPSUQ2uA3f9FHy8fkf7hAJZI0KtUP82yK1hAzA9KtX7UftdSt05VHGcat SLAA== X-Gm-Message-State: ALoCoQkFW+MAiw3tZLDWfUCJH1KyX93BL0squdGRavDRhrBx7HVfmXJJMYZoxb3hmVbfJlrLOtM4 X-Received: by 10.112.182.8 with SMTP id ea8mr620999lbc.11.1412252718278; Thu, 02 Oct 2014 05:25:18 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.1.40 with SMTP id 8ls250441laj.65.gmail; Thu, 02 Oct 2014 05:25:17 -0700 (PDT) X-Received: by 10.112.7.101 with SMTP id i5mr14195113lba.57.1412252717828; Thu, 02 Oct 2014 05:25:17 -0700 (PDT) Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com [209.85.217.171]) by mx.google.com with ESMTPS id o3si6357307laj.62.2014.10.02.05.25.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 02 Oct 2014 05:25:17 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.171 as permitted sender) client-ip=209.85.217.171; Received: by mail-lb0-f171.google.com with SMTP id z12so2102570lbi.2 for ; Thu, 02 Oct 2014 05:25:17 -0700 (PDT) X-Received: by 10.112.130.226 with SMTP id oh2mr24882252lbb.100.1412252717743; Thu, 02 Oct 2014 05:25:17 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.130.169 with SMTP id of9csp42253lbb; Thu, 2 Oct 2014 05:25:16 -0700 (PDT) X-Received: by 10.229.84.133 with SMTP id j5mr80931381qcl.14.1412252716382; Thu, 02 Oct 2014 05:25:16 -0700 (PDT) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id q8si5977307qcj.35.2014.10.02.05.25.15 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 02 Oct 2014 05:25:16 -0700 (PDT) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XZfRa-00040F-8X; Thu, 02 Oct 2014 12:25:14 +0000 Received: from mail-qc0-f178.google.com ([209.85.216.178]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XZfRT-00040A-LJ for lng-odp@lists.linaro.org; Thu, 02 Oct 2014 12:25:07 +0000 Received: by mail-qc0-f178.google.com with SMTP id c9so1992372qcz.23 for ; Thu, 02 Oct 2014 05:25:02 -0700 (PDT) X-Received: by 10.140.101.9 with SMTP id t9mr43262087qge.67.1412252702426; Thu, 02 Oct 2014 05:25:02 -0700 (PDT) Received: from localhost.localdomain (ec2-23-23-178-99.compute-1.amazonaws.com. [23.23.178.99]) by mx.google.com with ESMTPSA id l9sm3084602qag.23.2014.10.02.05.25.00 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 02 Oct 2014 05:25:01 -0700 (PDT) From: Petri Savolainen To: lng-odp@lists.linaro.org Date: Thu, 2 Oct 2014 15:24:41 +0300 Message-Id: <1412252681-17915-1-git-send-email-petri.savolainen@linaro.org> X-Mailer: git-send-email 2.1.1 X-Topics: patch Subject: [lng-odp] [PATCH] Added process mode to example app X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: petri.savolainen@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.171 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 - Added an option to run odp_example as Linux processes (vs. pthreads) - Removed thread dependency from odp_local_init - Added Linux helpers for forking processes - modified odp_thread.c to allocate global variables from shared memory Signed-off-by: Petri Savolainen --- example/generator/odp_generator.c | 10 +- example/ipsec/odp_ipsec.c | 7 +- example/l2fwd/odp_l2fwd.c | 11 +- example/odp_example/odp_example.c | 184 ++++++++++++++++-------- example/packet/odp_pktio.c | 11 +- example/timer/odp_timer_test.c | 14 +- helper/include/odph_linux.h | 66 ++++++++- platform/linux-generic/include/api/odp_init.h | 4 +- platform/linux-generic/include/api/odp_thread.h | 17 +-- platform/linux-generic/include/odp_internal.h | 4 +- platform/linux-generic/odp_init.c | 14 +- platform/linux-generic/odp_linux.c | 113 +++++++++++++-- platform/linux-generic/odp_thread.c | 84 ++++++++--- 13 files changed, 401 insertions(+), 138 deletions(-) diff --git a/example/generator/odp_generator.c b/example/generator/odp_generator.c index 78d9df5..6055324 100644 --- a/example/generator/odp_generator.c +++ b/example/generator/odp_generator.c @@ -511,7 +511,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; odp_buffer_pool_t pool; - int thr_id; int num_workers; void *pool_base; int i; @@ -525,6 +524,11 @@ int main(int argc, char *argv[]) exit(EXIT_FAILURE); } + if (odp_init_local()) { + ODP_ERR("Error: ODP local init failed.\n"); + exit(EXIT_FAILURE); + } + /* init counters */ odp_atomic_init_u64(&counters.seq); odp_atomic_init_u64(&counters.ip); @@ -574,10 +578,6 @@ int main(int argc, char *argv[]) printf("First core: %i\n\n", first_core); - /* Init this thread */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); - /* Create packet pool */ shm = odp_shm_reserve("shm_packet_pool", SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0); diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c index cd94d9a..fa4100f 100644 --- a/example/ipsec/odp_ipsec.c +++ b/example/ipsec/odp_ipsec.c @@ -1166,7 +1166,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; - int thr_id; int num_workers; void *pool_base; int i; @@ -1182,8 +1181,10 @@ main(int argc, char *argv[]) } /* Init this thread */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); + if (odp_init_local()) { + ODP_ERR("Error: ODP local init failed.\n"); + exit(EXIT_FAILURE); + } /* Reserve memory for args from shared mem */ shm = odp_shm_reserve("shm_args", sizeof(args_t), ODP_CACHE_LINE_SIZE, diff --git a/example/l2fwd/odp_l2fwd.c b/example/l2fwd/odp_l2fwd.c index fb325c4..8aa0ba0 100644 --- a/example/l2fwd/odp_l2fwd.c +++ b/example/l2fwd/odp_l2fwd.c @@ -311,7 +311,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; odp_buffer_pool_t pool; - int thr_id; void *pool_base; int i; int first_core; @@ -325,6 +324,12 @@ int main(int argc, char *argv[]) exit(EXIT_FAILURE); } + /* Init this thread */ + if (odp_init_local()) { + ODP_ERR("Error: ODP local init failed.\n"); + exit(EXIT_FAILURE); + } + /* Reserve memory for args from shared mem */ shm = odp_shm_reserve("shm_args", sizeof(args_t), ODP_CACHE_LINE_SIZE, 0); @@ -374,10 +379,6 @@ int main(int argc, char *argv[]) printf("First core: %i\n\n", first_core); - /* Init this thread */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); - /* Create packet pool */ shm = odp_shm_reserve("shm_packet_pool", SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0); diff --git a/example/odp_example/odp_example.c b/example/odp_example/odp_example.c index c80dbbc..47d764e 100644 --- a/example/odp_example/odp_example.c +++ b/example/odp_example/odp_example.c @@ -46,12 +46,15 @@ typedef struct { /** Test arguments */ typedef struct { - int core_count; /**< Core count*/ + int core_count; /**< Core count */ + int proc_mode; /**< Process mode */ } test_args_t; -/** @private Barrier for test synchronisation */ -static odp_barrier_t test_barrier; +/** Test global variables */ +typedef struct { + odp_barrier_t barrier;/**< @private Barrier for test synchronisation */ +} test_globals_t; /** @@ -318,7 +321,8 @@ static int test_poll_queue(int thr, odp_buffer_pool_t msg_pool) * @return 0 if successful */ static int test_schedule_one_single(const char *str, int thr, - odp_buffer_pool_t msg_pool, int prio) + odp_buffer_pool_t msg_pool, + int prio, odp_barrier_t *barrier) { odp_buffer_t buf; odp_queue_t queue; @@ -348,7 +352,7 @@ static int test_schedule_one_single(const char *str, int thr, ns = odp_time_cycles_to_ns(cycles); tot = i; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); clear_sched_queues(); if (tot) { @@ -379,7 +383,8 @@ static int test_schedule_one_single(const char *str, int thr, * @return 0 if successful */ static int test_schedule_one_many(const char *str, int thr, - odp_buffer_pool_t msg_pool, int prio) + odp_buffer_pool_t msg_pool, + int prio, odp_barrier_t *barrier) { odp_buffer_t buf; odp_queue_t queue; @@ -412,7 +417,7 @@ static int test_schedule_one_many(const char *str, int thr, ns = odp_time_cycles_to_ns(cycles); tot = i; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); clear_sched_queues(); if (tot) { @@ -443,7 +448,8 @@ static int test_schedule_one_many(const char *str, int thr, * @return 0 if successful */ static int test_schedule_single(const char *str, int thr, - odp_buffer_pool_t msg_pool, int prio) + odp_buffer_pool_t msg_pool, + int prio, odp_barrier_t *barrier) { odp_buffer_t buf; odp_queue_t queue; @@ -490,7 +496,7 @@ static int test_schedule_single(const char *str, int thr, cycles = odp_time_diff_cycles(t1, t2); ns = odp_time_cycles_to_ns(cycles); - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); clear_sched_queues(); if (tot) { @@ -522,7 +528,8 @@ static int test_schedule_single(const char *str, int thr, * @return 0 if successful */ static int test_schedule_many(const char *str, int thr, - odp_buffer_pool_t msg_pool, int prio) + odp_buffer_pool_t msg_pool, + int prio, odp_barrier_t *barrier) { odp_buffer_t buf; odp_queue_t queue; @@ -572,7 +579,7 @@ static int test_schedule_many(const char *str, int thr, cycles = odp_time_diff_cycles(t1, t2); ns = odp_time_cycles_to_ns(cycles); - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); clear_sched_queues(); if (tot) { @@ -600,7 +607,8 @@ static int test_schedule_many(const char *str, int thr, * @return 0 if successful */ static int test_schedule_multi(const char *str, int thr, - odp_buffer_pool_t msg_pool, int prio) + odp_buffer_pool_t msg_pool, + int prio, odp_barrier_t *barrier) { odp_buffer_t buf[MULTI_BUFS_MAX]; odp_queue_t queue; @@ -682,7 +690,7 @@ static int test_schedule_multi(const char *str, int thr, cycles = odp_time_diff_cycles(t1, t2); ns = odp_time_cycles_to_ns(cycles); - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); clear_sched_queues(); if (tot) { @@ -710,18 +718,31 @@ static void *run_thread(void *arg) { int thr; odp_buffer_pool_t msg_pool; + odp_shm_t shm; + test_globals_t *globals; + odp_barrier_t *barrier; thr = odp_thread_id(); printf("Thread %i starts on core %i\n", thr, odp_thread_core()); + shm = odp_shm_lookup("test_globals"); + globals = odp_shm_addr(shm); + + if (globals == NULL) { + ODP_ERR("Shared mem lookup failed\n"); + return NULL; + } + + barrier = &globals->barrier; + /* * Test barriers back-to-back */ - odp_barrier_sync(&test_barrier); - odp_barrier_sync(&test_barrier); - odp_barrier_sync(&test_barrier); - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); + odp_barrier_sync(barrier); + odp_barrier_sync(barrier); + odp_barrier_sync(barrier); /* * Find the buffer pool @@ -733,83 +754,83 @@ static void *run_thread(void *arg) return NULL; } - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_alloc_single(thr, msg_pool)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_alloc_multi(thr, msg_pool)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_poll_queue(thr, msg_pool)) return NULL; /* Low prio */ - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_one_single("sched_one_s_lo", thr, msg_pool, - ODP_SCHED_PRIO_LOWEST)) + ODP_SCHED_PRIO_LOWEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_single("sched_____s_lo", thr, msg_pool, - ODP_SCHED_PRIO_LOWEST)) + ODP_SCHED_PRIO_LOWEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_one_many("sched_one_m_lo", thr, msg_pool, - ODP_SCHED_PRIO_LOWEST)) + ODP_SCHED_PRIO_LOWEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_many("sched_____m_lo", thr, msg_pool, - ODP_SCHED_PRIO_LOWEST)) + ODP_SCHED_PRIO_LOWEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_multi("sched_multi_lo", thr, msg_pool, - ODP_SCHED_PRIO_LOWEST)) + ODP_SCHED_PRIO_LOWEST, barrier)) return NULL; /* High prio */ - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_one_single("sched_one_s_hi", thr, msg_pool, - ODP_SCHED_PRIO_HIGHEST)) + ODP_SCHED_PRIO_HIGHEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_single("sched_____s_hi", thr, msg_pool, - ODP_SCHED_PRIO_HIGHEST)) + ODP_SCHED_PRIO_HIGHEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_one_many("sched_one_m_hi", thr, msg_pool, - ODP_SCHED_PRIO_HIGHEST)) + ODP_SCHED_PRIO_HIGHEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_many("sched_____m_hi", thr, msg_pool, - ODP_SCHED_PRIO_HIGHEST)) + ODP_SCHED_PRIO_HIGHEST, barrier)) return NULL; - odp_barrier_sync(&test_barrier); + odp_barrier_sync(barrier); if (test_schedule_multi("sched_multi_hi", thr, msg_pool, - ODP_SCHED_PRIO_HIGHEST)) + ODP_SCHED_PRIO_HIGHEST, barrier)) return NULL; @@ -884,6 +905,7 @@ static void print_usage(void) printf("Options:\n"); printf(" -c, --count core count, core IDs start from 1\n"); printf(" -h, --help this help\n"); + printf(" --proc process mode\n"); printf("\n\n"); } @@ -902,6 +924,7 @@ static void parse_args(int argc, char *argv[], test_args_t *args) static struct option longopts[] = { {"count", required_argument, NULL, 'c'}, {"help", no_argument, NULL, 'h'}, + {"proc", no_argument, NULL, 0}, {NULL, 0, NULL, 0} }; @@ -912,6 +935,10 @@ static void parse_args(int argc, char *argv[], test_args_t *args) break; /* No more options */ switch (opt) { + case 0: + args->proc_mode = 1; + break; + case 'c': args->core_count = atoi(optarg); break; @@ -935,7 +962,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; test_args_t args; - int thr_id; int num_workers; odp_buffer_pool_t pool; void *pool_base; @@ -944,16 +970,32 @@ int main(int argc, char *argv[]) int prios; int first_core; odp_shm_t shm; + test_globals_t *globals; - printf("\nODP example starts\n"); + printf("\nODP example starts\n\n"); memset(&args, 0, sizeof(args)); parse_args(argc, argv, &args); + if (args.proc_mode) + printf("Process mode\n"); + else + printf("Thread mode\n"); + memset(thread_tbl, 0, sizeof(thread_tbl)); + /* ODP global init */ if (odp_init_global()) { - printf("ODP global init failed.\n"); + ODP_ERR("ODP global init failed.\n"); + return -1; + } + + /* + * Init this thread. It makes also ODP calls when + * setting up resources for worker threads. + */ + if (odp_init_local()) { + ODP_ERR("ODP global init failed.\n"); return -1; } @@ -991,16 +1033,22 @@ int main(int argc, char *argv[]) printf("first core: %i\n", first_core); - /* - * Init this thread. It makes also ODP calls when - * setting up resources for worker threads. - */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); /* Test cycle count accuracy */ test_time(); + shm = odp_shm_reserve("test_globals", + sizeof(test_globals_t), ODP_CACHE_LINE_SIZE, 0); + + globals = odp_shm_addr(shm); + + if (globals == NULL) { + ODP_ERR("Shared memory reserve failed.\n"); + return -1; + } + + memset(globals, 0, sizeof(test_globals_t)); + /* * Create message pool */ @@ -1072,16 +1120,40 @@ int main(int argc, char *argv[]) odp_shm_print_all(); /* Barrier to sync test case execution */ - odp_barrier_init_count(&test_barrier, num_workers); + odp_barrier_init_count(&globals->barrier, num_workers); - /* Create and launch worker threads */ - odph_linux_pthread_create(thread_tbl, num_workers, first_core, - run_thread, NULL); + if (args.proc_mode) { + int ret; + odph_linux_process_t proc[MAX_WORKERS]; - /* Wait for worker threads to exit */ - odph_linux_pthread_join(thread_tbl, num_workers); + /* Fork worker processes */ + ret = odph_linux_process_fork_n(proc, num_workers, + first_core); - printf("ODP example complete\n\n"); + if (ret < 0) { + ODP_ERR("Fork workers failed %i\n", ret); + return -1; + } + + if (ret == 0) { + /* Child process */ + run_thread(NULL); + } else { + /* Parent process */ + odph_linux_process_wait_n(proc, num_workers); + printf("ODP example complete\n\n"); + } + + } else { + /* Create and launch worker threads */ + odph_linux_pthread_create(thread_tbl, num_workers, first_core, + run_thread, NULL); + + /* Wait for worker threads to terminate */ + odph_linux_pthread_join(thread_tbl, num_workers); + + printf("ODP example complete\n\n"); + } return 0; } diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c index a949a05..145ae47 100644 --- a/example/packet/odp_pktio.c +++ b/example/packet/odp_pktio.c @@ -291,7 +291,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; odp_buffer_pool_t pool; - int thr_id; int num_workers; void *pool_base; int i; @@ -305,6 +304,12 @@ int main(int argc, char *argv[]) exit(EXIT_FAILURE); } + /* Init this thread */ + if (odp_init_local()) { + ODP_ERR("Error: ODP local init failed.\n"); + exit(EXIT_FAILURE); + } + /* Reserve memory for args from shared mem */ shm = odp_shm_reserve("shm_args", sizeof(args_t), ODP_CACHE_LINE_SIZE, 0); @@ -344,10 +349,6 @@ int main(int argc, char *argv[]) printf("First core: %i\n\n", first_core); - /* Init this thread */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); - /* Create packet pool */ shm = odp_shm_reserve("shm_packet_pool", SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0); diff --git a/example/timer/odp_timer_test.c b/example/timer/odp_timer_test.c index 6e1715d..87900fc 100644 --- a/example/timer/odp_timer_test.c +++ b/example/timer/odp_timer_test.c @@ -240,7 +240,6 @@ int main(int argc, char *argv[]) { odph_linux_pthread_t thread_tbl[MAX_WORKERS]; test_args_t args; - int thr_id; int num_workers; odp_buffer_pool_t pool; void *pool_base; @@ -262,6 +261,12 @@ int main(int argc, char *argv[]) return -1; } + /* Init this thread. */ + if (odp_init_local()) { + printf("ODP local init failed.\n"); + return -1; + } + printf("\n"); printf("ODP system info\n"); printf("---------------\n"); @@ -302,13 +307,6 @@ int main(int argc, char *argv[]) printf("timeouts: %i\n", args.tmo_count); /* - * Init this thread. It makes also ODP calls when - * setting up resources for worker threads. - */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); - - /* * Create message pool */ shm = odp_shm_reserve("msg_pool", diff --git a/helper/include/odph_linux.h b/helper/include/odph_linux.h index 1ea349a..8671dc0 100644 --- a/helper/include/odph_linux.h +++ b/helper/include/odph_linux.h @@ -10,8 +10,9 @@ * * ODP Linux helper API * - * This file is an optional helper to odp.h APIs. Application can manage - * pthreads also by other means. + * This file is an optional helper to odp.h APIs. These functions are provided + * to ease common setups in a Linux system. User is free to implement the same + * setups in otherways (not via this API). */ #ifndef ODP_LINUX_H_ @@ -23,15 +24,24 @@ extern "C" { #include +#include -/** Pthread status data */ +/** Linux pthread state information */ typedef struct { - pthread_t thread; /**< @private Pthread */ - pthread_attr_t attr; /**< @private Pthread attributes */ - + pthread_t thread; /**< Pthread ID */ + pthread_attr_t attr; /**< Pthread attributes */ + int core; /**< Core ID */ } odph_linux_pthread_t; +/** Linux process state information */ +typedef struct { + pid_t pid; /**< Process ID */ + int core; /**< Core ID */ + int status; /**< Process state change status */ +} odph_linux_process_t; + + /** * Creates and launches pthreads * @@ -61,6 +71,50 @@ void odph_linux_pthread_create(odph_linux_pthread_t *thread_tbl, void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); +/** + * Fork a process + * + * Forks and sets core affinity for the child process + * + * @param proc Pointer to process state info (for output) + * @param core Destination core for the child process + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork(odph_linux_process_t *proc, int core); + + +/** + * Fork a number of processes + * + * Forks and sets core affinity for child processes + * + * @param proc_tbl Process state info table (for output) + * @param num Number of processes to create + * @param first_core Destination core for the first process + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + int num, int first_core); + + +/** + * Wait for a number of processes + * + * Waits for a number of child processes to terminate. Records process state + * change status into the process state info structure. + * + * @param proc_tbl Process state info table (previously filled by fork) + * @param num Number of processes to wait + * + * @return 0 on success, -1 on failure + */ +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); + + #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/include/api/odp_init.h b/platform/linux-generic/include/api/odp_init.h index 490324a..13c8e44 100644 --- a/platform/linux-generic/include/api/odp_init.h +++ b/platform/linux-generic/include/api/odp_init.h @@ -41,10 +41,10 @@ int odp_init_global(void); * * All threads must call this function before calling * any other ODP API functions. - * @param thr_id Thread id + * * @return 0 if successful */ -int odp_init_local(int thr_id); +int odp_init_local(void); diff --git a/platform/linux-generic/include/api/odp_thread.h b/platform/linux-generic/include/api/odp_thread.h index e8e8c8a..5567748 100644 --- a/platform/linux-generic/include/api/odp_thread.h +++ b/platform/linux-generic/include/api/odp_thread.h @@ -8,7 +8,7 @@ /** * @file * - * ODP Linux helper API + * ODP thread API */ #ifndef ODP_THREAD_H_ @@ -19,19 +19,6 @@ extern "C" { #endif - -#include - - - -/** - * Create thread id - * - * @param core Core dedicated for the thread - * @return New thread id - */ -int odp_thread_create(int core); - /** * Get thread id * @@ -41,7 +28,7 @@ int odp_thread_id(void); /** - * Get thread id + * Get core id * * @return Core id where the thread is running currently */ diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index aa79493..f8c1596 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -21,8 +21,8 @@ extern "C" { int odp_system_info_init(void); -void odp_thread_init_global(void); -void odp_thread_init_local(int thr_id); +int odp_thread_init_global(void); +int odp_thread_init_local(void); int odp_shm_init_global(void); int odp_shm_init_local(void); diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 5b7e192..55fa53a 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -11,8 +11,6 @@ int odp_init_global(void) { - odp_thread_init_global(); - odp_system_info_init(); if (odp_shm_init_global()) { @@ -20,6 +18,11 @@ int odp_init_global(void) return -1; } + if (odp_thread_init_global()) { + ODP_ERR("ODP thread init failed.\n"); + return -1; + } + if (odp_buffer_pool_init_global()) { ODP_ERR("ODP buffer pool init failed.\n"); return -1; @@ -54,9 +57,12 @@ int odp_init_global(void) } -int odp_init_local(int thr_id) +int odp_init_local(void) { - odp_thread_init_local(thr_id); + if (odp_thread_init_local()) { + ODP_ERR("ODP thread local init failed.\n"); + return -1; + } if (odp_pktio_init_local()) { ODP_ERR("ODP packet io local init failed.\n"); diff --git a/platform/linux-generic/odp_linux.c b/platform/linux-generic/odp_linux.c index 9251ec9..cba6637 100644 --- a/platform/linux-generic/odp_linux.c +++ b/platform/linux-generic/odp_linux.c @@ -8,12 +8,16 @@ #define _GNU_SOURCE #endif #include +#include +#include +#include #include #include #include #include + #include #include #include @@ -23,7 +27,6 @@ typedef struct { - int thr_id; void *(*start_routine) (void *); void *arg; @@ -35,9 +38,8 @@ static void *odp_run_start_routine(void *arg) odp_start_args_t *start_args = arg; /* ODP thread local init */ - if (odp_init_local(start_args->thr_id)) { - ODP_ERR("Local init failed for thread: %d\n", - start_args->thr_id); + if (odp_init_local()) { + ODP_ERR("Local init failed\n"); return NULL; } @@ -65,9 +67,9 @@ void odph_linux_pthread_create(odph_linux_pthread_t *thread_tbl, int num, for (i = 0; i < num; i++) { pthread_attr_init(&thread_tbl[i].attr); - CPU_ZERO(&cpu_set); - cpu = (first_core + i) % core_count; + thread_tbl[i].core = cpu; + CPU_ZERO(&cpu_set); CPU_SET(cpu, &cpu_set); pthread_attr_setaffinity_np(&thread_tbl[i].attr, @@ -81,8 +83,6 @@ void odph_linux_pthread_create(odph_linux_pthread_t *thread_tbl, int num, start_args->start_routine = start_routine; start_args->arg = arg; - start_args->thr_id = odp_thread_create(cpu); - pthread_create(&thread_tbl[i].thread, &thread_tbl[i].attr, odp_run_start_routine, start_args); } @@ -98,3 +98,100 @@ void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) pthread_join(thread_tbl[i].thread, NULL); } } + + +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + int num, int first_core) +{ + cpu_set_t cpu_set; + pid_t pid; + int core_count; + int cpu; + int i; + + memset(proc_tbl, 0, num*sizeof(odph_linux_process_t)); + + core_count = odp_sys_core_count(); + + if (first_core < 0 || first_core >= core_count) { + ODP_ERR("Bad first_core\n"); + return -1; + } + + if (num < 0 || num > core_count) { + ODP_ERR("Bad num\n"); + return -1; + } + + for (i = 0; i < num; i++) { + cpu = (first_core + i) % core_count; + pid = fork(); + + if (pid < 0) { + ODP_ERR("fork() failed\n"); + return -1; + } + + /* Parent continues to fork */ + if (pid > 0) { + proc_tbl[i].pid = pid; + proc_tbl[i].core = cpu; + continue; + } + + /* Child process */ + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { + ODP_ERR("sched_setaffinity() failed\n"); + return -2; + } + + if (odp_init_local()) { + ODP_ERR("Local init failed\n"); + return -2; + } + + return 0; + } + + return 1; +} + + +int odph_linux_process_fork(odph_linux_process_t *proc, int core) +{ + return odph_linux_process_fork_n(proc, 1, core); +} + + +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) +{ + pid_t pid; + int i, j; + int status; + + for (i = 0; i < num; i++) { + pid = wait(&status); + + if (pid < 0) { + ODP_ERR("wait() failed\n"); + return -1; + } + + for (j = 0; j < num; j++) { + if (proc_tbl[j].pid == pid) { + proc_tbl[j].status = status; + break; + } + } + + if (j == num) { + ODP_ERR("Bad pid\n"); + return -1; + } + } + + return 0; +} diff --git a/platform/linux-generic/odp_thread.c b/platform/linux-generic/odp_thread.c index eaa480e..b869b27 100644 --- a/platform/linux-generic/odp_thread.c +++ b/platform/linux-generic/odp_thread.c @@ -4,65 +4,111 @@ * SPDX-License-Identifier: BSD-3-Clause */ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif +#include + #include #include #include #include +#include +#include +#include #include #include +#include typedef struct { int thr_id; - int phys_core; + int cpu; + +} thread_state_t; + -} odp_thread_tbl_t; +typedef struct { + thread_state_t thr[ODP_CONFIG_MAX_THREADS]; + odp_atomic_int_t num; + +} thread_globals_t; /* Globals */ -static odp_thread_tbl_t odp_thread_tbl[ODP_CONFIG_MAX_THREADS]; -static odp_atomic_int_t num_threads; +static thread_globals_t *thread_globals; + /* Thread local */ -static __thread odp_thread_tbl_t *odp_this_thread; +static __thread thread_state_t *this_thread; -void odp_thread_init_global(void) +int odp_thread_init_global(void) { - memset(odp_thread_tbl, 0, sizeof(odp_thread_tbl)); - num_threads = 0; -} + odp_shm_t shm; + shm = odp_shm_reserve("odp_thread_globals", + sizeof(thread_globals_t), + ODP_CACHE_LINE_SIZE, 0); -void odp_thread_init_local(int thr_id) -{ - odp_this_thread = &odp_thread_tbl[thr_id]; + thread_globals = odp_shm_addr(shm); + + if (thread_globals == NULL) + return -1; + + memset(thread_globals, 0, sizeof(thread_globals_t)); + return 0; } -int odp_thread_create(int phys_core) +static int thread_id(void) { int id; + int cpu; + + id = odp_atomic_fetch_add_int(&thread_globals->num, 1); - id = odp_atomic_fetch_add_int(&num_threads, 1); + if (id >= ODP_CONFIG_MAX_THREADS) { + ODP_ERR("Too many threads\n"); + return -1; + } + + cpu = sched_getcpu(); - if (id < ODP_CONFIG_MAX_THREADS) { - odp_thread_tbl[id].thr_id = id; - odp_thread_tbl[id].phys_core = phys_core; + if (cpu < 0) { + ODP_ERR("getcpu failed\n"); + return -1; } + thread_globals->thr[id].thr_id = id; + thread_globals->thr[id].cpu = cpu; + return id; } +int odp_thread_init_local(void) +{ + int id; + + id = thread_id(); + + if (id < 0) + return -1; + + this_thread = &thread_globals->thr[id]; + return 0; +} + + int odp_thread_id(void) { - return odp_this_thread->thr_id; + return this_thread->thr_id; } int odp_thread_core(void) { - return odp_this_thread->phys_core; + return this_thread->cpu; }