From patchwork Fri Dec 23 19:15:40 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Holmes X-Patchwork-Id: 88971 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp3472344qgi; Fri, 23 Dec 2016 11:20:06 -0800 (PST) X-Received: by 10.200.44.118 with SMTP id e51mr15412603qta.264.1482520806678; Fri, 23 Dec 2016 11:20:06 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id k30si6860326qtd.259.2016.12.23.11.20.06; Fri, 23 Dec 2016 11:20:06 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 46BE460721; Fri, 23 Dec 2016 19:20:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 0135760C6D; Fri, 23 Dec 2016 19:17:04 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B221F60C62; Fri, 23 Dec 2016 19:16:47 +0000 (UTC) Received: from mail-qt0-f169.google.com (mail-qt0-f169.google.com [209.85.216.169]) by lists.linaro.org (Postfix) with ESMTPS id 5F58B60C6B for ; Fri, 23 Dec 2016 19:15:53 +0000 (UTC) Received: by mail-qt0-f169.google.com with SMTP id k15so28624926qtg.3 for ; Fri, 23 Dec 2016 11:15:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/zdxnZH8dA542eaN3t7KtrFHdSkuSuvVezhKilxgaPI=; b=ZIctcSXMNxi3Fc3vPi7ijqnTUakIQHruXXA1ao5YivtM4kNU8/wcz9hBxMRDE6qYMp dIPQWIEtVs4yUBuZpCFrbkuMLGyFdSavCcos/GMhqFj8sXtclha3KU/Gs3zGNTSY4A7s q2Kqpkl6tD1kjJyixkZ5exyf98MXRcn5Zam+gPidr8eqyK/PWxrMBgdqZFP3oVYHINWm v46fhDVTm/fmKa20jUV7ChRMdsMLxFchPauI7qNxLN5x8QwpPlZeFEI001gvvAkFMwNz bAiB2xsHJ7UWWZPB0VnPDG93oAjO74sN34maDva87YQxAPt9eWvZHZ8PLYFOb6bs2YDC MBkQ== X-Gm-Message-State: AIkVDXLkLJvxwVGXmlKroAdSm2wHznkkdX16YMjq1pA7URG5QAJMEiSwdjdk9ZfMspCEC9kTEEc= X-Received: by 10.237.60.51 with SMTP id t48mr16043382qte.24.1482520551424; Fri, 23 Dec 2016 11:15:51 -0800 (PST) Received: from localhost (c-98-221-136-245.hsd1.nj.comcast.net. [98.221.136.245]) by smtp.gmail.com with ESMTPSA id o44sm5365876qtc.8.2016.12.23.11.15.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Dec 2016 11:15:49 -0800 (PST) From: Mike Holmes To: lng-odp@lists.linaro.org Date: Fri, 23 Dec 2016 14:15:40 -0500 Message-Id: <20161223191541.31493-4-mike.holmes@linaro.org> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161223191541.31493-1-mike.holmes@linaro.org> References: <20161223191541.31493-1-mike.holmes@linaro.org> Subject: [lng-odp] [PATCH 3/4] helper: move thread implementation under platform X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Only the portable api is built by default, use --enable-helper-extn to enable non portable APIs for a helper platform Signed-off-by: Mike Holmes --- configure.ac | 17 +- example/Makefile.inc | 2 +- helper/Makefile.am | 15 +- .../helper/platform/linux-generic/threads_extn.h | 112 ++++++++ helper/include/odp/helper/threads.h | 76 ----- helper/m4/configure.m4 | 11 + helper/odph_thread_internal.h | 27 ++ helper/platform/linux-generic/thread.c | 256 +++++++++++++++++ helper/test/Makefile.am | 17 +- helper/test/linux-generic/Makefile.am | 5 + helper/test/{ => linux-generic}/process.c | 2 +- helper/test/{ => linux-generic}/thread.c | 2 +- helper/threads.c | 314 ++++----------------- ...inux.pc.in => libodphelper-linux-generic.pc.in} | 4 +- test/Makefile.inc | 2 +- test/common_plat/validation/api/Makefile.inc | 2 +- test/linux-generic/Makefile.inc | 2 +- 17 files changed, 515 insertions(+), 351 deletions(-) create mode 100644 helper/include/odp/helper/platform/linux-generic/threads_extn.h create mode 100644 helper/odph_thread_internal.h create mode 100644 helper/platform/linux-generic/thread.c create mode 100644 helper/test/linux-generic/Makefile.am rename helper/test/{ => linux-generic}/process.c (97%) rename helper/test/{ => linux-generic}/thread.c (97%) rename pkgconfig/{libodphelper-linux.pc.in => libodphelper-linux-generic.pc.in} (72%) -- 2.9.3 diff --git a/configure.ac b/configure.ac index 4fec9d5..001cab9 100644 --- a/configure.ac +++ b/configure.ac @@ -138,6 +138,18 @@ AC_SUBST([with_platform]) AC_SUBST([platform_with_platform], ["platform/${with_platform}"]) ########################################################################## +# Determine which helper platform to build for +########################################################################## +AC_ARG_WITH([helper_platform], + [AS_HELP_STRING([--with-helper_platform=platform], + [select helper platform to be used, default linux-generic])], + [], + [with_helper_platform=${with_platform} + ]) + +AC_SUBST([with_helper_platform]) + +########################################################################## # Run platform specific checks and settings ########################################################################## IMPLEMENTATION_NAME="" @@ -202,6 +214,7 @@ AM_CONDITIONAL([test_example], [test x$test_example = xyes ]) AM_CONDITIONAL([HAVE_DOXYGEN], [test "x${DOXYGEN}" = "xdoxygen"]) AM_CONDITIONAL([user_guide], [test "x${user_guides}" = "xyes" ]) AM_CONDITIONAL([HAVE_MSCGEN], [test "x${MSCGEN}" = "xmscgen"]) +AM_CONDITIONAL([helper_ext], [test x$helper_ext = xyes ]) ########################################################################## # Setup doxygen documentation @@ -293,7 +306,7 @@ AM_CXXFLAGS="-std=c++11" AC_CONFIG_FILES([Makefile pkgconfig/libodp-linux.pc - pkgconfig/libodphelper-linux.pc + pkgconfig/libodphelper-linux-generic.pc ]) AC_SEARCH_LIBS([timer_create],[rt posix4]) @@ -320,6 +333,8 @@ AC_MSG_RESULT([ implementation_name: ${IMPLEMENTATION_NAME} ARCH_DIR ${ARCH_DIR} with_platform: ${with_platform} + with_helper_platform: ${with_helper_platform} + helper_ext: ${helper_ext} prefix: ${prefix} sysconfdir: ${sysconfdir} libdir: ${libdir} diff --git a/example/Makefile.inc b/example/Makefile.inc index 19d3994..ea596d5 100644 --- a/example/Makefile.inc +++ b/example/Makefile.inc @@ -1,6 +1,6 @@ include $(top_srcdir)/platform/@with_platform@/Makefile.inc LIB = $(top_builddir)/lib -LDADD = $(LIB)/libodp-linux.la $(LIB)/libodphelper-linux.la +LDADD = $(LIB)/libodp-linux.la $(LIB)/libodphelper-@with_helper_platform@.la AM_CFLAGS += \ -I$(srcdir) \ -I$(top_srcdir)/example \ diff --git a/helper/Makefile.am b/helper/Makefile.am index 71c6975..66d60dd 100644 --- a/helper/Makefile.am +++ b/helper/Makefile.am @@ -1,7 +1,7 @@ include $(top_srcdir)/platform/@with_platform@/Makefile.inc pkgconfigdir = $(libdir)/pkgconfig -pkgconfig_DATA = $(top_builddir)/pkgconfig/libodphelper-linux.pc +pkgconfig_DATA = $(top_builddir)/pkgconfig/libodphelper-@with_helper_platform@.pc LIB = $(top_builddir)/lib AM_CFLAGS = -I$(srcdir)/include @@ -25,18 +25,25 @@ helperinclude_HEADERS = \ $(srcdir)/include/odp/helper/table.h\ $(srcdir)/include/odp/helper/udp.h +if helper_ext +helperinclude_HEADERS += \ + $(srcdir)/include/odp/helper/platform/@with_helper_platform@/threads_extn.h +endif + noinst_HEADERS = \ $(srcdir)/odph_debug.h \ $(srcdir)/odph_hashtable.h \ $(srcdir)/odph_lineartable.h \ - $(srcdir)/odph_list_internal.h + $(srcdir)/odph_list_internal.h \ + $(srcdir)/odph_thread_internal.h -__LIB__libodphelper_linux_la_SOURCES = \ +__LIB__libodphelper_@with_helper_platform@_la_SOURCES = \ eth.c \ ip.c \ chksum.c \ threads.c \ + platform/@with_helper_platform@/thread.c \ hashtable.c \ lineartable.c -lib_LTLIBRARIES = $(LIB)/libodphelper-linux.la +lib_LTLIBRARIES = $(LIB)/libodphelper-@with_helper_platform@.la diff --git a/helper/include/odp/helper/platform/linux-generic/threads_extn.h b/helper/include/odp/helper/platform/linux-generic/threads_extn.h new file mode 100644 index 0000000..1d4036d --- /dev/null +++ b/helper/include/odp/helper/platform/linux-generic/threads_extn.h @@ -0,0 +1,112 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP Linux helper extension API + * + * This file is an optional helper to odp.h APIs. These functions are provided + * to ease common setups in a Linux system. User is free to implement the same + * setups in otherways (not via this API). + */ + +#ifndef ODPH_LINUX_EXT_H_ +#define ODPH_LINUX_EXT_H_ + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** @addtogroup odph_linux ODPH LINUX + * @{ + */ + +/** + * Creates and launches pthreads + * + * Creates, pins and launches threads to separate CPU's based on the cpumask. + * + * @param[out] pthread_tbl Table of pthread state information records. Table + * must have at least as many entries as there are + * CPUs in the CPU mask. + * @param mask CPU mask + * @param thr_params Linux helper thread parameters + * + * @return Number of threads created + */ +int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params); + +/** + * Waits pthreads to exit + * + * Returns when all threads have been exit. + * + * @param thread_tbl Thread table + * @param num Number of threads to create + * + */ +void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); + +/** + * Fork a process + * + * Forks and sets CPU affinity for the child process. Ignores 'start' and 'arg' + * thread parameters. + * + * @param[out] proc Pointer to process state info (for output) + * @param cpu Destination CPU for the child process + * @param thr_params Linux helper thread parameters + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, + const odph_linux_thr_params_t *thr_params); + +/** + * Fork a number of processes + * + * Forks and sets CPU affinity for child processes. Ignores 'start' and 'arg' + * thread parameters. + * + * @param[out] proc_tbl Process state info table (for output) + * @param mask CPU mask of processes to create + * @param thr_params Linux helper thread parameters + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params); + +/** + * Wait for a number of processes + * + * Waits for a number of child processes to terminate. Records process state + * change status into the process state info structure. + * + * @param proc_tbl Process state info table (previously filled by fork) + * @param num Number of processes to wait + * + * @return 0 on success, -1 on failure + */ +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); + +/** + * @} + */ + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/helper/include/odp/helper/threads.h b/helper/include/odp/helper/threads.h index b8d975a..5682bab 100644 --- a/helper/include/odp/helper/threads.h +++ b/helper/include/odp/helper/threads.h @@ -93,82 +93,6 @@ typedef struct { } odph_odpthread_t; /** - * Creates and launches pthreads - * - * Creates, pins and launches threads to separate CPU's based on the cpumask. - * - * @param[out] pthread_tbl Table of pthread state information records. Table - * must have at least as many entries as there are - * CPUs in the CPU mask. - * @param mask CPU mask - * @param thr_params Linux helper thread parameters - * - * @return Number of threads created - */ -int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params); - -/** - * Waits pthreads to exit - * - * Returns when all threads have been exit. - * - * @param thread_tbl Thread table - * @param num Number of threads to create - * - */ -void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); - -/** - * Fork a process - * - * Forks and sets CPU affinity for the child process. Ignores 'start' and 'arg' - * thread parameters. - * - * @param[out] proc Pointer to process state info (for output) - * @param cpu Destination CPU for the child process - * @param thr_params Linux helper thread parameters - * - * @return On success: 1 for the parent, 0 for the child - * On failure: -1 for the parent, -2 for the child - */ -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, - const odph_linux_thr_params_t *thr_params); - - -/** - * Fork a number of processes - * - * Forks and sets CPU affinity for child processes. Ignores 'start' and 'arg' - * thread parameters. - * - * @param[out] proc_tbl Process state info table (for output) - * @param mask CPU mask of processes to create - * @param thr_params Linux helper thread parameters - * - * @return On success: 1 for the parent, 0 for the child - * On failure: -1 for the parent, -2 for the child - */ -int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params); - - -/** - * Wait for a number of processes - * - * Waits for a number of child processes to terminate. Records process state - * change status into the process state info structure. - * - * @param proc_tbl Process state info table (previously filled by fork) - * @param num Number of processes to wait - * - * @return 0 on success, -1 on failure - */ -int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); - -/** * Creates and launches odpthreads (as linux threads or processes) * * Creates, pins and launches threads to separate CPU's based on the cpumask. diff --git a/helper/m4/configure.m4 b/helper/m4/configure.m4 index b8a21f7..8f1c929 100644 --- a/helper/m4/configure.m4 +++ b/helper/m4/configure.m4 @@ -8,5 +8,16 @@ AC_ARG_ENABLE([test-helper], test_helper=yes fi]) +########################################################################## +# Enable/disable helper-ext +# platform specific non portable extensions +########################################################################## +helper_ext=no +AC_ARG_ENABLE([helper-ext], + [ --enable-helper-ext build helper platform extensions (not portable)], + [if test "x$enableval" = "xyes"; then + helper_ext=yes + fi]) + AC_CONFIG_FILES([helper/Makefile helper/test/Makefile]) diff --git a/helper/odph_thread_internal.h b/helper/odph_thread_internal.h new file mode 100644 index 0000000..24d5e09 --- /dev/null +++ b/helper/odph_thread_internal.h @@ -0,0 +1,27 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP Helper internal thread APIs + */ + +#ifndef ODPH_THREAD_INTERNAL_H_ +#define ODPH_THREAD_INTERNAL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +void *_odph_run_start_routine(void *arg); +void *_odph_thread_run_start_routine(void *arg); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/helper/platform/linux-generic/thread.c b/helper/platform/linux-generic/thread.c new file mode 100644 index 0000000..b6424a7 --- /dev/null +++ b/helper/platform/linux-generic/thread.c @@ -0,0 +1,256 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include +#include "odph_debug.h" + +int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params) +{ + int i; + int num; + int cpu_count; + int cpu; + int ret; + + num = odp_cpumask_count(mask); + + memset(pthread_tbl, 0, num * sizeof(odph_linux_pthread_t)); + + cpu_count = odp_cpu_count(); + + if (num < 1 || num > cpu_count) { + ODPH_ERR("Invalid number of threads:%d (%d cores available)\n", + num, cpu_count); + return 0; + } + + cpu = odp_cpumask_first(mask); + for (i = 0; i < num; i++) { + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pthread_attr_init(&pthread_tbl[i].attr); + + pthread_tbl[i].cpu = cpu; + + pthread_attr_setaffinity_np(&pthread_tbl[i].attr, + sizeof(cpu_set_t), &cpu_set); + + pthread_tbl[i].thr_params.start = thr_params->start; + pthread_tbl[i].thr_params.arg = thr_params->arg; + pthread_tbl[i].thr_params.thr_type = thr_params->thr_type; + pthread_tbl[i].thr_params.instance = thr_params->instance; + + ret = pthread_create(&pthread_tbl[i].thread, + &pthread_tbl[i].attr, + _odph_run_start_routine, + &pthread_tbl[i].thr_params); + if (ret != 0) { + ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); + break; + } + + cpu = odp_cpumask_next(mask, cpu); + } + + return i; +} + +void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) +{ + int i; + int ret; + + for (i = 0; i < num; i++) { + /* Wait thread to exit */ + ret = pthread_join(thread_tbl[i].thread, NULL); + if (ret != 0) { + ODPH_ERR("Failed to join thread from cpu #%d\n", + thread_tbl[i].cpu); + } + pthread_attr_destroy(&thread_tbl[i].attr); + } +} + +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params) +{ + pid_t pid; + int num; + int cpu_count; + int cpu; + int i; + + num = odp_cpumask_count(mask); + + memset(proc_tbl, 0, num * sizeof(odph_linux_process_t)); + + cpu_count = odp_cpu_count(); + + if (num < 1 || num > cpu_count) { + ODPH_ERR("Bad num\n"); + return -1; + } + + cpu = odp_cpumask_first(mask); + for (i = 0; i < num; i++) { + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pid = fork(); + + if (pid < 0) { + ODPH_ERR("fork() failed\n"); + return -1; + } + + /* Parent continues to fork */ + if (pid > 0) { + proc_tbl[i].pid = pid; + proc_tbl[i].cpu = cpu; + + cpu = odp_cpumask_next(mask, cpu); + continue; + } + + /* Child process */ + + /* Request SIGTERM if parent dies */ + prctl(PR_SET_PDEATHSIG, SIGTERM); + /* Parent died already? */ + if (getppid() == 1) + kill(getpid(), SIGTERM); + + if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { + ODPH_ERR("sched_setaffinity() failed\n"); + return -2; + } + + if (odp_init_local(thr_params->instance, + thr_params->thr_type)) { + ODPH_ERR("Local init failed\n"); + return -2; + } + + return 0; + } + + return 1; +} + +int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, + const odph_linux_thr_params_t *thr_params) +{ + odp_cpumask_t mask; + + odp_cpumask_zero(&mask); + odp_cpumask_set(&mask, cpu); + return odph_linux_process_fork_n(proc, &mask, thr_params); +} + +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) +{ + pid_t pid; + int i, j; + int status = 0; + + for (i = 0; i < num; i++) { + pid = wait(&status); + + if (pid < 0) { + ODPH_ERR("wait() failed\n"); + return -1; + } + + for (j = 0; j < num; j++) { + if (proc_tbl[j].pid == pid) { + proc_tbl[j].status = status; + break; + } + } + + if (j == num) { + ODPH_ERR("Bad pid:%d\n", (int)pid); + return -1; + } + + /* Examine the child process' termination status */ + if (WIFEXITED(status) && WEXITSTATUS(status) != EXIT_SUCCESS) { + ODPH_ERR("Child exit status:%d (pid:%d)\n", + WEXITSTATUS(status), (int)pid); + return -1; + } + if (WIFSIGNALED(status)) { + int signo = WTERMSIG(status); + + ODPH_ERR("Child term signo:%d - %s (pid:%d)\n", + signo, strsignal(signo), (int)pid); + return -1; + } + } + + return 0; +} + +/* + * Create a single ODPthread as a linux thread + */ +static int odph_linux_thread_create(odph_odpthread_t *thread_tbl, + int cpu, + const odph_odpthread_params_t *thr_params) +{ + int ret; + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pthread_attr_init(&thread_tbl->thread.attr); + + thread_tbl->cpu = cpu; + + pthread_attr_setaffinity_np(&thread_tbl->thread.attr, + sizeof(cpu_set_t), &cpu_set); + + thread_tbl->start_args.thr_params = *thr_params; /* copy */ + thread_tbl->start_args.linuxtype = ODPTHREAD_PTHREAD; + + ret = pthread_create(&thread_tbl->thread.thread_id, + &thread_tbl->thread.attr, + _odph_thread_run_start_routine, + &thread_tbl->start_args); + if (ret != 0) { + ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); + thread_tbl->start_args.linuxtype = ODPTHREAD_NOT_STARTED; + return ret; + } + + return 0; +} diff --git a/helper/test/Makefile.am b/helper/test/Makefile.am index 545db73..47dcb86 100644 --- a/helper/test/Makefile.am +++ b/helper/test/Makefile.am @@ -6,11 +6,18 @@ AM_LDFLAGS += -static TESTS_ENVIRONMENT += TEST_DIR=${builddir} EXECUTABLES = chksum$(EXEEXT) \ - thread$(EXEEXT) \ parse$(EXEEXT)\ - process$(EXEEXT)\ table$(EXEEXT) +#These are platform specific extensions that are not portable +#They are a convenience to app writers who have chosen to +#restrict their application to Linux. + +if helper_ext +EXECUTABLES += @with_helper_platform@/thread$(EXEEXT) \ + @with_helper_platform@/process$(EXEEXT) +endif + COMPILE_ONLY = odpthreads TESTSCRIPTS = odpthreads_as_processes \ @@ -28,10 +35,6 @@ EXTRA_DIST = odpthreads_as_processes odpthreads_as_pthreads dist_chksum_SOURCES = chksum.c dist_odpthreads_SOURCES = odpthreads.c -odpthreads_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la -dist_thread_SOURCES = thread.c -thread_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la -dist_process_SOURCES = process.c +odpthreads_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la dist_parse_SOURCES = parse.c -process_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la dist_table_SOURCES = table.c diff --git a/helper/test/linux-generic/Makefile.am b/helper/test/linux-generic/Makefile.am new file mode 100644 index 0000000..28d54a8 --- /dev/null +++ b/helper/test/linux-generic/Makefile.am @@ -0,0 +1,5 @@ + +thread_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la +dist_thread_SOURCES = thread.c +dist_process_SOURCES = process.c +process_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la diff --git a/helper/test/process.c b/helper/test/linux-generic/process.c similarity index 97% rename from helper/test/process.c rename to helper/test/linux-generic/process.c index f3c6d50..f641128 100644 --- a/helper/test/process.c +++ b/helper/test/linux-generic/process.c @@ -6,7 +6,7 @@ #include #include -#include +#include #define NUMBER_WORKERS 16 /* 0 = max */ diff --git a/helper/test/thread.c b/helper/test/linux-generic/thread.c similarity index 97% rename from helper/test/thread.c rename to helper/test/linux-generic/thread.c index da94b49..f1ca1b7 100644 --- a/helper/test/thread.c +++ b/helper/test/linux-generic/thread.c @@ -6,7 +6,7 @@ #include #include -#include +#include #define NUMBER_WORKERS 16 static void *worker_fn(void *arg TEST_UNUSED) diff --git a/helper/threads.c b/helper/threads.c index d5215c2..edfbe0e 100644 --- a/helper/threads.c +++ b/helper/threads.c @@ -9,25 +9,24 @@ #endif #include #include -#include #include #include #include -#include -#include -#include -#include - #include #include #include "odph_debug.h" +static int _odph_linux_process_create(odph_odpthread_t *thread_tbl, + int cpu, + const odph_odpthread_params_t *thr_params + ); + static struct { int proc; /* true when process mode is required, false otherwise */ } helper_options; -static void *odp_run_start_routine(void *arg) +void *_odph_run_start_routine(void *arg) { odph_linux_thr_params_t *thr_params = arg; @@ -48,206 +47,11 @@ static void *odp_run_start_routine(void *arg) return ret_ptr; } -int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params) -{ - int i; - int num; - int cpu_count; - int cpu; - int ret; - - num = odp_cpumask_count(mask); - - memset(pthread_tbl, 0, num * sizeof(odph_linux_pthread_t)); - - cpu_count = odp_cpu_count(); - - if (num < 1 || num > cpu_count) { - ODPH_ERR("Invalid number of threads:%d (%d cores available)\n", - num, cpu_count); - return 0; - } - - cpu = odp_cpumask_first(mask); - for (i = 0; i < num; i++) { - cpu_set_t cpu_set; - - CPU_ZERO(&cpu_set); - CPU_SET(cpu, &cpu_set); - - pthread_attr_init(&pthread_tbl[i].attr); - - pthread_tbl[i].cpu = cpu; - - pthread_attr_setaffinity_np(&pthread_tbl[i].attr, - sizeof(cpu_set_t), &cpu_set); - - pthread_tbl[i].thr_params.start = thr_params->start; - pthread_tbl[i].thr_params.arg = thr_params->arg; - pthread_tbl[i].thr_params.thr_type = thr_params->thr_type; - pthread_tbl[i].thr_params.instance = thr_params->instance; - - ret = pthread_create(&pthread_tbl[i].thread, - &pthread_tbl[i].attr, - odp_run_start_routine, - &pthread_tbl[i].thr_params); - if (ret != 0) { - ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); - break; - } - - cpu = odp_cpumask_next(mask, cpu); - } - - return i; -} - -void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) -{ - int i; - int ret; - - for (i = 0; i < num; i++) { - /* Wait thread to exit */ - ret = pthread_join(thread_tbl[i].thread, NULL); - if (ret != 0) { - ODPH_ERR("Failed to join thread from cpu #%d\n", - thread_tbl[i].cpu); - } - pthread_attr_destroy(&thread_tbl[i].attr); - } -} - -int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params) -{ - pid_t pid; - int num; - int cpu_count; - int cpu; - int i; - - num = odp_cpumask_count(mask); - - memset(proc_tbl, 0, num * sizeof(odph_linux_process_t)); - - cpu_count = odp_cpu_count(); - - if (num < 1 || num > cpu_count) { - ODPH_ERR("Bad num\n"); - return -1; - } - - cpu = odp_cpumask_first(mask); - for (i = 0; i < num; i++) { - cpu_set_t cpu_set; - - CPU_ZERO(&cpu_set); - CPU_SET(cpu, &cpu_set); - - pid = fork(); - - if (pid < 0) { - ODPH_ERR("fork() failed\n"); - return -1; - } - - /* Parent continues to fork */ - if (pid > 0) { - proc_tbl[i].pid = pid; - proc_tbl[i].cpu = cpu; - - cpu = odp_cpumask_next(mask, cpu); - continue; - } - - /* Child process */ - - /* Request SIGTERM if parent dies */ - prctl(PR_SET_PDEATHSIG, SIGTERM); - /* Parent died already? */ - if (getppid() == 1) - kill(getpid(), SIGTERM); - - if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { - ODPH_ERR("sched_setaffinity() failed\n"); - return -2; - } - - if (odp_init_local(thr_params->instance, - thr_params->thr_type)) { - ODPH_ERR("Local init failed\n"); - return -2; - } - - return 0; - } - - return 1; -} - -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, - const odph_linux_thr_params_t *thr_params) -{ - odp_cpumask_t mask; - - odp_cpumask_zero(&mask); - odp_cpumask_set(&mask, cpu); - return odph_linux_process_fork_n(proc, &mask, thr_params); -} - -int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) -{ - pid_t pid; - int i, j; - int status = 0; - - for (i = 0; i < num; i++) { - pid = wait(&status); - - if (pid < 0) { - ODPH_ERR("wait() failed\n"); - return -1; - } - - for (j = 0; j < num; j++) { - if (proc_tbl[j].pid == pid) { - proc_tbl[j].status = status; - break; - } - } - - if (j == num) { - ODPH_ERR("Bad pid:%d\n", (int)pid); - return -1; - } - - /* Examine the child process' termination status */ - if (WIFEXITED(status) && WEXITSTATUS(status) != EXIT_SUCCESS) { - ODPH_ERR("Child exit status:%d (pid:%d)\n", - WEXITSTATUS(status), (int)pid); - return -1; - } - if (WIFSIGNALED(status)) { - int signo = WTERMSIG(status); - - ODPH_ERR("Child term signo:%d - %s (pid:%d)\n", - signo, strsignal(signo), (int)pid); - return -1; - } - } - - return 0; -} - /* * wrapper for odpthreads, either implemented as linux threads or processes. * (in process mode, if start_routine returns NULL, the process return FAILURE). */ -static void *odpthread_run_start_routine(void *arg) +void *_odph_thread_run_start_routine(void *arg) { int status; int ret; @@ -289,54 +93,6 @@ static void *odpthread_run_start_routine(void *arg) } /* - * Create a single ODPthread as a linux process - */ -static int odph_linux_process_create(odph_odpthread_t *thread_tbl, - int cpu, - const odph_odpthread_params_t *thr_params) -{ - cpu_set_t cpu_set; - pid_t pid; - - CPU_ZERO(&cpu_set); - CPU_SET(cpu, &cpu_set); - - thread_tbl->start_args.thr_params = *thr_params; /* copy */ - thread_tbl->start_args.linuxtype = ODPTHREAD_PROCESS; - thread_tbl->cpu = cpu; - - pid = fork(); - if (pid < 0) { - ODPH_ERR("fork() failed\n"); - thread_tbl->start_args.linuxtype = ODPTHREAD_NOT_STARTED; - return -1; - } - - /* Parent continues to fork */ - if (pid > 0) { - thread_tbl->proc.pid = pid; - return 0; - } - - /* Child process */ - - /* Request SIGTERM if parent dies */ - prctl(PR_SET_PDEATHSIG, SIGTERM); - /* Parent died already? */ - if (getppid() == 1) - kill(getpid(), SIGTERM); - - if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { - ODPH_ERR("sched_setaffinity() failed\n"); - return -2; - } - - odpthread_run_start_routine(&thread_tbl->start_args); - - return 0; /* never reached */ -} - -/* * Create a single ODPthread as a linux thread */ static int odph_linux_thread_create(odph_odpthread_t *thread_tbl, @@ -361,7 +117,7 @@ static int odph_linux_thread_create(odph_odpthread_t *thread_tbl, ret = pthread_create(&thread_tbl->thread.thread_id, &thread_tbl->thread.attr, - odpthread_run_start_routine, + _odph_thread_run_start_routine, &thread_tbl->start_args); if (ret != 0) { ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); @@ -405,9 +161,9 @@ int odph_odpthreads_create(odph_odpthread_t *thread_tbl, thr_params)) break; } else { - if (odph_linux_process_create(&thread_tbl[i], - cpu, - thr_params)) + if (_odph_linux_process_create(&thread_tbl[i], + cpu, + thr_params)) break; } @@ -677,3 +433,51 @@ int odph_parse_options(int argc, char *argv[], return res; } + +/* + * Create a single ODPthread as a linux process + */ +static int _odph_linux_process_create(odph_odpthread_t *thread_tbl, + int cpu, + const odph_odpthread_params_t *thr_params) +{ + cpu_set_t cpu_set; + pid_t pid; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + thread_tbl->start_args.thr_params = *thr_params; /* copy */ + thread_tbl->start_args.linuxtype = ODPTHREAD_PROCESS; + thread_tbl->cpu = cpu; + + pid = fork(); + if (pid < 0) { + ODPH_ERR("fork() failed\n"); + thread_tbl->start_args.linuxtype = ODPTHREAD_NOT_STARTED; + return -1; + } + + /* Parent continues to fork */ + if (pid > 0) { + thread_tbl->proc.pid = pid; + return 0; + } + + /* Child process */ + + /* Request SIGTERM if parent dies */ + prctl(PR_SET_PDEATHSIG, SIGTERM); + /* Parent died already? */ + if (getppid() == 1) + kill(getpid(), SIGTERM); + + if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { + ODPH_ERR("sched_setaffinity() failed\n"); + return -2; + } + + _odph_thread_run_start_routine(&thread_tbl->start_args); + + return 0; /* never reached */ +} diff --git a/pkgconfig/libodphelper-linux.pc.in b/pkgconfig/libodphelper-linux-generic.pc.in similarity index 72% rename from pkgconfig/libodphelper-linux.pc.in rename to pkgconfig/libodphelper-linux-generic.pc.in index 3987f4c..cab7be2 100644 --- a/pkgconfig/libodphelper-linux.pc.in +++ b/pkgconfig/libodphelper-linux-generic.pc.in @@ -3,9 +3,9 @@ exec_prefix=@exec_prefix@ libdir=@libdir@ includedir=@includedir@ -Name: libodphelper-linux +Name: libodphelper-linux-generic Description: Helper for the ODP packet processing engine Version: @PKGCONFIG_VERSION@ -Libs: -L${libdir} -lodphelper-linux +Libs: -L${libdir} -lodphelper-linux-generic Libs.private: Cflags: -I${includedir} diff --git a/test/Makefile.inc b/test/Makefile.inc index 1ebc047..243a616 100644 --- a/test/Makefile.inc +++ b/test/Makefile.inc @@ -4,7 +4,7 @@ LIB = $(top_builddir)/lib #in the following line, the libs using the symbols should come before #the libs containing them! The includer is given a chance to add things #before libodp by setting PRE_LDADD before the inclusion. -LDADD = $(PRE_LDADD) $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LDADD = $(PRE_LDADD) $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la INCFLAGS = \ -I$(top_builddir)/platform/@with_platform@/include \ diff --git a/test/common_plat/validation/api/Makefile.inc b/test/common_plat/validation/api/Makefile.inc index ffba620..a0afd26 100644 --- a/test/common_plat/validation/api/Makefile.inc +++ b/test/common_plat/validation/api/Makefile.inc @@ -13,4 +13,4 @@ AM_LDFLAGS += -static LIBCUNIT_COMMON = $(COMMON_DIR)/libcunit_common.la LIBCPUMASK_COMMON = $(COMMON_DIR)/libcpumask_common.la LIBTHRMASK_COMMON = $(COMMON_DIR)/libthrmask_common.la -LIBODP = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LIBODP = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la diff --git a/test/linux-generic/Makefile.inc b/test/linux-generic/Makefile.inc index 36745fe..2a49076 100644 --- a/test/linux-generic/Makefile.inc +++ b/test/linux-generic/Makefile.inc @@ -6,7 +6,7 @@ AM_LDFLAGS += -static LIBCUNIT_COMMON = $(top_builddir)/test/common_plat/common/libcunit_common.la LIB = $(top_builddir)/lib -LIBODP = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LIBODP = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la INCCUNIT_COMMON = -I$(top_srcdir)/test/common_plat/common INCODP = \