From patchwork Wed Apr 27 09:13:01 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Uvarov X-Patchwork-Id: 66774 Delivered-To: patch@linaro.org Received: by 10.140.93.198 with SMTP id d64csp2073861qge; Wed, 27 Apr 2016 02:17:05 -0700 (PDT) X-Received: by 10.50.40.234 with SMTP id a10mr21459846igl.37.1461748625313; Wed, 27 Apr 2016 02:17:05 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id l11si4126775igf.102.2016.04.27.02.17.04; Wed, 27 Apr 2016 02:17:05 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id C3A65610DE; Wed, 27 Apr 2016 09:17:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 0896E606D0; Wed, 27 Apr 2016 09:16:52 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id E24C0610AA; Wed, 27 Apr 2016 09:16:33 +0000 (UTC) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) by lists.linaro.org (Postfix) with ESMTPS id 45A56604F6 for ; Wed, 27 Apr 2016 09:13:08 +0000 (UTC) Received: by mail-lf0-f45.google.com with SMTP id u64so43195882lff.3 for ; Wed, 27 Apr 2016 02:13:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=fQvbF2MJQdKpwnKFDhPg7yRTj+Gc8ak/3NHWlJcA0HI=; b=gL9LONleHXdPRbyE+KwibO3d2iEvBzGekmHALzka5+AQ69K142ycbyQnIHi30HMmtg o0Hedz+CTQjfwCv85Nwo3T1pjDMcWjU44vZxZJAlNI59SzLJpJXIm+wNZnaUM6ckWc0A Z/IntJuJJVGIPY0Foy07IG3ZB52dZsY0JTbJjt+8o8x5le45DlT66SeYF+Mmydq9AWZk pE49z5ofOGQ0BQ0pig7D6tZ5AuXnP/JwLqcPjsc3Fz1/X0wtJV6Sl2zP7WvOTTFRLUm3 zcsa4mAF5LO4bp4Uaykg4f7zD7uUfZvJpKxVi67mTCP/d2OLWBVucRTatdaYC1Y037Is HKOQ== X-Gm-Message-State: AOPr4FXZ2qiDKVWNePE9udcBLe+dusFo3lk6q1hQ5FdDiCqqZCjjogQ5MwQOnZI478P9V8qF5Mo= X-Received: by 10.25.152.147 with SMTP id a141mr3125167lfe.83.1461748386825; Wed, 27 Apr 2016 02:13:06 -0700 (PDT) Received: from maxim-Aspire-VN7-791.d-systems.local (mserver.d-systems.su. [185.75.190.112]) by smtp.gmail.com with ESMTPSA id e185sm596965lfg.45.2016.04.27.02.13.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 27 Apr 2016 02:13:06 -0700 (PDT) From: Maxim Uvarov To: lng-odp@lists.linaro.org Date: Wed, 27 Apr 2016 12:13:01 +0300 Message-Id: <1461748381-25967-1-git-send-email-maxim.uvarov@linaro.org> X-Mailer: git-send-email 2.7.1.250.gff4ea60 X-Topics: patch Subject: [lng-odp] [PATCH API-NEXT] linux-generic: repair ipc_pktio work after ipc ns changes X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Signed-off-by: Maxim Uvarov --- Note: by default ipc_pkt is turned on. But examples compilation and execution is turned off. You need to add --enable-pktio_ipc-support to ./configure I think there will be separate patch to finally enable it after several loops in CI. Maxim. .../include/odp/api/plat/init_types.h | 1 + platform/linux-generic/include/odp_internal.h | 1 + platform/linux-generic/include/odp_shm_internal.h | 5 ++++ platform/linux-generic/odp_init.c | 5 +++- platform/linux-generic/odp_shared_memory.c | 12 ++++++--- platform/linux-generic/pktio/ipc.c | 28 ++++++++++++++------- platform/linux-generic/test/pktio_ipc/ipc_common.c | 20 ++++++++++++--- platform/linux-generic/test/pktio_ipc/ipc_common.h | 3 +++ platform/linux-generic/test/pktio_ipc/pktio_ipc1.c | 29 +++++++++++++--------- platform/linux-generic/test/pktio_ipc/pktio_ipc2.c | 24 +++++++++++------- .../linux-generic/test/pktio_ipc/pktio_ipc_run | 17 ++++++------- 11 files changed, 96 insertions(+), 49 deletions(-) diff --git a/platform/linux-generic/include/odp/api/plat/init_types.h b/platform/linux-generic/include/odp/api/plat/init_types.h index 6a1052d..888b04a 100644 --- a/platform/linux-generic/include/odp/api/plat/init_types.h +++ b/platform/linux-generic/include/odp/api/plat/init_types.h @@ -25,6 +25,7 @@ typedef uint64_t odp_instance_t; * @internal platform specific data */ typedef struct odp_platform_init_t { + int ipc_ns; /**< Name space for ipc shared objects. */ } odp_platform_init_t; #ifdef __cplusplus diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index d8aa7af..28a4fc4 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -46,6 +46,7 @@ struct odp_global_data_s { odp_cpumask_t control_cpus; odp_cpumask_t worker_cpus; int num_cpus_installed; + int ipc_ns; }; enum init_stage { diff --git a/platform/linux-generic/include/odp_shm_internal.h b/platform/linux-generic/include/odp_shm_internal.h index 1fd7a3c..30e60f7 100644 --- a/platform/linux-generic/include/odp_shm_internal.h +++ b/platform/linux-generic/include/odp_shm_internal.h @@ -11,6 +11,11 @@ extern "C" { #endif +#include + +#define SHM_DEVNAME_MAXLEN (ODP_SHM_NAME_LEN + 16) +#define SHM_DEVNAME_FORMAT "/odp-%d-%s" /* /dev/shm/odp-- */ + #define _ODP_SHM_PROC_NOCREAT 0x4 /**< Do not create shm if not exist */ #define _ODP_SHM_O_EXCL 0x8 /**< Do not create shm if exist */ diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 51aaa9a..e8d12c9 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -12,9 +12,12 @@ struct odp_global_data_s odp_global_data; int odp_init_global(odp_instance_t *instance, const odp_init_t *params, - const odp_platform_init_t *platform_params ODP_UNUSED) + const odp_platform_init_t *platform_params) { + memset(&odp_global_data, 0, sizeof(struct odp_global_data_s)); odp_global_data.main_pid = getpid(); + if (platform_params) + odp_global_data.ipc_ns = platform_params->ipc_ns; enum init_stage stage = NO_INIT; odp_global_data.log_fn = odp_override_log; diff --git a/platform/linux-generic/odp_shared_memory.c b/platform/linux-generic/odp_shared_memory.c index 276a785..568711a 100644 --- a/platform/linux-generic/odp_shared_memory.c +++ b/platform/linux-generic/odp_shared_memory.c @@ -27,9 +27,6 @@ #include #include -#define SHM_DEVNAME_MAXLEN (ODP_SHM_NAME_LEN + 16) -#define SHM_DEVNAME_FORMAT "/odp-%d-%s" /* /dev/shm/odp-- */ - ODP_STATIC_ASSERT(ODP_CONFIG_SHM_BLOCKS >= ODP_CONFIG_POOLS, "ODP_CONFIG_SHM_BLOCKS < ODP_CONFIG_POOLS"); @@ -231,11 +228,18 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, uint64_t align, oflag |= O_EXCL; if (flags & (ODP_SHM_PROC | _ODP_SHM_PROC_NOCREAT)) { + int shm_ns_id; + + if (odp_global_data.ipc_ns) + shm_ns_id = odp_global_data.ipc_ns; + else + shm_ns_id = odp_global_data.main_pid; + need_huge_page = 0; /* Creates a file to /dev/shm/odp */ snprintf(shm_devname, SHM_DEVNAME_MAXLEN, - SHM_DEVNAME_FORMAT, odp_global_data.main_pid, name); + SHM_DEVNAME_FORMAT, shm_ns_id, name); fd = shm_open(shm_devname, oflag, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH); if (fd == -1) { diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index 4a17e36..b16c611 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -45,13 +45,23 @@ static const char *_ipc_odp_buffer_pool_shm_name(odp_pool_t pool_hdl) static int _ipc_shm_lookup(const char *name) { int shm; + char shm_devname[SHM_DEVNAME_MAXLEN]; - shm = shm_open(name, O_RDWR, S_IRUSR | S_IWUSR); + if (!odp_global_data.ipc_ns) + ODP_ABORT("ipc_ns not set\n"); + + snprintf(shm_devname, SHM_DEVNAME_MAXLEN, + SHM_DEVNAME_FORMAT, + odp_global_data.ipc_ns, name); + + shm = shm_open(shm_devname, O_RDWR, S_IRUSR | S_IWUSR); if (shm == -1) { - if (errno == ENOENT) + if (errno == ENOENT) { + ODP_DBG("no file %s\n", shm_devname); return -1; + } ODP_ABORT("shm_open for %s err %s\n", - name, strerror(errno)); + shm_devname, strerror(errno)); } close(shm); return 0; @@ -583,6 +593,8 @@ static int ipc_pktio_send(pktio_entry_t *pktio_entry, int ret; unsigned i; uint32_t ready = odp_atomic_load_u32(&pktio_entry->s.ipc.ready); + odp_packet_t pkt_table_mapped[len]; /**< Ready to send packet has to be + * in memory mapped pool. */ if (odp_unlikely(!ready)) return 0; @@ -610,8 +622,9 @@ static int ipc_pktio_send(pktio_entry_t *pktio_entry, ODP_ABORT("Unable to copy packet\n"); odp_packet_free(pkt); - /* Cannot do this. Packet table is const. */ - /*pkt_table[i] = newpkt;*/ + pkt_table_mapped[i] = newpkt; + } else { + pkt_table_mapped[i] = pkt; } rbuf_p = (void *)&pkt; @@ -626,10 +639,7 @@ static int ipc_pktio_send(pktio_entry_t *pktio_entry, } /* Put packets to ring to be processed by other process. */ - /* BUG: Cannot store pointer to user provided memory, - which is likely allocated from the stack and will be overwritten - after this function returns. */ -/* rbuf_p = (void *)&pkt_table[0]; */ + rbuf_p = (void *)&pkt_table_mapped[0]; r = pktio_entry->s.ipc.tx.send; ret = _ring_mp_enqueue_burst(r, rbuf_p, len); if (odp_unlikely(ret < 0)) { diff --git a/platform/linux-generic/test/pktio_ipc/ipc_common.c b/platform/linux-generic/test/pktio_ipc/ipc_common.c index 101f597..34d2632 100644 --- a/platform/linux-generic/test/pktio_ipc/ipc_common.c +++ b/platform/linux-generic/test/pktio_ipc/ipc_common.c @@ -8,6 +8,7 @@ /** Run time in seconds */ int run_time_sec; +int ipc_name_space; int ipc_odp_packet_sendall(odp_pktio_t pktio, odp_packet_t pkt_tbl[], int num) @@ -80,14 +81,16 @@ void parse_args(int argc, char *argv[]) int long_index; static struct option longopts[] = { {"time", required_argument, NULL, 't'}, - {"help", no_argument, NULL, 'h'}, /* return 'h' */ + {"ns", required_argument, NULL, 'n'}, /* ipc name space */ + {"help", no_argument, NULL, 'h'}, /* return 'h' */ {NULL, 0, NULL, 0} }; run_time_sec = 0; /* loop forever if time to run is 0 */ + ipc_name_space = 0; while (1) { - opt = getopt_long(argc, argv, "+t:h", + opt = getopt_long(argc, argv, "+t:n:h", longopts, &long_index); if (opt == -1) @@ -97,17 +100,24 @@ void parse_args(int argc, char *argv[]) case 't': run_time_sec = atoi(optarg); break; + case 'n': + ipc_name_space = atoi(optarg); + break; case 'h': usage(argv[0]); exit(EXIT_SUCCESS); break; - default: break; } } optind = 1; /* reset 'extern optind' from the getopt lib */ + + if (!ipc_name_space) { + usage(argv[0]); + exit(1); + } } /** @@ -138,10 +148,12 @@ void usage(char *progname) { printf("\n" "Usage: %s OPTIONS\n" - " E.g. %s -t seconds\n" + " E.g. -n ipc_name_space %s -t seconds\n" "\n" "OpenDataPlane linux-generic ipc test application.\n" "\n" + "Mandatory OPTIONS:\n" + " -n, --ns IPC name space ID /dev/shm/odp--objname.\n" "Optional OPTIONS\n" " -h, --help Display help and exit.\n" " -t, --time Time to run in seconds.\n" diff --git a/platform/linux-generic/test/pktio_ipc/ipc_common.h b/platform/linux-generic/test/pktio_ipc/ipc_common.h index c5e3eb7..7bc483f 100644 --- a/platform/linux-generic/test/pktio_ipc/ipc_common.h +++ b/platform/linux-generic/test/pktio_ipc/ipc_common.h @@ -62,6 +62,9 @@ char *pktio_name; /** Run time in seconds */ int run_time_sec; +/** IPC name space id /dev/shm/odp-nsid-objname */ +int ipc_name_space; + /* helper funcs */ void parse_args(int argc, char *argv[]); void print_info(char *progname); diff --git a/platform/linux-generic/test/pktio_ipc/pktio_ipc1.c b/platform/linux-generic/test/pktio_ipc/pktio_ipc1.c index cec2b07..a4eed88 100644 --- a/platform/linux-generic/test/pktio_ipc/pktio_ipc1.c +++ b/platform/linux-generic/test/pktio_ipc/pktio_ipc1.c @@ -113,9 +113,9 @@ static int pktio_run_loop(odp_pool_t pool) EXAMPLE_ABORT("invalid l4 offset\n"); off += ODPH_UDPHDR_LEN; - ret = odp_packet_copydata_out(pkt, off, - sizeof(head), - &head); + ret = odp_packet_copy_to_mem(pkt, off, + sizeof(head), + &head); if (ret) { stat_errors++; odp_packet_free(pkt); @@ -137,9 +137,9 @@ static int pktio_run_loop(odp_pool_t pool) } off = odp_packet_len(pkt) - sizeof(pkt_tail_t); - ret = odp_packet_copydata_out(pkt, off, - sizeof(tail), - &tail); + ret = odp_packet_copy_to_mem(pkt, off, + sizeof(tail), + &tail); if (ret) { stat_errors++; odp_packet_free(pkt); @@ -210,15 +210,15 @@ static int pktio_run_loop(odp_pool_t pool) head.seq = cnt++; off += ODPH_UDPHDR_LEN; - ret = odp_packet_copydata_in(pkt, off, sizeof(head), - &head); + ret = odp_packet_copy_from_mem(pkt, off, sizeof(head), + &head); if (ret) EXAMPLE_ABORT("unable to copy in head data"); tail.magic = TEST_SEQ_MAGIC; off = odp_packet_len(pkt) - sizeof(pkt_tail_t); - ret = odp_packet_copydata_in(pkt, off, sizeof(tail), - &tail); + ret = odp_packet_copy_from_mem(pkt, off, sizeof(tail), + &tail); if (ret) EXAMPLE_ABORT("unable to copy in tail data"); } @@ -280,19 +280,24 @@ int main(int argc, char *argv[]) { odp_pool_t pool; odp_pool_param_t params; + odp_instance_t instance; + odp_platform_init_t plat_idata; int ret; /* Parse and store the application arguments */ parse_args(argc, argv); + memset(&plat_idata, 0, sizeof(odp_platform_init_t)); + plat_idata.ipc_ns = ipc_name_space; + /* Init ODP before calling anything else */ - if (odp_init_global(NULL, NULL)) { + if (odp_init_global(&instance, NULL, &plat_idata)) { EXAMPLE_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } /* Init this thread */ - if (odp_init_local(ODP_THREAD_CONTROL)) { + if (odp_init_local(instance, ODP_THREAD_CONTROL)) { EXAMPLE_ERR("Error: ODP local init failed.\n"); exit(EXIT_FAILURE); } diff --git a/platform/linux-generic/test/pktio_ipc/pktio_ipc2.c b/platform/linux-generic/test/pktio_ipc/pktio_ipc2.c index a133d8b..c0c6ff5 100644 --- a/platform/linux-generic/test/pktio_ipc/pktio_ipc2.c +++ b/platform/linux-generic/test/pktio_ipc/pktio_ipc2.c @@ -101,8 +101,8 @@ static int ipc_second_process(void) EXAMPLE_ABORT("invalid l4 offset\n"); off += ODPH_UDPHDR_LEN; - ret = odp_packet_copydata_out(pkt, off, sizeof(head), - &head); + ret = odp_packet_copy_to_mem(pkt, off, sizeof(head), + &head); if (ret) EXAMPLE_ABORT("unable copy out head data"); @@ -111,8 +111,8 @@ static int ipc_second_process(void) /* Modify magic number in packet */ head.magic = TEST_SEQ_MAGIC_2; - ret = odp_packet_copydata_in(pkt, off, sizeof(head), - &head); + ret = odp_packet_copy_from_mem(pkt, off, sizeof(head), + &head); if (ret) EXAMPLE_ABORT("unable to copy in head data"); } @@ -136,9 +136,9 @@ static int ipc_second_process(void) off = odp_packet_l4_offset(alloc_pkt); off += ODPH_UDPHDR_LEN; - ret = odp_packet_copydata_in(alloc_pkt, off, - sizeof(head), - &head); + ret = odp_packet_copy_from_mem(alloc_pkt, off, + sizeof(head), + &head); if (ret) EXAMPLE_ABORT("unable to copy in head data"); @@ -173,16 +173,22 @@ exit: int main(int argc, char *argv[]) { + odp_instance_t instance; + odp_platform_init_t plat_idata; + /* Parse and store the application arguments */ parse_args(argc, argv); - if (odp_init_global(NULL, NULL)) { + memset(&plat_idata, 0, sizeof(odp_platform_init_t)); + plat_idata.ipc_ns = ipc_name_space; + + if (odp_init_global(&instance, NULL, &plat_idata)) { EXAMPLE_ERR("Error: ODP global init failed.\n"); exit(EXIT_FAILURE); } /* Init this thread */ - if (odp_init_local(ODP_THREAD_CONTROL)) { + if (odp_init_local(instance, ODP_THREAD_CONTROL)) { EXAMPLE_ERR("Error: ODP local init failed.\n"); exit(EXIT_FAILURE); } diff --git a/platform/linux-generic/test/pktio_ipc/pktio_ipc_run b/platform/linux-generic/test/pktio_ipc/pktio_ipc_run index 08a7457..1128002 100755 --- a/platform/linux-generic/test/pktio_ipc/pktio_ipc_run +++ b/platform/linux-generic/test/pktio_ipc/pktio_ipc_run @@ -20,18 +20,17 @@ PATH=.:$PATH run() { local ret=0 + IPC_NS=$$ #if test was interrupted with CTRL+c than files #might remain in shm. Needed cleanely delete them. - rm -rf /dev/shm/ipc_pktio_* 2>&1 > /dev/null - rm -rf /dev/shm/packet_pool1 2>&1 > /dev/null - rm -rf /dev/shm/packet_pool2 2>&1 > /dev/null + rm -rf /dev/shm/odp-${IPC_NS}* 2>&1 > /dev/null echo "==== run pktio_ipc1 then pktio_ipc2 ====" - pktio_ipc1${EXEEXT} -t 30 & + pktio_ipc1${EXEEXT} -n ${IPC_NS} -t 30 & IPC_PID=$! - pktio_ipc2${EXEEXT} -t 10 + pktio_ipc2${EXEEXT} -n ${IPC_NS} -t 10 ret=$? # pktio_ipc1 should do clean up and exit just # after pktio_ipc2 exited. If it does not happen @@ -39,9 +38,7 @@ run() sleep 1 kill ${IPC_PID} 2>&1 > /dev/null if [ $? -eq 0 ]; then - rm -rf /dev/shm/ipc_pktio_* 2>&1 > /dev/null - rm -rf /dev/shm/packet_pool1 2>&1 > /dev/null - rm -rf /dev/shm/packet_pool2 2>&1 > /dev/null + rm -rf /dev/shm/odp-${IPC_NS}* 2>&1 > /dev/null fi if [ $ret -ne 0 ]; then @@ -52,10 +49,10 @@ run() fi echo "==== run pktio_ipc2 then pktio_ipc1 ====" - pktio_ipc2${EXEEXT} -t 10 & + pktio_ipc2${EXEEXT} -n ${IPC_NS} -t 10 & IPC_PID=$! - pktio_ipc1${EXEEXT} -t 20 + pktio_ipc1${EXEEXT} -n ${IPC_NS} -t 20 ret=$? (kill ${IPC_PID} 2>&1 > /dev/null) > /dev/null || true