From patchwork Tue Jul 4 22:00:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 107034 Delivered-To: patch@linaro.org Received: by 10.140.101.44 with SMTP id t41csp60109qge; Tue, 4 Jul 2017 15:05:20 -0700 (PDT) X-Received: by 10.200.45.182 with SMTP id p51mr50423314qta.208.1499205919961; Tue, 04 Jul 2017 15:05:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1499205919; cv=none; d=google.com; s=arc-20160816; b=tNd2vIRAiiXyszo9gedSczLfn+u6XsB63XsVttmGOHUVP1eLZUzmjo0iSUOk0IGUjG /9FJI4E9O12e9mTb1/wMsqjWKn6qy6b+GvIr+qsHBNTFuqYHbAvKcVnnZyrBFsVcWsLm DUkT9iMMCY+tVYhGqf3SeaaCF7WkRJMCI9M2Ml3R2aYEBlXZSiMWPhhEG/1RAzSjjqbO clq8KrVu5mDaKJ/64DuwVbwcf693cMakHWC1oHNv18VIoS4TJbECDo6823JRc/2UuuQV Pu+rYEJdgOVo1qIb0l+yZmo2Bw0VbagwN8XU2pMSlinyjy+Oo/IsuGDnS3he1L5GdsPj o4VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=OvCErNfO4qNeFVI2+FLvGslADqnh48T6EQcGrsDjaGQ=; b=HjyQCl8uYRg0dHJXwjFZf5fdpwab60WejT5tzw6tvx3MtfO/QxEbdFn6wWfBkoH6qE /SBiWqW8+7nXBclDvFpYMFqJ0pO3afq+llUf5ZKn7rRMQNjdM4l7obpLkOyDySgpBebD d9eBsV0q5h3l7YBtoB/2H8tFuB0u1/j1LsM6SSbtQB4Nc3dBdMRoeEcN5Njo0Qr06P1/ DAk5udV1aU1bK9WHtZXRWyezOLoyl5r+8JX1cNEtaaQ4Lu7qJo1sBa/Nzp+YvDenFfiF iNClsdqoCQ4lZUYxmD6PslVdtW1qcwvK46ReQ27YfQn+Il41sH964L+RK6G9+JNdUJad bwYw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id a12si6786758qti.262.2017.07.04.15.05.19; Tue, 04 Jul 2017 15:05:19 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 9610760665; Tue, 4 Jul 2017 22:05:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id C261D6074A; Tue, 4 Jul 2017 22:01:22 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 972E160C2A; Tue, 4 Jul 2017 22:01:11 +0000 (UTC) Received: from forward5h.cmail.yandex.net (forward5h.cmail.yandex.net [87.250.230.20]) by lists.linaro.org (Postfix) with ESMTPS id C884A6071A for ; Tue, 4 Jul 2017 22:00:19 +0000 (UTC) Received: from smtp2p.mail.yandex.net (smtp2p.mail.yandex.net [77.88.29.85]) by forward5h.cmail.yandex.net (Yandex) with ESMTP id 6F526210F5 for ; Wed, 5 Jul 2017 01:00:18 +0300 (MSK) Received: from smtp2p.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2p.mail.yandex.net (Yandex) with ESMTP id 4D03B1A8003E for ; Wed, 5 Jul 2017 01:00:17 +0300 (MSK) Received: by smtp2p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id b3wfOILtJj-0HPecAWW; Wed, 05 Jul 2017 01:00:17 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Wed, 5 Jul 2017 01:00:07 +0300 Message-Id: <1499205607-30786-8-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1499205607-30786-1-git-send-email-odpbot@yandex.ru> References: <1499205607-30786-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 67 Subject: [lng-odp] [PATCH v1 7/7] linux-gen: shm: user can define place for shm files X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Maxim Uvarov Signed-off-by: Maxim Uvarov --- /** Email created from pull request 67 (muvarov:master_ipc_fixes) ** https://github.com/Linaro/odp/pull/67 ** Patch: https://github.com/Linaro/odp/pull/67.patch ** Base sha: ceeab69f3af67701adb524c7b9757d19cefb1110 ** Merge commit sha: 6f88e14c89b861d66bb7043b22f9b299b1f73ddf **/ platform/linux-generic/_ishm.c | 83 ++++++++++++++++++++-- platform/linux-generic/include/_ishm_internal.h | 1 + platform/linux-generic/include/odp_internal.h | 1 + platform/linux-generic/odp_init.c | 72 ++----------------- .../validation/api/shmem/shmem_common.h | 5 +- .../validation/api/shmem/shmem_linux.c | 8 ++- .../validation/api/shmem/shmem_odp1.c | 8 ++- .../validation/api/shmem/shmem_odp2.c | 4 +- 8 files changed, 102 insertions(+), 80 deletions(-) diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c index 6545bfce..cde2dbc3 100644 --- a/platform/linux-generic/_ishm.c +++ b/platform/linux-generic/_ishm.c @@ -71,6 +71,8 @@ #include #include #include +#include +#include /* * Maximum number of internal shared memory blocks. @@ -99,6 +101,7 @@ #define ISHM_FILENAME_MAXLEN (ISHM_NAME_MAXLEN + 64) #define ISHM_FILENAME_FORMAT "%s/odp-%d-ishm-%s" #define ISHM_FILENAME_NORMAL_PAGE_DIR "/dev/shm" +#define _ODP_FILES_FMT "odp-%d-" /* * when the memory is to be shared with an external entity (such as another @@ -106,7 +109,7 @@ * export file is created describing the exported memory: this defines the * location and the filename format of this description file */ -#define ISHM_EXPTNAME_FORMAT "/dev/shm/%s/odp-%d-shm-%s" +#define ISHM_EXPTNAME_FORMAT "%s/%s/odp-%d-shm-%s" /* * At worse case the virtual space gets so fragmented that there is @@ -437,7 +440,7 @@ static int create_file(int block_index, huge_flag_t huge, uint64_t len, odp_global_data.uid); else snprintf(dir, ISHM_FILENAME_MAXLEN, "%s/%s", - ISHM_FILENAME_NORMAL_PAGE_DIR, + odp_global_data.shm_dir, odp_global_data.uid); snprintf(filename, ISHM_FILENAME_MAXLEN, @@ -476,6 +479,7 @@ static int create_file(int block_index, huge_flag_t huge, uint64_t len, ISHM_FILENAME_MAXLEN - 1); snprintf(new_block->exptname, ISHM_FILENAME_MAXLEN, ISHM_EXPTNAME_FORMAT, + odp_global_data.shm_dir, odp_global_data.uid, odp_global_data.main_pid, (name && name[0]) ? name : seq_string); @@ -955,6 +959,7 @@ int _odp_ishm_find_exported(const char *remote_name, pid_t external_odp_pid, /* try to read the block description file: */ snprintf(export_filename, ISHM_FILENAME_MAXLEN, ISHM_EXPTNAME_FORMAT, + odp_global_data.shm_dir, odp_global_data.uid, external_odp_pid, remote_name); @@ -1383,22 +1388,86 @@ static int do_odp_ishm_init_local(void) return 0; } +/* remove all files staring with "odp-" from a directory "dir" */ +int _odp_ishm_cleanup_files(const char *dirpath) +{ + struct dirent *e; + DIR *dir; + char userdir[PATH_MAX]; + char prefix[PATH_MAX]; + char *fullpath; + int d_len = strlen(dirpath); + int p_len; + int f_len; + + snprintf(userdir, PATH_MAX, "%s/%s", dirpath, odp_global_data.uid); + + dir = opendir(userdir); + if (!dir) { + /* ok if the dir does not exist. no much to delete then! */ + ODP_DBG("opendir failed for %s: %s\n", + dirpath, strerror(errno)); + return 0; + } + snprintf(prefix, PATH_MAX, _ODP_FILES_FMT, odp_global_data.main_pid); + p_len = strlen(prefix); + while ((e = readdir(dir)) != NULL) { + if (strncmp(e->d_name, prefix, p_len) == 0) { + f_len = strlen(e->d_name); + fullpath = malloc(d_len + f_len + 2); + if (fullpath == NULL) { + closedir(dir); + return -1; + } + snprintf(fullpath, PATH_MAX, "%s/%s", + dirpath, e->d_name); + ODP_DBG("deleting obsolete file: %s\n", fullpath); + if (unlink(fullpath)) + ODP_ERR("unlink failed for %s: %s\n", + fullpath, strerror(errno)); + free(fullpath); + } + } + closedir(dir); + + return 0; +} + int _odp_ishm_init_global(void) { void *addr; void *spce_addr; int i; + uid_t uid; - if ((getpid() != odp_global_data.main_pid) || - (syscall(SYS_gettid) != getpid())) - ODP_ERR("odp_init_global() must be performed by the main " + odp_global_data.main_pid = getpid(); + odp_global_data.shm_dir = getenv("ODP_SHM_DIR"); + odp_global_data.shm_dir = + calloc(1, sizeof(ISHM_FILENAME_NORMAL_PAGE_DIR)); + sprintf(odp_global_data.shm_dir, "%s", ISHM_FILENAME_NORMAL_PAGE_DIR); + + ODP_DBG("ishm: using dir %s\n", odp_global_data.shm_dir); + + uid = getuid(); + snprintf(odp_global_data.uid, UID_MAXLEN, "%d", + uid); + + if ((syscall(SYS_gettid)) != odp_global_data.main_pid) { + ODP_ERR("ishm init must be performed by the main " "ODP process!\n."); + return -1; + } if (!odp_global_data.hugepage_info.default_huge_page_dir) ODP_DBG("NOTE: No support for huge pages\n"); - else + else { ODP_DBG("Huge pages mount point is: %s\n", odp_global_data.hugepage_info.default_huge_page_dir); + _odp_ishm_cleanup_files( + odp_global_data.hugepage_info.default_huge_page_dir); + } + + _odp_ishm_cleanup_files(odp_global_data.shm_dir); /* allocate space for the internal shared mem block table: */ addr = mmap(NULL, sizeof(ishm_table_t), @@ -1592,6 +1661,8 @@ int _odp_ishm_term_global(void) if (_odp_ishmphy_unbook_va()) ret |= -1; + free(odp_global_data.shm_dir); + return ret; } diff --git a/platform/linux-generic/include/_ishm_internal.h b/platform/linux-generic/include/_ishm_internal.h index c7c33077..005d6b55 100644 --- a/platform/linux-generic/include/_ishm_internal.h +++ b/platform/linux-generic/include/_ishm_internal.h @@ -44,6 +44,7 @@ int _odp_ishm_find_exported(const char *remote_name, void *_odp_ishm_address(int block_index); int _odp_ishm_info(int block_index, _odp_ishm_info_t *info); int _odp_ishm_status(const char *title); +int _odp_ishm_cleanup_files(const char *dirpath); #ifdef __cplusplus } diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index 7e6811f3..dd3374b7 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -42,6 +42,7 @@ typedef struct { } hugepage_info_t; struct odp_global_data_s { + char *shm_dir; /*< directory for odp mmaped files */ pid_t main_pid; char uid[UID_MAXLEN]; odp_log_func_t log_fn; diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index f62ccec0..ab2d6fb0 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -20,70 +20,15 @@ #include #include -#define _ODP_FILES_FMT "odp-%d-" -#define _ODP_TMPDIR "/dev/shm" - struct odp_global_data_s odp_global_data; -/* remove all files staring with "odp-" from a directory "dir" */ -static int cleanup_files(const char *dirpath, int odp_pid) -{ - struct dirent *e; - DIR *dir; - char userdir[PATH_MAX]; - char prefix[PATH_MAX]; - char *fullpath; - int d_len = strlen(dirpath); - int p_len; - int f_len; - - snprintf(userdir, PATH_MAX, "%s/%s", dirpath, odp_global_data.uid); - - dir = opendir(userdir); - if (!dir) { - /* ok if the dir does not exist. no much to delete then! */ - ODP_DBG("opendir failed for %s: %s\n", - dirpath, strerror(errno)); - return 0; - } - snprintf(prefix, PATH_MAX, _ODP_FILES_FMT, odp_pid); - p_len = strlen(prefix); - while ((e = readdir(dir)) != NULL) { - if (strncmp(e->d_name, prefix, p_len) == 0) { - f_len = strlen(e->d_name); - fullpath = malloc(d_len + f_len + 2); - if (fullpath == NULL) { - closedir(dir); - return -1; - } - snprintf(fullpath, PATH_MAX, "%s/%s", - dirpath, e->d_name); - ODP_DBG("deleting obsolete file: %s\n", fullpath); - if (unlink(fullpath)) - ODP_ERR("unlink failed for %s: %s\n", - fullpath, strerror(errno)); - free(fullpath); - } - } - closedir(dir); - - return 0; -} - int odp_init_global(odp_instance_t *instance, const odp_init_t *params, const odp_platform_init_t *platform_params ODP_UNUSED) { - char *hpdir; - uid_t uid; - memset(&odp_global_data, 0, sizeof(struct odp_global_data_s)); odp_global_data.main_pid = getpid(); - uid = getuid(); - snprintf(odp_global_data.uid, UID_MAXLEN, "%d", - uid); - enum init_stage stage = NO_INIT; odp_global_data.log_fn = odp_override_log; odp_global_data.abort_fn = odp_override_abort; @@ -95,8 +40,6 @@ int odp_init_global(odp_instance_t *instance, odp_global_data.abort_fn = params->abort_fn; } - cleanup_files(_ODP_TMPDIR, odp_global_data.main_pid); - if (odp_cpumask_init_global(params)) { ODP_ERR("ODP cpumask init failed.\n"); goto init_failed; @@ -113,23 +56,20 @@ int odp_init_global(odp_instance_t *instance, ODP_ERR("ODP system_info init failed.\n"); goto init_failed; } - hpdir = odp_global_data.hugepage_info.default_huge_page_dir; - /* cleanup obsolete huge page files, if any */ - if (hpdir) - cleanup_files(hpdir, odp_global_data.main_pid); stage = SYSINFO_INIT; + if (_odp_ishm_init_global()) { + ODP_ERR("ODP ishm init failed.\n"); + goto init_failed; + } + stage = ISHM_INIT; + if (_odp_fdserver_init_global()) { ODP_ERR("ODP fdserver init failed.\n"); goto init_failed; } stage = FDSERVER_INIT; - if (_odp_ishm_init_global()) { - ODP_ERR("ODP ishm init failed.\n"); - goto init_failed; - } - stage = ISHM_INIT; if (odp_thread_init_global()) { ODP_ERR("ODP thread init failed.\n"); diff --git a/test/linux-generic/validation/api/shmem/shmem_common.h b/test/linux-generic/validation/api/shmem/shmem_common.h index 621e3431..e6c04d81 100644 --- a/test/linux-generic/validation/api/shmem/shmem_common.h +++ b/test/linux-generic/validation/api/shmem/shmem_common.h @@ -7,8 +7,9 @@ #ifndef _COMMON_TEST_SHMEM_H_ #define _COMMON_TEST_SHMEM_H_ -#define ODP_SHM_NAME "odp_linux_shared_mem" -#define FIFO_NAME_FMT "/dev/shm/%d/shmem_test_fifo-%d" +#define SHM_NAME "odp_linux_shared_mem" +#define DEFAULT_SHM_DIR "/dev/shm" +#define FIFO_NAME_FMT "/%s/%d/shmem_test_fifo-%d" #define ALIGN_SIZE (128) #define TEST_SHARE_FOO (0xf0f0f0f0) #define TEST_SHARE_BAR (0xf0f0f0f) diff --git a/test/linux-generic/validation/api/shmem/shmem_linux.c b/test/linux-generic/validation/api/shmem/shmem_linux.c index d5931447..e7e699e7 100644 --- a/test/linux-generic/validation/api/shmem/shmem_linux.c +++ b/test/linux-generic/validation/api/shmem/shmem_linux.c @@ -79,6 +79,7 @@ #include #include #include +#include #include "shmem_linux.h" #include "shmem_common.h" @@ -210,6 +211,7 @@ int main(int argc __attribute__((unused)), char *argv[]) test_shared_linux_data_t *addr; int app2_status; uid_t uid = getuid(); + char *shm_dir = getenv("ODP_SHM_DIR"); /* odp_app1 is in the same directory as this file: */ strncpy(prg_name, argv[0], PATH_MAX - 1); @@ -228,7 +230,9 @@ int main(int argc __attribute__((unused)), char *argv[]) /* wait max 30 sec for the fifo to be created by the ODP side. * Just die if time expire as there is no fifo to communicate * through... */ - sprintf(fifo_name, FIFO_NAME_FMT, uid, odp_app1); + sprintf(fifo_name, FIFO_NAME_FMT, + shm_dir ? shm_dir : DEFAULT_SHM_DIR, + uid, odp_app1); for (nb_sec = 0; nb_sec < MAX_FIFO_WAIT; nb_sec++) { fifo_fd = open(fifo_name, O_WRONLY); if (fifo_fd >= 0) @@ -244,7 +248,7 @@ int main(int argc __attribute__((unused)), char *argv[]) * check to see if linux can see the created shared memory: */ /* read the shared memory attributes (includes the shm filename): */ - if (read_shmem_attribues(odp_app1, ODP_SHM_NAME, + if (read_shmem_attribues(odp_app1, SHM_NAME, shm_filename, &len, &flags, &user_len, &user_flags, &align) != 0) test_failure(fifo_name, fifo_fd, odp_app1); diff --git a/test/linux-generic/validation/api/shmem/shmem_odp1.c b/test/linux-generic/validation/api/shmem/shmem_odp1.c index c402365a..26abc94b 100644 --- a/test/linux-generic/validation/api/shmem/shmem_odp1.c +++ b/test/linux-generic/validation/api/shmem/shmem_odp1.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include "shmem_odp1.h" @@ -26,9 +27,10 @@ void shmem_test_odp_shm_proc(void) odp_shm_t shm; test_shared_data_t *test_shared_data; char test_result; + char *shm_dir = getenv("ODP_SHM_DIR"); /* reminder: ODP_SHM_PROC => export to linux, ODP_SHM_EXPORT=>to odp */ - shm = odp_shm_reserve(ODP_SHM_NAME, + shm = odp_shm_reserve(SHM_NAME, sizeof(test_shared_data_t), ALIGN_SIZE, ODP_SHM_PROC | ODP_SHM_EXPORT); CU_ASSERT_FATAL(ODP_SHM_INVALID != shm); @@ -41,7 +43,9 @@ void shmem_test_odp_shm_proc(void) /* open the fifo: this will indicate to linux process that it can * start the shmem lookups and check if it sees the data */ - sprintf(fifo_name, FIFO_NAME_FMT, getuid(), getpid()); + sprintf(fifo_name, FIFO_NAME_FMT, + shm_dir ? shm_dir : DEFAULT_SHM_DIR, + getuid(), getpid()); CU_ASSERT_FATAL(mkfifo(fifo_name, 0666) == 0); /* read from the fifo: the linux process result: */ diff --git a/test/linux-generic/validation/api/shmem/shmem_odp2.c b/test/linux-generic/validation/api/shmem/shmem_odp2.c index 7d8c682b..2a4b67d7 100644 --- a/test/linux-generic/validation/api/shmem/shmem_odp2.c +++ b/test/linux-generic/validation/api/shmem/shmem_odp2.c @@ -53,8 +53,8 @@ int main(int argc, char *argv[]) odp1 = (odp_instance_t)atoi(argv[1]); printf("shmem_odp2: trying to grab %s from pid %d\n", - ODP_SHM_NAME, (int)odp1); - shm = odp_shm_import(ODP_SHM_NAME, odp1, ODP_SHM_NAME); + SHM_NAME, (int)odp1); + shm = odp_shm_import(SHM_NAME, odp1, SHM_NAME); if (shm == ODP_SHM_INVALID) { fprintf(stderr, "error: odp_shm_lookup_external failed.\n"); return 1;