From patchwork Mon Sep 10 09:00:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 146294 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp2251668ljw; Mon, 10 Sep 2018 02:01:20 -0700 (PDT) X-Google-Smtp-Source: ANB0VdapEyHn/rjnC0og0fkOkkEUIotMlmuHM7VChUYNavbz599YNpmkRCM+LvO+2LJ3ccnYrCk2 X-Received: by 2002:aed:3c55:: with SMTP id u21-v6mr14958927qte.198.1536570080753; Mon, 10 Sep 2018 02:01:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536570080; cv=none; d=google.com; s=arc-20160816; b=XXprWgcsdqq9BweCZbIX5SrfpRrPXq76eKujA/K0V4BOA7P1gLskayLsW9nU+xQKHS WaT3so0PVOVUXxdDd/UKnRadJbG0wT8VGm3C7zi8xV/BGhh5BxHdUxL5MuewmeU8CKTx n+kuWRLvLXZQKP+sEYgkTt3oTmjJLW+A2IcX4A1uyQJqnXMSx/l+DmY4/+MSfW1T28u3 r7uynFknDguQOjVy5wVX6mMUjAk+D0oORRak12a/HO3Wnqro1s8qOvKuya1t8NHogj5t r/Y2eLflxxVMietcSsquREQwqhQDEwbk8KCCF6KO3xEF38AQhLHfGI+Cs0Ffyi1lhdgZ 6seA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to; bh=PNoCjTEAt5ymIoOe+P6O3o29hr/hl2JLDOvF+gAEEqo=; b=MGqwTQNP6XdQM+K6Lq3yHdjzdT02iZt0264gNepamiFfx0rKF2xYgJuHrIk50/qWb2 5oJEtG0KHn5F5EqObtb7QqkqD//EoWUKbrqArHrKsRTCOGCK8z/xLHvdW/VqgFyWL1gI AGJUmEy4mVPsh7s+JFt2nU0HRzNiFKnGYrE4J0hsx/F81Fs9YPQ4lKDZnAItT7bESDSk z01aoaLhoXuMewYXCz7QWYrddv9ICjAzl5IUJ8Ah4IBDK4/Tp9sM4wU3aG/Gj8qgE9Eo 2iSVpWDFEyNfwe76ohK8+FMC/IL0+s8rpXftCrs6VOWfRBSTP8UvW+XA4PSloaMlsK0x hllw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id s11-v6si359112qvk.0.2018.09.10.02.01.20; Mon, 10 Sep 2018 02:01:20 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 65068618DC; Mon, 10 Sep 2018 09:01:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_LOW autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id DCF576080E; Mon, 10 Sep 2018 09:00:30 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id A432A61889; Mon, 10 Sep 2018 09:00:20 +0000 (UTC) Received: from forward103o.mail.yandex.net (forward103o.mail.yandex.net [37.140.190.177]) by lists.linaro.org (Postfix) with ESMTPS id E78AE60F1E for ; Mon, 10 Sep 2018 09:00:15 +0000 (UTC) Received: from mxback8j.mail.yandex.net (mxback8j.mail.yandex.net [IPv6:2a02:6b8:0:1619::111]) by forward103o.mail.yandex.net (Yandex) with ESMTP id 8DA3B588165E for ; Mon, 10 Sep 2018 12:00:13 +0300 (MSK) Received: from smtp2o.mail.yandex.net (smtp2o.mail.yandex.net [2a02:6b8:0:1a2d::26]) by mxback8j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id h86hIL8fuS-0DjufKW2; Mon, 10 Sep 2018 12:00:13 +0300 Received: by smtp2o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id jM0XhXSRvO-0CraX32P; Mon, 10 Sep 2018 12:00:12 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Mon, 10 Sep 2018 09:00:09 +0000 Message-Id: <1536570009-6388-3-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536570009-6388-1-git-send-email-odpbot@yandex.ru> References: <1536570009-6388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 685 Subject: [lng-odp] [PATCH v4 2/2] linux-gen: ishm: make huge page cache size dynamic X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Josep Puigdemont Signed-off-by: Josep Puigdemont --- /** Email created from pull request 685 (joseppc:fix/cache_huge_pages) ** https://github.com/Linaro/odp/pull/685 ** Patch: https://github.com/Linaro/odp/pull/685.patch ** Base sha: 6d48d7f7f684b8aa87f7eb4f922d45be345ed771 ** Merge commit sha: 71e95a3d2ddb0df6ba6f0e3a62bf6aa98afbb035 **/ config/odp-linux-generic.conf | 2 +- platform/linux-generic/odp_ishm.c | 73 +++++++++++++++++++------------ 2 files changed, 46 insertions(+), 29 deletions(-) diff --git a/config/odp-linux-generic.conf b/config/odp-linux-generic.conf index 0dd2a6c13..318af0ad7 100644 --- a/config/odp-linux-generic.conf +++ b/config/odp-linux-generic.conf @@ -21,7 +21,7 @@ config_file_version = "0.0.1" # Internal shared memory allocator shm: { # ODP will try to reserve as many huge pages as the number indicated - # here, up to 64. A zero value means that no pages should be reserved. + # here. A zero value means that no pages should be reserved. # When using process mode threads, this value should be set to 0 # because the current implementation won't work properly otherwise. # These pages will only be freed when the application calls diff --git a/platform/linux-generic/odp_ishm.c b/platform/linux-generic/odp_ishm.c index aeda50bec..11fbe8ef0 100644 --- a/platform/linux-generic/odp_ishm.c +++ b/platform/linux-generic/odp_ishm.c @@ -239,15 +239,15 @@ typedef struct { } ishm_ftable_t; static ishm_ftable_t *ishm_ftbl; -#define HP_CACHE_SIZE 64 struct huge_page_cache { uint64_t len; + int max_fds; /* maximum amount requested of pre-allocated huge pages */ int total; /* amount of actually pre-allocated huge pages */ int idx; /* retrieve fd[idx] to get a free file descriptor */ - int fd[HP_CACHE_SIZE]; /* list of file descriptors */ + int fd[]; /* list of file descriptors */ }; -static struct huge_page_cache hpc; +static struct huge_page_cache *hpc; #ifndef MAP_ANONYMOUS #define MAP_ANONYMOUS MAP_ANON @@ -301,19 +301,14 @@ static void hp_init(void) char filename[ISHM_FILENAME_MAXLEN]; char dir[ISHM_FILENAME_MAXLEN]; int count; - - hpc.total = 0; - hpc.idx = -1; - hpc.len = odp_sys_huge_page_size(); + void *addr; if (!_odp_libconfig_lookup_ext_int("shm", NULL, "num_cached_hp", &count)) { return; } - if (count > HP_CACHE_SIZE) - count = HP_CACHE_SIZE; - else if (count <= 0) + if (count <= 0) return; ODP_DBG("Init HP cache with up to %d pages\n", count); @@ -339,55 +334,77 @@ static void hp_init(void) dir, odp_global_data.main_pid); + addr = mmap(NULL, + sizeof(struct huge_page_cache) + sizeof(int) * count, + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ODP_ERR("Unable to mmap memory for huge page cache\n."); + return; + } + + hpc = addr; + + hpc->max_fds = count; + hpc->total = 0; + hpc->idx = -1; + hpc->len = odp_sys_huge_page_size(); + for (int i = 0; i < count; ++i) { int fd; - fd = hp_create_file(hpc.len, filename); - if (fd == -1) + fd = hp_create_file(hpc->len, filename); + if (fd == -1) { + do { + hpc->fd[i++] = -1; + } while (i < count); break; - hpc.total++; - hpc.fd[i] = fd; + } + hpc->total++; + hpc->fd[i] = fd; } - hpc.idx = hpc.total - 1; + hpc->idx = hpc->total - 1; ODP_DBG("HP cache has %d huge pages of size 0x%08" PRIx64 "\n", - hpc.total, hpc.len); + hpc->total, hpc->len); } static void hp_term(void) { - for (int i = 0; i < hpc.total; i++) { - if (hpc.fd[i] != -1) - close(hpc.fd[i]); + if (NULL == hpc) + return; + + for (int i = 0; i < hpc->total; i++) { + if (hpc->fd[i] != -1) + close(hpc->fd[i]); } - hpc.total = 0; - hpc.idx = -1; - hpc.len = 0; + hpc->total = 0; + hpc->idx = -1; + hpc->len = 0; } static int hp_get_cached(uint64_t len) { int fd; - if (hpc.idx < 0 || len != hpc.len) + if (NULL == hpc || hpc->idx < 0 || len != hpc->len) return -1; - fd = hpc.fd[hpc.idx]; - hpc.fd[hpc.idx--] = -1; + fd = hpc->fd[hpc->idx]; + hpc->fd[hpc->idx--] = -1; return fd; } static int hp_put_cached(int fd) { - if (odp_unlikely(++hpc.idx >= hpc.total)) { - hpc.idx--; + if (NULL == hpc || odp_unlikely(++hpc->idx >= hpc->total)) { + hpc->idx--; ODP_ERR("Trying to put more FD than allowed: %d\n", fd); return -1; } - hpc.fd[hpc.idx] = fd; + hpc->fd[hpc->idx] = fd; return 0; }