From patchwork Thu Jun 29 14:09:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bogdan Pricope X-Patchwork-Id: 106649 Delivered-To: patch@linaro.org Received: by 10.182.135.102 with SMTP id pr6csp4757826obb; Thu, 29 Jun 2017 07:13:35 -0700 (PDT) X-Received: by 10.55.101.206 with SMTP id z197mr19434668qkb.207.1498745615854; Thu, 29 Jun 2017 07:13:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498745615; cv=none; d=google.com; s=arc-20160816; b=gzNQt7nHV9WjQWV5l079LDUKfG0DnHHaUfih1cT+TfIbKjiOxidIy2ad2Vrhv2xc/V /hp/H3rztc5vdw9Rv+LHwpvEc7qHgI6jNnXN76DFbLPGe+vD3b8n6GS2q1ib2J5faE4S YHqjo+PzNDmnkKkiczE5I1VZL62rZFnRz0rPygPjnkIJw6uFfBm/UWSq9k7Yng3vy1Xw ZReKjF+7ZiuZyfPceQXQM9mzMSHwCARvgCyv78JtVZAHEFFlzNbEYmGIeI1eNMQTDMgR 5umPgNKFSvplKPXWAy90GLt+Ih9VE+UeeDWNLi3/jWc8HAw9yhBiAIfqcvDXWLwbRhzW 3hVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:delivered-to:arc-authentication-results; bh=sXSXpoNZAkxZXUPr9ThyXCJYI6e4vWb7POFtonauO8k=; b=FyMw5sbJrYUKpmJ6WZ7k50sKoB+KgYrahVKas6s4QPx1etTTAt3yvhahNDhrJGVsex mftpiw/9QKx/ukw1hacBlg65c4rQdCJMehI8osZr+b0WGou3xgsI5ImwMaAkDd57v2vt uBvI1aIdxJeEM2n4QoZ/vn/nNOUUYGubVWZtlGC+MaEIL2aYjLYiObI9ChQezkE9JUYp imUt39aeM8VN4AKyLW+LVJydmo+K4xoUhCevOufI8xaOz67mnBCWgy/OZhCc2OT/N/rd 36ei7LPU6N2Hm7AyH8PukbwysyeOsJRI/2uya9BNR0g3fanJSvwA9EGQ9mimRjERe0au hNtQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id k13si5020607qtf.38.2017.06.29.07.13.35; Thu, 29 Jun 2017 07:13:35 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 857326067C; Thu, 29 Jun 2017 14:13:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RCVD_IN_SORBS_SPAM, URIBL_BLOCKED, URIBL_SBL,URIBL_SBL_A autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id F229662875; Thu, 29 Jun 2017 14:10:55 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 2FA4D607F9; Thu, 29 Jun 2017 14:10:26 +0000 (UTC) Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by lists.linaro.org (Postfix) with ESMTPS id DC48761DD5 for ; Thu, 29 Jun 2017 14:09:54 +0000 (UTC) Received: by mail-wm0-f44.google.com with SMTP id b184so16043163wme.1 for ; Thu, 29 Jun 2017 07:09:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sXSXpoNZAkxZXUPr9ThyXCJYI6e4vWb7POFtonauO8k=; b=EvXj67oguCoUjXajio8BHZW5VlSIPbjOvlooOKIeK0KxNRacwQV+xVCoaF99O1b/0I WWrJUnSNQjdI9lBZeHMxNUF1fzHoJ4IF7bsOlVCvx9bB+mrBI3mEYNGK934e7bCQ7bnW qO2hnlqXRTub/texR+MAo9tJ/PxzhhW0f5QOucASQgbGXMBE9wwyOhTGzBugSwNGc/Pf n0sv1KnQwS55Jq4Fspbbvarce53VAdcFRW3uLlv3k7e3ErrBQGUal2XiHFxdVPAkG16O mSm7L2ATdOyprOtXPn2Aco6qkh+vAN2ZQEPwF9hu6GxyDGoPmCjb/NduLKcezxyxQsmF g9zg== X-Gm-Message-State: AKS2vOwUK4PXYi+0XVwC5lePWGVCEBGmDMbBjYTNtxAavcJn4Z5EpCX+ PaWc6xagt0iWxTzc7CtTOfE3 X-Received: by 10.28.195.67 with SMTP id t64mr10729487wmf.41.1498745393421; Thu, 29 Jun 2017 07:09:53 -0700 (PDT) Received: from netbricks-team.enea.se ([193.202.22.66]) by smtp.gmail.com with ESMTPSA id 143sm1635863wmo.11.2017.06.29.07.09.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 29 Jun 2017 07:09:52 -0700 (PDT) From: Bogdan Pricope To: lng-odp@lists.linaro.org Date: Thu, 29 Jun 2017 17:09:37 +0300 Message-Id: <1498745377-2501-2-git-send-email-bogdan.pricope@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498745377-2501-1-git-send-email-bogdan.pricope@linaro.org> References: <1498745377-2501-1-git-send-email-bogdan.pricope@linaro.org> Subject: [lng-odp] [RFCv2] ddf: example: build module with dpdk support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Signed-off-by: Bogdan Pricope --- example/ddf_app/odp_ddf_app.c | 14 +++- example/ddf_ifs/Makefile.am | 9 ++- example/ddf_ifs/ddf_ifs_enumr_dpdk.c | 126 ++++++++++++++++++++++++++++++++++- example/m4/configure.m4 | 2 + example/m4/example_dpdk.m4 | 43 ++++++++++++ scripts/build-example-ddf-dpdk | 33 +++++++++ 6 files changed, 223 insertions(+), 4 deletions(-) create mode 100644 example/m4/example_dpdk.m4 create mode 100755 scripts/build-example-ddf-dpdk -- 1.9.1 diff --git a/example/ddf_app/odp_ddf_app.c b/example/ddf_app/odp_ddf_app.c index af864ee..18c6994 100644 --- a/example/ddf_app/odp_ddf_app.c +++ b/example/ddf_app/odp_ddf_app.c @@ -20,9 +20,12 @@ int main(int argc, char *argv[]) { odp_instance_t instance; + odp_pktio_t pktio = ODP_PKTIO_INVALID; - (void)argc; - (void)argv; + if (argc == 1 || argc > 2) { + printf("Error: invalid parameter.\nUsage:\n\t%s \n", argv[0]); + exit(0); + } EXAMPLE_DBG("Start DDF Application...\n"); @@ -40,6 +43,13 @@ int main(int argc, char *argv[]) /* Print ddf objects*/ odpdrv_print_all(); + /* Open pktio*/ + /*odp_pktio_open();*/ + + /* Close pktio*/ + if (odp_pktio_close(pktio)) + EXAMPLE_ERR("Error: Failed to close pktio \"%s\".\n", argv[1]); + /* Terminate ODP */ odp_term_local(); odp_term_global(instance); diff --git a/example/ddf_ifs/Makefile.am b/example/ddf_ifs/Makefile.am index aa892ac..77e56e6 100644 --- a/example/ddf_ifs/Makefile.am +++ b/example/ddf_ifs/Makefile.am @@ -3,7 +3,14 @@ LIB = $(top_builddir)/lib AM_CPPFLAGS += -I$(srcdir) \ -I$(top_srcdir)/include \ -I$(top_srcdir)/platform/@with_platform@/include \ - -I$(top_srcdir)/platform/@with_platform@/arch/@ARCH_DIR@ + -I$(top_srcdir)/platform/@with_platform@/arch/@ARCH_DIR@ \ + -I$(top_srcdir)/include/odp/arch/@ARCH_ABI@ \ + -I$(top_builddir)/include + +if EXAMPLE_DPDK_SUPPORT +AM_CPPFLAGS += @EXAMPLE_DPDK_CPPFLAGS@ +AM_LDFLAGS += @EXAMPLE_DPDK_LDFLAGS@ +endif lib_LTLIBRARIES = $(LIB)/libddf_ifs.la diff --git a/example/ddf_ifs/ddf_ifs_enumr_dpdk.c b/example/ddf_ifs/ddf_ifs_enumr_dpdk.c index 782f7b7..25585dc 100644 --- a/example/ddf_ifs/ddf_ifs_enumr_dpdk.c +++ b/example/ddf_ifs/ddf_ifs_enumr_dpdk.c @@ -4,12 +4,27 @@ * SPDX-License-Identifier: BSD-3-Clause */ +#include + #include +#include +#include +#include + #include "odp_drv.h" #include "ddf_ifs_api.h" #include "ddf_ifs_enumr_dpdk.h" #include "ddf_ifs_dev_dpdk.h" +/*#include */ + +#include + +#include +#include +#include +#include + static odpdrv_enumr_t dpdk_enumr; #define TEST_DPDK_DEV_CNT 3 @@ -17,6 +32,106 @@ static odpdrv_enumr_t dpdk_enumr; static odpdrv_device_t dpdk_dev[DDF_DPDK_DEV_MAX]; static int dpdk_dev_cnt; + +#define DPDK_MEMORY_MB 512 +static int dpdk_pktio_init(void) +{ + int dpdk_argc; + int i; + odp_cpumask_t mask; + char mask_str[ODP_CPUMASK_STR_SIZE]; + const char *cmdline; + int32_t masklen; + int mem_str_len; + int cmd_len; + cpu_set_t original_cpuset; + struct rte_config *cfg; + + /** + * DPDK init changes the affinity of the calling thread, so after it + * returns the original affinity is restored. Only the first active + * core is passed to rte_eal_init(), as the rest would be used for + * DPDK's special lcore threads, which are only available through + * rte_eal_[mp_]remote_launch(), but not through ODP API's. + * Nevertheless, odp_local_init() makes sure for the rest of + * the DPDK libraries ODP threads look like proper DPDK threads. + */ + CPU_ZERO(&original_cpuset); + i = pthread_getaffinity_np(pthread_self(), + sizeof(original_cpuset), &original_cpuset); + if (i != 0) { + printf("Failed to read thread affinity: %d\n", i); + return -1; + } + + odp_cpumask_zero(&mask); + for (i = 0; i < CPU_SETSIZE; i++) { + if (CPU_ISSET(i, &original_cpuset)) { + odp_cpumask_set(&mask, i); + break; + } + } + masklen = odp_cpumask_to_str(&mask, mask_str, ODP_CPUMASK_STR_SIZE); + + if (masklen < 0) { + printf("CPU mask error: %d\n", masklen); + return -1; + } + + mem_str_len = snprintf(NULL, 0, "%d", DPDK_MEMORY_MB); + + cmdline = getenv("ODP_PKTIO_DPDK_PARAMS"); + if (cmdline == NULL) + cmdline = ""; + + /* masklen includes the terminating null as well */ + cmd_len = strlen("ddfdpdk -c -m ") + masklen + mem_str_len + + strlen(cmdline) + strlen(" "); + + char full_cmd[cmd_len]; + + /* first argument is facility log, simply bind it to odpdpdk for now.*/ + cmd_len = snprintf(full_cmd, cmd_len, "ddfdpdk -c %s -m %d %s", + mask_str, DPDK_MEMORY_MB, cmdline); + + for (i = 0, dpdk_argc = 1; i < cmd_len; ++i) { + if (isspace(full_cmd[i])) + ++dpdk_argc; + } + + char *dpdk_argv[dpdk_argc]; + + dpdk_argc = rte_strsplit(full_cmd, strlen(full_cmd), dpdk_argv, + dpdk_argc, ' '); + for (i = 0; i < dpdk_argc; ++i) + printf("arg[%d]: %s\n", i, dpdk_argv[i]); + + i = rte_eal_init(dpdk_argc, dpdk_argv); + + if (i < 0) { + printf("Cannot init the Intel DPDK EAL!\n"); + return -1; + } else if (i + 1 != dpdk_argc) { + printf("Some DPDK args were not processed!\n"); + printf("Passed: %d Consumed %d\n", dpdk_argc, i + 1); + } + printf("rte_eal_init OK\n"); + + rte_set_log_level(RTE_LOG_WARNING); + + i = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), + &original_cpuset); + if (i) + printf("Failed to reset thread affinity: %d\n", i); + + cfg = rte_eal_get_configuration(); + for (i = 0; i < RTE_MAX_LCORE; i++) + cfg->lcore_role[i] = ROLE_RTE; + + return 0; +} + + static int dpdk_enumr_probe(void) { int dpdk_dev_cnt_detected = TEST_DPDK_DEV_CNT; /* detected with @@ -24,7 +139,16 @@ static int dpdk_enumr_probe(void) char dev_addr[ODPDRV_NAME_ADDR_SZ]; int i; - printf("%s() - %d devices found\n", __func__, dpdk_dev_cnt_detected); + printf("%s()\n", __func__); + + if (dpdk_pktio_init()) { + printf("Failed to initialize dpdk\n"); + return -1; + } + + printf("(Real) Detected DPDK devices: %d\n", rte_eth_dev_count()); + + printf("%d devices found\n", dpdk_dev_cnt_detected); if (dpdk_dev_cnt_detected > DDF_DPDK_DEV_MAX) { dpdk_dev_cnt_detected = DDF_DPDK_DEV_MAX; diff --git a/example/m4/configure.m4 b/example/m4/configure.m4 index 270aa89..bb2eb43 100644 --- a/example/m4/configure.m4 +++ b/example/m4/configure.m4 @@ -10,6 +10,8 @@ AC_ARG_ENABLE([test-example], test_example=no fi]) +m4_include([example/m4/example_dpdk.m4]) + AC_CONFIG_FILES([example/classifier/Makefile example/generator/Makefile example/hello/Makefile diff --git a/example/m4/example_dpdk.m4 b/example/m4/example_dpdk.m4 new file mode 100644 index 0000000..1a12187 --- /dev/null +++ b/example/m4/example_dpdk.m4 @@ -0,0 +1,43 @@ +########################################################################## +# Enable DPDK support +########################################################################## +example_dpdk_support=no +AC_ARG_WITH([example-dpdk-path], +AC_HELP_STRING([--with-example-dpdk-path=DIR path to dpdk build directory]), + [DPDK_PATH=$withval + example_dpdk_support=yes],[]) + +########################################################################## +# Save and set temporary compilation flags +########################################################################## +OLD_CPPFLAGS=$CPPFLAGS +CPPFLAGS="$AM_CPPFLAGS $CPPFLAGS" + +########################################################################## +# Check for DPDK availability +# +# DPDK pmd drivers are not linked unless the --whole-archive option is +# used. No spaces are allowed between the --whole-arhive flags. +########################################################################## +if test x$example_dpdk_support = xyes +then + EXAMPLE_DPDK_CPPFLAGS="-msse4.2 -isystem $DPDK_PATH/include" + + CPPFLAGS="$EXAMPLE_DPDK_CPPFLAGS $CPPFLAGS" + AC_CHECK_HEADERS([rte_config.h], [], + [AC_MSG_FAILURE(["can't find DPDK header"])]) + + EXAMPLE_DPDK_LDFLAGS="-L$DPDK_PATH/lib -Wl,--no-as-needed,-ldpdk,--as-needed -ldl -lm -lpcap" +else + example_dpdk_support=no + EXAMPLE_DPDK_CPPFLAGS="" + EXAMPLE_DPDK_LDFLAGS="" +fi + +AM_CONDITIONAL([EXAMPLE_DPDK_SUPPORT], [test x$example_dpdk_support = xyes ]) +AC_SUBST(EXAMPLE_DPDK_CPPFLAGS) +AC_SUBST(EXAMPLE_DPDK_LDFLAGS) +########################################################################## +# Restore old saved variables +########################################################################## +CPPFLAGS=$OLD_CPPFLAGS diff --git a/scripts/build-example-ddf-dpdk b/scripts/build-example-ddf-dpdk new file mode 100755 index 0000000..dbcd49a --- /dev/null +++ b/scripts/build-example-ddf-dpdk @@ -0,0 +1,33 @@ +#!/bin/bash + +TARGET=${TARGET:-"x86_64-native-linuxapp-gcc"} + +export ROOT_DIR=$(readlink -e $(dirname $0) | sed 's|/scripts||') +pushd ${ROOT_DIR} + +echo '#include "pcap.h"' | cpp -H -o /dev/null 2>&1 +if [ "$?" != "0" ]; then + echo "Error: pcap is not installed. You may need to install libpcap-dev" +fi + +git -c advice.detachedHead=false clone -q --depth=1 --single-branch --branch=v17.02 http://dpdk.org/git/dpdk dpdk +pushd dpdk +git log --oneline --decorate + +#Make and edit DPDK configuration +make config T=${TARGET} O=${TARGET} +pushd ${TARGET} +#To use I/O without DPDK supported NIC's enable pcap pmd: +sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config +sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config +popd + +#Build DPDK +make install T=${TARGET} EXTRA_CFLAGS="-fPIC" +popd + +#Build ODP +./bootstrap; +./configure --enable-debug --enable-debug-print \ + --with-example-dpdk-path=`pwd`/dpdk/${TARGET} +make