From patchwork Tue Jun 27 08:45:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bogdan Pricope X-Patchwork-Id: 106406 Delivered-To: patch@linaro.org Received: by 10.140.101.48 with SMTP id t45csp964218qge; Tue, 27 Jun 2017 01:46:36 -0700 (PDT) X-Received: by 10.55.1.65 with SMTP id 62mr4757209qkb.210.1498553196645; Tue, 27 Jun 2017 01:46:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1498553196; cv=none; d=google.com; s=arc-20160816; b=mQ5eYH8iQVmz+H6cO9Zo8C5B7wfHmA+mTDINW9wimQv2WKTo4163+Z7chFu0sNhjgx soIVYZ5D0JMziDLuw9YphB0RjYOxZfw80sztumWW8koDp9izgLYQbWxTf6lYzOqcoHQf T8763U2CTeJxMHLfzPkbHPEu6ZViM2ZYYkjJ6rAlNi17Ac6UTvjjwr1Dfm/mb/eVSRJV CDAd/+vWF9TXhdbybQqWQoSw6gn7a2awYpOqsjje4ycg+Bb6LnXmVLiOUIFjqpbmdEyg Yb4LfoPb5LKwaJMLN/tFT/H21Ip2bz+S44iPnkk4+a5JfYSvt8YFw+CvTeVD91N+OHWm BWzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:delivered-to:arc-authentication-results; bh=xnpMHryMczbOOZv9WLbeZZTE+Zt7J0OR8Ql3FBGi5dY=; b=H5sjYqQmD4We2cMO2/3nmxFNAzbR4ey9GVvmZ81LkN2mv8HUlC4Xhf13mL/JAOj+W3 v+3opxe3y+EJep3sLWhxNYUU/sYdXnz/FabGTI3WecDlz42NaiL1SREoTLzjsDFaZCLA BS7QwPEU95d4gvcqf/sXdOFF7XUKZyzS7IBq/kzMVoo0FSDH9r3h3WF1uQOYe4RWpqii OAcxMWFzTX//Bb//T3jU7/8fNNMui3pA0QSzaKRL9hffge9A1a14er+bAqXytCZsSk3L 1Wzj9dH9Y98ZNkIe7iFUQEHdKMKdmQm+TXIywGffIyhJhCy9Ozz3J2V2NOEdpPuG2Uf2 7EGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id r190si2187732qkd.124.2017.06.27.01.46.36; Tue, 27 Jun 2017 01:46:36 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 5CC6560BED; Tue, 27 Jun 2017 08:46:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, URIBL_SBL, URIBL_SBL_A autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id D2FB060B54; Tue, 27 Jun 2017 08:46:27 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 3FB3960B64; Tue, 27 Jun 2017 08:46:25 +0000 (UTC) Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by lists.linaro.org (Postfix) with ESMTPS id 5EB7A60BE6 for ; Tue, 27 Jun 2017 08:46:00 +0000 (UTC) Received: by mail-wr0-f173.google.com with SMTP id r103so155704922wrb.0 for ; Tue, 27 Jun 2017 01:46:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xnpMHryMczbOOZv9WLbeZZTE+Zt7J0OR8Ql3FBGi5dY=; b=rzmpc4QHX3Nezgi7bSz/8p2Qlph7AwrLFbz8W9J7mMFODiw747nDHnj57c7ElO1Bb+ NL7qyn3ytdnBb6r0mGTNi9P6GuXcaELn+Yo2i4/T1QD62pmhSRtjGBoxa56QX/bTQtIU uCfuJsi7w/DQD6FkvlITGsBzcQpwXmIBGOBxePKGfnZg/0NI4yfk7ajqutfdR+a3Tr8u iZKTXCguvWLJWQKPWDolu4G5/aFG+cT3peCCIT+m5ZbEiPcfbGUmlK7+FKmB9iuWf5+j 1m9HtRaCnW/fA/nVDcNE/KZOnggta80UNv8ZBBoMx6KLQJWQAoF39UBMguAKSSNoOPlu wNLw== X-Gm-Message-State: AKS2vOxR1hHbut3hYqPDu1uNsUDM0u3oFeANTM+wmYEq1Q0PqTv5AkVR pk7EznOAeazrT8duyaLQW16g X-Received: by 10.223.148.66 with SMTP id 60mr16766766wrq.75.1498553158991; Tue, 27 Jun 2017 01:45:58 -0700 (PDT) Received: from netbricks-team.enea.se ([193.202.22.66]) by smtp.gmail.com with ESMTPSA id p68sm1780553wmf.21.2017.06.27.01.45.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Jun 2017 01:45:58 -0700 (PDT) From: Bogdan Pricope To: lng-odp@lists.linaro.org Date: Tue, 27 Jun 2017 11:45:46 +0300 Message-Id: <1498553146-24236-2-git-send-email-bogdan.pricope@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1498553146-24236-1-git-send-email-bogdan.pricope@linaro.org> References: <1498553146-24236-1-git-send-email-bogdan.pricope@linaro.org> Subject: [lng-odp] [RFC] ddf: example: build module with dpdk support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Signed-off-by: Bogdan Pricope --- example/ddf_app/odp_ddf_app.c | 14 +++- example/ddf_ifs/Makefile.am | 9 ++- example/ddf_ifs/ddf_ifs_enumr_dpdk.c | 126 ++++++++++++++++++++++++++++++++++- example/m4/configure.m4 | 2 + example/m4/example_dpdk.m4 | 57 ++++++++++++++++ scripts/build-example-ddf-dpdk | 33 +++++++++ 6 files changed, 237 insertions(+), 4 deletions(-) create mode 100644 example/m4/example_dpdk.m4 create mode 100755 scripts/build-example-ddf-dpdk -- 1.9.1 diff --git a/example/ddf_app/odp_ddf_app.c b/example/ddf_app/odp_ddf_app.c index af864ee..18c6994 100644 --- a/example/ddf_app/odp_ddf_app.c +++ b/example/ddf_app/odp_ddf_app.c @@ -20,9 +20,12 @@ int main(int argc, char *argv[]) { odp_instance_t instance; + odp_pktio_t pktio = ODP_PKTIO_INVALID; - (void)argc; - (void)argv; + if (argc == 1 || argc > 2) { + printf("Error: invalid parameter.\nUsage:\n\t%s \n", argv[0]); + exit(0); + } EXAMPLE_DBG("Start DDF Application...\n"); @@ -40,6 +43,13 @@ int main(int argc, char *argv[]) /* Print ddf objects*/ odpdrv_print_all(); + /* Open pktio*/ + /*odp_pktio_open();*/ + + /* Close pktio*/ + if (odp_pktio_close(pktio)) + EXAMPLE_ERR("Error: Failed to close pktio \"%s\".\n", argv[1]); + /* Terminate ODP */ odp_term_local(); odp_term_global(instance); diff --git a/example/ddf_ifs/Makefile.am b/example/ddf_ifs/Makefile.am index aa892ac..77e56e6 100644 --- a/example/ddf_ifs/Makefile.am +++ b/example/ddf_ifs/Makefile.am @@ -3,7 +3,14 @@ LIB = $(top_builddir)/lib AM_CPPFLAGS += -I$(srcdir) \ -I$(top_srcdir)/include \ -I$(top_srcdir)/platform/@with_platform@/include \ - -I$(top_srcdir)/platform/@with_platform@/arch/@ARCH_DIR@ + -I$(top_srcdir)/platform/@with_platform@/arch/@ARCH_DIR@ \ + -I$(top_srcdir)/include/odp/arch/@ARCH_ABI@ \ + -I$(top_builddir)/include + +if EXAMPLE_DPDK_SUPPORT +AM_CPPFLAGS += @EXAMPLE_DPDK_CPPFLAGS@ +AM_LDFLAGS += @EXAMPLE_DPDK_LDFLAGS@ +endif lib_LTLIBRARIES = $(LIB)/libddf_ifs.la diff --git a/example/ddf_ifs/ddf_ifs_enumr_dpdk.c b/example/ddf_ifs/ddf_ifs_enumr_dpdk.c index 782f7b7..25585dc 100644 --- a/example/ddf_ifs/ddf_ifs_enumr_dpdk.c +++ b/example/ddf_ifs/ddf_ifs_enumr_dpdk.c @@ -4,12 +4,27 @@ * SPDX-License-Identifier: BSD-3-Clause */ +#include + #include +#include +#include +#include + #include "odp_drv.h" #include "ddf_ifs_api.h" #include "ddf_ifs_enumr_dpdk.h" #include "ddf_ifs_dev_dpdk.h" +/*#include */ + +#include + +#include +#include +#include +#include + static odpdrv_enumr_t dpdk_enumr; #define TEST_DPDK_DEV_CNT 3 @@ -17,6 +32,106 @@ static odpdrv_enumr_t dpdk_enumr; static odpdrv_device_t dpdk_dev[DDF_DPDK_DEV_MAX]; static int dpdk_dev_cnt; + +#define DPDK_MEMORY_MB 512 +static int dpdk_pktio_init(void) +{ + int dpdk_argc; + int i; + odp_cpumask_t mask; + char mask_str[ODP_CPUMASK_STR_SIZE]; + const char *cmdline; + int32_t masklen; + int mem_str_len; + int cmd_len; + cpu_set_t original_cpuset; + struct rte_config *cfg; + + /** + * DPDK init changes the affinity of the calling thread, so after it + * returns the original affinity is restored. Only the first active + * core is passed to rte_eal_init(), as the rest would be used for + * DPDK's special lcore threads, which are only available through + * rte_eal_[mp_]remote_launch(), but not through ODP API's. + * Nevertheless, odp_local_init() makes sure for the rest of + * the DPDK libraries ODP threads look like proper DPDK threads. + */ + CPU_ZERO(&original_cpuset); + i = pthread_getaffinity_np(pthread_self(), + sizeof(original_cpuset), &original_cpuset); + if (i != 0) { + printf("Failed to read thread affinity: %d\n", i); + return -1; + } + + odp_cpumask_zero(&mask); + for (i = 0; i < CPU_SETSIZE; i++) { + if (CPU_ISSET(i, &original_cpuset)) { + odp_cpumask_set(&mask, i); + break; + } + } + masklen = odp_cpumask_to_str(&mask, mask_str, ODP_CPUMASK_STR_SIZE); + + if (masklen < 0) { + printf("CPU mask error: %d\n", masklen); + return -1; + } + + mem_str_len = snprintf(NULL, 0, "%d", DPDK_MEMORY_MB); + + cmdline = getenv("ODP_PKTIO_DPDK_PARAMS"); + if (cmdline == NULL) + cmdline = ""; + + /* masklen includes the terminating null as well */ + cmd_len = strlen("ddfdpdk -c -m ") + masklen + mem_str_len + + strlen(cmdline) + strlen(" "); + + char full_cmd[cmd_len]; + + /* first argument is facility log, simply bind it to odpdpdk for now.*/ + cmd_len = snprintf(full_cmd, cmd_len, "ddfdpdk -c %s -m %d %s", + mask_str, DPDK_MEMORY_MB, cmdline); + + for (i = 0, dpdk_argc = 1; i < cmd_len; ++i) { + if (isspace(full_cmd[i])) + ++dpdk_argc; + } + + char *dpdk_argv[dpdk_argc]; + + dpdk_argc = rte_strsplit(full_cmd, strlen(full_cmd), dpdk_argv, + dpdk_argc, ' '); + for (i = 0; i < dpdk_argc; ++i) + printf("arg[%d]: %s\n", i, dpdk_argv[i]); + + i = rte_eal_init(dpdk_argc, dpdk_argv); + + if (i < 0) { + printf("Cannot init the Intel DPDK EAL!\n"); + return -1; + } else if (i + 1 != dpdk_argc) { + printf("Some DPDK args were not processed!\n"); + printf("Passed: %d Consumed %d\n", dpdk_argc, i + 1); + } + printf("rte_eal_init OK\n"); + + rte_set_log_level(RTE_LOG_WARNING); + + i = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), + &original_cpuset); + if (i) + printf("Failed to reset thread affinity: %d\n", i); + + cfg = rte_eal_get_configuration(); + for (i = 0; i < RTE_MAX_LCORE; i++) + cfg->lcore_role[i] = ROLE_RTE; + + return 0; +} + + static int dpdk_enumr_probe(void) { int dpdk_dev_cnt_detected = TEST_DPDK_DEV_CNT; /* detected with @@ -24,7 +139,16 @@ static int dpdk_enumr_probe(void) char dev_addr[ODPDRV_NAME_ADDR_SZ]; int i; - printf("%s() - %d devices found\n", __func__, dpdk_dev_cnt_detected); + printf("%s()\n", __func__); + + if (dpdk_pktio_init()) { + printf("Failed to initialize dpdk\n"); + return -1; + } + + printf("(Real) Detected DPDK devices: %d\n", rte_eth_dev_count()); + + printf("%d devices found\n", dpdk_dev_cnt_detected); if (dpdk_dev_cnt_detected > DDF_DPDK_DEV_MAX) { dpdk_dev_cnt_detected = DDF_DPDK_DEV_MAX; diff --git a/example/m4/configure.m4 b/example/m4/configure.m4 index 9e7973b..ba23943 100644 --- a/example/m4/configure.m4 +++ b/example/m4/configure.m4 @@ -10,6 +10,8 @@ AC_ARG_ENABLE([test-example], test_example=no fi]) +m4_include([example/m4/example_dpdk.m4]) + AC_CONFIG_FILES([example/classifier/Makefile example/generator/Makefile example/hello/Makefile diff --git a/example/m4/example_dpdk.m4 b/example/m4/example_dpdk.m4 new file mode 100644 index 0000000..2aa2d6f --- /dev/null +++ b/example/m4/example_dpdk.m4 @@ -0,0 +1,57 @@ +########################################################################## +# Enable DPDK support +########################################################################## +example_dpdk_support=no +AC_ARG_WITH([example-dpdk-path], +AC_HELP_STRING([--with-example-dpdk-path=DIR path to dpdk build directory]), + [DPDK_PATH=$withval + example_dpdk_support=yes],[]) + +########################################################################## +# Save and set temporary compilation flags +########################################################################## +OLD_CPPFLAGS=$CPPFLAGS +CPPFLAGS="$AM_CPPFLAGS $CPPFLAGS" + +########################################################################## +# Check for DPDK availability +# +# DPDK pmd drivers are not linked unless the --whole-archive option is +# used. No spaces are allowed between the --whole-arhive flags. +########################################################################## +if test x$example_dpdk_support = xyes +then + EXAMPLE_DPDK_CPPFLAGS="-msse4.2 -isystem $DPDK_PATH/include" + + CPPFLAGS="$EXAMPLE_DPDK_CPPFLAGS $CPPFLAGS" + AC_CHECK_HEADERS([rte_config.h], [], + [AC_MSG_FAILURE(["can't find DPDK header"])]) + + DPDK_PMD=--whole-archive, + for filename in $with_example_dpdk_path/lib/*.a; do + cur_driver=`echo $(basename "$filename" .a) | \ + sed -n 's/^\(librte_pmd_\)/-lrte_pmd_/p' | sed -n 's/$/,/p'` + # rte_pmd_nfp has external dependencies which break linking + if test "$cur_driver" = "-lrte_pmd_nfp,"; then + echo "skip linking rte_pmd_nfp" + else + DPDK_PMD+=$cur_driver + fi + done + + DPDK_PMD+=--no-whole-archive + + EXAMPLE_DPDK_LDFLAGS="-L$DPDK_PATH/lib -Wl,$DPDK_PMD -ldpdk" +else + example_dpdk_support=no + EXAMPLE_DPDK_CPPFLAGS="" + EXAMPLE_DPDK_LDFLAGS="" +fi + +AM_CONDITIONAL([EXAMPLE_DPDK_SUPPORT], [test x$example_dpdk_support = xyes ]) +AC_SUBST(EXAMPLE_DPDK_CPPFLAGS) +AC_SUBST(EXAMPLE_DPDK_LDFLAGS) +########################################################################## +# Restore old saved variables +########################################################################## +CPPFLAGS=$OLD_CPPFLAGS diff --git a/scripts/build-example-ddf-dpdk b/scripts/build-example-ddf-dpdk new file mode 100755 index 0000000..dbcd49a --- /dev/null +++ b/scripts/build-example-ddf-dpdk @@ -0,0 +1,33 @@ +#!/bin/bash + +TARGET=${TARGET:-"x86_64-native-linuxapp-gcc"} + +export ROOT_DIR=$(readlink -e $(dirname $0) | sed 's|/scripts||') +pushd ${ROOT_DIR} + +echo '#include "pcap.h"' | cpp -H -o /dev/null 2>&1 +if [ "$?" != "0" ]; then + echo "Error: pcap is not installed. You may need to install libpcap-dev" +fi + +git -c advice.detachedHead=false clone -q --depth=1 --single-branch --branch=v17.02 http://dpdk.org/git/dpdk dpdk +pushd dpdk +git log --oneline --decorate + +#Make and edit DPDK configuration +make config T=${TARGET} O=${TARGET} +pushd ${TARGET} +#To use I/O without DPDK supported NIC's enable pcap pmd: +sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config +sed -ri 's,(CONFIG_RTE_BUILD_SHARED_LIB=).*,\1y,' .config +popd + +#Build DPDK +make install T=${TARGET} EXTRA_CFLAGS="-fPIC" +popd + +#Build ODP +./bootstrap; +./configure --enable-debug --enable-debug-print \ + --with-example-dpdk-path=`pwd`/dpdk/${TARGET} +make