From patchwork Thu Nov 6 14:52:46 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ciprian Barbu X-Patchwork-Id: 40318 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f71.google.com (mail-wg0-f71.google.com [74.125.82.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2C49B24237 for ; Thu, 6 Nov 2014 14:53:31 +0000 (UTC) Received: by mail-wg0-f71.google.com with SMTP id b13sf744234wgh.10 for ; Thu, 06 Nov 2014 06:53:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:subject :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:errors-to:sender :x-original-sender:x-original-authentication-results:mailing-list :content-type:content-transfer-encoding; bh=Qh7njzYilq2trP8Fa3nmZJdGG7PLOl9xBjSAfkIcoUY=; b=gfOz9jwOKfuAPudsfja5vuDQJBGTte/nJTOFHLDQoOrtjp2orlp0z4RPd3kc2eZCMq MR3LpxpCn+D95FKRx9iuaeFkVqovd8KtDONHRmiD05r30bwv2tQr6knta5caKHAiG/bz IRj2usb0jjaJ5LuCn6KNrVgxrLMoW2x7/j/uVFUOjSB54hD2Flf3CwTT/gx8vghN5S+c NFrnGk/lS+DZtyhCelREahapLI7JLuYPmHH5Ff0Yem3BTv5c8Zm4lb9x9BTWFnQ0L+hd 46eroycUbS2/yi5ESs7O2ch38uxI1KOLhdgKi9QApkLy/XmjczRGpVVVJy/gk135Pnel bODw== X-Gm-Message-State: ALoCoQl4a1wt9nAtuDhDfZ07Pcc4qCKDJaDKuIYe1lLgKMAa4a0r/1nVxJ1PmU0YylA/JPK9QJ3V X-Received: by 10.112.159.199 with SMTP id xe7mr55098lbb.21.1415285610091; Thu, 06 Nov 2014 06:53:30 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.43.166 with SMTP id x6ls92289lal.93.gmail; Thu, 06 Nov 2014 06:53:29 -0800 (PST) X-Received: by 10.112.162.41 with SMTP id xx9mr5406732lbb.21.1415285609096; Thu, 06 Nov 2014 06:53:29 -0800 (PST) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com. [209.85.217.180]) by mx.google.com with ESMTPS id ra3si11818836lbb.24.2014.11.06.06.53.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Nov 2014 06:53:28 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) client-ip=209.85.217.180; Received: by mail-lb0-f180.google.com with SMTP id u10so1050962lbd.11 for ; Thu, 06 Nov 2014 06:53:28 -0800 (PST) X-Received: by 10.152.5.38 with SMTP id p6mr5493109lap.44.1415285608624; Thu, 06 Nov 2014 06:53:28 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.184.201 with SMTP id ew9csp55733lbc; Thu, 6 Nov 2014 06:53:12 -0800 (PST) X-Received: by 10.140.106.138 with SMTP id e10mr7135159qgf.71.1415285591928; Thu, 06 Nov 2014 06:53:11 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id h5si12189009qcm.23.2014.11.06.06.53.11 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 06 Nov 2014 06:53:11 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XmOQv-0003qS-6c; Thu, 06 Nov 2014 14:53:09 +0000 Received: from mail-lb0-f171.google.com ([209.85.217.171]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XmOQn-0003pm-U0 for lng-odp@lists.linaro.org; Thu, 06 Nov 2014 14:53:02 +0000 Received: by mail-lb0-f171.google.com with SMTP id b6so1027600lbj.30 for ; Thu, 06 Nov 2014 06:52:55 -0800 (PST) X-Received: by 10.152.45.1 with SMTP id i1mr5495633lam.7.1415285575210; Thu, 06 Nov 2014 06:52:55 -0800 (PST) Received: from cipriantemp.enea.se (sestofw01.enea.se. [192.36.1.252]) by mx.google.com with ESMTPSA id bk3sm2549721lbc.34.2014.11.06.06.52.54 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 06 Nov 2014 06:52:54 -0800 (PST) From: Ciprian Barbu To: lng-odp@lists.linaro.org Date: Thu, 6 Nov 2014 16:52:46 +0200 Message-Id: <1415285566-19069-1-git-send-email-ciprian.barbu@linaro.org> X-Mailer: git-send-email 1.8.3.2 X-Topics: patch Subject: [lng-odp] [PATCH NETMAP] Removed netmap specific example X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ciprian.barbu@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Ciprian Barbu --- This patch removes the packet_netmap example and adds instruction on how to use the regular packet example but with the added limitation of using a single thread. configure.ac | 1 - example/Makefile.am | 2 +- example/packet_netmap/Makefile.am | 8 - example/packet_netmap/odp_pktio_netmap.c | 587 ---------------------- platform/linux-netmap/README | 38 +- platform/linux-netmap/include/odp_packet_netmap.h | 66 +++ platform/linux-netmap/odp_packet_netmap.c | 132 ++--- platform/linux-netmap/odp_packet_netmap.h | 67 --- 8 files changed, 125 insertions(+), 776 deletions(-) delete mode 100644 example/packet_netmap/Makefile.am delete mode 100644 example/packet_netmap/odp_pktio_netmap.c create mode 100644 platform/linux-netmap/include/odp_packet_netmap.h delete mode 100644 platform/linux-netmap/odp_packet_netmap.h diff --git a/configure.ac b/configure.ac index a1c68ba..f8ea5a8 100644 --- a/configure.ac +++ b/configure.ac @@ -157,7 +157,6 @@ AC_CONFIG_FILES([Makefile example/l2fwd/Makefile example/odp_example/Makefile example/packet/Makefile - example/packet_netmap/Makefile example/timer/Makefile test/Makefile test/api_test/Makefile diff --git a/example/Makefile.am b/example/Makefile.am index 72663b9..b2a22a3 100644 --- a/example/Makefile.am +++ b/example/Makefile.am @@ -1 +1 @@ -SUBDIRS = generator ipsec l2fwd odp_example packet packet_netmap timer +SUBDIRS = generator ipsec l2fwd odp_example packet timer diff --git a/example/packet_netmap/Makefile.am b/example/packet_netmap/Makefile.am deleted file mode 100644 index 5a5913c..0000000 --- a/example/packet_netmap/Makefile.am +++ /dev/null @@ -1,8 +0,0 @@ -include $(top_srcdir)/example/Makefile.inc - -if ODP_NETMAP_ENABLED -bin_PROGRAMS = odp_pktio_netmap -odp_pktio_netmap_LDFLAGS = $(AM_LDFLAGS) -static -endif - -dist_odp_pktio_netmap_SOURCES = odp_pktio_netmap.c diff --git a/example/packet_netmap/odp_pktio_netmap.c b/example/packet_netmap/odp_pktio_netmap.c deleted file mode 100644 index a2756ee..0000000 --- a/example/packet_netmap/odp_pktio_netmap.c +++ /dev/null @@ -1,587 +0,0 @@ -/* Copyright (c) 2013, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -/** - * @file - * - * ODP basic packet IO loopback test application - */ - -#include -#include -#include -#include -#include - -#include -#include -#include - -#include -#include -#include -#include -#include -#include - -#include - -/** @def MAX_WORKERS - * @brief Maximum number of worker threads - */ -#define MAX_WORKERS 32 - -/** @def MAX_IFS - * @brief Maximum number of netmap interfaces - */ -#define MAX_IFS 16 - -/** @def SHM_PKT_POOL_SIZE - * @brief Size of the shared memory block - */ -#define SHM_PKT_POOL_SIZE (512*2048) - -/** @def SHM_PKT_POOL_BUF_SIZE - * @brief Buffer size of the packet pool buffer - */ -#define SHM_PKT_POOL_BUF_SIZE 1856 - -/** @def MAX_PKT_BURST - * @brief Maximum number of packet bursts - */ -#define MAX_PKT_BURST 16 - -/** @def PKTIO_MODE_SOCK - * @brief PKTIO is set in socket mode - */ -#define PKTIO_MODE_SOCK 0 - -/** @def PKTIO_MODE_NETMAP - * @brief PKTIO is set in netmap mode - */ -#define PKTIO_MODE_NETMAP 1 - -/** Get rid of path in filename - only for unix-type paths using '/' */ -#define NO_PATH(file_name) (strrchr((file_name), '/') ? \ - strrchr((file_name), '/') + 1 : (file_name)) - -/** - * Interface parameters obatained from app arguments - */ -typedef struct { - char if_name[32]; /**< Interface name */ - int pktio_mode; /**< Socket mode or netmap mode */ -} if_info_t; - -/** - * Parsed command line application arguments - */ -typedef struct { - int if_count; /**< Number of interfaces to be used */ - if_info_t *ifs; /**< Array of interface config options */ - odp_buffer_pool_t pool; /**< Buffer pool for packet IO */ -} appl_args_t; - -/** - * Thread specific arguments - * In this netmap example, there is a thread polling a network interface - * and another thread polling the ring that is used by the software stack - * to send packets to the same network interface. Each of the two threads - * needs to know which is the output queue corresponding to the other thread - * to be able to pass packets between the stack and the nic. This queue is - * defined by bridge_q below. - */ -typedef struct { - odp_pktio_t pktio; /**< ODP packet IO handler */ - odp_buffer_pool_t pool; /**< Buffer pool for packet IO */ - char *pktio_dev; /**< Interface name to use */ - int netmap_mode; /**< Either poll the hardware rings or the - rings associated with the host stack */ - odp_queue_t bridge_q; /**< Connect the network stack with the NIC */ -} pktio_info_t; - -/** - * Grouping of both parsed CL args and thread specific args - alloc together - */ -typedef struct { - /** Application (parsed) arguments */ - appl_args_t appl; - /** pktio entries: one for SW ring and one for HW ring */ - pktio_info_t pktios[2 * MAX_IFS]; - /** TODO: find a way to associate private data with pktios */ - /** Lookup table: find pktio_info_t based on pktio id */ - pktio_info_t *pktio_lt[ODP_CONFIG_PKTIO_ENTRIES]; -} args_t; - -/** Global pointer to args */ -static args_t *args; - -/* helper funcs */ -static int drop_err_pkts(odp_packet_t pkt_tbl[], unsigned len); -static void swap_pkt_addrs(odp_packet_t pkt_tbl[], unsigned len); -static void parse_args(int argc, char *argv[], appl_args_t *appl_args); -static void print_info(char *progname, appl_args_t *appl_args); -static void usage(char *progname); - -/** - * Packet IO loopback worker thread using ODP queues - * - * @param arg thread arguments of type 'thread_args_t *' - */ -static void *pktio_queue_thread(void *arg) -{ - int thr; - odp_buffer_pool_t pkt_pool; - odp_packet_t pkt; - odp_buffer_t buf; - unsigned long pkt_cnt = 0; - unsigned long err_cnt = 0; - - (void)arg; - - thr = odp_thread_id(); - printf("Pktio thread [%02i] starts\n", thr); - - /* Lookup the packet pool */ - pkt_pool = odp_buffer_pool_lookup("packet_pool"); - if (pkt_pool == ODP_BUFFER_POOL_INVALID) { - ODP_ERR(" [%02i] Error: pkt_pool not found\n", thr); - return NULL; - } - - /* Loop packets */ - for (;;) { - odp_pktio_t pktio_tmp; - odp_queue_t outq_def; - pktio_info_t *pktio_info; - - /* Use schedule to get buf from any input queue */ - buf = odp_schedule(NULL, ODP_SCHED_WAIT); - - pkt = odp_packet_from_buffer(buf); - - /* Drop packets with errors */ - if (odp_unlikely(drop_err_pkts(&pkt, 1) == 0)) { - ODP_ERR("Drop frame - err_cnt:%lu\n", ++err_cnt); - continue; - } - - pktio_tmp = odp_pktio_get_input(pkt); - if (pktio_tmp == ODP_PKTIO_INVALID) { - ODP_ERR("[%02i] Error: invalid pktio\n", thr); - return NULL; - } - - outq_def = odp_pktio_outq_getdef(pktio_tmp); - - if (outq_def == ODP_QUEUE_INVALID) { - ODP_ERR(" [%02i] Error: def output-Q query\n", - thr); - return NULL; - } - - /* Lookup the thread associated with the entry */ - pktio_info = args->pktio_lt[pktio_tmp]; - - /* Send back packets arrived on physical interface */ - if (pktio_info->netmap_mode == ODP_NETMAP_RING_HW) { - odp_packet_t pkt_copy; - - pkt_copy = odph_packet_alloc(pkt_pool); - - if (odp_packet_copy(pkt_copy, pkt) != 0) { - ODP_ERR("Packet copy failed!\n"); - odph_packet_free(pkt_copy); - } else { - swap_pkt_addrs(&pkt_copy, 1); - odp_queue_enq(outq_def, - odp_buffer_from_packet(pkt_copy)); - } - } - - odp_queue_enq(pktio_info->bridge_q, buf); - - /* Print packet counts every once in a while */ - if (odp_unlikely(pkt_cnt++ % 100000 == 0)) { - printf(" [%02i] pkt_cnt:%lu\n", thr, pkt_cnt); - fflush(NULL); - } - } - -/* unreachable */ -} - -/** - * ODP packet example main function - */ -int main(int argc, char *argv[]) -{ - odph_linux_pthread_t thread_tbl[MAX_WORKERS]; - odp_buffer_pool_t pool; - int thr_id; - int num_workers; - void *pool_base; - odp_shm_t shm; - int i; - - /* Init ODP before calling anything else */ - if (odp_init_global()) { - ODP_ERR("Error: ODP global init failed.\n"); - exit(EXIT_FAILURE); - } - - /* Reserve memory for args from shared mem */ - shm = odp_shm_reserve("shm_args", sizeof(args_t), - ODP_CACHE_LINE_SIZE, 0); - args = odp_shm_addr(shm); - if (args == NULL) { - ODP_ERR("Error: shared mem alloc failed.\n"); - exit(EXIT_FAILURE); - } - memset(args, 0, sizeof(*args)); - - /* Parse and store the application arguments */ - parse_args(argc, argv, &args->appl); - - /* Print both system and application information */ - print_info(NO_PATH(argv[0]), &args->appl); - - num_workers = odp_sys_core_count(); - if (num_workers > MAX_WORKERS) - num_workers = MAX_WORKERS; - - /* Init this thread */ - thr_id = odp_thread_create(0); - odp_init_local(thr_id); - - /* Create packet pool */ - shm = odp_shm_reserve("shm_packet_pool", - SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0); - pool_base = odp_shm_addr(shm); - if (pool_base == NULL) { - ODP_ERR("Error: packet pool mem alloc failed.\n"); - exit(EXIT_FAILURE); - } - - pool = odp_buffer_pool_create("packet_pool", pool_base, - SHM_PKT_POOL_SIZE, - SHM_PKT_POOL_BUF_SIZE, - ODP_CACHE_LINE_SIZE, - ODP_BUFFER_TYPE_PACKET); - if (pool == ODP_BUFFER_POOL_INVALID) { - ODP_ERR("Error: packet pool create failed.\n"); - exit(EXIT_FAILURE); - } - odp_buffer_pool_print(pool); - - for (i = 0; i < 2 * args->appl.if_count; ++i) { - odp_pktio_params_t params; - netmap_params_t *nm_params = ¶ms.nm_params; - char inq_name[ODP_QUEUE_NAME_LEN]; - odp_queue_t inq_def; - odp_queue_param_t qparam; - odp_pktio_t pktio; - int ret; - - /* Create a pktio polling the hardware rings and one that polls - * the software ring associated with the physical interface - */ - - args->pktios[i].pktio_dev = args->appl.ifs[i / 2].if_name; - memset(nm_params, 0, sizeof(*nm_params)); - nm_params->type = ODP_PKTIO_TYPE_NETMAP; - if (i % 2) { - nm_params->netmap_mode = ODP_NETMAP_RING_SW; - nm_params->ringid = 0; - } else { - nm_params->netmap_mode = ODP_NETMAP_RING_HW; - nm_params->ringid = 0; - } - pktio = odp_pktio_open(args->pktios[i].pktio_dev, - pool, ¶ms); - /* Open a packet IO instance for this thread */ - if (pktio == ODP_PKTIO_INVALID) { - ODP_ERR(" [%02i] Err: pktio create\n", i); - return -1; - } - - args->pktios[i].pktio = pktio; - args->pktios[i].pool = pool; - args->pktios[i].netmap_mode = nm_params->netmap_mode; - /* Save pktio_info in the lookup table */ - args->pktio_lt[pktio] = &args->pktios[i]; - /* - * Create and set the default INPUT queue associated with the - * 'pktio' resource - */ - qparam.sched.prio = ODP_SCHED_PRIO_DEFAULT; - qparam.sched.sync = ODP_SCHED_SYNC_NONE; - qparam.sched.group = ODP_SCHED_GROUP_DEFAULT; - snprintf(inq_name, sizeof(inq_name), "%i-pktio_inq_def", - (int)pktio); - inq_name[ODP_QUEUE_NAME_LEN - 1] = '\0'; - - inq_def = odp_queue_create(inq_name, ODP_QUEUE_TYPE_PKTIN, - &qparam); - if (inq_def == ODP_QUEUE_INVALID) { - ODP_ERR(" [%02i] Err: pktio q create\n", i); - return -1; - } - - ret = odp_pktio_inq_setdef(pktio, inq_def); - if (ret != 0) { - ODP_ERR(" [%02i] Err: default input-Q setup\n" - , i); - return -1; - } - - printf(" [%02i] created pktio:%02i, queue mode\n" - " default pktio%02i-INPUT queue:%u\n", - i, pktio, pktio, inq_def); - - /* Prepare for bridging: set bridge_q queue ids */ - if (i % 2) { - odp_pktio_t pktio_bridge; - odp_queue_t outq_def; - - pktio_bridge = args->pktios[i-1].pktio; - outq_def = odp_pktio_outq_getdef(pktio_bridge); - args->pktios[i].bridge_q = outq_def; - - pktio_bridge = args->pktios[i].pktio; - outq_def = odp_pktio_outq_getdef(pktio_bridge); - args->pktios[i-1].bridge_q = outq_def; - } - } - - memset(thread_tbl, 0, sizeof(thread_tbl)); - for (i = 0; i < num_workers; ++i) { - - /* - * Create threads one-by-one instead of all-at-once, - * because each thread might get different arguments - */ - odph_linux_pthread_create(&thread_tbl[i], 1, i, - pktio_queue_thread, NULL); - } - - /* Master thread waits for other threads to exit */ - odph_linux_pthread_join(thread_tbl, num_workers); - - printf("Exit\n\n"); - - return 0; -} - -/** - * Drop packets which input parsing marked as containing errors. - * - * Frees packets with error and modifies pkt_tbl[] to only contain packets with - * no detected errors. - * - * @param pkt_tbl Array of packet - * @param len Length of pkt_tbl[] - * - * @return Number of packets with no detected error - */ -static int drop_err_pkts(odp_packet_t pkt_tbl[], unsigned len) -{ - odp_packet_t pkt; - unsigned pkt_cnt = len; - unsigned i, j; - - for (i = 0, j = 0; i < len; ++i) { - pkt = pkt_tbl[i]; - - if (odp_unlikely(odp_packet_error(pkt))) { - odph_packet_free(pkt); /* Drop */ - pkt_cnt--; - } else if (odp_unlikely(i != j++)) { - pkt_tbl[j-1] = pkt; - } - } - - return pkt_cnt; -} - -/** - * Swap eth src<->dst and IP src<->dst addresses - * - * @param pkt_tbl Array of packets - * @param len Length of pkt_tbl[] - */ -static void swap_pkt_addrs(odp_packet_t pkt_tbl[], unsigned len) -{ - odp_packet_t pkt; - odph_ethhdr_t *eth; - odph_ethaddr_t tmp_addr; - odph_ipv4hdr_t *ip; - uint32be_t ip_tmp_addr; /* tmp ip addr */ - unsigned i; - - for (i = 0; i < len; ++i) { - pkt = pkt_tbl[i]; - if (odp_packet_inflag_eth(pkt)) { - eth = (odph_ethhdr_t *)odp_packet_l2(pkt); - - tmp_addr = eth->dst; - eth->dst = eth->src; - eth->src = tmp_addr; - - if (odp_packet_inflag_ipv4(pkt)) { - /* IPv4 */ - ip = (odph_ipv4hdr_t *)odp_packet_l3(pkt); - - ip_tmp_addr = ip->src_addr; - ip->src_addr = ip->dst_addr; - ip->dst_addr = ip_tmp_addr; - } - } - } -} - -/** - * Parse and store the command line arguments - * - * @param argc argument count - * @param argv[] argument vector - * @param appl_args Store application arguments here - */ -static void parse_args(int argc, char *argv[], appl_args_t *appl_args) -{ - int opt; - int long_index; - char *names, *str, *token, *save; - size_t len; - int i; - static struct option longopts[] = { - {"interface", required_argument, NULL, 'i'}, /* return 'i' */ - {"help", no_argument, NULL, 'h'}, /* return 'h' */ - {NULL, 0, NULL, 0} - }; - - while (1) { - opt = getopt_long(argc, argv, "+i:h", longopts, &long_index); - - if (opt == -1) - break; /* No more options */ - - switch (opt) { - /* parse packet-io interface names */ - case 'i': - len = strlen(optarg); - if (len == 0) { - usage(argv[0]); - exit(EXIT_FAILURE); - } - len += 1; /* add room for '\0' */ - - names = malloc(len); - if (names == NULL) { - usage(argv[0]); - exit(EXIT_FAILURE); - } - - /* count the number of tokens separated by ',' */ - strcpy(names, optarg); - for (str = names, i = 0;; str = NULL, i++) { - token = strtok_r(str, ",", &save); - if (token == NULL) - break; - } - appl_args->if_count = i; - - if (appl_args->if_count == 0) { - usage(argv[0]); - exit(EXIT_FAILURE); - } - - /* allocate storage for the if names */ - appl_args->ifs = - calloc(appl_args->if_count, sizeof(if_info_t)); - - /* store the if names (reset names string) */ - strcpy(names, optarg); - for (str = names, i = 0;; str = NULL, i++) { - token = strtok_r(str, ",", &save); - if (token == NULL) - break; - strncpy(appl_args->ifs[i].if_name, token, - sizeof(appl_args->ifs[i].if_name)); - appl_args->ifs[i].pktio_mode = - PKTIO_MODE_NETMAP; - } - break; - - case 'h': - usage(argv[0]); - exit(EXIT_SUCCESS); - break; - - default: - break; - } - } - - if (appl_args->if_count == 0) { - usage(argv[0]); - exit(EXIT_FAILURE); - } - - optind = 1; /* reset 'extern optind' from the getopt lib */ -} - -/** - * Print system and application info - */ -static void print_info(char *progname, appl_args_t *appl_args) -{ - int i; - - printf("\n" - "ODP system info\n" - "---------------\n" - "ODP API version: %s\n" - "CPU model: %s\n" - "CPU freq (hz): %"PRIu64"\n" - "Cache line size: %i\n" - "Core count: %i\n" - "\n", - odp_version_api_str(), odp_sys_cpu_model_str(), odp_sys_cpu_hz(), - odp_sys_cache_line_size(), odp_sys_core_count() - ); - printf("Running ODP appl: \"%s\"\n" - "-----------------\n" - "IF-count: %i\n" - "Using IFs: ", - progname, appl_args->if_count); - for (i = 0; i < appl_args->if_count; ++i) - printf(" %s", appl_args->ifs[i].if_name); - printf("\n" - "Mode: "); - printf("\n\n"); - fflush(NULL); -} - -/** - * Prinf usage information - */ -static void usage(char *progname) -{ - printf("\n" - "Usage: %s OPTIONS\n" - " E.g. %s -i eth1,eth2,eth3\n" - "\n" - "OpenDataPlane example application.\n" - "\n" - "Mandatory OPTIONS:\n" - " -i, --interface Eth interfaces (comma-separated, no spaces)\n" - "\n" - "Optional OPTIONS\n" - " -h, --help Display help and exit.\n" - "\n", NO_PATH(progname), NO_PATH(progname) - ); -} diff --git a/platform/linux-netmap/README b/platform/linux-netmap/README index 42b7595..afe491e 100644 --- a/platform/linux-netmap/README +++ b/platform/linux-netmap/README @@ -82,21 +82,23 @@ To configure ODP for linux-generic: make -3. Running the example application -================================== - -The example application for netmap-linux is odp_pktio_netmap. The main purpose -of the example application is to show how to implement a simple packet loopback -application using ODP. The example also implements a bridge between the Linux -IP stack and the physical interfaces since netmap disrupts the communication -between the two. The bridging is achieved by passing packets between the -"software" ring attached to the host stack and the physical rings of the NIC. -More information on netmap architecture and software rings can be found in the -"netmap: a novel framework for fast packet I/O" paper by Luigi Rizzo on his -research page: http://info.iet.unipi.it/~luigi/research.html - - sudo ./odp_pktio_netmap -i eth0 -m 1 - -From another machine you can simply run ping and observe the duplicate ICMP -replies. Also the host running the example application should still have network -connectivity due to the bridging performed as explained above. +3. Running the packet loopback example +====================================== + +ODP has a packet example that works across platforms. It is a simple example +that receives packets and sends them back to the interface they originated, +exchanging the sender and receiver in l2 and l3. + +There is a limitation though, if more than one thread is used there will be a +race condition because multiple threads will try to receive frames from the +same interface at the same time. + + cd example + sudo ./odp_pktio -i eth0 -m 1 -t 4 -c 1 + +Note the netmap disrupts the connection between the NIC and the host stack. +To do a quick test however you can try pinging the machine running the example +from another machine connected to it. You will probably need to manually add an +ARP entry on the other machine. Then ping will work because the looped back ICMP +packets will be interpreted as response. You can inspect the content with +tcpdump and observe the only thing different will be swapped MAC addresses. diff --git a/platform/linux-netmap/include/odp_packet_netmap.h b/platform/linux-netmap/include/odp_packet_netmap.h new file mode 100644 index 0000000..84b9b18 --- /dev/null +++ b/platform/linux-netmap/include/odp_packet_netmap.h @@ -0,0 +1,66 @@ +/* Copyright (c) 2013, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef ODP_PACKET_NETMAP_H +#define ODP_PACKET_NETMAP_H + +#include + +#include +#include +#include + +#include +#include +#include +#include + +#include + +#define ODP_NETMAP_RING_HW 0 +#define ODP_NETMAP_RING_SW 1 + +#define NETMAP_BLOCKING_IO + +/** Packet socket using netmap mmaped rings for both Rx and Tx */ +typedef struct { + odp_buffer_pool_t pool; + size_t max_frame_len; /**< max frame len = buf_size - sizeof(pkt_hdr) */ + size_t frame_offset; /**< frame start offset from start of pkt buf */ + size_t buf_size; /**< size of buffer payload in 'pool' */ + struct nm_desc_t *nm_desc; + uint32_t begin; + uint32_t end; + struct netmap_ring *rxring; + struct netmap_ring *txring; + odp_queue_t tx_access; /* Used for exclusive access to send packets */ + uint32_t if_flags; + char ifname[32]; +} pkt_netmap_t; + +/** + * Configure an interface to work in netmap mode + */ +int setup_pkt_netmap(pkt_netmap_t * const pkt_nm, const char *netdev, + odp_buffer_pool_t pool, netmap_params_t *nm_params); + +/** + * Switch interface from netmap mode to normal mode + */ +int close_pkt_netmap(pkt_netmap_t * const pkt_nm); + +/** + * Receive packets using netmap + */ +int recv_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], + unsigned len); + +/** + * Send packets using netmap + */ +int send_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], + unsigned len); +#endif diff --git a/platform/linux-netmap/odp_packet_netmap.c b/platform/linux-netmap/odp_packet_netmap.c index 6b9b120..23970a5 100644 --- a/platform/linux-netmap/odp_packet_netmap.c +++ b/platform/linux-netmap/odp_packet_netmap.c @@ -105,14 +105,12 @@ done: } int setup_pkt_netmap(pkt_netmap_t * const pkt_nm, const char *netdev, - odp_buffer_pool_t pool, netmap_params_t *nm_params) + odp_buffer_pool_t pool, + netmap_params_t *nm_params ODP_UNUSED) { - char qname[ODP_QUEUE_NAME_LEN]; char ifname[32]; odp_packet_t pkt; - odp_buffer_t token; uint8_t *pkt_buf; - uint16_t ringid; uint8_t *l2_hdr; int ret; @@ -132,98 +130,55 @@ int setup_pkt_netmap(pkt_netmap_t * const pkt_nm, const char *netdev, pkt_nm->buf_size = odph_packet_buf_size(pkt); /* max frame len taking into account the l2-offset */ pkt_nm->max_frame_len = pkt_nm->buf_size - pkt_nm->frame_offset; - /* save netmap_mode for later use */ - pkt_nm->netmap_mode = nm_params->netmap_mode; odph_packet_free(pkt); - if (nm_params->netmap_mode == ODP_NETMAP_RING_SW) - ringid = NETMAP_SW_RING; - else - ringid = nm_params->ringid; - strncpy(pkt_nm->ifname, netdev, sizeof(pkt_nm->ifname)); snprintf(ifname, sizeof(ifname), "netmap:%s", netdev); - pkt_nm->nm_desc = nm_open(ifname, NULL, ringid, 0); + pkt_nm->nm_desc = nm_open(ifname, NULL, 0, 0); if (pkt_nm->nm_desc == NULL) { ODP_ERR("Error opening nm interface: %s\n", strerror(errno)); return -1; } - ODP_DBG("thread %d mode %s mmap addr %p\n", + ODP_DBG("thread %d mmap addr %p\n", odp_thread_id(), - nm_params->netmap_mode == ODP_NETMAP_RING_SW ? "SW" : "HW", pkt_nm->nm_desc->mem); - if (nm_params->netmap_mode == ODP_NETMAP_RING_SW) { - pkt_nm->begin = pkt_nm->nm_desc->req.nr_rx_rings; - pkt_nm->end = pkt_nm->begin + 1; - pkt_nm->rxring = NETMAP_RXRING(pkt_nm->nm_desc->nifp, - pkt_nm->nm_desc->req.nr_rx_rings); - pkt_nm->txring = NETMAP_TXRING(pkt_nm->nm_desc->nifp, - pkt_nm->nm_desc->req.nr_tx_rings); - } else if (nm_params->ringid & NETMAP_HW_RING) { - pkt_nm->begin = nm_params->ringid & NETMAP_RING_MASK; - pkt_nm->end = pkt_nm->begin + 1; - pkt_nm->rxring = NETMAP_RXRING(pkt_nm->nm_desc->nifp, - pkt_nm->begin); - pkt_nm->txring = NETMAP_TXRING(pkt_nm->nm_desc->nifp, - pkt_nm->begin); - } else { - pkt_nm->begin = 0; - pkt_nm->end = pkt_nm->nm_desc->req.nr_rx_rings; - pkt_nm->rxring = NETMAP_RXRING(pkt_nm->nm_desc->nifp, 0); - pkt_nm->txring = NETMAP_TXRING(pkt_nm->nm_desc->nifp, 0); + pkt_nm->begin = 0; + pkt_nm->end = pkt_nm->nm_desc->req.nr_rx_rings; + pkt_nm->rxring = NETMAP_RXRING(pkt_nm->nm_desc->nifp, 0); + pkt_nm->txring = NETMAP_TXRING(pkt_nm->nm_desc->nifp, 0); + + ret = nm_do_ioctl(pkt_nm, SIOCGIFFLAGS, 0); + if (ret) + return ret; + if ((pkt_nm->if_flags & IFF_UP) == 0) { + ODP_DBG("%s is down, bringing up...\n", pkt_nm->ifname); + pkt_nm->if_flags |= IFF_UP; } - - /* Set TX checksumming if hardware rings */ - if (nm_params->netmap_mode == ODP_NETMAP_RING_HW) { - ret = nm_do_ioctl(pkt_nm, SIOCGIFFLAGS, 0); - if (ret) - return ret; - if ((pkt_nm->if_flags & IFF_UP) == 0) { - ODP_DBG("%s is down, bringing up...\n", pkt_nm->ifname); - pkt_nm->if_flags |= IFF_UP; - } - if (ETH_PROMISC) { - pkt_nm->if_flags |= IFF_PROMISC; - nm_do_ioctl(pkt_nm, SIOCSIFFLAGS, 0); - } - ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_SGSO); - if (ret) - ODP_DBG("ETHTOOL_SGSO not supported\n"); - - ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_STSO); - if (ret) - ODP_DBG("ETHTOOL_STSO not supported\n"); - /* TODO: This seems to cause the app to not receive frames - * first time it is launched after netmap driver is inserted. - * Should be investigated further. - */ - /* - nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_SRXCSUM); - */ - ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_STXCSUM); - if (ret) - ODP_DBG("ETHTOOL_STXCSUM not supported\n"); + if (ETH_PROMISC) { + pkt_nm->if_flags |= IFF_PROMISC; + nm_do_ioctl(pkt_nm, SIOCSIFFLAGS, 0); } - - /* Set up the TX access queue */ - snprintf(qname, sizeof(qname), "%s:%s-pktio_tx_access", netdev, - nm_params->netmap_mode == ODP_NETMAP_RING_SW ? "SW" : "HW"); - pkt_nm->tx_access = odp_queue_create(qname, ODP_QUEUE_TYPE_POLL, NULL); - if (pkt_nm->tx_access == ODP_QUEUE_INVALID) { - ODP_ERR("Error: pktio queue creation failed\n"); - return -1; - } - token = odp_buffer_alloc(pool); - if (!odp_buffer_is_valid(token)) { - ODP_ERR("Error: token creation failed\n"); - return -1; - } - - odp_queue_enq(pkt_nm->tx_access, token); + ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_SGSO); + if (ret) + ODP_DBG("ETHTOOL_SGSO not supported\n"); + + ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_STSO); + if (ret) + ODP_DBG("ETHTOOL_STSO not supported\n"); + /* TODO: This seems to cause the app to not receive frames + * first time it is launched after netmap driver is inserted. + * Should be investigated further. + */ + /* + nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_SRXCSUM); + */ + ret = nm_do_ioctl(pkt_nm, SIOCETHTOOL, ETHTOOL_STXCSUM); + if (ret) + ODP_DBG("ETHTOOL_STXCSUM not supported\n"); ODP_DBG("Wait for link to come up\n"); sleep(WAITLINK_TMO); @@ -323,15 +278,9 @@ int recv_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], continue; } if (odp_unlikely(frame_len < ODPH_ETH_LEN_MIN)) { - if (odp_unlikely(pkt_nm->netmap_mode != - ODP_NETMAP_RING_SW)) { - ODP_ERR("RX: Frame truncated: %u\n", - (unsigned)frame_len); - continue; - } - memset(l2_hdr + frame_len, 0, - ODPH_ETH_LEN_MIN - frame_len); - frame_len = ODPH_ETH_LEN_MIN; + ODP_ERR("RX: Frame truncated: %u\n", + (unsigned)frame_len); + continue; } /* For now copy the data in the mbuf, @@ -369,7 +318,6 @@ int send_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], uint32_t limit, tx; uint32_t ringid = pkt_nm->begin; odp_packet_t pkt; - odp_buffer_t token; #ifdef NETMAP_BLOCKING_IO struct pollfd fds[2]; @@ -382,8 +330,6 @@ int send_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], fds[0].events = POLLOUT; #endif - token = odp_queue_deq(pkt_nm->tx_access); - while (nb_tx < len) { #ifdef NETMAP_BLOCKING_IO ret = poll(&fds[0], 1, POLL_TMO); @@ -437,8 +383,6 @@ int send_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], } } - odp_queue_enq(pkt_nm->tx_access, token); - #ifndef NETMAP_BLOCKING_IO ioctl(fd, NIOCTXSYNC, NULL); #endif diff --git a/platform/linux-netmap/odp_packet_netmap.h b/platform/linux-netmap/odp_packet_netmap.h deleted file mode 100644 index 4b66d3d..0000000 --- a/platform/linux-netmap/odp_packet_netmap.h +++ /dev/null @@ -1,67 +0,0 @@ -/* Copyright (c) 2013, Linaro Limited - * All rights reserved. - * - * SPDX-License-Identifier: BSD-3-Clause - */ - -#ifndef ODP_PACKET_NETMAP_H -#define ODP_PACKET_NETMAP_H - -#include - -#include -#include -#include - -#include -#include -#include -#include - -#include - -#define ODP_NETMAP_RING_HW 0 -#define ODP_NETMAP_RING_SW 1 - -#define NETMAP_BLOCKING_IO - -/** Packet socket using netmap mmaped rings for both Rx and Tx */ -typedef struct { - odp_buffer_pool_t pool; - size_t max_frame_len; /**< max frame len = buf_size - sizeof(pkt_hdr) */ - size_t frame_offset; /**< frame start offset from start of pkt buf */ - size_t buf_size; /**< size of buffer payload in 'pool' */ - int netmap_mode; - struct nm_desc_t *nm_desc; - uint32_t begin; - uint32_t end; - struct netmap_ring *rxring; - struct netmap_ring *txring; - odp_queue_t tx_access; /* Used for exclusive access to send packets */ - uint32_t if_flags; - char ifname[32]; -} pkt_netmap_t; - -/** - * Configure an interface to work in netmap mode - */ -int setup_pkt_netmap(pkt_netmap_t * const pkt_nm, const char *netdev, - odp_buffer_pool_t pool, netmap_params_t *nm_params); - -/** - * Switch interface from netmap mode to normal mode - */ -int close_pkt_netmap(pkt_netmap_t * const pkt_nm); - -/** - * Receive packets using netmap - */ -int recv_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], - unsigned len); - -/** - * Send packets using netmap - */ -int send_pkt_netmap(pkt_netmap_t * const pkt_nm, odp_packet_t pkt_table[], - unsigned len); -#endif