From patchwork Tue May 30 07:20:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bogdan Pricope X-Patchwork-Id: 100694 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp88653qge; Tue, 30 May 2017 00:21:20 -0700 (PDT) X-Received: by 10.107.130.25 with SMTP id e25mr15365188iod.31.1496128880335; Tue, 30 May 2017 00:21:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496128880; cv=none; d=google.com; s=arc-20160816; b=qAoYMgkh66MXAtTWLAO8DguhPzZwPZLlxG9kutONtF5zw+4CEk0aNGz9JEQ6Q46kYR /sqpC2VAsQM9UsRymh66UxE+ufjfTQLlxi7g6Uei8/MeWIb2nnqosAM3w6AOO1xFks7U VJHSLUI7i6iPkEdDJebd9EPVu96Kcpt9TBbSiN5OpLYtDAVnaA+zvmzswULue76Q1UUz ZZyyFIX7Zh0WbMFZYYRwg9nPxh4IFfQrsD/ZtWFWr/lpY2YoRHAyXH2k1E8c164oB9yT +ZamAGNq10NvkZU2lnCWeyjmANM+TA4HlteiOAuKZk0uqV1a8aVZ41PVISRWcLOk7+ky j2Qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:delivered-to:arc-authentication-results; bh=paZNpkmaRb0NmcCZqMMJ4ZLyIN7fIcJchQDLk2mLR5Y=; b=vKDxLYK7EFpynXkH8uuL8apLlubaSgRLDCDYMuY+uKQNIV4h9c0+fjbn6FGmUWaoZt HnEFG0KtGn56vSG+B8YvojSxwkNi+peCKiDYmg3K9dG6Fn5S9WrFCwh7Lo3P2QViUm13 2qdOsa0vcZWdohBo7TXkh5ebad7CiLvC7YhtrQ4FpngAzG2WA/jgGIy2S/ObDjB+ihmn Ftvhb8hG4m3ow5UIfpIHqXI8izPm9zkhukNyO8R+7Apzv8GvumADBb+A3ewGbVMjSdn3 dt1as/v5+7wQlKpaCk9mdelXRtzAooRI0pHPo5L81T62im1biaUvY5p9MhGMapfzkGeJ Szhg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id e28si11251282ioj.189.2017.05.30.00.21.20; Tue, 30 May 2017 00:21:20 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D1BBF6094A; Tue, 30 May 2017 07:21:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, RCVD_IN_SORBS_SPAM, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 54AB360521; Tue, 30 May 2017 07:21:08 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 4F3CA607E5; Tue, 30 May 2017 07:21:05 +0000 (UTC) Received: from mail-wm0-f47.google.com (mail-wm0-f47.google.com [74.125.82.47]) by lists.linaro.org (Postfix) with ESMTPS id 55429607B3 for ; Tue, 30 May 2017 07:20:46 +0000 (UTC) Received: by mail-wm0-f47.google.com with SMTP id e127so86454779wmg.1 for ; Tue, 30 May 2017 00:20:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=paZNpkmaRb0NmcCZqMMJ4ZLyIN7fIcJchQDLk2mLR5Y=; b=UjAtU1z5vSiU8AlgnXPT/c087MZB7NMnAq9umrLwFWyFv5KiQ+hWsGrJyXAQbNfDI3 swyB8RCe5KD+JnLoR5kuSnjSc1BwNDdptEESuHb9Fo2y1dmcHawQg3Bl1OI2kJtUZBZd 5tu2Atpp3SkclUcpUYc0KEnyfINJ7hqOiT+IacvVAte5aIQWSgXtW/G93/Nniu5kd1Qr zdvC5JNowgnnydPYa0h1Mg9ySjK1cG0eKAkjbJNrp6HpliPV//1Z7lmYqOWQsBYtO8dj 4Q8CmtGz+rIISkbCm/gsYmh5jYGlnOML6DGKAlfijJKy8mzFfipxWKTfv644YVSTsld3 doaA== X-Gm-Message-State: AODbwcCP2L3X0iT9FY66dT7px94li8h0o7LxrEgxJUTlqvwXUjfGZ260 a/6j/OgFQvFW+skk26vnr/Sz X-Received: by 10.28.165.210 with SMTP id o201mr585228wme.8.1496128844951; Tue, 30 May 2017 00:20:44 -0700 (PDT) Received: from netbricks-team.enea.se ([193.202.22.66]) by smtp.gmail.com with ESMTPSA id a9sm21291304wrc.22.2017.05.30.00.20.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 30 May 2017 00:20:44 -0700 (PDT) From: Bogdan Pricope To: lng-odp@lists.linaro.org Date: Tue, 30 May 2017 10:20:29 +0300 Message-Id: <1496128829-28125-2-git-send-email-bogdan.pricope@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1496128829-28125-1-git-send-email-bogdan.pricope@linaro.org> References: <1496128829-28125-1-git-send-email-bogdan.pricope@linaro.org> Subject: [lng-odp] [RFCv6] dpdk: enable hardware checksum support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Signed-off-by: Bogdan Pricope --- example/generator/odp_generator.c | 100 ++++++++++++++--- platform/linux-generic/odp_packet_io.c | 2 + platform/linux-generic/pktio/dpdk.c | 190 ++++++++++++++++++++++++++++++++- 3 files changed, 273 insertions(+), 19 deletions(-) -- 1.9.1 diff --git a/example/generator/odp_generator.c b/example/generator/odp_generator.c index 3ec7d8d..1cd3e29 100644 --- a/example/generator/odp_generator.c +++ b/example/generator/odp_generator.c @@ -46,6 +46,7 @@ typedef struct { odp_pktio_t pktio; + odp_pktio_config_t config; odp_pktout_queue_t pktout[MAX_WORKERS]; unsigned pktout_count; } interface_t; @@ -92,6 +93,7 @@ static struct { */ typedef struct { odp_pktout_queue_t pktout; /**< Packet output queue to use*/ + odp_pktout_config_opt_t *pktout_cfg; /**< Packet output offload config*/ odp_pool_t pool; /**< Pool for packet IO */ odp_timer_pool_t tp; /**< Timer pool handle */ odp_queue_t tq; /**< Queue for timeouts */ @@ -116,6 +118,9 @@ static args_t *args; /** Barrier to sync threads execution */ static odp_barrier_t barrier; +/** List of interfaces */ +static interface_t *ifs; + /* helper funcs */ static void parse_args(int argc, char *argv[], appl_args_t *appl_args); static void print_info(char *progname, appl_args_t *appl_args); @@ -193,7 +198,8 @@ static int scan_ip(char *buf, unsigned int *paddr) * @return Handle of created packet * @retval ODP_PACKET_INVALID Packet could not be created */ -static odp_packet_t setup_udp_pkt_ref(odp_pool_t pool) +static odp_packet_t setup_udp_pkt_ref(odp_pool_t pool, + odp_pktout_config_opt_t *pktout_cfg) { odp_packet_t pkt; char *buf; @@ -237,8 +243,10 @@ static odp_packet_t setup_udp_pkt_ref(odp_pool_t pool) udp->src_port = odp_cpu_to_be_16(args->appl.srcport); udp->dst_port = odp_cpu_to_be_16(args->appl.dstport); udp->length = odp_cpu_to_be_16(args->appl.payload + ODPH_UDPHDR_LEN); - udp->chksum = 0; - udp->chksum = odph_ipv4_udp_chksum(pkt); + if (!pktout_cfg->bit.udp_chksum) { + udp->chksum = 0; + udp->chksum = odph_ipv4_udp_chksum(pkt); + } return pkt; } @@ -252,7 +260,8 @@ static odp_packet_t setup_udp_pkt_ref(odp_pool_t pool) * @return Handle of created packet * @retval ODP_PACKET_INVALID Packet could not be created */ -static odp_packet_t pack_udp_pkt(odp_pool_t pool, odp_packet_t pkt_ref) +static odp_packet_t pack_udp_pkt(odp_pool_t pool, odp_packet_t pkt_ref, + odp_pktout_config_opt_t *pktout_cfg) { odp_packet_t pkt; char *buf; @@ -274,7 +283,14 @@ static odp_packet_t pack_udp_pkt(odp_pool_t pool, odp_packet_t pkt_ref) ip = (odph_ipv4hdr_t *)(buf + ODPH_ETHHDR_LEN); seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0xFFFF; ip->id = odp_cpu_to_be_16(seq); - ip->chksum = odph_chksum(ip, ODPH_IPV4HDR_LEN); + if (!pktout_cfg->bit.ipv4_chksum) + ip->chksum = odph_chksum(ip, ODPH_IPV4HDR_LEN); + + if (pktout_cfg->bit.ipv4_chksum || pktout_cfg->bit.udp_chksum) { + odp_packet_l2_offset_set(pkt, 0); + odp_packet_l3_offset_set(pkt, ODPH_ETHHDR_LEN); + odp_packet_l4_offset_set(pkt, ODPH_ETHHDR_LEN + ODPH_IPV4HDR_LEN); + } return pkt; } @@ -343,7 +359,8 @@ static odp_packet_t setup_icmp_pkt_ref(odp_pool_t pool) * @return Handle of created packet * @retval ODP_PACKET_INVALID Packet could not be created */ -static odp_packet_t pack_icmp_pkt(odp_pool_t pool, odp_packet_t pkt_ref) +static odp_packet_t pack_icmp_pkt(odp_pool_t pool, odp_packet_t pkt_ref, + odp_pktout_config_opt_t *pktout_cfg) { odp_packet_t pkt; char *buf; @@ -368,7 +385,8 @@ static odp_packet_t pack_icmp_pkt(odp_pool_t pool, odp_packet_t pkt_ref) ip = (odph_ipv4hdr_t *)(buf + ODPH_ETHHDR_LEN); seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0xffff; ip->id = odp_cpu_to_be_16(seq); - ip->chksum = odph_chksum(ip, ODPH_IPV4HDR_LEN); + if (!pktout_cfg->bit.ipv4_chksum) + ip->chksum = odph_chksum(ip, ODPH_IPV4HDR_LEN); /* icmp */ icmp = (odph_icmphdr_t *)(buf + ODPH_ETHHDR_LEN + ODPH_IPV4HDR_LEN); @@ -382,6 +400,12 @@ static odp_packet_t pack_icmp_pkt(odp_pool_t pool, odp_packet_t pkt_ref) icmp->chksum = 0; icmp->chksum = odph_chksum(icmp, args->appl.payload + ODPH_ICMPHDR_LEN); + if (pktout_cfg->bit.ipv4_chksum) { + odp_packet_l2_offset_set(pkt, 0); + odp_packet_l3_offset_set(pkt, ODPH_ETHHDR_LEN); + odp_packet_l4_offset_set(pkt, ODPH_ETHHDR_LEN + ODPH_IPV4HDR_LEN); + } + return pkt; } @@ -422,6 +446,22 @@ static int create_pktio(const char *dev, odp_pool_t pool, dev); return -1; } + + odp_pktio_config_init(&itf->config); + itf->config.pktin.bit.ipv4_chksum = capa.config.pktin.bit.ipv4_chksum; + itf->config.pktin.bit.udp_chksum = capa.config.pktin.bit.udp_chksum; + itf->config.pktin.bit.drop_ipv4_err = capa.config.pktin.bit.drop_ipv4_err; + itf->config.pktin.bit.drop_udp_err = capa.config.pktin.bit.drop_udp_err; + + itf->config.pktout.bit.ipv4_chksum = capa.config.pktout.bit.ipv4_chksum; + itf->config.pktout.bit.udp_chksum = capa.config.pktout.bit.udp_chksum; + + if (odp_pktio_config(itf->pktio, &itf->config)) { + EXAMPLE_ERR("Error: Failed to set interface configuration %s\n", + dev); + return -1; + } + if (num_rx_queues > capa.max_input_queues) num_rx_queues = capa.max_input_queues; @@ -482,6 +522,7 @@ static int gen_send_thread(void *arg) int ret, i; thread_args_t *thr_args; odp_pktout_queue_t pktout; + odp_pktout_config_opt_t *pktout_cfg; odp_packet_t pkt_array[MAX_UDP_TX_BURST]; int pkt_array_size; int burst_start, burst_size; @@ -491,9 +532,10 @@ static int gen_send_thread(void *arg) thr_args = arg; pktout = thr_args->pktout; + pktout_cfg = thr_args->pktout_cfg; if (args->appl.mode == APPL_MODE_UDP) { - pkt_ref = setup_udp_pkt_ref(thr_args->pool); + pkt_ref = setup_udp_pkt_ref(thr_args->pool, pktout_cfg); pkt_array_size = args->appl.udp_tx_burst; } else if (args->appl.mode == APPL_MODE_PING) { pkt_ref = setup_icmp_pkt_ref(thr_args->pool); @@ -522,7 +564,7 @@ static int gen_send_thread(void *arg) if (args->appl.mode == APPL_MODE_UDP) { for (i = 0; i < pkt_array_size; i++) { pkt_array[i] = pack_udp_pkt(thr_args->pool, - pkt_ref); + pkt_ref, pktout_cfg); if (!odp_packet_is_valid(pkt_array[i])) break; } @@ -533,7 +575,7 @@ static int gen_send_thread(void *arg) break; } } else if (args->appl.mode == APPL_MODE_PING) { - pkt_array[0] = pack_icmp_pkt(thr_args->pool, pkt_ref); + pkt_array[0] = pack_icmp_pkt(thr_args->pool, pkt_ref, pktout_cfg); if (!odp_packet_is_valid(pkt_array[0])) { EXAMPLE_ERR(" [%2i] alloc_single failed\n", thr); @@ -684,6 +726,7 @@ static int gen_recv_thread(void *arg) odp_packet_t pkts[MAX_RX_BURST], pkt; odp_event_t events[MAX_RX_BURST]; int pkt_cnt, ev_cnt, i; + interface_t *itf; thr = odp_thread_id(); (void)arg; @@ -705,6 +748,27 @@ static int gen_recv_thread(void *arg) continue; for (i = 0, pkt_cnt = 0; i < ev_cnt; i++) { pkt = odp_packet_from_event(events[i]); + itf = &ifs[odp_pktio_index(odp_packet_input(pkt))]; + + if (odp_packet_has_ipv4(pkt)) { + if (itf->config.pktin.bit.ipv4_chksum) { /* HW validation */ + if (odp_packet_has_l3_error(pkt)) + printf("HW detected L3 error\n"); + } /* else SW validation */ + } + + if (odp_packet_has_udp(pkt)) { + if (itf->config.pktin.bit.udp_chksum) { /* HW validation*/ + if (odp_packet_has_l4_error(pkt)) + printf("HW detected L4 error\n"); + } /* else SW validation */ + } + if (odp_packet_has_tcp(pkt)) { + if (itf->config.pktin.bit.tcp_chksum) { /* HW validation */ + if (odp_packet_has_l4_error(pkt)) + printf("HW detected L4 error\n"); + } /*else SW validation */ + } /* Drop packets with errors */ if (odp_unlikely(odp_packet_has_error(pkt))) { @@ -714,9 +778,11 @@ static int gen_recv_thread(void *arg) pkts[pkt_cnt++] = pkt; } - print_pkts(thr, pkts, pkt_cnt); + if (pkt_cnt) { + print_pkts(thr, pkts, pkt_cnt); - odp_packet_free_multi(pkts, pkt_cnt); + odp_packet_free_multi(pkts, pkt_cnt); + } } return 0; @@ -810,7 +876,6 @@ int main(int argc, char *argv[]) odp_pool_t tmop; odp_queue_t tq; odp_event_t ev; - interface_t *ifs; odp_instance_t instance; odph_odpthread_params_t thr_params; @@ -972,6 +1037,7 @@ int main(int argc, char *argv[]) abort(); } (void)args->thread[1].pktout; /* Not used*/ + (void)args->thread[1].pktout_cfg; /* Not used*/ args->thread[1].pool = pool; args->thread[1].tp = tp; args->thread[1].tq = tq; @@ -1001,6 +1067,7 @@ int main(int argc, char *argv[]) abort(); } args->thread[0].pktout = ifs[0].pktout[0]; + args->thread[0].pktout_cfg = &ifs[0].config.pktout; args->thread[0].pool = pool; args->thread[0].tp = tp; args->thread[0].tq = tq; @@ -1032,15 +1099,18 @@ int main(int argc, char *argv[]) int (*thr_run_func)(void *); int if_idx, pktout_idx; - if (args->appl.mode == APPL_MODE_RCV) + if (args->appl.mode == APPL_MODE_RCV) { (void)args->thread[i].pktout; /*not used*/ - else { + (void)args->thread[i].pktout_cfg; /*not used*/ + } else { if_idx = i % args->appl.if_count; pktout_idx = (i / args->appl.if_count) % ifs[if_idx].pktout_count; args->thread[i].pktout = ifs[if_idx].pktout[pktout_idx]; + args->thread[i].pktout_cfg = + &ifs[if_idx].config.pktout; } tq = odp_queue_create("", NULL); if (tq == ODP_QUEUE_INVALID) { diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index 50a000e..dc24ee9 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -150,6 +150,8 @@ static void init_pktio_entry(pktio_entry_t *entry) init_out_queues(entry); pktio_classifier_init(entry); + + odp_pktio_config_init(&entry->s.config); } static odp_pktio_t alloc_lock_pktio_entry(void) diff --git a/platform/linux-generic/pktio/dpdk.c b/platform/linux-generic/pktio/dpdk.c index 6ac89bd..77ff35f 100644 --- a/platform/linux-generic/pktio/dpdk.c +++ b/platform/linux-generic/pktio/dpdk.c @@ -23,10 +23,15 @@ #include #include +#include #include #include #include +#include +#include +#include +#include #include static int disable_pktio; /** !0 this pktio disabled, 0 enabled */ @@ -189,6 +194,7 @@ static int dpdk_setup_port(pktio_entry_t *pktio_entry) int ret; pkt_dpdk_t *pkt_dpdk = &pktio_entry->s.pkt_dpdk; struct rte_eth_rss_conf rss_conf; + uint16_t hw_ip_checksum = 0; /* Always set some hash functions to enable DPDK RSS hash calculation */ if (pkt_dpdk->hash.all_bits == 0) { @@ -198,13 +204,18 @@ static int dpdk_setup_port(pktio_entry_t *pktio_entry) rss_conf_to_hash_proto(&rss_conf, &pkt_dpdk->hash); } + if (pktio_entry->s.config.pktin.bit.ipv4_chksum || + pktio_entry->s.config.pktin.bit.udp_chksum || + pktio_entry->s.config.pktin.bit.tcp_chksum) + hw_ip_checksum = 1; + struct rte_eth_conf port_conf = { .rxmode = { .mq_mode = ETH_MQ_RX_RSS, .max_rx_pkt_len = pkt_dpdk->data_room, .split_hdr_size = 0, .header_split = 0, - .hw_ip_checksum = 0, + .hw_ip_checksum = hw_ip_checksum, .hw_vlan_filter = 0, .jumbo_frame = 1, .hw_strip_crc = 0, @@ -420,6 +431,11 @@ static void dpdk_init_capability(pktio_entry_t *pktio_entry, { pkt_dpdk_t *pkt_dpdk = &pktio_entry->s.pkt_dpdk; odp_pktio_capability_t *capa = &pkt_dpdk->capa; + int ptype_cnt; + int ptype_l3_ipv4 = 0; + int ptype_l4_tcp = 0; + int ptype_l4_udp = 0; + uint32_t ptype_mask = RTE_PTYPE_L3_MASK | RTE_PTYPE_L4_MASK; memset(dev_info, 0, sizeof(struct rte_eth_dev_info)); memset(capa, 0, sizeof(odp_pktio_capability_t)); @@ -431,9 +447,55 @@ static void dpdk_init_capability(pktio_entry_t *pktio_entry, PKTIO_MAX_QUEUES); capa->set_op.op.promisc_mode = 1; + ptype_cnt = rte_eth_dev_get_supported_ptypes(pkt_dpdk->port_id, + ptype_mask, NULL, 0); + if (ptype_cnt > 0) { + uint32_t ptypes[ptype_cnt]; + int i; + + ptype_cnt = rte_eth_dev_get_supported_ptypes(pkt_dpdk->port_id, + ptype_mask, ptypes, ptype_cnt); + for (i = 0; i < ptype_cnt; i++) + switch (ptypes[i]) { + case RTE_PTYPE_L3_IPV4: + case RTE_PTYPE_L3_IPV4_EXT_UNKNOWN: + case RTE_PTYPE_L3_IPV4_EXT: + ptype_l3_ipv4 = 1; + break; + case RTE_PTYPE_L4_TCP: + ptype_l4_tcp = 1; + break; + case RTE_PTYPE_L4_UDP: + ptype_l4_udp = 1; + break; + } + } + odp_pktio_config_init(&capa->config); capa->config.pktin.bit.ts_all = 1; capa->config.pktin.bit.ts_ptp = 1; + + capa->config.pktin.bit.ipv4_chksum = ptype_l3_ipv4 && + (dev_info->rx_offload_capa & DEV_RX_OFFLOAD_IPV4_CKSUM)? 1:0; + if (capa->config.pktin.bit.ipv4_chksum) + capa->config.pktin.bit.drop_ipv4_err = 1; + + capa->config.pktin.bit.udp_chksum = ptype_l4_udp && + (dev_info->rx_offload_capa & DEV_RX_OFFLOAD_UDP_CKSUM)? 1:0; + if (capa->config.pktin.bit.udp_chksum) + capa->config.pktin.bit.drop_udp_err = 1; + + capa->config.pktin.bit.tcp_chksum = ptype_l4_tcp && + (dev_info->rx_offload_capa & DEV_RX_OFFLOAD_TCP_CKSUM)? 1:0; + if (capa->config.pktin.bit.tcp_chksum) + capa->config.pktin.bit.drop_tcp_err = 1; + + capa->config.pktout.bit.ipv4_chksum = + (dev_info->tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM)? 1:0; + capa->config.pktout.bit.udp_chksum = + (dev_info->tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM)? 1:0; + capa->config.pktout.bit.tcp_chksum = + (dev_info->tx_offload_capa & DEV_TX_OFFLOAD_TCP_CKSUM)? 1:0; } static int dpdk_open(odp_pktio_t id ODP_UNUSED, @@ -605,9 +667,11 @@ static inline int mbuf_to_pkt(pktio_entry_t *pktio_entry, int nb_pkts = 0; int alloc_len, num; odp_pool_t pool = pktio_entry->s.pkt_dpdk.pool; + odp_pktin_config_opt_t *pktin_cfg; /* Allocate maximum sized packets */ alloc_len = pktio_entry->s.pkt_dpdk.data_room; + pktin_cfg = &pktio_entry->s.config.pktin; num = packet_alloc_multi(pool, alloc_len, pkt_table, mbuf_num); if (num != mbuf_num) { @@ -658,6 +722,37 @@ static inline int mbuf_to_pkt(pktio_entry_t *pktio_entry, if (mbuf->ol_flags & PKT_RX_RSS_HASH) odp_packet_flow_hash_set(pkt, mbuf->hash.rss); + if (pktin_cfg->bit.ipv4_chksum && + (mbuf->packet_type & RTE_PTYPE_L3_IPV4) && /* covers IPv4, IPv4_EXT, IPv4_EXT_UKN */ + mbuf->ol_flags & PKT_RX_IP_CKSUM_BAD) { + if (pktin_cfg->bit.drop_ipv4_err) { + odp_packet_free(pkt); + rte_pktmbuf_free(mbuf); + continue; + } else + pkt_hdr->p.error_flags.ip_err = 1; + } + + if (pktin_cfg->bit.udp_chksum && + (mbuf->packet_type & RTE_PTYPE_L4_MASK) == RTE_PTYPE_L4_UDP && + mbuf->ol_flags & PKT_RX_L4_CKSUM_BAD) { + if (pktin_cfg->bit.drop_udp_err) { + odp_packet_free(pkt); + rte_pktmbuf_free(mbuf); + continue; + } else + pkt_hdr->p.error_flags.udp_err = 1; + } else if (pktin_cfg->bit.tcp_chksum && + (mbuf->packet_type & RTE_PTYPE_L4_MASK) == RTE_PTYPE_L4_TCP && + mbuf->ol_flags & PKT_RX_L4_CKSUM_BAD) { + if (pktin_cfg->bit.drop_tcp_err) { + odp_packet_free(pkt); + rte_pktmbuf_free(mbuf); + continue; + } else + pkt_hdr->p.error_flags.tcp_err = 1; + } + packet_set_ts(pkt_hdr, ts); pkt_table[nb_pkts++] = pkt; @@ -675,6 +770,40 @@ fail: return (i > 0 ? i : -1); } +static inline uint16_t phdr_csum(int ipv4, + void *l3_hdr, + uint64_t ol_flags) +{ + if (ipv4) + return rte_ipv4_phdr_cksum(l3_hdr, ol_flags); + else /*ipv6*/ + return rte_ipv6_phdr_cksum(l3_hdr, ol_flags); +} + +static int packet_parse(void *l3_hdr, uint8_t *l3_proto_v4, uint8_t *l4_proto) +{ + uint8_t l3_proto_ver = _ODP_IPV4HDR_VER(*(uint8_t *)l3_hdr); + + if (l3_proto_ver == _ODP_IPV4) { + struct ipv4_hdr *ip = (struct ipv4_hdr *)l3_hdr; + + *l3_proto_v4 = 1; + if (!rte_ipv4_frag_pkt_is_fragmented(ip)) + *l4_proto = ip->next_proto_id; + else + *l4_proto = 0; + + return 0; + } else if (l3_proto_ver == _ODP_IPV6) { + struct ipv6_hdr *ipv6 = (struct ipv6_hdr *)l3_hdr; + + *l3_proto_v4 = 0; + *l4_proto = ipv6->proto; + return 0; + } + + return -1; +} static inline int pkt_to_mbuf(pktio_entry_t *pktio_entry, struct rte_mbuf *mbuf_table[], @@ -683,15 +812,29 @@ static inline int pkt_to_mbuf(pktio_entry_t *pktio_entry, pkt_dpdk_t *pkt_dpdk = &pktio_entry->s.pkt_dpdk; int i, j; char *data; + odp_bool_t ipv4_chksum_cfg, udp_chksum_cfg, tcp_chksum_cfg; + odp_bool_t ipv4_chksum_pkt, udp_chksum_pkt, tcp_chksum_pkt; + uint8_t l3_proto_v4, l4_proto; + odp_packet_t pkt; uint16_t pkt_len; + packet_parser_t *pkt_p; + struct rte_mbuf *mbuf; + void *l3_hdr; if (odp_unlikely((rte_pktmbuf_alloc_bulk(pkt_dpdk->pkt_pool, mbuf_table, num)))) { ODP_ERR("Failed to alloc mbuf\n"); return 0; } + ipv4_chksum_cfg = pktio_entry->s.config.pktout.bit.ipv4_chksum; + udp_chksum_cfg = pktio_entry->s.config.pktout.bit.udp_chksum; + tcp_chksum_cfg = pktio_entry->s.config.pktout.bit.tcp_chksum; + for (i = 0; i < num; i++) { - pkt_len = _odp_packet_len(pkt_table[i]); + pkt = pkt_table[i]; + mbuf = mbuf_table[i]; + pkt_len = _odp_packet_len(pkt); + pkt_p = &odp_packet_hdr(pkt)->p; if (pkt_len > pkt_dpdk->mtu) { if (i == 0) @@ -700,9 +843,48 @@ static inline int pkt_to_mbuf(pktio_entry_t *pktio_entry, } /* Packet always fits in mbuf */ - data = rte_pktmbuf_append(mbuf_table[i], pkt_len); + data = rte_pktmbuf_append(mbuf, pkt_len); + + odp_packet_copy_to_mem(pkt, 0, pkt_len, data); + + if (!pktio_entry->s.config.pktout.all_bits) + continue; + + l3_hdr = (void *)(data + pkt_p->l3_offset); + if (packet_parse(l3_hdr, &l3_proto_v4, &l4_proto)) + continue; - odp_packet_copy_to_mem(pkt_table[i], 0, pkt_len, data); + ipv4_chksum_pkt = ipv4_chksum_cfg && l3_proto_v4; + udp_chksum_pkt = udp_chksum_cfg && (l4_proto == _ODP_IPPROTO_UDP); + tcp_chksum_pkt = tcp_chksum_cfg && (l4_proto == _ODP_IPPROTO_TCP); + + if (ipv4_chksum_pkt || udp_chksum_pkt || tcp_chksum_pkt) { + mbuf->l2_len = pkt_p->l3_offset - pkt_p->l2_offset; + mbuf->l3_len = pkt_p->l4_offset - pkt_p->l3_offset; + + if (l3_proto_v4) + mbuf->ol_flags = PKT_TX_IPV4; + else + mbuf->ol_flags = PKT_TX_IPV6; + + if (ipv4_chksum_pkt) { + mbuf->ol_flags |= PKT_TX_IP_CKSUM; + + ((struct ipv4_hdr *)l3_hdr)->hdr_checksum = 0; + } + + if (udp_chksum_pkt) { + mbuf->ol_flags |= PKT_TX_UDP_CKSUM; + + ((struct udp_hdr *)(data + pkt_p->l4_offset))->dgram_cksum = + phdr_csum(l3_proto_v4, l3_hdr, mbuf->ol_flags); + } else if (tcp_chksum_pkt) { + mbuf->ol_flags |= PKT_TX_TCP_CKSUM; + + ((struct tcp_hdr *)(data + pkt_p->l4_offset))->cksum = + phdr_csum(l3_proto_v4, l3_hdr, mbuf->ol_flags); + } + } } return i;