From patchwork Thu Nov 27 16:32:25 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Balasubramanian Manoharan X-Patchwork-Id: 41631 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f198.google.com (mail-lb0-f198.google.com [209.85.217.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 3604E244C2 for ; Thu, 27 Nov 2014 16:33:15 +0000 (UTC) Received: by mail-lb0-f198.google.com with SMTP id p9sf3221256lbv.5 for ; Thu, 27 Nov 2014 08:33:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:subject:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=4pNbaADcSiVfe6O2dY/xeYfUN9fcHPZhGcnAri0Jh6I=; b=R4mA0YI8cvyiXv4vin529g5fWej9i1wMImPDkmZsDgfrWg4j9/+zgAyaLwbdAP9PtT YXCyNCB2SGJ0QwfEoP0kH9uJ3nKlyH32RBUX2SCgVMjgV61Q/YH6vuPCU1m6s2xaYoyG H6P4gh0F872HWoh4R0X3rZXxfoaWVso/xM+slpqDFFKNpqZAKoBZ94GF1nsNe1io1LPn 3rPAwRymjazq9nZecW7rlKcB2RrGTOdcHGuun4WQ1I3UzqpBd/wlibpibvCTfohqCSQz /tRFFwkuc9OhvtDHGDUEzM6ehUyDS7nIRWS9FjloNPi9PiC4VNbOnFGGx0Slmf8OpBly vo0w== X-Gm-Message-State: ALoCoQmMHFrXvbjdcKaZFc5Z7MxHLtbdSRd8WxaPEd2YUugk4uDU5NNa6qaO2I5bJ9zDLc9VBZW+ X-Received: by 10.112.204.71 with SMTP id kw7mr714439lbc.13.1417105994194; Thu, 27 Nov 2014 08:33:14 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.205.10 with SMTP id lc10ls849259lac.22.gmail; Thu, 27 Nov 2014 08:33:13 -0800 (PST) X-Received: by 10.152.203.137 with SMTP id kq9mr39689936lac.51.1417105993623; Thu, 27 Nov 2014 08:33:13 -0800 (PST) Received: from mail-la0-f52.google.com (mail-la0-f52.google.com. [209.85.215.52]) by mx.google.com with ESMTPS id q2si1632294lah.126.2014.11.27.08.33.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Nov 2014 08:33:13 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.52 as permitted sender) client-ip=209.85.215.52; Received: by mail-la0-f52.google.com with SMTP id q1so4525131lam.11 for ; Thu, 27 Nov 2014 08:33:13 -0800 (PST) X-Received: by 10.152.9.7 with SMTP id v7mr39998043laa.40.1417105993422; Thu, 27 Nov 2014 08:33:13 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.184.201 with SMTP id ew9csp822686lbc; Thu, 27 Nov 2014 08:33:11 -0800 (PST) X-Received: by 10.52.85.132 with SMTP id h4mr8813150vdz.40.1417105989088; Thu, 27 Nov 2014 08:33:09 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id qr16si2882968vdb.90.2014.11.27.08.33.08 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 27 Nov 2014 08:33:09 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Xu20A-00059O-Sx; Thu, 27 Nov 2014 16:33:06 +0000 Received: from mail-yh0-f44.google.com ([209.85.213.44]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Xu200-000592-Vm for lng-odp@lists.linaro.org; Thu, 27 Nov 2014 16:32:57 +0000 Received: by mail-yh0-f44.google.com with SMTP id c41so2371376yho.31 for ; Thu, 27 Nov 2014 08:32:51 -0800 (PST) X-Received: by 10.170.142.139 with SMTP id j133mr25808783ykc.81.1417105971671; Thu, 27 Nov 2014 08:32:51 -0800 (PST) Received: from bala-PowerEdge-T110-II.caveonetworks.com ([111.93.218.67]) by mx.google.com with ESMTPSA id j5sm3998270yhj.40.2014.11.27.08.32.49 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Nov 2014 08:32:51 -0800 (PST) From: Balasubramanian Manoharan To: lng-odp@lists.linaro.org Date: Thu, 27 Nov 2014 22:02:25 +0530 Message-Id: <1417105945-28700-2-git-send-email-bala.manoharan@linaro.org> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1417105945-28700-1-git-send-email-bala.manoharan@linaro.org> References: <1417105945-28700-1-git-send-email-bala.manoharan@linaro.org> X-Topics: Classification patch Subject: [lng-odp] [PATCH v3 2/2] linux-generic: Classification Implementation for v1.0 X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bala.manoharan@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.52 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 This patch contains classification implementation for ODP v1.0. The salient features of this classification version are as follows: * Attaches PMR, PMR_SET to a Pktio entry * Attaches CoS values for L2 and L3 QoS to a Pktio entry * Selects ClassOfService for a packet based on PMR, L2 QoS and L3 QoS values * Selects a default CoS if packet does not match any of the assigned rules * Selects an Error CoS for an Error packet * Enqueues the packet to the queue associated with the selected CoS This patch also adds classifier object to pktio entry and moves static inline functions to header files in Buffer and PKTIO modules. Signed-off-by: Balasubramanian Manoharan --- V3: Patch has been merged as independently compilable unit helper/include/odph_ip.h | 6 + platform/linux-generic/include/api/odp.h | 1 + .../include/odp_buffer_pool_internal.h | 10 + .../include/odp_classification_datamodel.h | 199 +++++ .../include/odp_classification_inlines.h | 259 +++++++ .../include/odp_classification_internal.h | 173 +++++ platform/linux-generic/include/odp_internal.h | 2 + .../linux-generic/include/odp_packet_io_internal.h | 17 + platform/linux-generic/odp_buffer_pool.c | 10 - platform/linux-generic/odp_classification.c | 831 +++++++++++++++++++-- platform/linux-generic/odp_init.c | 4 + platform/linux-generic/odp_packet_io.c | 85 ++- 12 files changed, 1475 insertions(+), 122 deletions(-) create mode 100644 platform/linux-generic/include/odp_classification_datamodel.h create mode 100644 platform/linux-generic/include/odp_classification_inlines.h create mode 100644 platform/linux-generic/include/odp_classification_internal.h diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h index 2c83c0f..f78724e 100644 --- a/helper/include/odph_ip.h +++ b/helper/include/odph_ip.h @@ -35,6 +35,9 @@ extern "C" { /** @internal Returns IPv4 header length */ #define ODPH_IPV4HDR_IHL(ver_ihl) ((ver_ihl) & 0x0f) +/** @internal Returns IPv4 DSCP */ +#define ODPH_IPV4HDR_DSCP(tos) (((tos) & 0xfc) >> 2) + /** @internal Returns IPv4 Don't fragment */ #define ODPH_IPV4HDR_FLAGS_DONT_FRAG(frag_offset) ((frag_offset) & 0x4000) @@ -47,6 +50,9 @@ extern "C" { /** @internal Returns true if IPv4 packet is a fragment */ #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff) +/** @internal Returns IPv4 DSCP */ +#define ODPH_IPV6HDR_DSCP(ver_tc_flow) (uint8_t)((((ver_tc_flow) & 0x0fc00000) >> 22) & 0xff) + /** IPv4 header */ typedef struct ODP_PACKED { uint8_t ver_ihl; /**< Version / Header length */ diff --git a/platform/linux-generic/include/api/odp.h b/platform/linux-generic/include/api/odp.h index 6e4f69e..b7b1ca9 100644 --- a/platform/linux-generic/include/api/odp.h +++ b/platform/linux-generic/include/api/odp.h @@ -47,6 +47,7 @@ extern "C" { #include #include #include +#include #include #ifdef __cplusplus diff --git a/platform/linux-generic/include/odp_buffer_pool_internal.h b/platform/linux-generic/include/odp_buffer_pool_internal.h index e0210bd..bdbefff 100644 --- a/platform/linux-generic/include/odp_buffer_pool_internal.h +++ b/platform/linux-generic/include/odp_buffer_pool_internal.h @@ -64,6 +64,12 @@ struct pool_entry_s { size_t hdr_size; }; +typedef union pool_entry_u { + struct pool_entry_s s; + + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; + +} pool_entry_t; extern void *pool_entry_ptr[]; @@ -73,6 +79,10 @@ static inline void *get_pool_entry(uint32_t pool_id) return pool_entry_ptr[pool_id]; } +static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) +{ + return pool_hdl - 1; +} static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf) { diff --git a/platform/linux-generic/include/odp_classification_datamodel.h b/platform/linux-generic/include/odp_classification_datamodel.h new file mode 100644 index 0000000..f7c9fb5 --- /dev/null +++ b/platform/linux-generic/include/odp_classification_datamodel.h @@ -0,0 +1,199 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Datamodel + * Describes the classification internal data model + */ + +#ifndef ODP_CLASSIFICATION_DATAMODEL_H_ +#define ODP_CLASSIFICATION_DATAMODEL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include + +/* Maximum Class Of Service Entry */ +#define ODP_COS_MAX_ENTRY 64 +/* Maximum PMR Set Entry */ +#define ODP_PMRSET_MAX_ENTRY 64 +/* Maximum PMR Entry */ +#define ODP_PMR_MAX_ENTRY 64 +/* Maximum PMR Terms in a PMR Set */ +#define ODP_PMRTERM_MAX 8 +/* Maximum PMRs attached in PKTIO Level */ +#define ODP_PKTIO_MAX_PMR 8 +/* L2 Priority Bits */ +#define ODP_COS_L2_QOS_BITS 3 +/* Max L2 QoS value */ +#define ODP_COS_MAX_L2_QOS (1 << ODP_COS_L2_QOS_BITS) +/* L2 DSCP Bits */ +#define ODP_COS_L3_QOS_BITS 6 +/* Max L3 QoS Value */ +#define ODP_COS_MAX_L3_QOS (1 << ODP_COS_L3_QOS_BITS) +/* Max PMR Term bits */ +#define ODP_PMR_TERM_BYTES_MAX 8 + +/* forward declaration */ +typedef union pmr_u pmr_t; + +/** +Packet Matching Rule Term Value + +Stores the Term and Value mapping for a PMR. +The maximum size of value currently supported in 64 bits +**/ +typedef struct pmr_term_value { + odp_pmr_match_type_e match_type; /**< Packet Match Type*/ + odp_pmr_term_e term; /* PMR Term */ + union { + struct { + uint64_t val; + uint64_t mask; + } mask; /**< Match a masked set of bits */ + struct { + uint64_t val1; + uint64_t val2; + } range; /**< Match an integer range */ + }; +} pmr_term_value_t; + +/* +Class Of Service +*/ +struct cos_s { + odp_rwlock_t lock; /* cos rwlock */ + queue_entry_t *queue; /* Associated Queue */ + odp_queue_group_t queue_group; /* Associated Queue Group */ + pool_entry_t *pool; /* Associated Buffer pool */ + pmr_t *pmr; /* Associated PMR */ + bool valid; /* validity Flag */ + odp_drop_e drop_policy; /* Associated Drop Policy */ + odp_cos_flow_set_t flow_set; /* Assigned Flow Set */ + char name[ODP_COS_NAME_LEN]; /* name */ + size_t headroom; /* Headroom for this CoS */ +}; + +typedef union cos_u { + struct cos_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct cos_s))]; +} cos_t; + + +/** +Packet Matching Rule + +**/ +struct pmr_s { + odp_rwlock_t lock; /* pmr rwlock*/ + cos_t *cos; /* Associated CoS */ + odp_atomic_u32_t count; /* num of packets matching this rule */ + uint16_t num_pmr; /* num of PMR Term Values*/ + bool valid; /* Validity Flag */ + pmr_term_value_t pmr_term_value[1]; /* Associated PMR Term */ +}; + +typedef union pmr_u { + struct pmr_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_s))]; +} pmr_t; + +/** +Packet Matching Rule Set + +This structure is implemented as a extension over struct pmr_s +In order to use same pointer to access both pmr_s and pmr_set_s +'num_pmr' value is used to differentiate between pmr_s and pmr_set_s struct +**/ +struct pmr_set_s { + pmr_t pmr; + pmr_term_value_t pmr_term_value[ODP_PMRTERM_MAX - 1]; + /* List of associated PMR Terms */ +}; + +typedef union pmr_set_u { + struct pmr_set_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_set_s))]; +} pmr_set_t; + +/** +L2 QoS and CoS Map + +This structure holds the mapping between L2 QoS value and +corresponding cos_t object +**/ +typedef struct pmr_l2_cos { + odp_rwlock_t lock; /* pmr_l2_cos rwlock */ + cos_t *cos[ODP_COS_MAX_L2_QOS]; /* Array of CoS objects */ +} pmr_l2_cos_t; + +/** +L3 QoS and CoS Map + +This structure holds the mapping between L3 QoS value and +corresponding cos_t object +**/ +typedef struct pmr_l3_cos { + odp_rwlock_t lock; /* pmr_l3_cos rwlock */ + cos_t *cos[ODP_COS_MAX_L3_QOS]; /* Array of CoS objects */ +} pmr_l3_cos_t; + +/** +Linux Generic Classifier + +This structure is stored in pktio_entry and holds all +the classifier configuration value. +**/ +typedef struct classifier { + odp_rwlock_t lock; /*pktio_cos rwlock */ + uint8_t num_pmr; /* num of PMRs linked to given PKTIO*/ + bool l3_precedence; /* L3 QoS precedence */ + odp_cos_flow_set_t flow_set; /* Flow Set to be calculated + for this pktio */ + pmr_l2_cos_t l2_cos_table; /* L2 QoS-CoS table map */ + pmr_l3_cos_t l3_cos_table; /* L3 Qos-CoS table map */ + pmr_t *pmr[ODP_PKTIO_MAX_PMR]; /* PMRs linked with this PKTIO */ + cos_t *error_cos; /* Associated Error CoS */ + cos_t *default_cos; /* Associated Default CoS */ + size_t headroom; /* Pktio Headroom */ + size_t skip; /* Pktio Skip Offset */ +} classifier_t; + +/** +Class of Service Table +**/ +typedef struct odp_cos_table { + cos_t cos_entry[ODP_COS_MAX_ENTRY]; +} cos_tbl_t; + +/** +PMR set table +**/ +typedef struct pmr_set_tbl { + pmr_set_t pmr_set[ODP_PMRSET_MAX_ENTRY]; +} pmr_set_tbl_t; + +/** +PMR table +**/ +typedef struct pmr_tbl { + pmr_t pmr[ODP_PMR_MAX_ENTRY]; +} pmr_tbl_t; + +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_classification_inlines.h b/platform/linux-generic/include/odp_classification_inlines.h new file mode 100644 index 0000000..6b20119 --- /dev/null +++ b/platform/linux-generic/include/odp_classification_inlines.h @@ -0,0 +1,259 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Inlines + * Classification Inlines Functions + */ +#ifndef __ODP_CLASSIFICATION_INLINES_H_ +#define __ODP_CLASSIFICATION_INLINES_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include + +/* PMR term value verification function +These functions verify the given PMR term value with the value in the packet +These following functions return 1 on success and 0 on failure +*/ + +static inline int verify_pmr_packet_len(odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (pkt_hdr->frame_len & + term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= pkt_hdr->frame_len) && + (pkt_hdr->frame_len <= term_value->range.val2)) + return 1; + } + return 0; +} +static inline int verify_pmr_ip_proto(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint8_t proto; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + proto = ip->proto; + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (proto & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= proto) && + (proto <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_ipv4_saddr(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint32_t ipaddr; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + ipaddr = odp_be_to_cpu_32(ip->src_addr); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= ipaddr) && + (ipaddr <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_ipv4_daddr(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint32_t ipaddr; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + ipaddr = odp_be_to_cpu_32(ip->dst_addr); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= ipaddr) && + (ipaddr <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_tcp_sport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t sport; + odph_tcphdr_t *tcp; + if (!pkt_hdr->input_flags.tcp) + return 0; + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + sport = odp_be_to_cpu_16(tcp->src_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (sport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= sport) && + (sport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_tcp_dport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t dport; + odph_tcphdr_t *tcp; + if (!pkt_hdr->input_flags.tcp) + return 0; + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + dport = odp_be_to_cpu_16(tcp->dst_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (dport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= dport) && + (dport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_udp_dport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t dport; + odph_udphdr_t *udp; + if (!pkt_hdr->input_flags.udp) + return 0; + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + dport = odp_be_to_cpu_16(udp->dst_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (dport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= dport) && + (dport <= term_value->range.val2)) + return 1; + } + return 0; +} +static inline int verify_pmr_udp_sport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t sport; + odph_udphdr_t *udp; + if (!pkt_hdr->input_flags.udp) + return 0; + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + sport = odp_be_to_cpu_16(udp->src_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (sport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= sport) && + (sport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_dmac(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} + +static inline int verify_pmr_ipv6_saddr(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ipv6_daddr(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_vlan_id_0(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_vlan_id_x(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ipsec_spi(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ld_vni(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_eth_type_0(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_eth_type_x(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_classification_internal.h b/platform/linux-generic/include/odp_classification_internal.h new file mode 100644 index 0000000..fd2c6af --- /dev/null +++ b/platform/linux-generic/include/odp_classification_internal.h @@ -0,0 +1,173 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Internal + * Describes the classification internal Functions + */ + +#ifndef __ODP_CLASSIFICATION_INTERNAL_H_ +#define __ODP_CLASSIFICATION_INTERNAL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include + +/** Classification Internal function **/ + +/** +@internal +Select a CoS for the given Packet based on pktio + +This function will call all the PMRs associated with a pktio for +a given packet and will return the matched COS object. +This function will check PMR, L2 and L3 QoS COS object associated +with the PKTIO interface. + +Returns the default cos if the packet does not match any PMR +Returns the error_cos if the packet has an error +**/ +cos_t *pktio_select_cos(pktio_entry_t *pktio, uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr); + +/** +@internal +match_qos_cos + +Select a CoS for the given Packet based on QoS values +This function returns the COS object matching the L2 and L3 QoS +based on the l3_preference value of the pktio +**/ +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); +/** +Packet Classifier + +Start function for Packet Classifier +This function calls Classifier module internal functions for a given packet and +enqueues the packet to specific Queue based on PMR and CoS selected. +**/ +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt); +/** +Packet IO classifier init + +This function does initialization of classifier object associated with pktio. +This function should be called during pktio initialization. +**/ +int pktio_classifier_init(pktio_entry_t *pktio); + +/** +@internal +match_pmr_cos + +Match a PMR chain with a Packet and return matching CoS +This function gets called recursively to check the chained PMR Term value +with the packet. + +**/ +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, + odp_packet_hdr_t *hdr); +/** +@internal +CoS associated with L3 QoS value + +This function returns the CoS associated with L3 QoS value +**/ +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); + +/** +@internal +CoS associated with L2 QoS value + +This function returns the CoS associated with L2 QoS value +**/ +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); +/** +@internal +Flow Signature Calculation + +This function calculates the Flow Signature for a packet based on +CoS and updates in Packet Meta Data +**/ +int update_flow_signature(uint8_t *pkt_addr, cos_t *cos); + +/** +@internal +Allocate a odp_pmr_set_t Handle +*/ +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr); + +/** +@internal +Allocate a odp_pmr_t Handle +*/ +odp_pmr_t alloc_pmr(pmr_t **pmr); + +/** +@internal +Pointer to pmr_set_t Handle +This function checks for validity of pmr_set_t Handle +*/ +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id); + +/** +@internal +Pointer to pmr_set_t Handle +*/ +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id); + +/** +@internal +Pointer to pmr_set_t Handle +This function checks for validity of pmr_set_t Handle +*/ +pmr_t *get_pmr_entry(odp_pmr_t pmr_id); + +/** +@internal +Pointer to pmr_set_t Handle +*/ +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id); + +/** +@internal +Pointer to odp_cos_t Handle +*/ +cos_t *get_cos_entry(odp_cos_t cos_id); + +/** +@internal +Pointer to odp_cos_t Handle +This function checks for validity of odp_cos_t Handle +*/ +cos_t *get_cos_entry_internal(odp_cos_t cos_id); + +/** +@internal +Verify PMR with a Packet + +This function goes through each PMR_TERM value in pmr_t structure and +calls verification function for each term.Returns 1 if PMR matches or 0 +Otherwise. +**/ +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr); + +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index f8c1596..04c1030 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -32,6 +32,8 @@ int odp_buffer_pool_init_global(void); int odp_pktio_init_global(void); int odp_pktio_init_local(void); +int odp_classification_init_global(void); + int odp_queue_init_global(void); int odp_crypto_init_global(void); diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index 0bc1e21..218e9d0 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -20,6 +20,7 @@ extern "C" { #include #include +#include #include @@ -40,6 +41,7 @@ struct pktio_entry { odp_pktio_type_t type; /**< pktio type */ pkt_sock_t pkt_sock; /**< using socket API for IO */ pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for IO */ + classifier_t cls; /**< classifier linked with this pktio*/ char name[IFNAMSIZ]; /**< name of pktio provided to pktio_open() */ }; @@ -49,6 +51,21 @@ typedef union { uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pktio_entry))]; } pktio_entry_t; +typedef struct { + pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; +} pktio_table_t; + +extern void *pktio_entry_ptr[]; + + +static inline pktio_entry_t *get_pktio_entry(odp_pktio_t id) +{ + if (odp_unlikely(id == ODP_PKTIO_INVALID || + id > ODP_CONFIG_PKTIO_ENTRIES)) + return NULL; + + return pktio_entry_ptr[id - 1]; +} #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/odp_buffer_pool.c b/platform/linux-generic/odp_buffer_pool.c index 6a0a6b2..b5bd613 100644 --- a/platform/linux-generic/odp_buffer_pool.c +++ b/platform/linux-generic/odp_buffer_pool.c @@ -56,12 +56,6 @@ typedef struct { } odp_any_buffer_hdr_t; -typedef union pool_entry_u { - struct pool_entry_s s; - - uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; - -} pool_entry_t; typedef struct pool_table_t { @@ -86,10 +80,6 @@ static inline odp_buffer_pool_t pool_index_to_handle(uint32_t pool_id) } -static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) -{ - return pool_hdl -1; -} static inline void set_handle(odp_buffer_hdr_t *hdr, diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 190d71e..ecf97a0 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -2,68 +2,289 @@ #include #include #include +#include #include +#include #include +#include +#include +#include +#include +#include +#include +#include +#include +#include -odp_cos_t odp_cos_create(const char *name) +#define WRITE_LOCK(a) odp_rwlock_write_lock(a) +#define WRITE_UNLOCK(a) odp_rwlock_write_unlock(a) +#define LOCK_INIT(a) odp_rwlock_init(a) + +#define READ_LOCK(a) odp_rwlock_write_lock(a) +#define READ_UNLOCK(a) odp_rwlock_write_unlock(a) + +static cos_tbl_t *cos_tbl; +static pmr_set_tbl_t *pmr_set_tbl; +static pmr_tbl_t *pmr_tbl; + +cos_t *get_cos_entry_internal(odp_cos_t cos_id) +{ + return &(cos_tbl->cos_entry[cos_id - 1]); +} + +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id) +{ + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); +} + +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id) +{ + return &(pmr_tbl->pmr[pmr_id - 1]); +} + +int odp_classification_init_global(void) { - (void) name; - ODP_UNIMPLEMENTED(); + odp_shm_t cos_shm; + odp_shm_t pmr_shm; + odp_shm_t pmr_set_shm; + int i; + + cos_shm = odp_shm_reserve("odp_cos_pools", + sizeof(cos_tbl_t), + sizeof(cos_t), 0); + + cos_tbl = odp_shm_addr(cos_shm); + if (cos_tbl == NULL) { + odp_shm_free(cos_shm); + return -1; + } + + memset(cos_tbl, 0, sizeof(cos_tbl_t)); + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { + /* init locks */ + cos_t *cos = get_cos_entry_internal(i + 1); + LOCK_INIT(&cos->s.lock); + } + + pmr_shm = odp_shm_reserve("odp_pmr_pools", + sizeof(pmr_tbl_t), + sizeof(pmr_t), 0); + pmr_tbl = odp_shm_addr(pmr_shm); + if (pmr_tbl == NULL) { + odp_shm_free(pmr_shm); + return -1; + } + + memset(pmr_tbl, 0, sizeof(pmr_tbl_t)); + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { + /* init locks */ + pmr_t *pmr = get_pmr_entry_internal(i + 1); + LOCK_INIT(&pmr->s.lock); + } + + pmr_set_shm = odp_shm_reserve("odp_pmr_set_pools", + sizeof(pmr_set_tbl_t), + sizeof(pmr_set_t), 0); + pmr_set_tbl = odp_shm_addr(pmr_set_shm); + if (pmr_set_tbl == NULL) { + odp_shm_free(pmr_set_shm); + return -1; + } + + memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { + /* init locks */ + pmr_set_t *pmr = get_pmr_set_entry_internal(i + 1); + LOCK_INIT(&pmr->s.pmr.s.lock); + } + return 0; } +odp_cos_t odp_cos_create(const char *name) +{ + int i; + + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { + WRITE_LOCK(&cos_tbl->cos_entry[i].s.lock); + if (0 == cos_tbl->cos_entry[i].s.valid) { + strncpy(cos_tbl->cos_entry[i].s.name, name, + ODP_COS_NAME_LEN - 1); + cos_tbl->cos_entry[i].s.name[ODP_COS_NAME_LEN - 1] = 0; + cos_tbl->cos_entry[i].s.pmr = NULL; + cos_tbl->cos_entry[i].s.queue = NULL; + cos_tbl->cos_entry[i].s.pool = NULL; + cos_tbl->cos_entry[i].s.flow_set = 0; + cos_tbl->cos_entry[i].s.headroom = 0; + cos_tbl->cos_entry[i].s.valid = 1; + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); + return i + 1; + } + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); + } + return ODP_COS_INVALID; +} + +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) +{ + int i; + + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { + WRITE_LOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + if (0 == pmr_set_tbl->pmr_set[i].s.pmr.s.valid) { + pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; + pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; + pmr_set_tbl->pmr_set[i].s.pmr.s.cos = NULL; + *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; + odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] + .s.pmr.s.count, 0); + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + return i + 1; + } + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + } + return ODP_PMR_INVAL; +} + +odp_pmr_t alloc_pmr(pmr_t **pmr) +{ + int i; + + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { + WRITE_LOCK(&pmr_tbl->pmr[i].s.lock); + if (0 == pmr_tbl->pmr[i].s.valid) { + pmr_tbl->pmr[i].s.valid = 1; + pmr_tbl->pmr[i].s.cos = NULL; + odp_atomic_init_u32(&pmr_tbl->pmr[i].s.count, 0); + pmr_tbl->pmr[i].s.num_pmr = 0; + *pmr = &pmr_tbl->pmr[i]; + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); + return i + 1; + } + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); + } + return ODP_PMR_INVAL; +} + + +cos_t *get_cos_entry(odp_cos_t cos_id) +{ + if (cos_id > ODP_COS_MAX_ENTRY || cos_id == ODP_COS_INVALID) + return NULL; + if (cos_tbl->cos_entry[cos_id - 1].s.valid == 0) + return NULL; + return &(cos_tbl->cos_entry[cos_id - 1]); +} + + +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) +{ + if (pmr_set_id > ODP_PMRSET_MAX_ENTRY || pmr_set_id == ODP_PMR_INVAL) + return NULL; + if (pmr_set_tbl->pmr_set[pmr_set_id - 1].s.pmr.s.valid == 0) + return NULL; + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); +} + +pmr_t *get_pmr_entry(odp_pmr_t pmr_id) +{ + if (pmr_id > ODP_PMR_MAX_ENTRY || pmr_id == ODP_PMR_INVAL) + return NULL; + if (pmr_tbl->pmr[pmr_id - 1].s.valid == 0) + return NULL; + return &(pmr_tbl->pmr[pmr_id - 1]); +} + int odp_cos_destroy(odp_cos_t cos_id) { - (void)cos_id; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (NULL == cos) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.valid = 0; + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) { - (void)cos_id; - (void)queue_id; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.queue = queue_to_qentry(queue_id); + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) { - (void)cos_id; - (void)drop_policy; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.drop_policy = drop_policy; + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_pktio_set_default_cos(odp_pktio_t pktio_in, odp_cos_t default_cos) { - (void)pktio_in; - (void)default_cos; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry; + cos_t *cos; + entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + cos = get_cos_entry(default_cos); + if (cos == NULL) + return -1; + + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.default_cos = cos; + WRITE_UNLOCK(&entry->s.cls.lock); + return 0; } int odp_pktio_set_error_cos(odp_pktio_t pktio_in, odp_cos_t error_cos) { - (void)pktio_in; - (void)error_cos; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry; + cos_t *cos; + + entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + cos = get_cos_entry(error_cos); + if (cos == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.error_cos = cos; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset) { - (void)pktio_in; - (void)offset; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.skip = offset; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } -int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom) +int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom) { - (void)port_id; - (void)headroom; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.headroom = headroom; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } @@ -72,11 +293,26 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, uint8_t qos_table[], odp_cos_t cos_table[]) { - (void)pktio_in; - (void)num_qos; - (void)qos_table; - (void)cos_table; - ODP_UNIMPLEMENTED(); + pmr_l2_cos_t *l2_cos; + size_t i; + cos_t *cos; + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + READ_LOCK(&entry->s.cls.lock); + l2_cos = &entry->s.cls.l2_cos_table; + READ_UNLOCK(&entry->s.cls.lock); + + WRITE_LOCK(&l2_cos->lock); + /* Update the L2 QoS table*/ + for (i = 0; i < num_qos; i++) { + cos = get_cos_entry(cos_table[i]); + if (cos != NULL) { + if (ODP_COS_MAX_L2_QOS > qos_table[i]) + l2_cos->cos[qos_table[i]] = cos; + } + } + WRITE_UNLOCK(&l2_cos->lock); return 0; } @@ -86,12 +322,28 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, odp_cos_t cos_table[], bool l3_preference) { - (void)pktio_in; - (void)num_qos; - (void)qos_table; - (void)cos_table; - (void)l3_preference; - ODP_UNIMPLEMENTED(); + pmr_l3_cos_t *l3_cos; + size_t i; + pktio_entry_t *entry = get_pktio_entry(pktio_in); + cos_t *cos; + + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.l3_precedence = l3_preference; + l3_cos = &entry->s.cls.l3_cos_table; + WRITE_UNLOCK(&entry->s.cls.lock); + + WRITE_LOCK(&l3_cos->lock); + /* Update the L3 QoS table*/ + for (i = 0; i < num_qos; i++) { + cos = get_cos_entry(cos_table[i]); + if (cos != NULL) { + if (ODP_COS_MAX_L3_QOS > qos_table[i]) + l3_cos->cos[qos_table[i]] = cos; + } + } + WRITE_UNLOCK(&l3_cos->lock); return 0; } @@ -100,12 +352,23 @@ odp_pmr_t odp_pmr_create_match(odp_pmr_term_e term, const void *mask, size_t val_sz) { - (void)term; - (void)val; - (void)mask; - (void)val_sz; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + odp_pmr_t id; + + id = alloc_pmr(&pmr); + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) + return ODP_PMR_INVAL; + + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = 1; + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; + pmr->s.pmr_term_value[0].term = term; + pmr->s.pmr_term_value[0].mask.val = 0; + pmr->s.pmr_term_value[0].mask.mask = 0; + memcpy(&pmr->s.pmr_term_value[0].mask.val, val, val_sz); + memcpy(&pmr->s.pmr_term_value[0].mask.mask, mask, val_sz); + WRITE_UNLOCK(&pmr->s.lock); + return id; } odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, @@ -113,18 +376,33 @@ odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, const void *val2, size_t val_sz) { - (void)term; - (void)val1; - (void)val2; - (void)val_sz; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + odp_pmr_t id; + + id = alloc_pmr(&pmr); + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) + return ODP_PMR_INVAL; + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = 1; + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; + pmr->s.pmr_term_value[0].term = term; + pmr->s.pmr_term_value[0].range.val1 = 0; + pmr->s.pmr_term_value[0].range.val2 = 0; + memcpy(&pmr->s.pmr_term_value[0].range.val1, val1, val_sz); + memcpy(&pmr->s.pmr_term_value[0].range.val2, val2, val_sz); + WRITE_UNLOCK(&pmr->s.lock); + return id; } int odp_pmr_destroy(odp_pmr_t pmr_id) { - (void)pmr_id; - ODP_UNIMPLEMENTED(); + pmr_t *pmr = get_pmr_entry(pmr_id); + + if (pmr == NULL) + return -1; + WRITE_LOCK(&pmr->s.lock); + pmr->s.valid = 0; + WRITE_UNLOCK(&pmr->s.lock); return 0; } @@ -132,64 +410,465 @@ int odp_pktio_pmr_cos(odp_pmr_t pmr_id, odp_pktio_t src_pktio, odp_cos_t dst_cos) { - (void)pmr_id; - (void)src_pktio; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + uint8_t num_pmr; + pktio_entry_t *pktio_entry; + pmr_t *pmr; + cos_t *cos; + + pktio_entry = get_pktio_entry(src_pktio); + if (pktio_entry == NULL) + return -1; + + pmr = get_pmr_entry(pmr_id); + if (pmr == NULL) + return -1; + + cos = get_cos_entry(dst_cos); + if (cos == NULL) + return -1; + + num_pmr = pktio_entry->s.cls.num_pmr; + if (num_pmr >= ODP_PKTIO_MAX_PMR) + return -1; + + WRITE_LOCK(&pktio_entry->s.cls.lock); + pktio_entry->s.cls.pmr[num_pmr] = pmr; + pktio_entry->s.cls.num_pmr++; + WRITE_UNLOCK(&pktio_entry->s.cls.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos; + WRITE_UNLOCK(&pmr->s.lock); return 0; } int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t dst_cos) { - (void)pmr_id; - (void)src_cos; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + cos_t *cos_src = get_cos_entry(src_cos); + cos_t *cos_dst = get_cos_entry(dst_cos); + pmr_t *pmr = get_pmr_entry(pmr_id); + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) + return -1; + + WRITE_LOCK(&cos_src->s.lock); + cos_src->s.pmr = pmr; + WRITE_UNLOCK(&cos_src->s.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos_dst; + WRITE_UNLOCK(&pmr->s.lock); return 0; } signed long odp_pmr_match_count(odp_pmr_t pmr_id) { - (void)pmr_id; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr = get_pmr_entry(pmr_id); + if (pmr == NULL) + return -1; + return (signed long)odp_atomic_load_u32(&pmr->s.count); } unsigned long long odp_pmr_terms_cap(void) { - ODP_UNIMPLEMENTED(); - return 0; + unsigned long long term_cap = 0; + + term_cap |= (1 << ODP_PMR_LEN); + term_cap |= (1 << ODP_PMR_IPPROTO); + term_cap |= (1 << ODP_PMR_UDP_DPORT); + term_cap |= (1 << ODP_PMR_TCP_DPORT); + term_cap |= (1 << ODP_PMR_UDP_SPORT); + term_cap |= (1 << ODP_PMR_TCP_SPORT); + term_cap |= (1 << ODP_PMR_SIP_ADDR); + term_cap |= (1 << ODP_PMR_DIP_ADDR); + return term_cap; } unsigned odp_pmr_terms_avail(void) { - ODP_UNIMPLEMENTED(); - return 0; + unsigned count = 0; + int i; + + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) + if (!pmr_tbl->pmr[i].s.valid) + count++; + return count; } int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, odp_pmr_set_t *pmr_set_id) { - (void)num_terms; - (void)terms; - (void)pmr_set_id; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + int i; + uint32_t id; + int val_sz; + int count = 0; + + id = alloc_pmr_set(&pmr); + if (id == ODP_PMR_INVAL) { + *pmr_set_id = id; + return -1; + } + + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = num_terms; + for (i = 0; i < num_terms; i++) { + pmr->s.pmr_term_value[i].match_type = terms[i].match_type; + if (terms[i].match_type == ODP_PMR_MASK) { + val_sz = terms[i].mask.val_sz; + if (val_sz > ODP_PMR_TERM_BYTES_MAX) + continue; + pmr->s.pmr_term_value[i].term = terms[i].mask.term; + pmr->s.pmr_term_value[i].mask.val = 0; + pmr->s.pmr_term_value[i].mask.mask = 0; + memcpy(&pmr->s.pmr_term_value[i].mask.val, + terms[i].mask.val, val_sz); + memcpy(&pmr->s.pmr_term_value[i].mask.mask, + terms[i].mask.mask, val_sz); + } else { + val_sz = terms[i].range.val_sz; + if (val_sz > ODP_PMR_TERM_BYTES_MAX) + continue; + pmr->s.pmr_term_value[i].term = terms[i].range.term; + pmr->s.pmr_term_value[i].range.val1 = 0; + pmr->s.pmr_term_value[i].range.val2 = 0; + memcpy(&pmr->s.pmr_term_value[i].range.val1, + terms[i].range.val1, val_sz); + memcpy(&pmr->s.pmr_term_value[i].range.val2, + terms[i].range.val2, val_sz); + } + count++; + } + WRITE_UNLOCK(&pmr->s.lock); + *pmr_set_id = id; + return count; } int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) { - (void)pmr_set_id; - ODP_UNIMPLEMENTED(); + pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); + if (pmr_set == NULL) + return -1; + WRITE_LOCK(&pmr_set->s.pmr.s.lock); + pmr_set->s.pmr.s.valid = 0; + WRITE_UNLOCK(&pmr_set->s.pmr.s.lock); return 0; } int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t src_pktio, - odp_cos_t dst_cos) + odp_cos_t dst_cos) +{ + uint8_t num_pmr; + pktio_entry_t *pktio_entry; + pmr_t *pmr; + cos_t *cos; + + pktio_entry = get_pktio_entry(src_pktio); + if (pktio_entry == NULL) + return -1; + + pmr = (pmr_t *)get_pmr_set_entry(pmr_set_id); + if (pmr == NULL) + return -1; + + cos = get_cos_entry(dst_cos); + if (cos == NULL) + return -1; + + num_pmr = pktio_entry->s.cls.num_pmr; + if (num_pmr >= ODP_PKTIO_MAX_PMR) + return -1; + + WRITE_LOCK(&pktio_entry->s.cls.lock); + pktio_entry->s.cls.pmr[num_pmr] = pmr; + pktio_entry->s.cls.num_pmr++; + WRITE_UNLOCK(&pktio_entry->s.cls.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos; + WRITE_UNLOCK(&pmr->s.lock); + return 0; +} + +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) +{ + int pmr_failure = 0; + int num_pmr; + int i; + pmr_term_value_t *term_value; + + READ_LOCK(&pmr->s.lock); + if (!pmr->s.valid) { + READ_UNLOCK(&pmr->s.lock); + return 0; + } + num_pmr = pmr->s.num_pmr; + + /* Iterate through list of PMR Term values in a pmr_t */ + for (i = 0; i < num_pmr; i++) { + term_value = &pmr->s.pmr_term_value[i]; + switch (term_value->term) { + case ODP_PMR_LEN: + if (!verify_pmr_packet_len(pkt_hdr, term_value)) + pmr_failure = 1; + break; + case ODP_PMR_ETHTYPE_0: + if (!verify_pmr_eth_type_0(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_ETHTYPE_X: + if (!verify_pmr_eth_type_x(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_VLAN_ID_0: + if (!verify_pmr_vlan_id_0(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_VLAN_ID_X: + if (!verify_pmr_vlan_id_x(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DMAC: + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_IPPROTO: + if (!verify_pmr_ip_proto(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_UDP_DPORT: + if (!verify_pmr_udp_dport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_TCP_DPORT: + if (!verify_pmr_tcp_dport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_UDP_SPORT: + if (!verify_pmr_udp_sport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_TCP_SPORT: + if (!verify_pmr_tcp_sport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_SIP_ADDR: + if (!verify_pmr_ipv4_saddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DIP_ADDR: + if (!verify_pmr_ipv4_daddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_SIP6_ADDR: + if (!verify_pmr_ipv6_saddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DIP6_ADDR: + if (!verify_pmr_ipv6_daddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_IPSEC_SPI: + if (!verify_pmr_ipsec_spi(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_LD_VNI: + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_INNER_HDR_OFF: + break; + } + if (pmr_failure) { + READ_UNLOCK(&pmr->s.lock); + return false; + } + } + READ_UNLOCK(&pmr->s.lock); + odp_atomic_inc_u32(&pmr->s.count); + return true; +} + +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, + odp_packet_hdr_t *hdr) +{ + cos_t *retcos = NULL; + + if (cos == NULL || pmr == NULL) + return NULL; + + READ_LOCK(&cos->s.lock); + if (!cos->s.valid) { + READ_UNLOCK(&cos->s.lock); + return NULL; + } + READ_UNLOCK(&cos->s.lock); + + if (verify_pmr(pmr, pkt_addr, hdr)) { + /** This gets called recursively to check all the PMRs in + * a PMR chain */ + retcos = match_pmr_cos(pmr->s.cos, pkt_addr, cos->s.pmr, hdr); + if (!retcos) + return cos; + } + return retcos; +} + +int pktio_classifier_init(pktio_entry_t *entry) { - (void)pmr_set_id; - (void)src_pktio; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + classifier_t *cls; + if (entry == NULL) + return -1; + cls = &entry->s.cls; + cls->num_pmr = 0; + cls->flow_set = 0; + cls->error_cos = NULL; + cls->default_cos = NULL; + cls->headroom = 0; + cls->skip = 0; return 0; } + +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt) +{ + pktio_entry_t *entry; + queue_entry_t *queue; + cos_t *cos; + odp_packet_hdr_t *pkt_hdr; + uint8_t *pkt_addr; + + entry = get_pktio_entry(pktio); + if (entry == NULL) + return -1; + + pkt_hdr = odp_packet_hdr(pkt); + pkt_addr = odp_packet_addr(pkt); + + /* Matching PMR and selecting the CoS for the packet*/ + cos = pktio_select_cos(entry, pkt_addr, pkt_hdr); + if (cos == NULL) + return -1; + + /* Enqueuing the Packet based on the CoS */ + READ_LOCK(&cos->s.lock); + queue = cos->s.queue; + READ_UNLOCK(&cos->s.lock); + return queue_enq(queue, odp_buf_to_hdr((odp_buffer_t)pkt)); +} + +cos_t *pktio_select_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr) +{ + pmr_t *pmr; + cos_t *cos = NULL; + int i; + classifier_t *cls = &entry->s.cls; + + /* Return error cos for error packet */ + if (pkt_hdr->error_flags.all) + return cls->error_cos; + /* Calls all the PMRs attached at the PKTIO level*/ + for (i = 0; i < cls->num_pmr; i++) { + pmr = entry->s.cls.pmr[i]; + if (pmr) { + cos = match_pmr_cos(pmr->s.cos, pkt_addr, pmr, pkt_hdr); + if (cos) + return cos; + } + } + + cos = match_qos_cos(entry, pkt_addr, pkt_hdr); + if (cos) + return cos; + + return cls->default_cos; +} + +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + uint8_t dscp; + cos_t *cos = NULL; + odph_ipv4hdr_t *ipv4; + odph_ipv6hdr_t *ipv6; + + READ_LOCK(&l3_cos->lock); + if (hdr->input_flags.l3 && hdr->input_flags.ipv4) { + ipv4 = (odph_ipv4hdr_t *)(pkt_addr + hdr->l3_offset); + dscp = ODPH_IPV4HDR_DSCP(ipv4->tos); + cos = l3_cos->cos[dscp]; + } else if (hdr->input_flags.l3 && hdr->input_flags.ipv6) { + ipv6 = (odph_ipv6hdr_t *)(pkt_addr + hdr->l3_offset); + dscp = ODPH_IPV6HDR_DSCP(ipv6->ver_tc_flow); + cos = l3_cos->cos[dscp]; + } + READ_UNLOCK(&l3_cos->lock); + + return cos; +} + +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + uint8_t qos; + cos_t *cos = NULL; + odph_ethhdr_t *eth; + odph_vlanhdr_t *vlan; + + if (hdr->input_flags.l2 && hdr->input_flags.vlan && + hdr->input_flags.eth) { + eth = (odph_ethhdr_t *)(pkt_addr + hdr->l2_offset); + vlan = (odph_vlanhdr_t *)(ð->type); + qos = ((vlan->tci >> 13) & 0xFF); + READ_LOCK(&l2_cos->lock); + cos = l2_cos->cos[qos]; + READ_UNLOCK(&l2_cos->lock); + } + return cos; +} + +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + classifier_t *cls = &entry->s.cls; + pmr_l2_cos_t *l2_cos; + pmr_l3_cos_t *l3_cos; + cos_t *cos; + + READ_LOCK(&cls->lock); + l2_cos = &cls->l2_cos_table; + l3_cos = &cls->l3_cos_table; + READ_UNLOCK(&cls->lock); + + if (cls->l3_precedence) { + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); + if (cos) + return cos; + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); + if (cos) + return cos; + } else { + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); + if (cos) + return cos; + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); + if (cos) + return cos; + } + return NULL; +} diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 672b3d6..c661231 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -54,6 +54,10 @@ int odp_init_global(odp_init_t *params ODP_UNUSED, ODP_ERR("ODP crypto init failed.\n"); return -1; } + if (odp_classification_init_global()) { + ODP_ERR("ODP crypto init failed.\n"); + return -1; + } return 0; } diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index 5c8146a..27f7b0a 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -11,32 +11,23 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include #include #include -typedef struct { - pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; -} pktio_table_t; - static pktio_table_t *pktio_tbl; - -static pktio_entry_t *get_entry(odp_pktio_t id) -{ - if (odp_unlikely(id == ODP_PKTIO_INVALID || - id > ODP_CONFIG_PKTIO_ENTRIES)) - return NULL; - - return &pktio_tbl->entries[id - 1]; -} +/* pktio pointer entries ( for inlines) */ +void *pktio_entry_ptr[ODP_CONFIG_PKTIO_ENTRIES]; int odp_pktio_init_global(void) { @@ -58,10 +49,12 @@ int odp_pktio_init_global(void) memset(pktio_tbl, 0, sizeof(pktio_table_t)); for (id = 1; id <= ODP_CONFIG_PKTIO_ENTRIES; ++id) { - pktio_entry = get_entry(id); + pktio_entry = &pktio_tbl->entries[id - 1]; odp_spinlock_init(&pktio_entry->s.lock); + odp_rwlock_init(&pktio_entry->s.cls.lock); + pktio_entry_ptr[id - 1] = pktio_entry; /* Create a default output queue for each pktio resource */ snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); name[ODP_QUEUE_NAME_LEN-1] = '\0'; @@ -108,12 +101,25 @@ static void unlock_entry(pktio_entry_t *entry) odp_spinlock_unlock(&entry->s.lock); } +static void lock_entry_classifier(pktio_entry_t *entry) +{ + odp_spinlock_lock(&entry->s.lock); + odp_rwlock_write_lock(&entry->s.cls.lock); +} + +static void unlock_entry_classifier(pktio_entry_t *entry) +{ + odp_rwlock_write_unlock(&entry->s.cls.lock); + odp_spinlock_unlock(&entry->s.lock); +} + static void init_pktio_entry(pktio_entry_t *entry) { set_taken(entry); entry->s.inq_default = ODP_QUEUE_INVALID; memset(&entry->s.pkt_sock, 0, sizeof(entry->s.pkt_sock)); memset(&entry->s.pkt_sock_mmap, 0, sizeof(entry->s.pkt_sock_mmap)); + pktio_classifier_init(entry); } static odp_pktio_t alloc_lock_pktio_entry(void) @@ -125,13 +131,13 @@ static odp_pktio_t alloc_lock_pktio_entry(void) for (i = 0; i < ODP_CONFIG_PKTIO_ENTRIES; ++i) { entry = &pktio_tbl->entries[i]; if (is_free(entry)) { - lock_entry(entry); + lock_entry_classifier(entry); if (is_free(entry)) { init_pktio_entry(entry); id = i + 1; return id; /* return with entry locked! */ } - unlock_entry(entry); + unlock_entry_classifier(entry); } } @@ -140,7 +146,7 @@ static odp_pktio_t alloc_lock_pktio_entry(void) static int free_pktio_entry(odp_pktio_t id) { - pktio_entry_t *entry = get_entry(id); + pktio_entry_t *entry = get_pktio_entry(id); if (entry == NULL) return -1; @@ -164,7 +170,7 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) } /* if successful, alloc_pktio_entry() returns with the entry locked */ - pktio_entry = get_entry(id); + pktio_entry = get_pktio_entry(id); if (!pktio_entry) return ODP_PKTIO_INVALID; @@ -200,14 +206,14 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) close_pkt_sock(&pktio_entry->s.pkt_sock); } - unlock_entry(pktio_entry); + unlock_entry_classifier(pktio_entry); free_pktio_entry(id); ODP_ERR("Unable to init any I/O type.\n"); return ODP_PKTIO_INVALID; done: strncpy(pktio_entry->s.name, dev, IFNAMSIZ); - unlock_entry(pktio_entry); + unlock_entry_classifier(pktio_entry); return id; } @@ -216,7 +222,7 @@ int odp_pktio_close(odp_pktio_t id) pktio_entry_t *entry; int res = -1; - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) return -1; @@ -255,7 +261,7 @@ odp_pktio_t odp_pktio_get_input(odp_packet_t pkt) int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); int pkts; int i; @@ -293,7 +299,7 @@ int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); int pkts; if (pktio_entry == NULL) @@ -323,7 +329,7 @@ int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) int odp_pktio_inq_setdef(odp_pktio_t id, odp_queue_t queue) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); queue_entry_t *qentry = queue_to_qentry(queue); if (pktio_entry == NULL || qentry == NULL) @@ -353,7 +359,7 @@ int odp_pktio_inq_remdef(odp_pktio_t id) odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); if (pktio_entry == NULL) return ODP_QUEUE_INVALID; @@ -363,7 +369,7 @@ odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); if (pktio_entry == NULL) return ODP_QUEUE_INVALID; @@ -423,7 +429,7 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) odp_buffer_t buf; odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; - int pkts, i; + int pkts, i, j; buf_hdr = queue_deq(qentry); if (buf_hdr != NULL) @@ -433,12 +439,15 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) if (pkts <= 0) return NULL; - for (i = 0; i < pkts; ++i) { + for (i = 0, j = 0; i < pkts; ++i) { buf = odp_packet_to_buffer(pkt_tbl[i]); - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); + buf_hdr = odp_buf_to_hdr(buf); + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) + tmp_hdr_tbl[j++] = buf_hdr; } - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); + if (j) + queue_enq_multi(qentry, tmp_hdr_tbl, j); buf_hdr = tmp_hdr_tbl[0]; return buf_hdr; } @@ -454,8 +463,9 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) int nbr; odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; + odp_buffer_hdr_t *tmp_hdr; odp_buffer_t buf; - int pkts, i; + int pkts, i, j; nbr = queue_deq_multi(qentry, buf_hdr, num); if (odp_unlikely(nbr > num)) @@ -472,12 +482,15 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) if (pkts <= 0) return nbr; - for (i = 0; i < pkts; ++i) { + for (i = 0, j = 0; i < pkts; ++i) { buf = odp_packet_to_buffer(pkt_tbl[i]); - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); + tmp_hdr = odp_buf_to_hdr(buf); + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) + tmp_hdr_tbl[j++] = tmp_hdr; } - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); + if (j) + queue_enq_multi(qentry, tmp_hdr_tbl, j); return nbr; } @@ -493,7 +506,7 @@ int odp_pktio_set_mtu(odp_pktio_t id, int mtu) return -1; } - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) { ODP_DBG("pktio entry %d does not exist\n", id); return -1; @@ -523,7 +536,7 @@ int odp_pktio_mtu(odp_pktio_t id) struct ifreq ifr; int ret; - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) { ODP_DBG("pktio entry %d does not exist\n", id); return -1;