| Message ID | 1417105945-28700-2-git-send-email-bala.manoharan@linaro.org |
|---|---|
| State | New |
| Headers | show |
Hi, I took some time to go through all the code. For the moment there is no support for selecting the pool based on the Class of Service, packets are allocated before hand. This is enough for this initial effort, of course. Also the drop policy is also not used, I suppose because of the first reason. The parsing is still performed with the odp_packet_parse function, the classification implementation relies on having the header field flags properly set. I mainly have minor stuff, there is one thing about locking that looks like more work, unless I'm wrong about it. All in all I think it's a good starting point, we would really need some test / sample app to carefully test everything at this point and then we can continue with improving the classifier. See comments below. On Thu, Nov 27, 2014 at 6:32 PM, Balasubramanian Manoharan <bala.manoharan@linaro.org> wrote: > This patch contains classification implementation for ODP v1.0. > > The salient features of this classification version are as follows: > * Attaches PMR, PMR_SET to a Pktio entry > * Attaches CoS values for L2 and L3 QoS to a Pktio entry > * Selects ClassOfService for a packet based on PMR, L2 QoS and L3 QoS values > * Selects a default CoS if packet does not match any of the assigned rules > * Selects an Error CoS for an Error packet > * Enqueues the packet to the queue associated with the selected CoS > > This patch also adds classifier object to pktio entry and moves static inline > functions to header files in Buffer and PKTIO modules. > > Signed-off-by: Balasubramanian Manoharan <bala.manoharan@linaro.org> > --- > V3: Patch has been merged as independently compilable unit > helper/include/odph_ip.h | 6 + > platform/linux-generic/include/api/odp.h | 1 + > .../include/odp_buffer_pool_internal.h | 10 + > .../include/odp_classification_datamodel.h | 199 +++++ > .../include/odp_classification_inlines.h | 259 +++++++ > .../include/odp_classification_internal.h | 173 +++++ > platform/linux-generic/include/odp_internal.h | 2 + > .../linux-generic/include/odp_packet_io_internal.h | 17 + > platform/linux-generic/odp_buffer_pool.c | 10 - > platform/linux-generic/odp_classification.c | 831 +++++++++++++++++++-- > platform/linux-generic/odp_init.c | 4 + > platform/linux-generic/odp_packet_io.c | 85 ++- > 12 files changed, 1475 insertions(+), 122 deletions(-) > create mode 100644 platform/linux-generic/include/odp_classification_datamodel.h > create mode 100644 platform/linux-generic/include/odp_classification_inlines.h > create mode 100644 platform/linux-generic/include/odp_classification_internal.h > > diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h > index 2c83c0f..f78724e 100644 > --- a/helper/include/odph_ip.h > +++ b/helper/include/odph_ip.h > @@ -35,6 +35,9 @@ extern "C" { > /** @internal Returns IPv4 header length */ > #define ODPH_IPV4HDR_IHL(ver_ihl) ((ver_ihl) & 0x0f) > > +/** @internal Returns IPv4 DSCP */ > +#define ODPH_IPV4HDR_DSCP(tos) (((tos) & 0xfc) >> 2) > + > /** @internal Returns IPv4 Don't fragment */ > #define ODPH_IPV4HDR_FLAGS_DONT_FRAG(frag_offset) ((frag_offset) & 0x4000) > > @@ -47,6 +50,9 @@ extern "C" { > /** @internal Returns true if IPv4 packet is a fragment */ > #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff) > > +/** @internal Returns IPv4 DSCP */ > +#define ODPH_IPV6HDR_DSCP(ver_tc_flow) (uint8_t)((((ver_tc_flow) & 0x0fc00000) >> 22) & 0xff) > + > /** IPv4 header */ > typedef struct ODP_PACKED { > uint8_t ver_ihl; /**< Version / Header length */ > diff --git a/platform/linux-generic/include/api/odp.h b/platform/linux-generic/include/api/odp.h > index 6e4f69e..b7b1ca9 100644 > --- a/platform/linux-generic/include/api/odp.h > +++ b/platform/linux-generic/include/api/odp.h > @@ -47,6 +47,7 @@ extern "C" { > #include <odp_packet_flags.h> > #include <odp_packet_io.h> > #include <odp_crypto.h> > +#include <odp_classification.h> > #include <odp_rwlock.h> > > #ifdef __cplusplus > diff --git a/platform/linux-generic/include/odp_buffer_pool_internal.h b/platform/linux-generic/include/odp_buffer_pool_internal.h > index e0210bd..bdbefff 100644 > --- a/platform/linux-generic/include/odp_buffer_pool_internal.h > +++ b/platform/linux-generic/include/odp_buffer_pool_internal.h > @@ -64,6 +64,12 @@ struct pool_entry_s { > size_t hdr_size; > }; > > +typedef union pool_entry_u { > + struct pool_entry_s s; > + > + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; > + > +} pool_entry_t; nit: moving these structures to internal headers (odp_buffer_pool_internal.h and odp_packet_io_internal.h) would have probably been best done in a separate patch, like a prerequisite to implementing the classifier. > > extern void *pool_entry_ptr[]; > > @@ -73,6 +79,10 @@ static inline void *get_pool_entry(uint32_t pool_id) > return pool_entry_ptr[pool_id]; > } > > +static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) > +{ > + return pool_hdl - 1; > +} > > static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf) > { > diff --git a/platform/linux-generic/include/odp_classification_datamodel.h b/platform/linux-generic/include/odp_classification_datamodel.h > new file mode 100644 > index 0000000..f7c9fb5 > --- /dev/null > +++ b/platform/linux-generic/include/odp_classification_datamodel.h > @@ -0,0 +1,199 @@ > +/* Copyright (c) 2014, Linaro Limited > + * All rights reserved. > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > + > +/** > + * @file > + * > + * ODP Classification Datamodel > + * Describes the classification internal data model > + */ > + > +#ifndef ODP_CLASSIFICATION_DATAMODEL_H_ > +#define ODP_CLASSIFICATION_DATAMODEL_H_ > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp_rwlock.h> > +#include <odp_classification.h> > +#include <odp_buffer_pool_internal.h> > +#include <odp_packet_internal.h> > +#include <odp_packet_io_internal.h> > +#include <odp_queue_internal.h> > + > +/* Maximum Class Of Service Entry */ > +#define ODP_COS_MAX_ENTRY 64 > +/* Maximum PMR Set Entry */ > +#define ODP_PMRSET_MAX_ENTRY 64 > +/* Maximum PMR Entry */ > +#define ODP_PMR_MAX_ENTRY 64 > +/* Maximum PMR Terms in a PMR Set */ > +#define ODP_PMRTERM_MAX 8 > +/* Maximum PMRs attached in PKTIO Level */ > +#define ODP_PKTIO_MAX_PMR 8 > +/* L2 Priority Bits */ > +#define ODP_COS_L2_QOS_BITS 3 > +/* Max L2 QoS value */ > +#define ODP_COS_MAX_L2_QOS (1 << ODP_COS_L2_QOS_BITS) > +/* L2 DSCP Bits */ > +#define ODP_COS_L3_QOS_BITS 6 > +/* Max L3 QoS Value */ > +#define ODP_COS_MAX_L3_QOS (1 << ODP_COS_L3_QOS_BITS) > +/* Max PMR Term bits */ > +#define ODP_PMR_TERM_BYTES_MAX 8 > + > +/* forward declaration */ > +typedef union pmr_u pmr_t; > + > +/** > +Packet Matching Rule Term Value > + > +Stores the Term and Value mapping for a PMR. > +The maximum size of value currently supported in 64 bits > +**/ > +typedef struct pmr_term_value { > + odp_pmr_match_type_e match_type; /**< Packet Match Type*/ > + odp_pmr_term_e term; /* PMR Term */ I would suggest keeping the same format with odp_pmr_match_t in odp_classification.h, with term inside the structure for mask and range. > + union { > + struct { > + uint64_t val; > + uint64_t mask; > + } mask; /**< Match a masked set of bits */ > + struct { > + uint64_t val1; > + uint64_t val2; > + } range; /**< Match an integer range */ > + }; > +} pmr_term_value_t; > + > +/* > +Class Of Service > +*/ > +struct cos_s { > + odp_rwlock_t lock; /* cos rwlock */ > + queue_entry_t *queue; /* Associated Queue */ > + odp_queue_group_t queue_group; /* Associated Queue Group */ > + pool_entry_t *pool; /* Associated Buffer pool */ > + pmr_t *pmr; /* Associated PMR */ > + bool valid; /* validity Flag */ > + odp_drop_e drop_policy; /* Associated Drop Policy */ > + odp_cos_flow_set_t flow_set; /* Assigned Flow Set */ > + char name[ODP_COS_NAME_LEN]; /* name */ > + size_t headroom; /* Headroom for this CoS */ > +}; > + > +typedef union cos_u { > + struct cos_s s; > + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct cos_s))]; > +} cos_t; > + > + > +/** > +Packet Matching Rule > + > +**/ > +struct pmr_s { > + odp_rwlock_t lock; /* pmr rwlock*/ > + cos_t *cos; /* Associated CoS */ > + odp_atomic_u32_t count; /* num of packets matching this rule */ > + uint16_t num_pmr; /* num of PMR Term Values*/ > + bool valid; /* Validity Flag */ > + pmr_term_value_t pmr_term_value[1]; /* Associated PMR Term */ > +}; > + > +typedef union pmr_u { > + struct pmr_s s; > + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_s))]; > +} pmr_t; > + > +/** > +Packet Matching Rule Set > + > +This structure is implemented as a extension over struct pmr_s > +In order to use same pointer to access both pmr_s and pmr_set_s > +'num_pmr' value is used to differentiate between pmr_s and pmr_set_s struct > +**/ > +struct pmr_set_s { > + pmr_t pmr; > + pmr_term_value_t pmr_term_value[ODP_PMRTERM_MAX - 1]; > + /* List of associated PMR Terms */ > +}; > + > +typedef union pmr_set_u { > + struct pmr_set_s s; > + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_set_s))]; > +} pmr_set_t; > + > +/** > +L2 QoS and CoS Map > + > +This structure holds the mapping between L2 QoS value and > +corresponding cos_t object > +**/ > +typedef struct pmr_l2_cos { > + odp_rwlock_t lock; /* pmr_l2_cos rwlock */ > + cos_t *cos[ODP_COS_MAX_L2_QOS]; /* Array of CoS objects */ > +} pmr_l2_cos_t; > + > +/** > +L3 QoS and CoS Map > + > +This structure holds the mapping between L3 QoS value and > +corresponding cos_t object > +**/ > +typedef struct pmr_l3_cos { > + odp_rwlock_t lock; /* pmr_l3_cos rwlock */ > + cos_t *cos[ODP_COS_MAX_L3_QOS]; /* Array of CoS objects */ > +} pmr_l3_cos_t; > + > +/** > +Linux Generic Classifier > + > +This structure is stored in pktio_entry and holds all > +the classifier configuration value. > +**/ > +typedef struct classifier { > + odp_rwlock_t lock; /*pktio_cos rwlock */ > + uint8_t num_pmr; /* num of PMRs linked to given PKTIO*/ > + bool l3_precedence; /* L3 QoS precedence */ > + odp_cos_flow_set_t flow_set; /* Flow Set to be calculated > + for this pktio */ > + pmr_l2_cos_t l2_cos_table; /* L2 QoS-CoS table map */ > + pmr_l3_cos_t l3_cos_table; /* L3 Qos-CoS table map */ > + pmr_t *pmr[ODP_PKTIO_MAX_PMR]; /* PMRs linked with this PKTIO */ > + cos_t *error_cos; /* Associated Error CoS */ > + cos_t *default_cos; /* Associated Default CoS */ > + size_t headroom; /* Pktio Headroom */ > + size_t skip; /* Pktio Skip Offset */ > +} classifier_t; > + > +/** > +Class of Service Table > +**/ > +typedef struct odp_cos_table { > + cos_t cos_entry[ODP_COS_MAX_ENTRY]; > +} cos_tbl_t; > + > +/** > +PMR set table > +**/ > +typedef struct pmr_set_tbl { > + pmr_set_t pmr_set[ODP_PMRSET_MAX_ENTRY]; > +} pmr_set_tbl_t; > + > +/** > +PMR table > +**/ > +typedef struct pmr_tbl { > + pmr_t pmr[ODP_PMR_MAX_ENTRY]; > +} pmr_tbl_t; > + > +#ifdef __cplusplus > +} > +#endif > +#endif > diff --git a/platform/linux-generic/include/odp_classification_inlines.h b/platform/linux-generic/include/odp_classification_inlines.h > new file mode 100644 > index 0000000..6b20119 > --- /dev/null > +++ b/platform/linux-generic/include/odp_classification_inlines.h > @@ -0,0 +1,259 @@ > +/* Copyright (c) 2014, Linaro Limited > + * All rights reserved. > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > + > +/** > + * @file > + * > + * ODP Classification Inlines > + * Classification Inlines Functions > + */ > +#ifndef __ODP_CLASSIFICATION_INLINES_H_ > +#define __ODP_CLASSIFICATION_INLINES_H_ > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp_debug.h> > +#include <odph_eth.h> > +#include <odph_ip.h> > +#include <odph_udp.h> > +#include <odph_tcp.h> > + > +/* PMR term value verification function > +These functions verify the given PMR term value with the value in the packet > +These following functions return 1 on success and 0 on failure > +*/ > + > +static inline int verify_pmr_packet_len(odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (pkt_hdr->frame_len & > + term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= pkt_hdr->frame_len) && > + (pkt_hdr->frame_len <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > +static inline int verify_pmr_ip_proto(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + odph_ipv4hdr_t *ip; > + uint8_t proto; > + if (!pkt_hdr->input_flags.ipv4) > + return 0; > + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); > + proto = ip->proto; > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (proto & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= proto) && > + (proto <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_ipv4_saddr(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + odph_ipv4hdr_t *ip; > + uint32_t ipaddr; > + if (!pkt_hdr->input_flags.ipv4) > + return 0; > + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); > + ipaddr = odp_be_to_cpu_32(ip->src_addr); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= ipaddr) && > + (ipaddr <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_ipv4_daddr(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + odph_ipv4hdr_t *ip; > + uint32_t ipaddr; > + if (!pkt_hdr->input_flags.ipv4) > + return 0; > + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); > + ipaddr = odp_be_to_cpu_32(ip->dst_addr); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= ipaddr) && > + (ipaddr <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_tcp_sport(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + uint16_t sport; > + odph_tcphdr_t *tcp; > + if (!pkt_hdr->input_flags.tcp) > + return 0; > + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); > + sport = odp_be_to_cpu_16(tcp->src_port); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (sport & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= sport) && > + (sport <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_tcp_dport(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + uint16_t dport; > + odph_tcphdr_t *tcp; > + if (!pkt_hdr->input_flags.tcp) > + return 0; > + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); > + dport = odp_be_to_cpu_16(tcp->dst_port); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (dport & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= dport) && > + (dport <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_udp_dport(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + uint16_t dport; > + odph_udphdr_t *udp; > + if (!pkt_hdr->input_flags.udp) > + return 0; > + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); > + dport = odp_be_to_cpu_16(udp->dst_port); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (dport & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= dport) && > + (dport <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > +static inline int verify_pmr_udp_sport(uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr, > + pmr_term_value_t *term_value) > +{ > + uint16_t sport; > + odph_udphdr_t *udp; > + if (!pkt_hdr->input_flags.udp) > + return 0; > + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); > + sport = odp_be_to_cpu_16(udp->src_port); > + if (term_value->match_type == ODP_PMR_MASK) { > + if (term_value->mask.val == (sport & term_value->mask.mask)) > + return 1; > + } else { > + if ((term_value->range.val1 <= sport) && > + (sport <= term_value->range.val2)) > + return 1; > + } > + return 0; > +} > + > +static inline int verify_pmr_dmac(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > + > +static inline int verify_pmr_ipv6_saddr(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_ipv6_daddr(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_vlan_id_0(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_vlan_id_x(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_ipsec_spi(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_ld_vni(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_eth_type_0(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +static inline int verify_pmr_eth_type_x(uint8_t *pkt_addr ODP_UNUSED, > + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, > + pmr_term_value_t *term_value ODP_UNUSED) > +{ > + ODP_UNIMPLEMENTED(); > + return 0; > +} > +#ifdef __cplusplus > +} > +#endif > +#endif > diff --git a/platform/linux-generic/include/odp_classification_internal.h b/platform/linux-generic/include/odp_classification_internal.h > new file mode 100644 > index 0000000..fd2c6af > --- /dev/null > +++ b/platform/linux-generic/include/odp_classification_internal.h > @@ -0,0 +1,173 @@ > +/* Copyright (c) 2014, Linaro Limited > + * All rights reserved. > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > + > +/** > + * @file > + * > + * ODP Classification Internal > + * Describes the classification internal Functions > + */ > + > +#ifndef __ODP_CLASSIFICATION_INTERNAL_H_ > +#define __ODP_CLASSIFICATION_INTERNAL_H_ > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <odp_classification.h> > +#include <odp_queue.h> > +#include <odp_packet_internal.h> > +#include <odp_packet_io.h> > +#include <odp_packet_io_internal.h> > +#include <odp_classification_datamodel.h> > + > +/** Classification Internal function **/ > + > +/** > +@internal > +Select a CoS for the given Packet based on pktio > + > +This function will call all the PMRs associated with a pktio for > +a given packet and will return the matched COS object. > +This function will check PMR, L2 and L3 QoS COS object associated > +with the PKTIO interface. > + > +Returns the default cos if the packet does not match any PMR > +Returns the error_cos if the packet has an error > +**/ > +cos_t *pktio_select_cos(pktio_entry_t *pktio, uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr); > + > +/** > +@internal > +match_qos_cos > + > +Select a CoS for the given Packet based on QoS values > +This function returns the COS object matching the L2 and L3 QoS > +based on the l3_preference value of the pktio > +**/ > +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr); > +/** > +Packet Classifier > + > +Start function for Packet Classifier > +This function calls Classifier module internal functions for a given packet and > +enqueues the packet to specific Queue based on PMR and CoS selected. > +**/ > +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt); > +/** > +Packet IO classifier init > + > +This function does initialization of classifier object associated with pktio. > +This function should be called during pktio initialization. > +**/ > +int pktio_classifier_init(pktio_entry_t *pktio); > + > +/** > +@internal > +match_pmr_cos > + > +Match a PMR chain with a Packet and return matching CoS > +This function gets called recursively to check the chained PMR Term value > +with the packet. > + > +**/ > +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, > + odp_packet_hdr_t *hdr); > +/** > +@internal > +CoS associated with L3 QoS value > + > +This function returns the CoS associated with L3 QoS value > +**/ > +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr); > + > +/** > +@internal > +CoS associated with L2 QoS value > + > +This function returns the CoS associated with L2 QoS value > +**/ > +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr); > +/** > +@internal > +Flow Signature Calculation > + > +This function calculates the Flow Signature for a packet based on > +CoS and updates in Packet Meta Data > +**/ > +int update_flow_signature(uint8_t *pkt_addr, cos_t *cos); > + > +/** > +@internal > +Allocate a odp_pmr_set_t Handle > +*/ > +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr); > + > +/** > +@internal > +Allocate a odp_pmr_t Handle > +*/ > +odp_pmr_t alloc_pmr(pmr_t **pmr); > + > +/** > +@internal > +Pointer to pmr_set_t Handle > +This function checks for validity of pmr_set_t Handle > +*/ > +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id); > + > +/** > +@internal > +Pointer to pmr_set_t Handle > +*/ > +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id); > + > +/** > +@internal > +Pointer to pmr_set_t Handle > +This function checks for validity of pmr_set_t Handle > +*/ > +pmr_t *get_pmr_entry(odp_pmr_t pmr_id); > + > +/** > +@internal > +Pointer to pmr_set_t Handle > +*/ > +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id); > + > +/** > +@internal > +Pointer to odp_cos_t Handle > +*/ > +cos_t *get_cos_entry(odp_cos_t cos_id); > + > +/** > +@internal > +Pointer to odp_cos_t Handle > +This function checks for validity of odp_cos_t Handle > +*/ > +cos_t *get_cos_entry_internal(odp_cos_t cos_id); > + > +/** > +@internal > +Verify PMR with a Packet > + > +This function goes through each PMR_TERM value in pmr_t structure and > +calls verification function for each term.Returns 1 if PMR matches or 0 > +Otherwise. > +**/ > +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr); > + > +#ifdef __cplusplus > +} > +#endif > +#endif > diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h > index f8c1596..04c1030 100644 > --- a/platform/linux-generic/include/odp_internal.h > +++ b/platform/linux-generic/include/odp_internal.h > @@ -32,6 +32,8 @@ int odp_buffer_pool_init_global(void); > int odp_pktio_init_global(void); > int odp_pktio_init_local(void); > > +int odp_classification_init_global(void); > + > int odp_queue_init_global(void); > > int odp_crypto_init_global(void); > diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h > index 0bc1e21..218e9d0 100644 > --- a/platform/linux-generic/include/odp_packet_io_internal.h > +++ b/platform/linux-generic/include/odp_packet_io_internal.h > @@ -20,6 +20,7 @@ extern "C" { > > #include <odp_spinlock.h> > #include <odp_packet_socket.h> > +#include <odp_classification_datamodel.h> > > #include <linux/if.h> > > @@ -40,6 +41,7 @@ struct pktio_entry { > odp_pktio_type_t type; /**< pktio type */ > pkt_sock_t pkt_sock; /**< using socket API for IO */ > pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for IO */ > + classifier_t cls; /**< classifier linked with this pktio*/ > char name[IFNAMSIZ]; /**< name of pktio provided to > pktio_open() */ > }; > @@ -49,6 +51,21 @@ typedef union { > uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pktio_entry))]; > } pktio_entry_t; > > +typedef struct { > + pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; > +} pktio_table_t; > + > +extern void *pktio_entry_ptr[]; Is it possible to use pktio_tbl like it was oringially instead of using this statically allocated array? > + > + > +static inline pktio_entry_t *get_pktio_entry(odp_pktio_t id) > +{ > + if (odp_unlikely(id == ODP_PKTIO_INVALID || > + id > ODP_CONFIG_PKTIO_ENTRIES)) > + return NULL; > + > + return pktio_entry_ptr[id - 1]; > +} > #ifdef __cplusplus > } > #endif > diff --git a/platform/linux-generic/odp_buffer_pool.c b/platform/linux-generic/odp_buffer_pool.c > index 6a0a6b2..b5bd613 100644 > --- a/platform/linux-generic/odp_buffer_pool.c > +++ b/platform/linux-generic/odp_buffer_pool.c > @@ -56,12 +56,6 @@ typedef struct { > } odp_any_buffer_hdr_t; > > > -typedef union pool_entry_u { > - struct pool_entry_s s; > - > - uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; > - > -} pool_entry_t; > > > typedef struct pool_table_t { > @@ -86,10 +80,6 @@ static inline odp_buffer_pool_t pool_index_to_handle(uint32_t pool_id) > } > > > -static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) > -{ > - return pool_hdl -1; > -} > > > static inline void set_handle(odp_buffer_hdr_t *hdr, > diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c > index 190d71e..ecf97a0 100644 > --- a/platform/linux-generic/odp_classification.c > +++ b/platform/linux-generic/odp_classification.c > @@ -2,68 +2,289 @@ > #include <odp_align.h> > #include <odp_queue.h> > #include <odp_debug.h> > +#include <odp_internal.h> > #include <odp_debug_internal.h> > +#include <odp_packet_internal.h> > #include <odp_packet_io.h> > +#include <odp_packet_io_internal.h> > +#include <odp_classification_datamodel.h> > +#include <odp_classification_inlines.h> > +#include <odp_classification_internal.h> > +#include <odp_buffer_pool_internal.h> > +#include <odp_shared_memory.h> > +#include <odph_eth.h> > +#include <string.h> > +#include <odp_rwlock.h> > > -odp_cos_t odp_cos_create(const char *name) > +#define WRITE_LOCK(a) odp_rwlock_write_lock(a) > +#define WRITE_UNLOCK(a) odp_rwlock_write_unlock(a) > +#define LOCK_INIT(a) odp_rwlock_init(a) > + > +#define READ_LOCK(a) odp_rwlock_write_lock(a) > +#define READ_UNLOCK(a) odp_rwlock_write_unlock(a) > + > +static cos_tbl_t *cos_tbl; > +static pmr_set_tbl_t *pmr_set_tbl; > +static pmr_tbl_t *pmr_tbl; > + > +cos_t *get_cos_entry_internal(odp_cos_t cos_id) > +{ > + return &(cos_tbl->cos_entry[cos_id - 1]); > +} > + > +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id) > +{ > + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); > +} > + > +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id) > +{ > + return &(pmr_tbl->pmr[pmr_id - 1]); > +} > + > +int odp_classification_init_global(void) > { > - (void) name; > - ODP_UNIMPLEMENTED(); > + odp_shm_t cos_shm; > + odp_shm_t pmr_shm; > + odp_shm_t pmr_set_shm; > + int i; > + > + cos_shm = odp_shm_reserve("odp_cos_pools", I would suggest naming these to shm_odp_xxx_table, pools would indicate you allocate some pools from the shm. > + sizeof(cos_tbl_t), > + sizeof(cos_t), 0); > + > + cos_tbl = odp_shm_addr(cos_shm); > + if (cos_tbl == NULL) { > + odp_shm_free(cos_shm); > + return -1; > + } > + > + memset(cos_tbl, 0, sizeof(cos_tbl_t)); > + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { > + /* init locks */ > + cos_t *cos = get_cos_entry_internal(i + 1); > + LOCK_INIT(&cos->s.lock); > + } > + > + pmr_shm = odp_shm_reserve("odp_pmr_pools", > + sizeof(pmr_tbl_t), > + sizeof(pmr_t), 0); > + pmr_tbl = odp_shm_addr(pmr_shm); > + if (pmr_tbl == NULL) { > + odp_shm_free(pmr_shm); > + return -1; > + } > + > + memset(pmr_tbl, 0, sizeof(pmr_tbl_t)); > + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { > + /* init locks */ > + pmr_t *pmr = get_pmr_entry_internal(i + 1); > + LOCK_INIT(&pmr->s.lock); > + } > + > + pmr_set_shm = odp_shm_reserve("odp_pmr_set_pools", > + sizeof(pmr_set_tbl_t), > + sizeof(pmr_set_t), 0); > + pmr_set_tbl = odp_shm_addr(pmr_set_shm); > + if (pmr_set_tbl == NULL) { > + odp_shm_free(pmr_set_shm); > + return -1; > + } > + > + memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); > + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { > + /* init locks */ > + pmr_set_t *pmr = get_pmr_set_entry_internal(i + 1); > + LOCK_INIT(&pmr->s.pmr.s.lock); > + } > + > return 0; > } > > +odp_cos_t odp_cos_create(const char *name) > +{ > + int i; > + > + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { > + WRITE_LOCK(&cos_tbl->cos_entry[i].s.lock); > + if (0 == cos_tbl->cos_entry[i].s.valid) { > + strncpy(cos_tbl->cos_entry[i].s.name, name, > + ODP_COS_NAME_LEN - 1); > + cos_tbl->cos_entry[i].s.name[ODP_COS_NAME_LEN - 1] = 0; > + cos_tbl->cos_entry[i].s.pmr = NULL; > + cos_tbl->cos_entry[i].s.queue = NULL; cost_tbl->cos_entry[i].s.queue_group is uninitialized. It's never used but I still think it should be properly reset here to make future work easier. The drop_policy also, if we have a default value for it. > + cos_tbl->cos_entry[i].s.pool = NULL; > + cos_tbl->cos_entry[i].s.flow_set = 0; > + cos_tbl->cos_entry[i].s.headroom = 0; > + cos_tbl->cos_entry[i].s.valid = 1; > + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); > + return i + 1; > + } > + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); > + } > + return ODP_COS_INVALID; > +} > + > +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) > +{ > + int i; > + > + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { > + WRITE_LOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); > + if (0 == pmr_set_tbl->pmr_set[i].s.pmr.s.valid) { > + pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; > + pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; > + pmr_set_tbl->pmr_set[i].s.pmr.s.cos = NULL; > + *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; > + odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] > + .s.pmr.s.count, 0); > + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); > + return i + 1; > + } > + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); > + } > + return ODP_PMR_INVAL; > +} > + > +odp_pmr_t alloc_pmr(pmr_t **pmr) > +{ > + int i; > + > + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { > + WRITE_LOCK(&pmr_tbl->pmr[i].s.lock); > + if (0 == pmr_tbl->pmr[i].s.valid) { > + pmr_tbl->pmr[i].s.valid = 1; > + pmr_tbl->pmr[i].s.cos = NULL; > + odp_atomic_init_u32(&pmr_tbl->pmr[i].s.count, 0); > + pmr_tbl->pmr[i].s.num_pmr = 0; > + *pmr = &pmr_tbl->pmr[i]; > + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); > + return i + 1; > + } > + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); > + } > + return ODP_PMR_INVAL; > +} > + > + > +cos_t *get_cos_entry(odp_cos_t cos_id) > +{ > + if (cos_id > ODP_COS_MAX_ENTRY || cos_id == ODP_COS_INVALID) > + return NULL; > + if (cos_tbl->cos_entry[cos_id - 1].s.valid == 0) In other parts of the file (e.g. odp_cos_destroy) access to 'valid' is done under lock. Don't we need it here too? If so, you might consider implementing a get_locked type of routine to keep subsequent access to fields in the entry in the same critical section. > + return NULL; > + return &(cos_tbl->cos_entry[cos_id - 1]); > +} > + > + > +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) > +{ > + if (pmr_set_id > ODP_PMRSET_MAX_ENTRY || pmr_set_id == ODP_PMR_INVAL) > + return NULL; > + if (pmr_set_tbl->pmr_set[pmr_set_id - 1].s.pmr.s.valid == 0) > + return NULL; > + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); > +} > + > +pmr_t *get_pmr_entry(odp_pmr_t pmr_id) > +{ > + if (pmr_id > ODP_PMR_MAX_ENTRY || pmr_id == ODP_PMR_INVAL) > + return NULL; > + if (pmr_tbl->pmr[pmr_id - 1].s.valid == 0) > + return NULL; > + return &(pmr_tbl->pmr[pmr_id - 1]); > +} > + > int odp_cos_destroy(odp_cos_t cos_id) > { > - (void)cos_id; > - ODP_UNIMPLEMENTED(); > + cos_t *cos = get_cos_entry(cos_id); > + if (NULL == cos) > + return -1; > + > + WRITE_LOCK(&cos->s.lock); > + cos->s.valid = 0; > + WRITE_UNLOCK(&cos->s.lock); > return 0; > } > > int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) > { > - (void)cos_id; > - (void)queue_id; > - ODP_UNIMPLEMENTED(); > + cos_t *cos = get_cos_entry(cos_id); > + if (cos == NULL) > + return -1; > + > + WRITE_LOCK(&cos->s.lock); > + cos->s.queue = queue_to_qentry(queue_id); > + WRITE_UNLOCK(&cos->s.lock); > return 0; > } > > int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) > { > - (void)cos_id; > - (void)drop_policy; > - ODP_UNIMPLEMENTED(); > + cos_t *cos = get_cos_entry(cos_id); > + if (cos == NULL) > + return -1; > + > + WRITE_LOCK(&cos->s.lock); > + cos->s.drop_policy = drop_policy; The drop_policy is never used, I assume that's part of a future work? > + WRITE_UNLOCK(&cos->s.lock); > return 0; > } > > int odp_pktio_set_default_cos(odp_pktio_t pktio_in, odp_cos_t default_cos) > { > - (void)pktio_in; > - (void)default_cos; > - ODP_UNIMPLEMENTED(); > + pktio_entry_t *entry; > + cos_t *cos; > + entry = get_pktio_entry(pktio_in); > + if (entry == NULL) > + return -1; > + cos = get_cos_entry(default_cos); > + if (cos == NULL) > + return -1; > + > + WRITE_LOCK(&entry->s.cls.lock); > + entry->s.cls.default_cos = cos; > + WRITE_UNLOCK(&entry->s.cls.lock); > + > return 0; > } > > int odp_pktio_set_error_cos(odp_pktio_t pktio_in, odp_cos_t error_cos) > { > - (void)pktio_in; > - (void)error_cos; > - ODP_UNIMPLEMENTED(); > + pktio_entry_t *entry; > + cos_t *cos; > + > + entry = get_pktio_entry(pktio_in); > + if (entry == NULL) > + return -1; > + cos = get_cos_entry(error_cos); > + if (cos == NULL) > + return -1; > + WRITE_LOCK(&entry->s.cls.lock); > + entry->s.cls.error_cos = cos; > + WRITE_UNLOCK(&entry->s.cls.lock); > return 0; > } > > int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset) > { > - (void)pktio_in; > - (void)offset; > - ODP_UNIMPLEMENTED(); > + pktio_entry_t *entry = get_pktio_entry(pktio_in); > + if (entry == NULL) > + return -1; > + WRITE_LOCK(&entry->s.cls.lock); > + entry->s.cls.skip = offset; > + WRITE_UNLOCK(&entry->s.cls.lock); > return 0; > } > > -int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom) > +int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom) > { > - (void)port_id; > - (void)headroom; > - ODP_UNIMPLEMENTED(); > + pktio_entry_t *entry = get_pktio_entry(pktio_in); > + if (entry == NULL) > + return -1; > + WRITE_LOCK(&entry->s.cls.lock); > + entry->s.cls.headroom = headroom; > + WRITE_UNLOCK(&entry->s.cls.lock); > return 0; > } > > @@ -72,11 +293,26 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, > uint8_t qos_table[], > odp_cos_t cos_table[]) > { > - (void)pktio_in; > - (void)num_qos; > - (void)qos_table; > - (void)cos_table; > - ODP_UNIMPLEMENTED(); > + pmr_l2_cos_t *l2_cos; > + size_t i; > + cos_t *cos; > + pktio_entry_t *entry = get_pktio_entry(pktio_in); > + if (entry == NULL) > + return -1; > + READ_LOCK(&entry->s.cls.lock); > + l2_cos = &entry->s.cls.l2_cos_table; > + READ_UNLOCK(&entry->s.cls.lock); > + > + WRITE_LOCK(&l2_cos->lock); > + /* Update the L2 QoS table*/ > + for (i = 0; i < num_qos; i++) { > + cos = get_cos_entry(cos_table[i]); > + if (cos != NULL) { > + if (ODP_COS_MAX_L2_QOS > qos_table[i]) > + l2_cos->cos[qos_table[i]] = cos; > + } > + } > + WRITE_UNLOCK(&l2_cos->lock); > return 0; > } > > @@ -86,12 +322,28 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, > odp_cos_t cos_table[], > bool l3_preference) > { > - (void)pktio_in; > - (void)num_qos; > - (void)qos_table; > - (void)cos_table; > - (void)l3_preference; > - ODP_UNIMPLEMENTED(); > + pmr_l3_cos_t *l3_cos; > + size_t i; > + pktio_entry_t *entry = get_pktio_entry(pktio_in); > + cos_t *cos; > + > + if (entry == NULL) > + return -1; > + WRITE_LOCK(&entry->s.cls.lock); > + entry->s.cls.l3_precedence = l3_preference; > + l3_cos = &entry->s.cls.l3_cos_table; > + WRITE_UNLOCK(&entry->s.cls.lock); > + > + WRITE_LOCK(&l3_cos->lock); > + /* Update the L3 QoS table*/ > + for (i = 0; i < num_qos; i++) { > + cos = get_cos_entry(cos_table[i]); > + if (cos != NULL) { > + if (ODP_COS_MAX_L3_QOS > qos_table[i]) > + l3_cos->cos[qos_table[i]] = cos; > + } > + } > + WRITE_UNLOCK(&l3_cos->lock); > return 0; > } > > @@ -100,12 +352,23 @@ odp_pmr_t odp_pmr_create_match(odp_pmr_term_e term, > const void *mask, > size_t val_sz) > { > - (void)term; > - (void)val; > - (void)mask; > - (void)val_sz; > - ODP_UNIMPLEMENTED(); > - return 0; > + pmr_t *pmr; > + odp_pmr_t id; > + > + id = alloc_pmr(&pmr); > + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) > + return ODP_PMR_INVAL; > + > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.num_pmr = 1; > + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; > + pmr->s.pmr_term_value[0].term = term; > + pmr->s.pmr_term_value[0].mask.val = 0; > + pmr->s.pmr_term_value[0].mask.mask = 0; > + memcpy(&pmr->s.pmr_term_value[0].mask.val, val, val_sz); > + memcpy(&pmr->s.pmr_term_value[0].mask.mask, mask, val_sz); > + WRITE_UNLOCK(&pmr->s.lock); > + return id; > } > > odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, > @@ -113,18 +376,33 @@ odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, > const void *val2, > size_t val_sz) > { > - (void)term; > - (void)val1; > - (void)val2; > - (void)val_sz; > - ODP_UNIMPLEMENTED(); > - return 0; > + pmr_t *pmr; > + odp_pmr_t id; > + > + id = alloc_pmr(&pmr); > + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) > + return ODP_PMR_INVAL; > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.num_pmr = 1; > + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; > + pmr->s.pmr_term_value[0].term = term; > + pmr->s.pmr_term_value[0].range.val1 = 0; > + pmr->s.pmr_term_value[0].range.val2 = 0; > + memcpy(&pmr->s.pmr_term_value[0].range.val1, val1, val_sz); > + memcpy(&pmr->s.pmr_term_value[0].range.val2, val2, val_sz); > + WRITE_UNLOCK(&pmr->s.lock); > + return id; > } > > int odp_pmr_destroy(odp_pmr_t pmr_id) > { > - (void)pmr_id; > - ODP_UNIMPLEMENTED(); > + pmr_t *pmr = get_pmr_entry(pmr_id); > + > + if (pmr == NULL) > + return -1; > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.valid = 0; > + WRITE_UNLOCK(&pmr->s.lock); > return 0; > } > > @@ -132,64 +410,465 @@ int odp_pktio_pmr_cos(odp_pmr_t pmr_id, > odp_pktio_t src_pktio, > odp_cos_t dst_cos) > { > - (void)pmr_id; > - (void)src_pktio; > - (void)dst_cos; > - ODP_UNIMPLEMENTED(); > + uint8_t num_pmr; > + pktio_entry_t *pktio_entry; > + pmr_t *pmr; > + cos_t *cos; > + > + pktio_entry = get_pktio_entry(src_pktio); > + if (pktio_entry == NULL) > + return -1; > + > + pmr = get_pmr_entry(pmr_id); > + if (pmr == NULL) > + return -1; > + > + cos = get_cos_entry(dst_cos); > + if (cos == NULL) > + return -1; > + > + num_pmr = pktio_entry->s.cls.num_pmr; > + if (num_pmr >= ODP_PKTIO_MAX_PMR) num_pmr > ODP_PKTIO_MAX_PMR is a serious error, I suggest using ODP_ASSERT for such case > + return -1; > + > + WRITE_LOCK(&pktio_entry->s.cls.lock); > + pktio_entry->s.cls.pmr[num_pmr] = pmr; > + pktio_entry->s.cls.num_pmr++; > + WRITE_UNLOCK(&pktio_entry->s.cls.lock); > + > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.cos = cos; > + WRITE_UNLOCK(&pmr->s.lock); > return 0; > } > > int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t dst_cos) > { > - (void)pmr_id; > - (void)src_cos; > - (void)dst_cos; > - ODP_UNIMPLEMENTED(); > + cos_t *cos_src = get_cos_entry(src_cos); > + cos_t *cos_dst = get_cos_entry(dst_cos); > + pmr_t *pmr = get_pmr_entry(pmr_id); > + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) > + return -1; > + > + WRITE_LOCK(&cos_src->s.lock); > + cos_src->s.pmr = pmr; > + WRITE_UNLOCK(&cos_src->s.lock); > + > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.cos = cos_dst; > + WRITE_UNLOCK(&pmr->s.lock); > return 0; > } > > signed long odp_pmr_match_count(odp_pmr_t pmr_id) > { > - (void)pmr_id; > - ODP_UNIMPLEMENTED(); > - return 0; > + pmr_t *pmr = get_pmr_entry(pmr_id); > + if (pmr == NULL) > + return -1; > + return (signed long)odp_atomic_load_u32(&pmr->s.count); > } > > unsigned long long odp_pmr_terms_cap(void) > { > - ODP_UNIMPLEMENTED(); > - return 0; > + unsigned long long term_cap = 0; > + > + term_cap |= (1 << ODP_PMR_LEN); > + term_cap |= (1 << ODP_PMR_IPPROTO); > + term_cap |= (1 << ODP_PMR_UDP_DPORT); > + term_cap |= (1 << ODP_PMR_TCP_DPORT); > + term_cap |= (1 << ODP_PMR_UDP_SPORT); > + term_cap |= (1 << ODP_PMR_TCP_SPORT); > + term_cap |= (1 << ODP_PMR_SIP_ADDR); > + term_cap |= (1 << ODP_PMR_DIP_ADDR); > + return term_cap; > } > > unsigned odp_pmr_terms_avail(void) > { > - ODP_UNIMPLEMENTED(); > - return 0; > + unsigned count = 0; > + int i; > + > + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) > + if (!pmr_tbl->pmr[i].s.valid) > + count++; > + return count; > } > > int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, > odp_pmr_set_t *pmr_set_id) > { > - (void)num_terms; > - (void)terms; > - (void)pmr_set_id; > - ODP_UNIMPLEMENTED(); > - return 0; > + pmr_t *pmr; > + int i; > + uint32_t id; > + int val_sz; > + int count = 0; > + Check num_terms <= ODP_PMRTERM_MAX ? > + id = alloc_pmr_set(&pmr); > + if (id == ODP_PMR_INVAL) { > + *pmr_set_id = id; > + return -1; > + } > + > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.num_pmr = num_terms; > + for (i = 0; i < num_terms; i++) { > + pmr->s.pmr_term_value[i].match_type = terms[i].match_type; > + if (terms[i].match_type == ODP_PMR_MASK) { > + val_sz = terms[i].mask.val_sz; > + if (val_sz > ODP_PMR_TERM_BYTES_MAX) > + continue; > + pmr->s.pmr_term_value[i].term = terms[i].mask.term; > + pmr->s.pmr_term_value[i].mask.val = 0; > + pmr->s.pmr_term_value[i].mask.mask = 0; > + memcpy(&pmr->s.pmr_term_value[i].mask.val, > + terms[i].mask.val, val_sz); > + memcpy(&pmr->s.pmr_term_value[i].mask.mask, > + terms[i].mask.mask, val_sz); > + } else { > + val_sz = terms[i].range.val_sz; > + if (val_sz > ODP_PMR_TERM_BYTES_MAX) > + continue; > + pmr->s.pmr_term_value[i].term = terms[i].range.term; > + pmr->s.pmr_term_value[i].range.val1 = 0; > + pmr->s.pmr_term_value[i].range.val2 = 0; > + memcpy(&pmr->s.pmr_term_value[i].range.val1, > + terms[i].range.val1, val_sz); > + memcpy(&pmr->s.pmr_term_value[i].range.val2, > + terms[i].range.val2, val_sz); > + } > + count++; > + } > + WRITE_UNLOCK(&pmr->s.lock); > + *pmr_set_id = id; > + return count; > } > > int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) > { > - (void)pmr_set_id; > - ODP_UNIMPLEMENTED(); > + pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); > + if (pmr_set == NULL) > + return -1; > + WRITE_LOCK(&pmr_set->s.pmr.s.lock); > + pmr_set->s.pmr.s.valid = 0; > + WRITE_UNLOCK(&pmr_set->s.pmr.s.lock); > return 0; > } > > int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t src_pktio, > - odp_cos_t dst_cos) > + odp_cos_t dst_cos) > +{ > + uint8_t num_pmr; > + pktio_entry_t *pktio_entry; > + pmr_t *pmr; > + cos_t *cos; > + > + pktio_entry = get_pktio_entry(src_pktio); > + if (pktio_entry == NULL) > + return -1; > + > + pmr = (pmr_t *)get_pmr_set_entry(pmr_set_id); > + if (pmr == NULL) > + return -1; > + > + cos = get_cos_entry(dst_cos); > + if (cos == NULL) > + return -1; > + > + num_pmr = pktio_entry->s.cls.num_pmr; > + if (num_pmr >= ODP_PKTIO_MAX_PMR) > + return -1; > + > + WRITE_LOCK(&pktio_entry->s.cls.lock); > + pktio_entry->s.cls.pmr[num_pmr] = pmr; > + pktio_entry->s.cls.num_pmr++; > + WRITE_UNLOCK(&pktio_entry->s.cls.lock); > + > + WRITE_LOCK(&pmr->s.lock); > + pmr->s.cos = cos; > + WRITE_UNLOCK(&pmr->s.lock); > + return 0; > +} > + > +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) > +{ > + int pmr_failure = 0; > + int num_pmr; > + int i; > + pmr_term_value_t *term_value; > + > + READ_LOCK(&pmr->s.lock); > + if (!pmr->s.valid) { > + READ_UNLOCK(&pmr->s.lock); > + return 0; > + } > + num_pmr = pmr->s.num_pmr; > + > + /* Iterate through list of PMR Term values in a pmr_t */ > + for (i = 0; i < num_pmr; i++) { > + term_value = &pmr->s.pmr_term_value[i]; > + switch (term_value->term) { > + case ODP_PMR_LEN: > + if (!verify_pmr_packet_len(pkt_hdr, term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_ETHTYPE_0: > + if (!verify_pmr_eth_type_0(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_ETHTYPE_X: > + if (!verify_pmr_eth_type_x(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_VLAN_ID_0: > + if (!verify_pmr_vlan_id_0(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_VLAN_ID_X: > + if (!verify_pmr_vlan_id_x(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_DMAC: > + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_IPPROTO: > + if (!verify_pmr_ip_proto(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_UDP_DPORT: > + if (!verify_pmr_udp_dport(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_TCP_DPORT: > + if (!verify_pmr_tcp_dport(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_UDP_SPORT: > + if (!verify_pmr_udp_sport(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_TCP_SPORT: > + if (!verify_pmr_tcp_sport(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_SIP_ADDR: > + if (!verify_pmr_ipv4_saddr(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_DIP_ADDR: > + if (!verify_pmr_ipv4_daddr(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_SIP6_ADDR: > + if (!verify_pmr_ipv6_saddr(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_DIP6_ADDR: > + if (!verify_pmr_ipv6_daddr(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_IPSEC_SPI: > + if (!verify_pmr_ipsec_spi(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_LD_VNI: > + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, > + term_value)) > + pmr_failure = 1; > + break; > + case ODP_PMR_INNER_HDR_OFF: > + break; > + } nit: the closing brace (and the block below) should be on tab further to match beginning of switch statement > + if (pmr_failure) { > + READ_UNLOCK(&pmr->s.lock); > + return false; > + } > + } > + READ_UNLOCK(&pmr->s.lock); > + odp_atomic_inc_u32(&pmr->s.count); > + return true; > +} > + > +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, > + odp_packet_hdr_t *hdr) > +{ > + cos_t *retcos = NULL; > + > + if (cos == NULL || pmr == NULL) > + return NULL; > + > + READ_LOCK(&cos->s.lock); > + if (!cos->s.valid) { > + READ_UNLOCK(&cos->s.lock); > + return NULL; > + } > + READ_UNLOCK(&cos->s.lock); > + > + if (verify_pmr(pmr, pkt_addr, hdr)) { > + /** This gets called recursively to check all the PMRs in > + * a PMR chain */ > + retcos = match_pmr_cos(pmr->s.cos, pkt_addr, cos->s.pmr, hdr); > + if (!retcos) > + return cos; > + } > + return retcos; > +} > + > +int pktio_classifier_init(pktio_entry_t *entry) > { > - (void)pmr_set_id; > - (void)src_pktio; > - (void)dst_cos; > - ODP_UNIMPLEMENTED(); > + classifier_t *cls; > + if (entry == NULL) > + return -1; > + cls = &entry->s.cls; > + cls->num_pmr = 0; > + cls->flow_set = 0; > + cls->error_cos = NULL; > + cls->default_cos = NULL; > + cls->headroom = 0; > + cls->skip = 0; > return 0; > } > + > +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt) > +{ > + pktio_entry_t *entry; > + queue_entry_t *queue; > + cos_t *cos; > + odp_packet_hdr_t *pkt_hdr; > + uint8_t *pkt_addr; > + > + entry = get_pktio_entry(pktio); > + if (entry == NULL) > + return -1; > + > + pkt_hdr = odp_packet_hdr(pkt); > + pkt_addr = odp_packet_addr(pkt); > + > + /* Matching PMR and selecting the CoS for the packet*/ > + cos = pktio_select_cos(entry, pkt_addr, pkt_hdr); > + if (cos == NULL) > + return -1; > + > + /* Enqueuing the Packet based on the CoS */ > + READ_LOCK(&cos->s.lock); > + queue = cos->s.queue; > + READ_UNLOCK(&cos->s.lock); > + return queue_enq(queue, odp_buf_to_hdr((odp_buffer_t)pkt)); > +} > + > +cos_t *pktio_select_cos(pktio_entry_t *entry, uint8_t *pkt_addr, > + odp_packet_hdr_t *pkt_hdr) > +{ > + pmr_t *pmr; > + cos_t *cos = NULL; > + int i; > + classifier_t *cls = &entry->s.cls; > + > + /* Return error cos for error packet */ > + if (pkt_hdr->error_flags.all) > + return cls->error_cos; > + /* Calls all the PMRs attached at the PKTIO level*/ > + for (i = 0; i < cls->num_pmr; i++) { > + pmr = entry->s.cls.pmr[i]; > + if (pmr) { > + cos = match_pmr_cos(pmr->s.cos, pkt_addr, pmr, pkt_hdr); > + if (cos) > + return cos; > + } > + } > + > + cos = match_qos_cos(entry, pkt_addr, pkt_hdr); > + if (cos) > + return cos; > + > + return cls->default_cos; > +} > + > +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr) > +{ > + uint8_t dscp; > + cos_t *cos = NULL; > + odph_ipv4hdr_t *ipv4; > + odph_ipv6hdr_t *ipv6; > + > + READ_LOCK(&l3_cos->lock); > + if (hdr->input_flags.l3 && hdr->input_flags.ipv4) { > + ipv4 = (odph_ipv4hdr_t *)(pkt_addr + hdr->l3_offset); > + dscp = ODPH_IPV4HDR_DSCP(ipv4->tos); > + cos = l3_cos->cos[dscp]; > + } else if (hdr->input_flags.l3 && hdr->input_flags.ipv6) { > + ipv6 = (odph_ipv6hdr_t *)(pkt_addr + hdr->l3_offset); > + dscp = ODPH_IPV6HDR_DSCP(ipv6->ver_tc_flow); > + cos = l3_cos->cos[dscp]; > + } > + READ_UNLOCK(&l3_cos->lock); > + > + return cos; > +} > + > +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr) > +{ > + uint8_t qos; > + cos_t *cos = NULL; > + odph_ethhdr_t *eth; > + odph_vlanhdr_t *vlan; > + > + if (hdr->input_flags.l2 && hdr->input_flags.vlan && > + hdr->input_flags.eth) { > + eth = (odph_ethhdr_t *)(pkt_addr + hdr->l2_offset); > + vlan = (odph_vlanhdr_t *)(ð->type); > + qos = ((vlan->tci >> 13) & 0xFF); > + READ_LOCK(&l2_cos->lock); > + cos = l2_cos->cos[qos]; > + READ_UNLOCK(&l2_cos->lock); > + } > + return cos; > +} > + > +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, > + odp_packet_hdr_t *hdr) > +{ > + classifier_t *cls = &entry->s.cls; > + pmr_l2_cos_t *l2_cos; > + pmr_l3_cos_t *l3_cos; > + cos_t *cos; > + > + READ_LOCK(&cls->lock); > + l2_cos = &cls->l2_cos_table; > + l3_cos = &cls->l3_cos_table; > + READ_UNLOCK(&cls->lock); > + > + if (cls->l3_precedence) { > + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); > + if (cos) > + return cos; > + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); > + if (cos) > + return cos; > + } else { > + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); > + if (cos) > + return cos; > + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); > + if (cos) > + return cos; > + } > + return NULL; > +} > diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c > index 672b3d6..c661231 100644 > --- a/platform/linux-generic/odp_init.c > +++ b/platform/linux-generic/odp_init.c > @@ -54,6 +54,10 @@ int odp_init_global(odp_init_t *params ODP_UNUSED, > ODP_ERR("ODP crypto init failed.\n"); > return -1; > } > + if (odp_classification_init_global()) { > + ODP_ERR("ODP crypto init failed.\n"); > + return -1; > + } > > return 0; > } > diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c > index 5c8146a..27f7b0a 100644 > --- a/platform/linux-generic/odp_packet_io.c > +++ b/platform/linux-generic/odp_packet_io.c > @@ -11,32 +11,23 @@ > #include <odp_packet_internal.h> > #include <odp_internal.h> > #include <odp_spinlock.h> > +#include <odp_rwlock.h> > #include <odp_shared_memory.h> > #include <odp_packet_socket.h> > #include <odp_hints.h> > #include <odp_config.h> > #include <odp_queue_internal.h> > #include <odp_schedule_internal.h> > +#include <odp_classification_internal.h> > #include <odp_debug.h> > > #include <string.h> > #include <sys/ioctl.h> > > -typedef struct { > - pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; > -} pktio_table_t; > - > static pktio_table_t *pktio_tbl; > > - > -static pktio_entry_t *get_entry(odp_pktio_t id) > -{ > - if (odp_unlikely(id == ODP_PKTIO_INVALID || > - id > ODP_CONFIG_PKTIO_ENTRIES)) > - return NULL; > - > - return &pktio_tbl->entries[id - 1]; > -} > +/* pktio pointer entries ( for inlines) */ > +void *pktio_entry_ptr[ODP_CONFIG_PKTIO_ENTRIES]; > > int odp_pktio_init_global(void) > { > @@ -58,10 +49,12 @@ int odp_pktio_init_global(void) > memset(pktio_tbl, 0, sizeof(pktio_table_t)); > > for (id = 1; id <= ODP_CONFIG_PKTIO_ENTRIES; ++id) { > - pktio_entry = get_entry(id); > + pktio_entry = &pktio_tbl->entries[id - 1]; > > odp_spinlock_init(&pktio_entry->s.lock); > + odp_rwlock_init(&pktio_entry->s.cls.lock); > > + pktio_entry_ptr[id - 1] = pktio_entry; > /* Create a default output queue for each pktio resource */ > snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); > name[ODP_QUEUE_NAME_LEN-1] = '\0'; > @@ -108,12 +101,25 @@ static void unlock_entry(pktio_entry_t *entry) > odp_spinlock_unlock(&entry->s.lock); > } > > +static void lock_entry_classifier(pktio_entry_t *entry) > +{ > + odp_spinlock_lock(&entry->s.lock); > + odp_rwlock_write_lock(&entry->s.cls.lock); > +} > + > +static void unlock_entry_classifier(pktio_entry_t *entry) > +{ > + odp_rwlock_write_unlock(&entry->s.cls.lock); > + odp_spinlock_unlock(&entry->s.lock); > +} > + > static void init_pktio_entry(pktio_entry_t *entry) > { > set_taken(entry); > entry->s.inq_default = ODP_QUEUE_INVALID; > memset(&entry->s.pkt_sock, 0, sizeof(entry->s.pkt_sock)); > memset(&entry->s.pkt_sock_mmap, 0, sizeof(entry->s.pkt_sock_mmap)); > + pktio_classifier_init(entry); > } > > static odp_pktio_t alloc_lock_pktio_entry(void) > @@ -125,13 +131,13 @@ static odp_pktio_t alloc_lock_pktio_entry(void) > for (i = 0; i < ODP_CONFIG_PKTIO_ENTRIES; ++i) { > entry = &pktio_tbl->entries[i]; > if (is_free(entry)) { > - lock_entry(entry); > + lock_entry_classifier(entry); > if (is_free(entry)) { > init_pktio_entry(entry); > id = i + 1; > return id; /* return with entry locked! */ > } > - unlock_entry(entry); > + unlock_entry_classifier(entry); > } > } > > @@ -140,7 +146,7 @@ static odp_pktio_t alloc_lock_pktio_entry(void) > > static int free_pktio_entry(odp_pktio_t id) > { > - pktio_entry_t *entry = get_entry(id); > + pktio_entry_t *entry = get_pktio_entry(id); > > if (entry == NULL) > return -1; > @@ -164,7 +170,7 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) > } > /* if successful, alloc_pktio_entry() returns with the entry locked */ > > - pktio_entry = get_entry(id); > + pktio_entry = get_pktio_entry(id); > if (!pktio_entry) > return ODP_PKTIO_INVALID; > > @@ -200,14 +206,14 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) > close_pkt_sock(&pktio_entry->s.pkt_sock); > } > > - unlock_entry(pktio_entry); > + unlock_entry_classifier(pktio_entry); > free_pktio_entry(id); > ODP_ERR("Unable to init any I/O type.\n"); > return ODP_PKTIO_INVALID; > > done: > strncpy(pktio_entry->s.name, dev, IFNAMSIZ); > - unlock_entry(pktio_entry); > + unlock_entry_classifier(pktio_entry); > return id; > } > > @@ -216,7 +222,7 @@ int odp_pktio_close(odp_pktio_t id) > pktio_entry_t *entry; > int res = -1; > > - entry = get_entry(id); > + entry = get_pktio_entry(id); > if (entry == NULL) > return -1; > > @@ -255,7 +261,7 @@ odp_pktio_t odp_pktio_get_input(odp_packet_t pkt) > > int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) > { > - pktio_entry_t *pktio_entry = get_entry(id); > + pktio_entry_t *pktio_entry = get_pktio_entry(id); > int pkts; > int i; > > @@ -293,7 +299,7 @@ int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) > > int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) > { > - pktio_entry_t *pktio_entry = get_entry(id); > + pktio_entry_t *pktio_entry = get_pktio_entry(id); > int pkts; > > if (pktio_entry == NULL) > @@ -323,7 +329,7 @@ int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) > > int odp_pktio_inq_setdef(odp_pktio_t id, odp_queue_t queue) > { > - pktio_entry_t *pktio_entry = get_entry(id); > + pktio_entry_t *pktio_entry = get_pktio_entry(id); > queue_entry_t *qentry = queue_to_qentry(queue); > > if (pktio_entry == NULL || qentry == NULL) > @@ -353,7 +359,7 @@ int odp_pktio_inq_remdef(odp_pktio_t id) > > odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) > { > - pktio_entry_t *pktio_entry = get_entry(id); > + pktio_entry_t *pktio_entry = get_pktio_entry(id); > > if (pktio_entry == NULL) > return ODP_QUEUE_INVALID; > @@ -363,7 +369,7 @@ odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) > > odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id) > { > - pktio_entry_t *pktio_entry = get_entry(id); > + pktio_entry_t *pktio_entry = get_pktio_entry(id); > > if (pktio_entry == NULL) > return ODP_QUEUE_INVALID; > @@ -423,7 +429,7 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) > odp_buffer_t buf; > odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; > odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; > - int pkts, i; > + int pkts, i, j; > > buf_hdr = queue_deq(qentry); > if (buf_hdr != NULL) > @@ -433,12 +439,15 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) > if (pkts <= 0) > return NULL; > > - for (i = 0; i < pkts; ++i) { > + for (i = 0, j = 0; i < pkts; ++i) { > buf = odp_packet_to_buffer(pkt_tbl[i]); > - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); > + buf_hdr = odp_buf_to_hdr(buf); > + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) > + tmp_hdr_tbl[j++] = buf_hdr; > } > > - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); > + if (j) > + queue_enq_multi(qentry, tmp_hdr_tbl, j); > buf_hdr = tmp_hdr_tbl[0]; > return buf_hdr; > } > @@ -454,8 +463,9 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) > int nbr; > odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; > odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; > + odp_buffer_hdr_t *tmp_hdr; > odp_buffer_t buf; > - int pkts, i; > + int pkts, i, j; > > nbr = queue_deq_multi(qentry, buf_hdr, num); > if (odp_unlikely(nbr > num)) > @@ -472,12 +482,15 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) > if (pkts <= 0) > return nbr; > > - for (i = 0; i < pkts; ++i) { > + for (i = 0, j = 0; i < pkts; ++i) { > buf = odp_packet_to_buffer(pkt_tbl[i]); > - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); > + tmp_hdr = odp_buf_to_hdr(buf); > + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) > + tmp_hdr_tbl[j++] = tmp_hdr; > } > > - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); > + if (j) > + queue_enq_multi(qentry, tmp_hdr_tbl, j); > return nbr; > } > > @@ -493,7 +506,7 @@ int odp_pktio_set_mtu(odp_pktio_t id, int mtu) > return -1; > } > > - entry = get_entry(id); > + entry = get_pktio_entry(id); > if (entry == NULL) { > ODP_DBG("pktio entry %d does not exist\n", id); > return -1; > @@ -523,7 +536,7 @@ int odp_pktio_mtu(odp_pktio_t id) > struct ifreq ifr; > int ret; > > - entry = get_entry(id); > + entry = get_pktio_entry(id); > if (entry == NULL) { > ODP_DBG("pktio entry %d does not exist\n", id); > return -1; > -- > 2.0.1.472.g6f92e5f > > > _______________________________________________ > lng-odp mailing list > lng-odp@lists.linaro.org > http://lists.linaro.org/mailman/listinfo/lng-odp
On Tuesday 02 December 2014 11:13 PM, Ciprian Barbu wrote: > Hi, > > I took some time to go through all the code. For the moment there is > no support for selecting the pool based on the Class of Service, > packets are allocated before hand. This is enough for this initial > effort, of course. Also the drop policy is also not used, I suppose > because of the first reason. The parsing is still performed with the > odp_packet_parse function, the classification implementation relies on > having the header field flags properly set. This current version does not support queue group, drop policy and pool selection based on CoS. The same will be taken up as enhancement after 1.0 > I mainly have minor stuff, there is one thing about locking that looks > like more work, unless I'm wrong about it. All in all I think it's a > good starting point, we would really need some test / sample app to > carefully test everything at this point and then we can continue with > improving the classifier. I am currently working on Cunit and an example program to test classification. > > See comments below. > > On Thu, Nov 27, 2014 at 6:32 PM, Balasubramanian Manoharan > <bala.manoharan@linaro.org> wrote: >> This patch contains classification implementation for ODP v1.0. >> >> The salient features of this classification version are as follows: >> * Attaches PMR, PMR_SET to a Pktio entry >> * Attaches CoS values for L2 and L3 QoS to a Pktio entry >> * Selects ClassOfService for a packet based on PMR, L2 QoS and L3 QoS values >> * Selects a default CoS if packet does not match any of the assigned rules >> * Selects an Error CoS for an Error packet >> * Enqueues the packet to the queue associated with the selected CoS >> >> This patch also adds classifier object to pktio entry and moves static inline >> functions to header files in Buffer and PKTIO modules. >> >> Signed-off-by: Balasubramanian Manoharan <bala.manoharan@linaro.org> >> --- >> V3: Patch has been merged as independently compilable unit >> helper/include/odph_ip.h | 6 + >> platform/linux-generic/include/api/odp.h | 1 + >> .../include/odp_buffer_pool_internal.h | 10 + >> .../include/odp_classification_datamodel.h | 199 +++++ >> .../include/odp_classification_inlines.h | 259 +++++++ >> .../include/odp_classification_internal.h | 173 +++++ >> platform/linux-generic/include/odp_internal.h | 2 + >> .../linux-generic/include/odp_packet_io_internal.h | 17 + >> platform/linux-generic/odp_buffer_pool.c | 10 - >> platform/linux-generic/odp_classification.c | 831 +++++++++++++++++++-- >> platform/linux-generic/odp_init.c | 4 + >> platform/linux-generic/odp_packet_io.c | 85 ++- >> 12 files changed, 1475 insertions(+), 122 deletions(-) >> create mode 100644 platform/linux-generic/include/odp_classification_datamodel.h >> create mode 100644 platform/linux-generic/include/odp_classification_inlines.h >> create mode 100644 platform/linux-generic/include/odp_classification_internal.h >> >> diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h >> index 2c83c0f..f78724e 100644 >> --- a/helper/include/odph_ip.h >> +++ b/helper/include/odph_ip.h >> @@ -35,6 +35,9 @@ extern "C" { >> /** @internal Returns IPv4 header length */ >> #define ODPH_IPV4HDR_IHL(ver_ihl) ((ver_ihl) & 0x0f) >> >> +/** @internal Returns IPv4 DSCP */ >> +#define ODPH_IPV4HDR_DSCP(tos) (((tos) & 0xfc) >> 2) >> + >> /** @internal Returns IPv4 Don't fragment */ >> #define ODPH_IPV4HDR_FLAGS_DONT_FRAG(frag_offset) ((frag_offset) & 0x4000) >> >> @@ -47,6 +50,9 @@ extern "C" { >> /** @internal Returns true if IPv4 packet is a fragment */ >> #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff) >> >> +/** @internal Returns IPv4 DSCP */ >> +#define ODPH_IPV6HDR_DSCP(ver_tc_flow) (uint8_t)((((ver_tc_flow) & 0x0fc00000) >> 22) & 0xff) >> + >> /** IPv4 header */ >> typedef struct ODP_PACKED { >> uint8_t ver_ihl; /**< Version / Header length */ >> diff --git a/platform/linux-generic/include/api/odp.h b/platform/linux-generic/include/api/odp.h >> index 6e4f69e..b7b1ca9 100644 >> --- a/platform/linux-generic/include/api/odp.h >> +++ b/platform/linux-generic/include/api/odp.h >> @@ -47,6 +47,7 @@ extern "C" { >> #include <odp_packet_flags.h> >> #include <odp_packet_io.h> >> #include <odp_crypto.h> >> +#include <odp_classification.h> >> #include <odp_rwlock.h> >> >> #ifdef __cplusplus >> diff --git a/platform/linux-generic/include/odp_buffer_pool_internal.h b/platform/linux-generic/include/odp_buffer_pool_internal.h >> index e0210bd..bdbefff 100644 >> --- a/platform/linux-generic/include/odp_buffer_pool_internal.h >> +++ b/platform/linux-generic/include/odp_buffer_pool_internal.h >> @@ -64,6 +64,12 @@ struct pool_entry_s { >> size_t hdr_size; >> }; >> >> +typedef union pool_entry_u { >> + struct pool_entry_s s; >> + >> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; >> + >> +} pool_entry_t; > nit: moving these structures to internal headers > (odp_buffer_pool_internal.h and odp_packet_io_internal.h) would have > probably been best done in a separate patch, like a prerequisite to > implementing the classifier. The initial patches were designed to have these as a separate patch but then it was decided that individual patches have to independently compilable. > >> extern void *pool_entry_ptr[]; >> >> @@ -73,6 +79,10 @@ static inline void *get_pool_entry(uint32_t pool_id) >> return pool_entry_ptr[pool_id]; >> } >> >> +static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) >> +{ >> + return pool_hdl - 1; >> +} >> >> static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf) >> { >> diff --git a/platform/linux-generic/include/odp_classification_datamodel.h b/platform/linux-generic/include/odp_classification_datamodel.h >> new file mode 100644 >> index 0000000..f7c9fb5 >> --- /dev/null >> +++ b/platform/linux-generic/include/odp_classification_datamodel.h >> @@ -0,0 +1,199 @@ >> +/* Copyright (c) 2014, Linaro Limited >> + * All rights reserved. >> + * >> + * SPDX-License-Identifier: BSD-3-Clause >> + */ >> + >> + >> +/** >> + * @file >> + * >> + * ODP Classification Datamodel >> + * Describes the classification internal data model >> + */ >> + >> +#ifndef ODP_CLASSIFICATION_DATAMODEL_H_ >> +#define ODP_CLASSIFICATION_DATAMODEL_H_ >> + >> +#ifdef __cplusplus >> +extern "C" { >> +#endif >> + >> +#include <odp_rwlock.h> >> +#include <odp_classification.h> >> +#include <odp_buffer_pool_internal.h> >> +#include <odp_packet_internal.h> >> +#include <odp_packet_io_internal.h> >> +#include <odp_queue_internal.h> >> + >> +/* Maximum Class Of Service Entry */ >> +#define ODP_COS_MAX_ENTRY 64 >> +/* Maximum PMR Set Entry */ >> +#define ODP_PMRSET_MAX_ENTRY 64 >> +/* Maximum PMR Entry */ >> +#define ODP_PMR_MAX_ENTRY 64 >> +/* Maximum PMR Terms in a PMR Set */ >> +#define ODP_PMRTERM_MAX 8 >> +/* Maximum PMRs attached in PKTIO Level */ >> +#define ODP_PKTIO_MAX_PMR 8 >> +/* L2 Priority Bits */ >> +#define ODP_COS_L2_QOS_BITS 3 >> +/* Max L2 QoS value */ >> +#define ODP_COS_MAX_L2_QOS (1 << ODP_COS_L2_QOS_BITS) >> +/* L2 DSCP Bits */ >> +#define ODP_COS_L3_QOS_BITS 6 >> +/* Max L3 QoS Value */ >> +#define ODP_COS_MAX_L3_QOS (1 << ODP_COS_L3_QOS_BITS) >> +/* Max PMR Term bits */ >> +#define ODP_PMR_TERM_BYTES_MAX 8 >> + >> +/* forward declaration */ >> +typedef union pmr_u pmr_t; >> + >> +/** >> +Packet Matching Rule Term Value >> + >> +Stores the Term and Value mapping for a PMR. >> +The maximum size of value currently supported in 64 bits >> +**/ >> +typedef struct pmr_term_value { >> + odp_pmr_match_type_e match_type; /**< Packet Match Type*/ >> + odp_pmr_term_e term; /* PMR Term */ > I would suggest keeping the same format with odp_pmr_match_t in > odp_classification.h, with term inside the structure for mask and > range. Since odp_pmr_term_e was common for both mask and range it was better for programming per se to have it outside of mask and range. >> + union { >> + struct { >> + uint64_t val; >> + uint64_t mask; >> + } mask; /**< Match a masked set of bits */ >> + struct { >> + uint64_t val1; >> + uint64_t val2; >> + } range; /**< Match an integer range */ >> + }; >> +} pmr_term_value_t; >> + >> +/* >> +Class Of Service >> +*/ >> +struct cos_s { >> + odp_rwlock_t lock; /* cos rwlock */ >> + queue_entry_t *queue; /* Associated Queue */ >> + odp_queue_group_t queue_group; /* Associated Queue Group */ >> + pool_entry_t *pool; /* Associated Buffer pool */ >> + pmr_t *pmr; /* Associated PMR */ >> + bool valid; /* validity Flag */ >> + odp_drop_e drop_policy; /* Associated Drop Policy */ >> + odp_cos_flow_set_t flow_set; /* Assigned Flow Set */ >> + char name[ODP_COS_NAME_LEN]; /* name */ >> + size_t headroom; /* Headroom for this CoS */ >> +}; >> + >> +typedef union cos_u { >> + struct cos_s s; >> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct cos_s))]; >> +} cos_t; >> + >> + >> +/** >> +Packet Matching Rule >> + >> +**/ >> +struct pmr_s { >> + odp_rwlock_t lock; /* pmr rwlock*/ >> + cos_t *cos; /* Associated CoS */ >> + odp_atomic_u32_t count; /* num of packets matching this rule */ >> + uint16_t num_pmr; /* num of PMR Term Values*/ >> + bool valid; /* Validity Flag */ >> + pmr_term_value_t pmr_term_value[1]; /* Associated PMR Term */ >> +}; >> + >> +typedef union pmr_u { >> + struct pmr_s s; >> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_s))]; >> +} pmr_t; >> + >> +/** >> +Packet Matching Rule Set >> + >> +This structure is implemented as a extension over struct pmr_s >> +In order to use same pointer to access both pmr_s and pmr_set_s >> +'num_pmr' value is used to differentiate between pmr_s and pmr_set_s struct >> +**/ >> +struct pmr_set_s { >> + pmr_t pmr; >> + pmr_term_value_t pmr_term_value[ODP_PMRTERM_MAX - 1]; >> + /* List of associated PMR Terms */ >> +}; >> + >> +typedef union pmr_set_u { >> + struct pmr_set_s s; >> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_set_s))]; >> +} pmr_set_t; >> + >> +/** >> +L2 QoS and CoS Map >> + >> +This structure holds the mapping between L2 QoS value and >> +corresponding cos_t object >> +**/ >> +typedef struct pmr_l2_cos { >> + odp_rwlock_t lock; /* pmr_l2_cos rwlock */ >> + cos_t *cos[ODP_COS_MAX_L2_QOS]; /* Array of CoS objects */ >> +} pmr_l2_cos_t; >> + >> +/** >> +L3 QoS and CoS Map >> + >> +This structure holds the mapping between L3 QoS value and >> +corresponding cos_t object >> +**/ >> +typedef struct pmr_l3_cos { >> + odp_rwlock_t lock; /* pmr_l3_cos rwlock */ >> + cos_t *cos[ODP_COS_MAX_L3_QOS]; /* Array of CoS objects */ >> +} pmr_l3_cos_t; >> + >> +/** >> +Linux Generic Classifier >> + >> +This structure is stored in pktio_entry and holds all >> +the classifier configuration value. >> +**/ >> +typedef struct classifier { >> + odp_rwlock_t lock; /*pktio_cos rwlock */ >> + uint8_t num_pmr; /* num of PMRs linked to given PKTIO*/ >> + bool l3_precedence; /* L3 QoS precedence */ >> + odp_cos_flow_set_t flow_set; /* Flow Set to be calculated >> + for this pktio */ >> + pmr_l2_cos_t l2_cos_table; /* L2 QoS-CoS table map */ >> + pmr_l3_cos_t l3_cos_table; /* L3 Qos-CoS table map */ >> + pmr_t *pmr[ODP_PKTIO_MAX_PMR]; /* PMRs linked with this PKTIO */ >> + cos_t *error_cos; /* Associated Error CoS */ >> + cos_t *default_cos; /* Associated Default CoS */ >> + size_t headroom; /* Pktio Headroom */ >> + size_t skip; /* Pktio Skip Offset */ >> +} classifier_t; >> + >> +/** >> +Class of Service Table >> +**/ >> +typedef struct odp_cos_table { >> + cos_t cos_entry[ODP_COS_MAX_ENTRY]; >> +} cos_tbl_t; >> + >> +/** >> +PMR set table >> +**/ >> +typedef struct pmr_set_tbl { >> + pmr_set_t pmr_set[ODP_PMRSET_MAX_ENTRY]; >> +} pmr_set_tbl_t; >> + >> +/** >> +PMR table >> +**/ >> +typedef struct pmr_tbl { >> + pmr_t pmr[ODP_PMR_MAX_ENTRY]; >> +} pmr_tbl_t; >> + >> +#ifdef __cplusplus >> +} >> +#endif >> +#endif >> diff --git a/platform/linux-generic/include/odp_classification_inlines.h b/platform/linux-generic/include/odp_classification_inlines.h >> new file mode 100644 >> index 0000000..6b20119 >> --- /dev/null >> +++ b/platform/linux-generic/include/odp_classification_inlines.h >> @@ -0,0 +1,259 @@ >> +/* Copyright (c) 2014, Linaro Limited >> + * All rights reserved. >> + * >> + * SPDX-License-Identifier: BSD-3-Clause >> + */ >> + >> + >> +/** >> + * @file >> + * >> + * ODP Classification Inlines >> + * Classification Inlines Functions >> + */ >> +#ifndef __ODP_CLASSIFICATION_INLINES_H_ >> +#define __ODP_CLASSIFICATION_INLINES_H_ >> + >> +#ifdef __cplusplus >> +extern "C" { >> +#endif >> + >> +#include <odp_debug.h> >> +#include <odph_eth.h> >> +#include <odph_ip.h> >> +#include <odph_udp.h> >> +#include <odph_tcp.h> >> + >> +/* PMR term value verification function >> +These functions verify the given PMR term value with the value in the packet >> +These following functions return 1 on success and 0 on failure >> +*/ >> + >> +static inline int verify_pmr_packet_len(odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (pkt_hdr->frame_len & >> + term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= pkt_hdr->frame_len) && >> + (pkt_hdr->frame_len <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> +static inline int verify_pmr_ip_proto(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + odph_ipv4hdr_t *ip; >> + uint8_t proto; >> + if (!pkt_hdr->input_flags.ipv4) >> + return 0; >> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >> + proto = ip->proto; >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (proto & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= proto) && >> + (proto <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_ipv4_saddr(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + odph_ipv4hdr_t *ip; >> + uint32_t ipaddr; >> + if (!pkt_hdr->input_flags.ipv4) >> + return 0; >> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >> + ipaddr = odp_be_to_cpu_32(ip->src_addr); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= ipaddr) && >> + (ipaddr <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_ipv4_daddr(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + odph_ipv4hdr_t *ip; >> + uint32_t ipaddr; >> + if (!pkt_hdr->input_flags.ipv4) >> + return 0; >> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >> + ipaddr = odp_be_to_cpu_32(ip->dst_addr); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= ipaddr) && >> + (ipaddr <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_tcp_sport(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + uint16_t sport; >> + odph_tcphdr_t *tcp; >> + if (!pkt_hdr->input_flags.tcp) >> + return 0; >> + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >> + sport = odp_be_to_cpu_16(tcp->src_port); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (sport & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= sport) && >> + (sport <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_tcp_dport(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + uint16_t dport; >> + odph_tcphdr_t *tcp; >> + if (!pkt_hdr->input_flags.tcp) >> + return 0; >> + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >> + dport = odp_be_to_cpu_16(tcp->dst_port); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (dport & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= dport) && >> + (dport <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_udp_dport(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + uint16_t dport; >> + odph_udphdr_t *udp; >> + if (!pkt_hdr->input_flags.udp) >> + return 0; >> + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >> + dport = odp_be_to_cpu_16(udp->dst_port); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (dport & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= dport) && >> + (dport <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> +static inline int verify_pmr_udp_sport(uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr, >> + pmr_term_value_t *term_value) >> +{ >> + uint16_t sport; >> + odph_udphdr_t *udp; >> + if (!pkt_hdr->input_flags.udp) >> + return 0; >> + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >> + sport = odp_be_to_cpu_16(udp->src_port); >> + if (term_value->match_type == ODP_PMR_MASK) { >> + if (term_value->mask.val == (sport & term_value->mask.mask)) >> + return 1; >> + } else { >> + if ((term_value->range.val1 <= sport) && >> + (sport <= term_value->range.val2)) >> + return 1; >> + } >> + return 0; >> +} >> + >> +static inline int verify_pmr_dmac(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> + >> +static inline int verify_pmr_ipv6_saddr(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_ipv6_daddr(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_vlan_id_0(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_vlan_id_x(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_ipsec_spi(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_ld_vni(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_eth_type_0(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +static inline int verify_pmr_eth_type_x(uint8_t *pkt_addr ODP_UNUSED, >> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >> + pmr_term_value_t *term_value ODP_UNUSED) >> +{ >> + ODP_UNIMPLEMENTED(); >> + return 0; >> +} >> +#ifdef __cplusplus >> +} >> +#endif >> +#endif >> diff --git a/platform/linux-generic/include/odp_classification_internal.h b/platform/linux-generic/include/odp_classification_internal.h >> new file mode 100644 >> index 0000000..fd2c6af >> --- /dev/null >> +++ b/platform/linux-generic/include/odp_classification_internal.h >> @@ -0,0 +1,173 @@ >> +/* Copyright (c) 2014, Linaro Limited >> + * All rights reserved. >> + * >> + * SPDX-License-Identifier: BSD-3-Clause >> + */ >> + >> + >> +/** >> + * @file >> + * >> + * ODP Classification Internal >> + * Describes the classification internal Functions >> + */ >> + >> +#ifndef __ODP_CLASSIFICATION_INTERNAL_H_ >> +#define __ODP_CLASSIFICATION_INTERNAL_H_ >> + >> +#ifdef __cplusplus >> +extern "C" { >> +#endif >> + >> +#include <odp_classification.h> >> +#include <odp_queue.h> >> +#include <odp_packet_internal.h> >> +#include <odp_packet_io.h> >> +#include <odp_packet_io_internal.h> >> +#include <odp_classification_datamodel.h> >> + >> +/** Classification Internal function **/ >> + >> +/** >> +@internal >> +Select a CoS for the given Packet based on pktio >> + >> +This function will call all the PMRs associated with a pktio for >> +a given packet and will return the matched COS object. >> +This function will check PMR, L2 and L3 QoS COS object associated >> +with the PKTIO interface. >> + >> +Returns the default cos if the packet does not match any PMR >> +Returns the error_cos if the packet has an error >> +**/ >> +cos_t *pktio_select_cos(pktio_entry_t *pktio, uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr); >> + >> +/** >> +@internal >> +match_qos_cos >> + >> +Select a CoS for the given Packet based on QoS values >> +This function returns the COS object matching the L2 and L3 QoS >> +based on the l3_preference value of the pktio >> +**/ >> +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr); >> +/** >> +Packet Classifier >> + >> +Start function for Packet Classifier >> +This function calls Classifier module internal functions for a given packet and >> +enqueues the packet to specific Queue based on PMR and CoS selected. >> +**/ >> +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt); >> +/** >> +Packet IO classifier init >> + >> +This function does initialization of classifier object associated with pktio. >> +This function should be called during pktio initialization. >> +**/ >> +int pktio_classifier_init(pktio_entry_t *pktio); >> + >> +/** >> +@internal >> +match_pmr_cos >> + >> +Match a PMR chain with a Packet and return matching CoS >> +This function gets called recursively to check the chained PMR Term value >> +with the packet. >> + >> +**/ >> +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, >> + odp_packet_hdr_t *hdr); >> +/** >> +@internal >> +CoS associated with L3 QoS value >> + >> +This function returns the CoS associated with L3 QoS value >> +**/ >> +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr); >> + >> +/** >> +@internal >> +CoS associated with L2 QoS value >> + >> +This function returns the CoS associated with L2 QoS value >> +**/ >> +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr); >> +/** >> +@internal >> +Flow Signature Calculation >> + >> +This function calculates the Flow Signature for a packet based on >> +CoS and updates in Packet Meta Data >> +**/ >> +int update_flow_signature(uint8_t *pkt_addr, cos_t *cos); >> + >> +/** >> +@internal >> +Allocate a odp_pmr_set_t Handle >> +*/ >> +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr); >> + >> +/** >> +@internal >> +Allocate a odp_pmr_t Handle >> +*/ >> +odp_pmr_t alloc_pmr(pmr_t **pmr); >> + >> +/** >> +@internal >> +Pointer to pmr_set_t Handle >> +This function checks for validity of pmr_set_t Handle >> +*/ >> +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id); >> + >> +/** >> +@internal >> +Pointer to pmr_set_t Handle >> +*/ >> +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id); >> + >> +/** >> +@internal >> +Pointer to pmr_set_t Handle >> +This function checks for validity of pmr_set_t Handle >> +*/ >> +pmr_t *get_pmr_entry(odp_pmr_t pmr_id); >> + >> +/** >> +@internal >> +Pointer to pmr_set_t Handle >> +*/ >> +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id); >> + >> +/** >> +@internal >> +Pointer to odp_cos_t Handle >> +*/ >> +cos_t *get_cos_entry(odp_cos_t cos_id); >> + >> +/** >> +@internal >> +Pointer to odp_cos_t Handle >> +This function checks for validity of odp_cos_t Handle >> +*/ >> +cos_t *get_cos_entry_internal(odp_cos_t cos_id); >> + >> +/** >> +@internal >> +Verify PMR with a Packet >> + >> +This function goes through each PMR_TERM value in pmr_t structure and >> +calls verification function for each term.Returns 1 if PMR matches or 0 >> +Otherwise. >> +**/ >> +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr); >> + >> +#ifdef __cplusplus >> +} >> +#endif >> +#endif >> diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h >> index f8c1596..04c1030 100644 >> --- a/platform/linux-generic/include/odp_internal.h >> +++ b/platform/linux-generic/include/odp_internal.h >> @@ -32,6 +32,8 @@ int odp_buffer_pool_init_global(void); >> int odp_pktio_init_global(void); >> int odp_pktio_init_local(void); >> >> +int odp_classification_init_global(void); >> + >> int odp_queue_init_global(void); >> >> int odp_crypto_init_global(void); >> diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h >> index 0bc1e21..218e9d0 100644 >> --- a/platform/linux-generic/include/odp_packet_io_internal.h >> +++ b/platform/linux-generic/include/odp_packet_io_internal.h >> @@ -20,6 +20,7 @@ extern "C" { >> >> #include <odp_spinlock.h> >> #include <odp_packet_socket.h> >> +#include <odp_classification_datamodel.h> >> >> #include <linux/if.h> >> >> @@ -40,6 +41,7 @@ struct pktio_entry { >> odp_pktio_type_t type; /**< pktio type */ >> pkt_sock_t pkt_sock; /**< using socket API for IO */ >> pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for IO */ >> + classifier_t cls; /**< classifier linked with this pktio*/ >> char name[IFNAMSIZ]; /**< name of pktio provided to >> pktio_open() */ >> }; >> @@ -49,6 +51,21 @@ typedef union { >> uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pktio_entry))]; >> } pktio_entry_t; >> >> +typedef struct { >> + pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; >> +} pktio_table_t; >> + >> +extern void *pktio_entry_ptr[]; > Is it possible to use pktio_tbl like it was oringially instead of > using this statically allocated array? Since pktio_tbl was being accessed by classifier module we had to drop static definition in pktio_tbl. Hence it was better to keep pktio_tbl as static and create an extern variable with pointers so that the same could be accessed by other modules. > >> + >> + >> +static inline pktio_entry_t *get_pktio_entry(odp_pktio_t id) >> +{ >> + if (odp_unlikely(id == ODP_PKTIO_INVALID || >> + id > ODP_CONFIG_PKTIO_ENTRIES)) >> + return NULL; >> + >> + return pktio_entry_ptr[id - 1]; >> +} >> #ifdef __cplusplus >> } >> #endif >> diff --git a/platform/linux-generic/odp_buffer_pool.c b/platform/linux-generic/odp_buffer_pool.c >> index 6a0a6b2..b5bd613 100644 >> --- a/platform/linux-generic/odp_buffer_pool.c >> +++ b/platform/linux-generic/odp_buffer_pool.c >> @@ -56,12 +56,6 @@ typedef struct { >> } odp_any_buffer_hdr_t; >> >> >> -typedef union pool_entry_u { >> - struct pool_entry_s s; >> - >> - uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; >> - >> -} pool_entry_t; >> >> >> typedef struct pool_table_t { >> @@ -86,10 +80,6 @@ static inline odp_buffer_pool_t pool_index_to_handle(uint32_t pool_id) >> } >> >> >> -static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) >> -{ >> - return pool_hdl -1; >> -} >> >> >> static inline void set_handle(odp_buffer_hdr_t *hdr, >> diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c >> index 190d71e..ecf97a0 100644 >> --- a/platform/linux-generic/odp_classification.c >> +++ b/platform/linux-generic/odp_classification.c >> @@ -2,68 +2,289 @@ >> #include <odp_align.h> >> #include <odp_queue.h> >> #include <odp_debug.h> >> +#include <odp_internal.h> >> #include <odp_debug_internal.h> >> +#include <odp_packet_internal.h> >> #include <odp_packet_io.h> >> +#include <odp_packet_io_internal.h> >> +#include <odp_classification_datamodel.h> >> +#include <odp_classification_inlines.h> >> +#include <odp_classification_internal.h> >> +#include <odp_buffer_pool_internal.h> >> +#include <odp_shared_memory.h> >> +#include <odph_eth.h> >> +#include <string.h> >> +#include <odp_rwlock.h> >> >> -odp_cos_t odp_cos_create(const char *name) >> +#define WRITE_LOCK(a) odp_rwlock_write_lock(a) >> +#define WRITE_UNLOCK(a) odp_rwlock_write_unlock(a) >> +#define LOCK_INIT(a) odp_rwlock_init(a) >> + >> +#define READ_LOCK(a) odp_rwlock_write_lock(a) >> +#define READ_UNLOCK(a) odp_rwlock_write_unlock(a) >> + >> +static cos_tbl_t *cos_tbl; >> +static pmr_set_tbl_t *pmr_set_tbl; >> +static pmr_tbl_t *pmr_tbl; >> + >> +cos_t *get_cos_entry_internal(odp_cos_t cos_id) >> +{ >> + return &(cos_tbl->cos_entry[cos_id - 1]); >> +} >> + >> +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id) >> +{ >> + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); >> +} >> + >> +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id) >> +{ >> + return &(pmr_tbl->pmr[pmr_id - 1]); >> +} >> + >> +int odp_classification_init_global(void) >> { >> - (void) name; >> - ODP_UNIMPLEMENTED(); >> + odp_shm_t cos_shm; >> + odp_shm_t pmr_shm; >> + odp_shm_t pmr_set_shm; >> + int i; >> + >> + cos_shm = odp_shm_reserve("odp_cos_pools", > I would suggest naming these to shm_odp_xxx_table, pools would > indicate you allocate some pools from the shm. Agreed. Will update in next version. > >> + sizeof(cos_tbl_t), >> + sizeof(cos_t), 0); >> + >> + cos_tbl = odp_shm_addr(cos_shm); >> + if (cos_tbl == NULL) { >> + odp_shm_free(cos_shm); >> + return -1; >> + } >> + >> + memset(cos_tbl, 0, sizeof(cos_tbl_t)); >> + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { >> + /* init locks */ >> + cos_t *cos = get_cos_entry_internal(i + 1); >> + LOCK_INIT(&cos->s.lock); >> + } >> + >> + pmr_shm = odp_shm_reserve("odp_pmr_pools", >> + sizeof(pmr_tbl_t), >> + sizeof(pmr_t), 0); >> + pmr_tbl = odp_shm_addr(pmr_shm); >> + if (pmr_tbl == NULL) { >> + odp_shm_free(pmr_shm); >> + return -1; >> + } >> + >> + memset(pmr_tbl, 0, sizeof(pmr_tbl_t)); >> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { >> + /* init locks */ >> + pmr_t *pmr = get_pmr_entry_internal(i + 1); >> + LOCK_INIT(&pmr->s.lock); >> + } >> + >> + pmr_set_shm = odp_shm_reserve("odp_pmr_set_pools", >> + sizeof(pmr_set_tbl_t), >> + sizeof(pmr_set_t), 0); >> + pmr_set_tbl = odp_shm_addr(pmr_set_shm); >> + if (pmr_set_tbl == NULL) { >> + odp_shm_free(pmr_set_shm); >> + return -1; >> + } >> + >> + memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); >> + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { >> + /* init locks */ >> + pmr_set_t *pmr = get_pmr_set_entry_internal(i + 1); >> + LOCK_INIT(&pmr->s.pmr.s.lock); >> + } >> + >> return 0; >> } >> >> +odp_cos_t odp_cos_create(const char *name) >> +{ >> + int i; >> + >> + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { >> + WRITE_LOCK(&cos_tbl->cos_entry[i].s.lock); >> + if (0 == cos_tbl->cos_entry[i].s.valid) { >> + strncpy(cos_tbl->cos_entry[i].s.name, name, >> + ODP_COS_NAME_LEN - 1); >> + cos_tbl->cos_entry[i].s.name[ODP_COS_NAME_LEN - 1] = 0; >> + cos_tbl->cos_entry[i].s.pmr = NULL; >> + cos_tbl->cos_entry[i].s.queue = NULL; > cost_tbl->cos_entry[i].s.queue_group is uninitialized. It's never used > but I still think it should be properly reset here to make future work > easier. The drop_policy also, if we have a default value for it. As said earlier, queue_group and drop policy are deferred from 1.0 But will handle the initialization of the same here. > >> + cos_tbl->cos_entry[i].s.pool = NULL; >> + cos_tbl->cos_entry[i].s.flow_set = 0; >> + cos_tbl->cos_entry[i].s.headroom = 0; >> + cos_tbl->cos_entry[i].s.valid = 1; >> + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); >> + return i + 1; >> + } >> + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); >> + } >> + return ODP_COS_INVALID; >> +} >> + >> +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) >> +{ >> + int i; >> + >> + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { >> + WRITE_LOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >> + if (0 == pmr_set_tbl->pmr_set[i].s.pmr.s.valid) { >> + pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; >> + pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; >> + pmr_set_tbl->pmr_set[i].s.pmr.s.cos = NULL; >> + *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; >> + odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] >> + .s.pmr.s.count, 0); >> + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >> + return i + 1; >> + } >> + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >> + } >> + return ODP_PMR_INVAL; >> +} >> + >> +odp_pmr_t alloc_pmr(pmr_t **pmr) >> +{ >> + int i; >> + >> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { >> + WRITE_LOCK(&pmr_tbl->pmr[i].s.lock); >> + if (0 == pmr_tbl->pmr[i].s.valid) { >> + pmr_tbl->pmr[i].s.valid = 1; >> + pmr_tbl->pmr[i].s.cos = NULL; >> + odp_atomic_init_u32(&pmr_tbl->pmr[i].s.count, 0); >> + pmr_tbl->pmr[i].s.num_pmr = 0; >> + *pmr = &pmr_tbl->pmr[i]; >> + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); >> + return i + 1; >> + } >> + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); >> + } >> + return ODP_PMR_INVAL; >> +} >> + >> + >> +cos_t *get_cos_entry(odp_cos_t cos_id) >> +{ >> + if (cos_id > ODP_COS_MAX_ENTRY || cos_id == ODP_COS_INVALID) >> + return NULL; >> + if (cos_tbl->cos_entry[cos_id - 1].s.valid == 0) > In other parts of the file (e.g. odp_cos_destroy) access to 'valid' is > done under lock. Don't we need it here too? If so, you might consider > implementing a get_locked type of routine to keep subsequent access to > fields in the entry in the same critical section. > >> + return NULL; >> + return &(cos_tbl->cos_entry[cos_id - 1]); >> +} >> + >> + >> +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) >> +{ >> + if (pmr_set_id > ODP_PMRSET_MAX_ENTRY || pmr_set_id == ODP_PMR_INVAL) >> + return NULL; >> + if (pmr_set_tbl->pmr_set[pmr_set_id - 1].s.pmr.s.valid == 0) >> + return NULL; >> + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); >> +} >> + >> +pmr_t *get_pmr_entry(odp_pmr_t pmr_id) >> +{ >> + if (pmr_id > ODP_PMR_MAX_ENTRY || pmr_id == ODP_PMR_INVAL) >> + return NULL; >> + if (pmr_tbl->pmr[pmr_id - 1].s.valid == 0) >> + return NULL; >> + return &(pmr_tbl->pmr[pmr_id - 1]); >> +} >> + >> int odp_cos_destroy(odp_cos_t cos_id) >> { >> - (void)cos_id; >> - ODP_UNIMPLEMENTED(); >> + cos_t *cos = get_cos_entry(cos_id); >> + if (NULL == cos) >> + return -1; >> + >> + WRITE_LOCK(&cos->s.lock); >> + cos->s.valid = 0; >> + WRITE_UNLOCK(&cos->s.lock); >> return 0; >> } >> >> int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) >> { >> - (void)cos_id; >> - (void)queue_id; >> - ODP_UNIMPLEMENTED(); >> + cos_t *cos = get_cos_entry(cos_id); >> + if (cos == NULL) >> + return -1; >> + >> + WRITE_LOCK(&cos->s.lock); >> + cos->s.queue = queue_to_qentry(queue_id); >> + WRITE_UNLOCK(&cos->s.lock); >> return 0; >> } >> >> int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) >> { >> - (void)cos_id; >> - (void)drop_policy; >> - ODP_UNIMPLEMENTED(); >> + cos_t *cos = get_cos_entry(cos_id); >> + if (cos == NULL) >> + return -1; >> + >> + WRITE_LOCK(&cos->s.lock); >> + cos->s.drop_policy = drop_policy; > The drop_policy is never used, I assume that's part of a future work? Yes. > >> + WRITE_UNLOCK(&cos->s.lock); >> return 0; >> } >> >> int odp_pktio_set_default_cos(odp_pktio_t pktio_in, odp_cos_t default_cos) >> { >> - (void)pktio_in; >> - (void)default_cos; >> - ODP_UNIMPLEMENTED(); >> + pktio_entry_t *entry; >> + cos_t *cos; >> + entry = get_pktio_entry(pktio_in); >> + if (entry == NULL) >> + return -1; >> + cos = get_cos_entry(default_cos); >> + if (cos == NULL) >> + return -1; >> + >> + WRITE_LOCK(&entry->s.cls.lock); >> + entry->s.cls.default_cos = cos; >> + WRITE_UNLOCK(&entry->s.cls.lock); >> + >> return 0; >> } >> >> int odp_pktio_set_error_cos(odp_pktio_t pktio_in, odp_cos_t error_cos) >> { >> - (void)pktio_in; >> - (void)error_cos; >> - ODP_UNIMPLEMENTED(); >> + pktio_entry_t *entry; >> + cos_t *cos; >> + >> + entry = get_pktio_entry(pktio_in); >> + if (entry == NULL) >> + return -1; >> + cos = get_cos_entry(error_cos); >> + if (cos == NULL) >> + return -1; >> + WRITE_LOCK(&entry->s.cls.lock); >> + entry->s.cls.error_cos = cos; >> + WRITE_UNLOCK(&entry->s.cls.lock); >> return 0; >> } >> >> int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset) >> { >> - (void)pktio_in; >> - (void)offset; >> - ODP_UNIMPLEMENTED(); >> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >> + if (entry == NULL) >> + return -1; >> + WRITE_LOCK(&entry->s.cls.lock); >> + entry->s.cls.skip = offset; >> + WRITE_UNLOCK(&entry->s.cls.lock); >> return 0; >> } >> >> -int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom) >> +int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom) >> { >> - (void)port_id; >> - (void)headroom; >> - ODP_UNIMPLEMENTED(); >> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >> + if (entry == NULL) >> + return -1; >> + WRITE_LOCK(&entry->s.cls.lock); >> + entry->s.cls.headroom = headroom; >> + WRITE_UNLOCK(&entry->s.cls.lock); >> return 0; >> } >> >> @@ -72,11 +293,26 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, >> uint8_t qos_table[], >> odp_cos_t cos_table[]) >> { >> - (void)pktio_in; >> - (void)num_qos; >> - (void)qos_table; >> - (void)cos_table; >> - ODP_UNIMPLEMENTED(); >> + pmr_l2_cos_t *l2_cos; >> + size_t i; >> + cos_t *cos; >> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >> + if (entry == NULL) >> + return -1; >> + READ_LOCK(&entry->s.cls.lock); >> + l2_cos = &entry->s.cls.l2_cos_table; >> + READ_UNLOCK(&entry->s.cls.lock); >> + >> + WRITE_LOCK(&l2_cos->lock); >> + /* Update the L2 QoS table*/ >> + for (i = 0; i < num_qos; i++) { >> + cos = get_cos_entry(cos_table[i]); >> + if (cos != NULL) { >> + if (ODP_COS_MAX_L2_QOS > qos_table[i]) >> + l2_cos->cos[qos_table[i]] = cos; >> + } >> + } >> + WRITE_UNLOCK(&l2_cos->lock); >> return 0; >> } >> >> @@ -86,12 +322,28 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, >> odp_cos_t cos_table[], >> bool l3_preference) >> { >> - (void)pktio_in; >> - (void)num_qos; >> - (void)qos_table; >> - (void)cos_table; >> - (void)l3_preference; >> - ODP_UNIMPLEMENTED(); >> + pmr_l3_cos_t *l3_cos; >> + size_t i; >> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >> + cos_t *cos; >> + >> + if (entry == NULL) >> + return -1; >> + WRITE_LOCK(&entry->s.cls.lock); >> + entry->s.cls.l3_precedence = l3_preference; >> + l3_cos = &entry->s.cls.l3_cos_table; >> + WRITE_UNLOCK(&entry->s.cls.lock); >> + >> + WRITE_LOCK(&l3_cos->lock); >> + /* Update the L3 QoS table*/ >> + for (i = 0; i < num_qos; i++) { >> + cos = get_cos_entry(cos_table[i]); >> + if (cos != NULL) { >> + if (ODP_COS_MAX_L3_QOS > qos_table[i]) >> + l3_cos->cos[qos_table[i]] = cos; >> + } >> + } >> + WRITE_UNLOCK(&l3_cos->lock); >> return 0; >> } >> >> @@ -100,12 +352,23 @@ odp_pmr_t odp_pmr_create_match(odp_pmr_term_e term, >> const void *mask, >> size_t val_sz) >> { >> - (void)term; >> - (void)val; >> - (void)mask; >> - (void)val_sz; >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + pmr_t *pmr; >> + odp_pmr_t id; >> + >> + id = alloc_pmr(&pmr); >> + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) >> + return ODP_PMR_INVAL; >> + >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.num_pmr = 1; >> + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; >> + pmr->s.pmr_term_value[0].term = term; >> + pmr->s.pmr_term_value[0].mask.val = 0; >> + pmr->s.pmr_term_value[0].mask.mask = 0; >> + memcpy(&pmr->s.pmr_term_value[0].mask.val, val, val_sz); >> + memcpy(&pmr->s.pmr_term_value[0].mask.mask, mask, val_sz); >> + WRITE_UNLOCK(&pmr->s.lock); >> + return id; >> } >> >> odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, >> @@ -113,18 +376,33 @@ odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, >> const void *val2, >> size_t val_sz) >> { >> - (void)term; >> - (void)val1; >> - (void)val2; >> - (void)val_sz; >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + pmr_t *pmr; >> + odp_pmr_t id; >> + >> + id = alloc_pmr(&pmr); >> + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) >> + return ODP_PMR_INVAL; >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.num_pmr = 1; >> + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; >> + pmr->s.pmr_term_value[0].term = term; >> + pmr->s.pmr_term_value[0].range.val1 = 0; >> + pmr->s.pmr_term_value[0].range.val2 = 0; >> + memcpy(&pmr->s.pmr_term_value[0].range.val1, val1, val_sz); >> + memcpy(&pmr->s.pmr_term_value[0].range.val2, val2, val_sz); >> + WRITE_UNLOCK(&pmr->s.lock); >> + return id; >> } >> >> int odp_pmr_destroy(odp_pmr_t pmr_id) >> { >> - (void)pmr_id; >> - ODP_UNIMPLEMENTED(); >> + pmr_t *pmr = get_pmr_entry(pmr_id); >> + >> + if (pmr == NULL) >> + return -1; >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.valid = 0; >> + WRITE_UNLOCK(&pmr->s.lock); >> return 0; >> } >> >> @@ -132,64 +410,465 @@ int odp_pktio_pmr_cos(odp_pmr_t pmr_id, >> odp_pktio_t src_pktio, >> odp_cos_t dst_cos) >> { >> - (void)pmr_id; >> - (void)src_pktio; >> - (void)dst_cos; >> - ODP_UNIMPLEMENTED(); >> + uint8_t num_pmr; >> + pktio_entry_t *pktio_entry; >> + pmr_t *pmr; >> + cos_t *cos; >> + >> + pktio_entry = get_pktio_entry(src_pktio); >> + if (pktio_entry == NULL) >> + return -1; >> + >> + pmr = get_pmr_entry(pmr_id); >> + if (pmr == NULL) >> + return -1; >> + >> + cos = get_cos_entry(dst_cos); >> + if (cos == NULL) >> + return -1; >> + >> + num_pmr = pktio_entry->s.cls.num_pmr; >> + if (num_pmr >= ODP_PKTIO_MAX_PMR) > num_pmr > ODP_PKTIO_MAX_PMR is a serious error, I suggest using > ODP_ASSERT for such case This error need not result in complete failure of the system. IMO, indicating the application that they have reached maximum PMR number should be sufficient. > >> + return -1; >> + >> + WRITE_LOCK(&pktio_entry->s.cls.lock); >> + pktio_entry->s.cls.pmr[num_pmr] = pmr; >> + pktio_entry->s.cls.num_pmr++; >> + WRITE_UNLOCK(&pktio_entry->s.cls.lock); >> + >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.cos = cos; >> + WRITE_UNLOCK(&pmr->s.lock); >> return 0; >> } >> >> int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t dst_cos) >> { >> - (void)pmr_id; >> - (void)src_cos; >> - (void)dst_cos; >> - ODP_UNIMPLEMENTED(); >> + cos_t *cos_src = get_cos_entry(src_cos); >> + cos_t *cos_dst = get_cos_entry(dst_cos); >> + pmr_t *pmr = get_pmr_entry(pmr_id); >> + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) >> + return -1; >> + >> + WRITE_LOCK(&cos_src->s.lock); >> + cos_src->s.pmr = pmr; >> + WRITE_UNLOCK(&cos_src->s.lock); >> + >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.cos = cos_dst; >> + WRITE_UNLOCK(&pmr->s.lock); >> return 0; >> } >> >> signed long odp_pmr_match_count(odp_pmr_t pmr_id) >> { >> - (void)pmr_id; >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + pmr_t *pmr = get_pmr_entry(pmr_id); >> + if (pmr == NULL) >> + return -1; >> + return (signed long)odp_atomic_load_u32(&pmr->s.count); >> } >> >> unsigned long long odp_pmr_terms_cap(void) >> { >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + unsigned long long term_cap = 0; >> + >> + term_cap |= (1 << ODP_PMR_LEN); >> + term_cap |= (1 << ODP_PMR_IPPROTO); >> + term_cap |= (1 << ODP_PMR_UDP_DPORT); >> + term_cap |= (1 << ODP_PMR_TCP_DPORT); >> + term_cap |= (1 << ODP_PMR_UDP_SPORT); >> + term_cap |= (1 << ODP_PMR_TCP_SPORT); >> + term_cap |= (1 << ODP_PMR_SIP_ADDR); >> + term_cap |= (1 << ODP_PMR_DIP_ADDR); >> + return term_cap; >> } >> >> unsigned odp_pmr_terms_avail(void) >> { >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + unsigned count = 0; >> + int i; >> + >> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) >> + if (!pmr_tbl->pmr[i].s.valid) >> + count++; >> + return count; >> } >> >> int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, >> odp_pmr_set_t *pmr_set_id) >> { >> - (void)num_terms; >> - (void)terms; >> - (void)pmr_set_id; >> - ODP_UNIMPLEMENTED(); >> - return 0; >> + pmr_t *pmr; >> + int i; >> + uint32_t id; >> + int val_sz; >> + int count = 0; >> + > Check num_terms <= ODP_PMRTERM_MAX ? Agreed. This is a coding error. > >> + id = alloc_pmr_set(&pmr); >> + if (id == ODP_PMR_INVAL) { >> + *pmr_set_id = id; >> + return -1; >> + } >> + >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.num_pmr = num_terms; >> + for (i = 0; i < num_terms; i++) { >> + pmr->s.pmr_term_value[i].match_type = terms[i].match_type; >> + if (terms[i].match_type == ODP_PMR_MASK) { >> + val_sz = terms[i].mask.val_sz; >> + if (val_sz > ODP_PMR_TERM_BYTES_MAX) >> + continue; >> + pmr->s.pmr_term_value[i].term = terms[i].mask.term; >> + pmr->s.pmr_term_value[i].mask.val = 0; >> + pmr->s.pmr_term_value[i].mask.mask = 0; >> + memcpy(&pmr->s.pmr_term_value[i].mask.val, >> + terms[i].mask.val, val_sz); >> + memcpy(&pmr->s.pmr_term_value[i].mask.mask, >> + terms[i].mask.mask, val_sz); >> + } else { >> + val_sz = terms[i].range.val_sz; >> + if (val_sz > ODP_PMR_TERM_BYTES_MAX) >> + continue; >> + pmr->s.pmr_term_value[i].term = terms[i].range.term; >> + pmr->s.pmr_term_value[i].range.val1 = 0; >> + pmr->s.pmr_term_value[i].range.val2 = 0; >> + memcpy(&pmr->s.pmr_term_value[i].range.val1, >> + terms[i].range.val1, val_sz); >> + memcpy(&pmr->s.pmr_term_value[i].range.val2, >> + terms[i].range.val2, val_sz); >> + } >> + count++; >> + } >> + WRITE_UNLOCK(&pmr->s.lock); >> + *pmr_set_id = id; >> + return count; >> } >> >> int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) >> { >> - (void)pmr_set_id; >> - ODP_UNIMPLEMENTED(); >> + pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); >> + if (pmr_set == NULL) >> + return -1; >> + WRITE_LOCK(&pmr_set->s.pmr.s.lock); >> + pmr_set->s.pmr.s.valid = 0; >> + WRITE_UNLOCK(&pmr_set->s.pmr.s.lock); >> return 0; >> } >> >> int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t src_pktio, >> - odp_cos_t dst_cos) >> + odp_cos_t dst_cos) >> +{ >> + uint8_t num_pmr; >> + pktio_entry_t *pktio_entry; >> + pmr_t *pmr; >> + cos_t *cos; >> + >> + pktio_entry = get_pktio_entry(src_pktio); >> + if (pktio_entry == NULL) >> + return -1; >> + >> + pmr = (pmr_t *)get_pmr_set_entry(pmr_set_id); >> + if (pmr == NULL) >> + return -1; >> + >> + cos = get_cos_entry(dst_cos); >> + if (cos == NULL) >> + return -1; >> + >> + num_pmr = pktio_entry->s.cls.num_pmr; >> + if (num_pmr >= ODP_PKTIO_MAX_PMR) >> + return -1; >> + >> + WRITE_LOCK(&pktio_entry->s.cls.lock); >> + pktio_entry->s.cls.pmr[num_pmr] = pmr; >> + pktio_entry->s.cls.num_pmr++; >> + WRITE_UNLOCK(&pktio_entry->s.cls.lock); >> + >> + WRITE_LOCK(&pmr->s.lock); >> + pmr->s.cos = cos; >> + WRITE_UNLOCK(&pmr->s.lock); >> + return 0; >> +} >> + >> +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) >> +{ >> + int pmr_failure = 0; >> + int num_pmr; >> + int i; >> + pmr_term_value_t *term_value; >> + >> + READ_LOCK(&pmr->s.lock); >> + if (!pmr->s.valid) { >> + READ_UNLOCK(&pmr->s.lock); >> + return 0; >> + } >> + num_pmr = pmr->s.num_pmr; >> + >> + /* Iterate through list of PMR Term values in a pmr_t */ >> + for (i = 0; i < num_pmr; i++) { >> + term_value = &pmr->s.pmr_term_value[i]; >> + switch (term_value->term) { >> + case ODP_PMR_LEN: >> + if (!verify_pmr_packet_len(pkt_hdr, term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_ETHTYPE_0: >> + if (!verify_pmr_eth_type_0(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_ETHTYPE_X: >> + if (!verify_pmr_eth_type_x(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_VLAN_ID_0: >> + if (!verify_pmr_vlan_id_0(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_VLAN_ID_X: >> + if (!verify_pmr_vlan_id_x(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_DMAC: >> + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_IPPROTO: >> + if (!verify_pmr_ip_proto(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_UDP_DPORT: >> + if (!verify_pmr_udp_dport(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_TCP_DPORT: >> + if (!verify_pmr_tcp_dport(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_UDP_SPORT: >> + if (!verify_pmr_udp_sport(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_TCP_SPORT: >> + if (!verify_pmr_tcp_sport(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_SIP_ADDR: >> + if (!verify_pmr_ipv4_saddr(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_DIP_ADDR: >> + if (!verify_pmr_ipv4_daddr(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_SIP6_ADDR: >> + if (!verify_pmr_ipv6_saddr(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_DIP6_ADDR: >> + if (!verify_pmr_ipv6_daddr(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_IPSEC_SPI: >> + if (!verify_pmr_ipsec_spi(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_LD_VNI: >> + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, >> + term_value)) >> + pmr_failure = 1; >> + break; >> + case ODP_PMR_INNER_HDR_OFF: >> + break; >> + } > nit: the closing brace (and the block below) should be on tab further > to match beginning of switch statement Agreed. > >> + if (pmr_failure) { >> + READ_UNLOCK(&pmr->s.lock); >> + return false; >> + } >> + } >> + READ_UNLOCK(&pmr->s.lock); >> + odp_atomic_inc_u32(&pmr->s.count); >> + return true; >> +} >> + >> +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, >> + odp_packet_hdr_t *hdr) >> +{ >> + cos_t *retcos = NULL; >> + >> + if (cos == NULL || pmr == NULL) >> + return NULL; >> + >> + READ_LOCK(&cos->s.lock); >> + if (!cos->s.valid) { >> + READ_UNLOCK(&cos->s.lock); >> + return NULL; >> + } >> + READ_UNLOCK(&cos->s.lock); >> + >> + if (verify_pmr(pmr, pkt_addr, hdr)) { >> + /** This gets called recursively to check all the PMRs in >> + * a PMR chain */ >> + retcos = match_pmr_cos(pmr->s.cos, pkt_addr, cos->s.pmr, hdr); >> + if (!retcos) >> + return cos; >> + } >> + return retcos; >> +} >> + >> +int pktio_classifier_init(pktio_entry_t *entry) >> { >> - (void)pmr_set_id; >> - (void)src_pktio; >> - (void)dst_cos; >> - ODP_UNIMPLEMENTED(); >> + classifier_t *cls; >> + if (entry == NULL) >> + return -1; >> + cls = &entry->s.cls; >> + cls->num_pmr = 0; >> + cls->flow_set = 0; >> + cls->error_cos = NULL; >> + cls->default_cos = NULL; >> + cls->headroom = 0; >> + cls->skip = 0; >> return 0; >> } >> + >> +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt) >> +{ >> + pktio_entry_t *entry; >> + queue_entry_t *queue; >> + cos_t *cos; >> + odp_packet_hdr_t *pkt_hdr; >> + uint8_t *pkt_addr; >> + >> + entry = get_pktio_entry(pktio); >> + if (entry == NULL) >> + return -1; >> + >> + pkt_hdr = odp_packet_hdr(pkt); >> + pkt_addr = odp_packet_addr(pkt); >> + >> + /* Matching PMR and selecting the CoS for the packet*/ >> + cos = pktio_select_cos(entry, pkt_addr, pkt_hdr); >> + if (cos == NULL) >> + return -1; >> + >> + /* Enqueuing the Packet based on the CoS */ >> + READ_LOCK(&cos->s.lock); >> + queue = cos->s.queue; >> + READ_UNLOCK(&cos->s.lock); >> + return queue_enq(queue, odp_buf_to_hdr((odp_buffer_t)pkt)); >> +} >> + >> +cos_t *pktio_select_cos(pktio_entry_t *entry, uint8_t *pkt_addr, >> + odp_packet_hdr_t *pkt_hdr) >> +{ >> + pmr_t *pmr; >> + cos_t *cos = NULL; >> + int i; >> + classifier_t *cls = &entry->s.cls; >> + >> + /* Return error cos for error packet */ >> + if (pkt_hdr->error_flags.all) >> + return cls->error_cos; >> + /* Calls all the PMRs attached at the PKTIO level*/ >> + for (i = 0; i < cls->num_pmr; i++) { >> + pmr = entry->s.cls.pmr[i]; >> + if (pmr) { >> + cos = match_pmr_cos(pmr->s.cos, pkt_addr, pmr, pkt_hdr); >> + if (cos) >> + return cos; >> + } >> + } >> + >> + cos = match_qos_cos(entry, pkt_addr, pkt_hdr); >> + if (cos) >> + return cos; >> + >> + return cls->default_cos; >> +} >> + >> +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr) >> +{ >> + uint8_t dscp; >> + cos_t *cos = NULL; >> + odph_ipv4hdr_t *ipv4; >> + odph_ipv6hdr_t *ipv6; >> + >> + READ_LOCK(&l3_cos->lock); >> + if (hdr->input_flags.l3 && hdr->input_flags.ipv4) { >> + ipv4 = (odph_ipv4hdr_t *)(pkt_addr + hdr->l3_offset); >> + dscp = ODPH_IPV4HDR_DSCP(ipv4->tos); >> + cos = l3_cos->cos[dscp]; >> + } else if (hdr->input_flags.l3 && hdr->input_flags.ipv6) { >> + ipv6 = (odph_ipv6hdr_t *)(pkt_addr + hdr->l3_offset); >> + dscp = ODPH_IPV6HDR_DSCP(ipv6->ver_tc_flow); >> + cos = l3_cos->cos[dscp]; >> + } >> + READ_UNLOCK(&l3_cos->lock); >> + >> + return cos; >> +} >> + >> +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr) >> +{ >> + uint8_t qos; >> + cos_t *cos = NULL; >> + odph_ethhdr_t *eth; >> + odph_vlanhdr_t *vlan; >> + >> + if (hdr->input_flags.l2 && hdr->input_flags.vlan && >> + hdr->input_flags.eth) { >> + eth = (odph_ethhdr_t *)(pkt_addr + hdr->l2_offset); >> + vlan = (odph_vlanhdr_t *)(ð->type); >> + qos = ((vlan->tci >> 13) & 0xFF); >> + READ_LOCK(&l2_cos->lock); >> + cos = l2_cos->cos[qos]; >> + READ_UNLOCK(&l2_cos->lock); >> + } >> + return cos; >> +} >> + >> +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, >> + odp_packet_hdr_t *hdr) >> +{ >> + classifier_t *cls = &entry->s.cls; >> + pmr_l2_cos_t *l2_cos; >> + pmr_l3_cos_t *l3_cos; >> + cos_t *cos; >> + >> + READ_LOCK(&cls->lock); >> + l2_cos = &cls->l2_cos_table; >> + l3_cos = &cls->l3_cos_table; >> + READ_UNLOCK(&cls->lock); >> + >> + if (cls->l3_precedence) { >> + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); >> + if (cos) >> + return cos; >> + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); >> + if (cos) >> + return cos; >> + } else { >> + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); >> + if (cos) >> + return cos; >> + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); >> + if (cos) >> + return cos; >> + } >> + return NULL; >> +} >> diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c >> index 672b3d6..c661231 100644 >> --- a/platform/linux-generic/odp_init.c >> +++ b/platform/linux-generic/odp_init.c >> @@ -54,6 +54,10 @@ int odp_init_global(odp_init_t *params ODP_UNUSED, >> ODP_ERR("ODP crypto init failed.\n"); >> return -1; >> } >> + if (odp_classification_init_global()) { >> + ODP_ERR("ODP crypto init failed.\n"); >> + return -1; >> + } >> >> return 0; >> } >> diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c >> index 5c8146a..27f7b0a 100644 >> --- a/platform/linux-generic/odp_packet_io.c >> +++ b/platform/linux-generic/odp_packet_io.c >> @@ -11,32 +11,23 @@ >> #include <odp_packet_internal.h> >> #include <odp_internal.h> >> #include <odp_spinlock.h> >> +#include <odp_rwlock.h> >> #include <odp_shared_memory.h> >> #include <odp_packet_socket.h> >> #include <odp_hints.h> >> #include <odp_config.h> >> #include <odp_queue_internal.h> >> #include <odp_schedule_internal.h> >> +#include <odp_classification_internal.h> >> #include <odp_debug.h> >> >> #include <string.h> >> #include <sys/ioctl.h> >> >> -typedef struct { >> - pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; >> -} pktio_table_t; >> - >> static pktio_table_t *pktio_tbl; >> >> - >> -static pktio_entry_t *get_entry(odp_pktio_t id) >> -{ >> - if (odp_unlikely(id == ODP_PKTIO_INVALID || >> - id > ODP_CONFIG_PKTIO_ENTRIES)) >> - return NULL; >> - >> - return &pktio_tbl->entries[id - 1]; >> -} >> +/* pktio pointer entries ( for inlines) */ >> +void *pktio_entry_ptr[ODP_CONFIG_PKTIO_ENTRIES]; >> >> int odp_pktio_init_global(void) >> { >> @@ -58,10 +49,12 @@ int odp_pktio_init_global(void) >> memset(pktio_tbl, 0, sizeof(pktio_table_t)); >> >> for (id = 1; id <= ODP_CONFIG_PKTIO_ENTRIES; ++id) { >> - pktio_entry = get_entry(id); >> + pktio_entry = &pktio_tbl->entries[id - 1]; >> >> odp_spinlock_init(&pktio_entry->s.lock); >> + odp_rwlock_init(&pktio_entry->s.cls.lock); >> >> + pktio_entry_ptr[id - 1] = pktio_entry; >> /* Create a default output queue for each pktio resource */ >> snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); >> name[ODP_QUEUE_NAME_LEN-1] = '\0'; >> @@ -108,12 +101,25 @@ static void unlock_entry(pktio_entry_t *entry) >> odp_spinlock_unlock(&entry->s.lock); >> } >> >> +static void lock_entry_classifier(pktio_entry_t *entry) >> +{ >> + odp_spinlock_lock(&entry->s.lock); >> + odp_rwlock_write_lock(&entry->s.cls.lock); >> +} >> + >> +static void unlock_entry_classifier(pktio_entry_t *entry) >> +{ >> + odp_rwlock_write_unlock(&entry->s.cls.lock); >> + odp_spinlock_unlock(&entry->s.lock); >> +} >> + >> static void init_pktio_entry(pktio_entry_t *entry) >> { >> set_taken(entry); >> entry->s.inq_default = ODP_QUEUE_INVALID; >> memset(&entry->s.pkt_sock, 0, sizeof(entry->s.pkt_sock)); >> memset(&entry->s.pkt_sock_mmap, 0, sizeof(entry->s.pkt_sock_mmap)); >> + pktio_classifier_init(entry); >> } >> >> static odp_pktio_t alloc_lock_pktio_entry(void) >> @@ -125,13 +131,13 @@ static odp_pktio_t alloc_lock_pktio_entry(void) >> for (i = 0; i < ODP_CONFIG_PKTIO_ENTRIES; ++i) { >> entry = &pktio_tbl->entries[i]; >> if (is_free(entry)) { >> - lock_entry(entry); >> + lock_entry_classifier(entry); >> if (is_free(entry)) { >> init_pktio_entry(entry); >> id = i + 1; >> return id; /* return with entry locked! */ >> } >> - unlock_entry(entry); >> + unlock_entry_classifier(entry); >> } >> } >> >> @@ -140,7 +146,7 @@ static odp_pktio_t alloc_lock_pktio_entry(void) >> >> static int free_pktio_entry(odp_pktio_t id) >> { >> - pktio_entry_t *entry = get_entry(id); >> + pktio_entry_t *entry = get_pktio_entry(id); >> >> if (entry == NULL) >> return -1; >> @@ -164,7 +170,7 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) >> } >> /* if successful, alloc_pktio_entry() returns with the entry locked */ >> >> - pktio_entry = get_entry(id); >> + pktio_entry = get_pktio_entry(id); >> if (!pktio_entry) >> return ODP_PKTIO_INVALID; >> >> @@ -200,14 +206,14 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) >> close_pkt_sock(&pktio_entry->s.pkt_sock); >> } >> >> - unlock_entry(pktio_entry); >> + unlock_entry_classifier(pktio_entry); >> free_pktio_entry(id); >> ODP_ERR("Unable to init any I/O type.\n"); >> return ODP_PKTIO_INVALID; >> >> done: >> strncpy(pktio_entry->s.name, dev, IFNAMSIZ); >> - unlock_entry(pktio_entry); >> + unlock_entry_classifier(pktio_entry); >> return id; >> } >> >> @@ -216,7 +222,7 @@ int odp_pktio_close(odp_pktio_t id) >> pktio_entry_t *entry; >> int res = -1; >> >> - entry = get_entry(id); >> + entry = get_pktio_entry(id); >> if (entry == NULL) >> return -1; >> >> @@ -255,7 +261,7 @@ odp_pktio_t odp_pktio_get_input(odp_packet_t pkt) >> >> int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) >> { >> - pktio_entry_t *pktio_entry = get_entry(id); >> + pktio_entry_t *pktio_entry = get_pktio_entry(id); >> int pkts; >> int i; >> >> @@ -293,7 +299,7 @@ int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) >> >> int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) >> { >> - pktio_entry_t *pktio_entry = get_entry(id); >> + pktio_entry_t *pktio_entry = get_pktio_entry(id); >> int pkts; >> >> if (pktio_entry == NULL) >> @@ -323,7 +329,7 @@ int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) >> >> int odp_pktio_inq_setdef(odp_pktio_t id, odp_queue_t queue) >> { >> - pktio_entry_t *pktio_entry = get_entry(id); >> + pktio_entry_t *pktio_entry = get_pktio_entry(id); >> queue_entry_t *qentry = queue_to_qentry(queue); >> >> if (pktio_entry == NULL || qentry == NULL) >> @@ -353,7 +359,7 @@ int odp_pktio_inq_remdef(odp_pktio_t id) >> >> odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) >> { >> - pktio_entry_t *pktio_entry = get_entry(id); >> + pktio_entry_t *pktio_entry = get_pktio_entry(id); >> >> if (pktio_entry == NULL) >> return ODP_QUEUE_INVALID; >> @@ -363,7 +369,7 @@ odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) >> >> odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id) >> { >> - pktio_entry_t *pktio_entry = get_entry(id); >> + pktio_entry_t *pktio_entry = get_pktio_entry(id); >> >> if (pktio_entry == NULL) >> return ODP_QUEUE_INVALID; >> @@ -423,7 +429,7 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) >> odp_buffer_t buf; >> odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; >> odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; >> - int pkts, i; >> + int pkts, i, j; >> >> buf_hdr = queue_deq(qentry); >> if (buf_hdr != NULL) >> @@ -433,12 +439,15 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) >> if (pkts <= 0) >> return NULL; >> >> - for (i = 0; i < pkts; ++i) { >> + for (i = 0, j = 0; i < pkts; ++i) { >> buf = odp_packet_to_buffer(pkt_tbl[i]); >> - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); >> + buf_hdr = odp_buf_to_hdr(buf); >> + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) >> + tmp_hdr_tbl[j++] = buf_hdr; >> } >> >> - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); >> + if (j) >> + queue_enq_multi(qentry, tmp_hdr_tbl, j); >> buf_hdr = tmp_hdr_tbl[0]; >> return buf_hdr; >> } >> @@ -454,8 +463,9 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) >> int nbr; >> odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; >> odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; >> + odp_buffer_hdr_t *tmp_hdr; >> odp_buffer_t buf; >> - int pkts, i; >> + int pkts, i, j; >> >> nbr = queue_deq_multi(qentry, buf_hdr, num); >> if (odp_unlikely(nbr > num)) >> @@ -472,12 +482,15 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) >> if (pkts <= 0) >> return nbr; >> >> - for (i = 0; i < pkts; ++i) { >> + for (i = 0, j = 0; i < pkts; ++i) { >> buf = odp_packet_to_buffer(pkt_tbl[i]); >> - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); >> + tmp_hdr = odp_buf_to_hdr(buf); >> + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) >> + tmp_hdr_tbl[j++] = tmp_hdr; >> } >> >> - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); >> + if (j) >> + queue_enq_multi(qentry, tmp_hdr_tbl, j); >> return nbr; >> } >> >> @@ -493,7 +506,7 @@ int odp_pktio_set_mtu(odp_pktio_t id, int mtu) >> return -1; >> } >> >> - entry = get_entry(id); >> + entry = get_pktio_entry(id); >> if (entry == NULL) { >> ODP_DBG("pktio entry %d does not exist\n", id); >> return -1; >> @@ -523,7 +536,7 @@ int odp_pktio_mtu(odp_pktio_t id) >> struct ifreq ifr; >> int ret; >> >> - entry = get_entry(id); >> + entry = get_pktio_entry(id); >> if (entry == NULL) { >> ODP_DBG("pktio entry %d does not exist\n", id); >> return -1; >> -- >> 2.0.1.472.g6f92e5f >> >> >> _______________________________________________ >> lng-odp mailing list >> lng-odp@lists.linaro.org >> http://lists.linaro.org/mailman/listinfo/lng-odp
On Wed, Dec 3, 2014 at 8:20 AM, Bala <bala.manoharan@linaro.org> wrote: > > On Tuesday 02 December 2014 11:13 PM, Ciprian Barbu wrote: >> >> Hi, >> >> I took some time to go through all the code. For the moment there is >> no support for selecting the pool based on the Class of Service, >> packets are allocated before hand. This is enough for this initial >> effort, of course. Also the drop policy is also not used, I suppose >> because of the first reason. The parsing is still performed with the >> odp_packet_parse function, the classification implementation relies on >> having the header field flags properly set. > > This current version does not support queue group, drop policy and pool > selection > based on CoS. The same will be taken up as enhancement after 1.0 >> >> I mainly have minor stuff, there is one thing about locking that looks >> like more work, unless I'm wrong about it. All in all I think it's a >> good starting point, we would really need some test / sample app to >> carefully test everything at this point and then we can continue with >> improving the classifier. > > I am currently working on Cunit and an example program to test > classification. >> >> >> See comments below. >> >> On Thu, Nov 27, 2014 at 6:32 PM, Balasubramanian Manoharan >> <bala.manoharan@linaro.org> wrote: >>> >>> This patch contains classification implementation for ODP v1.0. >>> >>> The salient features of this classification version are as follows: >>> * Attaches PMR, PMR_SET to a Pktio entry >>> * Attaches CoS values for L2 and L3 QoS to a Pktio entry >>> * Selects ClassOfService for a packet based on PMR, L2 QoS and L3 QoS >>> values >>> * Selects a default CoS if packet does not match any of the assigned >>> rules >>> * Selects an Error CoS for an Error packet >>> * Enqueues the packet to the queue associated with the selected CoS >>> >>> This patch also adds classifier object to pktio entry and moves static >>> inline >>> functions to header files in Buffer and PKTIO modules. >>> >>> Signed-off-by: Balasubramanian Manoharan <bala.manoharan@linaro.org> >>> --- >>> V3: Patch has been merged as independently compilable unit >>> helper/include/odph_ip.h | 6 + >>> platform/linux-generic/include/api/odp.h | 1 + >>> .../include/odp_buffer_pool_internal.h | 10 + >>> .../include/odp_classification_datamodel.h | 199 +++++ >>> .../include/odp_classification_inlines.h | 259 +++++++ >>> .../include/odp_classification_internal.h | 173 +++++ >>> platform/linux-generic/include/odp_internal.h | 2 + >>> .../linux-generic/include/odp_packet_io_internal.h | 17 + >>> platform/linux-generic/odp_buffer_pool.c | 10 - >>> platform/linux-generic/odp_classification.c | 831 >>> +++++++++++++++++++-- >>> platform/linux-generic/odp_init.c | 4 + >>> platform/linux-generic/odp_packet_io.c | 85 ++- >>> 12 files changed, 1475 insertions(+), 122 deletions(-) >>> create mode 100644 >>> platform/linux-generic/include/odp_classification_datamodel.h >>> create mode 100644 >>> platform/linux-generic/include/odp_classification_inlines.h >>> create mode 100644 >>> platform/linux-generic/include/odp_classification_internal.h >>> >>> diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h >>> index 2c83c0f..f78724e 100644 >>> --- a/helper/include/odph_ip.h >>> +++ b/helper/include/odph_ip.h >>> @@ -35,6 +35,9 @@ extern "C" { >>> /** @internal Returns IPv4 header length */ >>> #define ODPH_IPV4HDR_IHL(ver_ihl) ((ver_ihl) & 0x0f) >>> >>> +/** @internal Returns IPv4 DSCP */ >>> +#define ODPH_IPV4HDR_DSCP(tos) (((tos) & 0xfc) >> 2) >>> + >>> /** @internal Returns IPv4 Don't fragment */ >>> #define ODPH_IPV4HDR_FLAGS_DONT_FRAG(frag_offset) ((frag_offset) & >>> 0x4000) >>> >>> @@ -47,6 +50,9 @@ extern "C" { >>> /** @internal Returns true if IPv4 packet is a fragment */ >>> #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff) >>> >>> +/** @internal Returns IPv4 DSCP */ >>> +#define ODPH_IPV6HDR_DSCP(ver_tc_flow) (uint8_t)((((ver_tc_flow) & >>> 0x0fc00000) >> 22) & 0xff) >>> + >>> /** IPv4 header */ >>> typedef struct ODP_PACKED { >>> uint8_t ver_ihl; /**< Version / Header length */ >>> diff --git a/platform/linux-generic/include/api/odp.h >>> b/platform/linux-generic/include/api/odp.h >>> index 6e4f69e..b7b1ca9 100644 >>> --- a/platform/linux-generic/include/api/odp.h >>> +++ b/platform/linux-generic/include/api/odp.h >>> @@ -47,6 +47,7 @@ extern "C" { >>> #include <odp_packet_flags.h> >>> #include <odp_packet_io.h> >>> #include <odp_crypto.h> >>> +#include <odp_classification.h> >>> #include <odp_rwlock.h> >>> >>> #ifdef __cplusplus >>> diff --git a/platform/linux-generic/include/odp_buffer_pool_internal.h >>> b/platform/linux-generic/include/odp_buffer_pool_internal.h >>> index e0210bd..bdbefff 100644 >>> --- a/platform/linux-generic/include/odp_buffer_pool_internal.h >>> +++ b/platform/linux-generic/include/odp_buffer_pool_internal.h >>> @@ -64,6 +64,12 @@ struct pool_entry_s { >>> size_t hdr_size; >>> }; >>> >>> +typedef union pool_entry_u { >>> + struct pool_entry_s s; >>> + >>> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct >>> pool_entry_s))]; >>> + >>> +} pool_entry_t; >> >> nit: moving these structures to internal headers >> (odp_buffer_pool_internal.h and odp_packet_io_internal.h) would have >> probably been best done in a separate patch, like a prerequisite to >> implementing the classifier. > > The initial patches were designed to have these as a separate patch but then > it was decided that individual patches have to independently compilable. Yes and it could have been done. But I'm ok with how it is at this moment, we are still exercising our skills on breaking up patches. >> >> >>> extern void *pool_entry_ptr[]; >>> >>> @@ -73,6 +79,10 @@ static inline void *get_pool_entry(uint32_t pool_id) >>> return pool_entry_ptr[pool_id]; >>> } >>> >>> +static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) >>> +{ >>> + return pool_hdl - 1; >>> +} >>> >>> static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf) >>> { >>> diff --git >>> a/platform/linux-generic/include/odp_classification_datamodel.h >>> b/platform/linux-generic/include/odp_classification_datamodel.h >>> new file mode 100644 >>> index 0000000..f7c9fb5 >>> --- /dev/null >>> +++ b/platform/linux-generic/include/odp_classification_datamodel.h >>> @@ -0,0 +1,199 @@ >>> +/* Copyright (c) 2014, Linaro Limited >>> + * All rights reserved. >>> + * >>> + * SPDX-License-Identifier: BSD-3-Clause >>> + */ >>> + >>> + >>> +/** >>> + * @file >>> + * >>> + * ODP Classification Datamodel >>> + * Describes the classification internal data model >>> + */ >>> + >>> +#ifndef ODP_CLASSIFICATION_DATAMODEL_H_ >>> +#define ODP_CLASSIFICATION_DATAMODEL_H_ >>> + >>> +#ifdef __cplusplus >>> +extern "C" { >>> +#endif >>> + >>> +#include <odp_rwlock.h> >>> +#include <odp_classification.h> >>> +#include <odp_buffer_pool_internal.h> >>> +#include <odp_packet_internal.h> >>> +#include <odp_packet_io_internal.h> >>> +#include <odp_queue_internal.h> >>> + >>> +/* Maximum Class Of Service Entry */ >>> +#define ODP_COS_MAX_ENTRY 64 >>> +/* Maximum PMR Set Entry */ >>> +#define ODP_PMRSET_MAX_ENTRY 64 >>> +/* Maximum PMR Entry */ >>> +#define ODP_PMR_MAX_ENTRY 64 >>> +/* Maximum PMR Terms in a PMR Set */ >>> +#define ODP_PMRTERM_MAX 8 >>> +/* Maximum PMRs attached in PKTIO Level */ >>> +#define ODP_PKTIO_MAX_PMR 8 >>> +/* L2 Priority Bits */ >>> +#define ODP_COS_L2_QOS_BITS 3 >>> +/* Max L2 QoS value */ >>> +#define ODP_COS_MAX_L2_QOS (1 << ODP_COS_L2_QOS_BITS) >>> +/* L2 DSCP Bits */ >>> +#define ODP_COS_L3_QOS_BITS 6 >>> +/* Max L3 QoS Value */ >>> +#define ODP_COS_MAX_L3_QOS (1 << ODP_COS_L3_QOS_BITS) >>> +/* Max PMR Term bits */ >>> +#define ODP_PMR_TERM_BYTES_MAX 8 >>> + >>> +/* forward declaration */ >>> +typedef union pmr_u pmr_t; >>> + >>> +/** >>> +Packet Matching Rule Term Value >>> + >>> +Stores the Term and Value mapping for a PMR. >>> +The maximum size of value currently supported in 64 bits >>> +**/ >>> +typedef struct pmr_term_value { >>> + odp_pmr_match_type_e match_type; /**< Packet Match Type*/ >>> + odp_pmr_term_e term; /* PMR Term */ >> >> I would suggest keeping the same format with odp_pmr_match_t in >> odp_classification.h, with term inside the structure for mask and >> range. > > Since odp_pmr_term_e was common for both mask and range it was better > for programming per se to have it outside of mask and range. >>> >>> + union { >>> + struct { >>> + uint64_t val; >>> + uint64_t mask; >>> + } mask; /**< Match a masked set of bits */ >>> + struct { >>> + uint64_t val1; >>> + uint64_t val2; >>> + } range; /**< Match an integer range */ >>> + }; >>> +} pmr_term_value_t; >>> + >>> +/* >>> +Class Of Service >>> +*/ >>> +struct cos_s { >>> + odp_rwlock_t lock; /* cos rwlock */ >>> + queue_entry_t *queue; /* Associated Queue */ >>> + odp_queue_group_t queue_group; /* Associated Queue Group */ >>> + pool_entry_t *pool; /* Associated Buffer pool */ >>> + pmr_t *pmr; /* Associated PMR */ >>> + bool valid; /* validity Flag */ >>> + odp_drop_e drop_policy; /* Associated Drop Policy */ >>> + odp_cos_flow_set_t flow_set; /* Assigned Flow Set */ >>> + char name[ODP_COS_NAME_LEN]; /* name */ >>> + size_t headroom; /* Headroom for this CoS */ >>> +}; >>> + >>> +typedef union cos_u { >>> + struct cos_s s; >>> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct cos_s))]; >>> +} cos_t; >>> + >>> + >>> +/** >>> +Packet Matching Rule >>> + >>> +**/ >>> +struct pmr_s { >>> + odp_rwlock_t lock; /* pmr rwlock*/ >>> + cos_t *cos; /* Associated CoS */ >>> + odp_atomic_u32_t count; /* num of packets matching this >>> rule */ >>> + uint16_t num_pmr; /* num of PMR Term Values*/ >>> + bool valid; /* Validity Flag */ >>> + pmr_term_value_t pmr_term_value[1]; /* Associated PMR Term */ >>> +}; >>> + >>> +typedef union pmr_u { >>> + struct pmr_s s; >>> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_s))]; >>> +} pmr_t; >>> + >>> +/** >>> +Packet Matching Rule Set >>> + >>> +This structure is implemented as a extension over struct pmr_s >>> +In order to use same pointer to access both pmr_s and pmr_set_s >>> +'num_pmr' value is used to differentiate between pmr_s and pmr_set_s >>> struct >>> +**/ >>> +struct pmr_set_s { >>> + pmr_t pmr; >>> + pmr_term_value_t pmr_term_value[ODP_PMRTERM_MAX - 1]; >>> + /* List of associated PMR Terms */ >>> +}; >>> + >>> +typedef union pmr_set_u { >>> + struct pmr_set_s s; >>> + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct >>> pmr_set_s))]; >>> +} pmr_set_t; >>> + >>> +/** >>> +L2 QoS and CoS Map >>> + >>> +This structure holds the mapping between L2 QoS value and >>> +corresponding cos_t object >>> +**/ >>> +typedef struct pmr_l2_cos { >>> + odp_rwlock_t lock; /* pmr_l2_cos rwlock */ >>> + cos_t *cos[ODP_COS_MAX_L2_QOS]; /* Array of CoS objects */ >>> +} pmr_l2_cos_t; >>> + >>> +/** >>> +L3 QoS and CoS Map >>> + >>> +This structure holds the mapping between L3 QoS value and >>> +corresponding cos_t object >>> +**/ >>> +typedef struct pmr_l3_cos { >>> + odp_rwlock_t lock; /* pmr_l3_cos rwlock */ >>> + cos_t *cos[ODP_COS_MAX_L3_QOS]; /* Array of CoS objects */ >>> +} pmr_l3_cos_t; >>> + >>> +/** >>> +Linux Generic Classifier >>> + >>> +This structure is stored in pktio_entry and holds all >>> +the classifier configuration value. >>> +**/ >>> +typedef struct classifier { >>> + odp_rwlock_t lock; /*pktio_cos rwlock */ >>> + uint8_t num_pmr; /* num of PMRs linked to given >>> PKTIO*/ >>> + bool l3_precedence; /* L3 QoS precedence */ >>> + odp_cos_flow_set_t flow_set; /* Flow Set to be calculated >>> + for this pktio */ >>> + pmr_l2_cos_t l2_cos_table; /* L2 QoS-CoS table map */ >>> + pmr_l3_cos_t l3_cos_table; /* L3 Qos-CoS table map */ >>> + pmr_t *pmr[ODP_PKTIO_MAX_PMR]; /* PMRs linked with this PKTIO */ >>> + cos_t *error_cos; /* Associated Error CoS */ >>> + cos_t *default_cos; /* Associated Default CoS */ >>> + size_t headroom; /* Pktio Headroom */ >>> + size_t skip; /* Pktio Skip Offset */ >>> +} classifier_t; >>> + >>> +/** >>> +Class of Service Table >>> +**/ >>> +typedef struct odp_cos_table { >>> + cos_t cos_entry[ODP_COS_MAX_ENTRY]; >>> +} cos_tbl_t; >>> + >>> +/** >>> +PMR set table >>> +**/ >>> +typedef struct pmr_set_tbl { >>> + pmr_set_t pmr_set[ODP_PMRSET_MAX_ENTRY]; >>> +} pmr_set_tbl_t; >>> + >>> +/** >>> +PMR table >>> +**/ >>> +typedef struct pmr_tbl { >>> + pmr_t pmr[ODP_PMR_MAX_ENTRY]; >>> +} pmr_tbl_t; >>> + >>> +#ifdef __cplusplus >>> +} >>> +#endif >>> +#endif >>> diff --git a/platform/linux-generic/include/odp_classification_inlines.h >>> b/platform/linux-generic/include/odp_classification_inlines.h >>> new file mode 100644 >>> index 0000000..6b20119 >>> --- /dev/null >>> +++ b/platform/linux-generic/include/odp_classification_inlines.h >>> @@ -0,0 +1,259 @@ >>> +/* Copyright (c) 2014, Linaro Limited >>> + * All rights reserved. >>> + * >>> + * SPDX-License-Identifier: BSD-3-Clause >>> + */ >>> + >>> + >>> +/** >>> + * @file >>> + * >>> + * ODP Classification Inlines >>> + * Classification Inlines Functions >>> + */ >>> +#ifndef __ODP_CLASSIFICATION_INLINES_H_ >>> +#define __ODP_CLASSIFICATION_INLINES_H_ >>> + >>> +#ifdef __cplusplus >>> +extern "C" { >>> +#endif >>> + >>> +#include <odp_debug.h> >>> +#include <odph_eth.h> >>> +#include <odph_ip.h> >>> +#include <odph_udp.h> >>> +#include <odph_tcp.h> >>> + >>> +/* PMR term value verification function >>> +These functions verify the given PMR term value with the value in the >>> packet >>> +These following functions return 1 on success and 0 on failure >>> +*/ >>> + >>> +static inline int verify_pmr_packet_len(odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (pkt_hdr->frame_len & >>> + term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= pkt_hdr->frame_len) && >>> + (pkt_hdr->frame_len <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> +static inline int verify_pmr_ip_proto(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + odph_ipv4hdr_t *ip; >>> + uint8_t proto; >>> + if (!pkt_hdr->input_flags.ipv4) >>> + return 0; >>> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >>> + proto = ip->proto; >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (proto & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= proto) && >>> + (proto <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_ipv4_saddr(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + odph_ipv4hdr_t *ip; >>> + uint32_t ipaddr; >>> + if (!pkt_hdr->input_flags.ipv4) >>> + return 0; >>> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >>> + ipaddr = odp_be_to_cpu_32(ip->src_addr); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (ipaddr & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= ipaddr) && >>> + (ipaddr <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_ipv4_daddr(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + odph_ipv4hdr_t *ip; >>> + uint32_t ipaddr; >>> + if (!pkt_hdr->input_flags.ipv4) >>> + return 0; >>> + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); >>> + ipaddr = odp_be_to_cpu_32(ip->dst_addr); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (ipaddr & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= ipaddr) && >>> + (ipaddr <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_tcp_sport(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + uint16_t sport; >>> + odph_tcphdr_t *tcp; >>> + if (!pkt_hdr->input_flags.tcp) >>> + return 0; >>> + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >>> + sport = odp_be_to_cpu_16(tcp->src_port); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (sport & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= sport) && >>> + (sport <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_tcp_dport(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + uint16_t dport; >>> + odph_tcphdr_t *tcp; >>> + if (!pkt_hdr->input_flags.tcp) >>> + return 0; >>> + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >>> + dport = odp_be_to_cpu_16(tcp->dst_port); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (dport & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= dport) && >>> + (dport <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_udp_dport(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + uint16_t dport; >>> + odph_udphdr_t *udp; >>> + if (!pkt_hdr->input_flags.udp) >>> + return 0; >>> + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >>> + dport = odp_be_to_cpu_16(udp->dst_port); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (dport & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= dport) && >>> + (dport <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> +static inline int verify_pmr_udp_sport(uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr, >>> + pmr_term_value_t *term_value) >>> +{ >>> + uint16_t sport; >>> + odph_udphdr_t *udp; >>> + if (!pkt_hdr->input_flags.udp) >>> + return 0; >>> + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); >>> + sport = odp_be_to_cpu_16(udp->src_port); >>> + if (term_value->match_type == ODP_PMR_MASK) { >>> + if (term_value->mask.val == (sport & >>> term_value->mask.mask)) >>> + return 1; >>> + } else { >>> + if ((term_value->range.val1 <= sport) && >>> + (sport <= term_value->range.val2)) >>> + return 1; >>> + } >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_dmac(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> + >>> +static inline int verify_pmr_ipv6_saddr(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_ipv6_daddr(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_vlan_id_0(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_vlan_id_x(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_ipsec_spi(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_ld_vni(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_eth_type_0(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +static inline int verify_pmr_eth_type_x(uint8_t *pkt_addr ODP_UNUSED, >>> + odp_packet_hdr_t *pkt_hdr >>> ODP_UNUSED, >>> + pmr_term_value_t *term_value >>> ODP_UNUSED) >>> +{ >>> + ODP_UNIMPLEMENTED(); >>> + return 0; >>> +} >>> +#ifdef __cplusplus >>> +} >>> +#endif >>> +#endif >>> diff --git a/platform/linux-generic/include/odp_classification_internal.h >>> b/platform/linux-generic/include/odp_classification_internal.h >>> new file mode 100644 >>> index 0000000..fd2c6af >>> --- /dev/null >>> +++ b/platform/linux-generic/include/odp_classification_internal.h >>> @@ -0,0 +1,173 @@ >>> +/* Copyright (c) 2014, Linaro Limited >>> + * All rights reserved. >>> + * >>> + * SPDX-License-Identifier: BSD-3-Clause >>> + */ >>> + >>> + >>> +/** >>> + * @file >>> + * >>> + * ODP Classification Internal >>> + * Describes the classification internal Functions >>> + */ >>> + >>> +#ifndef __ODP_CLASSIFICATION_INTERNAL_H_ >>> +#define __ODP_CLASSIFICATION_INTERNAL_H_ >>> + >>> +#ifdef __cplusplus >>> +extern "C" { >>> +#endif >>> + >>> +#include <odp_classification.h> >>> +#include <odp_queue.h> >>> +#include <odp_packet_internal.h> >>> +#include <odp_packet_io.h> >>> +#include <odp_packet_io_internal.h> >>> +#include <odp_classification_datamodel.h> >>> + >>> +/** Classification Internal function **/ >>> + >>> +/** >>> +@internal >>> +Select a CoS for the given Packet based on pktio >>> + >>> +This function will call all the PMRs associated with a pktio for >>> +a given packet and will return the matched COS object. >>> +This function will check PMR, L2 and L3 QoS COS object associated >>> +with the PKTIO interface. >>> + >>> +Returns the default cos if the packet does not match any PMR >>> +Returns the error_cos if the packet has an error >>> +**/ >>> +cos_t *pktio_select_cos(pktio_entry_t *pktio, uint8_t *pkt_addr, >>> + odp_packet_hdr_t *pkt_hdr); >>> + >>> +/** >>> +@internal >>> +match_qos_cos >>> + >>> +Select a CoS for the given Packet based on QoS values >>> +This function returns the COS object matching the L2 and L3 QoS >>> +based on the l3_preference value of the pktio >>> +**/ >>> +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, >>> + odp_packet_hdr_t *hdr); >>> +/** >>> +Packet Classifier >>> + >>> +Start function for Packet Classifier >>> +This function calls Classifier module internal functions for a given >>> packet and >>> +enqueues the packet to specific Queue based on PMR and CoS selected. >>> +**/ >>> +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt); >>> +/** >>> +Packet IO classifier init >>> + >>> +This function does initialization of classifier object associated with >>> pktio. >>> +This function should be called during pktio initialization. >>> +**/ >>> +int pktio_classifier_init(pktio_entry_t *pktio); >>> + >>> +/** >>> +@internal >>> +match_pmr_cos >>> + >>> +Match a PMR chain with a Packet and return matching CoS >>> +This function gets called recursively to check the chained PMR Term >>> value >>> +with the packet. >>> + >>> +**/ >>> +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, >>> + odp_packet_hdr_t *hdr); >>> +/** >>> +@internal >>> +CoS associated with L3 QoS value >>> + >>> +This function returns the CoS associated with L3 QoS value >>> +**/ >>> +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, >>> + odp_packet_hdr_t *hdr); >>> + >>> +/** >>> +@internal >>> +CoS associated with L2 QoS value >>> + >>> +This function returns the CoS associated with L2 QoS value >>> +**/ >>> +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, >>> + odp_packet_hdr_t *hdr); >>> +/** >>> +@internal >>> +Flow Signature Calculation >>> + >>> +This function calculates the Flow Signature for a packet based on >>> +CoS and updates in Packet Meta Data >>> +**/ >>> +int update_flow_signature(uint8_t *pkt_addr, cos_t *cos); >>> + >>> +/** >>> +@internal >>> +Allocate a odp_pmr_set_t Handle >>> +*/ >>> +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr); >>> + >>> +/** >>> +@internal >>> +Allocate a odp_pmr_t Handle >>> +*/ >>> +odp_pmr_t alloc_pmr(pmr_t **pmr); >>> + >>> +/** >>> +@internal >>> +Pointer to pmr_set_t Handle >>> +This function checks for validity of pmr_set_t Handle >>> +*/ >>> +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id); >>> + >>> +/** >>> +@internal >>> +Pointer to pmr_set_t Handle >>> +*/ >>> +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id); >>> + >>> +/** >>> +@internal >>> +Pointer to pmr_set_t Handle >>> +This function checks for validity of pmr_set_t Handle >>> +*/ >>> +pmr_t *get_pmr_entry(odp_pmr_t pmr_id); >>> + >>> +/** >>> +@internal >>> +Pointer to pmr_set_t Handle >>> +*/ >>> +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id); >>> + >>> +/** >>> +@internal >>> +Pointer to odp_cos_t Handle >>> +*/ >>> +cos_t *get_cos_entry(odp_cos_t cos_id); >>> + >>> +/** >>> +@internal >>> +Pointer to odp_cos_t Handle >>> +This function checks for validity of odp_cos_t Handle >>> +*/ >>> +cos_t *get_cos_entry_internal(odp_cos_t cos_id); >>> + >>> +/** >>> +@internal >>> +Verify PMR with a Packet >>> + >>> +This function goes through each PMR_TERM value in pmr_t structure and >>> +calls verification function for each term.Returns 1 if PMR matches or 0 >>> +Otherwise. >>> +**/ >>> +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t >>> *pkt_hdr); >>> + >>> +#ifdef __cplusplus >>> +} >>> +#endif >>> +#endif >>> diff --git a/platform/linux-generic/include/odp_internal.h >>> b/platform/linux-generic/include/odp_internal.h >>> index f8c1596..04c1030 100644 >>> --- a/platform/linux-generic/include/odp_internal.h >>> +++ b/platform/linux-generic/include/odp_internal.h >>> @@ -32,6 +32,8 @@ int odp_buffer_pool_init_global(void); >>> int odp_pktio_init_global(void); >>> int odp_pktio_init_local(void); >>> >>> +int odp_classification_init_global(void); >>> + >>> int odp_queue_init_global(void); >>> >>> int odp_crypto_init_global(void); >>> diff --git a/platform/linux-generic/include/odp_packet_io_internal.h >>> b/platform/linux-generic/include/odp_packet_io_internal.h >>> index 0bc1e21..218e9d0 100644 >>> --- a/platform/linux-generic/include/odp_packet_io_internal.h >>> +++ b/platform/linux-generic/include/odp_packet_io_internal.h >>> @@ -20,6 +20,7 @@ extern "C" { >>> >>> #include <odp_spinlock.h> >>> #include <odp_packet_socket.h> >>> +#include <odp_classification_datamodel.h> >>> >>> #include <linux/if.h> >>> >>> @@ -40,6 +41,7 @@ struct pktio_entry { >>> odp_pktio_type_t type; /**< pktio type */ >>> pkt_sock_t pkt_sock; /**< using socket API for IO */ >>> pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for >>> IO */ >>> + classifier_t cls; /**< classifier linked with this >>> pktio*/ >>> char name[IFNAMSIZ]; /**< name of pktio provided to >>> pktio_open() */ >>> }; >>> @@ -49,6 +51,21 @@ typedef union { >>> uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct >>> pktio_entry))]; >>> } pktio_entry_t; >>> >>> +typedef struct { >>> + pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; >>> +} pktio_table_t; >>> + >>> +extern void *pktio_entry_ptr[]; >> >> Is it possible to use pktio_tbl like it was oringially instead of >> using this statically allocated array? > > Since pktio_tbl was being accessed by classifier module we had to drop > static definition in pktio_tbl. Hence it was better to keep pktio_tbl as > static and > create an extern variable with pointers so that the same could be accessed > by other > modules. Hm, odp_buffer_pool_internal.h has a similar thing, pool_entry_ptr. I suppose it's ok to have pktio_entry_ptr the same way. >> >> >>> + >>> + >>> +static inline pktio_entry_t *get_pktio_entry(odp_pktio_t id) >>> +{ >>> + if (odp_unlikely(id == ODP_PKTIO_INVALID || >>> + id > ODP_CONFIG_PKTIO_ENTRIES)) >>> + return NULL; >>> + >>> + return pktio_entry_ptr[id - 1]; >>> +} >>> #ifdef __cplusplus >>> } >>> #endif >>> diff --git a/platform/linux-generic/odp_buffer_pool.c >>> b/platform/linux-generic/odp_buffer_pool.c >>> index 6a0a6b2..b5bd613 100644 >>> --- a/platform/linux-generic/odp_buffer_pool.c >>> +++ b/platform/linux-generic/odp_buffer_pool.c >>> @@ -56,12 +56,6 @@ typedef struct { >>> } odp_any_buffer_hdr_t; >>> >>> >>> -typedef union pool_entry_u { >>> - struct pool_entry_s s; >>> - >>> - uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct >>> pool_entry_s))]; >>> - >>> -} pool_entry_t; >>> >>> >>> typedef struct pool_table_t { >>> @@ -86,10 +80,6 @@ static inline odp_buffer_pool_t >>> pool_index_to_handle(uint32_t pool_id) >>> } >>> >>> >>> -static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) >>> -{ >>> - return pool_hdl -1; >>> -} >>> >>> >>> static inline void set_handle(odp_buffer_hdr_t *hdr, >>> diff --git a/platform/linux-generic/odp_classification.c >>> b/platform/linux-generic/odp_classification.c >>> index 190d71e..ecf97a0 100644 >>> --- a/platform/linux-generic/odp_classification.c >>> +++ b/platform/linux-generic/odp_classification.c >>> @@ -2,68 +2,289 @@ >>> #include <odp_align.h> >>> #include <odp_queue.h> >>> #include <odp_debug.h> >>> +#include <odp_internal.h> >>> #include <odp_debug_internal.h> >>> +#include <odp_packet_internal.h> >>> #include <odp_packet_io.h> >>> +#include <odp_packet_io_internal.h> >>> +#include <odp_classification_datamodel.h> >>> +#include <odp_classification_inlines.h> >>> +#include <odp_classification_internal.h> >>> +#include <odp_buffer_pool_internal.h> >>> +#include <odp_shared_memory.h> >>> +#include <odph_eth.h> >>> +#include <string.h> >>> +#include <odp_rwlock.h> >>> >>> -odp_cos_t odp_cos_create(const char *name) >>> +#define WRITE_LOCK(a) odp_rwlock_write_lock(a) >>> +#define WRITE_UNLOCK(a) odp_rwlock_write_unlock(a) >>> +#define LOCK_INIT(a) odp_rwlock_init(a) >>> + >>> +#define READ_LOCK(a) odp_rwlock_write_lock(a) >>> +#define READ_UNLOCK(a) odp_rwlock_write_unlock(a) >>> + >>> +static cos_tbl_t *cos_tbl; >>> +static pmr_set_tbl_t *pmr_set_tbl; >>> +static pmr_tbl_t *pmr_tbl; >>> + >>> +cos_t *get_cos_entry_internal(odp_cos_t cos_id) >>> +{ >>> + return &(cos_tbl->cos_entry[cos_id - 1]); >>> +} >>> + >>> +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id) >>> +{ >>> + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); >>> +} >>> + >>> +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id) >>> +{ >>> + return &(pmr_tbl->pmr[pmr_id - 1]); >>> +} >>> + >>> +int odp_classification_init_global(void) >>> { >>> - (void) name; >>> - ODP_UNIMPLEMENTED(); >>> + odp_shm_t cos_shm; >>> + odp_shm_t pmr_shm; >>> + odp_shm_t pmr_set_shm; >>> + int i; >>> + >>> + cos_shm = odp_shm_reserve("odp_cos_pools", >> >> I would suggest naming these to shm_odp_xxx_table, pools would >> indicate you allocate some pools from the shm. > > Agreed. Will update in next version. >> >> >>> + sizeof(cos_tbl_t), >>> + sizeof(cos_t), 0); >>> + >>> + cos_tbl = odp_shm_addr(cos_shm); >>> + if (cos_tbl == NULL) { >>> + odp_shm_free(cos_shm); >>> + return -1; >>> + } >>> + >>> + memset(cos_tbl, 0, sizeof(cos_tbl_t)); >>> + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { >>> + /* init locks */ >>> + cos_t *cos = get_cos_entry_internal(i + 1); >>> + LOCK_INIT(&cos->s.lock); >>> + } >>> + >>> + pmr_shm = odp_shm_reserve("odp_pmr_pools", >>> + sizeof(pmr_tbl_t), >>> + sizeof(pmr_t), 0); >>> + pmr_tbl = odp_shm_addr(pmr_shm); >>> + if (pmr_tbl == NULL) { >>> + odp_shm_free(pmr_shm); >>> + return -1; >>> + } >>> + >>> + memset(pmr_tbl, 0, sizeof(pmr_tbl_t)); >>> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { >>> + /* init locks */ >>> + pmr_t *pmr = get_pmr_entry_internal(i + 1); >>> + LOCK_INIT(&pmr->s.lock); >>> + } >>> + >>> + pmr_set_shm = odp_shm_reserve("odp_pmr_set_pools", >>> + sizeof(pmr_set_tbl_t), >>> + sizeof(pmr_set_t), 0); >>> + pmr_set_tbl = odp_shm_addr(pmr_set_shm); >>> + if (pmr_set_tbl == NULL) { >>> + odp_shm_free(pmr_set_shm); >>> + return -1; >>> + } >>> + >>> + memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); >>> + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { >>> + /* init locks */ >>> + pmr_set_t *pmr = get_pmr_set_entry_internal(i + 1); >>> + LOCK_INIT(&pmr->s.pmr.s.lock); >>> + } >>> + >>> return 0; >>> } >>> >>> +odp_cos_t odp_cos_create(const char *name) >>> +{ >>> + int i; >>> + >>> + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { >>> + WRITE_LOCK(&cos_tbl->cos_entry[i].s.lock); >>> + if (0 == cos_tbl->cos_entry[i].s.valid) { >>> + strncpy(cos_tbl->cos_entry[i].s.name, name, >>> + ODP_COS_NAME_LEN - 1); >>> + cos_tbl->cos_entry[i].s.name[ODP_COS_NAME_LEN - >>> 1] = 0; >>> + cos_tbl->cos_entry[i].s.pmr = NULL; >>> + cos_tbl->cos_entry[i].s.queue = NULL; >> >> cost_tbl->cos_entry[i].s.queue_group is uninitialized. It's never used >> but I still think it should be properly reset here to make future work >> easier. The drop_policy also, if we have a default value for it. > > As said earlier, queue_group and drop policy are deferred from 1.0 > But will handle the initialization of the same here. >> >> >>> + cos_tbl->cos_entry[i].s.pool = NULL; >>> + cos_tbl->cos_entry[i].s.flow_set = 0; >>> + cos_tbl->cos_entry[i].s.headroom = 0; >>> + cos_tbl->cos_entry[i].s.valid = 1; >>> + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); >>> + return i + 1; >>> + } >>> + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); >>> + } >>> + return ODP_COS_INVALID; >>> +} >>> + >>> +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) >>> +{ >>> + int i; >>> + >>> + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { >>> + WRITE_LOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >>> + if (0 == pmr_set_tbl->pmr_set[i].s.pmr.s.valid) { >>> + pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; >>> + pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; >>> + pmr_set_tbl->pmr_set[i].s.pmr.s.cos = NULL; >>> + *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; >>> + odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] >>> + .s.pmr.s.count, 0); >>> + >>> WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >>> + return i + 1; >>> + } >>> + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); >>> + } >>> + return ODP_PMR_INVAL; >>> +} >>> + >>> +odp_pmr_t alloc_pmr(pmr_t **pmr) >>> +{ >>> + int i; >>> + >>> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { >>> + WRITE_LOCK(&pmr_tbl->pmr[i].s.lock); >>> + if (0 == pmr_tbl->pmr[i].s.valid) { >>> + pmr_tbl->pmr[i].s.valid = 1; >>> + pmr_tbl->pmr[i].s.cos = NULL; >>> + odp_atomic_init_u32(&pmr_tbl->pmr[i].s.count, 0); >>> + pmr_tbl->pmr[i].s.num_pmr = 0; >>> + *pmr = &pmr_tbl->pmr[i]; >>> + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); >>> + return i + 1; >>> + } >>> + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); >>> + } >>> + return ODP_PMR_INVAL; >>> +} >>> + >>> + >>> +cos_t *get_cos_entry(odp_cos_t cos_id) >>> +{ >>> + if (cos_id > ODP_COS_MAX_ENTRY || cos_id == ODP_COS_INVALID) >>> + return NULL; >>> + if (cos_tbl->cos_entry[cos_id - 1].s.valid == 0) >> >> In other parts of the file (e.g. odp_cos_destroy) access to 'valid' is >> done under lock. Don't we need it here too? If so, you might consider >> implementing a get_locked type of routine to keep subsequent access to >> fields in the entry in the same critical section. >> >>> + return NULL; >>> + return &(cos_tbl->cos_entry[cos_id - 1]); >>> +} >>> + >>> + >>> +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) >>> +{ >>> + if (pmr_set_id > ODP_PMRSET_MAX_ENTRY || pmr_set_id == >>> ODP_PMR_INVAL) >>> + return NULL; >>> + if (pmr_set_tbl->pmr_set[pmr_set_id - 1].s.pmr.s.valid == 0) >>> + return NULL; >>> + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); >>> +} >>> + >>> +pmr_t *get_pmr_entry(odp_pmr_t pmr_id) >>> +{ >>> + if (pmr_id > ODP_PMR_MAX_ENTRY || pmr_id == ODP_PMR_INVAL) >>> + return NULL; >>> + if (pmr_tbl->pmr[pmr_id - 1].s.valid == 0) >>> + return NULL; >>> + return &(pmr_tbl->pmr[pmr_id - 1]); >>> +} >>> + >>> int odp_cos_destroy(odp_cos_t cos_id) >>> { >>> - (void)cos_id; >>> - ODP_UNIMPLEMENTED(); >>> + cos_t *cos = get_cos_entry(cos_id); >>> + if (NULL == cos) >>> + return -1; >>> + >>> + WRITE_LOCK(&cos->s.lock); >>> + cos->s.valid = 0; >>> + WRITE_UNLOCK(&cos->s.lock); >>> return 0; >>> } >>> >>> int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) >>> { >>> - (void)cos_id; >>> - (void)queue_id; >>> - ODP_UNIMPLEMENTED(); >>> + cos_t *cos = get_cos_entry(cos_id); >>> + if (cos == NULL) >>> + return -1; >>> + >>> + WRITE_LOCK(&cos->s.lock); >>> + cos->s.queue = queue_to_qentry(queue_id); >>> + WRITE_UNLOCK(&cos->s.lock); >>> return 0; >>> } >>> >>> int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) >>> { >>> - (void)cos_id; >>> - (void)drop_policy; >>> - ODP_UNIMPLEMENTED(); >>> + cos_t *cos = get_cos_entry(cos_id); >>> + if (cos == NULL) >>> + return -1; >>> + >>> + WRITE_LOCK(&cos->s.lock); >>> + cos->s.drop_policy = drop_policy; >> >> The drop_policy is never used, I assume that's part of a future work? > > Yes. >> >> >>> + WRITE_UNLOCK(&cos->s.lock); >>> return 0; >>> } >>> >>> int odp_pktio_set_default_cos(odp_pktio_t pktio_in, odp_cos_t >>> default_cos) >>> { >>> - (void)pktio_in; >>> - (void)default_cos; >>> - ODP_UNIMPLEMENTED(); >>> + pktio_entry_t *entry; >>> + cos_t *cos; >>> + entry = get_pktio_entry(pktio_in); >>> + if (entry == NULL) >>> + return -1; >>> + cos = get_cos_entry(default_cos); >>> + if (cos == NULL) >>> + return -1; >>> + >>> + WRITE_LOCK(&entry->s.cls.lock); >>> + entry->s.cls.default_cos = cos; >>> + WRITE_UNLOCK(&entry->s.cls.lock); >>> + >>> return 0; >>> } >>> >>> int odp_pktio_set_error_cos(odp_pktio_t pktio_in, odp_cos_t error_cos) >>> { >>> - (void)pktio_in; >>> - (void)error_cos; >>> - ODP_UNIMPLEMENTED(); >>> + pktio_entry_t *entry; >>> + cos_t *cos; >>> + >>> + entry = get_pktio_entry(pktio_in); >>> + if (entry == NULL) >>> + return -1; >>> + cos = get_cos_entry(error_cos); >>> + if (cos == NULL) >>> + return -1; >>> + WRITE_LOCK(&entry->s.cls.lock); >>> + entry->s.cls.error_cos = cos; >>> + WRITE_UNLOCK(&entry->s.cls.lock); >>> return 0; >>> } >>> >>> int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset) >>> { >>> - (void)pktio_in; >>> - (void)offset; >>> - ODP_UNIMPLEMENTED(); >>> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >>> + if (entry == NULL) >>> + return -1; >>> + WRITE_LOCK(&entry->s.cls.lock); >>> + entry->s.cls.skip = offset; >>> + WRITE_UNLOCK(&entry->s.cls.lock); >>> return 0; >>> } >>> >>> -int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom) >>> +int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom) >>> { >>> - (void)port_id; >>> - (void)headroom; >>> - ODP_UNIMPLEMENTED(); >>> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >>> + if (entry == NULL) >>> + return -1; >>> + WRITE_LOCK(&entry->s.cls.lock); >>> + entry->s.cls.headroom = headroom; >>> + WRITE_UNLOCK(&entry->s.cls.lock); >>> return 0; >>> } >>> >>> @@ -72,11 +293,26 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, >>> uint8_t qos_table[], >>> odp_cos_t cos_table[]) >>> { >>> - (void)pktio_in; >>> - (void)num_qos; >>> - (void)qos_table; >>> - (void)cos_table; >>> - ODP_UNIMPLEMENTED(); >>> + pmr_l2_cos_t *l2_cos; >>> + size_t i; >>> + cos_t *cos; >>> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >>> + if (entry == NULL) >>> + return -1; >>> + READ_LOCK(&entry->s.cls.lock); >>> + l2_cos = &entry->s.cls.l2_cos_table; >>> + READ_UNLOCK(&entry->s.cls.lock); >>> + >>> + WRITE_LOCK(&l2_cos->lock); >>> + /* Update the L2 QoS table*/ >>> + for (i = 0; i < num_qos; i++) { >>> + cos = get_cos_entry(cos_table[i]); >>> + if (cos != NULL) { >>> + if (ODP_COS_MAX_L2_QOS > qos_table[i]) >>> + l2_cos->cos[qos_table[i]] = cos; >>> + } >>> + } >>> + WRITE_UNLOCK(&l2_cos->lock); >>> return 0; >>> } >>> >>> @@ -86,12 +322,28 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, >>> odp_cos_t cos_table[], >>> bool l3_preference) >>> { >>> - (void)pktio_in; >>> - (void)num_qos; >>> - (void)qos_table; >>> - (void)cos_table; >>> - (void)l3_preference; >>> - ODP_UNIMPLEMENTED(); >>> + pmr_l3_cos_t *l3_cos; >>> + size_t i; >>> + pktio_entry_t *entry = get_pktio_entry(pktio_in); >>> + cos_t *cos; >>> + >>> + if (entry == NULL) >>> + return -1; >>> + WRITE_LOCK(&entry->s.cls.lock); >>> + entry->s.cls.l3_precedence = l3_preference; >>> + l3_cos = &entry->s.cls.l3_cos_table; >>> + WRITE_UNLOCK(&entry->s.cls.lock); >>> + >>> + WRITE_LOCK(&l3_cos->lock); >>> + /* Update the L3 QoS table*/ >>> + for (i = 0; i < num_qos; i++) { >>> + cos = get_cos_entry(cos_table[i]); >>> + if (cos != NULL) { >>> + if (ODP_COS_MAX_L3_QOS > qos_table[i]) >>> + l3_cos->cos[qos_table[i]] = cos; >>> + } >>> + } >>> + WRITE_UNLOCK(&l3_cos->lock); >>> return 0; >>> } >>> >>> @@ -100,12 +352,23 @@ odp_pmr_t odp_pmr_create_match(odp_pmr_term_e term, >>> const void *mask, >>> size_t val_sz) >>> { >>> - (void)term; >>> - (void)val; >>> - (void)mask; >>> - (void)val_sz; >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + pmr_t *pmr; >>> + odp_pmr_t id; >>> + >>> + id = alloc_pmr(&pmr); >>> + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) >>> + return ODP_PMR_INVAL; >>> + >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.num_pmr = 1; >>> + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; >>> + pmr->s.pmr_term_value[0].term = term; >>> + pmr->s.pmr_term_value[0].mask.val = 0; >>> + pmr->s.pmr_term_value[0].mask.mask = 0; >>> + memcpy(&pmr->s.pmr_term_value[0].mask.val, val, val_sz); >>> + memcpy(&pmr->s.pmr_term_value[0].mask.mask, mask, val_sz); >>> + WRITE_UNLOCK(&pmr->s.lock); >>> + return id; >>> } >>> >>> odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, >>> @@ -113,18 +376,33 @@ odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, >>> const void *val2, >>> size_t val_sz) >>> { >>> - (void)term; >>> - (void)val1; >>> - (void)val2; >>> - (void)val_sz; >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + pmr_t *pmr; >>> + odp_pmr_t id; >>> + >>> + id = alloc_pmr(&pmr); >>> + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) >>> + return ODP_PMR_INVAL; >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.num_pmr = 1; >>> + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; >>> + pmr->s.pmr_term_value[0].term = term; >>> + pmr->s.pmr_term_value[0].range.val1 = 0; >>> + pmr->s.pmr_term_value[0].range.val2 = 0; >>> + memcpy(&pmr->s.pmr_term_value[0].range.val1, val1, val_sz); >>> + memcpy(&pmr->s.pmr_term_value[0].range.val2, val2, val_sz); >>> + WRITE_UNLOCK(&pmr->s.lock); >>> + return id; >>> } >>> >>> int odp_pmr_destroy(odp_pmr_t pmr_id) >>> { >>> - (void)pmr_id; >>> - ODP_UNIMPLEMENTED(); >>> + pmr_t *pmr = get_pmr_entry(pmr_id); >>> + >>> + if (pmr == NULL) >>> + return -1; >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.valid = 0; >>> + WRITE_UNLOCK(&pmr->s.lock); >>> return 0; >>> } >>> >>> @@ -132,64 +410,465 @@ int odp_pktio_pmr_cos(odp_pmr_t pmr_id, >>> odp_pktio_t src_pktio, >>> odp_cos_t dst_cos) >>> { >>> - (void)pmr_id; >>> - (void)src_pktio; >>> - (void)dst_cos; >>> - ODP_UNIMPLEMENTED(); >>> + uint8_t num_pmr; >>> + pktio_entry_t *pktio_entry; >>> + pmr_t *pmr; >>> + cos_t *cos; >>> + >>> + pktio_entry = get_pktio_entry(src_pktio); >>> + if (pktio_entry == NULL) >>> + return -1; >>> + >>> + pmr = get_pmr_entry(pmr_id); >>> + if (pmr == NULL) >>> + return -1; >>> + >>> + cos = get_cos_entry(dst_cos); >>> + if (cos == NULL) >>> + return -1; >>> + >>> + num_pmr = pktio_entry->s.cls.num_pmr; >>> + if (num_pmr >= ODP_PKTIO_MAX_PMR) >> >> num_pmr > ODP_PKTIO_MAX_PMR is a serious error, I suggest using >> ODP_ASSERT for such case > > This error need not result in complete failure of the system. > IMO, indicating the application that they have reached maximum PMR number > should be sufficient. It's not supposed to go beyond ODP_PKTIO_MAX_PMR, unless there is some synchronization error or a memory corruption. >> >> >>> + return -1; >>> + >>> + WRITE_LOCK(&pktio_entry->s.cls.lock); >>> + pktio_entry->s.cls.pmr[num_pmr] = pmr; >>> + pktio_entry->s.cls.num_pmr++; >>> + WRITE_UNLOCK(&pktio_entry->s.cls.lock); >>> + >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.cos = cos; >>> + WRITE_UNLOCK(&pmr->s.lock); >>> return 0; >>> } >>> >>> int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t >>> dst_cos) >>> { >>> - (void)pmr_id; >>> - (void)src_cos; >>> - (void)dst_cos; >>> - ODP_UNIMPLEMENTED(); >>> + cos_t *cos_src = get_cos_entry(src_cos); >>> + cos_t *cos_dst = get_cos_entry(dst_cos); >>> + pmr_t *pmr = get_pmr_entry(pmr_id); >>> + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) >>> + return -1; >>> + >>> + WRITE_LOCK(&cos_src->s.lock); >>> + cos_src->s.pmr = pmr; >>> + WRITE_UNLOCK(&cos_src->s.lock); >>> + >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.cos = cos_dst; >>> + WRITE_UNLOCK(&pmr->s.lock); >>> return 0; >>> } >>> >>> signed long odp_pmr_match_count(odp_pmr_t pmr_id) >>> { >>> - (void)pmr_id; >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + pmr_t *pmr = get_pmr_entry(pmr_id); >>> + if (pmr == NULL) >>> + return -1; >>> + return (signed long)odp_atomic_load_u32(&pmr->s.count); >>> } >>> >>> unsigned long long odp_pmr_terms_cap(void) >>> { >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + unsigned long long term_cap = 0; >>> + >>> + term_cap |= (1 << ODP_PMR_LEN); >>> + term_cap |= (1 << ODP_PMR_IPPROTO); >>> + term_cap |= (1 << ODP_PMR_UDP_DPORT); >>> + term_cap |= (1 << ODP_PMR_TCP_DPORT); >>> + term_cap |= (1 << ODP_PMR_UDP_SPORT); >>> + term_cap |= (1 << ODP_PMR_TCP_SPORT); >>> + term_cap |= (1 << ODP_PMR_SIP_ADDR); >>> + term_cap |= (1 << ODP_PMR_DIP_ADDR); >>> + return term_cap; >>> } >>> >>> unsigned odp_pmr_terms_avail(void) >>> { >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + unsigned count = 0; >>> + int i; >>> + >>> + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) >>> + if (!pmr_tbl->pmr[i].s.valid) >>> + count++; >>> + return count; >>> } >>> >>> int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, >>> odp_pmr_set_t *pmr_set_id) >>> { >>> - (void)num_terms; >>> - (void)terms; >>> - (void)pmr_set_id; >>> - ODP_UNIMPLEMENTED(); >>> - return 0; >>> + pmr_t *pmr; >>> + int i; >>> + uint32_t id; >>> + int val_sz; >>> + int count = 0; >>> + >> >> Check num_terms <= ODP_PMRTERM_MAX ? > > Agreed. This is a coding error. >> >> >>> + id = alloc_pmr_set(&pmr); >>> + if (id == ODP_PMR_INVAL) { >>> + *pmr_set_id = id; >>> + return -1; >>> + } >>> + >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.num_pmr = num_terms; >>> + for (i = 0; i < num_terms; i++) { >>> + pmr->s.pmr_term_value[i].match_type = >>> terms[i].match_type; >>> + if (terms[i].match_type == ODP_PMR_MASK) { >>> + val_sz = terms[i].mask.val_sz; >>> + if (val_sz > ODP_PMR_TERM_BYTES_MAX) >>> + continue; >>> + pmr->s.pmr_term_value[i].term = >>> terms[i].mask.term; >>> + pmr->s.pmr_term_value[i].mask.val = 0; >>> + pmr->s.pmr_term_value[i].mask.mask = 0; >>> + memcpy(&pmr->s.pmr_term_value[i].mask.val, >>> + terms[i].mask.val, val_sz); >>> + memcpy(&pmr->s.pmr_term_value[i].mask.mask, >>> + terms[i].mask.mask, val_sz); >>> + } else { >>> + val_sz = terms[i].range.val_sz; >>> + if (val_sz > ODP_PMR_TERM_BYTES_MAX) >>> + continue; >>> + pmr->s.pmr_term_value[i].term = >>> terms[i].range.term; >>> + pmr->s.pmr_term_value[i].range.val1 = 0; >>> + pmr->s.pmr_term_value[i].range.val2 = 0; >>> + memcpy(&pmr->s.pmr_term_value[i].range.val1, >>> + terms[i].range.val1, val_sz); >>> + memcpy(&pmr->s.pmr_term_value[i].range.val2, >>> + terms[i].range.val2, val_sz); >>> + } >>> + count++; >>> + } >>> + WRITE_UNLOCK(&pmr->s.lock); >>> + *pmr_set_id = id; >>> + return count; >>> } >>> >>> int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) >>> { >>> - (void)pmr_set_id; >>> - ODP_UNIMPLEMENTED(); >>> + pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); >>> + if (pmr_set == NULL) >>> + return -1; >>> + WRITE_LOCK(&pmr_set->s.pmr.s.lock); >>> + pmr_set->s.pmr.s.valid = 0; >>> + WRITE_UNLOCK(&pmr_set->s.pmr.s.lock); >>> return 0; >>> } >>> >>> int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t >>> src_pktio, >>> - odp_cos_t dst_cos) >>> + odp_cos_t dst_cos) >>> +{ >>> + uint8_t num_pmr; >>> + pktio_entry_t *pktio_entry; >>> + pmr_t *pmr; >>> + cos_t *cos; >>> + >>> + pktio_entry = get_pktio_entry(src_pktio); >>> + if (pktio_entry == NULL) >>> + return -1; >>> + >>> + pmr = (pmr_t *)get_pmr_set_entry(pmr_set_id); >>> + if (pmr == NULL) >>> + return -1; >>> + >>> + cos = get_cos_entry(dst_cos); >>> + if (cos == NULL) >>> + return -1; >>> + >>> + num_pmr = pktio_entry->s.cls.num_pmr; >>> + if (num_pmr >= ODP_PKTIO_MAX_PMR) >>> + return -1; >>> + >>> + WRITE_LOCK(&pktio_entry->s.cls.lock); >>> + pktio_entry->s.cls.pmr[num_pmr] = pmr; >>> + pktio_entry->s.cls.num_pmr++; >>> + WRITE_UNLOCK(&pktio_entry->s.cls.lock); >>> + >>> + WRITE_LOCK(&pmr->s.lock); >>> + pmr->s.cos = cos; >>> + WRITE_UNLOCK(&pmr->s.lock); >>> + return 0; >>> +} >>> + >>> +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) >>> +{ >>> + int pmr_failure = 0; >>> + int num_pmr; >>> + int i; >>> + pmr_term_value_t *term_value; >>> + >>> + READ_LOCK(&pmr->s.lock); >>> + if (!pmr->s.valid) { >>> + READ_UNLOCK(&pmr->s.lock); >>> + return 0; >>> + } >>> + num_pmr = pmr->s.num_pmr; >>> + >>> + /* Iterate through list of PMR Term values in a pmr_t */ >>> + for (i = 0; i < num_pmr; i++) { >>> + term_value = &pmr->s.pmr_term_value[i]; >>> + switch (term_value->term) { >>> + case ODP_PMR_LEN: >>> + if (!verify_pmr_packet_len(pkt_hdr, term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_ETHTYPE_0: >>> + if (!verify_pmr_eth_type_0(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_ETHTYPE_X: >>> + if (!verify_pmr_eth_type_x(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_VLAN_ID_0: >>> + if (!verify_pmr_vlan_id_0(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_VLAN_ID_X: >>> + if (!verify_pmr_vlan_id_x(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_DMAC: >>> + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_IPPROTO: >>> + if (!verify_pmr_ip_proto(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_UDP_DPORT: >>> + if (!verify_pmr_udp_dport(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_TCP_DPORT: >>> + if (!verify_pmr_tcp_dport(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_UDP_SPORT: >>> + if (!verify_pmr_udp_sport(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_TCP_SPORT: >>> + if (!verify_pmr_tcp_sport(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_SIP_ADDR: >>> + if (!verify_pmr_ipv4_saddr(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_DIP_ADDR: >>> + if (!verify_pmr_ipv4_daddr(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_SIP6_ADDR: >>> + if (!verify_pmr_ipv6_saddr(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_DIP6_ADDR: >>> + if (!verify_pmr_ipv6_daddr(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_IPSEC_SPI: >>> + if (!verify_pmr_ipsec_spi(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_LD_VNI: >>> + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, >>> + term_value)) >>> + pmr_failure = 1; >>> + break; >>> + case ODP_PMR_INNER_HDR_OFF: >>> + break; >>> + } >> >> nit: the closing brace (and the block below) should be on tab further >> to match beginning of switch statement > > Agreed. >> >> >>> + if (pmr_failure) { >>> + READ_UNLOCK(&pmr->s.lock); >>> + return false; >>> + } >>> + } >>> + READ_UNLOCK(&pmr->s.lock); >>> + odp_atomic_inc_u32(&pmr->s.count); >>> + return true; >>> +} >>> + >>> +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr,
diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h index 2c83c0f..f78724e 100644 --- a/helper/include/odph_ip.h +++ b/helper/include/odph_ip.h @@ -35,6 +35,9 @@ extern "C" { /** @internal Returns IPv4 header length */ #define ODPH_IPV4HDR_IHL(ver_ihl) ((ver_ihl) & 0x0f) +/** @internal Returns IPv4 DSCP */ +#define ODPH_IPV4HDR_DSCP(tos) (((tos) & 0xfc) >> 2) + /** @internal Returns IPv4 Don't fragment */ #define ODPH_IPV4HDR_FLAGS_DONT_FRAG(frag_offset) ((frag_offset) & 0x4000) @@ -47,6 +50,9 @@ extern "C" { /** @internal Returns true if IPv4 packet is a fragment */ #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff) +/** @internal Returns IPv4 DSCP */ +#define ODPH_IPV6HDR_DSCP(ver_tc_flow) (uint8_t)((((ver_tc_flow) & 0x0fc00000) >> 22) & 0xff) + /** IPv4 header */ typedef struct ODP_PACKED { uint8_t ver_ihl; /**< Version / Header length */ diff --git a/platform/linux-generic/include/api/odp.h b/platform/linux-generic/include/api/odp.h index 6e4f69e..b7b1ca9 100644 --- a/platform/linux-generic/include/api/odp.h +++ b/platform/linux-generic/include/api/odp.h @@ -47,6 +47,7 @@ extern "C" { #include <odp_packet_flags.h> #include <odp_packet_io.h> #include <odp_crypto.h> +#include <odp_classification.h> #include <odp_rwlock.h> #ifdef __cplusplus diff --git a/platform/linux-generic/include/odp_buffer_pool_internal.h b/platform/linux-generic/include/odp_buffer_pool_internal.h index e0210bd..bdbefff 100644 --- a/platform/linux-generic/include/odp_buffer_pool_internal.h +++ b/platform/linux-generic/include/odp_buffer_pool_internal.h @@ -64,6 +64,12 @@ struct pool_entry_s { size_t hdr_size; }; +typedef union pool_entry_u { + struct pool_entry_s s; + + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; + +} pool_entry_t; extern void *pool_entry_ptr[]; @@ -73,6 +79,10 @@ static inline void *get_pool_entry(uint32_t pool_id) return pool_entry_ptr[pool_id]; } +static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) +{ + return pool_hdl - 1; +} static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf) { diff --git a/platform/linux-generic/include/odp_classification_datamodel.h b/platform/linux-generic/include/odp_classification_datamodel.h new file mode 100644 index 0000000..f7c9fb5 --- /dev/null +++ b/platform/linux-generic/include/odp_classification_datamodel.h @@ -0,0 +1,199 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Datamodel + * Describes the classification internal data model + */ + +#ifndef ODP_CLASSIFICATION_DATAMODEL_H_ +#define ODP_CLASSIFICATION_DATAMODEL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <odp_rwlock.h> +#include <odp_classification.h> +#include <odp_buffer_pool_internal.h> +#include <odp_packet_internal.h> +#include <odp_packet_io_internal.h> +#include <odp_queue_internal.h> + +/* Maximum Class Of Service Entry */ +#define ODP_COS_MAX_ENTRY 64 +/* Maximum PMR Set Entry */ +#define ODP_PMRSET_MAX_ENTRY 64 +/* Maximum PMR Entry */ +#define ODP_PMR_MAX_ENTRY 64 +/* Maximum PMR Terms in a PMR Set */ +#define ODP_PMRTERM_MAX 8 +/* Maximum PMRs attached in PKTIO Level */ +#define ODP_PKTIO_MAX_PMR 8 +/* L2 Priority Bits */ +#define ODP_COS_L2_QOS_BITS 3 +/* Max L2 QoS value */ +#define ODP_COS_MAX_L2_QOS (1 << ODP_COS_L2_QOS_BITS) +/* L2 DSCP Bits */ +#define ODP_COS_L3_QOS_BITS 6 +/* Max L3 QoS Value */ +#define ODP_COS_MAX_L3_QOS (1 << ODP_COS_L3_QOS_BITS) +/* Max PMR Term bits */ +#define ODP_PMR_TERM_BYTES_MAX 8 + +/* forward declaration */ +typedef union pmr_u pmr_t; + +/** +Packet Matching Rule Term Value + +Stores the Term and Value mapping for a PMR. +The maximum size of value currently supported in 64 bits +**/ +typedef struct pmr_term_value { + odp_pmr_match_type_e match_type; /**< Packet Match Type*/ + odp_pmr_term_e term; /* PMR Term */ + union { + struct { + uint64_t val; + uint64_t mask; + } mask; /**< Match a masked set of bits */ + struct { + uint64_t val1; + uint64_t val2; + } range; /**< Match an integer range */ + }; +} pmr_term_value_t; + +/* +Class Of Service +*/ +struct cos_s { + odp_rwlock_t lock; /* cos rwlock */ + queue_entry_t *queue; /* Associated Queue */ + odp_queue_group_t queue_group; /* Associated Queue Group */ + pool_entry_t *pool; /* Associated Buffer pool */ + pmr_t *pmr; /* Associated PMR */ + bool valid; /* validity Flag */ + odp_drop_e drop_policy; /* Associated Drop Policy */ + odp_cos_flow_set_t flow_set; /* Assigned Flow Set */ + char name[ODP_COS_NAME_LEN]; /* name */ + size_t headroom; /* Headroom for this CoS */ +}; + +typedef union cos_u { + struct cos_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct cos_s))]; +} cos_t; + + +/** +Packet Matching Rule + +**/ +struct pmr_s { + odp_rwlock_t lock; /* pmr rwlock*/ + cos_t *cos; /* Associated CoS */ + odp_atomic_u32_t count; /* num of packets matching this rule */ + uint16_t num_pmr; /* num of PMR Term Values*/ + bool valid; /* Validity Flag */ + pmr_term_value_t pmr_term_value[1]; /* Associated PMR Term */ +}; + +typedef union pmr_u { + struct pmr_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_s))]; +} pmr_t; + +/** +Packet Matching Rule Set + +This structure is implemented as a extension over struct pmr_s +In order to use same pointer to access both pmr_s and pmr_set_s +'num_pmr' value is used to differentiate between pmr_s and pmr_set_s struct +**/ +struct pmr_set_s { + pmr_t pmr; + pmr_term_value_t pmr_term_value[ODP_PMRTERM_MAX - 1]; + /* List of associated PMR Terms */ +}; + +typedef union pmr_set_u { + struct pmr_set_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pmr_set_s))]; +} pmr_set_t; + +/** +L2 QoS and CoS Map + +This structure holds the mapping between L2 QoS value and +corresponding cos_t object +**/ +typedef struct pmr_l2_cos { + odp_rwlock_t lock; /* pmr_l2_cos rwlock */ + cos_t *cos[ODP_COS_MAX_L2_QOS]; /* Array of CoS objects */ +} pmr_l2_cos_t; + +/** +L3 QoS and CoS Map + +This structure holds the mapping between L3 QoS value and +corresponding cos_t object +**/ +typedef struct pmr_l3_cos { + odp_rwlock_t lock; /* pmr_l3_cos rwlock */ + cos_t *cos[ODP_COS_MAX_L3_QOS]; /* Array of CoS objects */ +} pmr_l3_cos_t; + +/** +Linux Generic Classifier + +This structure is stored in pktio_entry and holds all +the classifier configuration value. +**/ +typedef struct classifier { + odp_rwlock_t lock; /*pktio_cos rwlock */ + uint8_t num_pmr; /* num of PMRs linked to given PKTIO*/ + bool l3_precedence; /* L3 QoS precedence */ + odp_cos_flow_set_t flow_set; /* Flow Set to be calculated + for this pktio */ + pmr_l2_cos_t l2_cos_table; /* L2 QoS-CoS table map */ + pmr_l3_cos_t l3_cos_table; /* L3 Qos-CoS table map */ + pmr_t *pmr[ODP_PKTIO_MAX_PMR]; /* PMRs linked with this PKTIO */ + cos_t *error_cos; /* Associated Error CoS */ + cos_t *default_cos; /* Associated Default CoS */ + size_t headroom; /* Pktio Headroom */ + size_t skip; /* Pktio Skip Offset */ +} classifier_t; + +/** +Class of Service Table +**/ +typedef struct odp_cos_table { + cos_t cos_entry[ODP_COS_MAX_ENTRY]; +} cos_tbl_t; + +/** +PMR set table +**/ +typedef struct pmr_set_tbl { + pmr_set_t pmr_set[ODP_PMRSET_MAX_ENTRY]; +} pmr_set_tbl_t; + +/** +PMR table +**/ +typedef struct pmr_tbl { + pmr_t pmr[ODP_PMR_MAX_ENTRY]; +} pmr_tbl_t; + +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_classification_inlines.h b/platform/linux-generic/include/odp_classification_inlines.h new file mode 100644 index 0000000..6b20119 --- /dev/null +++ b/platform/linux-generic/include/odp_classification_inlines.h @@ -0,0 +1,259 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Inlines + * Classification Inlines Functions + */ +#ifndef __ODP_CLASSIFICATION_INLINES_H_ +#define __ODP_CLASSIFICATION_INLINES_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <odp_debug.h> +#include <odph_eth.h> +#include <odph_ip.h> +#include <odph_udp.h> +#include <odph_tcp.h> + +/* PMR term value verification function +These functions verify the given PMR term value with the value in the packet +These following functions return 1 on success and 0 on failure +*/ + +static inline int verify_pmr_packet_len(odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (pkt_hdr->frame_len & + term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= pkt_hdr->frame_len) && + (pkt_hdr->frame_len <= term_value->range.val2)) + return 1; + } + return 0; +} +static inline int verify_pmr_ip_proto(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint8_t proto; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + proto = ip->proto; + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (proto & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= proto) && + (proto <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_ipv4_saddr(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint32_t ipaddr; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + ipaddr = odp_be_to_cpu_32(ip->src_addr); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= ipaddr) && + (ipaddr <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_ipv4_daddr(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + odph_ipv4hdr_t *ip; + uint32_t ipaddr; + if (!pkt_hdr->input_flags.ipv4) + return 0; + ip = (odph_ipv4hdr_t *)(pkt_addr + pkt_hdr->l3_offset); + ipaddr = odp_be_to_cpu_32(ip->dst_addr); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (ipaddr & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= ipaddr) && + (ipaddr <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_tcp_sport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t sport; + odph_tcphdr_t *tcp; + if (!pkt_hdr->input_flags.tcp) + return 0; + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + sport = odp_be_to_cpu_16(tcp->src_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (sport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= sport) && + (sport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_tcp_dport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t dport; + odph_tcphdr_t *tcp; + if (!pkt_hdr->input_flags.tcp) + return 0; + tcp = (odph_tcphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + dport = odp_be_to_cpu_16(tcp->dst_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (dport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= dport) && + (dport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_udp_dport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t dport; + odph_udphdr_t *udp; + if (!pkt_hdr->input_flags.udp) + return 0; + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + dport = odp_be_to_cpu_16(udp->dst_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (dport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= dport) && + (dport <= term_value->range.val2)) + return 1; + } + return 0; +} +static inline int verify_pmr_udp_sport(uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr, + pmr_term_value_t *term_value) +{ + uint16_t sport; + odph_udphdr_t *udp; + if (!pkt_hdr->input_flags.udp) + return 0; + udp = (odph_udphdr_t *)(pkt_addr + pkt_hdr->l4_offset); + sport = odp_be_to_cpu_16(udp->src_port); + if (term_value->match_type == ODP_PMR_MASK) { + if (term_value->mask.val == (sport & term_value->mask.mask)) + return 1; + } else { + if ((term_value->range.val1 <= sport) && + (sport <= term_value->range.val2)) + return 1; + } + return 0; +} + +static inline int verify_pmr_dmac(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} + +static inline int verify_pmr_ipv6_saddr(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ipv6_daddr(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_vlan_id_0(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_vlan_id_x(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ipsec_spi(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_ld_vni(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_eth_type_0(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +static inline int verify_pmr_eth_type_x(uint8_t *pkt_addr ODP_UNUSED, + odp_packet_hdr_t *pkt_hdr ODP_UNUSED, + pmr_term_value_t *term_value ODP_UNUSED) +{ + ODP_UNIMPLEMENTED(); + return 0; +} +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_classification_internal.h b/platform/linux-generic/include/odp_classification_internal.h new file mode 100644 index 0000000..fd2c6af --- /dev/null +++ b/platform/linux-generic/include/odp_classification_internal.h @@ -0,0 +1,173 @@ +/* Copyright (c) 2014, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + + +/** + * @file + * + * ODP Classification Internal + * Describes the classification internal Functions + */ + +#ifndef __ODP_CLASSIFICATION_INTERNAL_H_ +#define __ODP_CLASSIFICATION_INTERNAL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include <odp_classification.h> +#include <odp_queue.h> +#include <odp_packet_internal.h> +#include <odp_packet_io.h> +#include <odp_packet_io_internal.h> +#include <odp_classification_datamodel.h> + +/** Classification Internal function **/ + +/** +@internal +Select a CoS for the given Packet based on pktio + +This function will call all the PMRs associated with a pktio for +a given packet and will return the matched COS object. +This function will check PMR, L2 and L3 QoS COS object associated +with the PKTIO interface. + +Returns the default cos if the packet does not match any PMR +Returns the error_cos if the packet has an error +**/ +cos_t *pktio_select_cos(pktio_entry_t *pktio, uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr); + +/** +@internal +match_qos_cos + +Select a CoS for the given Packet based on QoS values +This function returns the COS object matching the L2 and L3 QoS +based on the l3_preference value of the pktio +**/ +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); +/** +Packet Classifier + +Start function for Packet Classifier +This function calls Classifier module internal functions for a given packet and +enqueues the packet to specific Queue based on PMR and CoS selected. +**/ +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt); +/** +Packet IO classifier init + +This function does initialization of classifier object associated with pktio. +This function should be called during pktio initialization. +**/ +int pktio_classifier_init(pktio_entry_t *pktio); + +/** +@internal +match_pmr_cos + +Match a PMR chain with a Packet and return matching CoS +This function gets called recursively to check the chained PMR Term value +with the packet. + +**/ +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, + odp_packet_hdr_t *hdr); +/** +@internal +CoS associated with L3 QoS value + +This function returns the CoS associated with L3 QoS value +**/ +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); + +/** +@internal +CoS associated with L2 QoS value + +This function returns the CoS associated with L2 QoS value +**/ +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr); +/** +@internal +Flow Signature Calculation + +This function calculates the Flow Signature for a packet based on +CoS and updates in Packet Meta Data +**/ +int update_flow_signature(uint8_t *pkt_addr, cos_t *cos); + +/** +@internal +Allocate a odp_pmr_set_t Handle +*/ +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr); + +/** +@internal +Allocate a odp_pmr_t Handle +*/ +odp_pmr_t alloc_pmr(pmr_t **pmr); + +/** +@internal +Pointer to pmr_set_t Handle +This function checks for validity of pmr_set_t Handle +*/ +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id); + +/** +@internal +Pointer to pmr_set_t Handle +*/ +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id); + +/** +@internal +Pointer to pmr_set_t Handle +This function checks for validity of pmr_set_t Handle +*/ +pmr_t *get_pmr_entry(odp_pmr_t pmr_id); + +/** +@internal +Pointer to pmr_set_t Handle +*/ +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id); + +/** +@internal +Pointer to odp_cos_t Handle +*/ +cos_t *get_cos_entry(odp_cos_t cos_id); + +/** +@internal +Pointer to odp_cos_t Handle +This function checks for validity of odp_cos_t Handle +*/ +cos_t *get_cos_entry_internal(odp_cos_t cos_id); + +/** +@internal +Verify PMR with a Packet + +This function goes through each PMR_TERM value in pmr_t structure and +calls verification function for each term.Returns 1 if PMR matches or 0 +Otherwise. +**/ +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr); + +#ifdef __cplusplus +} +#endif +#endif diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index f8c1596..04c1030 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -32,6 +32,8 @@ int odp_buffer_pool_init_global(void); int odp_pktio_init_global(void); int odp_pktio_init_local(void); +int odp_classification_init_global(void); + int odp_queue_init_global(void); int odp_crypto_init_global(void); diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index 0bc1e21..218e9d0 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -20,6 +20,7 @@ extern "C" { #include <odp_spinlock.h> #include <odp_packet_socket.h> +#include <odp_classification_datamodel.h> #include <linux/if.h> @@ -40,6 +41,7 @@ struct pktio_entry { odp_pktio_type_t type; /**< pktio type */ pkt_sock_t pkt_sock; /**< using socket API for IO */ pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for IO */ + classifier_t cls; /**< classifier linked with this pktio*/ char name[IFNAMSIZ]; /**< name of pktio provided to pktio_open() */ }; @@ -49,6 +51,21 @@ typedef union { uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pktio_entry))]; } pktio_entry_t; +typedef struct { + pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; +} pktio_table_t; + +extern void *pktio_entry_ptr[]; + + +static inline pktio_entry_t *get_pktio_entry(odp_pktio_t id) +{ + if (odp_unlikely(id == ODP_PKTIO_INVALID || + id > ODP_CONFIG_PKTIO_ENTRIES)) + return NULL; + + return pktio_entry_ptr[id - 1]; +} #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/odp_buffer_pool.c b/platform/linux-generic/odp_buffer_pool.c index 6a0a6b2..b5bd613 100644 --- a/platform/linux-generic/odp_buffer_pool.c +++ b/platform/linux-generic/odp_buffer_pool.c @@ -56,12 +56,6 @@ typedef struct { } odp_any_buffer_hdr_t; -typedef union pool_entry_u { - struct pool_entry_s s; - - uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pool_entry_s))]; - -} pool_entry_t; typedef struct pool_table_t { @@ -86,10 +80,6 @@ static inline odp_buffer_pool_t pool_index_to_handle(uint32_t pool_id) } -static inline uint32_t pool_handle_to_index(odp_buffer_pool_t pool_hdl) -{ - return pool_hdl -1; -} static inline void set_handle(odp_buffer_hdr_t *hdr, diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index 190d71e..ecf97a0 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -2,68 +2,289 @@ #include <odp_align.h> #include <odp_queue.h> #include <odp_debug.h> +#include <odp_internal.h> #include <odp_debug_internal.h> +#include <odp_packet_internal.h> #include <odp_packet_io.h> +#include <odp_packet_io_internal.h> +#include <odp_classification_datamodel.h> +#include <odp_classification_inlines.h> +#include <odp_classification_internal.h> +#include <odp_buffer_pool_internal.h> +#include <odp_shared_memory.h> +#include <odph_eth.h> +#include <string.h> +#include <odp_rwlock.h> -odp_cos_t odp_cos_create(const char *name) +#define WRITE_LOCK(a) odp_rwlock_write_lock(a) +#define WRITE_UNLOCK(a) odp_rwlock_write_unlock(a) +#define LOCK_INIT(a) odp_rwlock_init(a) + +#define READ_LOCK(a) odp_rwlock_write_lock(a) +#define READ_UNLOCK(a) odp_rwlock_write_unlock(a) + +static cos_tbl_t *cos_tbl; +static pmr_set_tbl_t *pmr_set_tbl; +static pmr_tbl_t *pmr_tbl; + +cos_t *get_cos_entry_internal(odp_cos_t cos_id) +{ + return &(cos_tbl->cos_entry[cos_id - 1]); +} + +pmr_set_t *get_pmr_set_entry_internal(odp_pmr_set_t pmr_set_id) +{ + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); +} + +pmr_t *get_pmr_entry_internal(odp_pmr_t pmr_id) +{ + return &(pmr_tbl->pmr[pmr_id - 1]); +} + +int odp_classification_init_global(void) { - (void) name; - ODP_UNIMPLEMENTED(); + odp_shm_t cos_shm; + odp_shm_t pmr_shm; + odp_shm_t pmr_set_shm; + int i; + + cos_shm = odp_shm_reserve("odp_cos_pools", + sizeof(cos_tbl_t), + sizeof(cos_t), 0); + + cos_tbl = odp_shm_addr(cos_shm); + if (cos_tbl == NULL) { + odp_shm_free(cos_shm); + return -1; + } + + memset(cos_tbl, 0, sizeof(cos_tbl_t)); + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { + /* init locks */ + cos_t *cos = get_cos_entry_internal(i + 1); + LOCK_INIT(&cos->s.lock); + } + + pmr_shm = odp_shm_reserve("odp_pmr_pools", + sizeof(pmr_tbl_t), + sizeof(pmr_t), 0); + pmr_tbl = odp_shm_addr(pmr_shm); + if (pmr_tbl == NULL) { + odp_shm_free(pmr_shm); + return -1; + } + + memset(pmr_tbl, 0, sizeof(pmr_tbl_t)); + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { + /* init locks */ + pmr_t *pmr = get_pmr_entry_internal(i + 1); + LOCK_INIT(&pmr->s.lock); + } + + pmr_set_shm = odp_shm_reserve("odp_pmr_set_pools", + sizeof(pmr_set_tbl_t), + sizeof(pmr_set_t), 0); + pmr_set_tbl = odp_shm_addr(pmr_set_shm); + if (pmr_set_tbl == NULL) { + odp_shm_free(pmr_set_shm); + return -1; + } + + memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { + /* init locks */ + pmr_set_t *pmr = get_pmr_set_entry_internal(i + 1); + LOCK_INIT(&pmr->s.pmr.s.lock); + } + return 0; } +odp_cos_t odp_cos_create(const char *name) +{ + int i; + + for (i = 0; i < ODP_COS_MAX_ENTRY; i++) { + WRITE_LOCK(&cos_tbl->cos_entry[i].s.lock); + if (0 == cos_tbl->cos_entry[i].s.valid) { + strncpy(cos_tbl->cos_entry[i].s.name, name, + ODP_COS_NAME_LEN - 1); + cos_tbl->cos_entry[i].s.name[ODP_COS_NAME_LEN - 1] = 0; + cos_tbl->cos_entry[i].s.pmr = NULL; + cos_tbl->cos_entry[i].s.queue = NULL; + cos_tbl->cos_entry[i].s.pool = NULL; + cos_tbl->cos_entry[i].s.flow_set = 0; + cos_tbl->cos_entry[i].s.headroom = 0; + cos_tbl->cos_entry[i].s.valid = 1; + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); + return i + 1; + } + WRITE_UNLOCK(&cos_tbl->cos_entry[i].s.lock); + } + return ODP_COS_INVALID; +} + +odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) +{ + int i; + + for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { + WRITE_LOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + if (0 == pmr_set_tbl->pmr_set[i].s.pmr.s.valid) { + pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; + pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; + pmr_set_tbl->pmr_set[i].s.pmr.s.cos = NULL; + *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; + odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] + .s.pmr.s.count, 0); + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + return i + 1; + } + WRITE_UNLOCK(&pmr_set_tbl->pmr_set[i].s.pmr.s.lock); + } + return ODP_PMR_INVAL; +} + +odp_pmr_t alloc_pmr(pmr_t **pmr) +{ + int i; + + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { + WRITE_LOCK(&pmr_tbl->pmr[i].s.lock); + if (0 == pmr_tbl->pmr[i].s.valid) { + pmr_tbl->pmr[i].s.valid = 1; + pmr_tbl->pmr[i].s.cos = NULL; + odp_atomic_init_u32(&pmr_tbl->pmr[i].s.count, 0); + pmr_tbl->pmr[i].s.num_pmr = 0; + *pmr = &pmr_tbl->pmr[i]; + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); + return i + 1; + } + WRITE_UNLOCK(&pmr_tbl->pmr[i].s.lock); + } + return ODP_PMR_INVAL; +} + + +cos_t *get_cos_entry(odp_cos_t cos_id) +{ + if (cos_id > ODP_COS_MAX_ENTRY || cos_id == ODP_COS_INVALID) + return NULL; + if (cos_tbl->cos_entry[cos_id - 1].s.valid == 0) + return NULL; + return &(cos_tbl->cos_entry[cos_id - 1]); +} + + +pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) +{ + if (pmr_set_id > ODP_PMRSET_MAX_ENTRY || pmr_set_id == ODP_PMR_INVAL) + return NULL; + if (pmr_set_tbl->pmr_set[pmr_set_id - 1].s.pmr.s.valid == 0) + return NULL; + return &(pmr_set_tbl->pmr_set[pmr_set_id - 1]); +} + +pmr_t *get_pmr_entry(odp_pmr_t pmr_id) +{ + if (pmr_id > ODP_PMR_MAX_ENTRY || pmr_id == ODP_PMR_INVAL) + return NULL; + if (pmr_tbl->pmr[pmr_id - 1].s.valid == 0) + return NULL; + return &(pmr_tbl->pmr[pmr_id - 1]); +} + int odp_cos_destroy(odp_cos_t cos_id) { - (void)cos_id; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (NULL == cos) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.valid = 0; + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) { - (void)cos_id; - (void)queue_id; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.queue = queue_to_qentry(queue_id); + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) { - (void)cos_id; - (void)drop_policy; - ODP_UNIMPLEMENTED(); + cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) + return -1; + + WRITE_LOCK(&cos->s.lock); + cos->s.drop_policy = drop_policy; + WRITE_UNLOCK(&cos->s.lock); return 0; } int odp_pktio_set_default_cos(odp_pktio_t pktio_in, odp_cos_t default_cos) { - (void)pktio_in; - (void)default_cos; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry; + cos_t *cos; + entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + cos = get_cos_entry(default_cos); + if (cos == NULL) + return -1; + + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.default_cos = cos; + WRITE_UNLOCK(&entry->s.cls.lock); + return 0; } int odp_pktio_set_error_cos(odp_pktio_t pktio_in, odp_cos_t error_cos) { - (void)pktio_in; - (void)error_cos; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry; + cos_t *cos; + + entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + cos = get_cos_entry(error_cos); + if (cos == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.error_cos = cos; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset) { - (void)pktio_in; - (void)offset; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.skip = offset; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } -int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom) +int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom) { - (void)port_id; - (void)headroom; - ODP_UNIMPLEMENTED(); + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.headroom = headroom; + WRITE_UNLOCK(&entry->s.cls.lock); return 0; } @@ -72,11 +293,26 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, uint8_t qos_table[], odp_cos_t cos_table[]) { - (void)pktio_in; - (void)num_qos; - (void)qos_table; - (void)cos_table; - ODP_UNIMPLEMENTED(); + pmr_l2_cos_t *l2_cos; + size_t i; + cos_t *cos; + pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) + return -1; + READ_LOCK(&entry->s.cls.lock); + l2_cos = &entry->s.cls.l2_cos_table; + READ_UNLOCK(&entry->s.cls.lock); + + WRITE_LOCK(&l2_cos->lock); + /* Update the L2 QoS table*/ + for (i = 0; i < num_qos; i++) { + cos = get_cos_entry(cos_table[i]); + if (cos != NULL) { + if (ODP_COS_MAX_L2_QOS > qos_table[i]) + l2_cos->cos[qos_table[i]] = cos; + } + } + WRITE_UNLOCK(&l2_cos->lock); return 0; } @@ -86,12 +322,28 @@ int odp_cos_with_l3_qos(odp_pktio_t pktio_in, odp_cos_t cos_table[], bool l3_preference) { - (void)pktio_in; - (void)num_qos; - (void)qos_table; - (void)cos_table; - (void)l3_preference; - ODP_UNIMPLEMENTED(); + pmr_l3_cos_t *l3_cos; + size_t i; + pktio_entry_t *entry = get_pktio_entry(pktio_in); + cos_t *cos; + + if (entry == NULL) + return -1; + WRITE_LOCK(&entry->s.cls.lock); + entry->s.cls.l3_precedence = l3_preference; + l3_cos = &entry->s.cls.l3_cos_table; + WRITE_UNLOCK(&entry->s.cls.lock); + + WRITE_LOCK(&l3_cos->lock); + /* Update the L3 QoS table*/ + for (i = 0; i < num_qos; i++) { + cos = get_cos_entry(cos_table[i]); + if (cos != NULL) { + if (ODP_COS_MAX_L3_QOS > qos_table[i]) + l3_cos->cos[qos_table[i]] = cos; + } + } + WRITE_UNLOCK(&l3_cos->lock); return 0; } @@ -100,12 +352,23 @@ odp_pmr_t odp_pmr_create_match(odp_pmr_term_e term, const void *mask, size_t val_sz) { - (void)term; - (void)val; - (void)mask; - (void)val_sz; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + odp_pmr_t id; + + id = alloc_pmr(&pmr); + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) + return ODP_PMR_INVAL; + + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = 1; + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; + pmr->s.pmr_term_value[0].term = term; + pmr->s.pmr_term_value[0].mask.val = 0; + pmr->s.pmr_term_value[0].mask.mask = 0; + memcpy(&pmr->s.pmr_term_value[0].mask.val, val, val_sz); + memcpy(&pmr->s.pmr_term_value[0].mask.mask, mask, val_sz); + WRITE_UNLOCK(&pmr->s.lock); + return id; } odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, @@ -113,18 +376,33 @@ odp_pmr_t odp_pmr_create_range(odp_pmr_term_e term, const void *val2, size_t val_sz) { - (void)term; - (void)val1; - (void)val2; - (void)val_sz; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + odp_pmr_t id; + + id = alloc_pmr(&pmr); + if (id == ODP_PMR_INVAL || val_sz > ODP_PMR_TERM_BYTES_MAX) + return ODP_PMR_INVAL; + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = 1; + pmr->s.pmr_term_value[0].match_type = ODP_PMR_MASK; + pmr->s.pmr_term_value[0].term = term; + pmr->s.pmr_term_value[0].range.val1 = 0; + pmr->s.pmr_term_value[0].range.val2 = 0; + memcpy(&pmr->s.pmr_term_value[0].range.val1, val1, val_sz); + memcpy(&pmr->s.pmr_term_value[0].range.val2, val2, val_sz); + WRITE_UNLOCK(&pmr->s.lock); + return id; } int odp_pmr_destroy(odp_pmr_t pmr_id) { - (void)pmr_id; - ODP_UNIMPLEMENTED(); + pmr_t *pmr = get_pmr_entry(pmr_id); + + if (pmr == NULL) + return -1; + WRITE_LOCK(&pmr->s.lock); + pmr->s.valid = 0; + WRITE_UNLOCK(&pmr->s.lock); return 0; } @@ -132,64 +410,465 @@ int odp_pktio_pmr_cos(odp_pmr_t pmr_id, odp_pktio_t src_pktio, odp_cos_t dst_cos) { - (void)pmr_id; - (void)src_pktio; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + uint8_t num_pmr; + pktio_entry_t *pktio_entry; + pmr_t *pmr; + cos_t *cos; + + pktio_entry = get_pktio_entry(src_pktio); + if (pktio_entry == NULL) + return -1; + + pmr = get_pmr_entry(pmr_id); + if (pmr == NULL) + return -1; + + cos = get_cos_entry(dst_cos); + if (cos == NULL) + return -1; + + num_pmr = pktio_entry->s.cls.num_pmr; + if (num_pmr >= ODP_PKTIO_MAX_PMR) + return -1; + + WRITE_LOCK(&pktio_entry->s.cls.lock); + pktio_entry->s.cls.pmr[num_pmr] = pmr; + pktio_entry->s.cls.num_pmr++; + WRITE_UNLOCK(&pktio_entry->s.cls.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos; + WRITE_UNLOCK(&pmr->s.lock); return 0; } int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t dst_cos) { - (void)pmr_id; - (void)src_cos; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + cos_t *cos_src = get_cos_entry(src_cos); + cos_t *cos_dst = get_cos_entry(dst_cos); + pmr_t *pmr = get_pmr_entry(pmr_id); + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) + return -1; + + WRITE_LOCK(&cos_src->s.lock); + cos_src->s.pmr = pmr; + WRITE_UNLOCK(&cos_src->s.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos_dst; + WRITE_UNLOCK(&pmr->s.lock); return 0; } signed long odp_pmr_match_count(odp_pmr_t pmr_id) { - (void)pmr_id; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr = get_pmr_entry(pmr_id); + if (pmr == NULL) + return -1; + return (signed long)odp_atomic_load_u32(&pmr->s.count); } unsigned long long odp_pmr_terms_cap(void) { - ODP_UNIMPLEMENTED(); - return 0; + unsigned long long term_cap = 0; + + term_cap |= (1 << ODP_PMR_LEN); + term_cap |= (1 << ODP_PMR_IPPROTO); + term_cap |= (1 << ODP_PMR_UDP_DPORT); + term_cap |= (1 << ODP_PMR_TCP_DPORT); + term_cap |= (1 << ODP_PMR_UDP_SPORT); + term_cap |= (1 << ODP_PMR_TCP_SPORT); + term_cap |= (1 << ODP_PMR_SIP_ADDR); + term_cap |= (1 << ODP_PMR_DIP_ADDR); + return term_cap; } unsigned odp_pmr_terms_avail(void) { - ODP_UNIMPLEMENTED(); - return 0; + unsigned count = 0; + int i; + + for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) + if (!pmr_tbl->pmr[i].s.valid) + count++; + return count; } int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, odp_pmr_set_t *pmr_set_id) { - (void)num_terms; - (void)terms; - (void)pmr_set_id; - ODP_UNIMPLEMENTED(); - return 0; + pmr_t *pmr; + int i; + uint32_t id; + int val_sz; + int count = 0; + + id = alloc_pmr_set(&pmr); + if (id == ODP_PMR_INVAL) { + *pmr_set_id = id; + return -1; + } + + WRITE_LOCK(&pmr->s.lock); + pmr->s.num_pmr = num_terms; + for (i = 0; i < num_terms; i++) { + pmr->s.pmr_term_value[i].match_type = terms[i].match_type; + if (terms[i].match_type == ODP_PMR_MASK) { + val_sz = terms[i].mask.val_sz; + if (val_sz > ODP_PMR_TERM_BYTES_MAX) + continue; + pmr->s.pmr_term_value[i].term = terms[i].mask.term; + pmr->s.pmr_term_value[i].mask.val = 0; + pmr->s.pmr_term_value[i].mask.mask = 0; + memcpy(&pmr->s.pmr_term_value[i].mask.val, + terms[i].mask.val, val_sz); + memcpy(&pmr->s.pmr_term_value[i].mask.mask, + terms[i].mask.mask, val_sz); + } else { + val_sz = terms[i].range.val_sz; + if (val_sz > ODP_PMR_TERM_BYTES_MAX) + continue; + pmr->s.pmr_term_value[i].term = terms[i].range.term; + pmr->s.pmr_term_value[i].range.val1 = 0; + pmr->s.pmr_term_value[i].range.val2 = 0; + memcpy(&pmr->s.pmr_term_value[i].range.val1, + terms[i].range.val1, val_sz); + memcpy(&pmr->s.pmr_term_value[i].range.val2, + terms[i].range.val2, val_sz); + } + count++; + } + WRITE_UNLOCK(&pmr->s.lock); + *pmr_set_id = id; + return count; } int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) { - (void)pmr_set_id; - ODP_UNIMPLEMENTED(); + pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); + if (pmr_set == NULL) + return -1; + WRITE_LOCK(&pmr_set->s.pmr.s.lock); + pmr_set->s.pmr.s.valid = 0; + WRITE_UNLOCK(&pmr_set->s.pmr.s.lock); return 0; } int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t src_pktio, - odp_cos_t dst_cos) + odp_cos_t dst_cos) +{ + uint8_t num_pmr; + pktio_entry_t *pktio_entry; + pmr_t *pmr; + cos_t *cos; + + pktio_entry = get_pktio_entry(src_pktio); + if (pktio_entry == NULL) + return -1; + + pmr = (pmr_t *)get_pmr_set_entry(pmr_set_id); + if (pmr == NULL) + return -1; + + cos = get_cos_entry(dst_cos); + if (cos == NULL) + return -1; + + num_pmr = pktio_entry->s.cls.num_pmr; + if (num_pmr >= ODP_PKTIO_MAX_PMR) + return -1; + + WRITE_LOCK(&pktio_entry->s.cls.lock); + pktio_entry->s.cls.pmr[num_pmr] = pmr; + pktio_entry->s.cls.num_pmr++; + WRITE_UNLOCK(&pktio_entry->s.cls.lock); + + WRITE_LOCK(&pmr->s.lock); + pmr->s.cos = cos; + WRITE_UNLOCK(&pmr->s.lock); + return 0; +} + +int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) +{ + int pmr_failure = 0; + int num_pmr; + int i; + pmr_term_value_t *term_value; + + READ_LOCK(&pmr->s.lock); + if (!pmr->s.valid) { + READ_UNLOCK(&pmr->s.lock); + return 0; + } + num_pmr = pmr->s.num_pmr; + + /* Iterate through list of PMR Term values in a pmr_t */ + for (i = 0; i < num_pmr; i++) { + term_value = &pmr->s.pmr_term_value[i]; + switch (term_value->term) { + case ODP_PMR_LEN: + if (!verify_pmr_packet_len(pkt_hdr, term_value)) + pmr_failure = 1; + break; + case ODP_PMR_ETHTYPE_0: + if (!verify_pmr_eth_type_0(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_ETHTYPE_X: + if (!verify_pmr_eth_type_x(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_VLAN_ID_0: + if (!verify_pmr_vlan_id_0(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_VLAN_ID_X: + if (!verify_pmr_vlan_id_x(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DMAC: + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_IPPROTO: + if (!verify_pmr_ip_proto(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_UDP_DPORT: + if (!verify_pmr_udp_dport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_TCP_DPORT: + if (!verify_pmr_tcp_dport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_UDP_SPORT: + if (!verify_pmr_udp_sport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_TCP_SPORT: + if (!verify_pmr_tcp_sport(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_SIP_ADDR: + if (!verify_pmr_ipv4_saddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DIP_ADDR: + if (!verify_pmr_ipv4_daddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_SIP6_ADDR: + if (!verify_pmr_ipv6_saddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_DIP6_ADDR: + if (!verify_pmr_ipv6_daddr(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_IPSEC_SPI: + if (!verify_pmr_ipsec_spi(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_LD_VNI: + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, + term_value)) + pmr_failure = 1; + break; + case ODP_PMR_INNER_HDR_OFF: + break; + } + if (pmr_failure) { + READ_UNLOCK(&pmr->s.lock); + return false; + } + } + READ_UNLOCK(&pmr->s.lock); + odp_atomic_inc_u32(&pmr->s.count); + return true; +} + +cos_t *match_pmr_cos(cos_t *cos, uint8_t *pkt_addr, pmr_t *pmr, + odp_packet_hdr_t *hdr) +{ + cos_t *retcos = NULL; + + if (cos == NULL || pmr == NULL) + return NULL; + + READ_LOCK(&cos->s.lock); + if (!cos->s.valid) { + READ_UNLOCK(&cos->s.lock); + return NULL; + } + READ_UNLOCK(&cos->s.lock); + + if (verify_pmr(pmr, pkt_addr, hdr)) { + /** This gets called recursively to check all the PMRs in + * a PMR chain */ + retcos = match_pmr_cos(pmr->s.cos, pkt_addr, cos->s.pmr, hdr); + if (!retcos) + return cos; + } + return retcos; +} + +int pktio_classifier_init(pktio_entry_t *entry) { - (void)pmr_set_id; - (void)src_pktio; - (void)dst_cos; - ODP_UNIMPLEMENTED(); + classifier_t *cls; + if (entry == NULL) + return -1; + cls = &entry->s.cls; + cls->num_pmr = 0; + cls->flow_set = 0; + cls->error_cos = NULL; + cls->default_cos = NULL; + cls->headroom = 0; + cls->skip = 0; return 0; } + +int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt) +{ + pktio_entry_t *entry; + queue_entry_t *queue; + cos_t *cos; + odp_packet_hdr_t *pkt_hdr; + uint8_t *pkt_addr; + + entry = get_pktio_entry(pktio); + if (entry == NULL) + return -1; + + pkt_hdr = odp_packet_hdr(pkt); + pkt_addr = odp_packet_addr(pkt); + + /* Matching PMR and selecting the CoS for the packet*/ + cos = pktio_select_cos(entry, pkt_addr, pkt_hdr); + if (cos == NULL) + return -1; + + /* Enqueuing the Packet based on the CoS */ + READ_LOCK(&cos->s.lock); + queue = cos->s.queue; + READ_UNLOCK(&cos->s.lock); + return queue_enq(queue, odp_buf_to_hdr((odp_buffer_t)pkt)); +} + +cos_t *pktio_select_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *pkt_hdr) +{ + pmr_t *pmr; + cos_t *cos = NULL; + int i; + classifier_t *cls = &entry->s.cls; + + /* Return error cos for error packet */ + if (pkt_hdr->error_flags.all) + return cls->error_cos; + /* Calls all the PMRs attached at the PKTIO level*/ + for (i = 0; i < cls->num_pmr; i++) { + pmr = entry->s.cls.pmr[i]; + if (pmr) { + cos = match_pmr_cos(pmr->s.cos, pkt_addr, pmr, pkt_hdr); + if (cos) + return cos; + } + } + + cos = match_qos_cos(entry, pkt_addr, pkt_hdr); + if (cos) + return cos; + + return cls->default_cos; +} + +cos_t *match_qos_l3_cos(pmr_l3_cos_t *l3_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + uint8_t dscp; + cos_t *cos = NULL; + odph_ipv4hdr_t *ipv4; + odph_ipv6hdr_t *ipv6; + + READ_LOCK(&l3_cos->lock); + if (hdr->input_flags.l3 && hdr->input_flags.ipv4) { + ipv4 = (odph_ipv4hdr_t *)(pkt_addr + hdr->l3_offset); + dscp = ODPH_IPV4HDR_DSCP(ipv4->tos); + cos = l3_cos->cos[dscp]; + } else if (hdr->input_flags.l3 && hdr->input_flags.ipv6) { + ipv6 = (odph_ipv6hdr_t *)(pkt_addr + hdr->l3_offset); + dscp = ODPH_IPV6HDR_DSCP(ipv6->ver_tc_flow); + cos = l3_cos->cos[dscp]; + } + READ_UNLOCK(&l3_cos->lock); + + return cos; +} + +cos_t *match_qos_l2_cos(pmr_l2_cos_t *l2_cos, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + uint8_t qos; + cos_t *cos = NULL; + odph_ethhdr_t *eth; + odph_vlanhdr_t *vlan; + + if (hdr->input_flags.l2 && hdr->input_flags.vlan && + hdr->input_flags.eth) { + eth = (odph_ethhdr_t *)(pkt_addr + hdr->l2_offset); + vlan = (odph_vlanhdr_t *)(ð->type); + qos = ((vlan->tci >> 13) & 0xFF); + READ_LOCK(&l2_cos->lock); + cos = l2_cos->cos[qos]; + READ_UNLOCK(&l2_cos->lock); + } + return cos; +} + +cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, + odp_packet_hdr_t *hdr) +{ + classifier_t *cls = &entry->s.cls; + pmr_l2_cos_t *l2_cos; + pmr_l3_cos_t *l3_cos; + cos_t *cos; + + READ_LOCK(&cls->lock); + l2_cos = &cls->l2_cos_table; + l3_cos = &cls->l3_cos_table; + READ_UNLOCK(&cls->lock); + + if (cls->l3_precedence) { + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); + if (cos) + return cos; + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); + if (cos) + return cos; + } else { + cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); + if (cos) + return cos; + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); + if (cos) + return cos; + } + return NULL; +} diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 672b3d6..c661231 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -54,6 +54,10 @@ int odp_init_global(odp_init_t *params ODP_UNUSED, ODP_ERR("ODP crypto init failed.\n"); return -1; } + if (odp_classification_init_global()) { + ODP_ERR("ODP crypto init failed.\n"); + return -1; + } return 0; } diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index 5c8146a..27f7b0a 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -11,32 +11,23 @@ #include <odp_packet_internal.h> #include <odp_internal.h> #include <odp_spinlock.h> +#include <odp_rwlock.h> #include <odp_shared_memory.h> #include <odp_packet_socket.h> #include <odp_hints.h> #include <odp_config.h> #include <odp_queue_internal.h> #include <odp_schedule_internal.h> +#include <odp_classification_internal.h> #include <odp_debug.h> #include <string.h> #include <sys/ioctl.h> -typedef struct { - pktio_entry_t entries[ODP_CONFIG_PKTIO_ENTRIES]; -} pktio_table_t; - static pktio_table_t *pktio_tbl; - -static pktio_entry_t *get_entry(odp_pktio_t id) -{ - if (odp_unlikely(id == ODP_PKTIO_INVALID || - id > ODP_CONFIG_PKTIO_ENTRIES)) - return NULL; - - return &pktio_tbl->entries[id - 1]; -} +/* pktio pointer entries ( for inlines) */ +void *pktio_entry_ptr[ODP_CONFIG_PKTIO_ENTRIES]; int odp_pktio_init_global(void) { @@ -58,10 +49,12 @@ int odp_pktio_init_global(void) memset(pktio_tbl, 0, sizeof(pktio_table_t)); for (id = 1; id <= ODP_CONFIG_PKTIO_ENTRIES; ++id) { - pktio_entry = get_entry(id); + pktio_entry = &pktio_tbl->entries[id - 1]; odp_spinlock_init(&pktio_entry->s.lock); + odp_rwlock_init(&pktio_entry->s.cls.lock); + pktio_entry_ptr[id - 1] = pktio_entry; /* Create a default output queue for each pktio resource */ snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); name[ODP_QUEUE_NAME_LEN-1] = '\0'; @@ -108,12 +101,25 @@ static void unlock_entry(pktio_entry_t *entry) odp_spinlock_unlock(&entry->s.lock); } +static void lock_entry_classifier(pktio_entry_t *entry) +{ + odp_spinlock_lock(&entry->s.lock); + odp_rwlock_write_lock(&entry->s.cls.lock); +} + +static void unlock_entry_classifier(pktio_entry_t *entry) +{ + odp_rwlock_write_unlock(&entry->s.cls.lock); + odp_spinlock_unlock(&entry->s.lock); +} + static void init_pktio_entry(pktio_entry_t *entry) { set_taken(entry); entry->s.inq_default = ODP_QUEUE_INVALID; memset(&entry->s.pkt_sock, 0, sizeof(entry->s.pkt_sock)); memset(&entry->s.pkt_sock_mmap, 0, sizeof(entry->s.pkt_sock_mmap)); + pktio_classifier_init(entry); } static odp_pktio_t alloc_lock_pktio_entry(void) @@ -125,13 +131,13 @@ static odp_pktio_t alloc_lock_pktio_entry(void) for (i = 0; i < ODP_CONFIG_PKTIO_ENTRIES; ++i) { entry = &pktio_tbl->entries[i]; if (is_free(entry)) { - lock_entry(entry); + lock_entry_classifier(entry); if (is_free(entry)) { init_pktio_entry(entry); id = i + 1; return id; /* return with entry locked! */ } - unlock_entry(entry); + unlock_entry_classifier(entry); } } @@ -140,7 +146,7 @@ static odp_pktio_t alloc_lock_pktio_entry(void) static int free_pktio_entry(odp_pktio_t id) { - pktio_entry_t *entry = get_entry(id); + pktio_entry_t *entry = get_pktio_entry(id); if (entry == NULL) return -1; @@ -164,7 +170,7 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) } /* if successful, alloc_pktio_entry() returns with the entry locked */ - pktio_entry = get_entry(id); + pktio_entry = get_pktio_entry(id); if (!pktio_entry) return ODP_PKTIO_INVALID; @@ -200,14 +206,14 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool) close_pkt_sock(&pktio_entry->s.pkt_sock); } - unlock_entry(pktio_entry); + unlock_entry_classifier(pktio_entry); free_pktio_entry(id); ODP_ERR("Unable to init any I/O type.\n"); return ODP_PKTIO_INVALID; done: strncpy(pktio_entry->s.name, dev, IFNAMSIZ); - unlock_entry(pktio_entry); + unlock_entry_classifier(pktio_entry); return id; } @@ -216,7 +222,7 @@ int odp_pktio_close(odp_pktio_t id) pktio_entry_t *entry; int res = -1; - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) return -1; @@ -255,7 +261,7 @@ odp_pktio_t odp_pktio_get_input(odp_packet_t pkt) int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); int pkts; int i; @@ -293,7 +299,7 @@ int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); int pkts; if (pktio_entry == NULL) @@ -323,7 +329,7 @@ int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) int odp_pktio_inq_setdef(odp_pktio_t id, odp_queue_t queue) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); queue_entry_t *qentry = queue_to_qentry(queue); if (pktio_entry == NULL || qentry == NULL) @@ -353,7 +359,7 @@ int odp_pktio_inq_remdef(odp_pktio_t id) odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); if (pktio_entry == NULL) return ODP_QUEUE_INVALID; @@ -363,7 +369,7 @@ odp_queue_t odp_pktio_inq_getdef(odp_pktio_t id) odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id) { - pktio_entry_t *pktio_entry = get_entry(id); + pktio_entry_t *pktio_entry = get_pktio_entry(id); if (pktio_entry == NULL) return ODP_QUEUE_INVALID; @@ -423,7 +429,7 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) odp_buffer_t buf; odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; - int pkts, i; + int pkts, i, j; buf_hdr = queue_deq(qentry); if (buf_hdr != NULL) @@ -433,12 +439,15 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) if (pkts <= 0) return NULL; - for (i = 0; i < pkts; ++i) { + for (i = 0, j = 0; i < pkts; ++i) { buf = odp_packet_to_buffer(pkt_tbl[i]); - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); + buf_hdr = odp_buf_to_hdr(buf); + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) + tmp_hdr_tbl[j++] = buf_hdr; } - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); + if (j) + queue_enq_multi(qentry, tmp_hdr_tbl, j); buf_hdr = tmp_hdr_tbl[0]; return buf_hdr; } @@ -454,8 +463,9 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) int nbr; odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; + odp_buffer_hdr_t *tmp_hdr; odp_buffer_t buf; - int pkts, i; + int pkts, i, j; nbr = queue_deq_multi(qentry, buf_hdr, num); if (odp_unlikely(nbr > num)) @@ -472,12 +482,15 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) if (pkts <= 0) return nbr; - for (i = 0; i < pkts; ++i) { + for (i = 0, j = 0; i < pkts; ++i) { buf = odp_packet_to_buffer(pkt_tbl[i]); - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); + tmp_hdr = odp_buf_to_hdr(buf); + if (0 > packet_classifier(qentry->s.pktin, pkt_tbl[i])) + tmp_hdr_tbl[j++] = tmp_hdr; } - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); + if (j) + queue_enq_multi(qentry, tmp_hdr_tbl, j); return nbr; } @@ -493,7 +506,7 @@ int odp_pktio_set_mtu(odp_pktio_t id, int mtu) return -1; } - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) { ODP_DBG("pktio entry %d does not exist\n", id); return -1; @@ -523,7 +536,7 @@ int odp_pktio_mtu(odp_pktio_t id) struct ifreq ifr; int ret; - entry = get_entry(id); + entry = get_pktio_entry(id); if (entry == NULL) { ODP_DBG("pktio entry %d does not exist\n", id); return -1;
This patch contains classification implementation for ODP v1.0. The salient features of this classification version are as follows: * Attaches PMR, PMR_SET to a Pktio entry * Attaches CoS values for L2 and L3 QoS to a Pktio entry * Selects ClassOfService for a packet based on PMR, L2 QoS and L3 QoS values * Selects a default CoS if packet does not match any of the assigned rules * Selects an Error CoS for an Error packet * Enqueues the packet to the queue associated with the selected CoS This patch also adds classifier object to pktio entry and moves static inline functions to header files in Buffer and PKTIO modules. Signed-off-by: Balasubramanian Manoharan <bala.manoharan@linaro.org> --- V3: Patch has been merged as independently compilable unit helper/include/odph_ip.h | 6 + platform/linux-generic/include/api/odp.h | 1 + .../include/odp_buffer_pool_internal.h | 10 + .../include/odp_classification_datamodel.h | 199 +++++ .../include/odp_classification_inlines.h | 259 +++++++ .../include/odp_classification_internal.h | 173 +++++ platform/linux-generic/include/odp_internal.h | 2 + .../linux-generic/include/odp_packet_io_internal.h | 17 + platform/linux-generic/odp_buffer_pool.c | 10 - platform/linux-generic/odp_classification.c | 831 +++++++++++++++++++-- platform/linux-generic/odp_init.c | 4 + platform/linux-generic/odp_packet_io.c | 85 ++- 12 files changed, 1475 insertions(+), 122 deletions(-) create mode 100644 platform/linux-generic/include/odp_classification_datamodel.h create mode 100644 platform/linux-generic/include/odp_classification_inlines.h create mode 100644 platform/linux-generic/include/odp_classification_internal.h