From patchwork Tue Mar 10 15:31:09 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taras Kondratiuk X-Patchwork-Id: 45592 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f70.google.com (mail-wg0-f70.google.com [74.125.82.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 72967214BF for ; Tue, 10 Mar 2015 15:32:41 +0000 (UTC) Received: by wghk14 with SMTP id k14sf2017668wgh.3 for ; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version :content-type:content-transfer-encoding:errors-to:sender :x-original-sender:x-original-authentication-results:mailing-list; bh=EjViiBfaQJJX+uGOucbyJEgwtWX4cbjlr63hDS7Dwn4=; b=WNTo4bWkKjADbjO8oIkH9axV903bXBnWWSQpWUxmhJkiV/5ngNs1PUetRfBERxtP/u L6qCi1iJpwL76h5OxaYET6/VJGE222XXYpLd/NdnZGBrSawd3XYX61PQvPtKLaXX71Dp hZvEYKBIv3VgkkriDO7EtiJhbTXUjoajqSuLoGCKCTuJJiUf41nfMVQx/299k5fLzLtj gG+xRWnxJ5uRIGMCBJaY35rL1o/WviyuaU8E7rhkg3teYwhPB4Z7BRoNkTihET81wSAW BQmCnoUpCyLv5P1N4zUpmO09Mv4h9XKlBK7qXEbGCeZx1EonYlE22vdy0QX7bV88DpWb HUkw== X-Gm-Message-State: ALoCoQlxhzNk+Usj7kq+89rbpXPet+GFKTa438/5669EVbGAiegjzOt9yAAZq7jbkDbxVFToNXaz X-Received: by 10.112.130.70 with SMTP id oc6mr4926533lbb.13.1426001560565; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.7.3 with SMTP id f3ls45241laa.31.gmail; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) X-Received: by 10.112.150.131 with SMTP id ui3mr31213842lbb.62.1426001560381; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) Received: from mail-lb0-f178.google.com (mail-lb0-f178.google.com. [209.85.217.178]) by mx.google.com with ESMTPS id x7si592044laa.33.2015.03.10.08.32.40 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Mar 2015 08:32:40 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.178 as permitted sender) client-ip=209.85.217.178; Received: by lbvp9 with SMTP id p9so2542315lbv.10 for ; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) X-Received: by 10.112.162.232 with SMTP id yd8mr30453589lbb.41.1426001560202; Tue, 10 Mar 2015 08:32:40 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp2207092lbj; Tue, 10 Mar 2015 08:32:39 -0700 (PDT) X-Received: by 10.140.43.199 with SMTP id e65mr41189566qga.34.1426001558448; Tue, 10 Mar 2015 08:32:38 -0700 (PDT) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id f110si733171qgf.106.2015.03.10.08.32.37 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 10 Mar 2015 08:32:38 -0700 (PDT) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1YVM94-0003zZ-D8; Tue, 10 Mar 2015 15:32:34 +0000 Received: from mail-lb0-f176.google.com ([209.85.217.176]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1YVM88-0003pR-Im for lng-odp@lists.linaro.org; Tue, 10 Mar 2015 15:31:37 +0000 Received: by lbvp9 with SMTP id p9so2553141lbv.8 for ; Tue, 10 Mar 2015 08:31:31 -0700 (PDT) X-Received: by 10.152.88.1 with SMTP id bc1mr2547385lab.20.1426001491023; Tue, 10 Mar 2015 08:31:31 -0700 (PDT) Received: from uglx0153363.synapse.com ([195.238.92.128]) by mx.google.com with ESMTPSA id a2sm106609lbm.32.2015.03.10.08.31.29 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 10 Mar 2015 08:31:30 -0700 (PDT) From: Taras Kondratiuk To: lng-odp@lists.linaro.org Date: Tue, 10 Mar 2015 17:31:09 +0200 Message-Id: <1426001473-14618-12-git-send-email-taras.kondratiuk@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1426001473-14618-1-git-send-email-taras.kondratiuk@linaro.org> References: <1426001473-14618-1-git-send-email-taras.kondratiuk@linaro.org> X-Topics: patch Cc: Taras Kondratiuk Subject: [lng-odp] [KEYSTONE2 PATCH 11/15] linux-ks2: packet: update module X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: taras.kondratiuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Taras Kondratiuk Signed-off-by: Taras Kondratiuk --- platform/linux-keystone2/Makefile.am | 5 +- platform/linux-keystone2/include/odp.h | 2 + platform/linux-keystone2/include/odp/packet.h | 524 +++++++----- .../linux-keystone2/include/odp/packet_flags.h | 75 ++ .../include/odp/plat/packet_types.h | 38 + .../linux-keystone2/include/odp_packet_internal.h | 144 +--- platform/linux-keystone2/odp_packet.c | 914 ++++++++++++++++----- platform/linux-keystone2/odp_pool.c | 34 +- 8 files changed, 1193 insertions(+), 543 deletions(-) create mode 100644 platform/linux-keystone2/include/odp/packet_flags.h create mode 100644 platform/linux-keystone2/include/odp/plat/packet_types.h diff --git a/platform/linux-keystone2/Makefile.am b/platform/linux-keystone2/Makefile.am index a156c00..c482142 100644 --- a/platform/linux-keystone2/Makefile.am +++ b/platform/linux-keystone2/Makefile.am @@ -25,6 +25,7 @@ odpinclude_HEADERS = \ $(srcdir)/include/odp/buffer.h \ $(srcdir)/include/odp/crypto.h \ $(srcdir)/include/odp/event.h \ + $(srcdir)/include/odp/packet_flags.h \ $(srcdir)/include/odp/packet_io.h \ $(srcdir)/include/odp/packet.h \ $(srcdir)/include/odp/pool.h \ @@ -41,7 +42,6 @@ odpinclude_HEADERS = \ $(linux_generic_srcdir)/include/odp/hints.h \ $(linux_generic_srcdir)/include/odp/init.h \ $(linux_generic_srcdir)/include/odp/random.h \ - $(linux_generic_srcdir)/include/odp/packet_flags.h \ $(linux_generic_srcdir)/include/odp/queue.h \ $(linux_generic_srcdir)/include/odp/rwlock.h \ $(linux_generic_srcdir)/include/odp/schedule.h \ @@ -64,6 +64,7 @@ odpplatinclude_HEADERS = \ $(srcdir)/include/odp/plat/event_types.h \ $(srcdir)/include/odp/plat/mcsdk_tune.h \ $(srcdir)/include/odp/plat/osal.h \ + $(srcdir)/include/odp/plat/packet_types.h \ $(srcdir)/include/odp/plat/pool_types.h \ $(srcdir)/include/odp/plat/state.h \ $(srcdir)/include/odp/plat/ti_mcsdk.h \ @@ -72,7 +73,6 @@ odpplatinclude_HEADERS = \ $(linux_generic_srcdir)/include/odp/plat/classification_types.h \ $(linux_generic_srcdir)/include/odp/plat/cpumask_types.h \ $(linux_generic_srcdir)/include/odp/plat/crypto_types.h \ - $(linux_generic_srcdir)/include/odp/plat/packet_types.h \ $(linux_generic_srcdir)/include/odp/plat/packet_io_types.h \ $(linux_generic_srcdir)/include/odp/plat/queue_types.h \ $(linux_generic_srcdir)/include/odp/plat/schedule_types.h \ @@ -134,6 +134,7 @@ __LIB__libodp_la_SOURCES = \ odp_init.c \ odp_pool.c \ odp_buffer.c \ + odp_packet.c \ mcsdk/mcsdk_init.c \ mcsdk/mcsdk_navig.c \ mcsdk/mcsdk_rmclient.c \ diff --git a/platform/linux-keystone2/include/odp.h b/platform/linux-keystone2/include/odp.h index cf52c25..8737b9f 100644 --- a/platform/linux-keystone2/include/odp.h +++ b/platform/linux-keystone2/include/odp.h @@ -38,6 +38,8 @@ extern "C" { #include #include #include +#include +#include #include #include #include diff --git a/platform/linux-keystone2/include/odp/packet.h b/platform/linux-keystone2/include/odp/packet.h index 9cfe41d..1104006 100644 --- a/platform/linux-keystone2/include/odp/packet.h +++ b/platform/linux-keystone2/include/odp/packet.h @@ -19,233 +19,387 @@ extern "C" { #endif -#include +#include +#include +#include +#include +#include +#include +#include +#include +#include /** - * ODP packet descriptor + * Packet input & protocol flags + */ +typedef union { + /* All input flags */ + uint32_t all; + + struct { + /* Bitfield flags for each protocol */ + uint32_t l2:1; /**< known L2 protocol present */ + uint32_t l3:1; /**< known L3 protocol present */ + uint32_t l4:1; /**< known L4 protocol present */ + + uint32_t eth:1; /**< Ethernet */ + uint32_t jumbo:1; /**< Jumbo frame */ + uint32_t vlan:1; /**< VLAN hdr found */ + uint32_t vlan_qinq:1; /**< Stacked VLAN found, QinQ */ + + uint32_t snap:1; /**< SNAP */ + uint32_t arp:1; /**< ARP */ + + uint32_t ipv4:1; /**< IPv4 */ + uint32_t ipv6:1; /**< IPv6 */ + uint32_t ipfrag:1; /**< IP fragment */ + uint32_t ipopt:1; /**< IP optional headers */ + uint32_t ipsec:1; /**< IPSec decryption may be needed */ + + uint32_t udp:1; /**< UDP */ + uint32_t tcp:1; /**< TCP */ + uint32_t sctp:1; /**< SCTP */ + uint32_t icmp:1; /**< ICMP */ + uint32_t tcpopt:1; /**< TCP options */ + }; +} input_flags_t; + +_ODP_STATIC_ASSERT(sizeof(input_flags_t) == sizeof(uint32_t), + "INPUT_FLAGS_SIZE_ERROR"); + +/** + * Packet error flags */ -typedef odp_buffer_t odp_packet_t; +typedef union { + /* All error flags */ + uint32_t all; + + struct { + /* Bitfield flags for each detected error */ + uint32_t frame_len:1; /**< Frame length error */ + uint32_t snap_len:1; /**< Snap length error */ + uint32_t l2_chksum:1; /**< L2 checksum error, checks TBD */ + uint32_t ip_err:1; /**< IP error, checks TBD */ + uint32_t tcp_err:1; /**< TCP error, checks TBD */ + uint32_t udp_err:1; /**< UDP error, checks TBD */ + }; +} error_flags_t; + +_ODP_STATIC_ASSERT(sizeof(error_flags_t) == sizeof(uint32_t), + "ERROR_FLAGS_SIZE_ERROR"); -/** Invalid packet */ -#define ODP_PACKET_INVALID ODP_BUFFER_INVALID +/** + * Packet output flags + */ +typedef union { + /* All output flags */ + uint32_t all; -/** Invalid offset */ -#define ODP_PACKET_OFFSET_INVALID ((size_t)-1) + struct { + /* Bitfield flags for each output option */ + uint32_t l4_chksum:1; /**< Request L4 checksum calculation */ + }; +} output_flags_t; +_ODP_STATIC_ASSERT(sizeof(output_flags_t) == sizeof(uint32_t), + "OUTPUT_FLAGS_SIZE_ERROR"); /** - * Initialize the packet - * - * Needs to be called if the user allocates a packet buffer, i.e. the packet - * has not been received from I/O through ODP. - * - * @param pkt Packet handle + * Internal Packet header */ -void odp_packet_init(odp_packet_t pkt); +typedef struct odp_pkthdr { + union usr_data { + uint64_t u64; + const void *ptr; + } usr_data; + + input_flags_t input_flags; + error_flags_t error_flags; + output_flags_t output_flags; + + uint16_t l2_offset; /**< offset to L2 hdr, e.g. Eth */ + uint16_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ + uint16_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + + + odp_pktio_t input; + + struct { + int16_t saved_buf_offset; + uint32_t hash_offset; + union { + struct { + } enc; + struct { + uint32_t hash_tag[5]; + } dec; + }; + + } crypto; + +} odp_packet_hdr_t; /** - * Convert from packet handle to buffer handle - * - * @param buf Buffer handle - * - * @return Packet handle + * Packet headroom size reserved for implementation metadata. */ -static inline odp_packet_t odp_packet_from_buffer(odp_buffer_t buf) +#define _ODP_PKTHDR_SIZE ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_packet_hdr_t)) + +_ODP_STATIC_ASSERT(_ODP_PKTHDR_SIZE <= ODP_CACHE_LINE_SIZE, + "_ODP_PKTHDR_SIZE <= ODP_CACHE_LINE_SIZE"); + +static inline odp_packet_t _odp_ev_to_pkt(odp_event_t ev) { - return buf; + return (odp_packet_t)ev; } -/** - * Convert a packet handle to a buffer handle - * - * @param pkt Packet handle - * - * @return Buffer handle - */ -static inline odp_buffer_t odp_packet_to_buffer(odp_packet_t pkt) +static inline odp_event_t _odp_pkt_to_ev(odp_packet_t pkt) { - return pkt; + return (odp_event_t)pkt; } -/** - * Set the packet length - * - * @param pkt Packet handle - * @param len Length of packet in bytes - */ -void odp_packet_set_len(odp_packet_t pkt, size_t len); +static inline Cppi_HostDesc *_odp_pkt_to_cppi_desc(odp_packet_t pkt) +{ + return _odp_ev_to_cppi_desc(_odp_pkt_to_ev(pkt)); +} -/** - * Get the packet length - * - * @param pkt Packet handle - * - * @return Packet length in bytes - */ -size_t odp_packet_get_len(odp_packet_t pkt); +static inline odp_packet_t _cppi_desc_to_odp_pkt(Cppi_HostDesc *desc) +{ + return _odp_ev_to_pkt(_cppi_desc_to_odp_ev(desc)); +} -/** - * Packet buffer start address - * - * Returns a pointer to the start of the packet buffer. The address is not - * necessarily the same as packet data address. E.g. on a received Ethernet - * frame, the protocol header may start 2 or 6 bytes within the buffer to - * ensure 32 or 64-bit alignment of the IP header. - * - * Use odp_packet_l2(pkt) to get the start address of a received valid frame - * or odp_packet_data(pkt) to get the current packet data address. - * - * @param pkt Packet handle - * - * @return Pointer to the start of the packet buffer - * - * @see odp_packet_l2(), odp_packet_data() - */ -uint8_t *odp_packet_addr(odp_packet_t pkt); +static inline Cppi_HostDesc *_odp_pktseg_to_cppi_desc(odp_packet_seg_t seg) +{ + return _odp_pkt_to_cppi_desc((odp_packet_t)seg); +} -/** - * Packet data address - * - * Returns the current packet data address. When a packet is received from - * packet input, the data address points to the first byte of the packet. - * - * @param pkt Packet handle - * - * @return Pointer to the packet data - * - * @see odp_packet_l2(), odp_packet_addr() - */ -uint8_t *odp_packet_data(odp_packet_t pkt); +static inline odp_packet_seg_t _cppi_desc_to_odp_pktseg(Cppi_HostDesc *desc) +{ + return (odp_packet_seg_t)_cppi_desc_to_odp_pkt(desc); +} -/** - * Get pointer to the start of the L2 frame - * - * The L2 frame header address is not necessarily the same as the address of the - * packet buffer, see odp_packet_buf_addr() - * - * @param pkt Packet handle - * - * @return Pointer to L2 header or NULL if not found - * - * @see odp_packet_buf_addr(), odp_packet_start() - */ -uint8_t *odp_packet_l2(odp_packet_t pkt); -/** - * Return the byte offset from the packet buffer to the L2 frame - * - * @param pkt Packet handle - * - * @return L2 byte offset or ODP_PACKET_OFFSET_INVALID if not found - */ -size_t odp_packet_l2_offset(odp_packet_t pkt); +static inline struct odp_pkthdr *_odp_pkt_hdr(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint8_t *orig_ptr = _cppi_desc_orig_vptr(desc); + ODP_ASSERT(ODP_ALIGNED_CHECK(orig_ptr, ODP_ALIGNOF(odp_packet_hdr_t)), + "Wrong packet header alignment"); + return (odp_packet_hdr_t *)(void *)orig_ptr; +} -/** - * Set the byte offset to the L2 frame - * - * @param pkt Packet handle - * @param offset L2 byte offset +/* + * Public API */ -void odp_packet_set_l2_offset(odp_packet_t pkt, size_t offset); -/** - * Get pointer to the start of the L3 packet - * - * @param pkt Packet handle - * - * @return Pointer to L3 packet or NULL if not found - * - */ -uint8_t *odp_packet_l3(odp_packet_t pkt); +static inline odp_packet_t odp_packet_from_event(odp_event_t ev) +{ + return _odp_ev_to_pkt(ev); +} -/** - * Return the byte offset from the packet buffer to the L3 packet - * - * @param pkt Packet handle - * - * @return L3 byte offset or ODP_PACKET_OFFSET_INVALID if not found - */ -size_t odp_packet_l3_offset(odp_packet_t pkt); +static inline odp_event_t odp_packet_to_event(odp_packet_t pkt) +{ + return _odp_pkt_to_ev(pkt); +} -/** - * Set the byte offset to the L3 packet - * - * @param pkt Packet handle - * @param offset L3 byte offset - */ -void odp_packet_set_l3_offset(odp_packet_t pkt, size_t offset); +static inline void *odp_packet_head(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_orig_vptr(desc) + _ODP_PKTHDR_SIZE; +} +static inline uint32_t odp_packet_buf_len(odp_packet_t pkt) +{ + /** @todo: cache total packet length in a first descriptor */ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_orig_len_total(desc) - _ODP_PKTHDR_SIZE; +} -/** - * Get pointer to the start of the L4 packet - * - * @param pkt Packet handle - * - * @return Pointer to L4 packet or NULL if not found - * - */ -uint8_t *odp_packet_l4(odp_packet_t pkt); +static inline void *odp_packet_data(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_buf_vptr(desc); +} -/** - * Return the byte offset from the packet buffer to the L4 packet - * - * @param pkt Packet handle - * - * @return L4 byte offset or ODP_PACKET_OFFSET_INVALID if not found - */ -size_t odp_packet_l4_offset(odp_packet_t pkt); +static inline uint32_t odp_packet_seg_len(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_buf_len(desc); +} -/** - * Set the byte offset to the L4 packet - * - * @param pkt Packet handle - * @param offset L4 byte offset - */ -void odp_packet_set_l4_offset(odp_packet_t pkt, size_t offset); +static inline uint32_t odp_packet_len(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_pkt_len(desc); +} -/** - * Print (debug) information about the packet - * - * @param pkt Packet handle - */ -void odp_packet_print(odp_packet_t pkt); +static inline uint32_t odp_packet_headroom(odp_packet_t pkt) +{ + return (uint8_t *)odp_packet_data(pkt) - + (uint8_t *)odp_packet_head(pkt); +} -/** - * Copy contents and metadata from pkt_src to pkt_dst - * Useful when creating copies of packets - * - * @param pkt_dst Destination packet - * @param pkt_src Source packet - * - * @return 0 if successful - */ -int odp_packet_copy(odp_packet_t pkt_dst, odp_packet_t pkt_src); +static inline uint32_t odp_packet_tailroom(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _cppi_desc_last(_odp_pkt_to_cppi_desc(pkt)); + uint8_t *seg_end = _cppi_desc_orig_end(desc); + uint8_t *data_end = _cppi_desc_buf_end(desc); + return seg_end - data_end; +} -/** - * Set packet user context - * - * @param pkt Packet handle - * @param ctx User context - * - */ -static inline void odp_packet_set_ctx(odp_packet_t pkt, void *ctx) +static inline void *odp_packet_tail(odp_packet_t pkt) { - odp_buffer_set_ctx(odp_packet_to_buffer(pkt), ctx); + Cppi_HostDesc *desc = _cppi_desc_last(_odp_pkt_to_cppi_desc(pkt)); + return _cppi_desc_buf_end(desc); } -/** - * Get packet user context - * - * @param pkt Packet handle - * - * @return User context - */ -static inline void *odp_packet_get_ctx(odp_packet_t pkt) +static inline odp_pool_t odp_packet_pool(odp_packet_t pkt) +{ + return _odp_event_pool(_odp_pkt_to_ev(pkt)); +} + +static inline odp_pktio_t odp_packet_input(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->input; +} + +static inline void odp_packet_user_ptr_set(odp_packet_t pkt, const void *ctx) { - return odp_buffer_get_ctx(odp_packet_to_buffer(pkt)); + _odp_pkt_hdr(pkt)->usr_data.ptr = ctx; } +static inline void *odp_packet_user_ptr(odp_packet_t pkt) +{ + return (void *)(uintptr_t)_odp_pkt_hdr(pkt)->usr_data.ptr; +} + +static inline uint64_t odp_packet_user_u64(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->usr_data.u64; +} + +static inline void odp_packet_user_u64_set(odp_packet_t pkt, uint64_t ctx) +{ + _odp_pkt_hdr(pkt)->usr_data.u64 = ctx; +} + +static inline uint32_t odp_packet_l2_offset(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->l2_offset; +} + +static inline int odp_packet_l2_offset_set(odp_packet_t pkt, uint32_t offset) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + if (_cppi_desc_pkt_len(desc) <= offset) + return -1; + _odp_pkt_hdr(pkt)->l2_offset = offset; + return 0; +} + +static inline uint32_t odp_packet_l3_offset(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->l3_offset; +} + +static inline int odp_packet_l3_offset_set(odp_packet_t pkt, uint32_t offset) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + if (_cppi_desc_pkt_len(desc) <= offset) + return -1; + _odp_pkt_hdr(pkt)->l3_offset = offset; + return 0; +} + +static inline uint32_t odp_packet_l4_offset(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->l4_offset; +} + +static inline int odp_packet_l4_offset_set(odp_packet_t pkt, uint32_t offset) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + if (_cppi_desc_pkt_len(desc) <= offset) + return -1; + _odp_pkt_hdr(pkt)->l4_offset = offset; + return 0; +} + +static inline int odp_packet_is_segmented(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_vnext(desc) != NULL; +} + +static inline int odp_packet_num_segs(odp_packet_t pkt) +{ + /** @todo: cache number of_segments in a first descriptor */ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_num_bufs(desc); +} + +static inline odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + return _cppi_desc_to_odp_pktseg(desc); +} + +static inline odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt) +{ + /** @todo: cache the last_segment in a first descriptor */ + Cppi_HostDesc *desc = _cppi_desc_last(_odp_pkt_to_cppi_desc(pkt)); + return _cppi_desc_to_odp_pktseg(desc); +} + +static inline +odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt ODP_UNUSED, + odp_packet_seg_t seg) +{ + Cppi_HostDesc *desc = _odp_pktseg_to_cppi_desc(seg); + return _cppi_desc_to_odp_pktseg(_cppi_desc_vnext(desc)); +} + +static inline +void *odp_packet_seg_buf_addr(odp_packet_t pkt, odp_packet_seg_t seg) +{ + Cppi_HostDesc *pkt_desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *seg_desc = _odp_pktseg_to_cppi_desc(seg); + uint32_t header_len; + + /* There is a packet header in a first segment */ + header_len = (pkt_desc == seg_desc) ? _ODP_PKTHDR_SIZE : 0; + return _cppi_desc_orig_vptr(seg_desc) + header_len; +} + +static inline +uint32_t odp_packet_seg_buf_len(odp_packet_t pkt, odp_packet_seg_t seg) +{ + Cppi_HostDesc *pkt_desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *seg_desc = _odp_pktseg_to_cppi_desc(seg); + uint32_t header_len; + + /* There is a packet header in a first segment */ + header_len = (pkt_desc == seg_desc) ? _ODP_PKTHDR_SIZE : 0; + return _cppi_desc_orig_len(seg_desc) - header_len; +} + +static inline +void *odp_packet_seg_data(odp_packet_t pkt ODP_UNUSED, odp_packet_seg_t seg) +{ + Cppi_HostDesc *desc = _odp_pktseg_to_cppi_desc(seg); + return _cppi_desc_buf_vptr(desc); +} + +static inline +uint32_t odp_packet_seg_data_len(odp_packet_t pkt ODP_UNUSED, + odp_packet_seg_t seg) +{ + Cppi_HostDesc *desc = _odp_pktseg_to_cppi_desc(seg); + return _cppi_desc_buf_len(desc); +} + +#include + #ifdef __cplusplus } #endif diff --git a/platform/linux-keystone2/include/odp/packet_flags.h b/platform/linux-keystone2/include/odp/packet_flags.h new file mode 100644 index 0000000..6b1d933 --- /dev/null +++ b/platform/linux-keystone2/include/odp/packet_flags.h @@ -0,0 +1,75 @@ +/* + * Copyright (c) 2014, Linaro Limited + * Copyright (c) 2014, Texas Instruments Incorporated + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP packet flags + */ + +#ifndef ODP_PACKET_FLAGS_H_ +#define ODP_PACKET_FLAGS_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#define _INFLAG_GET_SET(flag) \ +static inline int odp_packet_has_##flag(odp_packet_t pkt) \ +{ \ + return _odp_pkt_hdr(pkt)->input_flags.flag; \ +} \ + \ +static inline void odp_packet_has_##flag##_set(odp_packet_t pkt, int val) \ +{ \ + _odp_pkt_hdr(pkt)->input_flags.flag = val; \ +} + +_INFLAG_GET_SET(l2) +_INFLAG_GET_SET(l3); +_INFLAG_GET_SET(l4); +_INFLAG_GET_SET(eth); +_INFLAG_GET_SET(jumbo); +_INFLAG_GET_SET(vlan); +_INFLAG_GET_SET(vlan_qinq); +_INFLAG_GET_SET(arp); +_INFLAG_GET_SET(ipv4); +_INFLAG_GET_SET(ipv6); +_INFLAG_GET_SET(ipfrag); +_INFLAG_GET_SET(ipopt); +_INFLAG_GET_SET(ipsec); +_INFLAG_GET_SET(udp); +_INFLAG_GET_SET(tcp); +_INFLAG_GET_SET(sctp); +_INFLAG_GET_SET(icmp); + + +static inline int odp_packet_has_error(odp_packet_t pkt) +{ + return (_odp_pkt_hdr(pkt)->error_flags.all != 0); +} + +static inline int odp_packet_errflag_frame_len(odp_packet_t pkt) +{ + return _odp_pkt_hdr(pkt)->error_flags.frame_len; +} + +static inline void odp_packet_override_l4_chksum(odp_packet_t pkt) +{ + _odp_pkt_hdr(pkt)->output_flags.l4_chksum = 1; +} + +#include + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/platform/linux-keystone2/include/odp/plat/packet_types.h b/platform/linux-keystone2/include/odp/plat/packet_types.h new file mode 100644 index 0000000..6f0c97d --- /dev/null +++ b/platform/linux-keystone2/include/odp/plat/packet_types.h @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2014, Linaro Limited + * Copyright (c) 2014, Texas Instruments Incorporated + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP packet types + */ + +#ifndef ODP_PLAT_PACKET_TYPES_H_ +#define ODP_PLAT_PACKET_TYPES_H_ + +#include +#include + +/** + * ODP packet + */ +typedef odp_handle_t odp_packet_t; + +/** Invalid packet */ +#define ODP_PACKET_INVALID ((odp_packet_t)0) + +/** Invalid offset */ +#define ODP_PACKET_OFFSET_INVALID ((uint16_t)-1) + +/** ODP packet segment */ +typedef odp_handle_t odp_packet_seg_t; + +/** Invalid packet segment */ +#define ODP_PACKET_SEG_INVALID ((odp_packet_seg_t)0) + +#endif diff --git a/platform/linux-keystone2/include/odp_packet_internal.h b/platform/linux-keystone2/include/odp_packet_internal.h index cfcd4de..1948b26 100644 --- a/platform/linux-keystone2/include/odp_packet_internal.h +++ b/platform/linux-keystone2/include/odp_packet_internal.h @@ -19,146 +19,22 @@ extern "C" { #endif -#include -#include -#include -#include -#include -#include - -/** - * Packet input & protocol flags - */ -typedef union { - /* All input flags */ - uint32_t all; - - struct { - /* Bitfield flags for each protocol */ - uint32_t l2:1; /**< known L2 protocol present */ - uint32_t l3:1; /**< known L3 protocol present */ - uint32_t l4:1; /**< known L4 protocol present */ - - uint32_t eth:1; /**< Ethernet */ - uint32_t jumbo:1; /**< Jumbo frame */ - uint32_t vlan:1; /**< VLAN hdr found */ - uint32_t vlan_qinq:1; /**< Stacked VLAN found, QinQ */ - - uint32_t arp:1; /**< ARP */ - - uint32_t ipv4:1; /**< IPv4 */ - uint32_t ipv6:1; /**< IPv6 */ - uint32_t ipfrag:1; /**< IP fragment */ - uint32_t ipopt:1; /**< IP optional headers */ - uint32_t ipsec:1; /**< IPSec decryption may be needed */ - - uint32_t udp:1; /**< UDP */ - uint32_t tcp:1; /**< TCP */ - uint32_t sctp:1; /**< SCTP */ - uint32_t icmp:1; /**< ICMP */ - }; -} input_flags_t; - -ODP_STATIC_ASSERT(sizeof(input_flags_t) == sizeof(uint32_t), "INPUT_FLAGS_SIZE_ERROR"); - -/** - * Packet error flags - */ -typedef union { - /* All error flags */ - uint32_t all; - - struct { - /* Bitfield flags for each detected error */ - uint32_t frame_len:1; /**< Frame length error */ - uint32_t l2_chksum:1; /**< L2 checksum error, checks TBD */ - uint32_t ip_err:1; /**< IP error, checks TBD */ - uint32_t tcp_err:1; /**< TCP error, checks TBD */ - uint32_t udp_err:1; /**< UDP error, checks TBD */ - }; -} error_flags_t; - -ODP_STATIC_ASSERT(sizeof(error_flags_t) == sizeof(uint32_t), "ERROR_FLAGS_SIZE_ERROR"); - -/** - * Packet output flags - */ -typedef union { - /* All output flags */ - uint32_t all; - - struct { - /* Bitfield flags for each output option */ - uint32_t l4_chksum:1; /**< Request L4 checksum calculation */ - }; -} output_flags_t; - -ODP_STATIC_ASSERT(sizeof(output_flags_t) == sizeof(uint32_t), "OUTPUT_FLAGS_SIZE_ERROR"); - -/** - * Internal Packet header - */ -struct odp_pkthdr { - /* common buffer header */ - struct odp_bufhdr buf_hdr; - - input_flags_t input_flags; - error_flags_t error_flags; - output_flags_t output_flags; - - uint16_t frame_offset; /**< offset to start of frame, even on error */ - uint16_t l2_offset; /**< offset to L2 hdr, e.g. Eth */ - uint16_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ - uint16_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ - - uint32_t frame_len; - - odp_pktio_t input; - - struct { - int16_t saved_buf_offset; - uint32_t hash_offset; - union { - struct { - } enc; - struct { - uint32_t hash_tag[5]; - } dec; - }; - - } crypto; - -}; - -ODP_STATIC_ASSERT(sizeof(struct odp_pkthdr) <= ODP_CACHE_LINE_SIZE, - "PACKET_HDR_T_SIZE_ERROR"); - -/** - * Return the packet header - */ -static inline struct odp_pkthdr *odp_packet_hdr(odp_packet_t pkt) -{ - odp_buffer_t buf = odp_packet_to_buffer(pkt); - return (struct odp_pkthdr *)odp_buffer_hdr(buf); -} +#include +#include /** * Parse packet and set internal metadata */ -void odp_packet_parse(odp_packet_t pkt, size_t len, size_t l2_offset); +int _odp_packet_parse(odp_packet_t pkt); -static inline void odp_pr_packet(int level, odp_packet_t pkt) -{ - if (level <= ODP_PRINT_LEVEL) - odp_packet_print(pkt); -} +#ifdef ODP_DEBUG_PRINT +#define odp_pr_dbg_packet odp_packet_print +#else +#define odp_pr_dbg_packet +#endif -#define odp_pr_err_packet(...) \ - odp_pr_packet(ODP_PRINT_LEVEL_ERR, ##__VA_ARGS__) -#define odp_pr_dbg_packet(...) \ - odp_pr_packet(ODP_PRINT_LEVEL_DBG, ##__VA_ARGS__) -#define odp_pr_vdbg_packet(...) \ - odp_pr_packet(ODP_PRINT_LEVEL_VDBG, ##__VA_ARGS__) +#define odp_pr_err_packet odp_packet_print +#define odp_pr_vdbg_packet odp_pr_dbg_packet #ifdef __cplusplus } diff --git a/platform/linux-keystone2/odp_packet.c b/platform/linux-keystone2/odp_packet.c index 999966b..b36f444 100644 --- a/platform/linux-keystone2/odp_packet.c +++ b/platform/linux-keystone2/odp_packet.c @@ -6,295 +6,505 @@ * SPDX-License-Identifier: BSD-3-Clause */ -#include +#include #include -#include -#include +#include +#include +#include +#include +#include -#include -#include +#include +#include +#include +#include #include #include -#define ODP_PACKET_HDR_OFFSET_INVALID ((uint16_t)-1) +static Cppi_HostDesc *alloc_buffers(Qmss_QueueHnd queue, uint32_t len) +{ + Cppi_HostDesc *desc, *first_desc = NULL; + uint32_t len_remain = len; + + while (len_remain) { + uint32_t buf_len, data_len; + Cppi_HostDesc *prev_desc; + uintptr_t desc_phys; + uint8_t *buf_addr; + + desc_phys = QMSS_DESC_PTR(Qmss_queuePopRaw(queue)); + desc = _odp_mem_phys_to_virt(desc_phys); + + if (!desc) + goto free_buffers; + + ODP_ASSERT(_cppi_desc_next(desc) == 0, + "Pool should not have linked buffers"); + if (!first_desc) { + /* + * Store a first descriptor to be able to free a chain + * in case of error + */ + first_desc = desc; + } else { + /* Link descriptor to previous one */ + _cppi_desc_vnext_set(prev_desc, desc); + } -static inline uint8_t parse_ipv4(struct odp_pkthdr *pkt_hdr, - odph_ipv4hdr_t *ipv4, - size_t *offset_out); -static inline uint8_t parse_ipv6(struct odp_pkthdr *pkt_hdr, - odph_ipv6hdr_t *ipv6, - size_t *offset_out); + buf_addr = _odp_mem_phys_to_virt(_cppi_desc_orig_ptr(desc)); + buf_len = _cppi_desc_orig_len(desc); + ODP_ASSERT(buf_addr, + "Packet pool should have a valid data buffers"); -void odp_packet_init(odp_packet_t pkt) -{ - struct odp_pkthdr *const pkt_hdr = odp_packet_hdr(pkt); + _cppi_desc_orig_vptr_set(desc, buf_addr); + _cppi_desc_buf_vptr_set(desc, buf_addr); + + data_len = (len_remain < buf_len) ? len_remain : buf_len; + _cppi_desc_buf_len_set(desc, data_len); + len_remain -= data_len; + prev_desc = desc; + } - pkt_hdr->l2_offset = ODP_PACKET_HDR_OFFSET_INVALID; - pkt_hdr->l3_offset = ODP_PACKET_HDR_OFFSET_INVALID; - pkt_hdr->l4_offset = ODP_PACKET_HDR_OFFSET_INVALID; + _cppi_desc_vnext_set(desc, NULL); + return first_desc; + +free_buffers: + desc = first_desc; + while (desc) { + Cppi_HostDesc *next_desc; + _cppi_desc_orig_ptr_set(desc, _odp_mem_virt_to_phys( + _cppi_desc_orig_vptr(desc))); + next_desc = _cppi_desc_vnext(desc); + _cppi_desc_next_set(desc, 0); + + Qmss_queuePushDescSizeRaw(queue, + (void *)_odp_mem_virt_to_phys(desc), + 16); + + desc = next_desc; + } + return NULL; } -void odp_packet_set_len(odp_packet_t pkt, size_t len) +static void packet_header_reset(struct odp_pkthdr *hdr) { - odp_buffer_t buf = odp_packet_to_buffer(pkt); - Pktlib_setPacketLen(_odp_buf_to_ti_pkt(buf), len); - /** - * @todo: Buffer length should be modified by buffer API when it - * become available - */ - _odp_buf_to_cppi_desc(buf)->buffLen = len; + hdr->l2_offset = ODP_PACKET_OFFSET_INVALID; + hdr->l3_offset = ODP_PACKET_OFFSET_INVALID; + hdr->l4_offset = ODP_PACKET_OFFSET_INVALID; + hdr->input = ODP_PKTIO_INVALID; + hdr->input_flags.all = 0; } -size_t odp_packet_get_len(odp_packet_t pkt) +static void packet_header_copy(struct odp_pkthdr *hdr_to, + struct odp_pkthdr *hdr_from) { - odp_buffer_t buf = odp_packet_to_buffer(pkt); - return Pktlib_getPacketLen(_odp_buf_to_ti_pkt(buf)); + memcpy(hdr_to, hdr_from, sizeof(struct odp_pkthdr)); } -uint8_t *odp_packet_addr(odp_packet_t pkt) +odp_packet_t odp_packet_alloc(odp_pool_t pool, uint32_t len) { - return odp_buffer_addr(odp_packet_to_buffer(pkt)); + Cppi_HostDesc *desc; + pool_entry_t *pool_entry = _odp_pool_entry(pool); + uint32_t headroom = pool_entry->headroom; + + desc = alloc_buffers(pool_entry->free_queue, len + headroom); + if (!desc) + return ODP_PACKET_INVALID; + + ODP_ASSERT(_cppi_desc_buf_len(desc) > headroom, + "Not enough space for headroom"); + _cppi_desc_buf_len_set(desc, + _cppi_desc_buf_len(desc) - headroom); + _cppi_desc_buf_vptr_set(desc, + _cppi_desc_buf_vptr(desc) + headroom); + _cppi_desc_pkt_len_set(desc, len); + + odp_packet_t pkt = _cppi_desc_to_odp_pkt(desc); + packet_header_reset(_odp_pkt_hdr(pkt)); + + return pkt; } -uint8_t *odp_packet_data(odp_packet_t pkt) +void odp_packet_free(odp_packet_t pkt) { - return odp_packet_addr(pkt) + odp_packet_hdr(pkt)->frame_offset; -} + Cppi_HostDesc *desc; + uint32_t pool_id; + pool_entry_t *pool_entry; + odp_pr_vdbg("pkt: %p\n", pkt); + desc = _odp_pkt_to_cppi_desc(pkt); -uint8_t *odp_packet_l2(odp_packet_t pkt) -{ - const size_t offset = odp_packet_l2_offset(pkt); + pool_id = _cppi_desc_pool_id(desc); + pool_entry = _odp_pool_id_to_entry(pool_id); - if (odp_unlikely(offset == ODP_PACKET_HDR_OFFSET_INVALID)) - return NULL; + while (desc) { + Cppi_HostDesc *next_desc; + _cppi_desc_orig_ptr_set(desc, _odp_mem_virt_to_phys( + _cppi_desc_orig_vptr(desc))); + next_desc = _cppi_desc_vnext(desc); + _cppi_desc_next_set(desc, 0); - return odp_packet_addr(pkt) + offset; + Qmss_queuePushDescSizeRaw(pool_entry->free_queue, + (void *)_odp_mem_virt_to_phys(desc), + 16); + + desc = next_desc; + } } -size_t odp_packet_l2_offset(odp_packet_t pkt) +static int reset_buffers(Cppi_HostDesc *desc, uint32_t len) { - return odp_packet_hdr(pkt)->l2_offset; + uint32_t len_remain = len; + + if (len > _cppi_desc_orig_len_total(desc)) + return -1; + + while (len_remain) { + ODP_ASSERT(desc, "Not enough buffers in a chain"); + uint32_t buf_len = _cppi_desc_orig_len(desc); + uint8_t *buf_addr = _cppi_desc_orig_vptr(desc); + uint32_t data_len; + + ODP_ASSERT(buf_addr, + "Packet pool should have a valid data buffers"); + + _cppi_desc_buf_vptr_set(desc, buf_addr); + + data_len = (len_remain < buf_len) ? len_remain : buf_len; + _cppi_desc_buf_len_set(desc, data_len); + len_remain -= data_len; + desc = _cppi_desc_vnext(desc); + } + + return 0; } -void odp_packet_set_l2_offset(odp_packet_t pkt, size_t offset) +int odp_packet_reset(odp_packet_t pkt, uint32_t len) { - odp_packet_hdr(pkt)->l2_offset = offset; + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint32_t pool_id = _cppi_desc_pool_id(desc); + pool_entry_t *pool_entry = _odp_pool_id_to_entry(pool_id); + uint32_t headroom = pool_entry->headroom; + + if (reset_buffers(desc, len + headroom)) + return -1; + + ODP_ASSERT(_cppi_desc_buf_len(desc) > headroom, + "Not enough space for headroom"); + _cppi_desc_buf_len_set(desc, + _cppi_desc_buf_len(desc) - headroom); + _cppi_desc_buf_vptr_set(desc, + _cppi_desc_buf_vptr(desc) + headroom); + _cppi_desc_pkt_len_set(desc, len); + + packet_header_reset(_odp_pkt_hdr(pkt)); + + return 0; } -uint8_t *odp_packet_l3(odp_packet_t pkt) +void *odp_packet_push_head(odp_packet_t pkt, uint32_t len) { - const size_t offset = odp_packet_l3_offset(pkt); + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint8_t *data_addr = odp_packet_data(pkt); + uint8_t *head_addr = odp_packet_head(pkt); - if (odp_unlikely(offset == ODP_PACKET_HDR_OFFSET_INVALID)) + ODP_ASSERT(data_addr >= head_addr, + "Data address should be within a buffer"); + + data_addr -= len; + if (data_addr < head_addr) return NULL; - return odp_packet_addr(pkt) + offset; -} + _cppi_desc_buf_vptr_set(desc, data_addr); + _cppi_desc_buf_len_set(desc, _cppi_desc_buf_len(desc) + len); + _cppi_desc_pkt_len_set(desc, _cppi_desc_pkt_len(desc) + len); -size_t odp_packet_l3_offset(odp_packet_t pkt) -{ - return odp_packet_hdr(pkt)->l3_offset; + return data_addr; } -void odp_packet_set_l3_offset(odp_packet_t pkt, size_t offset) +void *odp_packet_pull_head(odp_packet_t pkt, uint32_t len) { - odp_packet_hdr(pkt)->l3_offset = offset; + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint32_t data_len = _cppi_desc_buf_len(desc); + uint8_t *data_addr = odp_packet_data(pkt); + + if (len >= data_len) + return NULL; + + data_addr += len; + _cppi_desc_buf_vptr_set(desc, data_addr); + _cppi_desc_buf_len_set(desc, data_len - len); + _cppi_desc_pkt_len_set(desc, _cppi_desc_pkt_len(desc) - len); + + return data_addr; } -uint8_t *odp_packet_l4(odp_packet_t pkt) +void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len) { - const size_t offset = odp_packet_l4_offset(pkt); + odp_packet_seg_t seg = odp_packet_last_seg(pkt); + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *last_desc = _odp_pktseg_to_cppi_desc(seg); + uint8_t *tail_addr = _cppi_desc_buf_end(last_desc); + uint8_t *seg_end = _cppi_desc_orig_end(last_desc); + + ODP_ASSERT(tail_addr < seg_end, + "Tail address should be within a buffer"); - if (odp_unlikely(offset == ODP_PACKET_HDR_OFFSET_INVALID)) + if (tail_addr + len >= seg_end) return NULL; - return odp_packet_addr(pkt) + offset; + _cppi_desc_buf_len_set(last_desc, _cppi_desc_buf_len(last_desc) + len); + _cppi_desc_pkt_len_set(desc, _cppi_desc_pkt_len(desc) + len); + + return tail_addr; } -size_t odp_packet_l4_offset(odp_packet_t pkt) +void *odp_packet_pull_tail(odp_packet_t pkt, uint32_t len) { - return odp_packet_hdr(pkt)->l4_offset; + odp_packet_seg_t seg = odp_packet_last_seg(pkt); + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *last_desc = _odp_pktseg_to_cppi_desc(seg); + uint32_t buf_len = _cppi_desc_buf_len(last_desc); + + if (len >= buf_len) + return NULL; + + buf_len -= len; + _cppi_desc_buf_len_set(last_desc, buf_len); + _cppi_desc_pkt_len_set(desc, _cppi_desc_pkt_len(desc) - len); + + return _cppi_desc_buf_vptr(last_desc) + buf_len; } -void odp_packet_set_l4_offset(odp_packet_t pkt, size_t offset) +void *odp_packet_offset(odp_packet_t pkt, uint32_t offset, uint32_t *len, + odp_packet_seg_t *seg) { - odp_packet_hdr(pkt)->l4_offset = offset; + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + + if (_cppi_desc_pkt_len(desc) <= offset) + return NULL; + + desc = _cppi_desc_offset(desc, &offset); + + if (len) + *len = _cppi_desc_buf_len(desc) - offset; + if (seg) + *seg = _cppi_desc_to_odp_pktseg(desc); + return _cppi_desc_buf_vptr(desc) + offset; } -/** - * Simple packet parser: eth, VLAN, IP, TCP/UDP/ICMP - * - * Internal function: caller is responsible for passing only - * valid packet handles, lengths and offsets - * (usually done&called in packet input). - * - * @param pkt Packet handle - * @param len Packet length in bytes - * @param frame_offset Byte offset to L2 header - */ -void odp_packet_parse(odp_packet_t pkt, size_t len, size_t frame_offset) + +void *odp_packet_l2_ptr(odp_packet_t pkt, uint32_t *len) { - struct odp_pkthdr *const pkt_hdr = odp_packet_hdr(pkt); - odph_ethhdr_t *eth; - odph_vlanhdr_t *vlan; - odph_ipv4hdr_t *ipv4; - odph_ipv6hdr_t *ipv6; - uint16_t ethtype; - size_t offset = 0; - uint8_t ip_proto = 0; + const size_t offset = odp_packet_l2_offset(pkt); - pkt_hdr->input_flags.eth = 1; - pkt_hdr->frame_offset = frame_offset; + if (odp_unlikely(offset == ODP_PACKET_OFFSET_INVALID)) + return NULL; - if (odp_unlikely(len < ODPH_ETH_LEN_MIN)) { - pkt_hdr->error_flags.frame_len = 1; - return; - } else if (len > ODPH_ETH_LEN_MAX) { - pkt_hdr->input_flags.jumbo = 1; - } + return odp_packet_offset(pkt, offset, len, NULL); +} - len -= 4; /* Crop L2 CRC */ - odp_packet_set_len(pkt, len); +void *odp_packet_l3_ptr(odp_packet_t pkt, uint32_t *len) +{ + const size_t offset = odp_packet_l3_offset(pkt); - /* Assume valid L2 header, no CRC/FCS check in SW */ - pkt_hdr->input_flags.l2 = 1; - pkt_hdr->l2_offset = frame_offset; + if (odp_unlikely(offset == ODP_PACKET_OFFSET_INVALID)) + return NULL; - eth = (odph_ethhdr_t *)odp_packet_data(pkt); - ethtype = odp_be_to_cpu_16(eth->type); - vlan = (odph_vlanhdr_t *)ð->type; + return odp_packet_offset(pkt, offset, len, NULL); +} - if (ethtype == ODPH_ETHTYPE_VLAN_OUTER) { - pkt_hdr->input_flags.vlan_qinq = 1; - ethtype = odp_be_to_cpu_16(vlan->tpid); - offset += sizeof(odph_vlanhdr_t); - vlan = &vlan[1]; - } +void *odp_packet_l4_ptr(odp_packet_t pkt, uint32_t *len) +{ + const size_t offset = odp_packet_l4_offset(pkt); - if (ethtype == ODPH_ETHTYPE_VLAN) { - pkt_hdr->input_flags.vlan = 1; - ethtype = odp_be_to_cpu_16(vlan->tpid); - offset += sizeof(odph_vlanhdr_t); - } + if (odp_unlikely(offset == ODP_PACKET_OFFSET_INVALID)) + return NULL; - /* Set l3_offset+flag only for known ethtypes */ - switch (ethtype) { - case ODPH_ETHTYPE_IPV4: - pkt_hdr->input_flags.ipv4 = 1; - pkt_hdr->input_flags.l3 = 1; - pkt_hdr->l3_offset = frame_offset + ODPH_ETHHDR_LEN + offset; - ipv4 = (odph_ipv4hdr_t *)odp_packet_l3(pkt); - ip_proto = parse_ipv4(pkt_hdr, ipv4, &offset); - break; - case ODPH_ETHTYPE_IPV6: - pkt_hdr->input_flags.ipv6 = 1; - pkt_hdr->input_flags.l3 = 1; - pkt_hdr->l3_offset = frame_offset + ODPH_ETHHDR_LEN + offset; - ipv6 = (odph_ipv6hdr_t *)odp_packet_l3(pkt); - ip_proto = parse_ipv6(pkt_hdr, ipv6, &offset); - break; - case ODPH_ETHTYPE_ARP: - pkt_hdr->input_flags.arp = 1; - /* fall through */ - default: - ip_proto = 0; - break; - } + return odp_packet_offset(pkt, offset, len, NULL); +} - switch (ip_proto) { - case ODPH_IPPROTO_UDP: - pkt_hdr->input_flags.udp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_TCP: - pkt_hdr->input_flags.tcp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_SCTP: - pkt_hdr->input_flags.sctp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_ICMP: - pkt_hdr->input_flags.icmp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - default: - /* 0 or unhandled IP protocols, don't set L4 flag+offset */ - if (pkt_hdr->input_flags.ipv6) { - /* IPv6 next_hdr is not L4, mark as IP-option instead */ - pkt_hdr->input_flags.ipopt = 1; +static int packet_data_copy(Cppi_HostDesc *desc_in, uint32_t offset_in, + Cppi_HostDesc *desc_out, uint32_t offset_out, + uint32_t len) +{ + desc_in = _cppi_desc_offset(desc_in, &offset_in); + desc_out = _cppi_desc_offset(desc_out, &offset_out); + + uint32_t len_in = _cppi_desc_buf_len(desc_in) - offset_in; + uint8_t *ptr_in = _cppi_desc_buf_vptr(desc_in) + offset_in; + uint32_t len_out = _cppi_desc_buf_len(desc_out) - offset_out; + uint8_t *ptr_out = _cppi_desc_buf_vptr(desc_out) + offset_out; + uint32_t copy_len = (len_in < len_out) ? len_in : len_out; + + while (copy_len < len) { + ODP_ASSERT(copy_len, "Zero data copy"); + memcpy(ptr_in, ptr_out, copy_len); + len -= copy_len; + + if (len_in < len_out) { + ptr_out += copy_len; + len_out -= copy_len; + desc_in = _cppi_desc_vnext(desc_in); + ODP_ASSERT(desc_in, "Not enough descriptors"); + len_in = _cppi_desc_buf_len(desc_in); + ptr_in = _cppi_desc_buf_vptr(desc_in); + } else { + ptr_in += copy_len; + len_in -= copy_len; + desc_out = _cppi_desc_vnext(desc_out); + ODP_ASSERT(desc_out, "Not enough descriptors"); + len_out = _cppi_desc_buf_len(desc_out); + ptr_out = _cppi_desc_buf_vptr(desc_out); } - break; + copy_len = (len_in < len_out) ? len_in : len_out; } + + memcpy(ptr_in, ptr_out, len); + return 0; } -static inline uint8_t parse_ipv4(struct odp_pkthdr *pkt_hdr, - odph_ipv4hdr_t *ipv4, - size_t *offset_out) +odp_packet_t odp_packet_copy(odp_packet_t pkt, odp_pool_t pool) { - uint8_t ihl; - uint16_t frag_offset; - - ihl = ODPH_IPV4HDR_IHL(ipv4->ver_ihl); - if (odp_unlikely(ihl < ODPH_IPV4HDR_IHL_MIN)) { - pkt_hdr->error_flags.ip_err = 1; - return 0; - } + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *new_desc; + uint32_t pkt_len = _cppi_desc_pkt_len(desc); + odp_packet_t new_pkt; + + new_pkt = odp_packet_alloc(pool, pkt_len); + if (new_pkt == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + ODP_ASSERT(odp_packet_len(new_pkt) == pkt_len, "Wrong packet length"); + + new_desc = _odp_pkt_to_cppi_desc(new_pkt); + packet_data_copy(new_desc, 0, desc, 0, pkt_len); + packet_header_copy(_odp_pkt_hdr(new_pkt), _odp_pkt_hdr(pkt)); + return new_pkt; +} - if (odp_unlikely(ihl > ODPH_IPV4HDR_IHL_MIN)) { - pkt_hdr->input_flags.ipopt = 1; - return 0; +int odp_packet_copydata_out(odp_packet_t pkt, uint32_t offset, + uint32_t len, void *dst) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint32_t seg_len; + uint8_t *src; + + if (_cppi_desc_pkt_len(desc) < offset + len) + return -1; + + desc = _cppi_desc_offset(desc, &offset); + src = _cppi_desc_buf_vptr(desc) + offset; + seg_len = _cppi_desc_buf_len(desc) - offset; + + /* Copy full segments */ + while (len > seg_len) { + memcpy(dst, src, seg_len); + len -= seg_len; + dst = (uint8_t *)dst + seg_len; + + desc = _cppi_desc_vnext(desc); + ODP_ASSERT(desc, "Not enough descriptors"); + seg_len = _cppi_desc_buf_len(desc); + src = _cppi_desc_buf_vptr(desc); } - /* A packet is a fragment if: - * "more fragments" flag is set (all fragments except the last) - * OR - * "fragment offset" field is nonzero (all fragments except the first) - */ - frag_offset = odp_be_to_cpu_16(ipv4->frag_offset); - if (odp_unlikely(ODPH_IPV4HDR_IS_FRAGMENT(frag_offset))) { - pkt_hdr->input_flags.ipfrag = 1; - return 0; - } + /* Copy a tail segment */ + memcpy(dst, src, seg_len); + return 0; +} - if (ipv4->proto == ODPH_IPPROTO_ESP || - ipv4->proto == ODPH_IPPROTO_AH) { - pkt_hdr->input_flags.ipsec = 1; - return 0; +int odp_packet_copydata_in(odp_packet_t pkt, uint32_t offset, + uint32_t len, const void *src) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + uint32_t seg_len; + uint8_t *dst; + + if (_cppi_desc_pkt_len(desc) < offset + len) + return -1; + + desc = _cppi_desc_offset(desc, &offset); + dst = _cppi_desc_buf_vptr(desc) + offset; + seg_len = _cppi_desc_buf_len(desc) - offset; + + /* Copy full segments */ + while (len > seg_len) { + memcpy(dst, src, seg_len); + len -= seg_len; + src = (const uint8_t *)src + seg_len; + + desc = _cppi_desc_vnext(desc); + ODP_ASSERT(desc, "Not enough descriptors"); + seg_len = _cppi_desc_buf_len(desc); + dst = _cppi_desc_buf_vptr(desc); } - /* Set pkt_hdr->input_flags.ipopt when checking L4 hdrs after return */ - - *offset_out = sizeof(uint32_t) * ihl; - return ipv4->proto; + /* Copy a tail segment */ + memcpy(dst, src, seg_len); + return 0; } -static inline uint8_t parse_ipv6(struct odp_pkthdr *pkt_hdr, - odph_ipv6hdr_t *ipv6, - size_t *offset_out) +/** + * @todo: This is a dumb straightforward implementation. It should be + * optimized if this API will start to be used by applications. + */ +odp_packet_t odp_packet_add_data(odp_packet_t pkt, uint32_t offset, + uint32_t len) { - if (ipv6->next_hdr == ODPH_IPPROTO_ESP || - ipv6->next_hdr == ODPH_IPPROTO_AH) { - pkt_hdr->input_flags.ipopt = 1; - pkt_hdr->input_flags.ipsec = 1; - return 0; - } - - if (odp_unlikely(ipv6->next_hdr == ODPH_IPPROTO_FRAG)) { - pkt_hdr->input_flags.ipopt = 1; - pkt_hdr->input_flags.ipfrag = 1; - return 0; - } + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *new_desc; + uint32_t pkt_len = _cppi_desc_pkt_len(desc); + odp_packet_t new_pkt; + + if (offset > pkt_len) + return ODP_PACKET_INVALID; + + new_pkt = odp_packet_alloc(odp_packet_pool(pkt), pkt_len + len); + if (new_pkt == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + ODP_ASSERT(odp_packet_len(new_pkt) == pkt_len + len, + "Wrong packet length"); + + new_desc = _odp_pkt_to_cppi_desc(new_pkt); + packet_data_copy(new_desc, 0, desc, 0, offset); + packet_data_copy(new_desc, offset + len, + desc, offset, + pkt_len - offset); + packet_header_copy(_odp_pkt_hdr(new_pkt), _odp_pkt_hdr(pkt)); + odp_packet_free(pkt); + return new_pkt; +} - /* Don't step through more extensions */ - *offset_out = ODPH_IPV6HDR_LEN; - return ipv6->next_hdr; +/** + * @todo: This is a dumb straightforward implementation. It should be + * optimized if this API will start to be used by applications. + */ +odp_packet_t odp_packet_rem_data(odp_packet_t pkt, uint32_t offset, + uint32_t len) +{ + Cppi_HostDesc *desc = _odp_pkt_to_cppi_desc(pkt); + Cppi_HostDesc *new_desc; + uint32_t pkt_len = _cppi_desc_pkt_len(desc); + odp_packet_t new_pkt; + + if (offset + len > pkt_len) + return ODP_PACKET_INVALID; + + new_pkt = odp_packet_alloc(odp_packet_pool(pkt), pkt_len - len); + if (new_pkt == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + ODP_ASSERT(odp_packet_len(new_pkt) == pkt_len - len, + "Wrong packet length"); + + new_desc = _odp_pkt_to_cppi_desc(new_pkt); + packet_data_copy(new_desc, 0, desc, 0, offset); + packet_data_copy(new_desc, offset, + desc, offset + len, + pkt_len - offset - len); + packet_header_copy(_odp_pkt_hdr(new_pkt), _odp_pkt_hdr(pkt)); + odp_packet_free(pkt); + return new_pkt; } void odp_packet_print(odp_packet_t pkt) @@ -304,10 +514,10 @@ void odp_packet_print(odp_packet_t pkt) int len = 0; int n = max_len-1; Cppi_HostDesc *desc; - struct odp_pkthdr *hdr = odp_packet_hdr(pkt); - odp_buffer_t buf = odp_packet_to_buffer(pkt); + struct odp_pkthdr *hdr = _odp_pkt_hdr(pkt); len += snprintf(&str[len], n-len, "Packet "); + /** @todo: avoid odp_buffer_snprintf here */ len += odp_buffer_snprint(&str[len], n-len, (odp_buffer_t) pkt); len += snprintf(&str[len], n-len, " input_flags 0x%x\n", hdr->input_flags.all); @@ -316,30 +526,292 @@ void odp_packet_print(odp_packet_t pkt) len += snprintf(&str[len], n-len, " output_flags 0x%x\n", hdr->output_flags.all); len += snprintf(&str[len], n-len, - " frame_offset %u\n", hdr->frame_offset); - len += snprintf(&str[len], n-len, " l2_offset %u\n", hdr->l2_offset); len += snprintf(&str[len], n-len, " l3_offset %u\n", hdr->l3_offset); len += snprintf(&str[len], n-len, " l4_offset %u\n", hdr->l4_offset); len += snprintf(&str[len], n-len, - " packet len %u\n", odp_packet_get_len(pkt)); + " packet len %u\n", odp_packet_len(pkt)); len += snprintf(&str[len], n-len, - " input %u\n", hdr->input); + " input %p\n", hdr->input); str[len] = '\0'; printf("\n%s\n", str); - desc = _odp_buf_to_cppi_desc(buf); - odp_print_mem(desc, sizeof(*desc), "Descriptor dump"); + desc = _odp_pkt_to_cppi_desc(pkt); + odp_print_mem(desc, NWAL_DESC_SIZE, "Descriptor dump"); odp_print_mem((void *)desc->origBuffPtr, desc->buffPtr - desc->origBuffPtr + 128, "Buffer start"); } -int odp_packet_copy(odp_packet_t pkt_dst, odp_packet_t pkt_src) +int odp_packet_is_valid(odp_packet_t pkt) +{ + return pkt != ODP_PACKET_INVALID; +} + +/** + * Parser helper function for IPv4 + */ +static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset, + uint32_t pkt_len) +{ + odph_ipv4hdr_t *ipv4 = (odph_ipv4hdr_t *)*parseptr; + uint8_t ver = ODPH_IPV4HDR_VER(ipv4->ver_ihl); + uint8_t ihl = ODPH_IPV4HDR_IHL(ipv4->ver_ihl); + uint16_t frag_offset; + uint32_t l3_len = odp_be_to_cpu_16(ipv4->tot_len); + + if (odp_unlikely(ihl < ODPH_IPV4HDR_IHL_MIN) || + odp_unlikely(ver != 4) || + (l3_len > pkt_len - *offset)) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + *offset += ihl * 4; + *parseptr += ihl * 4; + + if (odp_unlikely(ihl > ODPH_IPV4HDR_IHL_MIN)) + pkt_hdr->input_flags.ipopt = 1; + + /* A packet is a fragment if: + * "more fragments" flag is set (all fragments except the last) + * OR + * "fragment offset" field is nonzero (all fragments except the first) + */ + frag_offset = odp_be_to_cpu_16(ipv4->frag_offset); + if (odp_unlikely(ODPH_IPV4HDR_IS_FRAGMENT(frag_offset))) + pkt_hdr->input_flags.ipfrag = 1; + + return ipv4->proto; +} + +/** + * Parser helper function for IPv6 + */ +static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset, + uint32_t pkt_len) +{ + odph_ipv6hdr_t *ipv6 = (odph_ipv6hdr_t *)*parseptr; + odph_ipv6hdr_ext_t *ipv6ext; + + uint32_t l3_len = odp_be_to_cpu_16(ipv6->payload_len); + + /* Basic sanity checks on IPv6 header */ + if ((ipv6->ver_tc_flow >> 28) != 6 || + l3_len > pkt_len - *offset) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + /* Skip past IPv6 header */ + *offset += sizeof(odph_ipv6hdr_t); + *parseptr += sizeof(odph_ipv6hdr_t); + + + /* Skip past any IPv6 extension headers */ + if (ipv6->next_hdr == ODPH_IPPROTO_HOPOPTS || + ipv6->next_hdr == ODPH_IPPROTO_ROUTE) { + pkt_hdr->input_flags.ipopt = 1; + + do { + ipv6ext = (odph_ipv6hdr_ext_t *)*parseptr; + uint16_t extlen = 8 + ipv6ext->ext_len * 8; + + *offset += extlen; + *parseptr += extlen; + } while ((ipv6ext->next_hdr == ODPH_IPPROTO_HOPOPTS || + ipv6ext->next_hdr == ODPH_IPPROTO_ROUTE) && + *offset < pkt_len); + + if (*offset >= pkt_hdr->l3_offset + ipv6->payload_len) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + if (ipv6ext->next_hdr == ODPH_IPPROTO_FRAG) + pkt_hdr->input_flags.ipfrag = 1; + + return ipv6ext->next_hdr; + } + + if (odp_unlikely(ipv6->next_hdr == ODPH_IPPROTO_FRAG)) { + pkt_hdr->input_flags.ipopt = 1; + pkt_hdr->input_flags.ipfrag = 1; + } + + return ipv6->next_hdr; +} + +/** + * Parser helper function for TCP + */ +static inline void parse_tcp(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_tcphdr_t *tcp = (odph_tcphdr_t *)*parseptr; + + if (tcp->hl < sizeof(odph_tcphdr_t)/sizeof(uint32_t)) + pkt_hdr->error_flags.tcp_err = 1; + else if ((uint32_t)tcp->hl * 4 > sizeof(odph_tcphdr_t)) + pkt_hdr->input_flags.tcpopt = 1; + + *offset += (uint32_t)tcp->hl * 4; + *parseptr += (uint32_t)tcp->hl * 4; +} + +/** + * Parser helper function for UDP + */ +static inline void parse_udp(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_udphdr_t *udp = (odph_udphdr_t *)*parseptr; + uint32_t udplen = odp_be_to_cpu_16(udp->length); + + if (udplen < sizeof(odph_udphdr_t)) + pkt_hdr->error_flags.udp_err = 1; + + *offset += sizeof(odph_udphdr_t); + *parseptr += sizeof(odph_udphdr_t); +} + +/** + * Simple packet parser + */ + +int _odp_packet_parse(odp_packet_t pkt) { - (void) pkt_dst; - (void) pkt_src; - return -1; + odp_packet_hdr_t *const pkt_hdr = _odp_pkt_hdr(pkt); + odph_ethhdr_t *eth; + uint16_t ethtype; + uint8_t *parseptr; + uint32_t offset; + uint8_t ip_proto = 0; + uint32_t pkt_len = odp_packet_len(pkt); + + /* Reset parser metadata for new parse */ + pkt_hdr->error_flags.all = 0; + pkt_hdr->input_flags.all = 0; + pkt_hdr->output_flags.all = 0; + pkt_hdr->l2_offset = 0; + pkt_hdr->l3_offset = ODP_PACKET_OFFSET_INVALID; + pkt_hdr->l4_offset = ODP_PACKET_OFFSET_INVALID; + + /* We only support Ethernet for now */ + pkt_hdr->input_flags.eth = 1; + + /* Detect jumbo frames */ + if (pkt_len > ODPH_ETH_LEN_MAX) + pkt_hdr->input_flags.jumbo = 1; + + /* Assume valid L2 header, no CRC/FCS check in SW */ + pkt_hdr->input_flags.l2 = 1; + + eth = odp_packet_data(pkt); + offset = sizeof(odph_ethhdr_t); + parseptr = (uint8_t *)ð->type; + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + + /* Parse the VLAN header(s), if present */ + if (ethtype == ODPH_ETHTYPE_VLAN_OUTER) { + pkt_hdr->input_flags.vlan_qinq = 1; + pkt_hdr->input_flags.vlan = 1; + offset += sizeof(odph_vlanhdr_t); + parseptr += sizeof(odph_vlanhdr_t); + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + if (ethtype == ODPH_ETHTYPE_VLAN) { + pkt_hdr->input_flags.vlan = 1; + offset += sizeof(odph_vlanhdr_t); + parseptr += sizeof(odph_vlanhdr_t); + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + /* Check for SNAP vs. DIX */ + if (ethtype < ODPH_ETH_LEN_MAX) { + pkt_hdr->input_flags.snap = 1; + if (ethtype > pkt_len - offset) { + pkt_hdr->error_flags.snap_len = 1; + goto parse_exit; + } + offset += 8; + parseptr += 8; + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + /* Consume Ethertype for Layer 3 parse */ + parseptr += 2; + + /* Set l3_offset+flag only for known ethtypes */ + pkt_hdr->input_flags.l3 = 1; + pkt_hdr->l3_offset = offset; + + /* Parse Layer 3 headers */ + switch (ethtype) { + case ODPH_ETHTYPE_IPV4: + pkt_hdr->input_flags.ipv4 = 1; + ip_proto = parse_ipv4(pkt_hdr, &parseptr, &offset, pkt_len); + break; + + case ODPH_ETHTYPE_IPV6: + pkt_hdr->input_flags.ipv6 = 1; + ip_proto = parse_ipv6(pkt_hdr, &parseptr, &offset, pkt_len); + break; + + case ODPH_ETHTYPE_ARP: + pkt_hdr->input_flags.arp = 1; + ip_proto = 255; /* Reserved invalid by IANA */ + break; + + default: + pkt_hdr->input_flags.l3 = 0; + pkt_hdr->l3_offset = ODP_PACKET_OFFSET_INVALID; + ip_proto = 255; /* Reserved invalid by IANA */ + } + + /* Set l4_offset+flag only for known ip_proto */ + pkt_hdr->input_flags.l4 = 1; + pkt_hdr->l4_offset = offset; + + /* Parse Layer 4 headers */ + switch (ip_proto) { + case ODPH_IPPROTO_ICMP: + pkt_hdr->input_flags.icmp = 1; + break; + + case ODPH_IPPROTO_TCP: + pkt_hdr->input_flags.tcp = 1; + parse_tcp(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_IPPROTO_UDP: + pkt_hdr->input_flags.udp = 1; + parse_udp(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_IPPROTO_AH: + case ODPH_IPPROTO_ESP: + pkt_hdr->input_flags.ipsec = 1; + break; + + default: + pkt_hdr->input_flags.l4 = 0; + pkt_hdr->l4_offset = ODP_PACKET_OFFSET_INVALID; + break; + } + + /* + * Anything beyond what we parse here is considered payload. + * Note: Payload is really only relevant for TCP and UDP. For + * all other protocols, the payload offset will point to the + * final header (ARP, ICMP, AH, ESP, or IP Fragment). + */ + +parse_exit: + return pkt_hdr->error_flags.all != 0; } diff --git a/platform/linux-keystone2/odp_pool.c b/platform/linux-keystone2/odp_pool.c index 93fac4b..4bf0ec0 100644 --- a/platform/linux-keystone2/odp_pool.c +++ b/platform/linux-keystone2/odp_pool.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -252,6 +253,37 @@ static int _odp_buffer_pool_init(pool_entry_t *entry) return 0; } +static int _odp_packet_pool_init(pool_entry_t *entry) +{ + uint32_t buf_size = entry->orig_params.pkt.len; + + ODP_ASSERT(entry->orig_params.pkt.len >= entry->orig_params.pkt.seg_len, + "Packet length should be >= segment length"); + + if (buf_size == 0) + buf_size = ODP_CONFIG_PACKET_SEG_LEN_MIN; + else if (buf_size > ODP_CONFIG_PACKET_SEG_LEN_MAX) + return -1; + + entry->headroom = _ODP_PKTHDR_SIZE + ODP_CONFIG_PACKET_HEADROOM; + entry->tailroom = 0; + + buf_size += entry->headroom; + buf_size = ODP_CACHE_LINE_SIZE_ROUNDUP(buf_size); + + entry->buf_size = buf_size; + entry->buf_align = ODP_CACHE_LINE_SIZE; + + if (_attach_buffers(entry->free_queue, + buf_size, + ODP_CACHE_LINE_SIZE, + entry->orig_params.type, + entry->id)) { + return -1; + } + + return 0; +} odp_pool_t odp_pool_create(const char *name, odp_shm_t shm, odp_pool_param_t *params) @@ -309,7 +341,7 @@ odp_pool_t odp_pool_create(const char *name, ret = _odp_buffer_pool_init(entry); break; case ODP_POOL_PACKET: - ret = -1; + ret = _odp_packet_pool_init(entry); break; case ODP_POOL_TIMEOUT: ret = -1;