From patchwork Tue Dec 16 12:30:39 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taras Kondratiuk X-Patchwork-Id: 42329 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f69.google.com (mail-wg0-f69.google.com [74.125.82.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 895F12456A for ; Tue, 16 Dec 2014 12:32:03 +0000 (UTC) Received: by mail-wg0-f69.google.com with SMTP id x12sf8545264wgg.8 for ; Tue, 16 Dec 2014 04:32:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:subject:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=Aj7E9U7zDIqCEfKWAwsj3xD0de67tfZcca/hjYWxoGw=; b=XwrA1EnGgSduu5dRkoUWO8CzbmTz/nMffhinLMxbSXCHqAja6DjOdrOMPUOkq8bG9s zXNDU9efBfOEFuTIwQwY59b1HaCShON2uz1Y1HkzJtq6li313Se83I6iYKMANsAjO3i9 I0aBCkMn1S7XyR2svjIDgz7oPf1VHKk5A2I+IXYJgo6ri1RsSv3ikCrH+/3oLSpDPbHO dxTjHzBYXo2J/+ldpEEIY2foPopceIMYb/H/rXRkWuN5ygTdBgnNhSRpCyIqddwnW28p Op3NksZOyF4Rw7+TouGjz8chjv7Da89u2PEFy/N0mj6ybbMmcM/A3fBV8h8RTPVXbf4o lHTw== X-Gm-Message-State: ALoCoQlcCSUs7i3x/6h5odbsuFWtVMarGi46H+75RMbKRHljifkR97k6IY6N3+E0LfhjLXShmHRo X-Received: by 10.152.10.233 with SMTP id l9mr5535162lab.0.1418733122777; Tue, 16 Dec 2014 04:32:02 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.2.165 with SMTP id 5ls924098lav.42.gmail; Tue, 16 Dec 2014 04:32:02 -0800 (PST) X-Received: by 10.112.138.9 with SMTP id qm9mr35610544lbb.62.1418733122588; Tue, 16 Dec 2014 04:32:02 -0800 (PST) Received: from mail-lb0-f169.google.com (mail-lb0-f169.google.com. [209.85.217.169]) by mx.google.com with ESMTPS id a10si611946laf.104.2014.12.16.04.32.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Dec 2014 04:32:02 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.169 as permitted sender) client-ip=209.85.217.169; Received: by mail-lb0-f169.google.com with SMTP id p9so10971125lbv.28 for ; Tue, 16 Dec 2014 04:32:02 -0800 (PST) X-Received: by 10.112.135.229 with SMTP id pv5mr35425752lbb.52.1418733122228; Tue, 16 Dec 2014 04:32:02 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.142.69 with SMTP id ru5csp1020079lbb; Tue, 16 Dec 2014 04:32:00 -0800 (PST) X-Received: by 10.224.67.132 with SMTP id r4mr1559360qai.1.1418733114930; Tue, 16 Dec 2014 04:31:54 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id a92si593687qge.117.2014.12.16.04.31.53 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 16 Dec 2014 04:31:54 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y0rI8-0006cw-7y; Tue, 16 Dec 2014 12:31:52 +0000 Received: from mail-lb0-f175.google.com ([209.85.217.175]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y0rHO-0006Ug-GS for lng-odp@lists.linaro.org; Tue, 16 Dec 2014 12:31:06 +0000 Received: by mail-lb0-f175.google.com with SMTP id u10so11009776lbd.34 for ; Tue, 16 Dec 2014 04:31:01 -0800 (PST) X-Received: by 10.152.20.7 with SMTP id j7mr24197694lae.37.1418733060932; Tue, 16 Dec 2014 04:31:00 -0800 (PST) Received: from uglx0153363.synapse.com ([195.238.92.128]) by mx.google.com with ESMTPSA id eg2sm169516lbb.29.2014.12.16.04.30.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 16 Dec 2014 04:31:00 -0800 (PST) From: Taras Kondratiuk To: lng-odp@lists.linaro.org Date: Tue, 16 Dec 2014 14:30:39 +0200 Message-Id: <1418733042-18047-7-git-send-email-taras.kondratiuk@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1418733042-18047-1-git-send-email-taras.kondratiuk@linaro.org> References: <1418733042-18047-1-git-send-email-taras.kondratiuk@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCHv5 6/9] linux-generic: packet: improve packet parsing X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: taras.kondratiuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.169 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Bill Fischofer Signed-off-by: Bill Fischofer Signed-off-by: Taras Kondratiuk --- helper/include/odph_ip.h | 28 +- .../linux-generic/include/odp_packet_internal.h | 31 +- platform/linux-generic/odp_packet.c | 458 +++++++++++++-------- platform/linux-generic/odp_packet_socket.c | 59 +-- 4 files changed, 349 insertions(+), 227 deletions(-) diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h index 16c8266..22ec43f 100644 --- a/helper/include/odph_ip.h +++ b/helper/include/odph_ip.h @@ -143,16 +143,30 @@ typedef struct ODP_PACKED { /** @internal Compile time assert */ ODP_STATIC_ASSERT(sizeof(odph_ipv6hdr_t) == ODPH_IPV6HDR_LEN, "ODPH_IPV6HDR_T__SIZE_ERROR"); +/** + * IPv6 Header extensions + */ +typedef struct ODP_PACKED { + uint8_t next_hdr; /**< Protocol of next header */ + uint8_t ext_len; /**< Length of this extention in 8 byte units, + not counting first 8 bytes, so 0 = 8 bytes + 1 = 16 bytes, etc. */ + uint8_t filler[6]; /**< Fill out first 8 byte segment */ +} odph_ipv6hdr_ext_t; + /** @name * IP protocol values (IPv4:'proto' or IPv6:'next_hdr') * @{*/ -#define ODPH_IPPROTO_ICMP 0x01 /**< Internet Control Message Protocol (1) */ -#define ODPH_IPPROTO_TCP 0x06 /**< Transmission Control Protocol (6) */ -#define ODPH_IPPROTO_UDP 0x11 /**< User Datagram Protocol (17) */ -#define ODPH_IPPROTO_SCTP 0x84 /**< Stream Control Transmission Protocol (132) */ -#define ODPH_IPPROTO_FRAG 0x2C /**< Fragment (44) */ -#define ODPH_IPPROTO_AH 0x33 /**< Authentication Header (51) */ -#define ODPH_IPPROTO_ESP 0x32 /**< Encapsulating Security Payload (50) */ +#define ODPH_IPPROTO_HOPOPTS 0x00 /**< IPv6 hop-by-hop options */ +#define ODPH_IPPROTO_ICMP 0x01 /**< Internet Control Message Protocol (1) */ +#define ODPH_IPPROTO_TCP 0x06 /**< Transmission Control Protocol (6) */ +#define ODPH_IPPROTO_UDP 0x11 /**< User Datagram Protocol (17) */ +#define ODPH_IPPROTO_ROUTE 0x2B /**< IPv6 Routing header (43) */ +#define ODPH_IPPROTO_FRAG 0x2C /**< IPv6 Fragment (44) */ +#define ODPH_IPPROTO_AH 0x33 /**< Authentication Header (51) */ +#define ODPH_IPPROTO_ESP 0x32 /**< Encapsulating Security Payload (50) */ +#define ODPH_IPPROTO_INVALID 0xFF /**< Reserved invalid by IANA */ + /**@}*/ #ifdef __cplusplus diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index fbaaa5f..0b24300 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -44,6 +44,7 @@ typedef union { uint32_t vlan:1; /**< VLAN hdr found */ uint32_t vlan_qinq:1; /**< Stacked VLAN found, QinQ */ + uint32_t snap:1; /**< SNAP */ uint32_t arp:1; /**< ARP */ uint32_t ipv4:1; /**< IPv4 */ @@ -54,6 +55,7 @@ typedef union { uint32_t udp:1; /**< UDP */ uint32_t tcp:1; /**< TCP */ + uint32_t tcpopt:1; /**< TCP options present */ uint32_t sctp:1; /**< SCTP */ uint32_t icmp:1; /**< ICMP */ }; @@ -70,7 +72,9 @@ typedef union { struct { /* Bitfield flags for each detected error */ + uint32_t app_error:1; /**< Error bit for application use */ uint32_t frame_len:1; /**< Frame length error */ + uint32_t snap_len:1; /**< Snap length error */ uint32_t l2_chksum:1; /**< L2 checksum error, checks TBD */ uint32_t ip_err:1; /**< IP error, checks TBD */ uint32_t tcp_err:1; /**< TCP error, checks TBD */ @@ -89,7 +93,10 @@ typedef union { struct { /* Bitfield flags for each output option */ - uint32_t l4_chksum:1; /**< Request L4 checksum calculation */ + uint32_t l3_chksum_set:1; /**< L3 chksum bit is valid */ + uint32_t l3_chksum:1; /**< L3 chksum override */ + uint32_t l4_chksum_set:1; /**< L3 chksum bit is valid */ + uint32_t l4_chksum:1; /**< L4 chksum override */ }; } output_flags_t; @@ -110,13 +117,19 @@ typedef struct { uint32_t l2_offset; /**< offset to L2 hdr, e.g. Eth */ uint32_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ uint32_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ + uint32_t payload_offset; /**< offset to payload */ + + uint32_t vlan_s_tag; /**< Parsed 1st VLAN header (S-TAG) */ + uint32_t vlan_c_tag; /**< Parsed 2nd VLAN header (C-TAG) */ + uint32_t l3_protocol; /**< Parsed L3 protocol */ + uint32_t l3_len; /**< Layer 3 length */ + uint32_t l4_protocol; /**< Parsed L4 protocol */ + uint32_t l4_len; /**< Layer 4 length */ uint32_t frame_len; uint32_t headroom; uint32_t tailroom; - uint64_t user_ctx; /* user context */ - odp_pktio_t input; } odp_packet_hdr_t; @@ -134,11 +147,6 @@ static inline odp_packet_hdr_t *odp_packet_hdr(odp_packet_t pkt) } /** - * Parse packet and set internal metadata - */ -void odp_packet_parse(odp_packet_t pkt, size_t len, size_t l2_offset); - -/** * Initialize packet buffer */ static inline void packet_init(pool_entry_t *pool, @@ -180,8 +188,15 @@ static inline void *packet_map(odp_packet_hdr_t *pkt_hdr, pkt_hdr->headroom + pkt_hdr->frame_len); } +static inline void packet_set_len(odp_packet_t pkt, uint32_t len) +{ + odp_packet_hdr(pkt)->frame_len = len; +} + odp_packet_t _odp_packet_alloc(odp_buffer_pool_t pool_hdl); +int _odp_packet_parse(odp_packet_t pkt); + #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index e5899fd..ff05849 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -12,15 +12,12 @@ #include #include +#include +#include #include #include -static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, - odph_ipv4hdr_t *ipv4, size_t *offset_out); -static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, - odph_ipv6hdr_t *ipv6, size_t *offset_out); - /* * * Alloc and free @@ -200,172 +197,6 @@ int odp_packet_seg_count(odp_packet_t pkt) return odp_packet_hdr(pkt)->buf_hdr.segcount; } - -/** - * Simple packet parser: eth, VLAN, IP, TCP/UDP/ICMP - * - * Internal function: caller is resposible for passing only valid packet handles - * , lengths and offsets (usually done&called in packet input). - * - * @param pkt Packet handle - * @param len Packet length in bytes - * @param frame_offset Byte offset to L2 header - */ -void odp_packet_parse(odp_packet_t pkt, size_t len, size_t frame_offset) -{ - odp_packet_hdr_t *const pkt_hdr = odp_packet_hdr(pkt); - odph_ethhdr_t *eth; - odph_vlanhdr_t *vlan; - odph_ipv4hdr_t *ipv4; - odph_ipv6hdr_t *ipv6; - uint16_t ethtype; - size_t offset = 0; - uint8_t ip_proto = 0; - - pkt_hdr->input_flags.eth = 1; - pkt_hdr->l2_offset = frame_offset; - pkt_hdr->frame_len = len; - - if (len > ODPH_ETH_LEN_MAX) - pkt_hdr->input_flags.jumbo = 1; - - /* Assume valid L2 header, no CRC/FCS check in SW */ - pkt_hdr->input_flags.l2 = 1; - pkt_hdr->l2_offset = frame_offset; - - eth = (odph_ethhdr_t *)odp_packet_data(pkt); - ethtype = odp_be_to_cpu_16(eth->type); - vlan = (odph_vlanhdr_t *)ð->type; - - if (ethtype == ODPH_ETHTYPE_VLAN_OUTER) { - pkt_hdr->input_flags.vlan_qinq = 1; - ethtype = odp_be_to_cpu_16(vlan->tpid); - offset += sizeof(odph_vlanhdr_t); - vlan = &vlan[1]; - } - - if (ethtype == ODPH_ETHTYPE_VLAN) { - pkt_hdr->input_flags.vlan = 1; - ethtype = odp_be_to_cpu_16(vlan->tpid); - offset += sizeof(odph_vlanhdr_t); - } - - /* Set l3_offset+flag only for known ethtypes */ - switch (ethtype) { - case ODPH_ETHTYPE_IPV4: - pkt_hdr->input_flags.ipv4 = 1; - pkt_hdr->input_flags.l3 = 1; - pkt_hdr->l3_offset = frame_offset + ODPH_ETHHDR_LEN + offset; - ipv4 = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt, NULL); - ip_proto = parse_ipv4(pkt_hdr, ipv4, &offset); - break; - case ODPH_ETHTYPE_IPV6: - pkt_hdr->input_flags.ipv6 = 1; - pkt_hdr->input_flags.l3 = 1; - pkt_hdr->l3_offset = frame_offset + ODPH_ETHHDR_LEN + offset; - ipv6 = (odph_ipv6hdr_t *)odp_packet_l3_ptr(pkt, NULL); - ip_proto = parse_ipv6(pkt_hdr, ipv6, &offset); - break; - case ODPH_ETHTYPE_ARP: - pkt_hdr->input_flags.arp = 1; - /* fall through */ - default: - ip_proto = 0; - break; - } - - switch (ip_proto) { - case ODPH_IPPROTO_UDP: - pkt_hdr->input_flags.udp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_TCP: - pkt_hdr->input_flags.tcp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_SCTP: - pkt_hdr->input_flags.sctp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - case ODPH_IPPROTO_ICMP: - pkt_hdr->input_flags.icmp = 1; - pkt_hdr->input_flags.l4 = 1; - pkt_hdr->l4_offset = pkt_hdr->l3_offset + offset; - break; - default: - /* 0 or unhandled IP protocols, don't set L4 flag+offset */ - if (pkt_hdr->input_flags.ipv6) { - /* IPv6 next_hdr is not L4, mark as IP-option instead */ - pkt_hdr->input_flags.ipopt = 1; - } - break; - } -} - -static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, - odph_ipv4hdr_t *ipv4, size_t *offset_out) -{ - uint8_t ihl; - uint16_t frag_offset; - - ihl = ODPH_IPV4HDR_IHL(ipv4->ver_ihl); - if (odp_unlikely(ihl < ODPH_IPV4HDR_IHL_MIN)) { - pkt_hdr->error_flags.ip_err = 1; - return 0; - } - - if (odp_unlikely(ihl > ODPH_IPV4HDR_IHL_MIN)) { - pkt_hdr->input_flags.ipopt = 1; - return 0; - } - - /* A packet is a fragment if: - * "more fragments" flag is set (all fragments except the last) - * OR - * "fragment offset" field is nonzero (all fragments except the first) - */ - frag_offset = odp_be_to_cpu_16(ipv4->frag_offset); - if (odp_unlikely(ODPH_IPV4HDR_IS_FRAGMENT(frag_offset))) { - pkt_hdr->input_flags.ipfrag = 1; - return 0; - } - - if (ipv4->proto == ODPH_IPPROTO_ESP || - ipv4->proto == ODPH_IPPROTO_AH) { - pkt_hdr->input_flags.ipsec = 1; - return 0; - } - - /* Set pkt_hdr->input_flags.ipopt when checking L4 hdrs after return */ - - *offset_out = sizeof(uint32_t) * ihl; - return ipv4->proto; -} - -static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, - odph_ipv6hdr_t *ipv6, size_t *offset_out) -{ - if (ipv6->next_hdr == ODPH_IPPROTO_ESP || - ipv6->next_hdr == ODPH_IPPROTO_AH) { - pkt_hdr->input_flags.ipopt = 1; - pkt_hdr->input_flags.ipsec = 1; - return 0; - } - - if (odp_unlikely(ipv6->next_hdr == ODPH_IPPROTO_FRAG)) { - pkt_hdr->input_flags.ipopt = 1; - pkt_hdr->input_flags.ipfrag = 1; - return 0; - } - - /* Don't step through more extensions */ - *offset_out = ODPH_IPV6HDR_LEN; - return ipv6->next_hdr; -} - void odp_packet_print(odp_packet_t pkt) { int max_len = 512; @@ -458,3 +289,288 @@ odp_packet_t _odp_packet_alloc(odp_buffer_pool_t pool_hdl) return (odp_packet_t)buffer_alloc(pool_hdl, pool->s.params.buf_size); } + +/** + * Parser helper function for IPv4 + */ +static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_ipv4hdr_t *ipv4 = (odph_ipv4hdr_t *)*parseptr; + uint8_t ver = ODPH_IPV4HDR_VER(ipv4->ver_ihl); + uint8_t ihl = ODPH_IPV4HDR_IHL(ipv4->ver_ihl); + uint16_t frag_offset; + + pkt_hdr->l3_len = odp_be_to_cpu_16(ipv4->tot_len); + + if (odp_unlikely(ihl < ODPH_IPV4HDR_IHL_MIN) || + odp_unlikely(ver != 4) || + (pkt_hdr->l3_len > pkt_hdr->frame_len - *offset)) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + *offset += ihl * 4; + *parseptr += ihl * 4; + + if (odp_unlikely(ihl > ODPH_IPV4HDR_IHL_MIN)) + pkt_hdr->input_flags.ipopt = 1; + + /* A packet is a fragment if: + * "more fragments" flag is set (all fragments except the last) + * OR + * "fragment offset" field is nonzero (all fragments except the first) + */ + frag_offset = odp_be_to_cpu_16(ipv4->frag_offset); + if (odp_unlikely(ODPH_IPV4HDR_IS_FRAGMENT(frag_offset))) + pkt_hdr->input_flags.ipfrag = 1; + + return ipv4->proto; +} + +/** + * Parser helper function for IPv6 + */ +static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_ipv6hdr_t *ipv6 = (odph_ipv6hdr_t *)*parseptr; + odph_ipv6hdr_ext_t *ipv6ext; + + pkt_hdr->l3_len = odp_be_to_cpu_16(ipv6->payload_len); + + /* Basic sanity checks on IPv6 header */ + if ((ipv6->ver_tc_flow >> 28) != 6 || + pkt_hdr->l3_len > pkt_hdr->frame_len - *offset) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + /* Skip past IPv6 header */ + *offset += sizeof(odph_ipv6hdr_t); + *parseptr += sizeof(odph_ipv6hdr_t); + + + /* Skip past any IPv6 extension headers */ + if (ipv6->next_hdr == ODPH_IPPROTO_HOPOPTS || + ipv6->next_hdr == ODPH_IPPROTO_ROUTE) { + pkt_hdr->input_flags.ipopt = 1; + + do { + ipv6ext = (odph_ipv6hdr_ext_t *)*parseptr; + uint16_t extlen = 8 + ipv6ext->ext_len * 8; + + *offset += extlen; + *parseptr += extlen; + } while ((ipv6ext->next_hdr == ODPH_IPPROTO_HOPOPTS || + ipv6ext->next_hdr == ODPH_IPPROTO_ROUTE) && + *offset < pkt_hdr->frame_len); + + if (*offset >= pkt_hdr->l3_offset + ipv6->payload_len) { + pkt_hdr->error_flags.ip_err = 1; + return 0; + } + + if (ipv6ext->next_hdr == ODPH_IPPROTO_FRAG) + pkt_hdr->input_flags.ipfrag = 1; + + return ipv6ext->next_hdr; + } + + if (odp_unlikely(ipv6->next_hdr == ODPH_IPPROTO_FRAG)) { + pkt_hdr->input_flags.ipopt = 1; + pkt_hdr->input_flags.ipfrag = 1; + } + + return ipv6->next_hdr; +} + +/** + * Parser helper function for TCP + */ +static inline void parse_tcp(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_tcphdr_t *tcp = (odph_tcphdr_t *)*parseptr; + + if (tcp->hl < sizeof(odph_tcphdr_t)/sizeof(uint32_t)) + pkt_hdr->error_flags.tcp_err = 1; + else if ((uint32_t)tcp->hl * 4 > sizeof(odph_tcphdr_t)) + pkt_hdr->input_flags.tcpopt = 1; + + pkt_hdr->l4_len = pkt_hdr->l3_len + + pkt_hdr->l3_offset - pkt_hdr->l4_offset; + + *offset += (uint32_t)tcp->hl * 4; + *parseptr += (uint32_t)tcp->hl * 4; +} + +/** + * Parser helper function for UDP + */ +static inline void parse_udp(odp_packet_hdr_t *pkt_hdr, + uint8_t **parseptr, uint32_t *offset) +{ + odph_udphdr_t *udp = (odph_udphdr_t *)*parseptr; + uint32_t udplen = odp_be_to_cpu_16(udp->length); + + if (udplen < sizeof(odph_udphdr_t) || + udplen > (pkt_hdr->l3_len + + pkt_hdr->l3_offset - pkt_hdr->l4_offset)) { + pkt_hdr->error_flags.udp_err = 1; + } + + pkt_hdr->l4_len = udplen; + + *offset += sizeof(odph_udphdr_t); + *parseptr += sizeof(odph_udphdr_t); +} + +/** + * Simple packet parser + */ + +int _odp_packet_parse(odp_packet_t pkt) +{ + odp_packet_hdr_t *const pkt_hdr = odp_packet_hdr(pkt); + odph_ethhdr_t *eth; + odph_vlanhdr_t *vlan; + uint16_t ethtype; + uint8_t *parseptr; + uint32_t offset, seglen; + uint8_t ip_proto = 0; + + /* Reset parser metadata for new parse */ + pkt_hdr->error_flags.all = 0; + pkt_hdr->input_flags.all = 0; + pkt_hdr->output_flags.all = 0; + pkt_hdr->l2_offset = 0; + pkt_hdr->l3_offset = 0; + pkt_hdr->l4_offset = 0; + pkt_hdr->payload_offset = 0; + pkt_hdr->vlan_s_tag = 0; + pkt_hdr->vlan_c_tag = 0; + pkt_hdr->l3_protocol = 0; + pkt_hdr->l4_protocol = 0; + + /* We only support Ethernet for now */ + pkt_hdr->input_flags.eth = 1; + + /* Detect jumbo frames */ + if (pkt_hdr->frame_len > ODPH_ETH_LEN_MAX) + pkt_hdr->input_flags.jumbo = 1; + + /* Assume valid L2 header, no CRC/FCS check in SW */ + pkt_hdr->input_flags.l2 = 1; + + eth = (odph_ethhdr_t *)packet_map(pkt_hdr, 0, &seglen); + offset = sizeof(odph_ethhdr_t); + parseptr = (uint8_t *)ð->type; + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + + /* Parse the VLAN header(s), if present */ + if (ethtype == ODPH_ETHTYPE_VLAN_OUTER) { + pkt_hdr->input_flags.vlan_qinq = 1; + pkt_hdr->input_flags.vlan = 1; + vlan = (odph_vlanhdr_t *)(void *)parseptr; + pkt_hdr->vlan_s_tag = ((ethtype << 16) | + odp_be_to_cpu_16(vlan->tci)); + offset += sizeof(odph_vlanhdr_t); + parseptr += sizeof(odph_vlanhdr_t); + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + if (ethtype == ODPH_ETHTYPE_VLAN) { + pkt_hdr->input_flags.vlan = 1; + vlan = (odph_vlanhdr_t *)(void *)parseptr; + pkt_hdr->vlan_c_tag = ((ethtype << 16) | + odp_be_to_cpu_16(vlan->tci)); + offset += sizeof(odph_vlanhdr_t); + parseptr += sizeof(odph_vlanhdr_t); + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + /* Check for SNAP vs. DIX */ + if (ethtype < ODPH_ETH_LEN_MAX) { + pkt_hdr->input_flags.snap = 1; + if (ethtype > pkt_hdr->frame_len - offset) { + pkt_hdr->error_flags.snap_len = 1; + goto parse_exit; + } + offset += 8; + parseptr += 8; + ethtype = odp_be_to_cpu_16(*((uint16_t *)(void *)parseptr)); + } + + /* Consume Ethertype for Layer 3 parse */ + parseptr += 2; + + /* Set l3_offset+flag only for known ethtypes */ + pkt_hdr->input_flags.l3 = 1; + pkt_hdr->l3_offset = offset; + pkt_hdr->l3_protocol = ethtype; + + /* Parse Layer 3 headers */ + switch (ethtype) { + case ODPH_ETHTYPE_IPV4: + pkt_hdr->input_flags.ipv4 = 1; + ip_proto = parse_ipv4(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_ETHTYPE_IPV6: + pkt_hdr->input_flags.ipv6 = 1; + ip_proto = parse_ipv6(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_ETHTYPE_ARP: + pkt_hdr->input_flags.arp = 1; + ip_proto = 255; /* Reserved invalid by IANA */ + break; + + default: + pkt_hdr->input_flags.l3 = 0; + ip_proto = 255; /* Reserved invalid by IANA */ + } + + /* Set l4_offset+flag only for known ip_proto */ + pkt_hdr->input_flags.l4 = 1; + pkt_hdr->l4_offset = offset; + pkt_hdr->l4_protocol = ip_proto; + + /* Parse Layer 4 headers */ + switch (ip_proto) { + case ODPH_IPPROTO_ICMP: + pkt_hdr->input_flags.icmp = 1; + break; + + case ODPH_IPPROTO_TCP: + pkt_hdr->input_flags.tcp = 1; + parse_tcp(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_IPPROTO_UDP: + pkt_hdr->input_flags.udp = 1; + parse_udp(pkt_hdr, &parseptr, &offset); + break; + + case ODPH_IPPROTO_AH: + case ODPH_IPPROTO_ESP: + pkt_hdr->input_flags.ipsec = 1; + break; + + default: + pkt_hdr->input_flags.l4 = 0; + break; + } + + /* + * Anything beyond what we parse here is considered payload. + * Note: Payload is really only relevant for TCP and UDP. For + * all other protocols, the payload offset will point to the + * final header (ARP, ICMP, AH, ESP, or IP Fragment). + */ + pkt_hdr->payload_offset = offset; + +parse_exit: + return pkt_hdr->error_flags.all != 0; +} diff --git a/platform/linux-generic/odp_packet_socket.c b/platform/linux-generic/odp_packet_socket.c index ead5d2f..2849065 100644 --- a/platform/linux-generic/odp_packet_socket.c +++ b/platform/linux-generic/odp_packet_socket.c @@ -34,6 +34,7 @@ #include #include +#include #include #include #include @@ -42,8 +43,6 @@ #include #include -#include -#include /** Provide a sendmmsg wrapper for systems with no libc or kernel support. * As it is implemented as a weak symbol, it has zero effect on systems @@ -207,28 +206,19 @@ int setup_pkt_sock(pkt_sock_t *const pkt_sock, const char *netdev, unsigned int if_idx; struct ifreq ethreq; struct sockaddr_ll sa_ll; - odp_packet_t pkt; - uint8_t *pkt_buf; - uint8_t *l2_hdr; if (pool == ODP_BUFFER_POOL_INVALID) return -1; pkt_sock->pool = pool; - pkt = _odp_packet_alloc(pool); - if (!odp_packet_is_valid(pkt)) - return -1; - - pkt_buf = odp_packet_addr(pkt); - l2_hdr = ETHBUF_ALIGN(pkt_buf); /* Store eth buffer offset for pkt buffers from this pool */ - pkt_sock->frame_offset = (uintptr_t)l2_hdr - (uintptr_t)pkt_buf; + pkt_sock->frame_offset = 0; /* pkt buffer size */ - pkt_sock->buf_size = odp_packet_buf_size(pkt); + pkt_sock->buf_size = odp_buffer_pool_segment_size(pool); /* max frame len taking into account the l2-offset */ - pkt_sock->max_frame_len = pkt_sock->buf_size - pkt_sock->frame_offset; - - odp_packet_free(pkt); + pkt_sock->max_frame_len = pkt_sock->buf_size - + odp_buffer_pool_headroom(pool) - + odp_buffer_pool_tailroom(pool); odp_spinlock_lock(&raw_sockets_lock); @@ -319,7 +309,6 @@ int recv_pkt_sock_basic(pkt_sock_t *const pkt_sock, int const sockfd = pkt_sock->sockfd; odp_packet_t pkt = ODP_PACKET_INVALID; uint8_t *pkt_buf; - uint8_t *l2_hdr; int nb_rx = 0; /* recvfrom: @@ -337,10 +326,9 @@ int recv_pkt_sock_basic(pkt_sock_t *const pkt_sock, break; } - pkt_buf = odp_packet_addr(pkt); - l2_hdr = pkt_buf + pkt_sock->frame_offset; + pkt_buf = odp_packet_data(pkt); - recv_bytes = recvfrom(sockfd, l2_hdr, + recv_bytes = recvfrom(sockfd, pkt_buf, pkt_sock->max_frame_len, MSG_DONTWAIT, (struct sockaddr *)&sll, &addrlen); /* no data or error: free recv buf and break out of loop */ @@ -351,7 +339,8 @@ int recv_pkt_sock_basic(pkt_sock_t *const pkt_sock, continue; /* Parse and set packet header data */ - odp_packet_parse(pkt, recv_bytes, pkt_sock->frame_offset); + packet_set_len(pkt, recv_bytes); + _odp_packet_parse(pkt); pkt_table[nb_rx] = pkt; pkt = ODP_PACKET_INVALID; @@ -433,7 +422,7 @@ int recv_pkt_sock_mmsg(pkt_sock_t *const pkt_sock, if (odp_unlikely(pkt_table[i] == ODP_PACKET_INVALID)) break; - pkt_buf = odp_packet_addr(pkt_table[i]); + pkt_buf = odp_packet_data(pkt_table[i]); l2_hdr = pkt_buf + pkt_sock->frame_offset; iovecs[i].iov_base = l2_hdr; iovecs[i].iov_len = pkt_sock->max_frame_len; @@ -456,8 +445,8 @@ int recv_pkt_sock_mmsg(pkt_sock_t *const pkt_sock, } /* Parse and set packet header data */ - odp_packet_parse(pkt_table[i], msgvec[i].msg_len, - pkt_sock->frame_offset); + packet_set_len(pkt_table[i], msgvec[i].msg_len); + _odp_packet_parse(pkt_table[i]); pkt_table[nb_rx] = pkt_table[i]; nb_rx++; @@ -570,7 +559,6 @@ static inline void mmap_tx_user_ready(struct tpacket2_hdr *hdr) static inline unsigned pkt_mmap_v2_rx(int sock, struct ring *ring, odp_packet_t pkt_table[], unsigned len, odp_buffer_pool_t pool, - size_t frame_offset, unsigned char if_mac[]) { union frame_map ppd; @@ -607,14 +595,14 @@ static inline unsigned pkt_mmap_v2_rx(int sock, struct ring *ring, if (odp_unlikely(pkt_table[i] == ODP_PACKET_INVALID)) break; - l2_hdr = odp_packet_addr(pkt_table[i]) - + frame_offset; + packet_set_len(pkt_table[i], pkt_len); + l2_hdr = odp_packet_data(pkt_table[i]); memcpy(l2_hdr, pkt_buf, pkt_len); mmap_rx_user_ready(ppd.raw); /* Parse and set packet header data */ - odp_packet_parse(pkt_table[i], pkt_len, frame_offset); + _odp_packet_parse(pkt_table[i]); frame_num = next_frame_num; i++; @@ -837,9 +825,6 @@ static int mmap_store_hw_addr(pkt_sock_mmap_t *const pkt_sock, int setup_pkt_sock_mmap(pkt_sock_mmap_t *const pkt_sock, const char *netdev, odp_buffer_pool_t pool, int fanout) { - odp_packet_t pkt; - uint8_t *pkt_buf; - uint8_t *l2_hdr; int if_idx; int ret = 0; @@ -848,16 +833,8 @@ int setup_pkt_sock_mmap(pkt_sock_mmap_t *const pkt_sock, const char *netdev, if (pool == ODP_BUFFER_POOL_INVALID) return -1; - pkt = _odp_packet_alloc(pool); - if (!odp_packet_is_valid(pkt)) - return -1; - - pkt_buf = odp_packet_addr(pkt); - l2_hdr = ETHBUF_ALIGN(pkt_buf); /* Store eth buffer offset for pkt buffers from this pool */ - pkt_sock->frame_offset = (uintptr_t)l2_hdr - (uintptr_t)pkt_buf; - - odp_packet_free(pkt); + pkt_sock->frame_offset = 0; pkt_sock->pool = pool; pkt_sock->sockfd = mmap_pkt_socket(); @@ -924,7 +901,7 @@ int recv_pkt_sock_mmap(pkt_sock_mmap_t *const pkt_sock, { return pkt_mmap_v2_rx(pkt_sock->rx_ring.sock, &pkt_sock->rx_ring, pkt_table, len, pkt_sock->pool, - pkt_sock->frame_offset, pkt_sock->if_mac); + pkt_sock->if_mac); } /*