From patchwork Tue Dec 16 12:30:42 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taras Kondratiuk X-Patchwork-Id: 42332 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f72.google.com (mail-wg0-f72.google.com [74.125.82.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id A8DDA2456A for ; Tue, 16 Dec 2014 12:32:16 +0000 (UTC) Received: by mail-wg0-f72.google.com with SMTP id y19sf8563989wgg.7 for ; Tue, 16 Dec 2014 04:32:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:subject:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=/zZSey9ykidgtOoEIJGhj1++M4g4jWn1H7mwR/0D5Xo=; b=jfmTRihCrx4Sghklm2lGaCCOEjVsfyAWKvJQ/qzW2cbUyxY+HXTH82SW3v2Jz00+La rsqg8H3deqDoCmkKo4ZyFV5wjUwiGi2xqA5mjlVE0PfFF37wDf7ZyRCxB6Lvtis2NdQw MG8+Uj7si0KD+gJeuCI896plrKOiEj9b6JeWwYcpRurdCFG3csp+VKZSYNxSqeR/Sh51 TvDPXNEB6U4pIh5+qura0zhPa5EhaDW4dud8s1VA6q5PwdQSDQKOhbUsEwLvxyE7U3om 64IA1J1QLVKEW/Hj3pj4rF82YrTdBssy4xxW41IeIF9Yqpf4fAcNY0Cqmovr6zl9vcsd gRog== X-Gm-Message-State: ALoCoQmFxLRkLaw6eXjD3FN53ahtaKgobB/cY49IGdZhFMhbL84x155ROYl9IOFC1Q2Mt7COJcKv X-Received: by 10.112.162.199 with SMTP id yc7mr452lbb.23.1418733135986; Tue, 16 Dec 2014 04:32:15 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.206.102 with SMTP id ln6ls842792lac.95.gmail; Tue, 16 Dec 2014 04:32:15 -0800 (PST) X-Received: by 10.152.5.100 with SMTP id r4mr35426590lar.26.1418733135817; Tue, 16 Dec 2014 04:32:15 -0800 (PST) Received: from mail-lb0-f169.google.com (mail-lb0-f169.google.com. [209.85.217.169]) by mx.google.com with ESMTPS id du3si687654lbc.32.2014.12.16.04.32.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Dec 2014 04:32:15 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.169 as permitted sender) client-ip=209.85.217.169; Received: by mail-lb0-f169.google.com with SMTP id p9so10885311lbv.0 for ; Tue, 16 Dec 2014 04:32:15 -0800 (PST) X-Received: by 10.112.136.69 with SMTP id py5mr21539373lbb.56.1418733135696; Tue, 16 Dec 2014 04:32:15 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.142.69 with SMTP id ru5csp1020134lbb; Tue, 16 Dec 2014 04:32:14 -0800 (PST) X-Received: by 10.224.115.75 with SMTP id h11mr21649949qaq.58.1418733133887; Tue, 16 Dec 2014 04:32:13 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id b8si605440qam.107.2014.12.16.04.32.11 (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 16 Dec 2014 04:32:13 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y0rIO-0006fT-QK; Tue, 16 Dec 2014 12:32:08 +0000 Received: from mail-lb0-f175.google.com ([209.85.217.175]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1Y0rHS-0006Uv-Ch for lng-odp@lists.linaro.org; Tue, 16 Dec 2014 12:31:10 +0000 Received: by mail-lb0-f175.google.com with SMTP id u10so10622366lbd.6 for ; Tue, 16 Dec 2014 04:31:04 -0800 (PST) X-Received: by 10.152.219.3 with SMTP id pk3mr35460748lac.19.1418733064859; Tue, 16 Dec 2014 04:31:04 -0800 (PST) Received: from uglx0153363.synapse.com ([195.238.92.128]) by mx.google.com with ESMTPSA id eg2sm169516lbb.29.2014.12.16.04.31.03 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 16 Dec 2014 04:31:03 -0800 (PST) From: Taras Kondratiuk To: lng-odp@lists.linaro.org Date: Tue, 16 Dec 2014 14:30:42 +0200 Message-Id: <1418733042-18047-10-git-send-email-taras.kondratiuk@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1418733042-18047-1-git-send-email-taras.kondratiuk@linaro.org> References: <1418733042-18047-1-git-send-email-taras.kondratiuk@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCHv5 9/9] api: packet: add segmentation and copy API X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: taras.kondratiuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.169 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Bill Fischofer Signed-off-by: Bill Fischofer Signed-off-by: Taras Kondratiuk --- helper/include/odph_ip.h | 4 +- platform/linux-generic/include/api/odp_packet.h | 304 +++++++++++------- .../linux-generic/include/api/odp_platform_types.h | 21 +- .../linux-generic/include/odp_buffer_inlines.h | 55 ++++ .../linux-generic/include/odp_packet_internal.h | 5 + platform/linux-generic/odp_crypto.c | 5 +- platform/linux-generic/odp_packet.c | 341 ++++++++++++++++++--- 7 files changed, 563 insertions(+), 172 deletions(-) diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h index 22ec43f..272fd96 100644 --- a/helper/include/odph_ip.h +++ b/helper/include/odph_ip.h @@ -88,7 +88,9 @@ static inline int odph_ipv4_csum_valid(odp_packet_t pkt) if (!odp_packet_l3_offset(pkt)) return 0; - memcpy(&ip, odp_packet_l3_ptr(pkt, NULL), sizeof(odph_ipv4hdr_t)); + odp_packet_copydata_out(pkt, odp_packet_l3_offset(pkt), + sizeof(odph_ipv4hdr_t), &ip); + w = (uint16_t *)(void *)&ip; chksum = ip.chksum; ip.chksum = 0x0; diff --git a/platform/linux-generic/include/api/odp_packet.h b/platform/linux-generic/include/api/odp_packet.h index 853709d..97c2cb6 100644 --- a/platform/linux-generic/include/api/odp_packet.h +++ b/platform/linux-generic/include/api/odp_packet.h @@ -1,7 +1,7 @@ /* Copyright (c) 2013, Linaro Limited * All rights reserved. * - * SPDX-License-Identifier: BSD-3-Clause + * SPDX-License-Identifier: BSD-3-Clause */ @@ -342,22 +342,58 @@ void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len); * @retval NULL The specified offset exceeds allowable data length */ void *odp_packet_pull_tail(odp_packet_t pkt, uint32_t len); + /** - * Set the packet length + * Packet offset pointer + * + * Returns pointer to data in the packet offset. The packet level byte offset is + * calculated from the current odp_packet_data() position. Optionally outputs + * handle to the segment and number of data bytes in the segment following the + * pointer. + * + * @param pkt Packet handle + * @param offset Byte offset into the packet + * @param[out] len Number of data bytes remaining in the segment (output). + * Ignored when NULL. + * @param[out] seg Handle to the segment containing the address (output). + * Ignored when NULL. + * + * @return Pointer to the offset + * @retval NULL Requested offset exceeds packet length + */ +void *odp_packet_offset(odp_packet_t pkt, uint32_t offset, uint32_t *len, + odp_packet_seg_t *seg); + +/* + * + * Meta-data + * ******************************************************** * - * @param pkt Packet handle - * @param len Length of packet in bytes */ -void odp_packet_set_len(odp_packet_t pkt, size_t len); /** - * Get the packet length + * Packet pool * - * @param pkt Packet handle + * Returns handle to the buffer pool where the packet was allocated from. + * + * @param pkt Packet handle + * + * @return Buffer pool handle + */ +odp_buffer_pool_t odp_packet_pool(odp_packet_t pkt); + +/** + * Packet input interface + * + * Returns handle to the packet IO interface which received the packet or + * ODP_PKTIO_INVALID when the packet was allocated/reset by the application. + * + * @param pkt Packet handle * - * @return Packet length in bytes + * @return Packet interface handle + * @retval ODP_PKTIO_INVALID Packet was not received */ -size_t odp_packet_get_len(odp_packet_t pkt); +odp_pktio_t odp_packet_input(odp_packet_t pkt); /** * User context pointer @@ -544,223 +580,259 @@ uint32_t odp_packet_l4_offset(odp_packet_t pkt); int odp_packet_l4_offset_set(odp_packet_t pkt, uint32_t offset); /** - * Print (debug) information about the packet + * Tests if packet is segmented * * @param pkt Packet handle + * + * @retval 0 Packet is not segmented + * @retval 1 Packet is segmented */ -void odp_packet_print(odp_packet_t pkt); +int odp_packet_is_segmented(odp_packet_t pkt); /** - * Copy contents and metadata from pkt_src to pkt_dst - * Useful when creating copies of packets + * Number of segments * - * @param pkt_dst Destination packet - * @param pkt_src Source packet + * Returns number of segments in the packet. A packet has always at least one + * segment. + * + * @param pkt Packet handle * - * @return 0 if successful + * @return Number of segments (>0) */ -int odp_packet_copy(odp_packet_t pkt_dst, odp_packet_t pkt_src); +int odp_packet_num_segs(odp_packet_t pkt); /** - * Tests if packet is segmented (a scatter/gather list) + * First segment in packet + * + * A packet has always the first segment (has at least one segment). * * @param pkt Packet handle * - * @return Non-zero if packet is segmented, otherwise 0 + * @return Handle to the first segment */ -int odp_packet_is_segmented(odp_packet_t pkt); +odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt); /** - * Segment count + * Last segment in packet * - * Returns number of segments in the packet. A packet has always at least one - * segment (the packet buffer itself). + * A packet has always the last segment (has at least one segment). * * @param pkt Packet handle * - * @return Segment count + * @return Handle to the last segment */ -int odp_packet_seg_count(odp_packet_t pkt); +odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt); /** - * Get segment by index + * Next segment in packet + * + * Returns handle to the next segment after the current segment, or + * ODP_PACKET_SEG_INVALID if there are no more segments. Use + * odp_packet_first_seg() to get handle to the first segment. * * @param pkt Packet handle - * @param index Segment index (0 ... seg_count-1) + * @param seg Current segment handle * - * @return Segment handle, or ODP_PACKET_SEG_INVALID on an error + * @return Handle to the next segment, or ODP_PACKET_SEG_INVALID */ -odp_packet_seg_t odp_packet_seg(odp_packet_t pkt, int index); +odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, odp_packet_seg_t seg); -/** - * Get next segment + +/* * - * @param pkt Packet handle - * @param seg Current segment handle + * Segment level + * ******************************************************** * - * @return Handle to next segment, or ODP_PACKET_SEG_INVALID on an error */ -odp_packet_seg_t odp_packet_seg_next(odp_packet_t pkt, odp_packet_seg_t seg); /** - * Segment info + * Segment buffer address * - * Copies segment parameters into the info structure. + * Returns start address of the segment. * * @param pkt Packet handle * @param seg Segment handle - * @param info Pointer to segment info structure * - * @return 0 if successful, otherwise non-zero + * @return Start address of the segment, or NULL on an error + * + * @see odp_packet_seg_buf_len() */ -int odp_packet_seg_info(odp_packet_t pkt, odp_packet_seg_t seg, - odp_packet_seg_info_t *info); +void *odp_packet_seg_buf_addr(odp_packet_t pkt, odp_packet_seg_t seg); /** - * Segment start address + * Segment buffer length + * + * Returns segment buffer length in bytes. * * @param pkt Packet handle * @param seg Segment handle * - * @return Segment start address, or NULL on an error + * @return Segment buffer length in bytes + * + * @see odp_packet_seg_buf_addr() */ -void *odp_packet_seg_addr(odp_packet_t pkt, odp_packet_seg_t seg); +uint32_t odp_packet_seg_buf_len(odp_packet_t pkt, odp_packet_seg_t seg); /** - * Segment maximum data size + * Segment data pointer + * + * Returns pointer to the first byte of data in the segment. * * @param pkt Packet handle * @param seg Segment handle * - * @return Segment maximum data size + * @return Pointer to the segment data, or NULL on an error + * + * @see odp_packet_seg_data_len() */ -size_t odp_packet_seg_size(odp_packet_t pkt, odp_packet_seg_t seg); +void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg); /** - * Segment data address + * Segment data length + * + * Returns segment data length in bytes. * * @param pkt Packet handle * @param seg Segment handle * - * @return Segment data address + * @return Segment data length in bytes + * + * @see odp_packet_seg_data() */ -void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg); +uint32_t odp_packet_seg_data_len(odp_packet_t pkt, odp_packet_seg_t seg); -/** - * Segment data length + +/* * - * @param pkt Packet handle - * @param seg Segment handle + * Manipulation + * ******************************************************** * - * @return Segment data length */ -size_t odp_packet_seg_data_len(odp_packet_t pkt, odp_packet_seg_t seg); + /** - * Segment headroom + * Add data into an offset * - * seg_headroom = seg_data - seg_addr + * Increases packet data length by adding new data area into the specified + * offset. The operation returns a new packet handle on success. It may modify + * packet segmentation and move data. Handles and pointers must be updated + * after the operation. User is responsible to update packet meta-data offsets + * when needed. The packet is not modified on an error. * - * @param pkt Packet handle - * @param seg Segment handle + * @param pkt Packet handle + * @param offset Byte offset into the packet + * @param len Number of bytes to add into the offset * - * @return Number of octets from seg_addr to seg_data + * @return New packet handle, or ODP_PACKET_INVALID in case of an error. */ -size_t odp_packet_seg_headroom(odp_packet_t pkt, odp_packet_seg_t seg); +odp_packet_t odp_packet_add_data(odp_packet_t pkt, uint32_t offset, + uint32_t len); /** - * Segment tailroom + * Remove data from an offset * - * seg_tailroom = seg_size - seg_headroom - seg_data_len + * Decreases packet data length by removing data from the specified offset. + * The operation returns a new packet handle on success, and may modify + * packet segmentation and move data. Handles and pointers must be updated + * after the operation. User is responsible to update packet meta-data offsets + * when needed. The packet is not modified on an error. * - * @param pkt Packet handle - * @param seg Segment handle + * @param pkt Packet handle + * @param offset Byte offset into the packet + * @param len Number of bytes to remove from the offset * - * @return Number of octets from end-of-data to end-of-segment + * @return New packet handle, or ODP_PACKET_INVALID in case of an error. */ -size_t odp_packet_seg_tailroom(odp_packet_t pkt, odp_packet_seg_t seg); +odp_packet_t odp_packet_rem_data(odp_packet_t pkt, uint32_t offset, + uint32_t len); -/** - * Push out segment head + +/* * - * Push out segment data address (away from data) and increase data length. - * Does not modify packet in case of an error. + * Copy + * ******************************************************** * - * seg_data -= len - * seg_data_len += len + */ + +/** + * Copy packet * - * @param pkt Packet handle - * @param seg Segment handle - * @param len Number of octets to push head (0 ... seg_headroom) + * Create a new copy of the packet. The new packet is exact copy of the source + * packet (incl. data and meta-data). The pool must have been created with + * buffer type ODP_BUFFER_TYPE_PACKET. + * + * @param pkt Packet handle + * @param pool Buffer pool for allocation of the new packet. * - * @return New segment data address, or NULL on an error + * @return Handle to the copy of the packet, or ODP_PACKET_INVALID */ -void *odp_packet_seg_push_head(odp_packet_t pkt, odp_packet_seg_t seg, - size_t len); +odp_packet_t odp_packet_copy(odp_packet_t pkt, odp_buffer_pool_t pool); /** - * Pull in segment head - * - * Pull in segment data address (towards data) and decrease data length. - * Does not modify packet in case of an error. + * Copy data from packet * - * seg_data += len - * seg_data_len -= len + * Copy 'len' bytes of data from the packet level offset to the destination + * address. * - * @param pkt Packet handle - * @param seg Segment handle - * @param len Number of octets to pull head (0 ... seg_data_len) + * @param pkt Packet handle + * @param offset Byte offset into the packet + * @param len Number of bytes to copy + * @param dst Destination address * - * @return New segment data address, or NULL on an error + * @retval 0 Success + * @retval Non-zero Failure */ -void *odp_packet_seg_pull_head(odp_packet_t pkt, odp_packet_seg_t seg, - size_t len); +int odp_packet_copydata_out(odp_packet_t pkt, uint32_t offset, + uint32_t len, void *dst); /** - * Push out segment tail + * Copy data into packet * - * Increase segment data length. - * Does not modify packet in case of an error. + * Copy 'len' bytes of data from the source address into the packet level + * offset. Maximum number of bytes to copy is packet data length minus the + * offset. Packet is not modified on an error. * - * seg_data_len += len + * @param pkt Packet handle + * @param offset Byte offset into the packet + * @param len Number of bytes to copy + * @param src Source address * - * @param pkt Packet handle - * @param seg Segment handle - * @param len Number of octets to push tail (0 ... seg_tailroom) + * @retval 0 Success + * @retval Non-zero Failure + */ +int odp_packet_copydata_in(odp_packet_t pkt, uint32_t offset, + uint32_t len, const void *src); + +/* + * + * Debugging + * ******************************************************** * - * @return New segment data length, or -1 on an error */ -int odp_packet_seg_push_tail(odp_packet_t pkt, odp_packet_seg_t seg, - size_t len); /** - * Pull in segment tail + * Print packet to the console * - * Decrease segment data length. - * Does not modify packet in case of an error. - * - * seg_data_len -= len + * Print all packet debug information to the console. * * @param pkt Packet handle - * @param seg Segment handle - * @param len Number of octets to pull tail (0 ... seg_data_len) - * - * @return New segment data length, or -1 on an error */ -int odp_packet_seg_pull_tail(odp_packet_t pkt, odp_packet_seg_t seg, - size_t len); +void odp_packet_print(odp_packet_t pkt); /** - * Tests if packet is valid + * Perform full packet validity check * - * Allows for more thorough checking than "if (pkt == ODP_PACKET_INVALID)" + * The operation may consume considerable number of cpu cycles depending on + * the check level. * * @param pkt Packet handle * - * @return 1 if valid, otherwise 0 + * @retval 0 Packet is not valid + * @retval 1 Packet is valid */ int odp_packet_is_valid(odp_packet_t pkt); + /** * @} */ diff --git a/platform/linux-generic/include/api/odp_platform_types.h b/platform/linux-generic/include/api/odp_platform_types.h index 2181eb6..2cfba87 100644 --- a/platform/linux-generic/include/api/odp_platform_types.h +++ b/platform/linux-generic/include/api/odp_platform_types.h @@ -35,28 +35,23 @@ typedef uint32_t odp_buffer_t; /** Invalid buffer */ #define ODP_BUFFER_INVALID (0xffffffff) +/** ODP buffer segment */ +typedef odp_buffer_t odp_buffer_seg_t; + +/** Invalid segment */ +#define ODP_SEGMENT_INVALID ODP_BUFFER_INVALID + /** ODP packet */ typedef odp_buffer_t odp_packet_t; /** Invalid packet */ #define ODP_PACKET_INVALID ODP_BUFFER_INVALID -/** Invalid offset */ -#define ODP_PACKET_OFFSET_INVALID ((uint32_t)-1) - /** ODP packet segment */ -typedef int odp_packet_seg_t; +typedef odp_buffer_t odp_packet_seg_t; /** Invalid packet segment */ -#define ODP_PACKET_SEG_INVALID -1 - -/** ODP packet segment info */ -typedef struct odp_packet_seg_info_t { - void *addr; /**< Segment start address */ - size_t size; /**< Segment maximum data size */ - void *data; /**< Segment data address */ - size_t data_len; /**< Segment data length */ -} odp_packet_seg_info_t; +#define ODP_PACKET_SEG_INVALID ODP_BUFFER_INVALID /** ODP packet IO handle */ typedef uint32_t odp_pktio_t; diff --git a/platform/linux-generic/include/odp_buffer_inlines.h b/platform/linux-generic/include/odp_buffer_inlines.h index 6227482..f880445 100644 --- a/platform/linux-generic/include/odp_buffer_inlines.h +++ b/platform/linux-generic/include/odp_buffer_inlines.h @@ -143,6 +143,61 @@ static inline void *buffer_map(odp_buffer_hdr_t *buf, return (void *)(seg_offset + (uint8_t *)buf->addr[seg_index]); } +static inline odp_buffer_seg_t segment_next(odp_buffer_hdr_t *buf, + odp_buffer_seg_t seg) +{ + odp_buffer_bits_t seghandle; + seghandle.u32 = seg; + + if (seg == ODP_SEGMENT_INVALID || + seghandle.prefix != buf->handle.prefix || + seghandle.seg >= buf->segcount - 1) + return ODP_SEGMENT_INVALID; + else { + seghandle.seg++; + return (odp_buffer_seg_t)seghandle.u32; + } +} + +static inline void *segment_map(odp_buffer_hdr_t *buf, + odp_buffer_seg_t seg, + uint32_t *seglen, + uint32_t limit, + uint32_t hr) +{ + uint32_t seg_offset, buf_left; + odp_buffer_bits_t seghandle; + uint8_t *seg_addr; + seghandle.u32 = seg; + + if (seghandle.prefix != buf->handle.prefix || + seghandle.seg >= buf->segcount) + return NULL; + + seg_addr = (uint8_t *)buf->addr[seghandle.seg]; + seg_offset = seghandle.seg * buf->segsize; + limit += hr; + + /* Can't map this segment if it's nothing but headroom or tailroom */ + if (hr >= seg_offset + buf->segsize || seg_offset > limit) + return NULL; + + /* Adjust address & offset if this segment contains any headroom */ + if (hr > seg_offset) { + seg_addr += hr % buf->segsize; + seg_offset += hr % buf->segsize; + } + + /* Set seglen if caller is asking for it */ + if (seglen != NULL) { + buf_left = limit - seg_offset; + *seglen = buf_left < buf->segsize ? buf_left : buf->segsize; + } + + return (void *)seg_addr; +} + + #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index bd5daf0..75fc6e6 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -226,6 +226,11 @@ static inline void packet_set_len(odp_packet_t pkt, uint32_t len) odp_packet_hdr(pkt)->frame_len = len; } +/* Forward declarations */ +int _odp_packet_copy_to_packet(odp_packet_t srcpkt, uint32_t srcoffset, + odp_packet_t dstpkt, uint32_t dstoffset, + uint32_t len); + odp_packet_t _odp_packet_alloc(odp_buffer_pool_t pool_hdl); int _odp_packet_parse(odp_packet_t pkt); diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index 89a5714..7490d52 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include @@ -356,7 +356,8 @@ odp_crypto_operation(odp_crypto_op_params_t *params, if (params->pkt != params->out_pkt) { if (odp_unlikely(ODP_PACKET_INVALID == params->out_pkt)) abort(); - odp_packet_copy(params->out_pkt, params->pkt); + _odp_packet_copy_to_packet(params->pkt, 0, params->out_pkt, 0, + odp_packet_len(params->pkt)); if (completion_event == odp_packet_to_buffer(params->pkt)) completion_event = odp_packet_to_buffer(params->out_pkt); diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 9f18540..0ab9866 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -37,12 +37,9 @@ odp_packet_t odp_packet_alloc(odp_buffer_pool_t pool_hdl, uint32_t len) odp_packet_t pkt = (odp_packet_t)buffer_alloc(pool_hdl, pool->s.params.buf_size); - if (pkt != ODP_PACKET_INVALID) { - odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); - uint32_t buf_size = pool->s.params.buf_size; - pkt_hdr->tailroom += buf_size; - pkt_hdr->frame_len -= buf_size; - } + if (pkt != ODP_PACKET_INVALID) + pull_tail(odp_packet_hdr(pkt), + pool->s.params.buf_size); return pkt; } @@ -178,6 +175,40 @@ void *odp_packet_pull_tail(odp_packet_t pkt, uint32_t len) return packet_map(pkt_hdr, pkt_hdr->frame_len, NULL); } +void *odp_packet_offset(odp_packet_t pkt, uint32_t offset, uint32_t *len, + odp_packet_seg_t *seg) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + void *addr = packet_map(pkt_hdr, offset, len); + + if (addr != NULL && seg != NULL) { + odp_buffer_bits_t seghandle; + seghandle.u32 = (uint32_t)pkt; + seghandle.seg = (pkt_hdr->headroom + offset) / + pkt_hdr->buf_hdr.segsize; + *seg = seghandle.handle; + } + + return addr; +} + +/* + * + * Meta-data + * ******************************************************** + * + */ + +odp_buffer_pool_t odp_packet_pool(odp_packet_t pkt) +{ + return odp_packet_hdr(pkt)->buf_hdr.pool_hdl; +} + +odp_pktio_t odp_packet_input(odp_packet_t pkt) +{ + return odp_packet_hdr(pkt)->input; +} + void *odp_packet_user_ptr(odp_packet_t pkt) { return odp_packet_hdr(pkt)->buf_hdr.buf_ctx; @@ -269,12 +300,240 @@ int odp_packet_is_segmented(odp_packet_t pkt) return odp_packet_hdr(pkt)->buf_hdr.segcount > 1; } - -int odp_packet_seg_count(odp_packet_t pkt) +int odp_packet_num_segs(odp_packet_t pkt) { return odp_packet_hdr(pkt)->buf_hdr.segcount; } +odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt) +{ + return (odp_packet_seg_t)pkt; +} + +odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + odp_buffer_bits_t seghandle; + + seghandle.u32 = (uint32_t)pkt; + seghandle.seg = pkt_hdr->buf_hdr.segcount - 1; + return seghandle.handle; +} + +odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, odp_packet_seg_t seg) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + return segment_next(&pkt_hdr->buf_hdr, seg); +} + +/* + * + * Segment level + * ******************************************************** + * + */ + +void *odp_packet_seg_buf_addr(odp_packet_t pkt, odp_packet_seg_t seg) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + return segment_map(&pkt_hdr->buf_hdr, seg, NULL, + pkt_hdr->headroom + pkt_hdr->frame_len, 0); +} + +uint32_t odp_packet_seg_buf_len(odp_packet_t pkt, + odp_packet_seg_t seg ODP_UNUSED) +{ + return odp_packet_hdr(pkt)->buf_hdr.segsize; +} + +void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + return segment_map(&pkt_hdr->buf_hdr, seg, NULL, + pkt_hdr->frame_len, pkt_hdr->headroom); +} + +uint32_t odp_packet_seg_data_len(odp_packet_t pkt, odp_packet_seg_t seg) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t seglen = 0; + + segment_map(&pkt_hdr->buf_hdr, seg, &seglen, + pkt_hdr->frame_len, pkt_hdr->headroom); + + return seglen; +} + +/* + * + * Manipulation + * ******************************************************** + * + */ + +odp_packet_t odp_packet_add_data(odp_packet_t pkt, uint32_t offset, + uint32_t len) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t pktlen = pkt_hdr->frame_len; + odp_packet_t newpkt; + + if (offset > pktlen) + return ODP_PACKET_INVALID; + + newpkt = odp_packet_alloc(pkt_hdr->buf_hdr.pool_hdl, pktlen + len); + + if (newpkt != ODP_PACKET_INVALID) { + if (_odp_packet_copy_to_packet(pkt, 0, + newpkt, 0, offset) != 0 || + _odp_packet_copy_to_packet(pkt, offset, newpkt, + offset + len, + pktlen - offset) != 0) { + odp_packet_free(newpkt); + newpkt = ODP_PACKET_INVALID; + } else { + odp_packet_hdr_t *new_hdr = odp_packet_hdr(newpkt); + new_hdr->buf_hdr.buf_u64 = pkt_hdr->buf_hdr.buf_u64; + odp_atomic_store_u32( + &new_hdr->buf_hdr.ref_count, + odp_atomic_load_u32( + &pkt_hdr->buf_hdr.ref_count)); + _odp_packet_parse(newpkt); + odp_packet_free(pkt); + } + } + + return newpkt; +} + +odp_packet_t odp_packet_rem_data(odp_packet_t pkt, uint32_t offset, + uint32_t len) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + uint32_t pktlen = pkt_hdr->frame_len; + odp_packet_t newpkt; + + if (offset > pktlen || offset + len > pktlen) + return ODP_PACKET_INVALID; + + newpkt = odp_packet_alloc(pkt_hdr->buf_hdr.pool_hdl, pktlen - len); + + if (newpkt != ODP_PACKET_INVALID) { + if (_odp_packet_copy_to_packet(pkt, 0, + newpkt, 0, offset) != 0 || + _odp_packet_copy_to_packet(pkt, offset + len, + newpkt, offset, + pktlen - offset - len) != 0) { + odp_packet_free(newpkt); + newpkt = ODP_PACKET_INVALID; + } else { + odp_packet_hdr_t *new_hdr = odp_packet_hdr(newpkt); + new_hdr->buf_hdr.buf_u64 = pkt_hdr->buf_hdr.buf_u64; + odp_atomic_store_u32( + &new_hdr->buf_hdr.ref_count, + odp_atomic_load_u32( + &pkt_hdr->buf_hdr.ref_count)); + _odp_packet_parse(newpkt); + odp_packet_free(pkt); + } + } + + return newpkt; +} + +/* + * + * Copy + * ******************************************************** + * + */ + +odp_packet_t odp_packet_copy(odp_packet_t pkt, odp_buffer_pool_t pool) +{ + odp_packet_hdr_t *srchdr = odp_packet_hdr(pkt); + uint32_t pktlen = srchdr->frame_len; + uint32_t meta_offset = ODP_FIELD_SIZEOF(odp_packet_hdr_t, buf_hdr); + odp_packet_t newpkt = odp_packet_alloc(pool, pktlen); + + if (newpkt != ODP_PACKET_INVALID) { + odp_packet_hdr_t *newhdr = odp_packet_hdr(newpkt); + uint8_t *newstart, *srcstart; + + /* Must copy meta data first, followed by packet data */ + newstart = (uint8_t *)newhdr + meta_offset; + srcstart = (uint8_t *)srchdr + meta_offset; + + memcpy(newstart, srcstart, + sizeof(odp_packet_hdr_t) - meta_offset); + + if (_odp_packet_copy_to_packet(pkt, 0, + newpkt, 0, pktlen) != 0) { + odp_packet_free(newpkt); + newpkt = ODP_PACKET_INVALID; + } + } + + return newpkt; +} + +int odp_packet_copydata_out(odp_packet_t pkt, uint32_t offset, + uint32_t len, void *dst) +{ + void *mapaddr; + uint32_t seglen = 0; /* GCC */ + uint32_t cpylen; + uint8_t *dstaddr = (uint8_t *)dst; + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + if (offset + len > pkt_hdr->frame_len) + return -1; + + while (len > 0) { + mapaddr = packet_map(pkt_hdr, offset, &seglen); + cpylen = len > seglen ? seglen : len; + memcpy(dstaddr, mapaddr, cpylen); + offset += cpylen; + dstaddr += cpylen; + len -= cpylen; + } + + return 0; +} + +int odp_packet_copydata_in(odp_packet_t pkt, uint32_t offset, + uint32_t len, const void *src) +{ + void *mapaddr; + uint32_t seglen = 0; /* GCC */ + uint32_t cpylen; + const uint8_t *srcaddr = (const uint8_t *)src; + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + + if (offset + len > pkt_hdr->frame_len) + return -1; + + while (len > 0) { + mapaddr = packet_map(pkt_hdr, offset, &seglen); + cpylen = len > seglen ? seglen : len; + memcpy(mapaddr, srcaddr, cpylen); + offset += cpylen; + srcaddr += cpylen; + len -= cpylen; + } + + return 0; +} + +/* + * + * Debugging + * ******************************************************** + * + */ + void odp_packet_print(odp_packet_t pkt) { int max_len = 512; @@ -306,41 +565,11 @@ void odp_packet_print(odp_packet_t pkt) ODP_PRINT("\n%s\n", str); } -int odp_packet_copy(odp_packet_t pkt_dst, odp_packet_t pkt_src) -{ - odp_packet_hdr_t *const pkt_hdr_dst = odp_packet_hdr(pkt_dst); - odp_packet_hdr_t *const pkt_hdr_src = odp_packet_hdr(pkt_src); - const size_t start_offset = ODP_FIELD_SIZEOF(odp_packet_hdr_t, buf_hdr); - uint8_t *start_src; - uint8_t *start_dst; - size_t len; - - if (pkt_dst == ODP_PACKET_INVALID || pkt_src == ODP_PACKET_INVALID) - return -1; - - if (pkt_hdr_dst->buf_hdr.size < pkt_hdr_src->frame_len) - return -1; - - /* Copy packet header */ - start_dst = (uint8_t *)pkt_hdr_dst + start_offset; - start_src = (uint8_t *)pkt_hdr_src + start_offset; - len = sizeof(odp_packet_hdr_t) - start_offset; - memcpy(start_dst, start_src, len); - - /* Copy frame payload */ - start_dst = (uint8_t *)odp_packet_data(pkt_dst); - start_src = (uint8_t *)odp_packet_data(pkt_src); - len = pkt_hdr_src->frame_len; - memcpy(start_dst, start_src, len); - - return 0; -} - int odp_packet_is_valid(odp_packet_t pkt) { - odp_buffer_t buf = odp_packet_to_buffer(pkt); + odp_buffer_hdr_t *buf = validate_buf((odp_buffer_t)pkt); - return odp_buffer_is_valid(buf); + return (buf != NULL && buf->type == ODP_BUFFER_TYPE_PACKET); } /* @@ -350,6 +579,38 @@ int odp_packet_is_valid(odp_packet_t pkt) * */ +int _odp_packet_copy_to_packet(odp_packet_t srcpkt, uint32_t srcoffset, + odp_packet_t dstpkt, uint32_t dstoffset, + uint32_t len) +{ + odp_packet_hdr_t *srchdr = odp_packet_hdr(srcpkt); + odp_packet_hdr_t *dsthdr = odp_packet_hdr(dstpkt); + void *srcmap; + void *dstmap; + uint32_t cpylen, minseg; + uint32_t srcseglen = 0; /* GCC */ + uint32_t dstseglen = 0; /* GCC */ + + if (srcoffset + len > srchdr->frame_len || + dstoffset + len > dsthdr->frame_len) + return -1; + + while (len > 0) { + srcmap = packet_map(srchdr, srcoffset, &srcseglen); + dstmap = packet_map(dsthdr, dstoffset, &dstseglen); + + minseg = dstseglen > srcseglen ? srcseglen : dstseglen; + cpylen = len > minseg ? minseg : len; + memcpy(dstmap, srcmap, cpylen); + + srcoffset += cpylen; + dstoffset += cpylen; + len -= cpylen; + } + + return 0; +} + odp_packet_t _odp_packet_alloc(odp_buffer_pool_t pool_hdl) { pool_entry_t *pool = odp_pool_to_entry(pool_hdl);