From patchwork Mon Jun 8 11:09:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Balasubramanian Manoharan X-Patchwork-Id: 49609 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f69.google.com (mail-wg0-f69.google.com [74.125.82.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 7AE9D21419 for ; Mon, 8 Jun 2015 11:10:17 +0000 (UTC) Received: by wgez8 with SMTP id z8sf31869219wge.2 for ; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:subject:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=s6gO02GAd1B75JrOd2mRj/BjaMvj5+QFGeHKHRRrvJo=; b=NmbuJJrxFTKq1xx/6jT8CwajMYo/L/opkpTF+6ktKl5F9/vFpVQggmslSIY3gugKD1 ybnQx0cLssec4Ol0/zGTtkmY4YxImPFwvAdamgvUBTsNQ7aCit9ezvLr+8Ux3LsRLZJL /nnjj4sZws6RWmQzgtpqI79SjRv+eEheRgqXz8Lypx9JozVfatxOETkftBwl1hikagdt XKfCXgkRB3tQ0jD/45Rd5/8N6ypR5LSTOXCZpYsxjvrNKdmSfhkebbyCkVPCSrdRWaO4 L5D5e+i0sGn9nsxrWW3hV/T7Jpk9LRpkJ/2plLFybIIAQVEhze0S8p9cpxnuGDM+Q89M 7EbQ== X-Gm-Message-State: ALoCoQlOQWcQw7gSs/WOQ88No+UDaWRFuko1YUps7IyX9GSb+EElFT9ew6DEb+nglTjSEzosMasm X-Received: by 10.152.203.233 with SMTP id kt9mr16613015lac.7.1433761816745; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.184.168 with SMTP id ev8ls322654lac.44.gmail; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) X-Received: by 10.152.6.69 with SMTP id y5mr15894579lay.72.1433761816547; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com. [209.85.217.171]) by mx.google.com with ESMTPS id au9si2420315lbc.168.2015.06.08.04.10.16 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Jun 2015 04:10:16 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.171 as permitted sender) client-ip=209.85.217.171; Received: by lbcue7 with SMTP id ue7so78372930lbc.0 for ; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) X-Received: by 10.152.4.137 with SMTP id k9mr15950156lak.29.1433761816360; Mon, 08 Jun 2015 04:10:16 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.108.230 with SMTP id hn6csp1781169lbb; Mon, 8 Jun 2015 04:10:14 -0700 (PDT) X-Received: by 10.55.20.19 with SMTP id e19mr30440827qkh.37.1433761813887; Mon, 08 Jun 2015 04:10:13 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id f3si2156546qgf.106.2015.06.08.04.10.12; Mon, 08 Jun 2015 04:10:13 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id DEED061CA0; Mon, 8 Jun 2015 11:10:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 1CB8361D13; Mon, 8 Jun 2015 11:10:02 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id E47F461E34; Mon, 8 Jun 2015 11:09:57 +0000 (UTC) Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) by lists.linaro.org (Postfix) with ESMTPS id 8F86A61CEA for ; Mon, 8 Jun 2015 11:09:54 +0000 (UTC) Received: by pdjm12 with SMTP id m12so102230199pdj.3 for ; Mon, 08 Jun 2015 04:09:53 -0700 (PDT) X-Received: by 10.66.142.42 with SMTP id rt10mr28315231pab.142.1433761793731; Mon, 08 Jun 2015 04:09:53 -0700 (PDT) Received: from manoharan-Vostro-2520.domain.name ([117.207.78.186]) by mx.google.com with ESMTPSA id zf1sm2270215pbc.43.2015.06.08.04.09.51 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 08 Jun 2015 04:09:52 -0700 (PDT) From: bala.manoharan@linaro.org To: lng-odp@lists.linaro.org Date: Mon, 8 Jun 2015 16:39:42 +0530 Message-Id: <1433761782-7088-1-git-send-email-bala.manoharan@linaro.org> X-Mailer: git-send-email 2.0.1.472.g6f92e5f X-Topics: Classification Subject: [lng-odp] [ODP/RFC 1/2] linux-generic: egress classification implementation X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bala.manoharan@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.171 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Balasubramanian Manoharan This is linux-generic implementation of egress classification. This is a lock-less implementation for output packet scheduling, shaping and rate limitting. Multiple packet output queues with different priority values can be created and attached with pktio interface and the packet enqueued into the output queues will be scheduled based on the priority value. Committed Information Rate, Committed Burst Size, Peak Information Rate and Peak Burst Size can be set individually on each packet output queues. Rate limitting can also be set on a pktio interface. The implementation supports multiple hierarchy level of packet scheduling but since the current APIs are defined only for a single level, this version of the implementation supports only one hierarchy level. Once the APIs are finalized for multiple hierarchy level the same will be incorporated. Signed-off-by: Balasubramanian Manoharan --- include/odp.h | 1 + include/odp/api/pktout.h | 276 +++++++++++++ platform/linux-generic/Makefile.am | 4 + platform/linux-generic/include/odp/pktout.h | 30 ++ .../linux-generic/include/odp/plat/pktout_types.h | 44 ++ platform/linux-generic/include/odp_internal.h | 3 + .../linux-generic/include/odp_packet_io_internal.h | 12 +- .../include/odp_packet_out_internal.h | 164 ++++++++ platform/linux-generic/odp_init.c | 5 + platform/linux-generic/odp_packet_io.c | 49 ++- platform/linux-generic/odp_packet_out.c | 454 +++++++++++++++++++++ 11 files changed, 1027 insertions(+), 15 deletions(-) create mode 100644 include/odp/api/pktout.h create mode 100644 platform/linux-generic/include/odp/pktout.h create mode 100644 platform/linux-generic/include/odp/plat/pktout_types.h create mode 100644 platform/linux-generic/include/odp_packet_out_internal.h create mode 100644 platform/linux-generic/odp_packet_out.c diff --git a/include/odp.h b/include/odp.h index 2bac510..b8df7d3 100644 --- a/include/odp.h +++ b/include/odp.h @@ -38,6 +38,7 @@ extern "C" { #include #include #include +#include #include #include #include diff --git a/include/odp/api/pktout.h b/include/odp/api/pktout.h new file mode 100644 index 0000000..0d64a61 --- /dev/null +++ b/include/odp/api/pktout.h @@ -0,0 +1,276 @@ +/* Copyright (c) 2015, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP packet descriptor + */ + +#ifndef ODP_API_PACKET_OUT_H_ +#define ODP_API_PACKET_OUT_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include + +/** + * Number of packet output scheduling priorities per pktio interface + * This function returns the number of pktout scheduling priority + * supported in the platform per pktio interface. + * + * @param[in] pktio pktio handle + * + * @retval Number of packet output scheduling priorities + * @retval 1 if the pktio does not support pktout priority + */ +int odp_pktout_num_prio(odp_pktio_t pktio); + +/** +* Gets the total number of hierarchy level supported by the platform +* for output packet shaping and scheduling +* +* @return Total number of hierarchy supported +*/ +uint8_t odp_pktout_num_heirarchy(void); + +/* Get Maximum nuber of packet output queues that can be attached with +* the given pktio interface. This will be the total number of packet output +* queues supported by the pktio interface. This number is different from the +* pktout node which are points in the hierarchial tree. +* +* @param[in] pktio pktio handle +* +* @retval Number of output queues that can be attached to the +* given pktio handle +* @retval 1 if the pktout scheduling is not supported +*/ +uint32_t odp_pktout_max_queues(odp_pktio_t pktio); + +/* Assign outq to pktio interface +* Sets the outq into the pktio interface, Any queues that maybe previously +* associated with this pktio will be superseded by this queue array. +* The number of queues associate in the pktio interface must not exceed the +* maximum supported output queue given by odp_pktout_max_queues() function +* for the same pktio interface. +* Calling this function with num_queues set to zero will disassociate all +* the queues that are linked to this pktio interface +* +* @param[in] pktio pktio handle +* @param[in] queue_hdl[] Array of queue handles +* @param[in] num_queues Number of queue handles in the array +* +* @return Number of queues associated +* <0 in case of failure +*/ +int odp_pktout_set_queues(odp_pktio_t pktio, + odp_pktout_queue_t queue_hdl[], + uint32_t num_queues); + +/* +* Get the packet output queues which are associated with the +* pktio interface. This function is used to read the list of +* queues which are currently associated with the given pktio interface +* +* @param[in] pktio pktio handle +* @param[out] queue_hdl Array of queue handle +* @param[in/out] queue_elements During input will be the size of the queue +* array and output will be the number of +* elements written to queue_hdl +* +* @return 0 on success +* @retval <0 on failure +*/ +int odp_pktout_queues(odp_pktio_t pktio, + odp_pktout_queue_t queue_hdl[], + uint32_t *queue_elements); + +#define MAX_PKTOUT_PRIO 10 + +/** Pktio PKTOUT queue parameters +* @note: The below parameter is just a placeholder and the actual +* signature of the parameter will be modified once hierarchial +* information is added to the parameters. +*/ +typedef struct odp_pktout_queue_param_t { + int num; /**< number of queues */ + odp_schedule_prio_t prio[MAX_PKTOUT_PRIO]; /**< priority value */ +} odp_pktout_queue_param_t; + +/* Creates the outq for the given pktio interface +* This function overrides any previously associated queues with the interface +* The queues created using this function will be of type ODP_QUEUE_TYE_PKTOUT +* @param[in] pktio_id pktio handle +* @param[in] qparam parameter for queue creation +* @param[out] queue[] Array of created queue handles +* +* @return Number of queues created +* <0 on failure +* +*/ +int odp_pktout_queue_create(odp_pktio_t pktio_id, + odp_pktout_queue_param_t param, + odp_pktout_queue_t queue[]); + +/** Packet out queue level parameters **/ + +/** +* Get packet output queue priority +* Returns the priority of the packet output queue +* +* @param[in] queue_hdl Packet out queue handle +* +* @return Priority value of the input queue +* @retval <0 on failure +*/ +int odp_pktout_queue_priority(odp_pktout_queue_t queue_hdl); + +/** +* Sets the queue limit in terms of number of packets. +* This limit determines the amount of data that a queue is allowed to accept. +* The queue will not receive any packet beyond the set limit and will return +* an error value. +* +* @param[in] queue_hdl Queue Handle +* @param[in] pkt_cnt Maximum number of packets the queue may contain +* +* @return Packet limit set +* @retval <0 on failure +* +* @note This function is called only for ODP_QUEUE_TYPE_PKTOUT +**/ +int odp_pktout_queue_limit_set(odp_pktout_queue_t queue_hdl, uint32_t pkt_cnt); + +/** +* Gets the number of packets limit set on the queue +* This function is used to get the limit in terms of number of packet which is +* set on the given queue +* +* @param[in] queue_hdl Queue Handle +* +* @return Packet limit set +* @retval <0 on failure +* +* @note This function is called only for ODP_QUEUE_TYPE_PKTOUT +**/ +int odp_pktout_queue_limit(odp_pktout_queue_t queue_hdl); + +/** +* Queue depth in terms of the number of packets +* Gets the current depth of the queue in terms of the number of packets +* which can be used by the application to find the current load on the queue +* +* @param[in] queue_hdl Queue handle +* +* @return Current packets count in the queue +* @retval <0 on failure +* +* @note This function is called only for ODP_QUEUE_TYPE_PKTOUT +**/ +int odp_pktout_queue_packets(odp_pktout_queue_t queue_hdl); + +/** +* Queue depth in terms of the number of octets +* Gets the current depth of the queue in terms of the number of Octets +* which can be used by the application to find the current load on the queue +* +* @param[in] queue_hdl Queue handle +* +* @return Current octect count in the queue +* @retval <0 on failure +* @note This function is called only for ODP_QUEUE_TYPE_PKTOUT +*/ +int odp_pktout_queue_octets(odp_pktout_queue_t queue_hdl); + +/** +* Set Committed Information Rate and Committed Burst size +* sets cir and cbs on the packet out queue. +* cir is set in kbps and CBS is in bytes. +* +* @param[in] queue_hdl Queue handle +* @param[in] rate_kbps CIR rate in kbps +* @param[in] burst_bytes CBS in bytes +* +* @retval 0 on success +* @retval <0 on failure +**/ +int odp_pktout_cir_set(odp_pktout_queue_t queue_hdl, + uint32_t rate_kbps, uint32_t burst_bytes); + +/** +* Set Peak Information Rate and Peak Burst size +* sets pir and pbs on the packet out queue. +* pir is set in kbps and PBS is in bytes. +* +* @param[in] queue_hdl Queue handle +* @param[in] rate_kbps PIR rate in kbps +* @param[in] burst_bytes PBS in bytes +* +* @retval 0 on success +* @retval <0 on failure +**/ + +int odp_pktout_pir_set(odp_pktout_queue_t queue_hdl, + uint32_t rate_kbps, uint32_t burst_bytes); + +/** +* Port level rate limitting +* sets the rate limiting and burst size at the interface level +* This value will be equal to the line rate +* +* @param[in] pktio Pktio handle +* @param[in] rate_kbps Port level rate in kbps +* @param[in] burst_bytes Port level burst bytes +* +* @retval 0 on success +* @retval <0 on failure +*/ +int odp_pktio_rate_set(odp_pktio_t pktio, uint32_t rate_kbps, + uint32_t burst_bytes); + +/** +* Packet output queue enq +* Enqueues the packet into odp_pktout_queue_t queue. +* +* @param[in] pkt Packet handle +* @param[in] queue Packet output queue handle +* +* @retval 0 Success +* @retval <0 Failure +*/ +int odp_pktout_enq(odp_pktout_queue_t queue, odp_packet_t pkt); + +/** +* Packet output queue enqueue with feedback +* Enqueues the packet into odp_pktout_queue_t queue. +* Additionally this function returns the depth of the packet output queue +* interms of the number of packets before this packet is enqueued. +* +* @retval >0 Number of Packets to the packet limit, before this packet. +* @retval 0 Queue full. +* @retval <0 Failure +*/ +int32_t odp_pktout_enq_depth(odp_pktout_queue_t queue, odp_packet_t pkt); + +/** +* Packet output queue priority +* +* @param queue Packet output queue handle +* +* @return Packet output queue schedule priority +*/ +odp_schedule_prio_t odp_pktout_sched_prio(odp_pktout_queue_t queue); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index 66f0474..ddb8c7c 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -4,6 +4,7 @@ include $(top_srcdir)/platform/Makefile.inc AM_CFLAGS += -I$(srcdir)/include AM_CFLAGS += -I$(top_srcdir)/include AM_CFLAGS += -I$(top_srcdir)/helper/include +AM_CFLAGS += -g SUBDIRS = test @@ -58,6 +59,7 @@ odpplatinclude_HEADERS = \ $(top_srcdir)/platform/linux-generic/include/odp/plat/crypto_types.h \ $(top_srcdir)/platform/linux-generic/include/odp/plat/event_types.h \ $(top_srcdir)/platform/linux-generic/include/odp/plat/packet_types.h \ + $(top_srcdir)/platform/linux-generic/include/odp/plat/pktout_types.h \ $(top_srcdir)/platform/linux-generic/include/odp/plat/packet_io_types.h \ $(top_srcdir)/platform/linux-generic/include/odp/plat/pool_types.h \ $(top_srcdir)/platform/linux-generic/include/odp/plat/queue_types.h \ @@ -91,6 +93,7 @@ odpapiinclude_HEADERS = \ $(top_srcdir)/include/odp/api/packet.h \ $(top_srcdir)/include/odp/api/packet_flags.h \ $(top_srcdir)/include/odp/api/packet_io.h \ + $(top_srcdir)/include/odp/api/pktout.h \ $(top_srcdir)/include/odp/api/pool.h \ $(top_srcdir)/include/odp/api/queue.h \ $(top_srcdir)/include/odp/api/random.h \ @@ -153,6 +156,7 @@ __LIB__libodp_la_SOURCES = \ odp_impl.c \ ../../helper/linux.c \ odp_packet.c \ + odp_packet_out.c \ odp_packet_flags.c \ odp_packet_io.c \ odp_packet_socket.c \ diff --git a/platform/linux-generic/include/odp/pktout.h b/platform/linux-generic/include/odp/pktout.h new file mode 100644 index 0000000..16628f0 --- /dev/null +++ b/platform/linux-generic/include/odp/pktout.h @@ -0,0 +1,30 @@ +/* Copyright (c) 2015, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP queue + */ + +#ifndef ODP_PLAT_PKTOUT_H_ +#define ODP_PLAT_PKTOUT_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +#include + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/platform/linux-generic/include/odp/plat/pktout_types.h b/platform/linux-generic/include/odp/plat/pktout_types.h new file mode 100644 index 0000000..2902eef --- /dev/null +++ b/platform/linux-generic/include/odp/plat/pktout_types.h @@ -0,0 +1,44 @@ +/* Copyright (c) 2015, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP Packet output queue + */ + +#ifndef ODP_PKTOUT_TYPES_H_ +#define ODP_PKTOUT_TYPES_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include + +typedef ODP_HANDLE_T(odp_pktout_queue_t); + +#define ODP_PKTOUT_INVALID _odp_cast_scalar(odp_pktout_queue_t, 0) + +#define MAX_PKTOUT_QUEUE 10 + +#define PKTIO_MAX_PRIORITY 10 + +#define ODP_PKTOUT_QUEUES 1024 + +#define ODP_CONFID_PKTOUT_PRIOS 10 + +#define PKTOUT_QUEUE_PER_PRIO 10 +/** + * @} + */ + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/platform/linux-generic/include/odp_internal.h b/platform/linux-generic/include/odp_internal.h index 8c5d339..186906f 100644 --- a/platform/linux-generic/include/odp_internal.h +++ b/platform/linux-generic/include/odp_internal.h @@ -64,6 +64,9 @@ int odp_classification_term_global(void); int odp_queue_init_global(void); int odp_queue_term_global(void); +int odp_pktout_init_global(void); +int odp_pktout_term_global(void); + int odp_crypto_init_global(void); int odp_crypto_term_global(void); diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index 18b59ef..367d1e2 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -23,10 +23,12 @@ extern "C" { #include #include #include - +#include +#include #include #include #include +#include /** * Packet IO types @@ -53,6 +55,12 @@ struct pktio_entry { char name[IFNAMSIZ]; /**< name of pktio provided to pktio_open() */ odp_bool_t promisc; /**< promiscuous mode state */ + int max_pktout_queues; + int priority; + pktout_node_entry_t root_node; + odp_pktout_queue_t pktout[MAX_PKTOUT_QUEUE]; + odph_linux_pthread_t pktout_thread; + int num_hierarchy; }; typedef union { @@ -88,6 +96,8 @@ static inline pktio_entry_t *get_pktio_entry(odp_pktio_t pktio) int pktin_poll(pktio_entry_t *entry); +int __odp_pktio_send(pktio_entry_t *entry, odp_packet_t pkt_table[], int len); + #ifdef __cplusplus } #endif diff --git a/platform/linux-generic/include/odp_packet_out_internal.h b/platform/linux-generic/include/odp_packet_out_internal.h new file mode 100644 index 0000000..f68fb6e --- /dev/null +++ b/platform/linux-generic/include/odp_packet_out_internal.h @@ -0,0 +1,164 @@ +/* Copyright (c) 2013, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef ODP_PKTOUT_INTERNAL_H_ +#define ODP_PKTOUT_INTERNAL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include +#include +#include +#include + +#define USE_TICKETLOCK + +#ifdef USE_TICKETLOCK +#include +#else +#include +#endif + +#define QUEUE_MULTI_MAX 8 + +#define QUEUE_STATUS_FREE 0 +#define QUEUE_STATUS_DESTROYED 1 +#define QUEUE_STATUS_READY 2 +#define QUEUE_STATUS_NOTSCHED 3 +#define QUEUE_STATUS_SCHED 4 + +#define QUEUE_CIR_KBPS 0 /* 0kbps */ +#define QUEUE_CBS_BYTE 2000 /* 2000 bytes */ +#define QUEUE_PIR_KBPS 0 /* 0 kbps */ +#define QUEUE_PBS_BYTE 4000 /* 4000 bytes */ + +#define NODE_CIR_KBPS 20 /* 20 kbps */ +#define NODE_CBS_BYTE 4000 /* 4000 bytes */ +#define NODE_PIR_KBPS 40 /* 40 kbps */ +#define NODE_PBS_BYTE 6000 /* 4000 bytes */ + +typedef struct pktout_shaper_param_s { + odp_atomic_u64_t cir_kbps; + odp_atomic_u64_t cbs_byte; + odp_atomic_u64_t cur_cir; + odp_atomic_u64_t pir_kbps; + odp_atomic_u64_t pbs_byte; + odp_atomic_u64_t cur_pir; + odp_atomic_u64_t time; +} pktout_shaper_param_t; + +struct pktout_entry_s { +#ifdef USE_TICKETLOCK + odp_ticketlock_t lock ODP_ALIGNED_CACHE; +#else + odp_spinlock_t lock ODP_ALIGNED_CACHE; +#endif + + int status; + odp_pktout_queue_t handle; + odph_ring_t *ring; + odp_pktout_queue_param_t param; + int priority; + odp_pktio_t pktio; + char name[ODP_QUEUE_NAME_LEN]; + pktout_shaper_param_t shaper; + uint32_t limit_pkt; + uint32_t limit_oct; +}; + +typedef union pktout_entry_u { + struct pktout_entry_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(struct pktout_entry_s))]; +} pktout_entry_t; + +struct pktout_node_entry_s { + int num_child; + int is_leaf; + int is_root; + int priority; + pktout_shaper_param_t shaper; + uint32_t pri_count[ODP_CONFID_PKTOUT_PRIOS]; + void *pri_queue[ODP_CONFID_PKTOUT_PRIOS][PKTOUT_QUEUE_PER_PRIO]; +}; + +typedef union pktout_node_entry_u { + struct pktout_node_entry_s s; + uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof + (struct pktout_node_entry_s))]; +} pktout_node_entry_t; + +pktout_entry_t *get_pktout_qentry(uint32_t pktout_id); + +odp_packet_t deq_pktout_queue(pktout_entry_t *queue, uint64_t *traffic); + +odp_packet_t deq_pktout_node(pktout_node_entry_t *node, uint64_t *traffic); + +odp_packet_t __deq_pktout_node(pktout_node_entry_t *node, uint64_t *traffic); + +void *__send_packet(void *args); + +static inline uint32_t pktout_to_id(odp_pktout_queue_t handle) +{ + return _odp_typeval(handle) - 1; +} + +static inline odp_pktout_queue_t pktout_from_id(uint32_t pktout_id) +{ + return _odp_cast_scalar(odp_pktout_queue_t, pktout_id + 1); +} + +int pktout_schedule_queue_init(pktout_entry_t *pktout, + pktout_node_entry_t *node); + +typedef odp_packet_t (*deq_pktout_node_func_t)(pktout_node_entry_t *node); +typedef odp_packet_t (*deq_pktout_queue_func_t)(pktout_entry_t *queue); + +static inline void pktout_init(pktout_entry_t *queue) +{ + queue->s.pktio = ODP_PKTIO_INVALID; + queue->s.limit_pkt = 1024; + queue->s.limit_oct = 1024 * 524; + odp_atomic_init_u64(&queue->s.shaper.cir_kbps, QUEUE_CIR_KBPS * 1000); + odp_atomic_init_u64(&queue->s.shaper.cbs_byte, QUEUE_CBS_BYTE * 1000); + odp_atomic_init_u64(&queue->s.shaper.pir_kbps, QUEUE_PIR_KBPS * 1000); + odp_atomic_init_u64(&queue->s.shaper.pbs_byte, QUEUE_PBS_BYTE * 1000); + odp_atomic_init_u64(&queue->s.shaper.cur_cir, 0); + odp_atomic_init_u64(&queue->s.shaper.cur_pir, 0); + odp_atomic_init_u64(&queue->s.shaper.time, time(NULL)); + queue->s.ring = odph_ring_create(queue->s.name, queue->s.limit_pkt, 0); +} + +static inline void pktout_node_init(pktout_node_entry_t *node) +{ + node->s.num_child = 0; + node->s.is_leaf = 0; + node->s.priority = PKTIO_MAX_PRIORITY; + int i, j; + + odp_atomic_init_u64(&node->s.shaper.cir_kbps, NODE_CIR_KBPS * 1000); + odp_atomic_init_u64(&node->s.shaper.cbs_byte, NODE_CBS_BYTE); + odp_atomic_init_u64(&node->s.shaper.pir_kbps, NODE_PIR_KBPS * 1000); + odp_atomic_init_u64(&node->s.shaper.pbs_byte, NODE_PBS_BYTE); + odp_atomic_init_u64(&node->s.shaper.cur_cir, 0); + odp_atomic_init_u64(&node->s.shaper.cur_pir, 0); + odp_atomic_init_u64(&node->s.shaper.time, time(NULL)); + for (i = 0; i < ODP_CONFID_PKTOUT_PRIOS; i++) { + node->s.pri_count[i] = 0; + for (j = 0; j < PKTOUT_QUEUE_PER_PRIO; j++) + node->s.pri_queue[i][j] = NULL; + } +} + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index bf36e68..1f1a55f 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -52,6 +52,11 @@ int odp_init_global(odp_init_t *params, return -1; } + if (odp_pktout_init_global()) { + ODP_ERR("ODP packet out init failed.\n"); + return -1; + } + if (odp_pktio_init_global()) { ODP_ERR("ODP packet io init failed.\n"); return -1; diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index 5ae24b9..765f763 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -76,6 +77,8 @@ int odp_pktio_init_global(void) queue_entry = queue_to_qentry(qid); queue_entry->s.pktout = _odp_cast_scalar(odp_pktio_t, id); + pktio_entry->s.root_node.s.is_root = 1; + pktout_node_init(&pktio_entry->s.root_node); } return 0; @@ -244,6 +247,7 @@ static odp_pktio_t setup_pktio_entry(const char *dev, odp_pool_t pool) odp_pktio_t id; pktio_entry_t *pktio_entry; int ret; + int cpu; if (strlen(dev) >= IFNAMSIZ) { /* ioctl names limitation */ @@ -278,7 +282,19 @@ static odp_pktio_t setup_pktio_entry(const char *dev, odp_pool_t pool) unlock_entry_classifier(pktio_entry); } + pktio_entry->s.priority = PKTIO_MAX_PRIORITY; + pktio_entry->s.handle = id; + odp_cpumask_t cpumask; + odp_cpumask_t mask; + /* These functions are planned to be modified in the new API + version and the same will be reflected once in the repo */ + odph_linux_cpumask_default(&cpumask, 1); + odp_cpumask_zero(&mask); + cpu = odp_cpumask_first(&cpumask); + odp_cpumask_set(&mask, cpu); + odph_linux_pthread_create(&pktio_entry->s.pktout_thread, &mask, + __send_packet, pktio_entry); return id; } @@ -297,7 +313,6 @@ odp_pktio_t odp_pktio_open(const char *dev, odp_pool_t pool) odp_spinlock_lock(&pktio_tbl->lock); id = setup_pktio_entry(dev, pool); odp_spinlock_unlock(&pktio_tbl->lock); - return id; } @@ -441,33 +456,39 @@ static int enq_loopback(pktio_entry_t *pktio_entry, odp_packet_t pkt_tbl[], int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], int len) { - pktio_entry_t *pktio_entry = get_pktio_entry(id); - int pkts; + pktio_entry_t *entry = get_pktio_entry(id); - if (pktio_entry == NULL) + if (!entry) return -1; + return __odp_pktio_send(entry, pkt_table, len); +} - lock_entry(pktio_entry); - switch (pktio_entry->s.type) { +int __odp_pktio_send(pktio_entry_t *entry, odp_packet_t pkt_table[], int len) +{ + int pkts; + + /* todo further optimization required in these functions */ + lock_entry(entry); + switch (entry->s.type) { case ODP_PKTIO_TYPE_SOCKET_BASIC: - pkts = send_pkt_sock_basic(&pktio_entry->s.pkt_sock, - pkt_table, len); + pkts = send_pkt_sock_basic(&entry->s.pkt_sock, + pkt_table, len); break; case ODP_PKTIO_TYPE_SOCKET_MMSG: - pkts = send_pkt_sock_mmsg(&pktio_entry->s.pkt_sock, - pkt_table, len); + pkts = send_pkt_sock_mmsg(&entry->s.pkt_sock, + pkt_table, len); break; case ODP_PKTIO_TYPE_SOCKET_MMAP: - pkts = send_pkt_sock_mmap(&pktio_entry->s.pkt_sock_mmap, - pkt_table, len); + pkts = send_pkt_sock_mmap(&entry->s.pkt_sock_mmap, + pkt_table, len); break; case ODP_PKTIO_TYPE_LOOPBACK: - pkts = enq_loopback(pktio_entry, pkt_table, len); + pkts = enq_loopback(entry, pkt_table, len); break; default: pkts = -1; } - unlock_entry(pktio_entry); + unlock_entry(entry); return pkts; } diff --git a/platform/linux-generic/odp_packet_out.c b/platform/linux-generic/odp_packet_out.c new file mode 100644 index 0000000..b4d262a --- /dev/null +++ b/platform/linux-generic/odp_packet_out.c @@ -0,0 +1,454 @@ +/* Copyright (c) 2015, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef USE_TICKETLOCK +#include +#define LOCK(a) odp_ticketlock_lock(a) +#define UNLOCK(a) odp_ticketlock_unlock(a) +#define LOCK_INIT(a) odp_ticketlock_init(a) +#else +#include +#define LOCK(a) odp_spinlock_lock(a) +#define UNLOCK(a) odp_spinlock_unlock(a) +#define LOCK_INIT(a) odp_spinlock_init(a) +#endif + +/* todo redefine this */ +#define MAX_BURST 10 + +typedef struct queue_table_t { + pktout_entry_t pktout_queue[ODP_PKTOUT_QUEUES]; +} pktout_queue_table_t; + +static pktout_queue_table_t *pktout_tbl; + +pktout_entry_t *get_pktout_qentry(uint32_t queue_id) +{ + return &pktout_tbl->pktout_queue[queue_id]; +} + +int odp_pktout_init_global(void) +{ + uint32_t i; + odp_shm_t shm; + + ODP_DBG("Queue init ... "); + + shm = odp_shm_reserve("odp_pktout_queues", + sizeof(pktout_queue_table_t), + sizeof(pktout_entry_t), 0); + + pktout_tbl = odp_shm_addr(shm); + + if (!pktout_tbl) + return -1; + + memset(pktout_tbl, 0, sizeof(pktout_queue_table_t)); + + for (i = 0; i < ODP_PKTOUT_QUEUES; i++) { + /* init locks */ + pktout_entry_t *pktout; + + pktout = get_pktout_qentry(i); + LOCK_INIT(&pktout->s.lock); + + pktout->s.handle = pktout_from_id(i); + } + + ODP_DBG("done\n"); + ODP_DBG("Pktout queue init global\n"); + ODP_DBG(" struct pktout_entry_s size %zu\n", + sizeof(struct pktout_entry_s)); + ODP_DBG(" pktout_entry_t size %zu\n", + sizeof(pktout_entry_t)); + ODP_DBG("\n"); + /** Init odp ring attached with pktout queues */ + odph_ring_tailq_init(); + return 0; +} + +int pktout_schedule_queue_init(pktout_entry_t *pktout, + pktout_node_entry_t *node) +{ + int priority; + int pri_count; + + node->s.is_leaf = 1; + priority = pktout->s.priority; + pri_count = node->s.pri_count[priority]++; + node->s.pri_queue[priority][pri_count] = pktout; + return 0; +} + +int odp_pktout_num_prio(odp_pktio_t pktio) +{ + pktio_entry_t *entry; + + entry = get_pktio_entry(pktio); + if (!entry) + return -1; + + return entry->s.priority; +} + +uint8_t odp_pktout_num_heirarchy(void) +{ + /** Linux-generic implementation can support multiple hierarchy + but the current API version only supports only hierarchy level*/ + return 1; +} + +uint32_t odp_pktout_max_queues(odp_pktio_t pktio) +{ + pktio_entry_t *entry = get_pktio_entry(pktio); + + return entry->s.max_pktout_queues; +} + +int odp_pktout_queue_create(odp_pktio_t pktio_id, + odp_pktout_queue_param_t param, + odp_pktout_queue_t queue_hdl[]) +{ + uint32_t i; + int j; + pktout_entry_t *queue; + odp_pktout_queue_t handle = ODP_PKTOUT_INVALID; + pktio_entry_t *entry = get_pktio_entry(pktio_id); + pktout_node_entry_t *node = &entry->s.root_node; + + for (j = 0; j < param.num; j++) { + for (i = 0; i < ODP_PKTOUT_QUEUES; i++) { + queue = &pktout_tbl->pktout_queue[i]; + + if (queue->s.status != QUEUE_STATUS_FREE) + continue; + + LOCK(&queue->s.lock); + if (queue->s.status == QUEUE_STATUS_FREE) { + sprintf(queue->s.name, "%s%d", + "pktout", (j + i)); + pktout_init(queue); + + queue->s.priority = param.prio[j]; + + queue->s.status = QUEUE_STATUS_READY; + queue->s.pktio = pktio_id; + + handle = queue->s.handle; + UNLOCK(&queue->s.lock); + break; + } + UNLOCK(&queue->s.lock); + } + if (i != ODP_PKTOUT_QUEUES) + queue_hdl[j] = handle; + else + break; + + if (handle != ODP_PKTOUT_INVALID) { + if (pktout_schedule_queue_init(queue, node)) { + ODP_ERR("pktout schedule queue init failed\n"); + return -1; + } + } + } + + return j; +} + +int odp_pktout_queue_limit_set(odp_pktout_queue_t queue_hdl, uint32_t pkt_cnt) +{ + uint32_t id; + pktout_entry_t *entry; + + id = pktout_to_id(queue_hdl); + entry = get_pktout_qentry(id); + entry->s.limit_pkt = pkt_cnt; + /* current implementation uses a fixed limit and once API + gets finalized the implementation will use the value set + by this function */ + return 1; +} + +int odp_pktout_queue_limit(odp_pktout_queue_t queue_hdl) +{ + uint32_t id; + pktout_entry_t *entry; + + id = pktout_to_id(queue_hdl); + entry = get_pktout_qentry(id); + return entry->s.limit_pkt; +} + +int odp_pktout_queue_priority(odp_pktout_queue_t queue_hdl) +{ + uint32_t id; + pktout_entry_t *entry; + + id = pktout_to_id(queue_hdl); + entry = get_pktout_qentry(id); + return entry->s.priority; +} + +int odp_pktout_queue_packets(odp_pktout_queue_t queue_hdl) +{ + uint32_t id; + pktout_entry_t *queue; + + id = pktout_to_id(queue_hdl); + queue = get_pktout_qentry(id); + return odph_ring_count(queue->s.ring); +} + +int odp_pktout_queue_octets(odp_pktout_queue_t queue_hdl ODP_UNUSED) +{ + return 0; +} + +odp_packet_t __deq_pktout_node(pktout_node_entry_t *node, uint64_t *traffic) +{ + int i, j; + int priority; + int num_queue; + odp_packet_t pkt; + pktout_node_entry_t *child; + pktout_entry_t *queue; + + pkt = ODP_PACKET_INVALID; + priority = node->s.priority; + + for (i = 0; i < priority; i++) { + num_queue = node->s.pri_count[i]; + for (j = 0; j < num_queue; j++) { + if (node->s.is_leaf) { + queue = node->s.pri_queue[i][j]; + pkt = deq_pktout_queue(queue, traffic); + } else { + child = node->s.pri_queue[i][j]; + pkt = deq_pktout_node(child, traffic); + } + if (pkt != ODP_PACKET_INVALID) + break; + } + if (pkt != ODP_PACKET_INVALID) + break; + } + return pkt; +} + +odp_packet_t deq_pktout_node(pktout_node_entry_t *node, uint64_t *traffic) +{ + odp_packet_t pkt = ODP_PACKET_INVALID; + + uint64_t cur_time; + uint64_t node_time; + uint64_t cir; + uint64_t pir; + + cur_time = time(NULL); + node_time = odp_atomic_load_u64(&node->s.shaper.time); + cir = odp_atomic_load_u64(&node->s.shaper.cir_kbps); + pir = odp_atomic_load_u64(&node->s.shaper.pir_kbps); + + if (node_time < cur_time) { + /* this snippet reached once every second and resets the + cir and pir values*/ + if (odp_atomic_load_u64(&node->s.shaper.cur_cir) > cir) + odp_atomic_sub_u64(&node->s.shaper.cur_cir, cir); + else + odp_atomic_store_u64(&node->s.shaper.cur_cir, 0); + + if (odp_atomic_load_u64(&node->s.shaper.cur_pir) > pir) + odp_atomic_sub_u64(&node->s.shaper.cur_pir, pir); + else + odp_atomic_store_u64(&node->s.shaper.cur_pir, 0); + + odp_atomic_store_u64(&node->s.shaper.time, cur_time); + if (node->s.is_root) + *traffic = 0; + } + + /* in the input if *traffic is not zero then the queues are + requested to dispatch the peak traffic. in the output this + *traffic denotes the data dequeued in bits */ + + if (odp_atomic_load_u64(&node->s.shaper.cur_cir) < cir) { + pkt = __deq_pktout_node(node, traffic); + odp_atomic_add_u64(&node->s.shaper.cur_cir, *traffic); + odp_atomic_add_u64(&node->s.shaper.cur_pir, *traffic); + } else if (*traffic) { + if (odp_atomic_load_u64(&node->s.shaper.cur_pir) < pir) { + pkt = __deq_pktout_node(node, traffic); + odp_atomic_add_u64(&node->s.shaper.cur_pir, *traffic); + } + } + if (node->s.is_root) + if (pkt == ODP_PACKET_INVALID) + *traffic = 1; + return pkt; +} + +odp_packet_t deq_pktout_queue(pktout_entry_t *queue, uint64_t *traffic) +{ + void *pkt_tbl[1]; + int ret; + uint64_t cur_time; + uint64_t queue_time; + uint64_t cir; + uint64_t pir; + odph_ipv4hdr_t *ip; + + cur_time = time(NULL); + queue_time = odp_atomic_load_u64(&queue->s.shaper.time); + cir = odp_atomic_load_u64(&queue->s.shaper.cir_kbps); + pir = odp_atomic_load_u64(&queue->s.shaper.pir_kbps); + + if (queue_time < cur_time) { + if (odp_atomic_load_u64(&queue->s.shaper.cur_cir) > cir) + odp_atomic_sub_u64(&queue->s.shaper.cur_cir, cir); + else + odp_atomic_store_u64(&queue->s.shaper.cur_cir, 0); + + if (odp_atomic_load_u64(&queue->s.shaper.cur_pir) > pir) + odp_atomic_sub_u64(&queue->s.shaper.cur_pir, pir); + else + odp_atomic_store_u64(&queue->s.shaper.cur_pir, 0); + + odp_atomic_store_u64(&queue->s.shaper.time, cur_time); + } + + if (odp_atomic_load_u64(&queue->s.shaper.cur_cir) < cir) { + ret = odph_ring_mc_dequeue_bulk(queue->s.ring, pkt_tbl, 1); + if (0 > ret) + /*todo: check for ENOENT */ + return ODP_PACKET_INVALID; + + ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt_tbl[0], NULL); + *traffic = odp_be_to_cpu_16(ip->tot_len); + *traffic = (*traffic) * 8; + odp_atomic_add_u64(&queue->s.shaper.cur_cir, *traffic); + odp_atomic_add_u64(&queue->s.shaper.cur_pir, *traffic); + return (odp_packet_t)pkt_tbl[0]; + } else if (*traffic) { + if (odp_atomic_load_u64(&queue->s.shaper.cur_pir) < pir) { + ret = odph_ring_mc_dequeue_bulk(queue->s.ring, + pkt_tbl, 1); + if (0 > ret) + return ODP_PACKET_INVALID; + + ip = (odph_ipv4hdr_t *)odp_packet_l3_ptr(pkt_tbl[0], + NULL); + *traffic = odp_be_to_cpu_16(ip->tot_len); + *traffic = (*traffic) * 8; + odp_atomic_add_u64(&queue->s.shaper.cur_pir, *traffic); + return (odp_packet_t)pkt_tbl[0]; + } + } + return ODP_PACKET_INVALID; +} + +void *__send_packet(void *pktio) +{ + /* goes over the loop and send the packet for the pktio */ + pktio_entry_t *entry; + pktout_node_entry_t *node; + odp_packet_t pkt_tbl[1]; + + entry = (pktio_entry_t *)pktio; + node = &entry->s.root_node; + uint64_t traffic; + + while (1) { + traffic = 0; + pkt_tbl[0] = deq_pktout_node(node, &traffic); + if (pkt_tbl[0] != ODP_PACKET_INVALID) + __odp_pktio_send(pktio, pkt_tbl, 1); + } + return NULL; +} + +int odp_pktout_enq(odp_pktout_queue_t queue_hdl, odp_packet_t pkt) +{ + uint32_t id; + pktout_entry_t *queue; + void *pkt_tbl[1]; + int ret; + + id = pktout_to_id(queue_hdl); + queue = get_pktout_qentry(id); + pkt_tbl[0] = pkt; + /** todo check the return value */ + ret = odph_ring_mp_enqueue_bulk(queue->s.ring, pkt_tbl, 1); + if (ret == -ENOBUFS) + return ret; + + return 0; +} + +int odp_pktout_cir_set(odp_pktout_queue_t queue_hdl, + uint32_t rate_kbps, uint32_t burst_bytes) +{ + uint32_t id; + pktout_entry_t *queue; + + id = pktout_to_id(queue_hdl); + queue = get_pktout_qentry(id); + odp_atomic_store_u64(&queue->s.shaper.cir_kbps, rate_kbps * 1000); + odp_atomic_store_u64(&queue->s.shaper.cbs_byte, burst_bytes); + return 0; +} + +int odp_pktout_pir_set(odp_pktout_queue_t queue_hdl, + uint32_t rate_kbps, uint32_t burst_bytes) +{ + uint32_t id; + pktout_entry_t *queue; + + id = pktout_to_id(queue_hdl); + queue = get_pktout_qentry(id); + odp_atomic_store_u64(&queue->s.shaper.pir_kbps, rate_kbps * 1000); + odp_atomic_store_u64(&queue->s.shaper.pbs_byte, burst_bytes); + return 0; +} + +int odp_pktio_rate_set(odp_pktio_t pktio, uint32_t rate_kbps, + uint32_t burst_bytes) +{ + pktio_entry_t *entry; + pktout_node_entry_t *node; + + entry = get_pktio_entry(pktio); + if (!entry) + return -1; + node = &entry->s.root_node; + + odp_atomic_store_u64(&node->s.shaper.cir_kbps, rate_kbps * 1000); + odp_atomic_store_u64(&node->s.shaper.cbs_byte, burst_bytes); + odp_atomic_store_u64(&node->s.shaper.pir_kbps, rate_kbps * 1000); + odp_atomic_store_u64(&node->s.shaper.pbs_byte, burst_bytes); + return 0; +}