From patchwork Wed Dec 27 18:00:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 122812 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp2480618qgn; Wed, 27 Dec 2017 10:00:58 -0800 (PST) X-Google-Smtp-Source: ACJfBouc1LxzYYBitFwELtjx8RnE9DSVjXmlKsBfNd43gAtbwk0J2HNff5Z9X7BiNYw2y+ht0OJc X-Received: by 10.55.78.211 with SMTP id c202mr12156810qkb.164.1514397658328; Wed, 27 Dec 2017 10:00:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1514397658; cv=none; d=google.com; s=arc-20160816; b=m/ByZUOmTL9zHWBLK2H5V/FjW7b9B+Po5b/0chEbghOV9bfcTJ1gGHURofSTHB3DdG OU+6ojczxK1njuo/L8glddMvrR+g0ZmWa6oe/cgHyusJc8szlebBAzZEAz0DB/FTJxqt 7fPxLm/Sd4K6i8QW8WrxNG5LiKQA78NrqXLHk6Eq0bCPqxG5Ydg4TzaQLNk48WCKdoZ+ 0GRXnimWPZWbgeKPx4p+MGy2UvanW8M0b8gHj0UzFHdOaUan3/hqPbTVsnmKwhrx0fJd m3yb84WKMZF45uLWmmCZIu9WvoWVhYP3O9oIX0lpQD/wc7vC9U/wKlioEzNjY8MMGKBW jotA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=yMpc4eudZeexmNqulgbczKboi31hZdeHXjRohc/vrwA=; b=mXIRVdWDqIGT3zT162rjIhoDoqLvUxGoB+fh2dB2YtCL8Rk0ibx6gQwjZGt7Lj174s Hx1eQcjkmCxshGBBabVsgXQhRHhS6y9+EsuEIn5wbfgFUi21ydSCGk5vHVXQbVU/eCgj +pCygXGfA0Pivv0nEBr+bI/+TXHcreTzqHVdUd7ZM62T+D1cS6WH8qmuppKuSyQgCoAb Qs7rZyBcNKFdbixAsNnFrQXGrrgBYbchG6v4tDIw5z85K9Bh21bNVJ7cu5dTW+0mcqz2 orl//lWaEgNrMs/0GU4X2aKQBF7sHkrIUoh9XWpz9L7zEvI/ERNyBveel01AQi7fDmyH C+gg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id v131si1266182qkb.315.2017.12.27.10.00.58; Wed, 27 Dec 2017 10:00:58 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 0932D608D9; Wed, 27 Dec 2017 18:00:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 3F421608F6; Wed, 27 Dec 2017 18:00:26 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B5DC7608D9; Wed, 27 Dec 2017 18:00:16 +0000 (UTC) Received: from forward106o.mail.yandex.net (forward106o.mail.yandex.net [37.140.190.187]) by lists.linaro.org (Postfix) with ESMTPS id ECFA8608CE for ; Wed, 27 Dec 2017 18:00:13 +0000 (UTC) Received: from mxback2j.mail.yandex.net (mxback2j.mail.yandex.net [IPv6:2a02:6b8:0:1619::10b]) by forward106o.mail.yandex.net (Yandex) with ESMTP id 362DC7836F7 for ; Wed, 27 Dec 2017 21:00:12 +0300 (MSK) Received: from smtp3p.mail.yandex.net (smtp3p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:8]) by mxback2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id gaxK1HGif9-0CKiJarC; Wed, 27 Dec 2017 21:00:12 +0300 Received: by smtp3p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id XZIPLxGN4u-0B5mf0PP; Wed, 27 Dec 2017 21:00:11 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Wed, 27 Dec 2017 21:00:09 +0300 Message-Id: <1514397609-22930-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1514397609-22930-1-git-send-email-odpbot@yandex.ru> References: <1514397609-22930-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 373 Subject: [lng-odp] [PATCH v3 1/1] changelog: updates for odp v1.17.0.0 X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Add updates for v1.17.0.0 (Tiger Moth RC 1) Signed-off-by: Bill Fischofer --- /** Email created from pull request 373 (Bill-Fischofer-Linaro:tmrc1-changelog) ** https://github.com/Linaro/odp/pull/373 ** Patch: https://github.com/Linaro/odp/pull/373.patch ** Base sha: f390388c10e47985e63b0e31c4ab63fd2508caa0 ** Merge commit sha: 88de5e4113bef8a2b293283b2c66fc80040de825 **/ CHANGELOG | 345 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 345 insertions(+) diff --git a/CHANGELOG b/CHANGELOG index af5757837..2164a3ca8 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,348 @@ +== OpenDataPlane (1.17.0.0) +=== New Features +ODP v1.17.0.0 is Tiger Moth Release Candidate 1 (RC 1). It introduces +significant new API support as well as functional refinements that expand +ODP offload support to cover IPsec, as well as other improvements. + +==== APIs +The following new and changed APIs are included in this release: + +===== Event Extensions +The ODP event model has been expanded to include new types as well as event +subtypes. Subtypes are used to qualify an event by indidating that the event +carries additional metadata relating to specific usage or operations. + +New event `ODP_EVENT_IPSEC_STATUS` (to be discussed +below) is added. The inital subtypes defined are: `ODP_EVENT_PACKET_BASIC`, +`ODP_EVENT_PACKET_CRYPTO`, `ODP_EVENT_PACKET_IPSEC`, and +`ODP_EVENT_NO_SUBTYPE`, which are also discussed below. + +Associated with this support, new event APIs are added: + +* `odp_event_subtype()` extracts the `odp_event_subtype_t` from an +`odp_event_t`. + +* `odp_event_types()` returns the `odp_event_type_t` and `odp_event_subtype_t` +of an `odp_event_t` in a single call. + +* `odp_event_type_multi()` scans a list of `odp_event_t` objects and returns +the number that share the same indicated `odp_event_type_t`. This allows +multiple events to be processed by groups of the same event type. + +* `odp_event_filter_packet()` scans a list of `odp_event_t` objects and +partitions them into a returned array of `odp_packet_t` objects and a remaining +array of non-packet events. + +* `odp_event_free_multi()` frees multiple events in a single call. + +* `odp_event_free_sp()` frees multiple events originating from the same +pool in a single call. The caller must assure that the input events are from +the same pool. + +===== IPsec support +ODP Tiger Moth introduces comprehensive protocol offload support for IPsec, +allowing applications to leverage the IPsec acceleration capabilities of many +SoCs. Support includes APIs for Security Association (SA) creation and +lifecycle management, as well as IPsec packet operations for inbound (RX) and +outbound (TX) processing. Packet operations are further divided into lookaside +and inline support. + +====== Lookaside Support +Lookaside processing enables IPsec packets to be decrypted into plain packets +or plain packets to be encrypted into IPsec packets in a single operation under +explicit application control. This is useful for packets that need pre- or +post-processing, or to better fit with existing application design. + +Two forms of lookaside processing are provided: the `odp_ipsec_in()` and +`odp_ipsec_out()` APIs provide synchronous decrypt and encrypt support, +respectively. The corresponding `odp_ipsec_in_enq()` and `odp_ipsec_out_enq()` +APIs provide these same services in asynchronous form where operations can be +launched and completed later. + +====== Inline Support +In contrast to lookaside support, IPsec inline support permits applications to +fully leverage the offload capabilities found in many SoCs by allowing inbound +IPsec packets to be recognized and decrypted automatically before they are +presented to the application for processing. This is done by configuring a +Security Association (SA) and its associated PktIO to operate in inline mode. + +Similarly, following output SA and PktIO configuration, the +`odp_ipsec_out_inline()` API permits a packet to be encrypted into an IPsec +packet and automatically scheduled for TX processing in a single +operation. Such "fire and forget" processing enables applications to leverage +IPsec HW support for such processing in a portable manner. + +Applications using IPsec inline support need only "touch" a packet once +compared to three times when using lookaside processing, leading to greater +processing efficiency. + +====== IPsec Events +New event types and subtypes are introduced to provide support for IPsec +processing. The `ODP_EVENT_PACKET` type has a new subtype: +`ODP_EVENT_PACKET_IPSEC` that provides extended metadata associated with +IPsec packets that have been processed. The new `ODP_EVENT_IPSEC_STATUS` +event, in turn, is used to report IPsec status events such as completion +notifications associated with `odp_ipsec_sa_disable()` calls. + +The `odp_ipsec_result()` API is used to obtain IPsec result metadata from +a packet that has event subtype `ODP_EVENT_PACKET_IPSEC`, while the +`odp_ipsec_status()` API is used to obtain IPsec status metadata from an +`ODP_EVENT_IPSEC_STATUS` event. + +===== Parser APIs +Packet parsing has been overhauled with the introduction of two new APIs: + +* `odp_packet_parse()` +* `odp_packet_parse_multi()` + +These use an `odp_packet_parse_param_t` struct to control the type and depth +of parsing to be performed. These routines are intended to be used to +process packets that have been decapsulated following IPsec decryption or other +tunneling or on IP fragments after they have been reassembled. + +Associated with this improved parse support, the `odp_parser_layer_t` struct +is deprecated and replaced with a more general `odp_proto_layer_t` struct that +is used both in PktIO configuration as well as the new parser APIs. + +===== Crypto AES-CTR and AES-GMAC Support +The ODP crypto APIs are extended to provide support for AES-CTR cipher and +AES-GMAC authentication modes, reflecting the growing availability of +accelerated support for these. + +===== Crypto removal of DES-CBC +DES-CBC is no longer considered secure and support for it is removed in ODP. + +===== Crypto move AAD length to sessions +The Additional Authentication Data (AAD) length is now part of the +`odp_crypto_session_t` rather than individual crypto operations. This provides +better compatibility with DPDK, which made a similar API adjustment in it's +17.08 release. + +===== Crypto Packet APIs +While the `odp_crypto_operation()` API is retained for compatibility, +new packet-oriented variants are introduced that provide additional +capabilities and flexibility. These APIs are: + +* `odp_crypto_op()` Performs synchronous crypto operations on one or more +input packets under the control of an associated `odp_crypto_packet_op_param_t` +struct. + +* `odp_crypto_op_enq()` Performs asynchronous crypto operations on or or more +input packets under the control of an associated `odp_crypto_packet_op_param_t` +struct. + +While `odp_crypto_operation()` calls result in `ODP_EVENT_CRYPTO_COMPL` events +for compatibility, the new packet-oriented APIs result in `ODP_EVENT_PACKET` +events that carry the new event subtype `ODP_EVENT_PACKET_CRYPTO`. These +packets contain additional metadata associated with the crypto operation. +New APIs added for manipulating this metadata include: + +* `odp_crypto_packet_from_event()` converts an `odp_event_t` of type +`ODP_EVENT_PACKET` and subtype `ODP_EVENT_PACKET_CRYPTO` to an `odp_packet_t`. + +* `odp_crypto_packet_to_event()` converts an `odp_packet_t` crypto packet +back into an `odp_event_t`. + +* `odp_crypto_result()` extracts the `odp_crypto_packet_result_t` struct that +contains the crypto metadata associated with an `odp_packet_t` of event +subtype `ODP_EVENT_PACKET_CRYPTO`. This struct provides a summary bit that +says whether the operation completed successfully as well as +`odp_crypto_op_status_t` fields for the `cipher_status` and `auth_status` if a +problem was detected. + +===== Classification Random Early Detection (RED) Support +Random Early Detection (RED) provides a means for input HW to ensure that +traffic is treated fairly under conditions of temporary resource overload due +to excessive inbound traffic. ODP RED support provides the ability to measure +and respond to resource pressure on either pools or queues, and only provides +flexibility in how such conditions are to be processed. They can result in +input packet drops or backpressure being indicated by transmitting pause +frames, depending on the underlying platform capabilities. + +The `odp_cls_capability_t` struct returned by the `odp_cls_capability()` API +has been expanded to cover this support. + +===== Time difference in nanoseconds +The new `odp_time_diff_ns()` API permits the delta between two `odp_time_t` +values to be computed in a single call. + +===== PktIO Maximum Frame Lengths +The `odp_pktio_mtu()` API is deprecated and replaced by two new APIs: +`odp_pktin_maxlen()` and `odp_pktout_maxlen()`. These return the maximum +sized packets that are supported for RX and TX processing, +respectively, on a given `odp_pktio_t`. + +===== Pool subparameter support +The `odp_pool_param_t` structure has been expanded to provide more flexibility +to support platforms that are able to offer multiple segment sizes within a +single pool. This can lead to greater storage efficiency. These are called +subparameters and implementations supporting up to 7 of these are accommodated +with these extensions. + +The `odp_pool_capability_t` structure is expanded to return the number of +subparameters supported by this implementation. The application, in turn, +specifies its expected packet size and number distribution in the +`odp_pool_pkt_subparam_t` structure that is part of the `odp_pool_param_t` +used to define the characteristics of `ODP_POOL_PACKET` pools. + +This is fully compatible with previous packet pool support since ODP +implementations are under no obligation to support pool subparameters and +these, when present, are advisory in nature. They simply serve to allow the +application to better communicate its expected packet distribution within a +pool so that the ODP implementation may better optimize storage use in the +pool. + +===== Checksum support +Checksum processing support has been formalized with the addition of APIs for +determining packet checksum status, controlling packet checksum processing, +retrieving partially computed checksums on packets, and computing checksum +partial sums for memory areas. + +====== Checksum status +The APIs `odp_packet_l3_chksum_status()` and `odp_packet_l4_status()` are +added to allow the results of packet input checksum processing to be +queried. These APIs return an `odp_packet_chksum_status_t` enum that indicates +whether checksum validation processing was performed and if so whether the +layer 3 or 4 checksum was found to be valid. This is applicable to both +normal packet input as well as those processed via IPsec. + +====== Checksum insertion +PktIOs output checksum processing is configured as part of the +`odp_pktout_config_opt_t` struct used as input to `odp_pktio_config()` API. +These control whether L3 and/or L4 checksums are to be inserted by default +as part of packet TX processing. + +Individual packets may override these defaults via the new +`odp_packet_l3_chksum_insert()` and `odp_packet_l4_chksum_insert()` APIs. These +take precedence over the PktIO default, allowing checksums to be inserted +when the PktIO default is to not insert checksums or to suppress checksum +insertion if when the PktIO default is to insert checksums. + +====== One's complement sums +Two new APIs: `odp_packet_ones_comp()` and `odp_chksum_ones_comp16()` are +added to assist in application-managed checksum processing. If an +implementation has computed a partial checksum as part of the receive +processing for an IP fragment, for example, then `odp_packet_ones_comp()` can +be used to retrieve this computed value, as well as the byte range over which +it was computed. The `odp_chksum_ones_comp16()` API, by contrast, allows the +application to perform a 16-bit ones-complement sum over a range of in-memory +bytes. Together these can be used to calculate IPv4, TCP, and UDP checksums. + +===== Packet multi-event conversion and single pool support +New packet APIs have been added to streamline packet processing: + +* `odp_packet_free_sp()` is the same as `odp_packet_free_multi()` except that +the application guarantees that all packets come from the same pool. + +* `odp_packet_from_event_multi()` allows multiple events to be converted +from `odp_event_t` to `odp_packet_t` objects in a single call. The caller +guarantees that all input `odp_event_t` objects are of type `ODP_EVENT_PACKET`. + +* `odp_packet_to_event_multi()` converts multiple `odp_packet_t` objects to +corresponding `odp_event_t` objects in a single call. + +===== Shared Memory API changes +Several changes have been made to the ODP shared memory APIs: + +* The `name` field used as input to `odp_shm_reserve()` is now optional. +If specified as `NULL` the shared memory area is anonymous and cannot be +looked up with `odp_shm_lookup()`. There is also no requirement that names be +unique. Duplicate names result in indeterminate output from `odp_shm_lookup()`. + +* The `odp_shm_info_t` now includes the `page_size` of the shared memory block +and it's (optional) name. + +* `odp_shm_print()` API is added to print implementation-defined information +associated with the `odp_shm_t` to the ODP log for diagnostic purposes. + +===== Add support for non-blocking Queues +New queue attributes are introduced to characterize queue behavior as +blocking or non-blocking. A blocking queue may stall other threads if a thread +is interrupted or suspending during an enqueue or dequeue operation. +Nonblocking queues may be either lock free or wait free and provide +progress and fairness guarantees to all threads regardless of +interruptions or stalls on the part of threads performing queue operations. + +The various `odp_nonblocking_t` attributes available are returned by the +`odp_queue_capability()` API for both plain and scheduled queues and are in +turn requested as part of the `odp_queue_param_t` struct passed to the +`odp_queue_create()` API. The intent is to allow applications that have +realtime response requirements to better express these needs and utilize +platform-specific capabilities in this area. + +===== Scheduler ordered lock API changes +The following changes have been made to the scheduler APIs: + +* Documentation clarifies that an ordered context may only hold one ordered +lock at a time. Results are undefined if a second ordered lock is attempted to +be acquired while already holding one. + +* The `odp_schedule_order_unlock_lock()` API is added to permit an ordered +context to switch from one ordered lock to another in a single operation. + +===== Timer Capabilities +The `odp_timer_capability()` API is added to return an `odp_timer_capability_t` +struct that details platform-specific timer capabilities for application use. +The only capability currently defined is `highest_res_ns`, which indicates the +highest supported resolution supported by a timer. This is the minimum valid +value for the `res_ns` timer pool parameter. + +=== Miscellaneous Fixes and Improvements +The following miscellaneous improvements have been made to the `linux-generic` +reference implementation of ODP. + +==== Additional packet inline functions +When compiling with `--enable-abi-compat=no` the following additional packet +functions are inlined: + +* `odp_packet_l2_offset()` +* `odp_packet_l2_ptr()` +* `odp_packet_l3_offset()` +* `odp_packet_l3_ptr()` +* `odp_packet_l4_offset()` +* `odp_packet_l4_ptr()` + +==== Crypto Security Improvements +GCM ciphers now use a counter instead of a random IV for improved security. + +==== Dependencies +The ODP test suite now automatically skips C++ tests if no C++ compiler +is available. + +The odp_pktio_ordered tests are only performed if PCAP is available. + +The DEPENDENCIES file has been updated to reflect build/test requirements for +running under Red Hat Linux distributions. + +==== DPDK 17.08 Support +PktIO DPDK support has been upgraded to DPDK 17.08. + +=== Test/Example improvements +=== l2fwd Example +A verbose option is added to provide more detail on test runs. + +=== ODP generator +Numerous performance improvements have been made that results in significantly +better I/O rates. This includes a configuration option to control checksum +usage. + +=== Bug Fixes +==== https://bugs.linaro.org/show_bug.cgi?id=3465[Bug 3465] +CID 1461688: odp_pool_create: Dereference before null check + +==== https://bugs.linaro.org/show_bug.cgi?id=3466[Bug 3466] +CID 1396968: buggy check + +==== https://bugs.linaro.org/show_bug.cgi?id=3491[Bug 3491] +l2fwd segfaults on api-next with dpdk checksum insertion override + +=== Known Issues +==== https://bugs.linaro.org/show_bug.cgi?id=3210[Bug 3210] +packet header parsing routines should verify header checksums + +==== https://bugs.linaro.org/show_bug.cgi?id=3517[Bug 3517] +timer test might fail + == OpenDataPlane (1.16.0.0) === New Features ODP v1.16.0.0 is the final preview release before the official release of