From patchwork Wed Jan 24 10:00:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 125631 Delivered-To: patch@linaro.org Received: by 10.46.66.141 with SMTP id h13csp307620ljf; Wed, 24 Jan 2018 02:01:24 -0800 (PST) X-Google-Smtp-Source: AH8x226J3iFBIo2wVg7gSwryCQjldslK54Soal5V11JhKA53K/burnGuwgdT8nhQcUStnvYwVPiT X-Received: by 10.237.38.71 with SMTP id z65mr8270032qtc.248.1516788083984; Wed, 24 Jan 2018 02:01:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516788083; cv=none; d=google.com; s=arc-20160816; b=b0t8d2LRDFUgPjZee0SoCTJT15AzOXRd/35AHQ+Lm1IhEjZnLh2Zs6LhoeOJjbMwQ4 N7GhUXur/S3oNeIom+L5VJ7/AjIEUbb0jVMEukybGzgT/R6XVjZyhCthEdhTAX+vKUyG 6qeej8xTRvA6/XTbik6lf2vcB0HXhhMSB3G97wQZKcxd3u78boAPjCf6dFLPNKjte0me 7ZZV+XyZt5ui9HsSqhxQXM+59KEsisMScKVmWa5T7wNg76ZGkDurKdypv+BwHG6Vpopa i9V5zOthJu0HRZPeQiXCp4YA/ebuYGxZaqehbNpUrsJUNxaKlcB1WgHoIUgzLWgKbE00 rfMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=KrzHtS2Bh5klJ2vCwt10lRRGZdc4NnOHt5KULg+Gujc=; b=grVWhfmzLewJFLPJQ+CDpSpCvDfc2q1vHQ94Brub2vX7LfRk0Nso0TOsbkJerOw8S3 ZzoGgQzYDQw/1gcVeU1nwxqxeb9+tCA8laFeI1n4rwLnFjePnIPJOA6noOpRj1lfYib5 SXCkc/2JufoNVyPC2t9ZfVTet0hGJRqjTxkovn3HyBmDVemg0I0DFQyynBRbjZRhdAWD f1QlbtADcXXZPhEHclQrvG5j6XdzxNGFYCBR2LI3rggdrzM6CsUpbVV/JId/XtuuUZea mI1m7lM7DxYFXKv38MxJKAx2gNcO6I5pOShMqzNSGPHtoJXkQab3M2MQzkYlXE/svAvP 72Tw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id f95si929753qkh.117.2018.01.24.02.01.23; Wed, 24 Jan 2018 02:01:23 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id A30B2617AB; Wed, 24 Jan 2018 10:01:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2 autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 29116617D0; Wed, 24 Jan 2018 10:00:40 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0A6BC617D0; Wed, 24 Jan 2018 10:00:35 +0000 (UTC) Received: from forward104p.mail.yandex.net (forward104p.mail.yandex.net [77.88.28.107]) by lists.linaro.org (Postfix) with ESMTPS id 8411360C4C for ; Wed, 24 Jan 2018 10:00:27 +0000 (UTC) Received: from mxback5j.mail.yandex.net (mxback5j.mail.yandex.net [IPv6:2a02:6b8:0:1619::10e]) by forward104p.mail.yandex.net (Yandex) with ESMTP id D0C37184838 for ; Wed, 24 Jan 2018 13:00:25 +0300 (MSK) Received: from smtp2p.mail.yandex.net (smtp2p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:7]) by mxback5j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id WFPk4fp02C-0P3W7IPX; Wed, 24 Jan 2018 13:00:25 +0300 Received: by smtp2p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 8PEj4kSoa2-0J9SHhqD; Wed, 24 Jan 2018 13:00:19 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Wed, 24 Jan 2018 13:00:10 +0300 Message-Id: <1516788011-7656-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1516788011-7656-1-git-send-email-odpbot@yandex.ru> References: <1516788011-7656-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 408 Subject: [lng-odp] [PATCH CATERPILLAR v3 1/2] linux-gen: pktio: coding style fixup X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Mykyta Iziumtsev Declare iterator variable used in "for" loop in containing scope instead of "for" scope itself. Although it's valid C99 syntax and "for" loop defines own scope it is still preferred to not use it in OpenDataPlane code. Signed-off-by: Mykyta Iziumtsev --- /** Email created from pull request 408 (MykytaI:caterpillar_mdev_e1000e) ** https://github.com/Linaro/odp/pull/408 ** Patch: https://github.com/Linaro/odp/pull/408.patch ** Base sha: f87bb5146fb25f7cd62246500cb4f7e625375654 ** Merge commit sha: 1d0d3e4b1cef1a8f746bbc2df65526e82a1cfca9 **/ platform/linux-generic/pktio/mdev/cxgb4.c | 6 ++++-- platform/linux-generic/pktio/mdev/i40e.c | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/platform/linux-generic/pktio/mdev/cxgb4.c b/platform/linux-generic/pktio/mdev/cxgb4.c index f09835d9f..5c55d87a4 100644 --- a/platform/linux-generic/pktio/mdev/cxgb4.c +++ b/platform/linux-generic/pktio/mdev/cxgb4.c @@ -509,20 +509,22 @@ static int cxgb4_open(odp_pktio_t id ODP_UNUSED, static int cxgb4_close(pktio_entry_t *pktio_entry) { + uint16_t i; + pktio_ops_cxgb4_data_t *pkt_cxgb4 = pktio_entry->s.ops_data; ODP_DBG("%s: close %s\n", MODULE_NAME, pkt_cxgb4->mdev.if_name); mdev_device_destroy(&pkt_cxgb4->mdev); - for (uint16_t i = 0; i < pkt_cxgb4->capa.max_input_queues; i++) { + for (i = 0; i < pkt_cxgb4->capa.max_input_queues; i++) { cxgb4_rx_queue_t *rxq = &pkt_cxgb4->rx_queues[i]; if (rxq->rx_data.size) mdev_dma_area_free(&pkt_cxgb4->mdev, &rxq->rx_data); } - for (uint16_t i = 0; i < pkt_cxgb4->capa.max_output_queues; i++) { + for (i = 0; i < pkt_cxgb4->capa.max_output_queues; i++) { cxgb4_tx_queue_t *txq = &pkt_cxgb4->tx_queues[i]; if (txq->tx_data.size) diff --git a/platform/linux-generic/pktio/mdev/i40e.c b/platform/linux-generic/pktio/mdev/i40e.c index 6f3b71712..f2be2bcf3 100644 --- a/platform/linux-generic/pktio/mdev/i40e.c +++ b/platform/linux-generic/pktio/mdev/i40e.c @@ -344,20 +344,22 @@ static int i40e_open(odp_pktio_t id ODP_UNUSED, static int i40e_close(pktio_entry_t *pktio_entry) { + uint16_t i; + pktio_ops_i40e_data_t *pkt_i40e = pktio_entry->s.ops_data; ODP_DBG("%s: close %s\n", MODULE_NAME, pkt_i40e->mdev.if_name); mdev_device_destroy(&pkt_i40e->mdev); - for (uint16_t i = 0; i < pkt_i40e->capa.max_input_queues; i++) { + for (i = 0; i < pkt_i40e->capa.max_input_queues; i++) { i40e_rx_queue_t *rxq = &pkt_i40e->rx_queues[i]; if (rxq->rx_data.size) mdev_dma_area_free(&pkt_i40e->mdev, &rxq->rx_data); } - for (uint16_t i = 0; i < pkt_i40e->capa.max_output_queues; i++) { + for (i = 0; i < pkt_i40e->capa.max_output_queues; i++) { i40e_tx_queue_t *txq = &pkt_i40e->tx_queues[i]; if (txq->tx_data.size) From patchwork Wed Jan 24 10:00:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 125632 Delivered-To: patch@linaro.org Received: by 10.46.66.141 with SMTP id h13csp308081ljf; Wed, 24 Jan 2018 02:02:09 -0800 (PST) X-Google-Smtp-Source: AH8x225Xwklu2+6R+lm0OmKWVxlphzY72FaQNH42z4YTtQrvANmOQ3PO43DVmkxXWso2GRlcGlmK X-Received: by 10.200.17.144 with SMTP id d16mr2549720qtj.339.1516788129122; Wed, 24 Jan 2018 02:02:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516788129; cv=none; d=google.com; s=arc-20160816; b=eUjJIUPk2VLfAQXrGOYi4uS/9v94MG38/4A1DIYe8IDvMfYb2O5bxdExtoS0R5O+/1 ll7gAXCbtqiZZvmw5mxb9sBmoc1vXZyME59aIVm95trw0eID0KgwAYPMpBRDqLo0l3UX 2+9sRu3BPCaK9W+ZlqfNivskUN4v4CrtequSBpIymPzCqirRwRJ9meq9+V1TEgEgsnO5 jo98+Vlp0qDch4jw03yJHM+ApJQ3kI7qBl5ssCn9280iXZ5lV6nKr7ldvtvar1ktMt75 pLXPcf/ziOeFPt5ypIi3FdgPNtLSz+AdlBeuM8kTvFfLtllQ30o/YHPuczl/rEHyJDna yceQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=TXX2K/y9UV6Vfzlry0FgMHHBuoDCObbubYhhLm1ZLVg=; b=EFjYEqPe3saoFWhoMMn6EtG8xOZ8JXzLAUtz/FViLEdEpjddFbnqo4NFnfcOqBIxMu 9zMpwjiB9dfbo6W6SmZPjXikwd/DhyCQ/W6uq9AhRHrwW2b5qcLqB2N/Ag+K1JxmGVFq qk798b+82ymjIJCE7CspXrsGEdsZT0OYWwJhYjPnrRQuRxhjZ02exHuYDriZkXXXyT27 s2piC3nPTCedlAuoxO9qBevAMs04Bc2kSlUkq5f743s7FPF8wt+gelBvRDC2/Gbp6Lzk cGF0deYf35H5BjB7Z2ltv6FNOYGFpqKPtPwZra0O3ERfke5tN8yiNl+LtInxmXPGTGSv ALlQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id p19si2432783qtg.124.2018.01.24.02.02.08; Wed, 24 Jan 2018 02:02:09 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id C26C361753; Wed, 24 Jan 2018 10:02:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2 autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 518E161775; Wed, 24 Jan 2018 10:01:31 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 2107161775; Wed, 24 Jan 2018 10:01:22 +0000 (UTC) Received: from forward101j.mail.yandex.net (forward101j.mail.yandex.net [5.45.198.241]) by lists.linaro.org (Postfix) with ESMTPS id 510F16081A for ; Wed, 24 Jan 2018 10:00:34 +0000 (UTC) Received: from mxback16j.mail.yandex.net (mxback16j.mail.yandex.net [IPv6:2a02:6b8:0:1619::92]) by forward101j.mail.yandex.net (Yandex) with ESMTP id BC841124492A for ; Wed, 24 Jan 2018 13:00:32 +0300 (MSK) Received: from smtp2p.mail.yandex.net (smtp2p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:7]) by mxback16j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id GbzxrGGO7R-0Waa5dvA; Wed, 24 Jan 2018 13:00:32 +0300 Received: by smtp2p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 8PEj4kSoa2-0P9aBBbj; Wed, 24 Jan 2018 13:00:25 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Wed, 24 Jan 2018 13:00:11 +0300 Message-Id: <1516788011-7656-3-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1516788011-7656-1-git-send-email-odpbot@yandex.ru> References: <1516788011-7656-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 408 Subject: [lng-odp] [PATCH CATERPILLAR v3 2/2] linux-gen: add e1000e mediated device driver X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Mykyta Iziumtsev Signed-off-by: Mykyta Iziumtsev --- /** Email created from pull request 408 (MykytaI:caterpillar_mdev_e1000e) ** https://github.com/Linaro/odp/pull/408 ** Patch: https://github.com/Linaro/odp/pull/408.patch ** Base sha: f87bb5146fb25f7cd62246500cb4f7e625375654 ** Merge commit sha: 1d0d3e4b1cef1a8f746bbc2df65526e82a1cfca9 **/ platform/linux-generic/Makefile.am | 1 + platform/linux-generic/pktio/mdev/e1000e.c | 607 +++++++++++++++++++++++++++++ platform/linux-generic/pktio/subsystem.c | 2 + 3 files changed, 610 insertions(+) create mode 100644 platform/linux-generic/pktio/mdev/e1000e.c diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index 84e539b9a..dc3284b67 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -252,6 +252,7 @@ __LIB__libodp_linux_la_SOURCES = \ pktio/loopback.c \ pktio/mdev.c \ pktio/mdev/cxgb4.c \ + pktio/mdev/e1000e.c \ pktio/mdev/i40e.c \ pktio/netmap.c \ pktio/ring.c \ diff --git a/platform/linux-generic/pktio/mdev/e1000e.c b/platform/linux-generic/pktio/mdev/e1000e.c new file mode 100644 index 000000000..e74161601 --- /dev/null +++ b/platform/linux-generic/pktio/mdev/e1000e.c @@ -0,0 +1,607 @@ +/* Copyright (c) 2018, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include "config.h" + +#ifdef ODP_MDEV + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#define MODULE_NAME "e1000e" + +#define E1000E_TX_BUF_SIZE 2048U +#define E1000E_RX_BUF_SIZE 2048U + +/* RX queue definitions */ +#define E1000E_RX_QUEUE_NUM_MAX 1 + +#define E1000_RDH_OFFSET 0x02810UL +#define E1000_RDT_OFFSET 0x02818UL + +/** RX descriptor */ +typedef struct { + odp_u64le_t addr; +#define E1000E_RXD_STATUS_DONE 0x00000001UL +#define E1000E_RXD_STATUS_ERR_MASK 0xff000000UL + odp_u32le_t status; + odp_u16le_t length; + odp_u16le_t vlan; +} e1000e_rx_desc_t; + +/** RX queue data */ +typedef struct ODP_ALIGNED_CACHE { + e1000e_rx_desc_t *rx_descs; /**< RX queue base */ + odp_u32le_t *doorbell; /**< RX queue doorbell */ + + uint16_t rx_queue_len; /**< Number of RX desc entries */ + + uint16_t cidx; /**< Next RX desc to read */ + odp_u32le_t *pidx; /**< Next RX desc HW is going to write */ + + mdev_dma_area_t rx_data; /**< RX packet payload area */ + + odp_ticketlock_t lock; /**< RX queue lock */ +} e1000e_rx_queue_t; + +/* TX queue definitions */ +#define E1000E_TX_QUEUE_NUM_MAX 1 + +#define E1000_TDH_OFFSET 0x03810UL +#define E1000_TDT_OFFSET 0x03818UL + +typedef struct { + odp_u64le_t addr; /* Address of data buffer */ +#define E1000_TXD_CMD_EOP 0x01000000 /* End of Packet */ +#define E1000_TXD_CMD_IFCS 0x02000000 /* Insert FCS (Ethernet CRC) */ + odp_u32le_t cmd; + odp_u32le_t reserved; +} e1000e_tx_desc_t; + +/** TX queue data */ +typedef struct ODP_ALIGNED_CACHE { + e1000e_tx_desc_t *tx_descs; /**< TX queue base */ + odp_u32le_t *doorbell; /**< TX queue doorbell */ + + uint16_t tx_queue_len; /**< Number of TX desc entries */ + + uint16_t pidx; /**< Next TX desc to write */ + odp_u32le_t *cidx; /**< Next TX desc HW is going to read */ + + mdev_dma_area_t tx_data; /**< TX packet payload area */ + + odp_ticketlock_t lock; /**< TX queue lock */ +} e1000e_tx_queue_t; + +/** Packet socket using mediated e1000e device */ +typedef struct { + /** RX queue hot data */ + e1000e_rx_queue_t rx_queues[E1000E_RX_QUEUE_NUM_MAX]; + + /** TX queue hot data */ + e1000e_tx_queue_t tx_queues[E1000E_TX_QUEUE_NUM_MAX]; + + odp_pool_t pool; /**< pool to alloc packets from */ + + odp_bool_t lockless_rx; /**< no locking for RX */ + odp_bool_t lockless_tx; /**< no locking for TX */ + + odp_pktio_capability_t capa; /**< interface capabilities */ + + uint8_t *mmio; /**< MMIO region */ + + int sockfd; /**< control socket */ + + mdev_device_t mdev; /**< Common mdev data */ +} pktio_ops_e1000e_data_t; + +static void e1000e_rx_refill(e1000e_rx_queue_t *rxq, uint16_t from, + uint16_t num); +static void e1000e_wait_link_up(pktio_entry_t *pktio_entry); +static int e1000e_close(pktio_entry_t *pktio_entry); + +static int e1000e_mmio_register(pktio_ops_e1000e_data_t *pkt_e1000e, + uint64_t offset, uint64_t size) +{ + ODP_ASSERT(pkt_e1000e->mmio == NULL); + + pkt_e1000e->mmio = mdev_region_mmap(&pkt_e1000e->mdev, offset, size); + if (pkt_e1000e->mmio == MAP_FAILED) { + ODP_ERR("Cannot mmap MMIO\n"); + return -1; + } + + ODP_DBG("Register MMIO region: 0x%llx@%016llx\n", size, offset); + + return 0; +} + +static int e1000e_rx_queue_register(pktio_ops_e1000e_data_t *pkt_e1000e, + uint64_t offset, uint64_t size) +{ + uint16_t rxq_idx = pkt_e1000e->capa.max_input_queues++; + e1000e_rx_queue_t *rxq = &pkt_e1000e->rx_queues[rxq_idx]; + struct ethtool_ringparam ering; + int ret; + + ODP_ASSERT(rxq_idx < ARRAY_SIZE(pkt_e1000e->rx_queues)); + + odp_ticketlock_init(&rxq->lock); + + ret = ethtool_ringparam_get_fd(pkt_e1000e->sockfd, + pkt_e1000e->mdev.if_name, &ering); + if (ret) { + ODP_ERR("Cannot get queue length\n"); + return -1; + } + rxq->rx_queue_len = ering.rx_pending; + + rxq->doorbell = + (odp_u32le_t *)(void *)(pkt_e1000e->mmio + E1000_RDT_OFFSET); + + ODP_ASSERT(rxq->rx_queue_len * sizeof(*rxq->rx_descs) <= size); + + rxq->rx_descs = mdev_region_mmap(&pkt_e1000e->mdev, offset, size); + if (rxq->rx_descs == MAP_FAILED) { + ODP_ERR("Cannot mmap RX queue\n"); + return -1; + } + + rxq->pidx = + (odp_u32le_t *)(void *)(pkt_e1000e->mmio + E1000_RDH_OFFSET); + + rxq->rx_data.size = rxq->rx_queue_len * E1000E_RX_BUF_SIZE; + ret = mdev_dma_area_alloc(&pkt_e1000e->mdev, &rxq->rx_data); + if (ret) { + ODP_ERR("Cannot allocate RX queue DMA area\n"); + return -1; + } + + /* Need 1 desc gap to keep tail from touching head */ + e1000e_rx_refill(rxq, 0, rxq->rx_queue_len - 1); + + ODP_DBG("Register RX queue region: 0x%llx@%016llx\n", size, offset); + ODP_DBG(" RX descriptors: %u\n", rxq->rx_queue_len); + + return 0; +} + +static int e1000e_tx_queue_register(pktio_ops_e1000e_data_t *pkt_e1000e, + uint64_t offset, uint64_t size) +{ + uint16_t txq_idx = pkt_e1000e->capa.max_output_queues++; + e1000e_tx_queue_t *txq = &pkt_e1000e->tx_queues[txq_idx]; + struct ethtool_ringparam ering; + int ret; + + ODP_ASSERT(txq_idx < ARRAY_SIZE(pkt_e1000e->tx_queues)); + + odp_ticketlock_init(&txq->lock); + + ret = ethtool_ringparam_get_fd(pkt_e1000e->sockfd, + pkt_e1000e->mdev.if_name, &ering); + if (ret) { + ODP_ERR("Cannot get queue length\n"); + return -1; + } + txq->tx_queue_len = ering.tx_pending; + + txq->doorbell = + (odp_u32le_t *)(void *)(pkt_e1000e->mmio + E1000_TDT_OFFSET); + + ODP_ASSERT(txq->tx_queue_len * sizeof(*txq->tx_descs) <= size); + + txq->tx_descs = mdev_region_mmap(&pkt_e1000e->mdev, offset, size); + if (txq->tx_descs == MAP_FAILED) { + ODP_ERR("Cannot mmap TX queue\n"); + return -1; + } + + txq->cidx = + (odp_u32le_t *)(void *)(pkt_e1000e->mmio + E1000_TDH_OFFSET); + + txq->tx_data.size = txq->tx_queue_len * E1000E_TX_BUF_SIZE; + ret = mdev_dma_area_alloc(&pkt_e1000e->mdev, &txq->tx_data); + if (ret) { + ODP_ERR("Cannot allocate TX queue DMA area\n"); + return -1; + } + + ODP_DBG("Register TX queue region: 0x%llx@%016llx\n", size, offset); + ODP_DBG(" TX descriptors: %u\n", txq->tx_queue_len); + + return 0; +} + +static int e1000e_region_info_cb(mdev_device_t *mdev, + struct vfio_region_info *region_info) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = + odp_container_of(mdev, pktio_ops_e1000e_data_t, mdev); + mdev_region_class_t class_info; + + if (vfio_get_region_cap_type(region_info, &class_info) < 0) { + ODP_ERR("Cannot find class_info in region %u\n", + region_info->index); + return -1; + } + + switch (class_info.type) { + case VFIO_NET_MDEV_MMIO: + return e1000e_mmio_register(pkt_e1000e, + region_info->offset, + region_info->size); + + case VFIO_NET_MDEV_RX_RING: + return e1000e_rx_queue_register(pkt_e1000e, + region_info->offset, + region_info->size); + + case VFIO_NET_MDEV_TX_RING: + return e1000e_tx_queue_register(pkt_e1000e, + region_info->offset, + region_info->size); + + default: + ODP_ERR("Unexpected region %u (class %u:%u)\n", + region_info->index, class_info.type, + class_info.subtype); + return -1; + } +} + +static int e1000e_open(odp_pktio_t id ODP_UNUSED, + pktio_entry_t *pktio_entry, + const char *resource, odp_pool_t pool) +{ + pktio_ops_e1000e_data_t *pkt_e1000e; + int ret; + + ODP_ASSERT(pool != ODP_POOL_INVALID); + + if (strncmp(resource, NET_MDEV_PREFIX, strlen(NET_MDEV_PREFIX))) + return -1; + + ODP_DBG("%s: probing resource %s\n", MODULE_NAME, resource); + + pkt_e1000e = ODP_OPS_DATA_ALLOC(sizeof(*pkt_e1000e)); + if (odp_unlikely(pkt_e1000e == NULL)) { + ODP_ERR("Failed to allocate pktio_ops_e1000e_data_t struct"); + return -1; + } + pktio_entry->s.ops_data = pkt_e1000e; + + memset(pkt_e1000e, 0, sizeof(*pkt_e1000e)); + + pkt_e1000e->sockfd = socket(AF_INET, SOCK_DGRAM, 0); + if (pkt_e1000e->sockfd == -1) { + ODP_ERR("Cannot get device control socket\n"); + goto out; + } + + ret = + mdev_device_create(&pkt_e1000e->mdev, MODULE_NAME, + resource + strlen(NET_MDEV_PREFIX), + e1000e_region_info_cb); + if (ret) + goto out; + + pkt_e1000e->pool = pool; + + e1000e_wait_link_up(pktio_entry); + + ODP_DBG("%s: open %s is successful\n", MODULE_NAME, + pkt_e1000e->mdev.if_name); + + return 0; + +out: + e1000e_close(pktio_entry); + return -1; +} + +static int e1000e_close(pktio_entry_t *pktio_entry) +{ + uint16_t i; + + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + ODP_DBG("%s: close %s\n", MODULE_NAME, pkt_e1000e->mdev.if_name); + + mdev_device_destroy(&pkt_e1000e->mdev); + + for (i = 0; i < pkt_e1000e->capa.max_input_queues; i++) { + e1000e_rx_queue_t *rxq = &pkt_e1000e->rx_queues[i]; + + if (rxq->rx_data.size) + mdev_dma_area_free(&pkt_e1000e->mdev, &rxq->rx_data); + } + + for (i = 0; i < pkt_e1000e->capa.max_output_queues; i++) { + e1000e_tx_queue_t *txq = &pkt_e1000e->tx_queues[i]; + + if (txq->tx_data.size) + mdev_dma_area_free(&pkt_e1000e->mdev, &txq->tx_data); + } + + if (pkt_e1000e->sockfd != -1) + close(pkt_e1000e->sockfd); + + ODP_OPS_DATA_FREE(pkt_e1000e); + + return 0; +} + +static void e1000e_rx_refill(e1000e_rx_queue_t *rxq, uint16_t from, + uint16_t num) +{ + uint16_t i = from; + + while (num) { + uint64_t iova = rxq->rx_data.iova + i * E1000E_RX_BUF_SIZE; + e1000e_rx_desc_t *rxd = &rxq->rx_descs[i]; + + rxd->addr = odp_cpu_to_le_64(iova); + + i++; + if (i >= rxq->rx_queue_len) + i = 0; + + num--; + } + + /* Ring the doorbell */ + odpdrv_mmio_u32le_write(i, rxq->doorbell); +} + +static int e1000e_recv(pktio_entry_t *pktio_entry, int rxq_idx, + odp_packet_t pkt_table[], int num) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + e1000e_rx_queue_t *rxq = &pkt_e1000e->rx_queues[rxq_idx]; + uint16_t refill_from; + uint16_t budget = 0; + int rx_pkts = 0; + int ret; + + if (!pkt_e1000e->lockless_rx) + odp_ticketlock_lock(&rxq->lock); + + /* Keep track of the start point to refill RX queue */ + refill_from = rxq->cidx ? rxq->cidx - 1 : rxq->rx_queue_len - 1; + + /* + * Determine how many packets are available in RX queue: + * (Write_index - Read_index) modulo RX queue size + */ + budget += odp_le_to_cpu_32(*rxq->pidx); + budget -= rxq->cidx; + budget &= rxq->rx_queue_len - 1; + + if (budget > num) + budget = num; + + ret = odp_packet_alloc_multi(pkt_e1000e->pool, E1000E_RX_BUF_SIZE, + pkt_table, budget); + budget = (ret > 0) ? ret : 0; + + while (rx_pkts < budget) { + volatile e1000e_rx_desc_t *rxd = &rxq->rx_descs[rxq->cidx]; + odp_packet_hdr_t *pkt_hdr; + odp_packet_t pkt = pkt_table[rx_pkts]; + uint16_t pkt_len; + + pkt_len = odp_le_to_cpu_16(rxd->length); + + ret = odp_packet_copy_from_mem(pkt, 0, pkt_len, + (uint8_t *)rxq->rx_data.vaddr + + rxq->cidx * E1000E_RX_BUF_SIZE); + if (odp_unlikely(ret)) + break; + + pkt_hdr = odp_packet_hdr(pkt); + pkt_hdr->input = pktio_entry->s.handle; + + rxq->cidx++; + if (odp_unlikely(rxq->cidx >= rxq->rx_queue_len)) + rxq->cidx = 0; + + rx_pkts++; + } + + if (rx_pkts) + e1000e_rx_refill(rxq, refill_from, rx_pkts); + + if (!pkt_e1000e->lockless_rx) + odp_ticketlock_unlock(&rxq->lock); + + if (rx_pkts < budget) + odp_packet_free_multi(pkt_table + rx_pkts, budget - rx_pkts); + + return rx_pkts; +} + +static int e1000e_send(pktio_entry_t *pktio_entry, int txq_idx, + const odp_packet_t pkt_table[], int num) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + e1000e_tx_queue_t *txq = &pkt_e1000e->tx_queues[txq_idx]; + uint16_t budget; + int tx_pkts = 0; + + if (!pkt_e1000e->lockless_tx) + odp_ticketlock_lock(&txq->lock); + + /* Determine how many packets will fit in TX queue */ + budget = txq->tx_queue_len - 1; + budget -= txq->pidx; + budget += odp_le_to_cpu_32(*txq->cidx); + budget &= txq->tx_queue_len - 1; + + if (budget > num) + budget = num; + + while (tx_pkts < budget) { + uint16_t pkt_len = _odp_packet_len(pkt_table[tx_pkts]); + uint32_t offset = txq->pidx * E1000E_TX_BUF_SIZE; + + e1000e_tx_desc_t *txd = &txq->tx_descs[txq->pidx]; + + uint32_t txd_cmd = E1000_TXD_CMD_IFCS | E1000_TXD_CMD_EOP; + + /* Skip oversized packets silently */ + if (odp_unlikely(pkt_len > E1000E_TX_BUF_SIZE)) { + tx_pkts++; + continue; + } + + odp_packet_copy_to_mem(pkt_table[tx_pkts], 0, pkt_len, + (uint8_t *)txq->tx_data.vaddr + offset); + + txd->addr = odp_cpu_to_le_64(txq->tx_data.iova + offset); + txd->cmd = odp_cpu_to_le_32(txd_cmd | pkt_len); + + txq->pidx++; + if (odp_unlikely(txq->pidx >= txq->tx_queue_len)) + txq->pidx = 0; + + tx_pkts++; + } + + /* Ring the doorbell */ + if (tx_pkts) + odpdrv_mmio_u32le_write(txq->pidx, txq->doorbell); + + if (!pkt_e1000e->lockless_tx) + odp_ticketlock_unlock(&txq->lock); + + odp_packet_free_multi(pkt_table, tx_pkts); + + return tx_pkts; +} + +static int e1000e_link_status(pktio_entry_t *pktio_entry) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + return link_status_fd(pkt_e1000e->sockfd, pkt_e1000e->mdev.if_name); +} + +static void e1000e_wait_link_up(pktio_entry_t *pktio_entry) +{ + while (!e1000e_link_status(pktio_entry)) + sleep(1); +} + +static int e1000e_capability(pktio_entry_t *pktio_entry, + odp_pktio_capability_t *capa) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + *capa = pkt_e1000e->capa; + return 0; +} + +static int e1000e_input_queues_config(pktio_entry_t *pktio_entry, + const odp_pktin_queue_param_t *p) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + if (p->op_mode == ODP_PKTIO_OP_MT_UNSAFE) + pkt_e1000e->lockless_rx = 1; + else + pkt_e1000e->lockless_rx = 0; + + return 0; +} + +static int e1000e_output_queues_config(pktio_entry_t *pktio_entry, + const odp_pktout_queue_param_t *p) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + if (p->op_mode == ODP_PKTIO_OP_MT_UNSAFE) + pkt_e1000e->lockless_tx = 1; + else + pkt_e1000e->lockless_tx = 0; + + return 0; +} + +static int e1000e_mac_get(pktio_entry_t *pktio_entry, void *mac_addr) +{ + pktio_ops_e1000e_data_t *pkt_e1000e = pktio_entry->s.ops_data; + + if (mac_addr_get_fd(pkt_e1000e->sockfd, pkt_e1000e->mdev.if_name, + mac_addr) < 0) + return -1; + + return ETH_ALEN; +} + +static int e1000e_init_global(void) +{ + ODP_PRINT("PKTIO: initialized " MODULE_NAME " interface\n"); + return 0; +} + +static pktio_ops_module_t e1000e_pktio_ops = { + .base = { + .name = MODULE_NAME, + .init_global = e1000e_init_global, + }, + + .open = e1000e_open, + .close = e1000e_close, + + .recv = e1000e_recv, + .send = e1000e_send, + + .link_status = e1000e_link_status, + + .capability = e1000e_capability, + + .mac_get = e1000e_mac_get, + + .input_queues_config = e1000e_input_queues_config, + .output_queues_config = e1000e_output_queues_config, +}; + +/** e1000e module entry point */ +ODP_MODULE_CONSTRUCTOR(e1000e_pktio_ops) +{ + odp_module_constructor(&e1000e_pktio_ops); + + odp_subsystem_register_module(pktio_ops, &e1000e_pktio_ops); +} + +/* + * Temporary variable to enable link this module, + * will remove in Makefile scheme changes. + */ +int enable_link_e1000e_pktio_ops; + +#endif /* ODP_MDEV */ diff --git a/platform/linux-generic/pktio/subsystem.c b/platform/linux-generic/pktio/subsystem.c index 199ed83fb..34f4e0b31 100644 --- a/platform/linux-generic/pktio/subsystem.c +++ b/platform/linux-generic/pktio/subsystem.c @@ -46,6 +46,7 @@ extern int enable_link_tap_pktio_ops; #endif #if defined(ODP_MDEV) && ODP_MDEV == 1 extern int enable_link_cxgb4_pktio_ops; +extern int enable_link_e1000e_pktio_ops; extern int enable_link_i40e_pktio_ops; #endif @@ -79,6 +80,7 @@ ODP_SUBSYSTEM_CONSTRUCTOR(pktio_ops) #endif #if defined(ODP_MDEV) && ODP_MDEV == 1 enable_link_cxgb4_pktio_ops = 1; + enable_link_e1000e_pktio_ops = 1; enable_link_i40e_pktio_ops = 1; #endif }