From patchwork Fri Oct 23 09:16:49 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stuart Haslam X-Patchwork-Id: 55468 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lf0-f70.google.com (mail-lf0-f70.google.com [209.85.215.70]) by patches.linaro.org (Postfix) with ESMTPS id 5C54F20581 for ; Fri, 23 Oct 2015 09:18:27 +0000 (UTC) Received: by lffz202 with SMTP id z202sf45630395lff.3 for ; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=CKYATCblnVy9+TdV7eaO2pwCrhKGZgiTsWioGwN/yAM=; b=nDBJj8qk1YgCF/fwB4jE+Lnn8g6B97D2wHCcJcP0kz+VP0A/ePDJS3BMFWHFHQ3kSt fHVm+1ZUVdgudUbvKZbgzdsgpjWyVkEck434HyyL2SVbKTXXC2FxUyJHgDzs/Vm7i9Ol BhrDS84MSQ+NslsgOBHvDhHdv1DMbYxGbnbVBR7+6+axkKuYmCKosqfBSJftQ2mnc+hU WpOhOnnugH/gmBb0IIC1GnMxjmpSBZc4Rki61EO29niO9zP06M4oNeqGjb2cIOaBRj1+ RPwp40ApY3aglkFUc2yaawCcV68I6m8V7HmZAmfIbbxWaj6Yp1uShMt75aGOV02nwqQs 5S2A== X-Gm-Message-State: ALoCoQm7V7LDUfZrr22EF0hRRJNDPoM89zlp4PKlgr7IP4JNgFc562J03yiuxERymvu8l8kwLsjS X-Received: by 10.112.142.195 with SMTP id ry3mr4719299lbb.22.1445591906363; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.25.19.100 with SMTP id j97ls446894lfi.50.gmail; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) X-Received: by 10.112.130.2 with SMTP id oa2mr10949290lbb.14.1445591906225; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com. [2a00:1450:4010:c07::22b]) by mx.google.com with ESMTPS id un4si12116721lbb.142.2015.10.23.02.18.26 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Oct 2015 02:18:26 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c07::22b as permitted sender) client-ip=2a00:1450:4010:c07::22b; Received: by lfaz124 with SMTP id z124so76641555lfa.1 for ; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) X-Received: by 10.112.161.168 with SMTP id xt8mr10508129lbb.88.1445591906003; Fri, 23 Oct 2015 02:18:26 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp1092966lbq; Fri, 23 Oct 2015 02:18:25 -0700 (PDT) X-Received: by 10.55.209.147 with SMTP id o19mr23780428qkl.61.1445591905004; Fri, 23 Oct 2015 02:18:25 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id y35si17715563qgd.46.2015.10.23.02.18.24; Fri, 23 Oct 2015 02:18:24 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 8369E62C7A; Fri, 23 Oct 2015 09:18:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, T_DKIM_INVALID, T_FRT_STOCK2,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 161B062C94; Fri, 23 Oct 2015 09:17:23 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0DE3262C8D; Fri, 23 Oct 2015 09:17:17 +0000 (UTC) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by lists.linaro.org (Postfix) with ESMTPS id 66D8962C77 for ; Fri, 23 Oct 2015 09:17:15 +0000 (UTC) Received: by wicfv8 with SMTP id fv8so22994519wic.0 for ; Fri, 23 Oct 2015 02:17:14 -0700 (PDT) X-Received: by 10.194.108.103 with SMTP id hj7mr22093698wjb.46.1445591834578; Fri, 23 Oct 2015 02:17:14 -0700 (PDT) Received: from e106441.emea.arm.com ([2001:41d0:a:3cb4::abcd]) by smtp.gmail.com with ESMTPSA id jt9sm6422177wjc.24.2015.10.23.02.17.13 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 23 Oct 2015 02:17:14 -0700 (PDT) From: Stuart Haslam To: lng-odp@lists.linaro.org Date: Fri, 23 Oct 2015 10:16:49 +0100 Message-Id: <1445591812-10857-3-git-send-email-stuart.haslam@linaro.org> X-Mailer: git-send-email 2.1.1 In-Reply-To: <1445591812-10857-1-git-send-email-stuart.haslam@linaro.org> References: <1445591812-10857-1-git-send-email-stuart.haslam@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCHv4 2/5] linux-generic: pktio: handle transmit errors correctly X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Original-Sender: stuart.haslam@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 2a00:1450:4010:c07::22b as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=neutral (body hash did not verify) header.i=@linaro_org.20150623.gappssmtp.com Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Errors which occur while sending packets via odp_pktio_send() aren't handled correctly or even consistently across the two socket based implementations. The problems being addressed are; - calls may block indefinitely in certain error conditions (mmsg) - packets may be freed and reported as being sent correctly even though they weren't really sent (mmap) - return value doesn't always accurately reflect number of packets sent - inconsistent use of __odp_errno Signed-off-by: Stuart Haslam --- .../linux-generic/include/odp_packet_io_internal.h | 6 ++ platform/linux-generic/pktio/socket.c | 27 +++--- platform/linux-generic/pktio/socket_mmap.c | 96 +++++++++++----------- 3 files changed, 70 insertions(+), 59 deletions(-) diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index 4745bd5..1a1118c 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -32,6 +32,12 @@ extern "C" { #define PKTIO_NAME_LEN 256 +/** Determine if a socket read/write error should be reported. Transient errors + * that simply require the caller to retry are ignored, the _send/_recv APIs + * are non-blocking and it is the caller's responsibility to retry if the + * requested number of packets were not handled. */ +#define SOCK_ERR_REPORT(e) (e != EAGAIN && e != EWOULDBLOCK && e != EINTR) + /* Forward declaration */ struct pktio_if_ops; diff --git a/platform/linux-generic/pktio/socket.c b/platform/linux-generic/pktio/socket.c index a95b9a8..9525665 100644 --- a/platform/linux-generic/pktio/socket.c +++ b/platform/linux-generic/pktio/socket.c @@ -393,9 +393,7 @@ static int sock_mmsg_send(pktio_entry_t *pktio_entry, struct iovec iovecs[ODP_PACKET_SOCKET_MAX_BURST_TX][ODP_BUFFER_MAX_SEG]; int ret; int sockfd; - unsigned i; - unsigned sent_msgs = 0; - unsigned flags; + unsigned n, i; if (odp_unlikely(len > ODP_PACKET_SOCKET_MAX_BURST_TX)) return -1; @@ -409,17 +407,24 @@ static int sock_mmsg_send(pktio_entry_t *pktio_entry, iovecs[i]); } - flags = MSG_DONTWAIT; - for (i = 0; i < len; i += sent_msgs) { - ret = sendmmsg(sockfd, &msgvec[i], len - i, flags); - sent_msgs = ret > 0 ? (unsigned)ret : 0; - flags = 0; /* blocking for next rounds */ + for (i = 0; i < len; ) { + ret = sendmmsg(sockfd, &msgvec[i], len - i, MSG_DONTWAIT); + if (odp_unlikely(ret <= -1)) { + if (i == 0 && SOCK_ERR_REPORT(errno)) { + __odp_errno = errno; + ODP_ERR("sendmmsg(): %s\n", strerror(errno)); + return -1; + } + break; + } + + i += ret; } - for (i = 0; i < len; i++) - odp_packet_free(pkt_table[i]); + for (n = 0; n < i; ++n) + odp_packet_free(pkt_table[n]); - return len; + return i; } /* diff --git a/platform/linux-generic/pktio/socket_mmap.c b/platform/linux-generic/pktio/socket_mmap.c index ba773a3..3f2f91e 100644 --- a/platform/linux-generic/pktio/socket_mmap.c +++ b/platform/linux-generic/pktio/socket_mmap.c @@ -174,49 +174,70 @@ static inline unsigned pkt_mmap_v2_tx(int sock, struct ring *ring, { union frame_map ppd; uint32_t pkt_len; - unsigned frame_num, next_frame_num; + unsigned first_frame_num, frame_num, frame_count; int ret; - unsigned i = 0; + uint8_t *buf; + unsigned n, i = 0; + unsigned nb_tx = 0; + int send_errno; - frame_num = ring->frame_num; + first_frame_num = ring->frame_num; + frame_num = first_frame_num; + frame_count = ring->rd_num; while (i < len) { - if (mmap_tx_kernel_ready(ring->rd[frame_num].iov_base)) { - ppd.raw = ring->rd[frame_num].iov_base; + ppd.raw = ring->rd[frame_num].iov_base; + if (!odp_unlikely(mmap_tx_kernel_ready(ppd.raw))) + break; - next_frame_num = (frame_num + 1) % ring->rd_num; + pkt_len = odp_packet_len(pkt_table[i]); + ppd.v2->tp_h.tp_snaplen = pkt_len; + ppd.v2->tp_h.tp_len = pkt_len; - pkt_len = odp_packet_len(pkt_table[i]); - ppd.v2->tp_h.tp_snaplen = pkt_len; - ppd.v2->tp_h.tp_len = pkt_len; + buf = (uint8_t *)ppd.raw + TPACKET2_HDRLEN - + sizeof(struct sockaddr_ll); + odp_packet_copydata_out(pkt_table[i], 0, pkt_len, buf); - odp_packet_copydata_out(pkt_table[i], 0, pkt_len, - (uint8_t *)ppd.raw + - TPACKET2_HDRLEN - - sizeof(struct sockaddr_ll)); + mmap_tx_user_ready(ppd.raw); - mmap_tx_user_ready(ppd.raw); + if (++frame_num >= frame_count) + frame_num = 0; - odp_packet_free(pkt_table[i]); - frame_num = next_frame_num; - i++; - } else { + i++; + } + + ret = sendto(sock, NULL, 0, MSG_DONTWAIT, NULL, 0); + send_errno = errno; + + /* On success, the return value indicates the number of bytes sent. On + * failure a value of -1 is returned, even if the failure occurred + * after some of the packets in the ring have already been sent, so we + * need to inspect the packet status to determine which were sent. */ + for (n = first_frame_num; n < first_frame_num + i; ++n) { + struct tpacket2_hdr *hdr = ring->rd[n].iov_base; + + if (odp_likely(hdr->tp_status == TP_STATUS_AVAILABLE)) { + nb_tx++; + } else if (hdr->tp_status & TP_STATUS_WRONG_FORMAT) { + /* status will be cleared on the next send request */ break; } } - ring->frame_num = frame_num; + ring->frame_num = (ring->frame_num + nb_tx) % frame_count; - ret = sendto(sock, NULL, 0, MSG_DONTWAIT, NULL, 0); - if (ret == -1) { - if (errno != EAGAIN) { - __odp_errno = errno; - ODP_ERR("sendto(pkt mmap): %s\n", strerror(errno)); - return -1; - } + if (odp_unlikely(ret == -1 && + nb_tx == 0 && + SOCK_ERR_REPORT(send_errno))) { + __odp_errno = send_errno; + ODP_ERR("sendto(pkt mmap): %s\n", strerror(send_errno)); + return -1; } - return i; + for (i = 0; i < nb_tx; ++i) + odp_packet_free(pkt_table[i]); + + return nb_tx; } static void mmap_fill_ring(struct ring *ring, odp_pool_t pool_hdl, int fanout) @@ -260,21 +281,6 @@ static void mmap_fill_ring(struct ring *ring, odp_pool_t pool_hdl, int fanout) ring->flen = ring->req.tp_frame_size; } -static int mmap_set_packet_loss_discard(int sock) -{ - int ret, discard = 1; - - ret = setsockopt(sock, SOL_PACKET, PACKET_LOSS, (void *)&discard, - sizeof(discard)); - if (ret == -1) { - __odp_errno = errno; - ODP_ERR("setsockopt(PACKET_LOSS): %s\n", strerror(errno)); - return -1; - } - - return 0; -} - static int mmap_setup_ring(int sock, struct ring *ring, int type, odp_pool_t pool_hdl, int fanout) { @@ -284,12 +290,6 @@ static int mmap_setup_ring(int sock, struct ring *ring, int type, ring->type = type; ring->version = TPACKET_V2; - if (type == PACKET_TX_RING) { - ret = mmap_set_packet_loss_discard(sock); - if (ret != 0) - return -1; - } - mmap_fill_ring(ring, pool_hdl, fanout); ret = setsockopt(sock, SOL_PACKET, type, &ring->req, sizeof(ring->req));