From patchwork Tue Sep 11 14:00:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 146458 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp3681249ljw; Tue, 11 Sep 2018 07:06:18 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZGTtNmYApnf3+/hT5XKkaFr2uQiXqNwePuQyjYlqQtXZDo/QF3zbVZOWJg50LbZ3AH+YFr X-Received: by 2002:ac8:c0e:: with SMTP id k14-v6mr19735849qti.20.1536674778621; Tue, 11 Sep 2018 07:06:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536674778; cv=none; d=google.com; s=arc-20160816; b=I5oRi2Ve4DdsehmZUwcQEaCNZduL/xxO1NK/A5wEfbH4n8qM8OrM/kwGxZjXMrJ3gu SBNPiGEkzpSiPoq+NfQfkIoKd2tjEnO7y2R5Tpr3eYm8iqyd06l9bWsg0AGfQdwkms3E zE8mnTH9v+WZ6yLXMZuC/TM4cQXwOf19KFLgBXLLR1g+Qqkk+OGRChPnUeOVTgaa6tCD w1vqmKJrGuzXofiTkGocIwlQGuanYFN0c1SaS18qOHQb1TMywcnqO6A3fiHVf7wfEeqI EHglrt5Qlnzuodg9wTylDbGwgY+F/xMXH0VvvKX+/Z7MMMFLuP3lmtEb+XLCWT8HOSoY 4Ndw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to; bh=HO3NVhjyQqXLCnwAhiYlzS7fK3B98wPRygQz/vHl5/8=; b=PVCgqzgzP2WMutpD9a5Rq3gSB/SSajfZKijMB0J10+DIPgPZmoxG6R5rR/p8HNTGyO fpntPFdgYC5lMC2/o/GrZyz8MR6NHCJPsjmxpHoPX4t+UjoWy+Rxpq8fFKqAGVMnTRMr 6Y9UGnn3XmPruNE88mNoT0Zs5QDiXQooPZOKE4IbdY+krfwPu1und0UrV0jawtUiL7zQ LaHdMxQ1nqpNEhrbH67wqOGmf2YANQ4rGVJ8fSt5xfoDHead/QaQk7rX7HUa2/7ePQvL aAMtJ2LYq+hLEcjagNLoOPPdxsH+FQjcfg4ulG5Sx2d9of0iElZ+HMS4UP+hujd3fFLm K+Iw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id j3-v6si4002957qvj.26.2018.09.11.07.06.18; Tue, 11 Sep 2018 07:06:18 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 4A4B261C27; Tue, 11 Sep 2018 14:06:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_LOW autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id CFF7661C2D; Tue, 11 Sep 2018 14:02:27 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id E00B961C1D; Tue, 11 Sep 2018 14:01:54 +0000 (UTC) Received: from forward103o.mail.yandex.net (forward103o.mail.yandex.net [37.140.190.177]) by lists.linaro.org (Postfix) with ESMTPS id D089961C27 for ; Tue, 11 Sep 2018 14:00:34 +0000 (UTC) Received: from mxback16j.mail.yandex.net (mxback16j.mail.yandex.net [IPv6:2a02:6b8:0:1619::92]) by forward103o.mail.yandex.net (Yandex) with ESMTP id 4BA385888814 for ; Tue, 11 Sep 2018 17:00:33 +0300 (MSK) Received: from smtp3p.mail.yandex.net (smtp3p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:8]) by mxback16j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id nobP0hMtzz-0WjCcsEM; Tue, 11 Sep 2018 17:00:32 +0300 Received: by smtp3p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id mBghUNGHde-0TSqgfno; Tue, 11 Sep 2018 17:00:30 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 11 Sep 2018 14:00:16 +0000 Message-Id: <1536674416-8465-7-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536674416-8465-1-git-send-email-odpbot@yandex.ru> References: <1536674416-8465-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 699 Subject: [lng-odp] [PATCH v1 6/6] linux-gen: ring: change ring_deq return value X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Petri Savolainen Return number of data values dequeued (0 or 1) instead of the data value. This improves error tolerance as there's no data value reserved to indicate empty ring. Also CPU may speculate further before the actual data value is actually needed. Signed-off-by: Petri Savolainen --- /** Email created from pull request 699 (psavol:master-sched-optim-clean-ups) ** https://github.com/Linaro/odp/pull/699 ** Patch: https://github.com/Linaro/odp/pull/699.patch ** Base sha: 33fbc04b6373960ec3f84de4e7e7b34c49d71508 ** Merge commit sha: 32d7a11f22e6f2e1e378b653993c5377d4116d8f **/ platform/linux-generic/include/odp_ring_internal.h | 10 ++++------ platform/linux-generic/odp_schedule_basic.c | 8 +++----- platform/linux-generic/odp_schedule_iquery.c | 5 ++--- platform/linux-generic/odp_schedule_sp.c | 3 +-- 4 files changed, 10 insertions(+), 16 deletions(-) diff --git a/platform/linux-generic/include/odp_ring_internal.h b/platform/linux-generic/include/odp_ring_internal.h index 97673bef4..9a637afb3 100644 --- a/platform/linux-generic/include/odp_ring_internal.h +++ b/platform/linux-generic/include/odp_ring_internal.h @@ -18,9 +18,6 @@ extern "C" { #include #include -/* Ring empty, not a valid data value. */ -#define RING_EMPTY ((uint32_t)-1) - /* Ring of uint32_t data * * Ring stores head and tail counters. Ring indexes are formed from these @@ -59,7 +56,7 @@ static inline void ring_init(ring_t *ring) } /* Dequeue data from the ring head */ -static inline uint32_t ring_deq(ring_t *ring, uint32_t mask) +static inline uint32_t ring_deq(ring_t *ring, uint32_t mask, uint32_t *data) { uint32_t head, tail, new_head; @@ -73,7 +70,7 @@ static inline uint32_t ring_deq(ring_t *ring, uint32_t mask) tail = odp_atomic_load_acq_u32(&ring->w_tail); if (head == tail) - return RING_EMPTY; + return 0; new_head = head + 1; @@ -83,7 +80,8 @@ static inline uint32_t ring_deq(ring_t *ring, uint32_t mask) /* Read data. CAS acquire-release ensures that data read * does not move above from here. */ - return ring->data[new_head & mask]; + *data = ring->data[new_head & mask]; + return 1; } /* Dequeue multiple data from the ring head. Num is smaller than ring size. */ diff --git a/platform/linux-generic/odp_schedule_basic.c b/platform/linux-generic/odp_schedule_basic.c index 77fee74dd..a285edc3c 100644 --- a/platform/linux-generic/odp_schedule_basic.c +++ b/platform/linux-generic/odp_schedule_basic.c @@ -416,8 +416,7 @@ static int schedule_term_global(void) ring_t *ring = &sched->prio_q[grp][i][j].ring; uint32_t qi; - while ((qi = ring_deq(ring, ring_mask)) != - RING_EMPTY) { + while (ring_deq(ring, ring_mask, &qi)) { odp_event_t events[1]; int num; @@ -907,10 +906,9 @@ static inline int do_schedule_grp(odp_queue_t *out_queue, odp_event_t out_ev[], /* Get queue index from the priority queue */ ring = &sched->prio_q[grp][prio][id].ring; - qi = ring_deq(ring, ring_mask); - /* Priority queue empty */ - if (qi == RING_EMPTY) { + if (ring_deq(ring, ring_mask, &qi) == 0) { + /* Priority queue empty */ i++; id++; continue; diff --git a/platform/linux-generic/odp_schedule_iquery.c b/platform/linux-generic/odp_schedule_iquery.c index f76942ff3..e1ef10c46 100644 --- a/platform/linux-generic/odp_schedule_iquery.c +++ b/platform/linux-generic/odp_schedule_iquery.c @@ -271,7 +271,7 @@ static int schedule_init_global(void) ring_init(&queue->ring); for (k = 0; k < PKTIO_RING_SIZE; k++) - queue->cmd_index[k] = RING_EMPTY; + queue->cmd_index[k] = -1; } for (i = 0; i < NUM_PKTIO_CMD; i++) @@ -668,9 +668,8 @@ static inline void pktio_poll_input(void) for (i = 0; i < PKTIO_CMD_QUEUES; i++, hash = (hash + 1) % PKTIO_CMD_QUEUES) { ring = &sched->pktio_poll.queues[hash].ring; - index = ring_deq(ring, PKTIO_RING_MASK); - if (odp_unlikely(index == RING_EMPTY)) + if (odp_unlikely(ring_deq(ring, PKTIO_RING_MASK, &index) == 0)) continue; cmd = &sched->pktio_poll.commands[index]; diff --git a/platform/linux-generic/odp_schedule_sp.c b/platform/linux-generic/odp_schedule_sp.c index 8ddd1e94e..6b9431b69 100644 --- a/platform/linux-generic/odp_schedule_sp.c +++ b/platform/linux-generic/odp_schedule_sp.c @@ -401,9 +401,8 @@ static inline sched_cmd_t *rem_head(int group, int prio) int pktio; prio_queue = &sched_global->prio_queue[group][prio]; - ring_idx = ring_deq(&prio_queue->ring, RING_MASK); - if (ring_idx == RING_EMPTY) + if (ring_deq(&prio_queue->ring, RING_MASK, &ring_idx) == 0) return NULL; pktio = index_from_ring_idx(&index, ring_idx);