From patchwork Sat Aug 8 03:03:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 52064 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f198.google.com (mail-lb0-f198.google.com [209.85.217.198]) by patches.linaro.org (Postfix) with ESMTPS id 06B6920539 for ; Sat, 8 Aug 2015 03:11:25 +0000 (UTC) Received: by lbcjf8 with SMTP id jf8sf39185318lbc.0 for ; Fri, 07 Aug 2015 20:11:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=SRSahJuIBOXGiooqbO1FG73+nGN8ZiLGO+q8O/dGw9M=; b=TvKyRIHpPH/i3sdbdfXa3rjOdQy4hPd/lIaIGHY/zfMNWoqXSt9+1YK4IT/XRF9XLX cEjNEVWgwC1OugS2rQXu/Jtj6G+4I0og5R/Kwiq39saVHbA9hnCOgUcKY39josEdlFJX KI7rZTGaQJaGjxIdvmgHMWr956UyIY+NyUAcVzdadFG07D/u5as6JTxXGfqZ+pi1EpTq 8chHzZ0KOKfBOGwuedvZe+qBgiJpaSt9wiyRLdB9Xmh5jOMX9Ye2kjq1uOe5mzzpozin 68GzEpIQ4dwavcdticEniXTZAxN39WnrOnsC18C5npOeXoI8fcArsFSWhSXdNTem44Ag 53Ig== X-Gm-Message-State: ALoCoQnYIjI2u/0bf4ezCdP8di9YFKIgbwC/ih4Nk9Cj02NCID8+DZKtFtdzZ2OEFm9s0/YbpQcv X-Received: by 10.180.83.33 with SMTP id n1mr397684wiy.4.1439003483948; Fri, 07 Aug 2015 20:11:23 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.170.195 with SMTP id ao3ls591826lac.53.gmail; Fri, 07 Aug 2015 20:11:23 -0700 (PDT) X-Received: by 10.152.22.168 with SMTP id e8mr11073700laf.40.1439003483774; Fri, 07 Aug 2015 20:11:23 -0700 (PDT) Received: from mail-la0-f46.google.com (mail-la0-f46.google.com. [209.85.215.46]) by mx.google.com with ESMTPS id eg8si3647144lac.1.2015.08.07.20.11.23 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 20:11:23 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.46 as permitted sender) client-ip=209.85.215.46; Received: by lagz9 with SMTP id z9so31235997lag.3 for ; Fri, 07 Aug 2015 20:11:23 -0700 (PDT) X-Received: by 10.112.133.72 with SMTP id pa8mr10961844lbb.117.1439003483648; Fri, 07 Aug 2015 20:11:23 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.7.198 with SMTP id l6csp719027lba; Fri, 7 Aug 2015 20:11:22 -0700 (PDT) X-Received: by 10.140.237.67 with SMTP id i64mr20229329qhc.5.1439003482036; Fri, 07 Aug 2015 20:11:22 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id 80si2087153qgi.59.2015.08.07.20.11.21; Fri, 07 Aug 2015 20:11:22 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 2C0516238F; Sat, 8 Aug 2015 03:11:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 30F0F623C0; Sat, 8 Aug 2015 03:04:41 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id A54A3623C2; Sat, 8 Aug 2015 03:04:33 +0000 (UTC) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by lists.linaro.org (Postfix) with ESMTPS id 876D462398 for ; Sat, 8 Aug 2015 03:03:26 +0000 (UTC) Received: by oip136 with SMTP id 136so63010154oip.1 for ; Fri, 07 Aug 2015 20:03:26 -0700 (PDT) X-Received: by 10.202.207.139 with SMTP id f133mr9411518oig.28.1439003005994; Fri, 07 Aug 2015 20:03:25 -0700 (PDT) Received: from localhost.localdomain (cpe-24-28-70-239.austin.res.rr.com. [24.28.70.239]) by smtp.gmail.com with ESMTPSA id kp7sm7375667oeb.14.2015.08.07.20.03.25 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 07 Aug 2015 20:03:25 -0700 (PDT) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Fri, 7 Aug 2015 22:03:07 -0500 Message-Id: <1439002992-29285-10-git-send-email-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1439002992-29285-1-git-send-email-bill.fischofer@linaro.org> References: <1439002992-29285-1-git-send-email-bill.fischofer@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv10 09/14] linux-generic: schedule: implement odp_schedule_release_ordered() X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bill.fischofer@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.46 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Bill Fischofer --- platform/linux-generic/odp_queue.c | 169 +++++++++++++++++++++++++++++++++---- 1 file changed, 152 insertions(+), 17 deletions(-) diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index 4d0e1b4..a2460e7 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -401,6 +402,7 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) return 0; } + /* We're in order, so account for this and proceed with enq */ origin_qe->s.order_out++; } @@ -426,16 +428,56 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) */ if (origin_qe) { odp_buffer_hdr_t *reorder_buf = origin_qe->s.reorder_head; - odp_buffer_hdr_t *reorder_prev; + odp_buffer_hdr_t *reorder_prev = NULL; + odp_buffer_hdr_t *placeholder_buf = NULL; + odp_buffer_hdr_t *next_buf; uint32_t release_count = 0; + uint32_t placeholder_count = 0; while (reorder_buf && - reorder_buf->target_qe == queue && - reorder_buf->order <= - origin_qe->s.order_out + release_count) { - release_count++; - reorder_prev = reorder_buf; - reorder_buf = reorder_buf->next; + reorder_buf->order <= origin_qe->s.order_out + + release_count + placeholder_count) { + /* + * Elements on the reorder list fall into one of + * three categories: + * + * 1. Those destined for the same queue. These + * can be enq'd now if they were waiting to + * be unblocked by this enq. + * + * 2. Those representing placeholders for events + * whose ordering was released by a prior + * odp_schedule_release_ordered() call. These + * can now just be freed. + * + * 3. Those representing events destined for another + * queue. These cannot be consolidated with this + * enq since they have a different target. + * + * Detecting an element with an order sequence gap, an + * element in category 3, or running out of elements + * stops the scan. + */ + next_buf = reorder_buf->next; + + if (odp_likely(reorder_buf->target_qe == queue)) { + reorder_prev = reorder_buf; + reorder_buf = next_buf; + release_count++; + } else if (!reorder_buf->target_qe) { + if (reorder_prev) + reorder_prev->next = next_buf; + else + origin_qe->s.reorder_head = next_buf; + + reorder_buf->next = placeholder_buf; + placeholder_buf = reorder_buf; + + reorder_buf = next_buf; + placeholder_count++; + } else { + break; + } } /* Add released buffers to the queue as well */ @@ -444,19 +486,28 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) queue->s.tail = reorder_prev; origin_qe->s.reorder_head = reorder_prev->next; reorder_prev->next = NULL; - origin_qe->s.order_out += release_count; } - /* Now handle unblocked buffers destined for other queues */ + /* Reflect the above two in the output sequence */ + origin_qe->s.order_out += release_count + placeholder_count; + + /* Now handle any unblocked buffers destined for other queues */ + UNLOCK(&queue->s.lock); if (reorder_buf && - reorder_buf->order <= origin_qe->s.order_out) { - UNLOCK(&origin_qe->s.lock); - UNLOCK(&queue->s.lock); - if (schedule_enq(reorder_buf->target_qe, origin_qe)) - ODP_ABORT("schedule_enq failed\n"); - } else { - UNLOCK(&origin_qe->s.lock); - UNLOCK(&queue->s.lock); + reorder_buf->order <= origin_qe->s.order_out) + origin_qe->s.reorder_head = reorder_buf->next; + else + reorder_buf = NULL; + UNLOCK(&origin_qe->s.lock); + if (reorder_buf) + odp_queue_enq(reorder_buf->target_qe->s.handle, + (odp_event_t)reorder_buf->handle.handle); + + /* Free all placeholder bufs that are now released */ + while (placeholder_buf) { + next_buf = placeholder_buf->next; + odp_buffer_free(buf_hdr->handle.handle); + placeholder_buf = next_buf; } } else { UNLOCK(&queue->s.lock); @@ -685,3 +736,87 @@ void odp_queue_param_init(odp_queue_param_t *params) { memset(params, 0, sizeof(odp_queue_param_t)); } + +/* This routine exists here rather than in odp_schedule + * because it operates on queue interenal structures + */ +int odp_schedule_release_ordered(odp_event_t ev) +{ + odp_buffer_t placeholder_buf; + odp_buffer_hdr_t *placeholder_buf_hdr, *reorder_buf, *reorder_prev; + odp_buffer_hdr_t *buf_hdr = + odp_buf_to_hdr(odp_buffer_from_event(ev)); + queue_entry_t *origin_qe = buf_hdr->origin_qe; + + /* Can't release if we didn't originate from an ordered queue */ + if (!origin_qe) + return -1; + + LOCK(&origin_qe->s.lock); + + /* If we are the first or second element beyond the next in order, + * we can release immediately since there can be no confusion about + * intermediate elements + */ + if (buf_hdr->order <= origin_qe->s.order_out + 1) { + buf_hdr->origin_qe = NULL; + origin_qe->s.order_out++; + + /* check if this release allows us to unblock waiters */ + reorder_buf = origin_qe->s.reorder_head; + if (reorder_buf && + reorder_buf->order <= origin_qe->s.order_out) + origin_qe->s.reorder_head = reorder_buf->next; + else + reorder_buf = NULL; + UNLOCK(&origin_qe->s.lock); + if (reorder_buf) + odp_queue_enq(reorder_buf->target_qe->s.handle, + (odp_event_t)reorder_buf->handle.handle); + return 0; + } + + /* If we are beyond the second element in the expected order, we need + * a placeholder to represent our "place in line". Just use an element + * from the same pool the buffer being released is from. + */ + placeholder_buf = odp_buffer_alloc(buf_hdr->pool_hdl); + + /* Can't release if no placeholder is available */ + if (placeholder_buf == ODP_BUFFER_INVALID) { + UNLOCK(&origin_qe->s.lock); + return -1; + } + + placeholder_buf_hdr = odp_buf_to_hdr(placeholder_buf); + reorder_buf = origin_qe->s.reorder_head; + + if (!reorder_buf) { + placeholder_buf_hdr->next = NULL; + origin_qe->s.reorder_head = placeholder_buf_hdr; + origin_qe->s.reorder_tail = placeholder_buf_hdr; + } else { + reorder_prev = NULL; + + while (buf_hdr->order > reorder_buf->order) { + reorder_prev = reorder_buf; + reorder_buf = reorder_buf->next; + if (!reorder_buf) + break; + } + + placeholder_buf_hdr->next = reorder_buf; + if (reorder_prev) + reorder_prev->next = placeholder_buf_hdr; + else + origin_qe->s.reorder_head = placeholder_buf_hdr; + + if (!reorder_buf) + origin_qe->s.reorder_tail = placeholder_buf_hdr; + } + + placeholder_buf_hdr->target_qe = NULL; + + UNLOCK(&origin_qe->s.lock); + return 0; +}