From patchwork Sun Aug 2 23:21:49 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 51815 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by patches.linaro.org (Postfix) with ESMTPS id E7B86229FD for ; Sun, 2 Aug 2015 23:28:16 +0000 (UTC) Received: by lbcjj5 with SMTP id jj5sf38712917lbc.1 for ; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=iu2Gy7qWz9G9/s6sDtoZ3YT6tBpgX0nQKRzqQtKDI9s=; b=R/5yNecjcqJsrlDqrOmiN3EMO9FfEdklcears/3dZ51xSU5890hKQ1yk2tKB2BYJ+I L0olkbyf15WHbCgru5qqqjuzE5f7UU/vO2exAaUHRoZjZl4JCHD+JP9MgZUflhO/Z/Rg tdQyf6bS06cNjxvEk7S6tl9+STMIRS3ktfdEZbhAXuOyaiaori4l80HyjeWbOesyuude 3rchasMstoxFSYs3qrRc7M9WvjRvBnDOfFC3sKl8bz0i9GSlu778WtdoCO5uZUtGuzhl OrgN6NED8wKxjp3tKhwT7ug2gK8ZyjQTdcgle6+69dVTHdBYPeFqWq0FISedzuXCLVsc vCZA== X-Gm-Message-State: ALoCoQmyVz0Hh4EqEMQr/DX+pDINdhhA/Q1SKzkc9tw8i87QfWGZGudAQBeRnLPqiI6GytMqqES0 X-Received: by 10.112.16.200 with SMTP id i8mr4652298lbd.20.1438558095893; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.6.231 with SMTP id e7ls469668laa.34.gmail; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) X-Received: by 10.112.234.163 with SMTP id uf3mr10995309lbc.37.1438558095747; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com. [209.85.215.45]) by mx.google.com with ESMTPS id ba18si10444686lab.100.2015.08.02.16.28.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Aug 2015 16:28:15 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) client-ip=209.85.215.45; Received: by labow3 with SMTP id ow3so5583675lab.1 for ; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) X-Received: by 10.153.7.66 with SMTP id da2mr257379lad.117.1438558095603; Sun, 02 Aug 2015 16:28:15 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.7.198 with SMTP id l6csp1570587lba; Sun, 2 Aug 2015 16:28:14 -0700 (PDT) X-Received: by 10.50.30.65 with SMTP id q1mr18251774igh.28.1438558093832; Sun, 02 Aug 2015 16:28:13 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id 7si12507830iop.86.2015.08.02.16.28.13; Sun, 02 Aug 2015 16:28:13 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id DCB6061D7E; Sun, 2 Aug 2015 23:28:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id B2E6E61D80; Sun, 2 Aug 2015 23:24:25 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 6B77461D95; Sun, 2 Aug 2015 23:24:18 +0000 (UTC) Received: from mail-qg0-f45.google.com (mail-qg0-f45.google.com [209.85.192.45]) by lists.linaro.org (Postfix) with ESMTPS id E1DC261CD3 for ; Sun, 2 Aug 2015 23:23:22 +0000 (UTC) Received: by qgeh16 with SMTP id h16so78308557qge.3 for ; Sun, 02 Aug 2015 16:23:22 -0700 (PDT) X-Received: by 10.140.91.246 with SMTP id z109mr21348033qgd.39.1438557802740; Sun, 02 Aug 2015 16:23:22 -0700 (PDT) Received: from localhost.localdomain ([64.88.227.134]) by smtp.gmail.com with ESMTPSA id b126sm5444557qhc.46.2015.08.02.16.23.18 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 02 Aug 2015 16:23:22 -0700 (PDT) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Sun, 2 Aug 2015 16:21:49 -0700 Message-Id: <1438557709-30355-9-git-send-email-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1438557709-30355-1-git-send-email-bill.fischofer@linaro.org> References: <1438557709-30355-1-git-send-email-bill.fischofer@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv4 8/8] linux-generic: queue: add ordered chain enq support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bill.fischofer@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Bill Fischofer --- .../linux-generic/include/odp_buffer_internal.h | 4 + platform/linux-generic/odp_queue.c | 149 +++++++++++++++------ 2 files changed, 114 insertions(+), 39 deletions(-) diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index c459fce..30f061e 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -109,6 +109,10 @@ typedef union queue_entry_u queue_entry_t; /* Common buffer header */ typedef struct odp_buffer_hdr_t { struct odp_buffer_hdr_t *next; /* next buf in a list */ + union { /* Multi-use secondary link */ + struct odp_buffer_hdr_t *prev; + struct odp_buffer_hdr_t *link; + }; odp_buffer_bits_t handle; /* handle */ union { uint32_t all; diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index a2460e7..8443ead 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -336,6 +336,7 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) { int sched = 0; queue_entry_t *origin_qe = buf_hdr->origin_qe; + odp_buffer_hdr_t *buf_tail; /* Need two locks for enq operations from ordered queues */ if (origin_qe) { @@ -404,17 +405,33 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) /* We're in order, so account for this and proceed with enq */ origin_qe->s.order_out++; + + /* if this element is linked, restore the linked chain */ + buf_tail = buf_hdr->link; + + if (buf_tail) { + buf_hdr->next = buf_tail; + buf_hdr->link = NULL; + + /* find end of the chain */ + while (buf_tail->next) + buf_tail = buf_tail->next; + } else { + buf_tail = buf_hdr; + } + } else { + buf_tail = buf_hdr; } if (queue->s.head == NULL) { /* Empty queue */ queue->s.head = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } else { queue->s.tail->next = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } if (queue->s.status == QUEUE_STATUS_NOTSCHED) { @@ -461,8 +478,23 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) next_buf = reorder_buf->next; if (odp_likely(reorder_buf->target_qe == queue)) { - reorder_prev = reorder_buf; - reorder_buf = next_buf; + /* promote any chain */ + odp_buffer_hdr_t *reorder_link = + reorder_buf->link; + + if (reorder_link) { + reorder_buf->next = reorder_link; + reorder_buf->link = NULL; + while (reorder_link->next) + reorder_link = + reorder_link->next; + reorder_link->next = next_buf; + reorder_prev = reorder_link; + } else { + reorder_prev = reorder_buf; + } + + reorder_buf = next_buf; release_count++; } else if (!reorder_buf->target_qe) { if (reorder_prev) @@ -523,53 +555,89 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) int queue_enq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { int sched = 0; - int i, j; + int i, rc, ret_count = 0; + int ordered_head[num]; + int ordered_count = 0; odp_buffer_hdr_t *tail; + /* Identify ordered chains in the input buffer list */ for (i = 0; i < num; i++) { - /* If any buffer is coming from an ordered queue, enqueue them - * individually since in the general case each might originate - * from a different ordered queue. If any of these fail, the - * return code tells the caller how many succeeded. - */ - if (buf_hdr[i]->origin_qe) { - for (j = 0; j < num; j++) { - if (queue_enq(queue, buf_hdr[j])) - return j; - } - return num; + if (buf_hdr[i]->origin_qe) + ordered_head[ordered_count++] = i; + if (i < num - 1) + buf_hdr[i]->next = buf_hdr[i + 1]; + } + + buf_hdr[num - 1] = NULL; + + if (ordered_count) { + if (ordered_head[0] > 0) { + tail = buf_hdr[ordered_head[0] - 1]; + tail->next = NULL; + ret_count = ordered_head[0]; + } else { + tail = NULL; + ret_count = 0; } - buf_hdr[i]->next = i == num - 1 ? NULL : buf_hdr[i + 1]; + } else { + tail = buf_hdr[num - 1]; + ret_count = num; } - tail = buf_hdr[num-1]; + /* Handle regular enq's at start of list */ + if (tail) { + tail->next = NULL; + LOCK(&queue->s.lock); + if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + UNLOCK(&queue->s.lock); + ODP_ERR("Bad queue status\n"); + return -1; + } - LOCK(&queue->s.lock); - if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + /* Empty queue */ + if (queue->s.head) + queue->s.tail->next = buf_hdr[0]; + else + queue->s.head = buf_hdr[0]; + + queue->s.tail = tail; + + if (queue->s.status == QUEUE_STATUS_NOTSCHED) { + queue->s.status = QUEUE_STATUS_SCHED; + sched = 1; /* retval: schedule queue */ + } UNLOCK(&queue->s.lock); - ODP_ERR("Bad queue status\n"); - return -1; + + /* Add queue to scheduling */ + if (sched && schedule_queue(queue)) + ODP_ABORT("schedule_queue failed\n"); } - /* Empty queue */ - if (queue->s.head == NULL) - queue->s.head = buf_hdr[0]; - else - queue->s.tail->next = buf_hdr[0]; + /* Handle ordered chains in the list */ + for (i = 0; i < ordered_count; i++) { + int eol = i < ordered_count - 1 ? ordered_head[i + 1] : num; + int list_count = eol - i; - queue->s.tail = tail; + if (i < ordered_count - 1) + buf_hdr[ordered_head[i + 1] - 1]->next = NULL; - if (queue->s.status == QUEUE_STATUS_NOTSCHED) { - queue->s.status = QUEUE_STATUS_SCHED; - sched = 1; /* retval: schedule queue */ - } - UNLOCK(&queue->s.lock); + if (ordered_head[i] < eol - 1) + buf_hdr[ordered_head[i]]->link = + buf_hdr[ordered_head[i] + 1]; + else + buf_hdr[ordered_head[i]]->link = NULL; - /* Add queue to scheduling */ - if (sched && schedule_queue(queue)) - ODP_ABORT("schedule_queue failed\n"); + rc = queue_enq(queue, buf_hdr[ordered_head[i]]); + if (rc < 0) + return ret_count; + + if (rc < list_count) + return ret_count + rc; - return num; /* All events enqueued */ + ret_count += rc; + } + + return ret_count; } int odp_queue_enq_multi(odp_queue_t handle, const odp_event_t ev[], int num) @@ -598,6 +666,9 @@ int odp_queue_enq(odp_queue_t handle, odp_event_t ev) queue = queue_to_qentry(handle); buf_hdr = odp_buf_to_hdr(odp_buffer_from_event(ev)); + /* No chains via this entry */ + buf_hdr->link = NULL; + return queue->s.enqueue(queue, buf_hdr); }