From patchwork Thu Dec 8 06:32:34 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yi He X-Patchwork-Id: 87199 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp697875qgi; Wed, 7 Dec 2016 22:33:01 -0800 (PST) X-Received: by 10.55.195.20 with SMTP id a20mr60082965qkj.85.1481178781192; Wed, 07 Dec 2016 22:33:01 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id f131si16528324qkb.34.2016.12.07.22.33.00; Wed, 07 Dec 2016 22:33:01 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 80A806340F; Thu, 8 Dec 2016 06:33:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id E7D0C62FF7; Thu, 8 Dec 2016 06:32:55 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 940BB6300A; Thu, 8 Dec 2016 06:32:53 +0000 (UTC) Received: from mail-pg0-f51.google.com (mail-pg0-f51.google.com [74.125.83.51]) by lists.linaro.org (Postfix) with ESMTPS id 65B1962FE1 for ; Thu, 8 Dec 2016 06:32:52 +0000 (UTC) Received: by mail-pg0-f51.google.com with SMTP id f188so170196600pgc.3 for ; Wed, 07 Dec 2016 22:32:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZteZlq08i3aN89TTxk2yQcwucb+8Q1RA7Ftk2rWmbAk=; b=Wqkm7QXh7HR62DCBgJnm5AqPQNlg5V3sNUkz5oab+/gjpur762f4Gd3HFg4b7erqec tjJobGGpEU6D98OS0dfzOTfkkV20k1b7QJVzvjweD4NwrMDuQww5oK8GV7jbc7MYyh1V Lbr8GKBXzXS9Wr5sgGfCrMMqzcGlEunLdmYtmc0wrfvUpcQPanRzKfLIJzFDgaZjFGjY 9WqYKYzJg9nbHQjqP9g4J8yZ1Y2StbOpyRNEai8RrjUxU5HiQyLLyk0VhrtyuEzPPzKx bCImseGWCSeyJXpqzaN8cJCzKte4S74riyK8CjN+hWzWXHzk7KU5OGE5PEiGWRUPdsES FsSQ== X-Gm-Message-State: AKaTC01p5ZMsprHNZDeBaPzB2bABgB5OBqAPYbmxvj9IObrGv0x9NP8D0LVczooUUcxGeX5n8hY= X-Received: by 10.84.128.46 with SMTP id 43mr156090664pla.100.1481178771604; Wed, 07 Dec 2016 22:32:51 -0800 (PST) Received: from ubuntu.heyii.co (ubuntu.heyii.co. [45.32.66.203]) by smtp.googlemail.com with ESMTPSA id 24sm47008537pfh.41.2016.12.07.22.32.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 07 Dec 2016 22:32:50 -0800 (PST) From: Yi He To: lng-odp@lists.linaro.org, bill.fischofer@linaro.org, matias.elo@nokia-bell-labs.com Date: Thu, 8 Dec 2016 06:32:34 +0000 Message-Id: <1481178754-21046-1-git-send-email-yi.he@linaro.org> X-Mailer: git-send-email 1.9.1 Subject: [lng-odp] [API-NEXT PATCH] linux-gen: scheduler: solve ordered context inversion X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" For ordered queue, a thread consumes events (dequeue) and acquires its unique sequential context in two steps, non atomic and preemptable. This leads to potential ordered context inversion in case the thread consumes prior events acquired subsequent context, while the thread consumes subsequent events but acquired prior context. This patch insert the ordered context acquisition into event dequeue operation to make these two steps atomic. Signed-off-by: Yi He --- When I'm rebase iquery scheduler onto Matias' new ordered queue impl I found this potential ordered context inversion issue, it didnot happen to the default scheduler but happens to the iquery scheduler, so after patchset will rely on this patch. please help review thanks. platform/linux-generic/include/odp_schedule_if.h | 3 +++ platform/linux-generic/odp_queue.c | 3 +++ platform/linux-generic/odp_schedule.c | 7 ++++++- platform/linux-generic/odp_schedule_sp.c | 7 ++++++- 4 files changed, 18 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/platform/linux-generic/include/odp_schedule_if.h b/platform/linux-generic/include/odp_schedule_if.h index 6c2b050..c0aee42 100644 --- a/platform/linux-generic/include/odp_schedule_if.h +++ b/platform/linux-generic/include/odp_schedule_if.h @@ -12,6 +12,7 @@ extern "C" { #endif #include +#include #include typedef void (*schedule_pktio_start_fn_t)(int pktio_index, int num_in_queue, @@ -33,6 +34,7 @@ typedef int (*schedule_term_local_fn_t)(void); typedef void (*schedule_order_lock_fn_t)(void); typedef void (*schedule_order_unlock_fn_t)(void); typedef unsigned (*schedule_max_ordered_locks_fn_t)(void); +typedef void (*schedule_save_context_fn_t)(queue_entry_t *queue); typedef struct schedule_fn_t { schedule_pktio_start_fn_t pktio_start; @@ -50,6 +52,7 @@ typedef struct schedule_fn_t { schedule_order_lock_fn_t order_lock; schedule_order_unlock_fn_t order_unlock; schedule_max_ordered_locks_fn_t max_ordered_locks; + schedule_save_context_fn_t save_context; } schedule_fn_t; /* Interface towards the scheduler */ diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index d9cb9f3..3b6a68b 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -568,6 +568,9 @@ static inline int deq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], if (hdr == NULL) queue->s.tail = NULL; + if (queue->s.type == ODP_QUEUE_TYPE_SCHED) + sched_fn->save_context(queue); + UNLOCK(&queue->s.lock); return i; diff --git a/platform/linux-generic/odp_schedule.c b/platform/linux-generic/odp_schedule.c index 645630a..264c58f 100644 --- a/platform/linux-generic/odp_schedule.c +++ b/platform/linux-generic/odp_schedule.c @@ -1205,6 +1205,10 @@ static int schedule_num_grps(void) return NUM_SCHED_GRPS; } +static void schedule_save_context(queue_entry_t *queue ODP_UNUSED) +{ +} + /* Fill in scheduler interface */ const schedule_fn_t schedule_default_fn = { .pktio_start = schedule_pktio_start, @@ -1221,7 +1225,8 @@ const schedule_fn_t schedule_default_fn = { .term_local = schedule_term_local, .order_lock = order_lock, .order_unlock = order_unlock, - .max_ordered_locks = schedule_max_ordered_locks + .max_ordered_locks = schedule_max_ordered_locks, + .save_context = schedule_save_context }; /* Fill in scheduler API calls */ diff --git a/platform/linux-generic/odp_schedule_sp.c b/platform/linux-generic/odp_schedule_sp.c index 76d1357..8481f29 100644 --- a/platform/linux-generic/odp_schedule_sp.c +++ b/platform/linux-generic/odp_schedule_sp.c @@ -676,6 +676,10 @@ static void order_unlock(void) { } +static void save_context(queue_entry_t *queue ODP_UNUSED) +{ +} + /* Fill in scheduler interface */ const schedule_fn_t schedule_sp_fn = { .pktio_start = pktio_start, @@ -692,7 +696,8 @@ const schedule_fn_t schedule_sp_fn = { .term_local = term_local, .order_lock = order_lock, .order_unlock = order_unlock, - .max_ordered_locks = max_ordered_locks + .max_ordered_locks = max_ordered_locks, + .save_context = save_context }; /* Fill in scheduler API calls */