From patchwork Thu Nov 20 11:53:44 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ola Liljedahl X-Patchwork-Id: 41233 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ee0-f71.google.com (mail-ee0-f71.google.com [74.125.83.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id D0CF523C27 for ; Thu, 20 Nov 2014 11:54:31 +0000 (UTC) Received: by mail-ee0-f71.google.com with SMTP id c13sf1968809eek.2 for ; Thu, 20 Nov 2014 03:54:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:subject:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list; bh=gcjS5opokrb4SfnzwApbfrxUa5K5p+iNunrOxTJb0kM=; b=YnZjLRuHfdRZC4pmsoB1+n6u4EV0XEiWvHB/6xcpnQnkQFlNmaLJ9s+f2bVjKbv15G xZDaq6lrxCBbqYz8up+EGvmpda+UvcpT5GRpHOZs9rKrM4hPkrPWAC+914QS8q7fMwQ+ JWDmTLDyTMQ/j1vzjXqlyT68M9W1rTkf53xw7yqtICQ++YZkwxpPXtWQQ4AUd3DIUk0c xjUEuRkupPETpyNPqtSeAoaA7tsM0miwFNp3dARqPfcGix5fkivzwAAflFCSYb5L1/YK 6YCPQV+2UWLJKP0a1Je2N7pK+gXaYaKCWiqgDXVXMQ4aUT1uGmpxYe6h94ggxNztmfym BR+A== X-Gm-Message-State: ALoCoQns661kou8/cT+zj3DGO58xO0NUZD6U1gqb+UffqAA07ID1tIPBiUNsBK1/FY9FBBdQUrs9 X-Received: by 10.181.27.135 with SMTP id jg7mr2674068wid.5.1416484471067; Thu, 20 Nov 2014 03:54:31 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.8.133 with SMTP id r5ls47762laa.65.gmail; Thu, 20 Nov 2014 03:54:30 -0800 (PST) X-Received: by 10.152.224.163 with SMTP id rd3mr47752249lac.24.1416484470709; Thu, 20 Nov 2014 03:54:30 -0800 (PST) Received: from mail-la0-f41.google.com (mail-la0-f41.google.com. [209.85.215.41]) by mx.google.com with ESMTPS id b19si1899513lbq.121.2014.11.20.03.54.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 20 Nov 2014 03:54:30 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.41 as permitted sender) client-ip=209.85.215.41; Received: by mail-la0-f41.google.com with SMTP id gf13so2273879lab.14 for ; Thu, 20 Nov 2014 03:54:30 -0800 (PST) X-Received: by 10.112.138.137 with SMTP id qq9mr2227877lbb.80.1416484470630; Thu, 20 Nov 2014 03:54:30 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.184.201 with SMTP id ew9csp246779lbc; Thu, 20 Nov 2014 03:54:29 -0800 (PST) X-Received: by 10.224.122.80 with SMTP id k16mr53705378qar.40.1416484469344; Thu, 20 Nov 2014 03:54:29 -0800 (PST) Received: from ip-10-35-177-41.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id v19si2276017qah.85.2014.11.20.03.54.28 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 20 Nov 2014 03:54:29 -0800 (PST) Received-SPF: none (google.com: lng-odp-bounces@lists.linaro.org does not designate permitted sender hosts) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-35-177-41.ec2.internal) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XrQJf-000182-Ai; Thu, 20 Nov 2014 11:54:27 +0000 Received: from mail-la0-f50.google.com ([209.85.215.50]) by ip-10-35-177-41.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1XrQJQ-00014z-FQ for lng-odp@lists.linaro.org; Thu, 20 Nov 2014 11:54:12 +0000 Received: by mail-la0-f50.google.com with SMTP id pv20so2254019lab.23 for ; Thu, 20 Nov 2014 03:54:06 -0800 (PST) X-Received: by 10.112.130.10 with SMTP id oa10mr31672766lbb.12.1416484446911; Thu, 20 Nov 2014 03:54:06 -0800 (PST) Received: from macmini.lan (84-217-193-77.tn.glocalnet.net. [84.217.193.77]) by mx.google.com with ESMTPSA id bb3sm534566lab.7.2014.11.20.03.54.05 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Nov 2014 03:54:06 -0800 (PST) From: Ola Liljedahl To: lng-odp@lists.linaro.org Date: Thu, 20 Nov 2014 12:53:44 +0100 Message-Id: <1416484428-23849-4-git-send-email-ola.liljedahl@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1416484428-23849-1-git-send-email-ola.liljedahl@linaro.org> References: <1416484428-23849-1-git-send-email-ola.liljedahl@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCH 3/7] linux-generic: odp_ring.c: use __atomic cmpset, added missing barriers, removed dangerous odp_mem_barrier X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ola.liljedahl@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.41 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Ola Liljedahl --- platform/linux-generic/include/odp_spin_internal.h | 9 ------- platform/linux-generic/odp_ring.c | 28 +++++++++++++++------- 2 files changed, 20 insertions(+), 17 deletions(-) diff --git a/platform/linux-generic/include/odp_spin_internal.h b/platform/linux-generic/include/odp_spin_internal.h index b7e2071..29c524f 100644 --- a/platform/linux-generic/include/odp_spin_internal.h +++ b/platform/linux-generic/include/odp_spin_internal.h @@ -15,15 +15,6 @@ extern "C" { /** - * GCC memory barrier for ODP internal use - */ -static inline void odp_mem_barrier(void) -{ - __asm__ __volatile__ ("" : : : "memory"); -} - - -/** * Spin loop for ODP internal use */ static inline void odp_spin(void) diff --git a/platform/linux-generic/odp_ring.c b/platform/linux-generic/odp_ring.c index 632aa66..1d3130a 100644 --- a/platform/linux-generic/odp_ring.c +++ b/platform/linux-generic/odp_ring.c @@ -259,13 +259,16 @@ int __odph_ring_mp_do_enqueue(odph_ring_t *r, void * const *obj_table, } prod_next = prod_head + n; - success = odp_atomic_cmpset_u32(&r->prod.head, prod_head, - prod_next); + success = __atomic_compare_exchange_n(&r->prod.head, + &prod_head, + prod_next, + false/*strong*/, + __ATOMIC_ACQUIRE, + __ATOMIC_RELAXED); } while (odp_unlikely(success == 0)); /* write entries in ring */ ENQUEUE_PTRS(); - odp_mem_barrier(); /* if we exceed the watermark */ if (odp_unlikely(((mask + 1) - free_entries + n) > r->prod.watermark)) { @@ -282,6 +285,8 @@ int __odph_ring_mp_do_enqueue(odph_ring_t *r, void * const *obj_table, while (odp_unlikely(r->prod.tail != prod_head)) odp_spin(); + /* Release our entries and the memory they refer to */ + __atomic_thread_fence(__ATOMIC_RELEASE); r->prod.tail = prod_next; return ret; } @@ -324,7 +329,6 @@ int __odph_ring_sp_do_enqueue(odph_ring_t *r, void * const *obj_table, /* write entries in ring */ ENQUEUE_PTRS(); - odp_mem_barrier(); /* if we exceed the watermark */ if (odp_unlikely(((mask + 1) - free_entries + n) > r->prod.watermark)) { @@ -334,6 +338,8 @@ int __odph_ring_sp_do_enqueue(odph_ring_t *r, void * const *obj_table, ret = (behavior == ODPH_RING_QUEUE_FIXED) ? 0 : n; } + /* Release our entries and the memory they refer to */ + __atomic_thread_fence(__ATOMIC_RELEASE); r->prod.tail = prod_next; return ret; } @@ -378,13 +384,16 @@ int __odph_ring_mc_do_dequeue(odph_ring_t *r, void **obj_table, } cons_next = cons_head + n; - success = odp_atomic_cmpset_u32(&r->cons.head, cons_head, - cons_next); + success = __atomic_compare_exchange_n(&r->cons.head, + &cons_head, + cons_next, + false/*strong*/, + __ATOMIC_ACQUIRE, + __ATOMIC_RELAXED); } while (odp_unlikely(success == 0)); /* copy in table */ DEQUEUE_PTRS(); - odp_mem_barrier(); /* * If there are other dequeues in progress that preceded us, @@ -393,6 +402,8 @@ int __odph_ring_mc_do_dequeue(odph_ring_t *r, void **obj_table, while (odp_unlikely(r->cons.tail != cons_head)) odp_spin(); + /* Release our entries and the memory they refer to */ + __atomic_thread_fence(__ATOMIC_RELEASE); r->cons.tail = cons_next; return behavior == ODPH_RING_QUEUE_FIXED ? 0 : n; @@ -431,9 +442,10 @@ int __odph_ring_sc_do_dequeue(odph_ring_t *r, void **obj_table, cons_next = cons_head + n; r->cons.head = cons_next; + /* Acquire the pointers and the memory they refer to */ + __atomic_thread_fence(__ATOMIC_ACQUIRE); /* copy in table */ DEQUEUE_PTRS(); - odp_mem_barrier(); r->cons.tail = cons_next; return behavior == ODPH_RING_QUEUE_FIXED ? 0 : n;