From patchwork Fri Feb 17 20:57:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 94170 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp346343qgi; Fri, 17 Feb 2017 12:57:14 -0800 (PST) X-Received: by 10.55.31.35 with SMTP id f35mr8804145qkf.312.1487365034911; Fri, 17 Feb 2017 12:57:14 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id s65si8259234qtd.192.2017.02.17.12.57.14; Fri, 17 Feb 2017 12:57:14 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 808D1634D7; Fri, 17 Feb 2017 20:57:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 2505462DC8; Fri, 17 Feb 2017 20:57:10 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id E094262F4A; Fri, 17 Feb 2017 20:57:08 +0000 (UTC) Received: from mail-oi0-f42.google.com (mail-oi0-f42.google.com [209.85.218.42]) by lists.linaro.org (Postfix) with ESMTPS id E78C062DA6 for ; Fri, 17 Feb 2017 20:57:07 +0000 (UTC) Received: by mail-oi0-f42.google.com with SMTP id 62so16644304oih.2 for ; Fri, 17 Feb 2017 12:57:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=INJYoxHEQImHZm9gSXsB+WThvgW316BNuX9erME5ofY=; b=e2WyhJChDi/DFYggzytTis0QNK/V9M6iy+gufaw1G8WH0sa8Rgchrje7WTfFBRiv6F Eoeyc9lGqZH6Jh7tMYmty3ytXxN+bszbhKd82BVT6xWQlA9YiXvYNdNFzBHezGKzrSdr +tDcHG+aJsTqpM88CP9oDzkVIG9/X20LHVfofB3jQhXNqmR0a/4NXGke/Ce7zIE261NN 3zNiBwTLJtmEGutLBqCk5FbVbVKnUjGu9Rorkq06Sb/gCIHh3u89W9zRAnF+3xMfPS02 dytVgtD3X4YgF6+5u5gx5t6jbh5F44bTS2Iy0/rXzvF34DmEs9rdKwr5x735mv2+75in lBlQ== X-Gm-Message-State: AMke39ntVIFFZ+GJxFJERCjYQE07UOUBWuhCT0szTOPTboHSHoEkPIGujwiThMtjKRefevgODh4= X-Received: by 10.202.71.137 with SMTP id u131mr5414353oia.111.1487365027175; Fri, 17 Feb 2017 12:57:07 -0800 (PST) Received: from localhost.localdomain (cpe-70-121-83-241.austin.res.rr.com. [70.121.83.241]) by smtp.gmail.com with ESMTPSA id j57sm4889749otb.1.2017.02.17.12.57.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Feb 2017 12:57:06 -0800 (PST) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Fri, 17 Feb 2017 14:57:03 -0600 Message-Id: <20170217205703.7230-1-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.12.0.rc1 Subject: [lng-odp] [API-NEXT PATCHv3] linux-generic: packet: minimize use of atomic ops on free paths X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" While reference counts are implemented as atomics, try to avoid unnecessary atomic decrement operations in free paths. This streamlines free processing in the expected case where packets do not contain shared segments. Signed-off-by: Bill Fischofer --- platform/linux-generic/include/odp_packet_internal.h | 6 +++++- platform/linux-generic/odp_packet.c | 20 +++++++++++--------- 2 files changed, 16 insertions(+), 10 deletions(-) -- 2.12.0.rc1 diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index 593f30dd..5864785a 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -257,7 +257,11 @@ static inline uint32_t packet_len(odp_packet_hdr_t *pkt_hdr) static inline uint32_t packet_ref_count(odp_packet_hdr_t *pkt_hdr) { - return odp_atomic_load_u32(&pkt_hdr->ref_count); + /* Breach the atomic type to do a peek at the ref count. This + * is used to bypass atomic operations if ref_count == 1 for + * performance reasons. + */ + return pkt_hdr->ref_count.v; } static inline void packet_ref_count_set(odp_packet_hdr_t *pkt_hdr, uint32_t n) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 4d2cde12..81bbcedd 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -477,7 +477,7 @@ static inline void free_bufs(odp_packet_hdr_t *pkt_hdr, int first, int num) for (i = 0, nfree = 0; i < num; i++) { odp_packet_hdr_t *hdr = pkt_hdr->buf_hdr.seg[first + i].hdr; - if (packet_ref_dec(hdr) == 1) + if (packet_ref_count(hdr) == 1 || packet_ref_dec(hdr) == 1) buf[nfree++] = buffer_handle(hdr); } @@ -499,7 +499,8 @@ static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, for (i = 0, nfree = 0; i < num; i++) { new_hdr = pkt_hdr->buf_hdr.seg[i].hdr; - if (packet_ref_dec(new_hdr) == 1) + if (packet_ref_count(new_hdr) == 1 || + packet_ref_dec(new_hdr) == 1) buf[nfree++] = buffer_handle(new_hdr); } @@ -643,12 +644,12 @@ static inline odp_packet_hdr_t *packet_free_to_list(odp_packet_hdr_t *pkt_hdr, int num_seg, i; do { - ref_count = packet_ref_count(pkt_hdr) - 1; + ref_count = packet_ref_count(pkt_hdr); num_seg = pkt_hdr->buf_hdr.segcount; ref_hdr = pkt_hdr->ref_hdr; if (odp_likely((CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1) && - ref_count == 0)) { + ref_count == 1)) { if (*nfree >= nbufs) break; @@ -661,11 +662,12 @@ static inline odp_packet_hdr_t *packet_free_to_list(odp_packet_hdr_t *pkt_hdr, odp_packet_hdr_t *hdr = pkt_hdr->buf_hdr.seg[i].hdr; - if (packet_ref_dec(hdr) == 1) + if (packet_ref_count(hdr) == 1 || + packet_ref_dec(hdr) == 1) buf[(*nfree)++] = buffer_handle(hdr); } - if (ref_count == 1) + if (ref_count == 2) pkt_hdr->unshared_len = pkt_hdr->frame_len; } @@ -682,18 +684,18 @@ static inline void packet_free(odp_packet_hdr_t *pkt_hdr) int num_seg; do { - ref_count = packet_ref_count(pkt_hdr) - 1; + ref_count = packet_ref_count(pkt_hdr); num_seg = pkt_hdr->buf_hdr.segcount; ref_hdr = pkt_hdr->ref_hdr; if (odp_likely((CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1) && - ref_count == 0)) { + ref_count == 1)) { buffer_free_multi((odp_buffer_t *) &pkt_hdr->buf_hdr.handle.handle, 1); } else { free_bufs(pkt_hdr, 0, num_seg); - if (ref_count == 1) + if (ref_count == 2) pkt_hdr->unshared_len = pkt_hdr->frame_len; }