From patchwork Tue Aug 25 15:26:40 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Holmes X-Patchwork-Id: 52699 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f72.google.com (mail-la0-f72.google.com [209.85.215.72]) by patches.linaro.org (Postfix) with ESMTPS id 55B3022E8F for ; Tue, 25 Aug 2015 15:30:05 +0000 (UTC) Received: by labia3 with SMTP id ia3sf28712756lab.3 for ; Tue, 25 Aug 2015 08:30:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=AmqBl+fN3NdoRAgmk1KyKLMx2xFA/hFoO/Y1Fe1DkqE=; b=VlRoXkgWbPmqgvwVZ94q/LRFldkrvj9HwyxxyFaLmXEb+Fk00pvC8TDeZEuylJ3Tnp XzEVmjma+XIanBvGHC9Kb1N9OutOjUvBJMjA9y58jkjO46iEcAYogrkFPn+2fXwuUs/x FIXnigtss/ZuxvrNu9CeKKy6T+orPzzRXVO8YAMG6/bzNll8OS+2L2Jx5SYW8HhwVwky 7R0pdzUNt0WBG89II63dr7p0vEFhOod52uUFDiGJCUGiiYcQ6mj8tTmPIc8+ZuWl1vMA z3kMalN1pfFHHJgWh9ksSbRLLvW21K3NLSUD3pnRbi8SSoSrMwky4KlY6FMC1MWeild1 WLaw== X-Gm-Message-State: ALoCoQkXYXfUrobw6LpdhxMPjllosuHmOv6Zzg7z82Rdu3bPXt/jtVm4A6ir9WNVWiDRbuHcoEO7 X-Received: by 10.112.138.2 with SMTP id qm2mr8573974lbb.19.1440516604289; Tue, 25 Aug 2015 08:30:04 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.9.7 with SMTP id v7ls751278laa.107.gmail; Tue, 25 Aug 2015 08:30:04 -0700 (PDT) X-Received: by 10.112.85.3 with SMTP id d3mr25940426lbz.33.1440516603943; Tue, 25 Aug 2015 08:30:03 -0700 (PDT) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com. [209.85.217.180]) by mx.google.com with ESMTPS id o6si16274998lag.87.2015.08.25.08.30.03 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Aug 2015 08:30:03 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) client-ip=209.85.217.180; Received: by lbcbn3 with SMTP id bn3so101847834lbc.2 for ; Tue, 25 Aug 2015 08:30:03 -0700 (PDT) X-Received: by 10.152.36.102 with SMTP id p6mr25885635laj.19.1440516603770; Tue, 25 Aug 2015 08:30:03 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.162.200 with SMTP id yc8csp3257574lbb; Tue, 25 Aug 2015 08:30:01 -0700 (PDT) X-Received: by 10.55.197.213 with SMTP id k82mr25655674qkl.49.1440516601564; Tue, 25 Aug 2015 08:30:01 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id f78si11044514qhe.78.2015.08.25.08.30.00; Tue, 25 Aug 2015 08:30:01 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id C058261F24; Tue, 25 Aug 2015 15:30:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 2D67C61F16; Tue, 25 Aug 2015 15:27:21 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D363261F1B; Tue, 25 Aug 2015 15:27:04 +0000 (UTC) Received: from mail-qk0-f170.google.com (mail-qk0-f170.google.com [209.85.220.170]) by lists.linaro.org (Postfix) with ESMTPS id AEA8161CC9 for ; Tue, 25 Aug 2015 15:26:51 +0000 (UTC) Received: by qkch123 with SMTP id h123so96414708qkc.0 for ; Tue, 25 Aug 2015 08:26:51 -0700 (PDT) X-Received: by 10.55.56.73 with SMTP id f70mr65221457qka.78.1440516411418; Tue, 25 Aug 2015 08:26:51 -0700 (PDT) Received: from localhost.localdomain (c-98-221-136-245.hsd1.nj.comcast.net. [98.221.136.245]) by smtp.gmail.com with ESMTPSA id z71sm13994853qkz.16.2015.08.25.08.26.50 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 25 Aug 2015 08:26:50 -0700 (PDT) From: Mike Holmes To: lng-odp@lists.linaro.org Date: Tue, 25 Aug 2015 11:26:40 -0400 Message-Id: <1440516404-4177-2-git-send-email-mike.holmes@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1440516404-4177-1-git-send-email-mike.holmes@linaro.org> References: <1440516404-4177-1-git-send-email-mike.holmes@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCH 2/6] linux-generic: correct whitespace issues X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: mike.holmes@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.180 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Correct whitespace issues seen by checkpatch using the odp_format tool Signed-off-by: Mike Holmes --- .../linux-generic/arch/linux/odp_time_cycles.c | 4 +- platform/linux-generic/odp_barrier.c | 2 +- platform/linux-generic/odp_buffer.c | 26 ++----- platform/linux-generic/odp_classification.c | 63 ++++++++-------- platform/linux-generic/odp_cpumask.c | 3 +- platform/linux-generic/odp_crypto.c | 64 ++++++---------- platform/linux-generic/odp_impl.c | 1 - platform/linux-generic/odp_init.c | 3 +- platform/linux-generic/odp_packet.c | 88 ++++++++++++---------- platform/linux-generic/odp_packet_flags.c | 1 + platform/linux-generic/odp_packet_io.c | 10 +-- platform/linux-generic/odp_pool.c | 31 ++++---- platform/linux-generic/odp_queue.c | 30 +++----- platform/linux-generic/odp_rwlock.c | 25 +++--- platform/linux-generic/odp_schedule.c | 47 +++++------- platform/linux-generic/odp_shared_memory.c | 31 +++----- platform/linux-generic/odp_spinlock.c | 5 -- platform/linux-generic/odp_system_info.c | 80 +++++++++----------- platform/linux-generic/odp_thread.c | 19 ++--- platform/linux-generic/odp_ticketlock.c | 3 - platform/linux-generic/odp_time.c | 9 +-- platform/linux-generic/odp_timer.c | 52 +++++++++---- platform/linux-generic/pktio/socket.c | 1 + platform/linux-generic/pktio/socket_mmap.c | 2 + 24 files changed, 279 insertions(+), 321 deletions(-) diff --git a/platform/linux-generic/arch/linux/odp_time_cycles.c b/platform/linux-generic/arch/linux/odp_time_cycles.c index 4dc0764..256ad7c 100644 --- a/platform/linux-generic/arch/linux/odp_time_cycles.c +++ b/platform/linux-generic/arch/linux/odp_time_cycles.c @@ -28,8 +28,8 @@ uint64_t odp_time_cycles(void) ODP_ABORT("clock_gettime failed\n"); hz = odp_sys_cpu_hz(); - sec = (uint64_t) time.tv_sec; - ns = (uint64_t) time.tv_nsec; + sec = (uint64_t)time.tv_sec; + ns = (uint64_t)time.tv_nsec; cycles = sec * hz; cycles += (ns * hz) / GIGA; diff --git a/platform/linux-generic/odp_barrier.c b/platform/linux-generic/odp_barrier.c index 53d83c0..ea2673e 100644 --- a/platform/linux-generic/odp_barrier.c +++ b/platform/linux-generic/odp_barrier.c @@ -37,7 +37,7 @@ void odp_barrier_wait(odp_barrier_t *barrier) count = odp_atomic_fetch_inc_u32(&barrier->bar); wasless = count < barrier->count; - if (count == 2*barrier->count-1) { + if (count == 2 * barrier->count - 1) { /* Wrap around *atomically* */ odp_atomic_sub_u32(&barrier->bar, 2 * barrier->count); } else { diff --git a/platform/linux-generic/odp_buffer.c b/platform/linux-generic/odp_buffer.c index d5d42a6..fb887da 100644 --- a/platform/linux-generic/odp_buffer.c +++ b/platform/linux-generic/odp_buffer.c @@ -13,7 +13,6 @@ #include #include - odp_buffer_t odp_buffer_from_event(odp_event_t ev) { return (odp_buffer_t)ev; @@ -31,7 +30,6 @@ void *odp_buffer_addr(odp_buffer_t buf) return hdr->addr[0]; } - uint32_t odp_buffer_size(odp_buffer_t buf) { odp_buffer_hdr_t *hdr = odp_buf_to_hdr(buf); @@ -39,7 +37,6 @@ uint32_t odp_buffer_size(odp_buffer_t buf) return hdr->size; } - int _odp_buffer_type(odp_buffer_t buf) { odp_buffer_hdr_t *hdr = odp_buf_to_hdr(buf); @@ -52,7 +49,6 @@ int odp_buffer_is_valid(odp_buffer_t buf) return validate_buf(buf) != NULL; } - int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf) { odp_buffer_hdr_t *hdr; @@ -65,32 +61,26 @@ int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf) hdr = odp_buf_to_hdr(buf); - len += snprintf(&str[len], n-len, - "Buffer\n"); - len += snprintf(&str[len], n-len, - " pool %" PRIu64 "\n", + len += snprintf(&str[len], n - len, "Buffer\n"); + len += snprintf(&str[len], n - len, " pool %" PRIu64 "\n", odp_pool_to_u64(hdr->pool_hdl)); - len += snprintf(&str[len], n-len, - " addr %p\n", hdr->addr); - len += snprintf(&str[len], n-len, - " size %" PRIu32 "\n", hdr->size); - len += snprintf(&str[len], n-len, - " ref_count %" PRIu32 "\n", + len += snprintf(&str[len], n - len, " addr %p\n", hdr->addr); + len += snprintf(&str[len], n - len, " size %" PRIu32 "\n", + hdr->size); + len += snprintf(&str[len], n - len, " ref_count %" PRIu32 "\n", odp_atomic_load_u32(&hdr->ref_count)); - len += snprintf(&str[len], n-len, - " type %i\n", hdr->type); + len += snprintf(&str[len], n - len, " type %i\n", hdr->type); return len; } - void odp_buffer_print(odp_buffer_t buf) { int max_len = 512; char str[max_len]; int len; - len = odp_buffer_snprint(str, max_len-1, buf); + len = odp_buffer_snprint(str, max_len - 1, buf); str[len] = 0; ODP_PRINT("\n%s\n", str); diff --git a/platform/linux-generic/odp_classification.c b/platform/linux-generic/odp_classification.c index fdb544d..452631a 100644 --- a/platform/linux-generic/odp_classification.c +++ b/platform/linux-generic/odp_classification.c @@ -53,8 +53,8 @@ int odp_classification_init_global(void) int i; cos_shm = odp_shm_reserve("shm_odp_cos_tbl", - sizeof(cos_tbl_t), - sizeof(cos_t), 0); + sizeof(cos_tbl_t), + sizeof(cos_t), 0); if (cos_shm == ODP_SHM_INVALID) { ODP_ERR("shm allocation failed for shm_odp_cos_tbl"); @@ -74,8 +74,8 @@ int odp_classification_init_global(void) } pmr_shm = odp_shm_reserve("shm_odp_pmr_tbl", - sizeof(pmr_tbl_t), - sizeof(pmr_t), 0); + sizeof(pmr_tbl_t), + sizeof(pmr_t), 0); if (pmr_shm == ODP_SHM_INVALID) { ODP_ERR("shm allocation failed for shm_odp_pmr_tbl"); @@ -90,13 +90,13 @@ int odp_classification_init_global(void) for (i = 0; i < ODP_PMR_MAX_ENTRY; i++) { /* init locks */ pmr_t *pmr = - get_pmr_entry_internal(_odp_cast_scalar(odp_pmr_t, i)); + get_pmr_entry_internal(_odp_cast_scalar(odp_pmr_t, i)); LOCK_INIT(&pmr->s.lock); } - pmr_set_shm = odp_shm_reserve("shm_odp_pmr_set_tbl", - sizeof(pmr_set_tbl_t), - sizeof(pmr_set_t), 0); + pmr_set_shm = + odp_shm_reserve("shm_odp_pmr_set_tbl", sizeof(pmr_set_tbl_t), + sizeof(pmr_set_t), 0); if (pmr_set_shm == ODP_SHM_INVALID) { ODP_ERR("shm allocation failed for shm_odp_pmr_set_tbl"); @@ -110,9 +110,8 @@ int odp_classification_init_global(void) memset(pmr_set_tbl, 0, sizeof(pmr_set_tbl_t)); for (i = 0; i < ODP_PMRSET_MAX_ENTRY; i++) { /* init locks */ - pmr_set_t *pmr = - get_pmr_set_entry_internal - (_odp_cast_scalar(odp_pmr_set_t, i)); + pmr_set_t *pmr = get_pmr_set_entry_internal( + _odp_cast_scalar(odp_pmr_set_t, i)); LOCK_INIT(&pmr->s.pmr.s.lock); } @@ -190,8 +189,8 @@ odp_pmr_set_t alloc_pmr_set(pmr_t **pmr) pmr_set_tbl->pmr_set[i].s.pmr.s.valid = 1; pmr_set_tbl->pmr_set[i].s.pmr.s.num_pmr = 0; *pmr = (pmr_t *)&pmr_set_tbl->pmr_set[i]; - odp_atomic_init_u32(&pmr_set_tbl->pmr_set[i] - .s.pmr.s.count, 0); + odp_atomic_init_u32( + &pmr_set_tbl->pmr_set[i].s.pmr.s.count, 0); /* return as locked */ return _odp_cast_scalar(odp_pmr_set_t, i); } @@ -221,7 +220,6 @@ odp_pmr_t alloc_pmr(pmr_t **pmr) return ODP_PMR_INVAL; } - cos_t *get_cos_entry(odp_cos_t cos_id) { if (_odp_typeval(cos_id) >= ODP_COS_MAX_ENTRY || @@ -232,7 +230,6 @@ cos_t *get_cos_entry(odp_cos_t cos_id) return &(cos_tbl->cos_entry[_odp_typeval(cos_id)]); } - pmr_set_t *get_pmr_set_entry(odp_pmr_set_t pmr_set_id) { if (_odp_typeval(pmr_set_id) >= ODP_PMRSET_MAX_ENTRY || @@ -256,6 +253,7 @@ pmr_t *get_pmr_entry(odp_pmr_t pmr_id) int odp_cos_destroy(odp_cos_t cos_id) { cos_t *cos = get_cos_entry(cos_id); + if (NULL == cos) { ODP_ERR("Invalid odp_cos_t handle"); return -1; @@ -268,6 +266,7 @@ int odp_cos_destroy(odp_cos_t cos_id) int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) { cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) { ODP_ERR("Invalid odp_cos_t handle"); return -1; @@ -281,6 +280,7 @@ int odp_cos_set_queue(odp_cos_t cos_id, odp_queue_t queue_id) int odp_cos_set_drop(odp_cos_t cos_id, odp_drop_e drop_policy) { cos_t *cos = get_cos_entry(cos_id); + if (cos == NULL) { ODP_ERR("Invalid odp_cos_t handle"); return -1; @@ -295,6 +295,7 @@ int odp_pktio_default_cos_set(odp_pktio_t pktio_in, odp_cos_t default_cos) { pktio_entry_t *entry; cos_t *cos; + entry = get_pktio_entry(pktio_in); if (entry == NULL) { ODP_ERR("Invalid odp_pktio_t handle"); @@ -334,6 +335,7 @@ int odp_pktio_error_cos_set(odp_pktio_t pktio_in, odp_cos_t error_cos) int odp_pktio_skip_set(odp_pktio_t pktio_in, uint32_t offset) { pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) { ODP_ERR("Invalid odp_cos_t handle"); return -1; @@ -346,6 +348,7 @@ int odp_pktio_skip_set(odp_pktio_t pktio_in, uint32_t offset) int odp_pktio_headroom_set(odp_pktio_t pktio_in, uint32_t headroom) { pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) { ODP_ERR("Invalid odp_pktio_t handle"); return -1; @@ -363,6 +366,7 @@ int odp_cos_with_l2_priority(odp_pktio_t pktio_in, uint32_t i; cos_t *cos; pktio_entry_t *entry = get_pktio_entry(pktio_in); + if (entry == NULL) { ODP_ERR("Invalid odp_pktio_t handle"); return -1; @@ -419,6 +423,7 @@ odp_pmr_t odp_pmr_create(odp_pmr_term_e term, const void *val, { pmr_t *pmr; odp_pmr_t id; + if (val_sz > ODP_PMR_TERM_BYTES_MAX) { ODP_ERR("val_sz greater than max supported limit"); return ODP_PMR_INVAL; @@ -431,8 +436,8 @@ odp_pmr_t odp_pmr_create(odp_pmr_term_e term, const void *val, pmr->s.num_pmr = 1; pmr->s.pmr_term_value[0].term = term; - pmr->s.pmr_term_value[0].val = 0; - pmr->s.pmr_term_value[0].mask = 0; + pmr->s.pmr_term_value[0].val = 0; + pmr->s.pmr_term_value[0].mask = 0; memcpy(&pmr->s.pmr_term_value[0].val, val, val_sz); memcpy(&pmr->s.pmr_term_value[0].mask, mask, val_sz); pmr->s.pmr_term_value[0].val &= pmr->s.pmr_term_value[0].mask; @@ -498,6 +503,7 @@ int odp_cos_pmr_cos(odp_pmr_t pmr_id, odp_cos_t src_cos, odp_cos_t dst_cos) cos_t *cos_src = get_cos_entry(src_cos); cos_t *cos_dst = get_cos_entry(dst_cos); pmr_t *pmr = get_pmr_entry(pmr_id); + if (NULL == cos_src || NULL == cos_dst || NULL == pmr) { ODP_ERR("Invalid input handle"); return -1; @@ -565,12 +571,9 @@ int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, pmr->s.pmr_term_value[i].term = terms[i].term; pmr->s.pmr_term_value[i].val = 0; pmr->s.pmr_term_value[i].mask = 0; - memcpy(&pmr->s.pmr_term_value[i].val, - terms[i].val, val_sz); - memcpy(&pmr->s.pmr_term_value[i].mask, - terms[i].mask, val_sz); - pmr->s.pmr_term_value[i].val &= pmr->s - .pmr_term_value[i].mask; + memcpy(&pmr->s.pmr_term_value[i].val, terms[i].val, val_sz); + memcpy(&pmr->s.pmr_term_value[i].mask, terms[i].mask, val_sz); + pmr->s.pmr_term_value[i].val &= pmr->s.pmr_term_value[i].mask; count++; } *pmr_set_id = id; @@ -581,6 +584,7 @@ int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms, int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) { pmr_set_t *pmr_set = get_pmr_set_entry(pmr_set_id); + if (pmr_set == NULL) return -1; @@ -589,7 +593,7 @@ int odp_pmr_match_set_destroy(odp_pmr_set_t pmr_set_id) } int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t src_pktio, - odp_cos_t dst_cos) + odp_cos_t dst_cos) { uint8_t num_pmr; pktio_entry_t *pktio_entry; @@ -673,8 +677,7 @@ int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) pmr_failure = 1; break; case ODP_PMR_DMAC: - if (!verify_pmr_dmac(pkt_addr, pkt_hdr, - term_value)) + if (!verify_pmr_dmac(pkt_addr, pkt_hdr, term_value)) pmr_failure = 1; break; case ODP_PMR_IPPROTO: @@ -728,8 +731,7 @@ int verify_pmr(pmr_t *pmr, uint8_t *pkt_addr, odp_packet_hdr_t *pkt_hdr) pmr_failure = 1; break; case ODP_PMR_LD_VNI: - if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, - term_value)) + if (!verify_pmr_ld_vni(pkt_addr, pkt_hdr, term_value)) pmr_failure = 1; break; case ODP_PMR_INNER_HDR_OFF: @@ -769,6 +771,7 @@ int pktio_classifier_init(pktio_entry_t *entry) { classifier_t *cls; int i; + /* classifier lock should be acquired by the calling function */ if (entry == NULL) return -1; @@ -814,7 +817,7 @@ int packet_classifier(odp_pktio_t pktio, odp_packet_t pkt) } cos_t *pktio_select_cos(pktio_entry_t *entry, uint8_t *pkt_addr, - odp_packet_hdr_t *pkt_hdr) + odp_packet_hdr_t *pkt_hdr) { pmr_t *pmr; cos_t *cos; @@ -898,7 +901,7 @@ cos_t *match_qos_cos(pktio_entry_t *entry, uint8_t *pkt_addr, l3_cos = &cls->l3_cos_table; if (cls->l3_precedence) { - cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); + cos = match_qos_l3_cos(l3_cos, pkt_addr, hdr); if (cos) return cos; cos = match_qos_l2_cos(l2_cos, pkt_addr, hdr); diff --git a/platform/linux-generic/odp_cpumask.c b/platform/linux-generic/odp_cpumask.c index b31e1ca..77d4236 100644 --- a/platform/linux-generic/odp_cpumask.c +++ b/platform/linux-generic/odp_cpumask.c @@ -167,8 +167,7 @@ void odp_cpumask_xor(odp_cpumask_t *dest, const odp_cpumask_t *src1, CPU_XOR(&dest->set, &src1->set, &src2->set); } -int odp_cpumask_equal(const odp_cpumask_t *mask1, - const odp_cpumask_t *mask2) +int odp_cpumask_equal(const odp_cpumask_t *mask1, const odp_cpumask_t *mask2) { return CPU_EQUAL(&mask1->set, &mask2->set); } diff --git a/platform/linux-generic/odp_crypto.c b/platform/linux-generic/odp_crypto.c index d49e256..9384291 100644 --- a/platform/linux-generic/odp_crypto.c +++ b/platform/linux-generic/odp_crypto.c @@ -30,21 +30,19 @@ typedef struct odp_crypto_global_s odp_crypto_global_t; struct odp_crypto_global_s { - odp_spinlock_t lock; + odp_spinlock_t lock; odp_crypto_generic_session_t *free; - odp_crypto_generic_session_t sessions[0]; + odp_crypto_generic_session_t sessions[0]; }; static odp_crypto_global_t *global; -static -odp_crypto_generic_op_result_t *get_op_result_from_event(odp_event_t ev) +static odp_crypto_generic_op_result_t *get_op_result_from_event(odp_event_t ev) { return &(odp_packet_hdr(odp_packet_from_event(ev))->op_result); } -static -odp_crypto_generic_session_t *alloc_session(void) +static odp_crypto_generic_session_t *alloc_session(void) { odp_crypto_generic_session_t *session = NULL; @@ -57,8 +55,7 @@ odp_crypto_generic_session_t *alloc_session(void) return session; } -static -void free_session(odp_crypto_generic_session_t *session) +static void free_session(odp_crypto_generic_session_t *session) { odp_spinlock_lock(&global->lock); session->next = global->free; @@ -85,16 +82,10 @@ enum crypto_alg_err md5_gen(odp_crypto_op_params_t *params, /* Adjust pointer for beginning of area to auth */ data += params->auth_range.offset; - icv += params->hash_result_offset; + icv += params->hash_result_offset; /* Hash it */ - HMAC(EVP_md5(), - session->auth.data.md5.key, - 16, - data, - len, - hash, - NULL); + HMAC(EVP_md5(), session->auth.data.md5.key, 16, data, len, hash, NULL); /* Copy to the output location */ memcpy(icv, hash, session->auth.data.md5.bytes); @@ -110,12 +101,12 @@ enum crypto_alg_err md5_check(odp_crypto_op_params_t *params, uint8_t *icv = data; uint32_t len = params->auth_range.length; uint32_t bytes = session->auth.data.md5.bytes; - uint8_t hash_in[EVP_MAX_MD_SIZE]; - uint8_t hash_out[EVP_MAX_MD_SIZE]; + uint8_t hash_in[EVP_MAX_MD_SIZE]; + uint8_t hash_out[EVP_MAX_MD_SIZE]; /* Adjust pointer for beginning of area to auth */ data += params->auth_range.offset; - icv += params->hash_result_offset; + icv += params->hash_result_offset; /* Copy current value out and clear it before authentication */ memset(hash_in, 0, sizeof(hash_in)); @@ -124,12 +115,7 @@ enum crypto_alg_err md5_check(odp_crypto_op_params_t *params, memset(hash_out, 0, sizeof(hash_out)); /* Hash it */ - HMAC(EVP_md5(), - session->auth.data.md5.key, - 16, - data, - len, - hash_out, + HMAC(EVP_md5(), session->auth.data.md5.key, 16, data, len, hash_out, NULL); /* Verify match */ @@ -282,17 +268,17 @@ odp_crypto_session_create(odp_crypto_session_params_t *params, /* Derive order */ if (ODP_CRYPTO_OP_ENCODE == params->op) - session->do_cipher_first = params->auth_cipher_text; + session->do_cipher_first = params->auth_cipher_text; else session->do_cipher_first = !params->auth_cipher_text; /* Copy stuff over */ session->op = params->op; session->compl_queue = params->compl_queue; - session->cipher.alg = params->cipher_alg; + session->cipher.alg = params->cipher_alg; session->cipher.iv.data = params->iv.data; - session->cipher.iv.len = params->iv.length; - session->auth.alg = params->auth_alg; + session->cipher.iv.len = params->iv.length; + session->auth.alg = params->auth_alg; session->output_pool = params->output_pool; /* Process based on cipher */ @@ -395,9 +381,8 @@ odp_crypto_operation(odp_crypto_op_params_t *params, local_result.cipher_status.hw_err = ODP_CRYPTO_HW_ERR_NONE; local_result.auth_status.alg_err = rc_auth; local_result.auth_status.hw_err = ODP_CRYPTO_HW_ERR_NONE; - local_result.ok = - (rc_cipher == ODP_CRYPTO_ALG_ERR_NONE) && - (rc_auth == ODP_CRYPTO_ALG_ERR_NONE); + local_result.ok = (rc_cipher == ODP_CRYPTO_ALG_ERR_NONE) && + (rc_auth == ODP_CRYPTO_ALG_ERR_NONE); /* If specified during creation post event to completion queue */ if (ODP_QUEUE_INVALID != session->compl_queue) { @@ -432,20 +417,18 @@ odp_crypto_operation(odp_crypto_op_params_t *params, return 0; } -int -odp_crypto_init_global(void) +int odp_crypto_init_global(void) { size_t mem_size; odp_shm_t shm; int idx; /* Calculate the memory size we need */ - mem_size = sizeof(*global); + mem_size = sizeof(*global); mem_size += (MAX_SESSIONS * sizeof(odp_crypto_generic_session_t)); /* Allocate our globally shared memory */ - shm = odp_shm_reserve("crypto_pool", mem_size, - ODP_CACHE_LINE_SIZE, 0); + shm = odp_shm_reserve("crypto_pool", mem_size, ODP_CACHE_LINE_SIZE, 0); global = odp_shm_addr(shm); @@ -485,12 +468,13 @@ int odp_crypto_term_global(void) return rc; } -int32_t -odp_random_data(uint8_t *buf, int32_t len, odp_bool_t use_entropy ODP_UNUSED) +int32_t odp_random_data(uint8_t *buf, int32_t len, + odp_bool_t use_entropy ODP_UNUSED) { int32_t rc; + rc = RAND_bytes(buf, len); - return (1 == rc) ? len /*success*/: -1 /*failure*/; + return (1 == rc) ? len /*success*/ : -1 /*failure*/; } odp_crypto_compl_t odp_crypto_compl_from_event(odp_event_t ev) diff --git a/platform/linux-generic/odp_impl.c b/platform/linux-generic/odp_impl.c index 46d0e40..32ff8b4 100644 --- a/platform/linux-generic/odp_impl.c +++ b/platform/linux-generic/odp_impl.c @@ -4,7 +4,6 @@ * SPDX-License-Identifier: BSD-3-Clause */ - /** * @file * diff --git a/platform/linux-generic/odp_init.c b/platform/linux-generic/odp_init.c index 9138d37..2bfabb6 100644 --- a/platform/linux-generic/odp_init.c +++ b/platform/linux-generic/odp_init.c @@ -12,7 +12,7 @@ struct odp_global_data_s odp_global_data; int odp_init_global(odp_init_t *params, - odp_platform_init_t *platform_params ODP_UNUSED) + odp_platform_init_t *platform_params ODP_UNUSED) { odp_global_data.log_fn = odp_override_log; odp_global_data.abort_fn = odp_override_abort; @@ -24,7 +24,6 @@ int odp_init_global(odp_init_t *params, odp_global_data.abort_fn = params->abort_fn; } - odp_system_info_init(); if (odp_shm_init_global()) { diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 5581cc4..bdd84a8 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -34,9 +34,8 @@ odp_packet_t odp_packet_alloc(odp_pool_t pool_hdl, uint32_t len) /* Handle special case for zero-length packets */ if (len == 0) { - odp_packet_t pkt = - (odp_packet_t)buffer_alloc(pool_hdl, - pool->s.params.buf.size); + odp_packet_t pkt = (odp_packet_t)buffer_alloc( + pool_hdl, pool->s.params.buf.size); if (pkt != ODP_PACKET_INVALID) pull_tail(odp_packet_hdr(pkt), pool->s.params.buf.size); @@ -95,6 +94,7 @@ odp_event_t odp_packet_to_event(odp_packet_t pkt) void *odp_packet_head(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + return buffer_map(&pkt_hdr->buf_hdr, 0, NULL, 0); } @@ -106,6 +106,7 @@ uint32_t odp_packet_buf_len(odp_packet_t pkt) void *odp_packet_data(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + return packet_map(pkt_hdr, 0, NULL); } @@ -137,6 +138,7 @@ uint32_t odp_packet_tailroom(odp_packet_t pkt) void *odp_packet_tail(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + return packet_map(pkt_hdr, pkt_hdr->frame_len, NULL); } @@ -193,9 +195,10 @@ void *odp_packet_offset(odp_packet_t pkt, uint32_t offset, uint32_t *len, if (addr != NULL && seg != NULL) { odp_buffer_bits_t seghandle; + seghandle.handle = (odp_buffer_t)pkt; - seghandle.seg = (pkt_hdr->headroom + offset) / - pkt_hdr->buf_hdr.segsize; + seghandle.seg = + (pkt_hdr->headroom + offset) / pkt_hdr->buf_hdr.segsize; *seg = (odp_packet_seg_t)seghandle.handle; } @@ -242,6 +245,7 @@ uint32_t odp_packet_user_area_size(odp_packet_t pkt) void *odp_packet_l2_ptr(odp_packet_t pkt, uint32_t *len) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return packet_map(pkt_hdr, pkt_hdr->l2_offset, len); @@ -250,6 +254,7 @@ void *odp_packet_l2_ptr(odp_packet_t pkt, uint32_t *len) uint32_t odp_packet_l2_offset(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return pkt_hdr->l2_offset; @@ -271,6 +276,7 @@ int odp_packet_l2_offset_set(odp_packet_t pkt, uint32_t offset) void *odp_packet_l3_ptr(odp_packet_t pkt, uint32_t *len) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return packet_map(pkt_hdr, pkt_hdr->l3_offset, len); @@ -279,6 +285,7 @@ void *odp_packet_l3_ptr(odp_packet_t pkt, uint32_t *len) uint32_t odp_packet_l3_offset(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return pkt_hdr->l3_offset; @@ -300,6 +307,7 @@ int odp_packet_l3_offset_set(odp_packet_t pkt, uint32_t offset) void *odp_packet_l4_ptr(odp_packet_t pkt, uint32_t *len) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return packet_map(pkt_hdr, pkt_hdr->l4_offset, len); @@ -308,6 +316,7 @@ void *odp_packet_l4_ptr(odp_packet_t pkt, uint32_t *len) uint32_t odp_packet_l4_offset(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return pkt_hdr->l4_offset; @@ -428,6 +437,7 @@ odp_packet_t odp_packet_add_data(odp_packet_t pkt, uint32_t offset, newpkt = ODP_PACKET_INVALID; } else { odp_packet_hdr_t *new_hdr = odp_packet_hdr(newpkt); + new_hdr->input = pkt_hdr->input; new_hdr->buf_hdr.buf_u64 = pkt_hdr->buf_hdr.buf_u64; if (new_hdr->buf_hdr.uarea_addr != NULL && @@ -472,6 +482,7 @@ odp_packet_t odp_packet_rem_data(odp_packet_t pkt, uint32_t offset, newpkt = ODP_PACKET_INVALID; } else { odp_packet_hdr_t *new_hdr = odp_packet_hdr(newpkt); + new_hdr->input = pkt_hdr->input; new_hdr->buf_hdr.buf_u64 = pkt_hdr->buf_hdr.buf_u64; if (new_hdr->buf_hdr.uarea_addr != NULL && @@ -569,9 +580,9 @@ int odp_packet_copydata_in(odp_packet_t pkt, uint32_t offset, mapaddr = packet_map(pkt_hdr, offset, &seglen); cpylen = len > seglen ? seglen : len; memcpy(mapaddr, srcaddr, cpylen); - offset += cpylen; + offset += cpylen; srcaddr += cpylen; - len -= cpylen; + len -= cpylen; } return 0; @@ -589,27 +600,26 @@ void odp_packet_print(odp_packet_t pkt) int max_len = 512; char str[max_len]; int len = 0; - int n = max_len-1; + int n = max_len - 1; odp_packet_hdr_t *hdr = odp_packet_hdr(pkt); - len += snprintf(&str[len], n-len, "Packet "); - len += odp_buffer_snprint(&str[len], n-len, (odp_buffer_t) pkt); - len += snprintf(&str[len], n-len, - " input_flags 0x%" PRIx32 "\n", hdr->input_flags.all); - len += snprintf(&str[len], n-len, - " error_flags 0x%" PRIx32 "\n", hdr->error_flags.all); - len += snprintf(&str[len], n-len, - " output_flags 0x%" PRIx32 "\n", hdr->output_flags.all); - len += snprintf(&str[len], n-len, - " l2_offset %" PRIu32 "\n", hdr->l2_offset); - len += snprintf(&str[len], n-len, - " l3_offset %" PRIu32 "\n", hdr->l3_offset); - len += snprintf(&str[len], n-len, - " l4_offset %" PRIu32 "\n", hdr->l4_offset); - len += snprintf(&str[len], n-len, - " frame_len %" PRIu32 "\n", hdr->frame_len); - len += snprintf(&str[len], n-len, - " input %" PRIu64 "\n", + len += snprintf(&str[len], n - len, "Packet "); + len += odp_buffer_snprint(&str[len], n - len, (odp_buffer_t)pkt); + len += snprintf(&str[len], n - len, " input_flags 0x%" PRIx32 "\n", + hdr->input_flags.all); + len += snprintf(&str[len], n - len, " error_flags 0x%" PRIx32 "\n", + hdr->error_flags.all); + len += snprintf(&str[len], n - len, " output_flags 0x%" PRIx32 "\n", + hdr->output_flags.all); + len += snprintf(&str[len], n - len, " l2_offset %" PRIu32 "\n", + hdr->l2_offset); + len += snprintf(&str[len], n - len, " l3_offset %" PRIu32 "\n", + hdr->l3_offset); + len += snprintf(&str[len], n - len, " l4_offset %" PRIu32 "\n", + hdr->l4_offset); + len += snprintf(&str[len], n - len, " frame_len %" PRIu32 "\n", + hdr->frame_len); + len += snprintf(&str[len], n - len, " input %" PRIu64 "\n", odp_pktio_to_u64(hdr->input)); str[len] = '\0'; @@ -755,14 +765,13 @@ static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, *offset += sizeof(odph_ipv6hdr_t); *parseptr += sizeof(odph_ipv6hdr_t); - /* Skip past any IPv6 extension headers */ if (ipv6->next_hdr == ODPH_IPPROTO_HOPOPTS || ipv6->next_hdr == ODPH_IPPROTO_ROUTE) { pkt_hdr->input_flags.ipopt = 1; - do { - ipv6ext = (odph_ipv6hdr_ext_t *)*parseptr; + do { + ipv6ext = (odph_ipv6hdr_ext_t *)*parseptr; uint16_t extlen = 8 + ipv6ext->ext_len * 8; *offset += extlen; @@ -771,7 +780,8 @@ static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, ipv6ext->next_hdr == ODPH_IPPROTO_ROUTE) && *offset < pkt_hdr->frame_len); - if (*offset >= pkt_hdr->l3_offset + odp_be_to_cpu_16(ipv6->payload_len)) { + if (*offset >= + pkt_hdr->l3_offset + odp_be_to_cpu_16(ipv6->payload_len)) { pkt_hdr->error_flags.ip_err = 1; return 0; } @@ -798,7 +808,7 @@ static inline void parse_tcp(odp_packet_hdr_t *pkt_hdr, { odph_tcphdr_t *tcp = (odph_tcphdr_t *)*parseptr; - if (tcp->hl < sizeof(odph_tcphdr_t)/sizeof(uint32_t)) + if (tcp->hl < sizeof(odph_tcphdr_t) / sizeof(uint32_t)) pkt_hdr->error_flags.tcp_err = 1; else if ((uint32_t)tcp->hl * 4 > sizeof(odph_tcphdr_t)) pkt_hdr->input_flags.tcpopt = 1; @@ -928,13 +938,13 @@ int _odp_packet_parse(odp_packet_hdr_t *pkt_hdr) case ODPH_ETHTYPE_ARP: pkt_hdr->input_flags.arp = 1; - ip_proto = 255; /* Reserved invalid by IANA */ + ip_proto = 255; /* Reserved invalid by IANA */ break; default: pkt_hdr->input_flags.l3 = 0; pkt_hdr->l3_offset = ODP_PACKET_OFFSET_INVALID; - ip_proto = 255; /* Reserved invalid by IANA */ + ip_proto = 255; /* Reserved invalid by IANA */ } /* Set l4_offset+flag only for known ip_proto */ @@ -969,12 +979,12 @@ int _odp_packet_parse(odp_packet_hdr_t *pkt_hdr) break; } - /* - * Anything beyond what we parse here is considered payload. - * Note: Payload is really only relevant for TCP and UDP. For - * all other protocols, the payload offset will point to the - * final header (ARP, ICMP, AH, ESP, or IP Fragment). - */ + /* + * Anything beyond what we parse here is considered payload. + * Note: Payload is really only relevant for TCP and UDP. For + * all other protocols, the payload offset will point to the + * final header (ARP, ICMP, AH, ESP, or IP Fragment). + */ pkt_hdr->payload_offset = offset; parse_exit: diff --git a/platform/linux-generic/odp_packet_flags.c b/platform/linux-generic/odp_packet_flags.c index 4f680a1..a0bb868 100644 --- a/platform/linux-generic/odp_packet_flags.c +++ b/platform/linux-generic/odp_packet_flags.c @@ -24,6 +24,7 @@ int odp_packet_has_error(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + if (pkt_hdr->input_flags.unparsed) _odp_packet_parse(pkt_hdr); return odp_packet_hdr(pkt)->error_flags.all != 0; diff --git a/platform/linux-generic/odp_packet_io.c b/platform/linux-generic/odp_packet_io.c index f3a4543..2c7240e 100644 --- a/platform/linux-generic/odp_packet_io.c +++ b/platform/linux-generic/odp_packet_io.c @@ -62,7 +62,7 @@ int odp_pktio_init_global(void) pktio_entry_ptr[id - 1] = pktio_entry; /* Create a default output queue for each pktio resource */ snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); - name[ODP_QUEUE_NAME_LEN-1] = '\0'; + name[ODP_QUEUE_NAME_LEN - 1] = '\0'; qid = odp_queue_create(name, ODP_QUEUE_TYPE_PKTOUT, NULL); if (qid == ODP_QUEUE_INVALID) @@ -315,8 +315,6 @@ odp_pktio_t odp_pktio_lookup(const char *dev) return id; } - - int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], int len) { pktio_entry_t *pktio_entry = get_pktio_entry(id); @@ -511,7 +509,7 @@ odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) return NULL; if (j > 1) - queue_enq_multi(qentry, &tmp_hdr_tbl[1], j-1); + queue_enq_multi(qentry, &tmp_hdr_tbl[1], j - 1); buf_hdr = tmp_hdr_tbl[0]; return buf_hdr; } @@ -536,7 +534,7 @@ int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) nbr = queue_deq_multi(qentry, buf_hdr, num); if (odp_unlikely(nbr > num)) ODP_ABORT("queue_deq_multi req: %d, returned %d\n", - num, nbr); + num, nbr); /** queue already has number of requsted buffers, * do not do receive in that case. @@ -596,6 +594,7 @@ int pktin_poll(pktio_entry_t *entry) if (num_enq) { queue_entry_t *qentry; + qentry = queue_to_qentry(entry->s.inq_default); queue_enq_multi(qentry, hdr_tbl, num_enq); } @@ -677,7 +676,6 @@ int odp_pktio_promisc_mode(odp_pktio_t id) return ret; } - int odp_pktio_mac_addr(odp_pktio_t id, void *mac_addr, int addr_size) { pktio_entry_t *entry; diff --git a/platform/linux-generic/odp_pool.c b/platform/linux-generic/odp_pool.c index dfdd2f5..79100a1 100644 --- a/platform/linux-generic/odp_pool.c +++ b/platform/linux-generic/odp_pool.c @@ -23,33 +23,29 @@ #include #include - #if ODP_CONFIG_POOLS > ODP_BUFFER_MAX_POOLS #error ODP_CONFIG_POOLS > ODP_BUFFER_MAX_POOLS #endif - typedef union buffer_type_any_u { - odp_buffer_hdr_t buf; - odp_packet_hdr_t pkt; + odp_buffer_hdr_t buf; + odp_packet_hdr_t pkt; odp_timeout_hdr_t tmo; } odp_anybuf_t; /* Any buffer type header */ typedef struct { - union buffer_type_any_u any_hdr; /* any buffer type */ + union buffer_type_any_u any_hdr; /* any buffer type */ } odp_any_buffer_hdr_t; typedef struct odp_any_hdr_stride { uint8_t pad[ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_any_buffer_hdr_t))]; } odp_any_hdr_stride; - typedef struct pool_table_t { pool_entry_t pool[ODP_CONFIG_POOLS]; } pool_table_t; - /* The pool table */ static pool_table_t *pool_tbl; static const char SHM_DEFAULT_NAME[] = "odp_buffer_pools"; @@ -65,8 +61,7 @@ int odp_pool_init_global(void) uint32_t i; odp_shm_t shm; - shm = odp_shm_reserve(SHM_DEFAULT_NAME, - sizeof(pool_table_t), + shm = odp_shm_reserve(SHM_DEFAULT_NAME, sizeof(pool_table_t), sizeof(pool_entry_t), 0); pool_tbl = odp_shm_addr(shm); @@ -79,6 +74,7 @@ int odp_pool_init_global(void) for (i = 0; i < ODP_CONFIG_POOLS; i++) { /* init locks */ pool_entry_t *pool = &pool_tbl->pool[i]; + POOL_LOCK_INIT(&pool->s.lock); POOL_LOCK_INIT(&pool->s.buf_lock); POOL_LOCK_INIT(&pool->s.blk_lock); @@ -260,8 +256,7 @@ odp_pool_t odp_pool_create(const char *name, odp_pool_param_t *params) if (name == NULL) { pool->s.name[0] = 0; } else { - strncpy(pool->s.name, name, - ODP_POOL_NAME_LEN - 1); + strncpy(pool->s.name, name, ODP_POOL_NAME_LEN - 1); pool->s.name[ODP_POOL_NAME_LEN - 1] = 0; pool->s.flags.has_name = 1; } @@ -403,7 +398,6 @@ odp_pool_t odp_pool_create(const char *name, odp_pool_param_t *params) return pool_hdl; } - odp_pool_t odp_pool_lookup(const char *name) { uint32_t i; @@ -449,8 +443,7 @@ int odp_pool_destroy(odp_pool_t pool_hdl) POOL_LOCK(&pool->s.lock); /* Call fails if pool is not allocated or predefined*/ - if (pool->s.pool_shm == ODP_SHM_INVALID || - pool->s.flags.predefined) { + if (pool->s.pool_shm == ODP_SHM_INVALID || pool->s.flags.predefined) { POOL_UNLOCK(&pool->s.lock); return -1; } @@ -498,8 +491,10 @@ odp_buffer_t buffer_alloc(odp_pool_t pool_hdl, size_t size) /* Get blocks for this buffer, if pool uses application data */ if (buf->buf.size < totsize) { intmax_t needed = totsize - buf->buf.size; + do { uint8_t *blk = get_blk(&pool->s); + if (blk == NULL) { ret_buf(&pool->s, &buf->buf); return ODP_BUFFER_INVALID; @@ -543,6 +538,7 @@ void _odp_flush_caches(void) for (i = 0; i < ODP_CONFIG_POOLS; i++) { pool_entry_t *pool = get_pool_entry(i); + flush_cache(&local_cache[i], &pool->s); } } @@ -604,16 +600,16 @@ void odp_pool_print(odp_pool_t pool_hdl) ODP_DBG(" pkt length %u requested, %u used\n", pool->s.params.pkt.len, pool->s.blk_size); } - ODP_DBG(" num bufs %u\n", pool->s.buf_num); + ODP_DBG(" num bufs %u\n", pool->s.buf_num); ODP_DBG(" bufs available %u %s\n", bufcount, pool->s.low_wm_assert ? " **low wm asserted**" : ""); - ODP_DBG(" bufs in use %u\n", pool->s.buf_num - bufcount); + ODP_DBG(" bufs in use %u\n", pool->s.buf_num - bufcount); ODP_DBG(" buf allocs %lu\n", bufallocs); ODP_DBG(" buf frees %lu\n", buffrees); ODP_DBG(" buf empty %lu\n", bufempty); ODP_DBG(" blk size %zu\n", pool->s.seg_size > ODP_MAX_INLINE_BUF ? pool->s.seg_size : 0); - ODP_DBG(" blks available %u\n", blkcount); + ODP_DBG(" blks available %u\n", blkcount); ODP_DBG(" blk allocs %lu\n", blkallocs); ODP_DBG(" blk frees %lu\n", blkfrees); ODP_DBG(" blk empty %lu\n", blkempty); @@ -623,7 +619,6 @@ void odp_pool_print(odp_pool_t pool_hdl) ODP_DBG(" low wm count %lu\n", lowmct); } - odp_pool_t odp_buffer_pool(odp_buffer_t buf) { return odp_buf_to_hdr(buf)->pool_hdl; diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index d1c2cfc..2cbcac2 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -36,14 +36,12 @@ #include - typedef struct queue_table_t { - queue_entry_t queue[ODP_CONFIG_QUEUES]; + queue_entry_t queue[ODP_CONFIG_QUEUES]; } queue_table_t; static queue_table_t *queue_tbl; - queue_entry_t *get_qentry(uint32_t queue_id) { return &queue_tbl->queue[queue_id]; @@ -93,7 +91,6 @@ static void queue_init(queue_entry_t *queue, const char *name, queue->s.cmd_ev = ODP_EVENT_INVALID; } - int odp_queue_init_global(void) { uint32_t i; @@ -101,8 +98,7 @@ int odp_queue_init_global(void) ODP_DBG("Queue init ... "); - shm = odp_shm_reserve("odp_queues", - sizeof(queue_table_t), + shm = odp_shm_reserve("odp_queues", sizeof(queue_table_t), sizeof(queue_entry_t), 0); queue_tbl = odp_shm_addr(shm); @@ -115,6 +111,7 @@ int odp_queue_init_global(void) for (i = 0; i < ODP_CONFIG_QUEUES; i++) { /* init locks */ queue_entry_t *queue = get_qentry(i); + LOCK_INIT(&queue->s.lock); queue->s.handle = queue_from_id(i); } @@ -123,8 +120,7 @@ int odp_queue_init_global(void) ODP_DBG("Queue init global\n"); ODP_DBG(" struct queue_entry_s size %zu\n", sizeof(struct queue_entry_s)); - ODP_DBG(" queue_entry_t size %zu\n", - sizeof(queue_entry_t)); + ODP_DBG(" queue_entry_t size %zu\n", sizeof(queue_entry_t)); ODP_DBG("\n"); return 0; @@ -247,6 +243,7 @@ void queue_destroy_finalize(queue_entry_t *queue) int odp_queue_destroy(odp_queue_t handle) { queue_entry_t *queue; + queue = queue_to_qentry(handle); LOCK(&queue->s.lock); @@ -289,6 +286,7 @@ int odp_queue_destroy(odp_queue_t handle) int odp_queue_set_context(odp_queue_t handle, void *context) { queue_entry_t *queue; + queue = queue_to_qentry(handle); odp_sync_stores(); queue->s.param.context = context; @@ -299,6 +297,7 @@ int odp_queue_set_context(odp_queue_t handle, void *context) void *odp_queue_get_context(odp_queue_t handle) { queue_entry_t *queue; + queue = queue_to_qentry(handle); return queue->s.param.context; } @@ -325,7 +324,6 @@ odp_queue_t odp_queue_lookup(const char *name) return ODP_QUEUE_INVALID; } - int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) { int sched = 0; @@ -368,10 +366,10 @@ int queue_enq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) odp_buffer_hdr_t *tail; for (i = 0; i < num - 1; i++) - buf_hdr[i]->next = buf_hdr[i+1]; + buf_hdr[i]->next = buf_hdr[i + 1]; - tail = buf_hdr[num-1]; - buf_hdr[num-1]->next = NULL; + tail = buf_hdr[num - 1]; + buf_hdr[num - 1]->next = NULL; LOCK(&queue->s.lock); if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { @@ -418,19 +416,17 @@ int odp_queue_enq_multi(odp_queue_t handle, const odp_event_t ev[], int num) return queue->s.enqueue_multi(queue, buf_hdr, num); } - int odp_queue_enq(odp_queue_t handle, odp_event_t ev) { odp_buffer_hdr_t *buf_hdr; queue_entry_t *queue; - queue = queue_to_qentry(handle); + queue = queue_to_qentry(handle); buf_hdr = odp_buf_to_hdr(odp_buffer_from_event(ev)); return queue->s.enqueue(queue, buf_hdr); } - odp_buffer_hdr_t *queue_deq(queue_entry_t *queue) { odp_buffer_hdr_t *buf_hdr; @@ -460,7 +456,6 @@ odp_buffer_hdr_t *queue_deq(queue_entry_t *queue) return buf_hdr; } - int queue_deq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { odp_buffer_hdr_t *hdr; @@ -522,7 +517,6 @@ int odp_queue_deq_multi(odp_queue_t handle, odp_event_t events[], int num) return ret; } - odp_event_t odp_queue_deq(odp_queue_t handle) { queue_entry_t *queue; @@ -537,13 +531,11 @@ odp_event_t odp_queue_deq(odp_queue_t handle) return ODP_EVENT_INVALID; } - void queue_lock(queue_entry_t *queue) { LOCK(&queue->s.lock); } - void queue_unlock(queue_entry_t *queue) { UNLOCK(&queue->s.lock); diff --git a/platform/linux-generic/odp_rwlock.c b/platform/linux-generic/odp_rwlock.c index 47c15ef..1a2dae9 100644 --- a/platform/linux-generic/odp_rwlock.c +++ b/platform/linux-generic/odp_rwlock.c @@ -19,7 +19,7 @@ void odp_rwlock_init(odp_rwlock_t *rwlock) void odp_rwlock_read_lock(odp_rwlock_t *rwlock) { uint32_t cnt; - int is_locked = 0; + int is_locked = 0; while (is_locked == 0) { cnt = _odp_atomic_u32_load_mm(&rwlock->cnt, _ODP_MEMMODEL_RLX); @@ -28,11 +28,12 @@ void odp_rwlock_read_lock(odp_rwlock_t *rwlock) odp_spin(); continue; } - is_locked = _odp_atomic_u32_cmp_xchg_strong_mm(&rwlock->cnt, - &cnt, - cnt + 1, - _ODP_MEMMODEL_ACQ, - _ODP_MEMMODEL_RLX); + is_locked = + _odp_atomic_u32_cmp_xchg_strong_mm(&rwlock->cnt, + &cnt, + cnt + 1, + _ODP_MEMMODEL_ACQ, + _ODP_MEMMODEL_RLX); } } @@ -48,17 +49,19 @@ void odp_rwlock_write_lock(odp_rwlock_t *rwlock) while (is_locked == 0) { uint32_t zero = 0; + cnt = _odp_atomic_u32_load_mm(&rwlock->cnt, _ODP_MEMMODEL_RLX); /* lock acquired, wait */ if (cnt != 0) { odp_spin(); continue; } - is_locked = _odp_atomic_u32_cmp_xchg_strong_mm(&rwlock->cnt, - &zero, - (uint32_t)-1, - _ODP_MEMMODEL_ACQ, - _ODP_MEMMODEL_RLX); + is_locked = + _odp_atomic_u32_cmp_xchg_strong_mm(&rwlock->cnt, + &zero, + (uint32_t)-1, + _ODP_MEMMODEL_ACQ, + _ODP_MEMMODEL_RLX); } } diff --git a/platform/linux-generic/odp_schedule.c b/platform/linux-generic/odp_schedule.c index 689ec3f..172d0a0 100644 --- a/platform/linux-generic/odp_schedule.c +++ b/platform/linux-generic/odp_schedule.c @@ -28,19 +28,17 @@ #define NUM_SCHED_CMD (ODP_CONFIG_QUEUES + ODP_CONFIG_PKTIO_ENTRIES) /* Scheduler sub queues */ -#define QUEUES_PER_PRIO 4 +#define QUEUES_PER_PRIO 4 /* Maximum number of dequeues */ #define MAX_DEQ 4 - /* Mask of queues per priority */ typedef uint8_t pri_mask_t; -_ODP_STATIC_ASSERT((8*sizeof(pri_mask_t)) >= QUEUES_PER_PRIO, +_ODP_STATIC_ASSERT((8 * sizeof(pri_mask_t)) >= QUEUES_PER_PRIO, "pri_mask_t_is_too_small"); - typedef struct { odp_queue_t pri_queue[ODP_CONFIG_SCHED_PRIOS][QUEUES_PER_PRIO]; pri_mask_t pri_mask[ODP_CONFIG_SCHED_PRIOS]; @@ -52,7 +50,7 @@ typedef struct { /* Schedule command */ typedef struct { - int cmd; + int cmd; union { queue_entry_t *qe; @@ -68,7 +66,6 @@ typedef struct { #define SCHED_CMD_DEQUEUE 0 #define SCHED_CMD_POLL_PKTIN 1 - typedef struct { odp_queue_t pri_queue; odp_event_t cmd_ev; @@ -110,8 +107,7 @@ int odp_schedule_init_global(void) ODP_DBG("Schedule init ... "); - shm = odp_shm_reserve("odp_scheduler", - sizeof(sched_t), + shm = odp_shm_reserve("odp_scheduler", sizeof(sched_t), ODP_CACHE_LINE_SIZE, 0); sched = odp_shm_addr(shm); @@ -144,11 +140,11 @@ int odp_schedule_init_global(void) char name[] = "odp_priXX_YY"; name[7] = '0' + i / 10; - name[8] = '0' + i - 10*(i / 10); + name[8] = '0' + i - 10 * (i / 10); for (j = 0; j < QUEUES_PER_PRIO; j++) { name[10] = '0' + j / 10; - name[11] = '0' + j - 10*(j / 10); + name[11] = '0' + j - 10 * (j / 10); queue = odp_queue_create(name, ODP_QUEUE_TYPE_POLL, NULL); @@ -248,12 +244,12 @@ int odp_schedule_term_local(void) static int pri_id_queue(odp_queue_t queue) { - return (QUEUES_PER_PRIO-1) & (queue_to_id(queue)); + return (QUEUES_PER_PRIO - 1) & (queue_to_id(queue)); } static int pri_id_pktio(odp_pktio_t pktio) { - return (QUEUES_PER_PRIO-1) & (pktio_to_id(pktio)); + return (QUEUES_PER_PRIO - 1) & (pktio_to_id(pktio)); } static odp_queue_t pri_set(int id, int prio) @@ -296,12 +292,14 @@ static odp_queue_t pri_set_pktio(odp_pktio_t pktio, int prio) static void pri_clr_queue(odp_queue_t queue, int prio) { int id = pri_id_queue(queue); + pri_clr(id, prio); } static void pri_clr_pktio(odp_pktio_t pktio, int prio) { int id = pri_id_pktio(pktio); + pri_clr(id, prio); } @@ -349,25 +347,24 @@ int schedule_pktio_start(odp_pktio_t pktio, int prio) if (buf == ODP_BUFFER_INVALID) return -1; - sched_cmd = odp_buffer_addr(buf); - sched_cmd->cmd = SCHED_CMD_POLL_PKTIN; + sched_cmd = odp_buffer_addr(buf); + sched_cmd->cmd = SCHED_CMD_POLL_PKTIN; sched_cmd->pktio = pktio; - sched_cmd->pe = get_pktio_entry(pktio); - sched_cmd->prio = prio; + sched_cmd->pe = get_pktio_entry(pktio); + sched_cmd->prio = prio; - pri_queue = pri_set_pktio(pktio, prio); + pri_queue = pri_set_pktio(pktio, prio); if (odp_queue_enq(pri_queue, odp_buffer_to_event(buf))) ODP_ABORT("schedule_pktio_start failed\n"); - return 0; } void odp_schedule_release_atomic(void) { if (sched_local.pri_queue != ODP_QUEUE_INVALID && - sched_local.num == 0) { + sched_local.num == 0) { /* Release current atomic queue */ if (odp_queue_enq(sched_local.pri_queue, sched_local.cmd_ev)) ODP_ABORT("odp_schedule_release_atomic failed\n"); @@ -375,13 +372,13 @@ void odp_schedule_release_atomic(void) } } - static inline int copy_events(odp_event_t out_ev[], unsigned int max) { int i = 0; while (sched_local.num && max) { odp_buffer_hdr_t *hdr = sched_local.buf_hdr[sched_local.index]; + out_ev[i] = odp_buffer_to_event(hdr->handle.handle); sched_local.index++; sched_local.num--; @@ -392,7 +389,6 @@ static inline int copy_events(odp_event_t out_ev[], unsigned int max) return i; } - /* * Schedule queues * @@ -427,7 +423,7 @@ static int schedule(odp_queue_t *out_queue, odp_event_t out_ev[], if (sched->pri_mask[i] == 0) continue; - id = thr & (QUEUES_PER_PRIO-1); + id = thr & (QUEUES_PER_PRIO - 1); for (j = 0; j < QUEUES_PER_PRIO; j++, id++) { odp_queue_t pri_q; @@ -508,7 +504,6 @@ static int schedule(odp_queue_t *out_queue, odp_event_t out_ev[], return 0; } - static int schedule_loop(odp_queue_t *out_queue, uint64_t wait, odp_event_t out_ev[], unsigned int max_num, unsigned int max_deq) @@ -545,7 +540,6 @@ static int schedule_loop(odp_queue_t *out_queue, uint64_t wait, return ret; } - odp_event_t odp_schedule(odp_queue_t *out_queue, uint64_t wait) { odp_event_t ev; @@ -557,26 +551,22 @@ odp_event_t odp_schedule(odp_queue_t *out_queue, uint64_t wait) return ev; } - int odp_schedule_multi(odp_queue_t *out_queue, uint64_t wait, odp_event_t events[], int num) { return schedule_loop(out_queue, wait, events, num, MAX_DEQ); } - void odp_schedule_pause(void) { sched_local.pause = 1; } - void odp_schedule_resume(void) { sched_local.pause = 0; } - uint64_t odp_schedule_wait_time(uint64_t ns) { if (ns <= ODP_SCHED_NO_WAIT) @@ -585,7 +575,6 @@ uint64_t odp_schedule_wait_time(uint64_t ns) return odp_time_ns_to_cycles(ns); } - int odp_schedule_num_prio(void) { return ODP_CONFIG_SCHED_PRIOS; diff --git a/platform/linux-generic/odp_shared_memory.c b/platform/linux-generic/odp_shared_memory.c index ab48dda..38b0b28 100644 --- a/platform/linux-generic/odp_shared_memory.c +++ b/platform/linux-generic/odp_shared_memory.c @@ -45,35 +45,29 @@ typedef struct { } odp_shm_block_t; - typedef struct { odp_shm_block_t block[ODP_CONFIG_SHM_BLOCKS]; odp_spinlock_t lock; } odp_shm_table_t; - #ifndef MAP_ANONYMOUS #define MAP_ANONYMOUS MAP_ANON #endif - /* Global shared memory table */ static odp_shm_table_t *odp_shm_tbl; - static inline uint32_t from_handle(odp_shm_t shm) { return _odp_typeval(shm) - 1; } - static inline odp_shm_t to_handle(uint32_t index) { return _odp_cast_scalar(odp_shm_t, index + 1); } - int odp_shm_init_global(void) { void *addr; @@ -107,13 +101,11 @@ int odp_shm_term_global(void) return ret; } - int odp_shm_init_local(void) { return 0; } - static int find_block(const char *name, uint32_t *index) { uint32_t i; @@ -258,7 +250,7 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, uint64_t align, } addr = mmap(NULL, alloc_hp_size, PROT_READ | PROT_WRITE, - map_flag | MAP_HUGETLB, fd, 0); + map_flag | MAP_HUGETLB, fd, 0); if (addr == MAP_FAILED) { ODP_DBG(" %s:\n" "\tNo huge pages, fall back to normal pages,\n" @@ -281,7 +273,7 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, uint64_t align, } addr = mmap(NULL, alloc_size, PROT_READ | PROT_WRITE, - map_flag, fd, 0); + map_flag, fd, 0); if (addr == MAP_FAILED) { odp_spinlock_unlock(&odp_shm_tbl->lock); ODP_DBG("%s mmap failed.\n", name); @@ -328,7 +320,6 @@ odp_shm_t odp_shm_lookup(const char *name) return hdl; } - void *odp_shm_addr(odp_shm_t shm) { uint32_t i; @@ -341,7 +332,6 @@ void *odp_shm_addr(odp_shm_t shm) return odp_shm_tbl->block[i].addr; } - int odp_shm_info(odp_shm_t shm, odp_shm_info_t *info) { odp_shm_block_t *block; @@ -354,25 +344,24 @@ int odp_shm_info(odp_shm_t shm, odp_shm_info_t *info) block = &odp_shm_tbl->block[i]; - info->name = block->name; - info->addr = block->addr; - info->size = block->size; + info->name = block->name; + info->addr = block->addr; + info->size = block->size; info->page_size = block->page_sz; - info->flags = block->flags; + info->flags = block->flags; return 0; } - void odp_shm_print_all(void) { int i; ODP_PRINT("\nShared memory\n"); ODP_PRINT("--------------\n"); - ODP_PRINT(" page size: %"PRIu64" kB\n", + ODP_PRINT(" page size: %" PRIu64 " kB\n", odp_sys_page_size() / 1024); - ODP_PRINT(" huge page size: %"PRIu64" kB\n", + ODP_PRINT(" huge page size: %" PRIu64 " kB\n", odp_sys_huge_page_size() / 1024); ODP_PRINT("\n"); @@ -384,11 +373,11 @@ void odp_shm_print_all(void) block = &odp_shm_tbl->block[i]; if (block->addr) { - ODP_PRINT(" %2i %-24s %4"PRIu64" %4"PRIu64 + ODP_PRINT(" %2i %-24s %4" PRIu64 " %4" PRIu64 " %2c %p\n", i, block->name, - block->size/1024, + block->size / 1024, block->align, (block->huge ? '*' : ' '), block->addr); diff --git a/platform/linux-generic/odp_spinlock.c b/platform/linux-generic/odp_spinlock.c index f165720..acdf5b6 100644 --- a/platform/linux-generic/odp_spinlock.c +++ b/platform/linux-generic/odp_spinlock.c @@ -8,13 +8,11 @@ #include #include - void odp_spinlock_init(odp_spinlock_t *spinlock) { _odp_atomic_flag_init(&spinlock->lock, 0); } - void odp_spinlock_lock(odp_spinlock_t *spinlock) { /* While the lock is already taken... */ @@ -26,19 +24,16 @@ void odp_spinlock_lock(odp_spinlock_t *spinlock) odp_spin(); } - int odp_spinlock_trylock(odp_spinlock_t *spinlock) { return (_odp_atomic_flag_tas(&spinlock->lock) == 0); } - void odp_spinlock_unlock(odp_spinlock_t *spinlock) { _odp_atomic_flag_clear(&spinlock->lock); } - int odp_spinlock_is_locked(odp_spinlock_t *spinlock) { return _odp_atomic_flag_load(&spinlock->lock) != 0; diff --git a/platform/linux-generic/odp_system_info.c b/platform/linux-generic/odp_system_info.c index 31df29e..8897832 100644 --- a/platform/linux-generic/odp_system_info.c +++ b/platform/linux-generic/odp_system_info.c @@ -23,20 +23,17 @@ #include #include - - typedef struct { const char *cpu_arch_str; int (*cpuinfo_parser)(FILE *file, odp_system_info_t *sysinfo); } odp_compiler_info_t; -#define CACHE_LNSZ_FILE \ +#define CACHE_LNSZ_FILE \ "/sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size" #define HUGE_PAGE_DIR "/sys/kernel/mm/hugepages" - /* * Report the number of CPUs in the affinity mask of the main thread */ @@ -45,22 +42,21 @@ static int sysconf_cpu_count(void) cpu_set_t cpuset; int ret; - ret = pthread_getaffinity_np(pthread_self(), - sizeof(cpuset), &cpuset); + ret = pthread_getaffinity_np(pthread_self(), sizeof(cpuset), &cpuset); if (ret != 0) return 0; return CPU_COUNT(&cpuset); } -#if defined __x86_64__ || defined __i386__ || defined __OCTEON__ || \ -defined __powerpc__ +#if defined __x86_64__ || defined __i386__ || defined __OCTEON__ || \ + defined __powerpc__ /* * Analysis of /sys/devices/system/cpu/ files */ static int systemcpu_cache_line_size(void) { - FILE *file; + FILE *file; char str[128]; int size = 0; @@ -72,7 +68,10 @@ static int systemcpu_cache_line_size(void) if (fgets(str, sizeof(str), file) != NULL) { /* Read cache line size */ - sscanf(str, "%i", &size); + if (!sscanf(str, "%i", &size) == EOF) { + ODP_ERR("failed to read chche line size\n"); + size = 0; + } } fclose(file); @@ -81,7 +80,6 @@ static int systemcpu_cache_line_size(void) } #endif - static int huge_page_size(void) { DIR *dir; @@ -96,6 +94,7 @@ static int huge_page_size(void) while ((dirent = readdir(dir)) != NULL) { int temp = 0; + sscanf(dirent->d_name, "hugepages-%i", &temp); if (temp > size) @@ -107,11 +106,9 @@ static int huge_page_size(void) return 0; } - return size*1024; + return size * 1024; } - - /* * HW specific /proc/cpuinfo file parsing */ @@ -138,8 +135,9 @@ static int cpuinfo_x86(FILE *file, odp_system_info_t *sysinfo) pos = strstr(str, "model name"); if (pos) { int len; + pos = strchr(str, ':'); - strncpy(sysinfo->model_str, pos+2, + strncpy(sysinfo->model_str, pos + 2, sizeof(sysinfo->model_str)); len = strlen(sysinfo->model_str); sysinfo->model_str[len - 1] = 0; @@ -149,7 +147,7 @@ static int cpuinfo_x86(FILE *file, odp_system_info_t *sysinfo) } } - sysinfo->cpu_hz = (uint64_t) (mhz * 1000000.0); + sysinfo->cpu_hz = (uint64_t)(mhz * 1000000.0); return 0; } @@ -157,7 +155,7 @@ static int cpuinfo_x86(FILE *file, odp_system_info_t *sysinfo) #elif defined __arm__ || defined __aarch64__ static int cpuinfo_arm(FILE *file ODP_UNUSED, -odp_system_info_t *sysinfo ODP_UNUSED) + odp_system_info_t *sysinfo ODP_UNUSED) { return 0; } @@ -187,8 +185,9 @@ static int cpuinfo_octeon(FILE *file, odp_system_info_t *sysinfo) if (pos) { int len; + pos = strchr(str, ':'); - strncpy(sysinfo->model_str, pos+2, + strncpy(sysinfo->model_str, pos + 2, sizeof(sysinfo->model_str)); len = strlen(sysinfo->model_str); sysinfo->model_str[len - 1] = 0; @@ -199,7 +198,7 @@ static int cpuinfo_octeon(FILE *file, odp_system_info_t *sysinfo) } /* bogomips seems to be 2x freq */ - sysinfo->cpu_hz = (uint64_t) (mhz * 1000000.0 / 2.0); + sysinfo->cpu_hz = (uint64_t)(mhz * 1000000.0 / 2.0); return 0; } @@ -227,8 +226,9 @@ static int cpuinfo_powerpc(FILE *file, odp_system_info_t *sysinfo) if (pos) { int len; + pos = strchr(str, ':'); - strncpy(sysinfo->model_str, pos+2, + strncpy(sysinfo->model_str, pos + 2, sizeof(sysinfo->model_str)); len = strlen(sysinfo->model_str); sysinfo->model_str[len - 1] = 0; @@ -237,42 +237,36 @@ static int cpuinfo_powerpc(FILE *file, odp_system_info_t *sysinfo) } } - sysinfo->cpu_hz = (uint64_t) (mhz * 1000000.0); + sysinfo->cpu_hz = (uint64_t)(mhz * 1000000.0); } - return 0; } #else - #error GCC target not found +#error GCC target not found #endif static odp_compiler_info_t compiler_info = { - #if defined __x86_64__ || defined __i386__ - .cpu_arch_str = "x86", - .cpuinfo_parser = cpuinfo_x86 +#if defined __x86_64__ || defined __i386__ + .cpu_arch_str = "x86", .cpuinfo_parser = cpuinfo_x86 - #elif defined __arm__ || defined __aarch64__ - .cpu_arch_str = "arm", - .cpuinfo_parser = cpuinfo_arm +#elif defined __arm__ || defined __aarch64__ + .cpu_arch_str = "arm", .cpuinfo_parser = cpuinfo_arm - #elif defined __OCTEON__ - .cpu_arch_str = "octeon", - .cpuinfo_parser = cpuinfo_octeon +#elif defined __OCTEON__ + .cpu_arch_str = "octeon", .cpuinfo_parser = cpuinfo_octeon - #elif defined __powerpc__ - .cpu_arch_str = "powerpc", - .cpuinfo_parser = cpuinfo_powerpc +#elif defined __powerpc__ + .cpu_arch_str = "powerpc", .cpuinfo_parser = cpuinfo_powerpc - #else - #error GCC target not found - #endif +#else +#error GCC target not found +#endif }; - -#if defined __x86_64__ || defined __i386__ || defined __OCTEON__ || \ -defined __powerpc__ +#if defined __x86_64__ || defined __i386__ || defined __OCTEON__ || \ + defined __powerpc__ /* * Analysis of /sys/devices/system/cpu/ files @@ -289,7 +283,6 @@ static int systemcpu(odp_system_info_t *sysinfo) sysinfo->cpu_count = ret; - ret = systemcpu_cache_line_size(); if (ret == 0) { ODP_ERR("systemcpu_cache_line_size failed.\n"); @@ -314,7 +307,6 @@ static int systemcpu(odp_system_info_t *sysinfo) * Use sysconf and dummy values in generic case */ - static int systemcpu(odp_system_info_t *sysinfo) { int ret; @@ -345,7 +337,7 @@ static int systemcpu(odp_system_info_t *sysinfo) */ int odp_system_info_init(void) { - FILE *file; + FILE *file; memset(&odp_global_data.system_info, 0, sizeof(odp_system_info_t)); diff --git a/platform/linux-generic/odp_thread.c b/platform/linux-generic/odp_thread.c index 9905c78..13c5ff2 100644 --- a/platform/linux-generic/odp_thread.c +++ b/platform/linux-generic/odp_thread.c @@ -29,33 +29,28 @@ typedef struct { odp_thread_type_t type; } thread_state_t; - typedef struct { thread_state_t thr[ODP_CONFIG_MAX_THREADS]; - odp_thrmask_t all; - odp_thrmask_t worker; - odp_thrmask_t control; - uint32_t num; - uint32_t num_worker; - uint32_t num_control; + odp_thrmask_t all; + odp_thrmask_t worker; + odp_thrmask_t control; + uint32_t num; + uint32_t num_worker; + uint32_t num_control; odp_spinlock_t lock; } thread_globals_t; - /* Globals */ static thread_globals_t *thread_globals; - /* Thread local */ static __thread thread_state_t *this_thread; - int odp_thread_init_global(void) { odp_shm_t shm; - shm = odp_shm_reserve("odp_thread_globals", - sizeof(thread_globals_t), + shm = odp_shm_reserve("odp_thread_globals", sizeof(thread_globals_t), ODP_CACHE_LINE_SIZE, 0); thread_globals = odp_shm_addr(shm); diff --git a/platform/linux-generic/odp_ticketlock.c b/platform/linux-generic/odp_ticketlock.c index 3e2a4ec..8d32535 100644 --- a/platform/linux-generic/odp_ticketlock.c +++ b/platform/linux-generic/odp_ticketlock.c @@ -10,14 +10,12 @@ #include #include - void odp_ticketlock_init(odp_ticketlock_t *ticketlock) { odp_atomic_init_u32(&ticketlock->next_ticket, 0); odp_atomic_init_u32(&ticketlock->cur_ticket, 0); } - void odp_ticketlock_lock(odp_ticketlock_t *ticketlock) { uint32_t ticket; @@ -82,7 +80,6 @@ void odp_ticketlock_unlock(odp_ticketlock_t *ticketlock) #endif } - int odp_ticketlock_is_locked(odp_ticketlock_t *ticketlock) { /* Compare 'cur_ticket' with 'next_ticket'. Ideally we should read diff --git a/platform/linux-generic/odp_time.c b/platform/linux-generic/odp_time.c index a08833d..2108081 100644 --- a/platform/linux-generic/odp_time.c +++ b/platform/linux-generic/odp_time.c @@ -25,18 +25,17 @@ uint64_t odp_time_cycles_to_ns(uint64_t cycles) uint64_t hz = odp_sys_cpu_hz(); if (cycles > (UINT64_MAX / GIGA)) - return (cycles/hz)*GIGA; + return (cycles / hz) * GIGA; - return (cycles*GIGA)/hz; + return (cycles * GIGA) / hz; } - uint64_t odp_time_ns_to_cycles(uint64_t ns) { uint64_t hz = odp_sys_cpu_hz(); if (ns > (UINT64_MAX / hz)) - return (ns/GIGA)*hz; + return (ns / GIGA) * hz; - return (ns*hz)/GIGA; + return (ns * hz) / GIGA; } diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index e8f0267..45074cc 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -84,6 +84,7 @@ static odp_timeout_hdr_t *timeout_hdr_from_buf(odp_buffer_t buf) static odp_timeout_hdr_t *timeout_hdr(odp_timeout_t tmo) { odp_buffer_t buf = odp_buffer_from_event(odp_timeout_to_event(tmo)); + return timeout_hdr_from_buf(buf); } @@ -111,9 +112,9 @@ typedef struct odp_timer_s { } odp_timer; static void timer_init(odp_timer *tim, - tick_buf_t *tb, - odp_queue_t _q, - void *_up) + tick_buf_t *tb, + odp_queue_t _q, + void *_up) { tim->queue = _q; tim->user_ptr = _up; @@ -178,8 +179,10 @@ static odp_timer_pool *timer_pool[MAX_TIMER_POOLS]; static inline odp_timer_pool *handle_to_tp(odp_timer_t hdl) { uint32_t tp_idx = hdl >> INDEX_BITS; + if (odp_likely(tp_idx < MAX_TIMER_POOLS)) { odp_timer_pool *tp = timer_pool[tp_idx]; + if (odp_likely(tp != NULL)) return timer_pool[tp_idx]; } @@ -187,9 +190,10 @@ static inline odp_timer_pool *handle_to_tp(odp_timer_t hdl) } static inline uint32_t handle_to_idx(odp_timer_t hdl, - struct odp_timer_pool_s *tp) + struct odp_timer_pool_s *tp) { uint32_t idx = hdl & ((1U << INDEX_BITS) - 1U); + PREFETCH(&tp->tick_buf[idx]); if (odp_likely(idx < odp_atomic_load_u32(&tp->high_wm))) return idx; @@ -197,7 +201,7 @@ static inline uint32_t handle_to_idx(odp_timer_t hdl, } static inline odp_timer_t tp_idx_to_handle(struct odp_timer_pool_s *tp, - uint32_t idx) + uint32_t idx) { ODP_ASSERT(idx < (1U << INDEX_BITS)); return (tp->tp_idx << INDEX_BITS) | idx; @@ -212,6 +216,7 @@ static odp_timer_pool *odp_timer_pool_new( const odp_timer_pool_param_t *param) { uint32_t tp_idx = odp_atomic_fetch_add_u32(&num_timer_pools, 1); + if (odp_unlikely(tp_idx >= MAX_TIMER_POOLS)) { /* Restore the previous value */ odp_atomic_sub_u32(&num_timer_pools, 1); @@ -230,6 +235,7 @@ static odp_timer_pool *odp_timer_pool_new( ODP_ABORT("%s: timer pool shm-alloc(%zuKB) failed\n", _name, (sz0 + sz1 + sz2) / 1024); odp_timer_pool *tp = (odp_timer_pool *)odp_shm_addr(shm); + odp_atomic_init_u64(&tp->cur_tick, 0); snprintf(tp->name, sizeof(tp->name), "%s", _name); tp->shm = shm; @@ -243,6 +249,7 @@ static odp_timer_pool *odp_timer_pool_new( tp->timers = (void *)((char *)odp_shm_addr(shm) + sz0 + sz1); /* Initialize all odp_timer entries */ uint32_t i; + for (i = 0; i < tp->param.num_timers; i++) { tp->timers[i].queue = ODP_QUEUE_INVALID; set_next_free(&tp->timers[i], i + 1); @@ -273,6 +280,7 @@ static void odp_timer_pool_del(odp_timer_pool *tp) if (tp->param.clk_src == ODP_CLOCK_CPU) itimer_fini(tp); int rc = odp_shm_free(tp->shm); + if (rc != 0) ODP_ABORT("Failed to free shared memory (%d)\n", rc); } @@ -282,6 +290,7 @@ static inline odp_timer_t timer_alloc(odp_timer_pool *tp, void *user_ptr) { odp_timer_t hdl; + odp_spinlock_lock(&tp->lock); if (odp_likely(tp->num_alloc < tp->param.num_timers)) { tp->num_alloc++; @@ -289,6 +298,7 @@ static inline odp_timer_t timer_alloc(odp_timer_pool *tp, ODP_ASSERT(tp->first_free != tp->param.num_timers); uint32_t idx = tp->first_free; odp_timer *tim = &tp->timers[idx]; + tp->first_free = get_next_free(tim); /* Initialize timer */ timer_init(tim, &tp->tick_buf[idx], queue, user_ptr); @@ -309,8 +319,8 @@ static inline odp_timer_t timer_alloc(odp_timer_pool *tp, } static odp_buffer_t timer_cancel(odp_timer_pool *tp, - uint32_t idx, - uint64_t new_state); + uint32_t idx, + uint64_t new_state); static inline odp_buffer_t timer_free(odp_timer_pool *tp, uint32_t idx) { @@ -340,9 +350,9 @@ static inline odp_buffer_t timer_free(odp_timer_pool *tp, uint32_t idx) *****************************************************************************/ static bool timer_reset(uint32_t idx, - uint64_t abs_tck, - odp_buffer_t *tmo_buf, - odp_timer_pool *tp) + uint64_t abs_tck, + odp_buffer_t *tmo_buf, + odp_timer_pool *tp) { bool success = true; tick_buf_t *tb = &tp->tick_buf[idx]; @@ -350,6 +360,7 @@ static bool timer_reset(uint32_t idx, if (tmo_buf == NULL || *tmo_buf == ODP_BUFFER_INVALID) { #ifdef ODP_ATOMIC_U128 tick_buf_t new, old; + do { /* Relaxed and non-atomic read of current values */ old.exp_tck.v = tb->exp_tck.v; @@ -383,7 +394,7 @@ static bool timer_reset(uint32_t idx, /* Swap in new expiration tick, get back old tick which * will indicate active/inactive timer state */ old = _odp_atomic_u64_xchg_mm(&tb->exp_tck, abs_tck, - _ODP_MEMMODEL_RLX); + _ODP_MEMMODEL_RLX); if ((old & TMO_INACTIVE) != 0) { /* Timer was inactive (cancelled or expired), * we can't reset a timer without a timeout buffer. @@ -440,6 +451,7 @@ static bool timer_reset(uint32_t idx, odp_buffer_t old_buf = ODP_BUFFER_INVALID; #ifdef ODP_ATOMIC_U128 tick_buf_t new, old; + new.exp_tck.v = abs_tck; new.tmo_buf = *tmo_buf; TB_SET_PAD(new); @@ -474,8 +486,8 @@ static bool timer_reset(uint32_t idx, } static odp_buffer_t timer_cancel(odp_timer_pool *tp, - uint32_t idx, - uint64_t new_state) + uint32_t idx, + uint64_t new_state) { tick_buf_t *tb = &tp->tick_buf[idx]; odp_buffer_t old_buf; @@ -526,6 +538,7 @@ static unsigned timer_expire(odp_timer_pool *tp, uint32_t idx, uint64_t tick) /* Attempt to grab timeout buffer, replace with inactive timer * and invalid buffer */ tick_buf_t new, old; + old.exp_tck.v = exp_tck; old.tmo_buf = tb->tmo_buf; TB_SET_PAD(old); @@ -617,6 +630,7 @@ static unsigned odp_timer_pool_expire(odp_timer_pool_t tpid, uint64_t tick) #endif /* Non-atomic read for speed */ uint64_t exp_tck = array[i++].exp_tck.v; + if (odp_unlikely(exp_tck <= tick)) { /* Attempt to expire timer */ nexp += timer_expire(tpid, i - 1, tick); @@ -636,11 +650,13 @@ static void timer_notify(sigval_t sigval) #ifdef __ARM_ARCH odp_timer *array = &tp->timers[0]; uint32_t i; + /* Prefetch initial cache lines (match 32 above) */ for (i = 0; i < 32; i += ODP_CACHE_LINE_SIZE / sizeof(array[0])) PREFETCH(&array[i]); #endif uint64_t prev_tick = odp_atomic_fetch_inc_u64(&tp->cur_tick); + /* Attempt to acquire the lock, check if the old value was clear */ if (odp_spinlock_trylock(&tp->itimer_running)) { /* Scan timer array, looking for timers to expire */ @@ -756,6 +772,7 @@ odp_timer_t odp_timer_alloc(odp_timer_pool_t tpid, /* We don't care about the validity of user_ptr because we will not * attempt to dereference it */ odp_timer_t hdl = timer_alloc(tpid, queue, user_ptr); + if (odp_likely(hdl != ODP_TIMER_INVALID)) { /* Success */ return hdl; @@ -769,6 +786,7 @@ odp_event_t odp_timer_free(odp_timer_t hdl) odp_timer_pool *tp = handle_to_tp(hdl); uint32_t idx = handle_to_idx(hdl, tp); odp_buffer_t old_buf = timer_free(tp, idx); + return odp_buffer_to_event(old_buf); } @@ -779,6 +797,7 @@ int odp_timer_set_abs(odp_timer_t hdl, odp_timer_pool *tp = handle_to_tp(hdl); uint32_t idx = handle_to_idx(hdl, tp); uint64_t cur_tick = odp_atomic_load_u64(&tp->cur_tick); + if (odp_unlikely(abs_tck < cur_tick + tp->min_rel_tck)) return ODP_TIMER_TOOEARLY; if (odp_unlikely(abs_tck > cur_tick + tp->max_rel_tck)) @@ -796,6 +815,7 @@ int odp_timer_set_rel(odp_timer_t hdl, odp_timer_pool *tp = handle_to_tp(hdl); uint32_t idx = handle_to_idx(hdl, tp); uint64_t abs_tck = odp_atomic_load_u64(&tp->cur_tick) + rel_tck; + if (odp_unlikely(rel_tck < tp->min_rel_tck)) return ODP_TIMER_TOOEARLY; if (odp_unlikely(rel_tck > tp->max_rel_tck)) @@ -810,8 +830,10 @@ int odp_timer_cancel(odp_timer_t hdl, odp_event_t *tmo_ev) { odp_timer_pool *tp = handle_to_tp(hdl); uint32_t idx = handle_to_idx(hdl, tp); + /* Set the expiration tick of the timer to TMO_INACTIVE */ odp_buffer_t old_buf = timer_cancel(tp, idx, TMO_INACTIVE); + if (old_buf != ODP_BUFFER_INVALID) { *tmo_ev = odp_buffer_to_event(old_buf); return 0; /* Active timer cancelled, timeout returned */ @@ -841,6 +863,7 @@ int odp_timeout_fresh(odp_timeout_t tmo) uint32_t idx = handle_to_idx(hdl, tp); tick_buf_t *tb = &tp->tick_buf[idx]; uint64_t exp_tck = odp_atomic_load_u64(&tb->exp_tck); + /* Return true if the timer still has the same expiration tick * (ignoring the inactive/expired bit) as the timeout */ return hdr->expiration == (exp_tck & ~TMO_INACTIVE); @@ -864,6 +887,7 @@ void *odp_timeout_user_ptr(odp_timeout_t tmo) odp_timeout_t odp_timeout_alloc(odp_pool_t pool) { odp_buffer_t buf = odp_buffer_alloc(pool); + if (odp_unlikely(buf == ODP_BUFFER_INVALID)) return ODP_TIMEOUT_INVALID; return odp_timeout_from_event(odp_buffer_to_event(buf)); @@ -872,6 +896,7 @@ odp_timeout_t odp_timeout_alloc(odp_pool_t pool) void odp_timeout_free(odp_timeout_t tmo) { odp_event_t ev = odp_timeout_to_event(tmo); + odp_buffer_free(odp_buffer_from_event(ev)); } @@ -879,6 +904,7 @@ int odp_timer_init_global(void) { #ifndef ODP_ATOMIC_U128 uint32_t i; + for (i = 0; i < NUM_LOCKS; i++) _odp_atomic_flag_clear(&locks[i]); #else diff --git a/platform/linux-generic/pktio/socket.c b/platform/linux-generic/pktio/socket.c index 45040fd..7cf7477 100644 --- a/platform/linux-generic/pktio/socket.c +++ b/platform/linux-generic/pktio/socket.c @@ -160,6 +160,7 @@ int promisc_mode_get_fd(int fd, const char *name) static int sock_close(pktio_entry_t *pktio_entry) { pkt_sock_t *pkt_sock = &pktio_entry->s.pkt_sock; + if (pkt_sock->sockfd != -1 && close(pkt_sock->sockfd) != 0) { __odp_errno = errno; ODP_ERR("close(sockfd): %s\n", strerror(errno)); diff --git a/platform/linux-generic/pktio/socket_mmap.c b/platform/linux-generic/pktio/socket_mmap.c index 7d42678..34f6e38 100644 --- a/platform/linux-generic/pktio/socket_mmap.c +++ b/platform/linux-generic/pktio/socket_mmap.c @@ -497,6 +497,7 @@ static int sock_mmap_recv(pktio_entry_t *pktio_entry, odp_packet_t pkt_table[], unsigned len) { pkt_sock_mmap_t *const pkt_sock = &pktio_entry->s.pkt_sock_mmap; + return pkt_mmap_v2_rx(pkt_sock->rx_ring.sock, &pkt_sock->rx_ring, pkt_table, len, pkt_sock->pool, pkt_sock->if_mac); @@ -506,6 +507,7 @@ static int sock_mmap_send(pktio_entry_t *pktio_entry, odp_packet_t pkt_table[], unsigned len) { pkt_sock_mmap_t *const pkt_sock = &pktio_entry->s.pkt_sock_mmap; + return pkt_mmap_v2_tx(pkt_sock->tx_ring.sock, &pkt_sock->tx_ring, pkt_table, len); }