From patchwork Fri Jul 24 13:31:33 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ivan Khoronzhuk X-Patchwork-Id: 51446 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by patches.linaro.org (Postfix) with ESMTPS id F12EC20323 for ; Fri, 24 Jul 2015 13:32:50 +0000 (UTC) Received: by laef2 with SMTP id f2sf7684714lae.0 for ; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=ycG5yZtKe+SQQcckmJOv295N7uX7CbVT5/xklhv88Fg=; b=HpkXhMhZJs0pIWiBd5Qb3szqJ256bclGkSIHyW3v5Doi68qL7Ox5IPjdIzlCnc3/bC 6iPcSWOF+PbuaLxb1CEGmD1PkpEU9Q8gpXHpKgkoR3Itu8GoAHYgYj6CSVw7Duu9pzKx Itgh24NM8kpkKh3VUkfqDFuT4xe610MXS2NNcrd27ymHSnN9ZHvYgfsM9QuT3DSYYGYj FFqq59ximfiYfkp4j3fHkK0sSTGdS0X6pljnM9iI/NMlIFt1rxTvrz76CgJI8TJPp7f5 S5j0Yj6gFlsbKJP06KptRKCJc3uq+ud4jAo4gbpG38zdMA2L4Z/iKkejsvQ7b4j+3qck nvpA== X-Gm-Message-State: ALoCoQlc1ZiLawu5lV9giK4d2HjaOcK7wUch1nSXMl3spWZEgsFn+89iaWhcOMcPmoFydHMaVWKf X-Received: by 10.112.171.41 with SMTP id ar9mr6257236lbc.24.1437744769941; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.179.171 with SMTP id dh11ls332884lac.47.gmail; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) X-Received: by 10.112.140.132 with SMTP id rg4mr12549720lbb.49.1437744769788; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com. [209.85.217.172]) by mx.google.com with ESMTPS id o1si7564779lbc.8.2015.07.24.06.32.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 24 Jul 2015 06:32:49 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.172 as permitted sender) client-ip=209.85.217.172; Received: by lbbyj8 with SMTP id yj8so15352446lbb.0 for ; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) X-Received: by 10.112.219.70 with SMTP id pm6mr13433934lbc.41.1437744769639; Fri, 24 Jul 2015 06:32:49 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.7.198 with SMTP id l6csp1186898lba; Fri, 24 Jul 2015 06:32:48 -0700 (PDT) X-Received: by 10.140.130.19 with SMTP id 19mr21716454qhc.2.1437744768011; Fri, 24 Jul 2015 06:32:48 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id d63si10055482qkh.35.2015.07.24.06.32.47; Fri, 24 Jul 2015 06:32:48 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 2780A61C83; Fri, 24 Jul 2015 13:32:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id D0E7F6182D; Fri, 24 Jul 2015 13:32:11 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D0A5A61B1D; Fri, 24 Jul 2015 13:31:56 +0000 (UTC) Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) by lists.linaro.org (Postfix) with ESMTPS id E700E61822 for ; Fri, 24 Jul 2015 13:31:54 +0000 (UTC) Received: by wibud3 with SMTP id ud3so28173233wib.0 for ; Fri, 24 Jul 2015 06:31:54 -0700 (PDT) X-Received: by 10.194.63.20 with SMTP id c20mr29568158wjs.134.1437744713905; Fri, 24 Jul 2015 06:31:53 -0700 (PDT) Received: from khorivan.synapse.com (104.46.77.93.pool.smart.vn.ua. [93.77.46.104]) by smtp.gmail.com with ESMTPSA id df1sm3569097wib.12.2015.07.24.06.31.52 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 24 Jul 2015 06:31:53 -0700 (PDT) From: Ivan Khoronzhuk To: lng-odp@lists.linaro.org Date: Fri, 24 Jul 2015 16:31:33 +0300 Message-Id: <1437744694-8615-2-git-send-email-ivan.khoronzhuk@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1437744694-8615-1-git-send-email-ivan.khoronzhuk@linaro.org> References: <1437744694-8615-1-git-send-email-ivan.khoronzhuk@linaro.org> X-Topics: timers patch Subject: [lng-odp] [Patch v3 1/2] example: timer: delete races while termination X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ivan.khoronzhuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.172 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Current implementation has at least two races that lead to several issues: - gbls->remain can overflow. One thread can decrement remain counter to 0. While another can decrement it once again and it will be > 0. After what some thread will loop very long time ... - Several threads can terminate the same timer and as result the same event. After out from the main loop a thread terminates a last timer it used. But a last timer saved in ttp for a thread can be received in another thread. So after leaving the main loop two threads can hold the same timer. - Some timer cannot be freed as several threads try to delete the same timer, as result one of the timer/tmo stay not freed after termination. - The test can send more events that requested. The receiving of requested number of tmos doesn't mean the test sent the same number. It rather sent more. This patch is intended to fix above drawbacks. The termination path must follow the next things: - An event can be in the following places: in a timer (waiting to be scheduled), in a queue for a thread to be scheduled, received in the main loop. - An event "holds" a timer, so when we receive an event we can delete it's timer. - a thread cannot delete timer w/o an event as it doesn't know who is owner of the event (and obvious the timer). - a thread shouldn't send events more than requested. - all threads have to be "held" in the loop till a last received event. The scheduler can assign event for any of the threads, so one thread can receive two last events for example. According to above, added several improvements: - don't send more timeouts that supposed to receive - free timer and tmo for a last received tmos = num of threads. - leave the main loop only if a last tmo/timer is free. Signed-off-by: Ivan Khoronzhuk --- example/timer/odp_timer_test.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/example/timer/odp_timer_test.c b/example/timer/odp_timer_test.c index 5e4306e..29d32af 100644 --- a/example/timer/odp_timer_test.c +++ b/example/timer/odp_timer_test.c @@ -47,6 +47,7 @@ typedef struct { odp_timer_pool_t tp; /**< Timer pool handle*/ odp_atomic_u32_t remain; /**< Number of timeouts to receive*/ struct test_timer tt[256]; /**< Array of all timer helper structs*/ + uint32_t num_workers; /**< Number of threads */ } test_globals_t; /** @private Timer set status ASCII strings */ @@ -139,16 +140,18 @@ static void test_abs_timeouts(int thr, test_globals_t *gbls) ttp->ev = odp_timeout_to_event(tmo); tick = odp_timer_current_tick(gbls->tp); - while ((int)odp_atomic_load_u32(&gbls->remain) > 0) { + while (1) { odp_event_t ev; odp_timer_set_t rc; - tick += period; - rc = odp_timer_set_abs(ttp->tim, tick, &ttp->ev); - if (odp_unlikely(rc != ODP_TIMER_SUCCESS)) { - /* Too early or too late timeout requested */ - EXAMPLE_ABORT("odp_timer_set_abs() failed: %s\n", - timerset2str(rc)); + if (ttp) { + tick += period; + rc = odp_timer_set_abs(ttp->tim, tick, &ttp->ev); + if (odp_unlikely(rc != ODP_TIMER_SUCCESS)) { + /* Too early or too late timeout requested */ + EXAMPLE_ABORT("odp_timer_set_abs() failed: %s\n", + timerset2str(rc)); + } } /* Get the next expired timeout. @@ -185,18 +188,18 @@ static void test_abs_timeouts(int thr, test_globals_t *gbls) } EXAMPLE_DBG(" [%i] timeout, tick %"PRIu64"\n", thr, tick); - odp_atomic_dec_u32(&gbls->remain); - } + uint32_t rx_num = odp_atomic_fetch_dec_u32(&gbls->remain); + + if (!rx_num) + EXAMPLE_ABORT("Unexpected timeout received (timer %x, tick %"PRIu64")\n", + ttp->tim, tick); + else if (rx_num > gbls->num_workers) + continue; - /* Cancel and free last timer used */ - (void)odp_timer_cancel(ttp->tim, &ttp->ev); - if (ttp->ev != ODP_EVENT_INVALID) odp_timeout_free(odp_timeout_from_event(ttp->ev)); - else - EXAMPLE_ERR("Lost timeout event at timer cancel\n"); - /* Since we have cancelled the timer, there is no timeout event to - * return from odp_timer_free() */ - (void)odp_timer_free(ttp->tim); + odp_timer_free(ttp->tim); + ttp = NULL; + } /* Remove any prescheduled events */ remove_prescheduled_events(); @@ -483,6 +486,8 @@ int main(int argc, char *argv[]) printf("\n"); + gbls->num_workers = num_workers; + /* Initialize number of timeouts to receive */ odp_atomic_init_u32(&gbls->remain, gbls->args.tmo_count * num_workers);