From patchwork Fri Jan 5 15:00:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 123533 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp917491qgn; Fri, 5 Jan 2018 07:03:16 -0800 (PST) X-Google-Smtp-Source: ACJfBovC5blMlKhomvcBBnqpFdDUXFMI5rBnfCKcdsiQ0M5K/MJ9pMDEf03hTVL/mUhaRyZYTxEL X-Received: by 10.200.39.167 with SMTP id w36mr4567596qtw.206.1515164596808; Fri, 05 Jan 2018 07:03:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515164596; cv=none; d=google.com; s=arc-20160816; b=S5Sf5+8K0h1YKSQz0fOefaGRTu7mrF1vJklzxfCMKU39ESWX8KBJqLzdIXYM7AIMLd KUwOXivc0IaCglo8NEugwHy1lsbm/qscZb8bh9skCalEkJtKcnYK4VdxWjBNKdu+KZEL odslHERUiPy95uYGKiDQaWbdeRAon+5rlT2T3D6CAk+y2tata7mw+dCW67gvnqsByzch SzsEZqsY77C1VK5ijqaNznJAoQhYomLONax2tWAgtRF4U+qxMx/kiER7aHgHhgITyKAv l+K8WU+vSazWGfxQ8sqBcVBTChQ1yh/VqaBz+F03QXGsHeeObcx+P7ILtOhcJ5do48mp /M5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=xBXwfuIP2RWFbMSasUfuVXk9dB2lNkUryStQVFkEXiA=; b=yNWfxrulbt4GzHLNvAHqARxMe33n+6MgmXXcG/zKwSRh1TUbJrZyZ8JcWXwxB9wWu4 b3puotJRChlNHcYG4TKoQ/0Crs+59Mdp41qYSjBRkDKHD8l4+UsVkhkCUWOcqq6qLLPC ATeNc1fGnPsK6Ue9hlmQT/v8ZQnVqhdK22lDKk66CLJR8dR87fht7chDwUFr/+hx+cVd oPi1QceZrddDSuUQ57C5pG+qKpSGaV1FdUdx9ZM7z5LZ2H/S+03ZrAoeyx8Mt5Anw2xl smWN9sr2Vc1X2Uejmu4bH8rWb6fDdJPpHMFiJ2Pm4gDlOJAdmZiPgApUJhW1HPajAZfZ 8FSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id j29si1481094qtj.339.2018.01.05.07.03.02; Fri, 05 Jan 2018 07:03:16 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id A75696107F; Fri, 5 Jan 2018 15:03:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 2592861104; Fri, 5 Jan 2018 15:01:09 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 98F2F60AEF; Fri, 5 Jan 2018 15:00:58 +0000 (UTC) Received: from forward105p.mail.yandex.net (forward105p.mail.yandex.net [77.88.28.108]) by lists.linaro.org (Postfix) with ESMTPS id 2330060859 for ; Fri, 5 Jan 2018 15:00:22 +0000 (UTC) Received: from mxback5o.mail.yandex.net (mxback5o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::1f]) by forward105p.mail.yandex.net (Yandex) with ESMTP id AFC19408242D for ; Fri, 5 Jan 2018 18:00:20 +0300 (MSK) Received: from smtp3p.mail.yandex.net (smtp3p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:8]) by mxback5o.mail.yandex.net (nwsmtp/Yandex) with ESMTP id R70MbjtKJG-0EImLFhH; Fri, 05 Jan 2018 18:00:14 +0300 Received: by smtp3p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id ty462EFbUF-0DdGmdx1; Fri, 05 Jan 2018 18:00:13 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Fri, 5 Jan 2018 18:00:09 +0300 Message-Id: <1515164409-28381-4-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515164409-28381-1-git-send-email-odpbot@yandex.ru> References: <1515164409-28381-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 381 Subject: [lng-odp] [PATCH CATERPILLAR v1 3/3] example: instrum: use low level API to retrieve performance counters X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bogdan Pricope Signed-off-by: Bogdan Pricope --- /** Email created from pull request 381 (bogdanPricope:cat_benchmark_poc) ** https://github.com/Linaro/odp/pull/381 ** Patch: https://github.com/Linaro/odp/pull/381.patch ** Base sha: 4d17f8ae64aba0e6f24877be30f86ae5880cef7e ** Merge commit sha: 85dce59a9248687c1522281ae7ac822405a71a20 **/ example/instrum/Makefile.am | 13 ++- example/instrum/drv.c | 39 +++++++ example/instrum/drv.h | 19 +++ example/instrum/init.c | 54 +++++++++ example/instrum/init.h | 19 +++ example/instrum/instrum.c | 17 +++ example/instrum/instrum_common.h | 34 ++++++ example/instrum/sched.c | 43 +++++++ example/instrum/sched.h | 19 +++ example/instrum/store.c | 244 +++++++++++++++++++++++++++++++++++++++ example/instrum/store.h | 36 ++++++ 11 files changed, 536 insertions(+), 1 deletion(-) create mode 100644 example/instrum/drv.c create mode 100644 example/instrum/drv.h create mode 100644 example/instrum/init.c create mode 100644 example/instrum/init.h create mode 100644 example/instrum/instrum_common.h create mode 100644 example/instrum/sched.c create mode 100644 example/instrum/sched.h create mode 100644 example/instrum/store.c create mode 100644 example/instrum/store.h diff --git a/example/instrum/Makefile.am b/example/instrum/Makefile.am index bf2a19c0d..741b970ed 100644 --- a/example/instrum/Makefile.am +++ b/example/instrum/Makefile.am @@ -16,5 +16,16 @@ AM_LDFLAGS = -L$(PAPI_PATH)/lib -lpapi lib_LTLIBRARIES = $(LIB)/libinstrum.la +noinst_HEADERS = \ + $(srcdir)/instrum_common.h \ + $(srcdir)/store.h \ + $(srcdir)/init.h \ + $(srcdir)/drv.h \ + $(srcdir)/sched.h + __LIB__libinstrum_la_SOURCES = \ - instrum.c + instrum.c \ + store.c \ + init.c \ + drv.c \ + sched.c diff --git a/example/instrum/drv.c b/example/instrum/drv.c new file mode 100644 index 000000000..9b100e3e8 --- /dev/null +++ b/example/instrum/drv.c @@ -0,0 +1,39 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include +#include +#include + +static int (*instr_odpdrv_print_all)(void); + +int instr_odpdrv_init(void) +{ + INSTR_FUNCTION(odpdrv_print_all); + + if (!instr_odpdrv_print_all) { + printf("odpdrv_print_all: Not Found\n"); + return -1; + } + + return 0; +} + +int odpdrv_print_all(void) +{ + int ret; + + STORE_SAMPLE_INIT; + + STORE_SAMPLE_START; + ret = (*instr_odpdrv_print_all)(); + STORE_SAMPLE_END; + + return ret; +} diff --git a/example/instrum/drv.h b/example/instrum/drv.h new file mode 100644 index 000000000..44cc6652f --- /dev/null +++ b/example/instrum/drv.h @@ -0,0 +1,19 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef __INSTRUM_DRV_H__ +#define __INSTRUM_DRV_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +int instr_odpdrv_init(void); + +#ifdef __cplusplus +} +#endif +#endif /* __INSTRUM_DRV_H__ */ diff --git a/example/instrum/init.c b/example/instrum/init.c new file mode 100644 index 000000000..a5b0e35dd --- /dev/null +++ b/example/instrum/init.c @@ -0,0 +1,54 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include +#include +#include + +static int (*instr_odp_init_local)(odp_instance_t instance, + odp_thread_type_t thr_type); + +static int (*instr_odp_term_local)(void); + +int instr_odpinit_init(void) +{ + INSTR_FUNCTION(odp_init_local); + + if (!instr_odp_init_local) { + printf("odp_init_local: Not Found\n"); + return -1; + } + + INSTR_FUNCTION(odp_term_local); + + if (!instr_odp_term_local) { + printf("odp_term_local: Not Found\n"); + return -1; + } + + return 0; +} + +int odp_init_local(odp_instance_t instance, odp_thread_type_t thr_type) +{ + int ret; + + ret = (*instr_odp_init_local)(instance, thr_type); + + instr_store_init_local(); + + return ret; +} + +int odp_term_local(void) +{ + instr_store_term_local(); + + return (*instr_odp_term_local)(); +} diff --git a/example/instrum/init.h b/example/instrum/init.h new file mode 100644 index 000000000..b92e9d4a3 --- /dev/null +++ b/example/instrum/init.h @@ -0,0 +1,19 @@ +/* Copyright (c) 2018, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef __INSTRUM_INIT_H__ +#define __INSTRUM_INIT_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +int instr_odpinit_init(void); + +#ifdef __cplusplus +} +#endif +#endif /*__INSTRUM_INIT_H__*/ diff --git a/example/instrum/instrum.c b/example/instrum/instrum.c index 88580c58f..71ab18159 100644 --- a/example/instrum/instrum.c +++ b/example/instrum/instrum.c @@ -6,13 +6,30 @@ #include #include +#include +#include +#include +#include static __attribute__((constructor)) void setup_wrappers(void) { printf("Setup Wrappers\n"); + + if (instr_store_init()) + return; + + if (instr_odpinit_init()) + return; + + if (instr_odpdrv_init()) + return; + + if (instr_odpsched_init()) + return; } static __attribute__((destructor)) void teardown_wrappers(void) { printf("Teardown Wrappers\n"); + instr_store_term(); } diff --git a/example/instrum/instrum_common.h b/example/instrum/instrum_common.h new file mode 100644 index 000000000..467314d83 --- /dev/null +++ b/example/instrum/instrum_common.h @@ -0,0 +1,34 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef __INSTRUM_COMMON_H__ +#define __INSTRUM_COMMON_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +#ifndef RTLD_NEXT +/*#define __GNU_SOURCE*/ +#define __USE_GNU +#endif + +#include +#include + +#define INSTR_FUNCTION(func) do { \ + instr_##func = dlsym(RTLD_NEXT, #func); \ + if (dlerror()) { \ + errno = EACCES; \ + instr_##func = NULL; \ + } \ + } while (0) + +#ifdef __cplusplus +} +#endif + +#endif /* __INSTRUM_COMMON_H__ */ diff --git a/example/instrum/sched.c b/example/instrum/sched.c new file mode 100644 index 000000000..a153be6e6 --- /dev/null +++ b/example/instrum/sched.c @@ -0,0 +1,43 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include +#include +#include + +static int (*instr_odp_schedule_multi)(odp_queue_t *from, + uint64_t wait, + odp_event_t events[], + int num); + +int instr_odpsched_init(void) +{ + INSTR_FUNCTION(odp_schedule_multi); + + if (!instr_odp_schedule_multi) { + printf("odp_schedule_multi: Not Found\n"); + return -1; + } + + return 0; +} + +int odp_schedule_multi(odp_queue_t *from, uint64_t wait, odp_event_t events[], + int num) +{ + int ret; + + STORE_SAMPLE_INIT; + + STORE_SAMPLE_START; + ret = (*instr_odp_schedule_multi)(from, wait, events, num); + STORE_SAMPLE_END; + + return ret; +} diff --git a/example/instrum/sched.h b/example/instrum/sched.h new file mode 100644 index 000000000..e3ccbb5b2 --- /dev/null +++ b/example/instrum/sched.h @@ -0,0 +1,19 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef __INSTRUM_SCHED_H__ +#define __INSTRUM_SCHED_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +int instr_odpsched_init(void); + +#ifdef __cplusplus +} +#endif +#endif /* __INSTRUM_SCHED_H__ */ diff --git a/example/instrum/store.c b/example/instrum/store.c new file mode 100644 index 000000000..750a18f81 --- /dev/null +++ b/example/instrum/store.c @@ -0,0 +1,244 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include +#include +#include "papi.h" + +#define PAPI_EVENT_CNT 2 +static int papi_event_tab[PAPI_EVENT_CNT] = {PAPI_BR_CN, PAPI_L2_DCM}; + +#define PROFILE_SAMPLE_CNT_MAX 50000 +#define PROFILE_SAMPLE_NAME_SIZE_MAX 20 + +typedef struct { + char name[PROFILE_SAMPLE_NAME_SIZE_MAX]; + long long timestamp_ns; + long long diff_cyc; + + long long counters[PAPI_EVENT_CNT]; +} profiling_sample_t; + +static __thread profiling_sample_t profile_sample_tab[PROFILE_SAMPLE_CNT_MAX]; +static __thread uint64_t profile_sample_idx; +static __thread uint64_t profile_sample_ovf; + +static __thread int event_set = PAPI_NULL; + +#define STORE_DIR_ENV "ODP_INSTRUM_STORE_DIR" +#define STORE_DIR_NAME_DFLT "/tmp" +#define STORE_DIR_NAME_SIZE_MAX 250 +#define STORE_FILE_NAME_SIZE_MAX 250 + +static char store_dir[STORE_DIR_NAME_SIZE_MAX]; + +static int papi_init(void) +{ + int retval, i; + + retval = PAPI_library_init(PAPI_VER_CURRENT); + if (retval != PAPI_VER_CURRENT) { + printf("PAPI Library initialization error!\n"); + return -1; + } + + retval = PAPI_thread_init((unsigned long(*)(void))(pthread_self)); + if (retval != PAPI_OK) { + printf("PAPI_thread_init error!\n"); + goto err_shutdown; + } + + if (PAPI_set_granularity(PAPI_GRN_THR) != PAPI_OK) { + printf("PAPI_set_granularity error!\n"); + goto err_shutdown; + } + + for (i = 0; i < PAPI_EVENT_CNT; i++) { + retval = PAPI_query_event(papi_event_tab[i]); + if (retval != PAPI_OK) { + printf("PAPI_query_event %d - error\n", i); + goto err_shutdown; + } + } + + return 0; + +err_shutdown: + PAPI_shutdown(); + + return -1; +} + +static void papi_term(void) +{ + PAPI_shutdown(); +} + +static int papi_init_local(void) +{ + int retval; + + retval = PAPI_register_thread(); + if (retval != PAPI_OK) { + printf("PAPI_register_thread failed - %d\n", retval); + return -1; + } + + /* Create LL event set */ + event_set = PAPI_NULL; + retval = PAPI_create_eventset(&event_set); + if (retval != PAPI_OK) { + printf("PAPI_create_eventset error: %d\n", retval); + return -1; + } + + retval = PAPI_add_events(event_set, papi_event_tab, PAPI_EVENT_CNT); + if (retval != PAPI_OK) { + printf("PAPI_add_events error: %d\n", retval); + goto err_clean_evset; + } + + retval = PAPI_start(event_set); + if (retval != PAPI_OK) { + printf("PAPI_start error: %d\n", retval); + goto err_clean_evset; + } + + return 0; + +err_clean_evset: + PAPI_cleanup_eventset(event_set); + PAPI_destroy_eventset(&event_set); + + return -1; +} + +static int papi_term_local(void) +{ + long long last_counters[PAPI_EVENT_CNT]; + + if (PAPI_stop(event_set, last_counters) == PAPI_OK) { + int i; + + for (i = 0; i < PAPI_EVENT_CNT; i++) + printf("Counter[%d] = %lld\n", i, last_counters[i]); + } + + PAPI_cleanup_eventset(event_set); + PAPI_destroy_eventset(&event_set); + + return 0; +} + +int instr_store_init(void) +{ + const char *store_dir_env = NULL; + + store_dir_env = getenv(STORE_DIR_ENV); + if (!store_dir_env) + store_dir_env = STORE_DIR_NAME_DFLT; + + strncpy(store_dir, store_dir_env, STORE_DIR_NAME_SIZE_MAX); + store_dir[STORE_DIR_NAME_SIZE_MAX - 1] = '\0'; + + if (papi_init()) + return -1; + + return 0; +} + +void instr_store_term(void) +{ + papi_term(); +} + +int instr_store_init_local(void) +{ + return papi_init_local(); +} + +int instr_store_term_local(void) +{ + return papi_term_local(); +} + +static void store_dump(void) +{ + FILE *f = NULL; + char file_name[STORE_DIR_NAME_SIZE_MAX + STORE_FILE_NAME_SIZE_MAX]; + char smpl[250], smpl_tmp[250]; + int i, j; + + sprintf(file_name, "%s/profile_%d_%ju.csv", + store_dir, odp_thread_id(), + profile_sample_ovf); + + f = fopen(file_name, "w"); + if (f == NULL) { + printf("Failed to create profiling file %s\n", file_name); + return; + } + + for (i = 0; i < PROFILE_SAMPLE_CNT_MAX; i++) { + sprintf(smpl, "%lld,%lld,%s", + profile_sample_tab[i].timestamp_ns, + profile_sample_tab[i].diff_cyc, + profile_sample_tab[i].name); + for (j = 0; j < PAPI_EVENT_CNT; j++) { + sprintf(smpl_tmp, ",%lld", + profile_sample_tab[i].counters[j]); + strcat(smpl, smpl_tmp); + } + fprintf(f, "%s\n", smpl); + } + + fclose(f); +} + +instr_profiling_sample_t store_sample_start(const char *func) +{ + profiling_sample_t *spl = NULL; + + spl = &profile_sample_tab[profile_sample_idx++]; + if (profile_sample_idx == PROFILE_SAMPLE_CNT_MAX) { + store_dump(); + profile_sample_idx = 0; + profile_sample_ovf++; + } + strncpy(spl->name, func, PROFILE_SAMPLE_NAME_SIZE_MAX); + spl->name[PROFILE_SAMPLE_NAME_SIZE_MAX - 1] = '\0'; + spl->timestamp_ns = PAPI_get_real_nsec(); + if (PAPI_read_ts(event_set, spl->counters, &spl->diff_cyc) != PAPI_OK) { + fprintf(stderr, "PAPI_read_counters - FAILED\n"); + spl->name[0] = 0; /* signal failed sample */ + } + + return spl; +} + +void store_sample_end(instr_profiling_sample_t _spl) +{ + profiling_sample_t *spl = _spl; + long long end_counters[PAPI_EVENT_CNT], end_cyc; + int i; + + if (!spl->name[0]) /* failed sample - on start */ + return; + + if (PAPI_read_ts(event_set, end_counters, &end_cyc) != PAPI_OK) { + fprintf(stderr, "PAPI_read_counters - FAILED\n"); + spl->name[0] = 0; /* signal failed sample*/ + return; + } + + for (i = 0; i < PAPI_EVENT_CNT; i++) + spl->counters[i] = end_counters[i] - spl->counters[i]; + + spl->diff_cyc = end_cyc - spl->diff_cyc; +} diff --git a/example/instrum/store.h b/example/instrum/store.h new file mode 100644 index 000000000..c7b1e4d7f --- /dev/null +++ b/example/instrum/store.h @@ -0,0 +1,36 @@ +/* Copyright (c) 2017, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef __INSTRUM_STORE_H__ +#define __INSTRUM_STORE_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +typedef void *instr_profiling_sample_t; + +#define STORE_SAMPLE_INIT \ + instr_profiling_sample_t _spl + +#define STORE_SAMPLE_START \ + (_spl = store_sample_start(__func__)) + +#define STORE_SAMPLE_END \ + store_sample_end(_spl) + +int instr_store_init(void); +void instr_store_term(void); +int instr_store_init_local(void); +int instr_store_term_local(void); + +instr_profiling_sample_t store_sample_start(const char *); +void store_sample_end(instr_profiling_sample_t); + +#ifdef __cplusplus +} +#endif +#endif /* __INSTRUM_STORE_H__ */