From patchwork Thu Apr 6 11:59:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Petri Savolainen X-Patchwork-Id: 96933 Delivered-To: patch@linaro.org Received: by 10.140.89.233 with SMTP id v96csp694710qgd; Thu, 6 Apr 2017 05:00:40 -0700 (PDT) X-Received: by 10.237.63.78 with SMTP id q14mr36611154qtf.266.1491480040828; Thu, 06 Apr 2017 05:00:40 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id q18si1200613qtg.203.2017.04.06.05.00.40; Thu, 06 Apr 2017 05:00:40 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 78EDF6437A; Thu, 6 Apr 2017 12:00:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id D782860734; Thu, 6 Apr 2017 11:59:53 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id BAB4F60A24; Thu, 6 Apr 2017 11:59:46 +0000 (UTC) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0123.outbound.protection.outlook.com [104.47.2.123]) by lists.linaro.org (Postfix) with ESMTPS id 18CD26072E for ; Thu, 6 Apr 2017 11:59:43 +0000 (UTC) Received: from DB4PR07CA042.eurprd07.prod.outlook.com (10.242.229.52) by VI1PR0701MB1902.eurprd07.prod.outlook.com (10.167.197.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.8; Thu, 6 Apr 2017 11:59:40 +0000 Received: from DB5EUR03FT051.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::201) by DB4PR07CA042.outlook.office365.com (2a01:111:e400:9828::52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.8 via Frontend Transport; Thu, 6 Apr 2017 11:59:40 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning linaro.org discourages use of 131.228.2.241 as permitted sender) Received: from mailrelay.int.nokia.com (131.228.2.241) by DB5EUR03FT051.mail.protection.outlook.com (10.152.21.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1005.5 via Frontend Transport; Thu, 6 Apr 2017 11:59:40 +0000 Received: from fihe3nok0735.emea.nsn-net.net (localhost [127.0.0.1]) by fihe3nok0735.emea.nsn-net.net (8.14.9/8.14.5) with ESMTP id v36Bx46u006028 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 6 Apr 2017 14:59:04 +0300 Received: from 10.144.19.15 ([10.144.104.219]) by fihe3nok0735.emea.nsn-net.net (8.14.9/8.14.5) with ESMTP id v36Bx4nK006023 (version=TLSv1/SSLv3 cipher=AES128-SHA256 bits=128 verify=NOT) for ; Thu, 6 Apr 2017 14:59:04 +0300 X-HPESVCS-Source-Ip: 10.144.104.219 From: Petri Savolainen To: Date: Thu, 6 Apr 2017 14:59:02 +0300 Message-ID: <1491479944-31232-1-git-send-email-petri.savolainen@linaro.org> X-Mailer: git-send-email 2.8.1 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:131.228.2.241; IPV:CAL; CTRY:FI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(39840400002)(39850400002)(39410400002)(39860400002)(39450400003)(39400400002)(2980300002)(199003)(189002)(9170700003)(22756006)(86362001)(110136004)(5660300001)(2351001)(5003940100001)(50986999)(47776003)(189998001)(53936002)(38730400002)(48376002)(36756003)(6916009)(50226002)(305945005)(106466001)(105596002)(8676002)(77096006)(2906002)(33646002)(81166006)(8936002)(356003); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR0701MB1902; H:mailrelay.int.nokia.com; FPR:; SPF:SoftFail; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; DB5EUR03FT051; 1:9aXxo/BoW/+HM0gDqJzPEzKkSUC2jc7sNOhENNVcCoZkfI4nc1vf6Fv3DzcKnZaDz7BewGQeZ5M12GQLMRUGgMjbsnKwlqJntdHKRX3YwqOMRpWnaC4zA3+MlQyDyjQ3Y8Kow48kjxy/YcDg+kx0Jc/w6UaTu0DUYyuemg4RvdBcKiJsYkYUhCQjcbRBq9Zb17WbK1nh94FAl7lLSc2bC+uPHK/KBhv68CeXeBcdrrhzGq5AIZ7VmuFJzDIlCMFoaU1pey+9Y+fc1KRmqhTrY74pTCQq3cNGIVBH6tDy77fv1DPCt+FlL34sdPn7Kp3wuI2N3LFJj43TFwLg4na2if5ySTwCqoRtN4LqWiLlKKAcw92hU6Q9FL+OFcLxe8KFUKGG51dsNl1ywqSC2U1xkg0Co5HzcTBKHv+oziaCjHdoFN9u0TCKFxBfN3Ku3G2nHISZq9bHv4rGQkQdUoD5+qssft6j7Z8K32gbkMx5CPhIPs3p8VZlOxLyf2WIerKPqxSQrKHn278nmWG787MNRg== MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 11ac36cb-441f-4218-50c8-08d47ce467ee X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075)(201703131423075)(201703031133081); SRVR:VI1PR0701MB1902; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 3:GPnjoj4kr1S+sxQZENQA6KdqBYdJbPJcXPgIBClwEOt5qLTKBzfCIxRrP3KkAKNvqVj8A3qcQndAndvN2qrJzinFplhlh6eP59ppQLPiseh1E56blwpZ0vIKiiMEtuX3mhN7C4y53WZ8b4kRrRcNa6l+Faffmtxbsav1+Gd0SI1H2N8e7u9rSK/wGmtdX2jf6AZ5shTBFw9yhDGFCzYg+DN6pX6DucV633baKNM0dg0WjuP/yjq1X5rsCb5aLIF6+PWV1TTW/MGSvHV/bEyChssbQjh0q95L800NNte247yi/PPZNlRKj3dw0lLTKBTgN31lpu3vgYVAvYwhTCd3ZnG4rO1XaMQAhS0Z3snkU/2IFJQfLrfQiAJvdsfqcuv0Mb7qw+x6ILJgBgRiD4eeQyQX0dIN21f0ei1n+OIRPwHOAANmoF3+0cPkbkWTjwpMUC2OwNfxbexyJ9LMdK8/xg== X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 25:pZz+eOAjnckgaBuAKcp8OpDx+SAXtJC8ahX3reJNxFpyOiRCK5k8k3oMbPv2/gneavkgqSrumCjDNF1l+slOmVDGyP06D/AteJ1eH6pBqejv/X0sgeNNzIiQ+ranc55yS/nCqIlxSGabC2GHoVY32K0aNHLfn0bjJObkUOR41VGxAylNXDo6ZUp5RfDQSZOdQWrVGf/kf4W75Zqt5WYuHLIM1M6JsxGt3ahfWtOw+E70JdhVXfXEK+DCQBUp26DzI8oYDHRKDpBSG/Ny1MmIfus+UUNPZk3ms+enabgHRmIlPAYpW4iH0udlfPm2hwCzRXKCmamDvUjmLJNbtEmvNT4Fm5AyRHZl2VzYScAR4X8t5Ra4P7oHnJjySax+Qy5BvZbQQ2+r7ASoKXN5mv5Sd9Tb04VexAtkjd/bICokoqUoWDb8nAkNf5sfMfX0EzHj1JffP3/ocyzij/OE7r9nQw==; 31:9gUilqDbosSxCzKVBSV1CyaWReGuthvzPslmuA8J0fsVVN7Mof4slC21+go1blqC+U2J6YePv/kag6zyWpsAFPVcNfcvZ2mMa/UihHm+sx+urVI6FH07PuBQSEYkBFYedfK/h0TuBl0Q79Fb5mV8lGKoKW7nPmjSIl67LazRxegYDizxPGg40hSzXIXpTQh4/pMz+BSSRy9yvmh23ThiQpkD4754MBQG1RuCPIuf3SpE6q3vl88ZjDjbBI+DqLvdwQc+1LpUuMzDHODyDpNNqQ== X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 20:5XsDyHOIksIFgg2c2qQAvRnhztEtsuWklR8bsJvNaGYy4KYi8DMcXTQ170ixHTTeL7FHN2FE4KkvjCgMYQ9w+yFGOa5B4YlGQV6ZO8xx/aoU0APXmOcq9/T4+OpsHqlWMPtLMC56IEb4sJNmsrEKvHN0UZkjEzb10sohQRlInuPYjRJbrtej0b2np6rqaAqZnuCHJIH+yYdRVe/pG1TfDMwj83vL1HL69lirhKw5LC2Fzluwmhpr947v4fTtZOu0JhsTcm8AZP1ca2q/ynIEPmJbluZUVxPv4TiOoOMVa+d/lXNzN8YWPCBQ1VZB4pzMSaMQv+QS0tgwzGPUt0txI7gxZYNmlxTa0lp/NtmdiNm9brTIKgYy7aVXXs48cVpShezcwjMRQwXrd1zstQpqX9aP537tOmsQLZiGasBbUJIj7wIyS49y5uSvlAsO3o2i5o7BDYtmTfbjY8oCj2UTe/8T/EdnV93uh8T6v4kUhAaJ/0dBvusYrFmpM6Hbyp25dz0ARq0IueNrsTDDakKtIAKIr2y/aKDb+FglkoUll1OODLFumTj/H5MAtoEkKcrN7BoSC454uP7EAJPWpK1AwLbHRQKSM8cGwo+nF+8tXMk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(13013025)(13021025)(5005006)(13023025)(8121501046)(3002001)(10201501046)(93006095)(93003095)(6055026)(6041248)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(20161123562025)(20161123555025)(6072148); SRVR:VI1PR0701MB1902; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0701MB1902; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 4:qVkArhje16OJExaZVZ87s6rIUsrNv/QGWcjOMY1p7QDj/SxmfnChEGCftYkOgNyjSXAThyDVkrKC0CAzex1FaXZiyUCM4fxxvZyiJkmj49Se9PDd0/vgh+G5RiZfOJlKj30aUkGqfkT5GHLaygTzr3XELUiuNQs93BXKrSWKAYYgZITKo10rDRp06JHTiLWJCPQlzXcmpk4MHiksf/ZZcysUs+rdOg9l8J2ylfqKb3YIJN3iL7w2xVQME3R6eBs0cjvwlk/+qZai6If2VkL7hn+oif5hHXq8E8NewkRBONQ44Kv9dJUrpAVuEmlyJ6asH0x7ygtkq3Lr6Q3kDyVtXncWt9JXFzIYA2eFn11JTxTBQ2FfZIVFD+RV/YSqUjCby6mcOQy4AQFOi+lFAkTa+L/HBuF/uCL46YC88e9+y/v/zjRKDPEiwU0lNig5e63EihsJXGHAnqgMrzWqwLqjpr2OzrMG9DtNxFI9rapMPNSKDGW/sEqROePvyIDkoo3Nc/JMEMBA4zuVTATguhzN4PCL4s9waxrcSwHrCrwuRadn2KX4NWvIV9t2Oha5yC7F+xbvGG2qIUFzHfEztc/3h1WsmCGimMgE/Kw2T+xy03Rgr0o7sLJglbE781yatNFxtv6X3GjccOBnMZeGeHFiWjr6sj5OSC1HTOAo1Nva5GsSQ5rqEaIfwR6UhPRNDWD8ZQh/pb9eeU1JaXO9szY10r8HgKk/vFZB4v0vl/sR9wSYe1V0/aYBS4nNuXrRsE12M3u2sH0etNUnKyCFI3RLTRpM29GZu0OinHfBySUORf0= X-Forefront-PRVS: 02698DF457 X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 23:pfdTJYaQiE4k/4ULNaf9wXv2dAy9PPlspMkLklWBwavAp0i9nN83zRCrz3uciO3YCPVJPFHq41hO23A4IYY/PxXBsvN49IMmAnzkTEX7TFwWqXkzKtXP+DQ1en+RnmcTuiAWd//PFWZWVTSsUczFi7JtceHNvIZuWtjMd+dDog9BSQUCMF6wnE97qxdRBvgEVIAoRQGvjaogIdBPvB6uPf4Oc4Tfg/rG6SyfC28x/w30RysczZdW0Aet4G9yFkbbO901kBvWMNCn1lm2/cNUzBpSJ4Ki+6zwrkhcvYbbiLe2wikzZVUxbWOdNd+SgaKfe68N9M9hh0DozrkhDTq0LdbNVnoxbz5pFGg3VPcspUwAMMov1qRxGlvVo78/ilB0+CnusALyeajKYSZvL4X/e3zEbn4G3XlwIDxIrzrituTKyYV52xmNXUGkm2Dnja0b4iIqlVO9CUG7LGqvUf3NoAl/UP1m3j9igpIVlxAEtwz3AWQrW2kAJwUCELOD/ad5FgJl+MSG3yqZbQ08Aqf+l4AJO6NJ5D8MHCU4OK9Ki9hcJacjo79USXKjeDwrL+bbAl/IkwMYTFYrZPx+J5BVI44QJqXUFwVQU5vAozuX/S5uXwej9mEPpKxkVa93lkUE1wgXoUqJbu2V3gvooLEaw2/uTw0ZyietTUjBvkBgB2SLEZVLTJbDQDVgMGZM+7eCmUpXhmkF0zOSYiNHl1/Z2hWds54XC7pVGRaWrVyO58Ar+2xeV+WXkYOjV3SYjlOJPdoWhwJIrpTsfOecBBIt4oXpgCWcEoDeqWFPkh9RbSfIB7nOL+M7Ka27E0ZQ0EtOLiH5/ywtvjr/y0aOL/P04I7CGzHq2VRysX8wSLVMpizwNbASPOCM0Yv1GoU3Eu5AEWwJcSvcz3MRkD10gBFVGIcL2ju0xQUKB9/zOEHlFWQ9I9z9agWFtf2GL7F2hH4q X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 6:vebPZ7OUgRBH0KmML53Q0oLbtq5LD7Z4YXhbXTDw+Qm05w3f5jXZhic7yp7Am+UptEq6CzqS3n1JIqn+8udhEyBVUnl7ZBRPrrIgF2qJ2v3+Qpm/QdwOCzF2eJQAQ0lJFgs9IBO9RtRF+6KSauzHP5gnO/eYWmWbC0ZJ8jxn7BlR7SvplqMQaHnhRSHNyZQL5L5pr+6MOzvzf/BE41vYT6UXrdXcRCoVmTddlrShe5J0tlup4YlWALDkBFbGOanPyn7fpBTCGC24q2tITIRz02iqMmgBZs3iboVmytj+gOOTvDCm02CjBp+cYIm9doRTFr0vJAEUoG5A+3aoy9JpwFkgM0nXuqX1Ha2L+1HoxqbSkW8SQ2tB2hXqbEwKzMZgiLt8mGJjQg7R1M4ZhHICtDq8SP9rDLAzZ60XNXUNdc5BFaQx7t39D+jDrwFnbBtSIopnClZnIA6xmKOnlwIx3XYw9mScg1ATZBjrbMxxLME=; 5:kTA9gDUXfhLrHdoKFO0ZHUOCUJp6znPQYDZ6SWGsBzWo1IAFAIwLn/3Xn9stKbnmc2CRYHhO4g9Lm8/F/COJmQh2lRLWW9+U4iGJFLlnD+KyENQM6dFEyxDPpwj3o8z2Gb7Wj7yrnZyI/ZfnevImvg==; 24:qVQuMb0iEYOJMQmUK6mGu+YTkFK2s63FXIk25lXR+/1yXC/WwvGjRmGFreQwlO5mh4H/r5Uvdkkvd6x51DL5932iU9G6pl8Evrnch2P15VQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0701MB1902; 7:GKHwvqbVFbqYulhtBaUZPJI3ODqCJHJeL5EPm4lS1K+hmmUDj3TAqQo51eYcbMHIZAPqRZU9DPmZ2ZbKpz8pD5UgpzRmwqwUgmecueMs7rrzcGlNCoxaAsTYnURwQ4BWPiTxx9Xu5NT3nkNMwN1KsBXESzuEJH4CiK1ayDENtNFMj/xdlcczMMH2pPFXL6lN7JILOcsclSrCDWfoZlRvYovDWSbm1mHRXQ/ooTBzEoJfektIybDLfBZYXpOTCQ498B3N/u3X3C9hDW1d7norlRQY7/M32JLJgIOuQ0wUS2YJujaXFebVjJVtjVIG0apzCluJZekQdrCJ32Sm6PPkYw== X-OriginatorOrg: nokia.onmicrosoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2017 11:59:40.2266 (UTC) X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5d471751-9675-428d-917b-70f44f9630b0; Ip=[131.228.2.241]; Helo=[mailrelay.int.nokia.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0701MB1902 Subject: [lng-odp] [PATCH 1/3] test: l2fwd: add group option X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" User may give number of scheduling groups to test scheduler performance with other that the default (all threads) group. Both pktios and threads are allocated into these groups with round robin. The number of groups may not exceed number of pktios or worker threads. Signed-off-by: Petri Savolainen --- test/common_plat/performance/odp_l2fwd.c | 148 ++++++++++++++++++++++++------- 1 file changed, 116 insertions(+), 32 deletions(-) -- 2.8.1 diff --git a/test/common_plat/performance/odp_l2fwd.c b/test/common_plat/performance/odp_l2fwd.c index 8f5c5e1..33efc02 100644 --- a/test/common_plat/performance/odp_l2fwd.c +++ b/test/common_plat/performance/odp_l2fwd.c @@ -104,6 +104,7 @@ typedef struct { int src_change; /**< Change source eth addresses */ int error_check; /**< Check packet errors */ int sched_mode; /**< Scheduler mode */ + int num_groups; /**< Number of scheduling groups */ } appl_args_t; static int exit_threads; /**< Break workers loop if set to 1 */ @@ -130,6 +131,7 @@ typedef union { typedef struct thread_args_t { int thr_idx; int num_pktio; + int num_groups; struct { odp_pktin_queue_t pktin; @@ -142,7 +144,12 @@ typedef struct thread_args_t { int tx_queue_idx; } pktio[MAX_PKTIOS]; - stats_t *stats; /**< Pointer to per thread stats */ + /* Groups to join */ + odp_schedule_group_t group[MAX_PKTIOS]; + + /* Pointer to per thread stats */ + stats_t *stats; + } thread_args_t; /** @@ -297,6 +304,22 @@ static int run_worker_sched_mode(void *arg) thr = odp_thread_id(); + if (gbl_args->appl.num_groups) { + odp_thrmask_t mask; + + odp_thrmask_zero(&mask); + odp_thrmask_set(&mask, thr); + + /* Join non-default groups */ + for (i = 0; i < thr_args->num_groups; i++) { + if (odp_schedule_group_join(thr_args->group[i], + &mask)) { + LOG_ERR("Join failed\n"); + return -1; + } + } + } + num_pktio = thr_args->num_pktio; if (num_pktio > MAX_PKTIOS) { @@ -590,7 +613,7 @@ static int run_worker_direct_mode(void *arg) * @retval -1 on failure */ static int create_pktio(const char *dev, int idx, int num_rx, int num_tx, - odp_pool_t pool) + odp_pool_t pool, odp_schedule_group_t group) { odp_pktio_t pktio; odp_pktio_param_t pktio_param; @@ -650,7 +673,7 @@ static int create_pktio(const char *dev, int idx, int num_rx, int num_tx, pktin_param.queue_param.sched.prio = ODP_SCHED_PRIO_DEFAULT; pktin_param.queue_param.sched.sync = sync_mode; - pktin_param.queue_param.sched.group = ODP_SCHED_GROUP_ALL; + pktin_param.queue_param.sched.group = group; } if (num_rx > (int)capa.max_input_queues) { @@ -1016,39 +1039,46 @@ static void usage(char *progname) printf("\n" "OpenDataPlane L2 forwarding application.\n" "\n" - "Usage: %s OPTIONS\n" + "Usage: %s [options]\n" + "\n" " E.g. %s -i eth0,eth1,eth2,eth3 -m 0 -t 1\n" - " In the above example,\n" - " eth0 will send pkts to eth1 and vice versa\n" - " eth2 will send pkts to eth3 and vice versa\n" + " In the above example,\n" + " eth0 will send pkts to eth1 and vice versa\n" + " eth2 will send pkts to eth3 and vice versa\n" "\n" "Mandatory OPTIONS:\n" - " -i, --interface Eth interfaces (comma-separated, no spaces)\n" - " Interface count min 1, max %i\n" + " -i, --interface Eth interfaces (comma-separated, no spaces)\n" + " Interface count min 1, max %i\n" "\n" "Optional OPTIONS:\n" - " -m, --mode Packet input mode\n" - " 0: Direct mode: PKTIN_MODE_DIRECT (default)\n" - " 1: Scheduler mode with parallel queues: PKTIN_MODE_SCHED + SCHED_SYNC_PARALLEL\n" - " 2: Scheduler mode with atomic queues: PKTIN_MODE_SCHED + SCHED_SYNC_ATOMIC\n" - " 3: Scheduler mode with ordered queues: PKTIN_MODE_SCHED + SCHED_SYNC_ORDERED\n" - " 4: Plain queue mode: ODP_PKTIN_MODE_QUEUE\n" - " -o, --out_mode Packet output mode\n" - " 0: Direct mode: PKTOUT_MODE_DIRECT (default)\n" - " 1: Queue mode: PKTOUT_MODE_QUEUE\n" - " -c, --count CPU count.\n" - " -t, --time Time in seconds to run.\n" - " -a, --accuracy Time in seconds get print statistics\n" + " -m, --mode Packet input mode\n" + " 0: Direct mode: PKTIN_MODE_DIRECT (default)\n" + " 1: Scheduler mode with parallel queues:\n" + " PKTIN_MODE_SCHED + SCHED_SYNC_PARALLEL\n" + " 2: Scheduler mode with atomic queues:\n" + " PKTIN_MODE_SCHED + SCHED_SYNC_ATOMIC\n" + " 3: Scheduler mode with ordered queues:\n" + " PKTIN_MODE_SCHED + SCHED_SYNC_ORDERED\n" + " 4: Plain queue mode: PKTIN_MODE_QUEUE\n" + " -o, --out_mode Packet output mode\n" + " 0: Direct mode: PKTOUT_MODE_DIRECT (default)\n" + " 1: Queue mode: PKTOUT_MODE_QUEUE\n" + " -c, --count CPU count.\n" + " -t, --time Time in seconds to run.\n" + " -a, --accuracy Time in seconds get print statistics\n" " (default is 1 second).\n" - " -d, --dst_change 0: Don't change packets' dst eth addresses\n" - " 1: Change packets' dst eth addresses (default)\n" - " -s, --src_change 0: Don't change packets' src eth addresses\n" - " 1: Change packets' src eth addresses (default)\n" - " -r, --dst_addr Destination addresses (comma-separated, no spaces)\n" - " Requires also the -d flag to be set\n" - " -e, --error_check 0: Don't check packet errors (default)\n" - " 1: Check packet errors\n" - " -h, --help Display help and exit.\n\n" + " -d, --dst_change 0: Don't change packets' dst eth addresses\n" + " 1: Change packets' dst eth addresses (default)\n" + " -s, --src_change 0: Don't change packets' src eth addresses\n" + " 1: Change packets' src eth addresses (default)\n" + " -r, --dst_addr Destination addresses (comma-separated, no spaces)\n" + " Requires also the -d flag to be set\n" + " -e, --error_check 0: Don't check packet errors (default)\n" + " 1: Check packet errors\n" + " -g, --groups Number of groups to use: 0 ... num\n" + " 0: SCHED_GROUP_ALL (default)\n" + " num: must not exceed number of interfaces or workers\n" + " -h, --help Display help and exit.\n\n" "\n", NO_PATH(progname), NO_PATH(progname), MAX_PKTIOS ); } @@ -1079,11 +1109,12 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) {"dst_change", required_argument, NULL, 'd'}, {"src_change", required_argument, NULL, 's'}, {"error_check", required_argument, NULL, 'e'}, + {"groups", required_argument, NULL, 'g'}, {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0} }; - static const char *shortopts = "+c:+t:+a:i:m:o:r:d:s:e:h"; + static const char *shortopts = "+c:+t:+a:i:m:o:r:d:s:e:g:h"; /* let helper collect its own arguments (e.g. --odph_proc) */ odph_parse_options(argc, argv, shortopts, longopts); @@ -1092,6 +1123,7 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) appl_args->accuracy = 1; /* get and print pps stats second */ appl_args->dst_change = 1; /* change eth dst address by default */ appl_args->src_change = 1; /* change eth src address by default */ + appl_args->num_groups = 0; /* use default group */ appl_args->error_check = 0; /* don't check packet errors by default */ opterr = 0; /* do not issue errors on helper options */ @@ -1217,6 +1249,9 @@ static void parse_args(int argc, char *argv[], appl_args_t *appl_args) case 'e': appl_args->error_check = atoi(optarg); break; + case 'g': + appl_args->num_groups = atoi(optarg); + break; case 'h': usage(argv[0]); exit(EXIT_SUCCESS); @@ -1305,6 +1340,24 @@ static void gbl_args_init(args_t *args) } } +static void create_groups(int num, odp_schedule_group_t *group) +{ + int i; + odp_thrmask_t zero; + + odp_thrmask_zero(&zero); + + /* Create groups */ + for (i = 0; i < num; i++) { + group[i] = odp_schedule_group_create(NULL, &zero); + + if (group[i] == ODP_SCHED_GROUP_INVALID) { + LOG_ERR("Group create failed\n"); + exit(EXIT_FAILURE); + } + } +} + /** * ODP L2 forwarding main function */ @@ -1325,6 +1378,8 @@ int main(int argc, char *argv[]) int if_count; int (*thr_run_func)(void *); odp_instance_t instance; + int num_groups; + odp_schedule_group_t group[MAX_PKTIOS]; /* Init ODP before calling anything else */ if (odp_init_global(&instance, NULL, NULL)) { @@ -1374,10 +1429,23 @@ int main(int argc, char *argv[]) if_count = gbl_args->appl.if_count; + num_groups = gbl_args->appl.num_groups; + printf("num worker threads: %i\n", num_workers); printf("first CPU: %i\n", odp_cpumask_first(&cpumask)); printf("cpu mask: %s\n", cpumaskstr); + if (num_groups) + printf("num groups: %i\n", num_groups); + + printf("\n"); + + if (num_groups > if_count || num_groups > num_workers) { + LOG_ERR("Too many groups. Number of groups may not exceed " + "number of interfaces or workers.\n"); + exit(EXIT_FAILURE); + } + /* Create packet pool */ odp_pool_param_init(¶ms); params.pkt.seg_len = SHM_PKT_POOL_BUF_SIZE; @@ -1399,9 +1467,18 @@ int main(int argc, char *argv[]) bind_workers(); + /* Default */ + if (num_groups == 0) { + group[0] = ODP_SCHED_GROUP_ALL; + num_groups = 1; + } else { + create_groups(num_groups, group); + } + for (i = 0; i < if_count; ++i) { const char *dev = gbl_args->appl.if_names[i]; int num_rx, num_tx; + odp_schedule_group_t grp; /* A queue per worker in scheduled mode */ num_rx = num_workers; @@ -1413,7 +1490,10 @@ int main(int argc, char *argv[]) num_tx = gbl_args->pktios[i].num_tx_thr; } - if (create_pktio(dev, i, num_rx, num_tx, pool)) + /* Round robin pktios to groups */ + grp = group[i % num_groups]; + + if (create_pktio(dev, i, num_rx, num_tx, pool, grp)) exit(EXIT_FAILURE); /* Save interface ethernet address */ @@ -1473,6 +1553,10 @@ int main(int argc, char *argv[]) thr_params.thr_type = ODP_THREAD_WORKER; thr_params.instance = instance; + /* Round robin threads to groups */ + gbl_args->thread[i].num_groups = 1; + gbl_args->thread[i].group[0] = group[i % num_groups]; + gbl_args->thread[i].stats = &stats[i]; odp_cpumask_zero(&thd_mask);