From patchwork Wed Jan 24 13:00:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 765846 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D69277F3D; Wed, 24 Jan 2024 13:00:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101246; cv=none; b=CLnOD9gaO7lbPKH8qQz3XD0hj6fcOdO8+k9loKDQP5dlLCcn6l2WqfsonMDRRg/wEU0BZX2dpYWs5dxOgS71rk1lGa+1sRIS6lZFPyRheIfaQMeEBEJfkz1ruyJIjU8/9JcpnyQ33dWxN7Id3HsDIACsRVjWwNIt4l52afgV5s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101246; c=relaxed/simple; bh=9o+HDIgpROySfFcjDsYJORGTrxcfESNCkeqXVhB1kMs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BMXvV/A+egqVEBt53XHo+BzabDAX9hSPKntVFQ4Sr4x8A8WC9y6pVJIlbUWZ2NwZGPYxgvSi15gRYz4EIHpl2Qhiv+cv4VxMBjwX/qfKuT4KRXs12VvLUl26Ja6YWQebqCPtWDFHQNdHWeIUf8tEMQ2STBDdm/nzfci6r5vz8Tc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JjsMtApZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JjsMtApZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5CC3FC43399; Wed, 24 Jan 2024 13:00:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706101246; bh=9o+HDIgpROySfFcjDsYJORGTrxcfESNCkeqXVhB1kMs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JjsMtApZ/LfOVy3w3MvsI0ylp+32K7Z9s8Vkc0JJNWOkxc0OdVfJG+G1nMTCgcIU6 qH4NGdLvCcdAjcH7RNNCWObmclTDUqMABp1q1R9CYBO/zsuavpNM/Yo800ECkgySdf dojdasyJtkpj45kunj9bPJ2e2A7raiaOHGNzNc2xBfePLG3m3CHmSU8zbUPVBJcR1J N/ew4UHt9HaTt6HdzaWI2BLMd8WeADHJE37X+aEGDAuKQKgyyzn1QxcEQ21OdcQb+1 zhoiSUD71lD4ilAOhdkGaOPR+91+YCPmo7rN6tNNNcMD0/8TSJdINVIZrsmaDBaEsP cvh+VWJpWGNwQ== From: Mark Brown Date: Wed, 24 Jan 2024 13:00:18 +0000 Subject: [PATCH v4 1/2] kselftest/seccomp: Use kselftest output functions for benchmark Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-1-cfd2bd2a31cf@kernel.org> References: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-0-cfd2bd2a31cf@kernel.org> In-Reply-To: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-0-cfd2bd2a31cf@kernel.org> To: Kees Cook , Andy Lutomirski , Will Drewry , Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Brown , Anders Roxell X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=6396; i=broonie@kernel.org; h=from:subject:message-id; bh=9o+HDIgpROySfFcjDsYJORGTrxcfESNCkeqXVhB1kMs=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlsQn49b2Zza7GkhVKcqYmrZhTblZq2V1s361lWZBk bJdChpeJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZbEJ+AAKCRAk1otyXVSH0Bf/B/ 4mqoLI4E/yGtZzKbIF6y/mexAgLxBG2XDnuMvc+Z57gybUe3ckEzXd6U2gc0wA4hMnDPztZ5d1OCD5 Z2iAjJJqHeiOG1pW5vrjwf7OeUsF/b3Ni+rFlSdMg3h/EeyQa4SdcRKzStj3HfcNie6qOQnyiduY2p DG8+jllccSAsBg9cxG5bSYcedIfQc363x/nOuiU/CWMs60XIoBQv0OXOFI8l711cnA84KZCN2enxu+ B0jPLww0QhNKbgKfZ9+bnrdYokCseAp1JTLGkABdAasEG8xUhD/9cuPf8RiY5gsvnLEdItgFvjBpIl oiVS5f0pNk3MTz8BmStsyUsauHtBDz X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In preparation for trying to output the test results themselves in TAP format rework all the prints in the benchmark to use the kselftest output functions. The uses of system() all produce single line output so we can avoid having to deal with fully managing the child process and continue to use system() by simply printing an empty message before we invoke system(). We also leave one printf() used to complete a line of output in place. Tested-by: Anders Roxell Acked-by: Kees Cook Signed-off-by: Mark Brown --- .../testing/selftests/seccomp/seccomp_benchmark.c | 45 ++++++++++++---------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/tools/testing/selftests/seccomp/seccomp_benchmark.c b/tools/testing/selftests/seccomp/seccomp_benchmark.c index 5b5c9d558dee..93168dd2c1e3 100644 --- a/tools/testing/selftests/seccomp/seccomp_benchmark.c +++ b/tools/testing/selftests/seccomp/seccomp_benchmark.c @@ -38,10 +38,10 @@ unsigned long long timing(clockid_t clk_id, unsigned long long samples) i *= 1000000000ULL; i += finish.tv_nsec - start.tv_nsec; - printf("%lu.%09lu - %lu.%09lu = %llu (%.1fs)\n", - finish.tv_sec, finish.tv_nsec, - start.tv_sec, start.tv_nsec, - i, (double)i / 1000000000.0); + ksft_print_msg("%lu.%09lu - %lu.%09lu = %llu (%.1fs)\n", + finish.tv_sec, finish.tv_nsec, + start.tv_sec, start.tv_nsec, + i, (double)i / 1000000000.0); return i; } @@ -53,7 +53,7 @@ unsigned long long calibrate(void) pid_t pid, ret; int seconds = 15; - printf("Calibrating sample size for %d seconds worth of syscalls ...\n", seconds); + ksft_print_msg("Calibrating sample size for %d seconds worth of syscalls ...\n", seconds); samples = 0; pid = getpid(); @@ -102,14 +102,14 @@ long compare(const char *name_one, const char *name_eval, const char *name_two, { bool good; - printf("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, - (long long)one, name_eval, (long long)two); + ksft_print_msg("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, + (long long)one, name_eval, (long long)two); if (one > INT_MAX) { - printf("Miscalculation! Measurement went negative: %lld\n", (long long)one); + ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)one); return 1; } if (two > INT_MAX) { - printf("Miscalculation! Measurement went negative: %lld\n", (long long)two); + ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)two); return 1; } @@ -145,12 +145,15 @@ int main(int argc, char *argv[]) setbuf(stdout, NULL); - printf("Running on:\n"); + ksft_print_msg("Running on:\n"); + ksft_print_msg(""); system("uname -a"); - printf("Current BPF sysctl settings:\n"); + ksft_print_msg("Current BPF sysctl settings:\n"); /* Avoid using "sysctl" which may not be installed. */ + ksft_print_msg(""); system("grep -H . /proc/sys/net/core/bpf_jit_enable"); + ksft_print_msg(""); system("grep -H . /proc/sys/net/core/bpf_jit_harden"); if (argc > 1) @@ -158,11 +161,11 @@ int main(int argc, char *argv[]) else samples = calibrate(); - printf("Benchmarking %llu syscalls...\n", samples); + ksft_print_msg("Benchmarking %llu syscalls...\n", samples); /* Native call */ native = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid native: %llu ns\n", native); + ksft_print_msg("getpid native: %llu ns\n", native); ret = prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0); assert(ret == 0); @@ -172,33 +175,33 @@ int main(int argc, char *argv[]) assert(ret == 0); bitmap1 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 1 filter (bitmap): %llu ns\n", bitmap1); + ksft_print_msg("getpid RET_ALLOW 1 filter (bitmap): %llu ns\n", bitmap1); /* Second filter resulting in a bitmap */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &bitmap_prog); assert(ret == 0); bitmap2 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 2 filters (bitmap): %llu ns\n", bitmap2); + ksft_print_msg("getpid RET_ALLOW 2 filters (bitmap): %llu ns\n", bitmap2); /* Third filter, can no longer be converted to bitmap */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog); assert(ret == 0); filter1 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 3 filters (full): %llu ns\n", filter1); + ksft_print_msg("getpid RET_ALLOW 3 filters (full): %llu ns\n", filter1); /* Fourth filter, can not be converted to bitmap because of filter 3 */ ret = prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &bitmap_prog); assert(ret == 0); filter2 = timing(CLOCK_PROCESS_CPUTIME_ID, samples) / samples; - printf("getpid RET_ALLOW 4 filters (full): %llu ns\n", filter2); + ksft_print_msg("getpid RET_ALLOW 4 filters (full): %llu ns\n", filter2); /* Estimations */ #define ESTIMATE(fmt, var, what) do { \ var = (what); \ - printf("Estimated " fmt ": %llu ns\n", var); \ + ksft_print_msg("Estimated " fmt ": %llu ns\n", var); \ if (var > INT_MAX) \ goto more_samples; \ } while (0) @@ -218,7 +221,7 @@ int main(int argc, char *argv[]) ESTIMATE("seccomp per-filter overhead (filters / 4)", per_filter2, (filter2 - native - entry) / 4); - printf("Expectations:\n"); + ksft_print_msg("Expectations:\n"); ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1); bits = compare("native", "≤", "1 filter", native, le, filter1); if (bits) @@ -230,7 +233,7 @@ int main(int argc, char *argv[]) bits = compare("1 bitmapped", "≈", "2 bitmapped", bitmap1 - native, approx, bitmap2 - native); if (bits) { - printf("Skipping constant action bitmap expectations: they appear unsupported.\n"); + ksft_print_msg("Skipping constant action bitmap expectations: they appear unsupported.\n"); goto out; } @@ -242,7 +245,7 @@ int main(int argc, char *argv[]) goto out; more_samples: - printf("Saw unexpected benchmark result. Try running again with more samples?\n"); + ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); out: return 0; } From patchwork Wed Jan 24 13:00:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 767883 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CBE97C0BD; Wed, 24 Jan 2024 13:00:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101248; cv=none; b=POke17UySNcdY0IKjnmJow0Bw7sysRJ27J4on1p5HUHz7YkIDhETVssc3tpz6X9wspABfBem6SYOeeIkTSG6MPF2GuPyzYTAh658UWhF2MZvIEymik3mxwpa+iiHyFtGO75NCF+EFt3TLwtJ1MApE3bJImhegbonBvF0WKKhJnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706101248; c=relaxed/simple; bh=X1GX0ByCnXN+q4cLRykG/1QyOWnsNOpR6oiWLUfDTno=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KVx/dNXsz4YugseC4SW3cBVuCZz7OSUXC+uHloNL4PUNxm8nuutyQaRLdBCBaduLHc/zX3o2RtEKAV5SBLvP5nyuds7EqFm9+H5rO+nyuMJkWmYbuiajEVum6USZBjWPLdCjGZW0kFmlx1tQ5WtAeZ2cto0Cq5V3b3tM2uJuYvk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FDIKaeqW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FDIKaeqW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 67A37C433B1; Wed, 24 Jan 2024 13:00:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706101248; bh=X1GX0ByCnXN+q4cLRykG/1QyOWnsNOpR6oiWLUfDTno=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FDIKaeqW33O8OwynqwvSImtN9obGZOJ75h5y4zwyq5I+90UbBcGrQc9akxfYPyCAh W6s/DIcw0Capd1k1mTIoLdlsrzkaMUOn+Cj/03DitpVKHdhk0Z3RDjxjduu4wJwjU5 VrzX45W9f6ygz/Aon83hyNVHpGgJeC8W3NPqn5CEH3QUYjUz5NFXZcCOdiRxQlPD5B kAuH4XpXHrP/ipMOl8B35i4Ohv3FZYgvw4/w4BUcxCzDG/RN+8kX1EEP4vv6hWXCBi A7fEPYJDON7Jtr1mng6+Au89llWkoRJX7u5Yec4dhaJm0IC6SMw7fpgKnxHXAyKJ4a mFXXzlLwWu7DQ== From: Mark Brown Date: Wed, 24 Jan 2024 13:00:19 +0000 Subject: [PATCH v4 2/2] kselftest/seccomp: Report each expectation we assert as a KTAP test Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-2-cfd2bd2a31cf@kernel.org> References: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-0-cfd2bd2a31cf@kernel.org> In-Reply-To: <20240124-b4-kselftest-seccomp-benchmark-ktap-v4-0-cfd2bd2a31cf@kernel.org> To: Kees Cook , Andy Lutomirski , Will Drewry , Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Mark Brown , Anders Roxell X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=5108; i=broonie@kernel.org; h=from:subject:message-id; bh=X1GX0ByCnXN+q4cLRykG/1QyOWnsNOpR6oiWLUfDTno=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlsQn5N93xH8lGYJR9CI+DPLMhROPr1XgTTQV5xqOg HhPSx7yJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZbEJ+QAKCRAk1otyXVSH0G8xB/ 9hWsgU+vB1MPfxtfy0eLiKu4r+niuaCgEm9zo8o+CVHYQgatiPKiHr2u99nYQ8ywvq1iy9WAWGiWq1 CrihEehbnAcfiAtd9fjxppadatXzAw2Nba5kQ2z52xtrfKsk1UC+kuk8e+QANRsVj9uA/ASX+sqVmY TKSYWQWrp18kNha4YQM8H6ww/o9uvDHhdH39C1DfyScFwzXOn599HkwVJbNzW5koYl+NRW47sAonxT Hrat3w8MZstRYfTiQzBoe6WdOUSLH+er3NUfeR09tKL8sPPeXoEtPxkyqFE7OCWvIFATJFUrFpk/KO NGX+lJMPrvNwNJ8UXQecK+eUlAuwME X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB The seccomp benchmark test makes a number of checks on the performance it measures and logs them to the output but does so in a custom format which none of the automated test runners understand meaning that the chances that anyone is paying attention are slim. Let's additionally log each result in KTAP format so that automated systems parsing the test output will see each comparison as a test case. The original logs are left in place since they provide the actual numbers for analysis. As part of this rework the flow for the main program so that when we skip tests we still log all the tests we skip, this is because the standard KTAP headers and footers include counts of the number of expected and run tests. Tested-by: Anders Roxell Acked-by: Kees Cook Signed-off-by: Mark Brown --- .../testing/selftests/seccomp/seccomp_benchmark.c | 61 +++++++++++++++------- 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/tools/testing/selftests/seccomp/seccomp_benchmark.c b/tools/testing/selftests/seccomp/seccomp_benchmark.c index 93168dd2c1e3..97b86980b768 100644 --- a/tools/testing/selftests/seccomp/seccomp_benchmark.c +++ b/tools/testing/selftests/seccomp/seccomp_benchmark.c @@ -98,24 +98,36 @@ bool le(int i_one, int i_two) } long compare(const char *name_one, const char *name_eval, const char *name_two, - unsigned long long one, bool (*eval)(int, int), unsigned long long two) + unsigned long long one, bool (*eval)(int, int), unsigned long long two, + bool skip) { bool good; + if (skip) { + ksft_test_result_skip("%s %s %s\n", name_one, name_eval, + name_two); + return 0; + } + ksft_print_msg("\t%s %s %s (%lld %s %lld): ", name_one, name_eval, name_two, (long long)one, name_eval, (long long)two); if (one > INT_MAX) { ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)one); - return 1; + good = false; + goto out; } if (two > INT_MAX) { ksft_print_msg("Miscalculation! Measurement went negative: %lld\n", (long long)two); - return 1; + good = false; + goto out; } good = eval(one, two); printf("%s\n", good ? "✔️" : "❌"); +out: + ksft_test_result(good, "%s %s %s\n", name_one, name_eval, name_two); + return good ? 0 : 1; } @@ -142,9 +154,13 @@ int main(int argc, char *argv[]) unsigned long long samples, calc; unsigned long long native, filter1, filter2, bitmap1, bitmap2; unsigned long long entry, per_filter1, per_filter2; + bool skip = false; setbuf(stdout, NULL); + ksft_print_header(); + ksft_set_plan(7); + ksft_print_msg("Running on:\n"); ksft_print_msg(""); system("uname -a"); @@ -202,8 +218,10 @@ int main(int argc, char *argv[]) #define ESTIMATE(fmt, var, what) do { \ var = (what); \ ksft_print_msg("Estimated " fmt ": %llu ns\n", var); \ - if (var > INT_MAX) \ - goto more_samples; \ + if (var > INT_MAX) { \ + skip = true; \ + ret |= 1; \ + } \ } while (0) ESTIMATE("total seccomp overhead for 1 bitmapped filter", calc, @@ -222,30 +240,33 @@ int main(int argc, char *argv[]) (filter2 - native - entry) / 4); ksft_print_msg("Expectations:\n"); - ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1); - bits = compare("native", "≤", "1 filter", native, le, filter1); + ret |= compare("native", "≤", "1 bitmap", native, le, bitmap1, + skip); + bits = compare("native", "≤", "1 filter", native, le, filter1, + skip); if (bits) - goto more_samples; + skip = true; ret |= compare("per-filter (last 2 diff)", "≈", "per-filter (filters / 4)", - per_filter1, approx, per_filter2); + per_filter1, approx, per_filter2, skip); bits = compare("1 bitmapped", "≈", "2 bitmapped", - bitmap1 - native, approx, bitmap2 - native); + bitmap1 - native, approx, bitmap2 - native, skip); if (bits) { ksft_print_msg("Skipping constant action bitmap expectations: they appear unsupported.\n"); - goto out; + skip = true; } - ret |= compare("entry", "≈", "1 bitmapped", entry, approx, bitmap1 - native); - ret |= compare("entry", "≈", "2 bitmapped", entry, approx, bitmap2 - native); + ret |= compare("entry", "≈", "1 bitmapped", entry, approx, + bitmap1 - native, skip); + ret |= compare("entry", "≈", "2 bitmapped", entry, approx, + bitmap2 - native, skip); ret |= compare("native + entry + (per filter * 4)", "≈", "4 filters total", - entry + (per_filter1 * 4) + native, approx, filter2); - if (ret == 0) - goto out; + entry + (per_filter1 * 4) + native, approx, filter2, + skip); -more_samples: - ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); -out: - return 0; + if (ret) + ksft_print_msg("Saw unexpected benchmark result. Try running again with more samples?\n"); + + ksft_finished(); }