From patchwork Mon Sep 13 15:35:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B434C433FE for ; Mon, 13 Sep 2021 15:36:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DED4610A2 for ; Mon, 13 Sep 2021 15:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241746AbhIMPhe (ORCPT ); Mon, 13 Sep 2021 11:37:34 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:63540 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237922AbhIMPh3 (ORCPT ); Mon, 13 Sep 2021 11:37:29 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWk6t015226; Mon, 13 Sep 2021 08:35:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=2oS4sXCnNcE5fScztHpWKr74xhb7xDYhheuArylBdPI=; b=UBeXI3KbRRPgCzvpS9nKn9IIP+qQXKbdE7Tk/wfsmZwpf0Cp3wTlOclYxdbhwXQxAwR7 AeGigcem82xqRDAsheEBmlM2onAHA1ti7afP5NFWggUJ4BV1dAgSsTiElkCvAOTkxTzU oP1J09Cqw7uAHLfKUY3me+nPVgmdZQun6oimlviOShPvLMQzEBJLkZjcHPK5n1nQBc64 ph8taROmB4lwsug93xLzxc+DKh1GQEHrBdXPqGD9MLjZsCDu4UpgjybsU2wrWmtJttql gcZNtQuIiWQ/McYca0iKjqbuZ2EzESKx8wxvVhw8CaCVNQTXf49bTazuB20KSlun5M6e cw== Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2173.outbound.protection.outlook.com [104.47.59.173]) by mx0a-0064b401.pphosted.com with ESMTP id 3b1kfn0pky-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 08:35:58 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HA5AHzfDQaSAx2MLGhfdClbLywOD/COFYKgaxzobvK0M/BntIULvD8dwbX6qBgVXHCXUCA0fsH6YYPWF4II17agGyXh5s8q1oM2bhpmvLkZ6Gi8zmYgY8v51xxvinM81rYWZAST9S5OB9UciP8ur99prT0FUiAqqjAYo6zBHLKmQ2ULizayg6Zeoy8ZXbQCIxpyN+E8ChmVFyviDiI4LF5jZ+wlJ7eYNHUkrNVmR/rW2ou3Rlrww/48yD9KuoEbUEnZ6Vf68o8L3z681l9VP1e5ffsZxNZFl2YuaPoWTYWOkt3/u7GHLXSdondX8C82Dq8gNmmYpcoJWgY9SZ9jBgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=2oS4sXCnNcE5fScztHpWKr74xhb7xDYhheuArylBdPI=; b=B+GkJbSg2G3IqFnkvV+EypApRN0CNhIzpiabKweTeSZCmHd/y657QAJ0b2Akzz3FmqudJLEmP5lo+uhiwRHs40y0NOPUtEA9TOI2Q62IbWUeQzwGzJ/ZmqI9E0DooEiDFdxvDhhy1IOBrGFDkZJQl94F1G+huZZtz5zUY2eg3dnrVLo3CD5qotAjISJQN4Lj33CPh4maCuiawMbsMZXC5u38AIHgqyRQhOqPA8f70T7iekkVxUGru373Gj16wmKhVY+aiBu5VHsAiMITpQgvrnsAsF3f8Q7BrxAb5H9Pqp9cZM39k8szHkD/hTfiGsRQ02vH/r/VRnxP5TPr43yUIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM4PR11MB5549.namprd11.prod.outlook.com (2603:10b6:5:388::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep 2021 15:35:57 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:35:57 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 01/13] bpf/verifier: per-register parent pointers Date: Mon, 13 Sep 2021 18:35:25 +0300 Message-Id: <20210913153537.2162465-2-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:35:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fe20d772-8114-40fd-2a8e-08d976cc2e3e X-MS-TrafficTypeDiagnostic: DM4PR11MB5549: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: In4HZtiXmZ9ihPedsrxzKEE76fQh8RSyQpdEx2dQhBW8/Gj3LK/Gy9S1mCw5al5jf4EdwHL9gWFxMgt6HCE/Q/tNA5cHQfx5OyOn9D6HbIe4kc2hNkwoum4TsFrHxYSWNfV/WqNXKpvm1boNO78AeVFvOepJ4GtnNRdkAhdbBSJwELLo+sr/ow05lPMiqYsF8uayJCKbyJdiy2Jk99dBPAKhOA0X3Nu1gYyqGD5PZkvwfcY9NgqTOydNpBlAAEwGzDCjE2hVkEXfj2sv69MEj/lhpFz7UZTnbbukYPhJI0z4NliygbVj8SJlepBG/I5jkJ2sxGrEr5wQ+op0KJjFJQP8hOcMGO5e/55hUD2n3j+5dyvyYOxGh/GqFaQHGoyj6NrZn3gEBvaZ852t4X3xcSS3j/WkHdVYkaEpxk+JpqZJh2zRs2EQcodeSwM4qEh1Zajj8dfIrrPeFqpbRUNPMfMymd0sktqnldTCq2LPI2+tDyKaBa0l0vkGOkJbaseVmeb9ZfpcCUOlhyvfsc9CIlPCFbq93Afou0AaxJm3DHKfCTvgI5s80RlFp7XF5MTLzpGxwZ3EsOEe1HV5tFXCDYxYJEyGU8PvDdlbxaRY2sPXqwSlZtITKQfvretxyXKA54QH/ODUUbqYTsM04IcMX+M9MlTKI62wlxQWqwroASzOCIzCoCZ05agZxaAwxiPXfUI4a46tvASLhY9OspOgRg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(366004)(39850400004)(136003)(376002)(478600001)(26005)(83380400001)(186003)(38350700002)(2616005)(8936002)(30864003)(38100700002)(44832011)(52116002)(8676002)(6916009)(5660300002)(6506007)(956004)(86362001)(316002)(36756003)(2906002)(4326008)(1076003)(66946007)(6666004)(66476007)(66556008)(6512007)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: h67sswmJetOp/mZatXJIoW30XDncvJaJm19XxlIOS/Q7xLHe9lyr5vu/7NO0yW4/3nZOVzxwjOf5pGb/AhJfYOdPui0hyGxW1VK4JIDGuVWdT9dTHtzSfBVIatxYKMBoH8N9sxkSvahK9K+PVWHZizIk5qqCLKxgA1h4Yzxpf8BkOaXr7ZYxwjZi+KB97mX7rWF5wlSpjnXpoLOnj9uItJFWDUvJUeZpmAKGDwwz3pU4U+l2/snalJ08IE9M6Liud8cZCarnpqiFfEThGag7SqUMX67f81ywLsLt5SWlMI717Q5rXXDdtR4PhznLrFdRUBkn2jvKoR6MnOp0nkDyhJTGOaelV0eBlwL5QewV2ICfWURXKRAhpDAwtInxNm/6fXP3ur7C+LFZWMClw/GKQ5e0JXlGu2F/Y+5RrNBNZlEP7ySJ/xJ3scpkmtLYTqMojCYD482kvM7j5NNv9rKYpNwqWJwrXR2vSBDc1eBxPUxTm8ncObELi10XxtGD09bVW89jMJ3cgjlkLvz4U/E2Ol0lsRfFhJBu8FWa9y268gflZhxiLLprGPCQhtMrycvo5c50ZZ1OX8w/fBKgeT1otJwWEcuZj+fuVnO8HlpFqu4Pu4Tv0K/ZZFTBXg98xqfwHV4Vs3iAj0uf20610qRLfVEoBfyX1hvQmTrlTqHVK0KXw/DiC+cbkadEx1tfufA7KVJ1/ij7YG574FfO//Ly+VaHTnsqqrb66rl7zgKs1mKzfCSVyDxMzeZKkNkTy9xURjTSzHtjms1vWzF8+L9QjBdBxAIe3GmZBC0o3w8riweElxVfiMbKJ/T5nLFffMkXTnzvp2vwS5/9eLZi9KAt1KJwqFxdzTQQtFnR5fs2dh+vc0T7tvp7rasStJ/feh0QVpF/aFZLxjgFVDVOE4lUuC9E0WtA7+CE4CXR07MP5HuhSbbSHVTckA0GIEn3YVx7ZVkAcNh+mkK061cLf7All66QjBxBW7WJpG9u1wvfxzCopqRIXqJ9/72V0ZTF1tLtHDmp1PgLVPyye7rJDtPA+YF8LSOrKpSpKmAnOwozYb6aFTPwsgzQg/3pyRbS8C0fJwAVImIqsfkrJWLS9WOr68tjmsPiKiJESzv+GMKBTjF1L0FGGUPl1W0Vc0EJIpw40Z9dDO19V2dQVbnAGsZHFq+03CH47KDgU/xWbB+fS52TC1vhK6xDeaHAqrQurajSeO99fM0EHzldJKEibGKznBVzGGt67EUis4+Fh9Fw/n+ePn4Dnc1rPGAGJSulMsnzdybiIhXy58i3u7rYkc7aVVm+Q1iMf62BYDBMJq16WGgZ/7FJ4IUQKNXI2FhNtFY+ X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe20d772-8114-40fd-2a8e-08d976cc2e3e X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:35:57.3496 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kPR2h/HRC4w3RVn0SLDLMmTczRlzdKPcQ66MiBS8uoC4GIlSQyS3K8Tq+lnm4jrT8JdGmVlOfNSbTEIwmPZtB9tifr73DJWyD0j92X+6Ri0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5549 X-Proofpoint-ORIG-GUID: XQZVm5yc8540GhlDSfzD43hSL9YWgUuj X-Proofpoint-GUID: XQZVm5yc8540GhlDSfzD43hSL9YWgUuj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Edward Cree commit 679c782de14bd48c19dd74cd1af20a2bc05dd936 upstream. By giving each register its own liveness chain, we elide the skip_callee() logic. Instead, each register's parent is the state it inherits from; both check_func_call() and prepare_func_exit() automatically connect reg states to the correct chain since when they copy the reg state across (r1-r5 into the callee as args, and r0 out as the return value) they also copy the parent pointer. Signed-off-by: Edward Cree Signed-off-by: Alexei Starovoitov [OP: adjusted context for 4.19] Signed-off-by: Ovidiu Panait --- include/linux/bpf_verifier.h | 8 +- kernel/bpf/verifier.c | 183 +++++++++-------------------------- 2 files changed, 47 insertions(+), 144 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 1c8517320ea6..daab0960c054 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -41,6 +41,7 @@ enum bpf_reg_liveness { }; struct bpf_reg_state { + /* Ordering of fields matters. See states_equal() */ enum bpf_reg_type type; union { /* valid when type == PTR_TO_PACKET */ @@ -62,7 +63,6 @@ struct bpf_reg_state { * came from, when one is tested for != NULL. */ u32 id; - /* Ordering of fields matters. See states_equal() */ /* For scalar types (SCALAR_VALUE), this represents our knowledge of * the actual value. * For pointer types, this represents the variable part of the offset @@ -79,15 +79,15 @@ struct bpf_reg_state { s64 smax_value; /* maximum possible (s64)value */ u64 umin_value; /* minimum possible (u64)value */ u64 umax_value; /* maximum possible (u64)value */ + /* parentage chain for liveness checking */ + struct bpf_reg_state *parent; /* Inside the callee two registers can be both PTR_TO_STACK like * R1=fp-8 and R2=fp-8, but one of them points to this function stack * while another to the caller's stack. To differentiate them 'frameno' * is used which is an index in bpf_verifier_state->frame[] array * pointing to bpf_func_state. - * This field must be second to last, for states_equal() reasons. */ u32 frameno; - /* This field must be last, for states_equal() reasons. */ enum bpf_reg_liveness live; }; @@ -110,7 +110,6 @@ struct bpf_stack_state { */ struct bpf_func_state { struct bpf_reg_state regs[MAX_BPF_REG]; - struct bpf_verifier_state *parent; /* index of call instruction that called into this func */ int callsite; /* stack frame number of this function state from pov of @@ -132,7 +131,6 @@ struct bpf_func_state { struct bpf_verifier_state { /* call stack tracking */ struct bpf_func_state *frame[MAX_CALL_FRAMES]; - struct bpf_verifier_state *parent; u32 curframe; bool speculative; }; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index abdc9eca463c..a5259ff30073 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -380,9 +380,9 @@ static int copy_stack_state(struct bpf_func_state *dst, /* do_check() starts with zero-sized stack in struct bpf_verifier_state to * make it consume minimal amount of memory. check_stack_write() access from * the program calls into realloc_func_state() to grow the stack size. - * Note there is a non-zero 'parent' pointer inside bpf_verifier_state - * which this function copies over. It points to previous bpf_verifier_state - * which is never reallocated + * Note there is a non-zero parent pointer inside each reg of bpf_verifier_state + * which this function copies over. It points to corresponding reg in previous + * bpf_verifier_state which is never reallocated */ static int realloc_func_state(struct bpf_func_state *state, int size, bool copy_old) @@ -467,7 +467,6 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state, } dst_state->speculative = src->speculative; dst_state->curframe = src->curframe; - dst_state->parent = src->parent; for (i = 0; i <= src->curframe; i++) { dst = dst_state->frame[i]; if (!dst) { @@ -739,6 +738,7 @@ static void init_reg_state(struct bpf_verifier_env *env, for (i = 0; i < MAX_BPF_REG; i++) { mark_reg_not_init(env, regs, i); regs[i].live = REG_LIVE_NONE; + regs[i].parent = NULL; } /* frame pointer */ @@ -883,74 +883,21 @@ static int check_subprogs(struct bpf_verifier_env *env) return 0; } -static -struct bpf_verifier_state *skip_callee(struct bpf_verifier_env *env, - const struct bpf_verifier_state *state, - struct bpf_verifier_state *parent, - u32 regno) -{ - struct bpf_verifier_state *tmp = NULL; - - /* 'parent' could be a state of caller and - * 'state' could be a state of callee. In such case - * parent->curframe < state->curframe - * and it's ok for r1 - r5 registers - * - * 'parent' could be a callee's state after it bpf_exit-ed. - * In such case parent->curframe > state->curframe - * and it's ok for r0 only - */ - if (parent->curframe == state->curframe || - (parent->curframe < state->curframe && - regno >= BPF_REG_1 && regno <= BPF_REG_5) || - (parent->curframe > state->curframe && - regno == BPF_REG_0)) - return parent; - - if (parent->curframe > state->curframe && - regno >= BPF_REG_6) { - /* for callee saved regs we have to skip the whole chain - * of states that belong to callee and mark as LIVE_READ - * the registers before the call - */ - tmp = parent; - while (tmp && tmp->curframe != state->curframe) { - tmp = tmp->parent; - } - if (!tmp) - goto bug; - parent = tmp; - } else { - goto bug; - } - return parent; -bug: - verbose(env, "verifier bug regno %d tmp %p\n", regno, tmp); - verbose(env, "regno %d parent frame %d current frame %d\n", - regno, parent->curframe, state->curframe); - return NULL; -} - +/* Parentage chain of this register (or stack slot) should take care of all + * issues like callee-saved registers, stack slot allocation time, etc. + */ static int mark_reg_read(struct bpf_verifier_env *env, - const struct bpf_verifier_state *state, - struct bpf_verifier_state *parent, - u32 regno) + const struct bpf_reg_state *state, + struct bpf_reg_state *parent) { bool writes = parent == state->parent; /* Observe write marks */ - if (regno == BPF_REG_FP) - /* We don't need to worry about FP liveness because it's read-only */ - return 0; - while (parent) { /* if read wasn't screened by an earlier write ... */ - if (writes && state->frame[state->curframe]->regs[regno].live & REG_LIVE_WRITTEN) + if (writes && state->live & REG_LIVE_WRITTEN) break; - parent = skip_callee(env, state, parent, regno); - if (!parent) - return -EFAULT; /* ... then we depend on parent's value */ - parent->frame[parent->curframe]->regs[regno].live |= REG_LIVE_READ; + parent->live |= REG_LIVE_READ; state = parent; parent = state->parent; writes = true; @@ -976,7 +923,10 @@ static int check_reg_arg(struct bpf_verifier_env *env, u32 regno, verbose(env, "R%d !read_ok\n", regno); return -EACCES; } - return mark_reg_read(env, vstate, vstate->parent, regno); + /* We don't need to worry about FP liveness because it's read-only */ + if (regno != BPF_REG_FP) + return mark_reg_read(env, ®s[regno], + regs[regno].parent); } else { /* check whether register used as dest operand can be written to */ if (regno == BPF_REG_FP) { @@ -1087,8 +1037,8 @@ static int check_stack_write(struct bpf_verifier_env *env, } else { u8 type = STACK_MISC; - /* regular write of data into stack */ - state->stack[spi].spilled_ptr = (struct bpf_reg_state) {}; + /* regular write of data into stack destroys any spilled ptr */ + state->stack[spi].spilled_ptr.type = NOT_INIT; /* only mark the slot as written if all 8 bytes were written * otherwise read propagation may incorrectly stop too soon @@ -1113,61 +1063,6 @@ static int check_stack_write(struct bpf_verifier_env *env, return 0; } -/* registers of every function are unique and mark_reg_read() propagates - * the liveness in the following cases: - * - from callee into caller for R1 - R5 that were used as arguments - * - from caller into callee for R0 that used as result of the call - * - from caller to the same caller skipping states of the callee for R6 - R9, - * since R6 - R9 are callee saved by implicit function prologue and - * caller's R6 != callee's R6, so when we propagate liveness up to - * parent states we need to skip callee states for R6 - R9. - * - * stack slot marking is different, since stacks of caller and callee are - * accessible in both (since caller can pass a pointer to caller's stack to - * callee which can pass it to another function), hence mark_stack_slot_read() - * has to propagate the stack liveness to all parent states at given frame number. - * Consider code: - * f1() { - * ptr = fp - 8; - * *ptr = ctx; - * call f2 { - * .. = *ptr; - * } - * .. = *ptr; - * } - * First *ptr is reading from f1's stack and mark_stack_slot_read() has - * to mark liveness at the f1's frame and not f2's frame. - * Second *ptr is also reading from f1's stack and mark_stack_slot_read() has - * to propagate liveness to f2 states at f1's frame level and further into - * f1 states at f1's frame level until write into that stack slot - */ -static void mark_stack_slot_read(struct bpf_verifier_env *env, - const struct bpf_verifier_state *state, - struct bpf_verifier_state *parent, - int slot, int frameno) -{ - bool writes = parent == state->parent; /* Observe write marks */ - - while (parent) { - if (parent->frame[frameno]->allocated_stack <= slot * BPF_REG_SIZE) - /* since LIVE_WRITTEN mark is only done for full 8-byte - * write the read marks are conservative and parent - * state may not even have the stack allocated. In such case - * end the propagation, since the loop reached beginning - * of the function - */ - break; - /* if read wasn't screened by an earlier write ... */ - if (writes && state->frame[frameno]->stack[slot].spilled_ptr.live & REG_LIVE_WRITTEN) - break; - /* ... then we depend on parent's value */ - parent->frame[frameno]->stack[slot].spilled_ptr.live |= REG_LIVE_READ; - state = parent; - parent = state->parent; - writes = true; - } -} - static int check_stack_read(struct bpf_verifier_env *env, struct bpf_func_state *reg_state /* func where register points to */, int off, int size, int value_regno) @@ -1205,8 +1100,8 @@ static int check_stack_read(struct bpf_verifier_env *env, */ state->regs[value_regno].live |= REG_LIVE_WRITTEN; } - mark_stack_slot_read(env, vstate, vstate->parent, spi, - reg_state->frameno); + mark_reg_read(env, ®_state->stack[spi].spilled_ptr, + reg_state->stack[spi].spilled_ptr.parent); return 0; } else { int zeros = 0; @@ -1222,8 +1117,8 @@ static int check_stack_read(struct bpf_verifier_env *env, off, i, size); return -EACCES; } - mark_stack_slot_read(env, vstate, vstate->parent, spi, - reg_state->frameno); + mark_reg_read(env, ®_state->stack[spi].spilled_ptr, + reg_state->stack[spi].spilled_ptr.parent); if (value_regno >= 0) { if (zeros == size) { /* any size read into register is zero extended, @@ -1927,8 +1822,8 @@ static int check_stack_boundary(struct bpf_verifier_env *env, int regno, /* reading any byte out of 8-byte 'spill_slot' will cause * the whole slot to be marked as 'read' */ - mark_stack_slot_read(env, env->cur_state, env->cur_state->parent, - spi, state->frameno); + mark_reg_read(env, &state->stack[spi].spilled_ptr, + state->stack[spi].spilled_ptr.parent); } return update_stack_depth(env, state, off); } @@ -2384,11 +2279,13 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn, state->curframe + 1 /* frameno within this callchain */, subprog /* subprog number within this prog */); - /* copy r1 - r5 args that callee can access */ + /* copy r1 - r5 args that callee can access. The copy includes parent + * pointers, which connects us up to the liveness chain + */ for (i = BPF_REG_1; i <= BPF_REG_5; i++) callee->regs[i] = caller->regs[i]; - /* after the call regsiters r0 - r5 were scratched */ + /* after the call registers r0 - r5 were scratched */ for (i = 0; i < CALLER_SAVED_REGS; i++) { mark_reg_not_init(env, caller->regs, caller_saved[i]); check_reg_arg(env, caller_saved[i], DST_OP_NO_MARK); @@ -4844,7 +4741,7 @@ static bool regsafe(struct bpf_reg_state *rold, struct bpf_reg_state *rcur, /* explored state didn't use this */ return true; - equal = memcmp(rold, rcur, offsetof(struct bpf_reg_state, frameno)) == 0; + equal = memcmp(rold, rcur, offsetof(struct bpf_reg_state, parent)) == 0; if (rold->type == PTR_TO_STACK) /* two stack pointers are equal only if they're pointing to @@ -5083,7 +4980,7 @@ static bool states_equal(struct bpf_verifier_env *env, * equivalent state (jump target or such) we didn't arrive by the straight-line * code, so read marks in the state must propagate to the parent regardless * of the state's write marks. That's what 'parent == state->parent' comparison - * in mark_reg_read() and mark_stack_slot_read() is for. + * in mark_reg_read() is for. */ static int propagate_liveness(struct bpf_verifier_env *env, const struct bpf_verifier_state *vstate, @@ -5104,7 +5001,8 @@ static int propagate_liveness(struct bpf_verifier_env *env, if (vparent->frame[vparent->curframe]->regs[i].live & REG_LIVE_READ) continue; if (vstate->frame[vstate->curframe]->regs[i].live & REG_LIVE_READ) { - err = mark_reg_read(env, vstate, vparent, i); + err = mark_reg_read(env, &vstate->frame[vstate->curframe]->regs[i], + &vparent->frame[vstate->curframe]->regs[i]); if (err) return err; } @@ -5119,7 +5017,8 @@ static int propagate_liveness(struct bpf_verifier_env *env, if (parent->stack[i].spilled_ptr.live & REG_LIVE_READ) continue; if (state->stack[i].spilled_ptr.live & REG_LIVE_READ) - mark_stack_slot_read(env, vstate, vparent, i, frame); + mark_reg_read(env, &state->stack[i].spilled_ptr, + &parent->stack[i].spilled_ptr); } } return err; @@ -5129,7 +5028,7 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx) { struct bpf_verifier_state_list *new_sl; struct bpf_verifier_state_list *sl; - struct bpf_verifier_state *cur = env->cur_state; + struct bpf_verifier_state *cur = env->cur_state, *new; int i, j, err, states_cnt = 0; sl = env->explored_states[insn_idx]; @@ -5175,16 +5074,18 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx) return -ENOMEM; /* add new state to the head of linked list */ - err = copy_verifier_state(&new_sl->state, cur); + new = &new_sl->state; + err = copy_verifier_state(new, cur); if (err) { - free_verifier_state(&new_sl->state, false); + free_verifier_state(new, false); kfree(new_sl); return err; } new_sl->next = env->explored_states[insn_idx]; env->explored_states[insn_idx] = new_sl; /* connect new state to parentage chain */ - cur->parent = &new_sl->state; + for (i = 0; i < BPF_REG_FP; i++) + cur_regs(env)[i].parent = &new->frame[new->curframe]->regs[i]; /* clear write marks in current state: the writes we did are not writes * our child did, so they don't screen off its reads from us. * (There are no read marks in current state, because reads always mark @@ -5197,9 +5098,13 @@ static int is_state_visited(struct bpf_verifier_env *env, int insn_idx) /* all stack frames are accessible from callee, clear them all */ for (j = 0; j <= cur->curframe; j++) { struct bpf_func_state *frame = cur->frame[j]; + struct bpf_func_state *newframe = new->frame[j]; - for (i = 0; i < frame->allocated_stack / BPF_REG_SIZE; i++) + for (i = 0; i < frame->allocated_stack / BPF_REG_SIZE; i++) { frame->stack[i].spilled_ptr.live = REG_LIVE_NONE; + frame->stack[i].spilled_ptr.parent = + &newframe->stack[i].spilled_ptr; + } } return 0; } From patchwork Mon Sep 13 15:35:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DABAC433F5 for ; Mon, 13 Sep 2021 15:36:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F1976108B for ; Mon, 13 Sep 2021 15:36:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234182AbhIMPhb (ORCPT ); Mon, 13 Sep 2021 11:37:31 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:62880 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241053AbhIMPh3 (ORCPT ); Mon, 13 Sep 2021 11:37:29 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWk6u015226; Mon, 13 Sep 2021 08:35:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=34FZp8QcNQGX7/H2kt6f38q+ycMsfFnOwwBK+1woWH8=; b=EjM6oBGAHLVLqnCJrO/2OBbgUd6Ty7fYfZj2XFEZp7c7eHFNjPDXPZSVqOO1NfjskI/K yNE61dm2QeiTG4fPY0OiN+o09Q6fS6mLfUtrosnv7jcEBfy+yg9f+AaYNfDUYmGba4Rm ITB952caoTYXkIutyYnBKB0kslJGfhC5eR2MWCcmKGA628ePd2qsexjvHZ/BO66t1bKs 4dLBJsi+p3+D3+MMSnkqERQL1usTEHSz7RbzuTtCq6Hr/llHRPFAcCHRdiKhC7DDLwmH DxkQLd9MOO6udsHygtk+Z+5mZXmqEfGmuFhNXcPt2rS/ItX8YZrnDggwpWlMN9m6QbPj Zw== Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2173.outbound.protection.outlook.com [104.47.59.173]) by mx0a-0064b401.pphosted.com with ESMTP id 3b1kfn0pky-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 08:35:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jxo+vSGwzCcB8DXukR4YvXWobxkL+Av++tLh/Nw2Hg0MyUcJgf5d6CYS8SjThbhF7qy+8+ltOGu7ovkYeaEC6KYXgxy4Og0C2KyzBlSXhoaRrDyUSZqMIT9YT6p+6LuF6G8fJXpDmXmXEmkji/FJMKm/uu1AmjmEHJ6vdUQaaiE/cAEtnkRWOoiJE5v+S+gXbZ1z3WVNc10gnSqhWRNpwssC10MSxUKcE9UTR7VjCdLx/AVi0w+IQsNx9sGDK71ehvIW0buOTHbcjz7hkucnJfGYxChnqXf2Qu1RM68UQObZsr0Y/KkVz2M4FDmsYMYdhFKzdzqRgDwPu6TiGWFoMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=34FZp8QcNQGX7/H2kt6f38q+ycMsfFnOwwBK+1woWH8=; b=c2yFMX9oA4UA0VlFWunlIL5luFgci99v0Jt0ky6PYlxCSeG/276kmfBWuEnPXbOTmxUDPsyfTJDJNH5427u2JBvTJRsPjX2i88iGkuPEcylJ+gbxDtHxihLdnU9i+K6fbjOXC1e7eXsQR8inBSzauhO97EfxtVGzE7BLSL7PqSvFv3crsSH9XwAuopvrSo8/kzdPW4Fn4/cQOxETyYoOe8aduRMrhrNEiWTfpeWMxthYRIFiYYw6NwRk/YqLLxLOjGEka7Y/ITk3PEKgKaurNQwH9KIf8dRbD96XnNS4yNc/65w/Tn13hUVOHnJFBw49X5TvZkB1yBqB10imN8a0RQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM4PR11MB5549.namprd11.prod.outlook.com (2603:10b6:5:388::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep 2021 15:35:58 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:35:58 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 02/13] bpf: correct slot_type marking logic to allow more stack slot sharing Date: Mon, 13 Sep 2021 18:35:26 +0300 Message-Id: <20210913153537.2162465-3-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:35:57 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 993ccacf-e8c3-42b5-385b-08d976cc2f08 X-MS-TrafficTypeDiagnostic: DM4PR11MB5549: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1360; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HFvRZH/SWjLQYUFewjwN9LHr08s1FZT8ClCltm43WzF30Y+oEUBiLMaPNh80aWIVjWclgZWhesYPOEkjDicO8J4SwXnKeDCdAPMEuf7FuSKhssefGV/LT4J9AAKRS9z8oKrQBpnkFNFr469JxZmO8PBdu0JC8WvKi7KDWEvaQYV7FbkY8Lci+ua+f57uKUgF0jDbm/x2O6E2zhR3MuKMAuLBLXjRkF04yTXRCj2/DC6w/YlunL5h8Yu8Zt4eF9duHoupxohO1DALDLbZ0BtRURSj9xwz8aj0kqz4AYYRotoF5DNX2+IdfO9meRacrtdod40NRgPLaDX+dpJAOvHvBkDOZ8iCzOD+FjbfhrNzI5JWtcy9hm4aodwfj7tJLZhSgC26zuhauj9h41MYkNVDYhcHzCCfsa5ZBXgxzyIRZ7zm2aEwbyzsd5Co1wINkynirHI4r0E+7kCQIV6AhQx+ZV4UCZddC2fHGnSSLGYGfqhXc++c83UD0A8F+s1RZ95R+TZ621BY3opbZN8AGYv3yEeYXju3CeOjmp5ot5cfA7HbuZItlC6Kh4C9VmGVb6D9rFfqzJIm/kkH8hwChmNCXLfKj0zm4bpeKEG9pka0lfqy5SGoZZRWNoMJX5uOeXtgUZLOA0pSo4yLIENVtAbY78l2ir87iF2Zkb0t/J8EuEg5VqPYVAtjsnwwaU+mdkJx X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(366004)(39850400004)(136003)(376002)(478600001)(26005)(83380400001)(186003)(38350700002)(2616005)(8936002)(38100700002)(44832011)(52116002)(8676002)(6916009)(5660300002)(6506007)(956004)(86362001)(316002)(36756003)(2906002)(4326008)(1076003)(66946007)(6666004)(66476007)(66556008)(6512007)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WnRydPktcREcWyIaai6HAoXDgwus7ACtnoiYX6rBjkc8C4yFq8IRus3lZ2H75MERkbmUh++31Dd5vVzjLAdtgSjuPj1wrGispTJB0evQSJKxuWvi9Dua+IHkn0rjeQtS+nn3KDp7kY2V6J6d2Ynaanh2z0ZlYd2qUi8wt9axSAlRly84o1xovxSNcJdX9UFaVzrbtq+wc6cqBI0MC6COa1IKpIxk7gH1UyAdfKYpRa0VVxRjMPVkw1gE2ErTJ+3FgpaUTF8uzGcUoY1zJL8B86396bM8pRsjjQJwHxVFPU2QJUBkiBH+hdDSJ0LAlMnqsPdl+wOblOaQRYVkyT+/iSl7y0UzkJDyj1T+gkmhldMH5vFe2mUuKHgTu66jU7Hc0tf/Eg0yCIdMuNjTKnLmW1EEyIv1P1A4LpsYvH+nmPXrpeUbI9lkZm2P8fhb6xIMvfnwfrjDNzxgFCzIjbiWo1OkYlWbyU3CUzi1vwMQHVksDn4P3LGsxOBd2D7CnAJdob5TDdII9AFTwfP5Dz4q+buzBKojT8zI4LhSZvLpBcEPYMb2xKJCQ2ljPawi27dGoiwzXEGZcxTx4hy42UjKrF1mrjrH3U93ZfWxzmH48nCa/98FbrLKZoGMcJtvdKriYOhbshumnUOeTSixtFWduJJgJEKdWSnlF82QTRpWTutLJ7VU1mGKN8ZUlFLLO0F2+QkRshbpePC0PUFMP8pYdF7Xdcq8vL/P/CdymBAoQopAnbD/jNre5vXbwLAOTQ77spw+iZjQ64iSxElrgcWchhtNtXcplBcStCj9oyN16Ul/58s9SLZ9nUymLd6EH94GDIUkxcJeXVAaGNrDlfEmUgjm3YNZwOWTKY/eWJhUZ9+vz/BWdDUzHvgxY9tUidCcD6VLziEAGC0dMyFAmfsqdfwyotDGuf/BY+08t2aEzrXtNfqOPF2gxp0tgsdiY5i/2T/1UgepXhHFaxWMPDPXqn2nZrTuq1CC+Sq0Mt9WTschGMMk3HTn82Lg0Vv8zPE4JUfEIrA7euo2X+wBPSjrKcd3GEs9qkatJcHGQryjyCLUUdBv8gqqNj3os0FJaAQ5uMbXUJrOa2Qy0txIpl2e8/ARdPt6L9wXbmwlHSbmFlm8ZqszmA7Rdzmv+yVcQp8EK4+vqvuB4ORFBbuQ1KfwSmtxH+gb0nUBwmIgb9nRAVLLWucIMZJqvJXPQSuGJn/HmrwapdcHhiOXMJVrgbtvNQFzZw/r6jAjHbTvI85s062iaEhPrggLXK0t4Y2fUCZS9v7ITimMQCnRCYUiLn9eRoc8vl+QrEV1AIQjSWa3YMz0zvKXQ1JWTR7F++yg8Dpt X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 993ccacf-e8c3-42b5-385b-08d976cc2f08 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:35:58.5463 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6WgASMZiDEZriB1fGZZxrMZQV9c+4ekpgcYo+9rdIbOCZKwQAg9m/UlLVHOlu/0r850QvwrIDhK3X5B8WhyP5Z7Nj6+tQxN1EscFTotCEUE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5549 X-Proofpoint-ORIG-GUID: Gno-mZy5kEcAWbB85lDKklgdImDmfTzw X-Proofpoint-GUID: Gno-mZy5kEcAWbB85lDKklgdImDmfTzw X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Jiong Wang commit 0bae2d4d62d523f06ff1a8e88ce38b45400acd28 upstream. Verifier is supposed to support sharing stack slot allocated to ptr with SCALAR_VALUE for privileged program. However this doesn't happen for some cases. The reason is verifier is not clearing slot_type STACK_SPILL for all bytes, it only clears part of them, while verifier is using: slot_type[0] == STACK_SPILL as a convention to check one slot is ptr type. So, the consequence of partial clearing slot_type is verifier could treat a partially overridden ptr slot, which should now be a SCALAR_VALUE slot, still as ptr slot, and rejects some valid programs. Before this patch, test_xdp_noinline.o under bpf selftests, bpf_lxc.o and bpf_netdev.o under Cilium bpf repo, when built with -mattr=+alu32 are rejected due to this issue. After this patch, they all accepted. There is no processed insn number change before and after this patch on Cilium bpf programs. Reviewed-by: Jakub Kicinski Signed-off-by: Jiong Wang Reviewed-by: Daniel Borkmann Signed-off-by: Alexei Starovoitov [OP: adjusted context for 4.19] Signed-off-by: Ovidiu Panait --- kernel/bpf/verifier.c | 5 +++ tools/testing/selftests/bpf/test_verifier.c | 34 +++++++++++++++++++-- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a5259ff30073..b6f008dcb30c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1039,6 +1039,10 @@ static int check_stack_write(struct bpf_verifier_env *env, /* regular write of data into stack destroys any spilled ptr */ state->stack[spi].spilled_ptr.type = NOT_INIT; + /* Mark slots as STACK_MISC if they belonged to spilled ptr. */ + if (state->stack[spi].slot_type[0] == STACK_SPILL) + for (i = 0; i < BPF_REG_SIZE; i++) + state->stack[spi].slot_type[i] = STACK_MISC; /* only mark the slot as written if all 8 bytes were written * otherwise read propagation may incorrectly stop too soon @@ -1056,6 +1060,7 @@ static int check_stack_write(struct bpf_verifier_env *env, register_is_null(&cur->regs[value_regno])) type = STACK_ZERO; + /* Mark slots affected by this stack write. */ for (i = 0; i < size; i++) state->stack[spi].slot_type[(slot - i) % BPF_REG_SIZE] = type; diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index c7d17781dbfe..6b9ed915c6b0 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -956,15 +956,45 @@ static struct bpf_test tests[] = { BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_1, -8), /* mess up with R1 pointer on stack */ BPF_ST_MEM(BPF_B, BPF_REG_10, -7, 0x23), - /* fill back into R0 should fail */ + /* fill back into R0 is fine for priv. + * R0 now becomes SCALAR_VALUE. + */ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8), + /* Load from R0 should fail. */ + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 8), BPF_EXIT_INSN(), }, .errstr_unpriv = "attempt to corrupt spilled", - .errstr = "corrupted spill", + .errstr = "R0 invalid mem access 'inv", .result = REJECT, .flags = F_NEEDS_EFFICIENT_UNALIGNED_ACCESS, }, + { + "check corrupted spill/fill, LSB", + .insns = { + BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_1, -8), + BPF_ST_MEM(BPF_H, BPF_REG_10, -8, 0xcafe), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8), + BPF_EXIT_INSN(), + }, + .errstr_unpriv = "attempt to corrupt spilled", + .result_unpriv = REJECT, + .result = ACCEPT, + .retval = POINTER_VALUE, + }, + { + "check corrupted spill/fill, MSB", + .insns = { + BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_1, -8), + BPF_ST_MEM(BPF_W, BPF_REG_10, -4, 0x12345678), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8), + BPF_EXIT_INSN(), + }, + .errstr_unpriv = "attempt to corrupt spilled", + .result_unpriv = REJECT, + .result = ACCEPT, + .retval = POINTER_VALUE, + }, { "invalid src register in STX", .insns = { From patchwork Mon Sep 13 15:35:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32E83C4332F for ; Mon, 13 Sep 2021 15:36:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1EBAF6103B for ; Mon, 13 Sep 2021 15:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233130AbhIMPhg (ORCPT ); Mon, 13 Sep 2021 11:37:36 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:63766 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241401AbhIMPh3 (ORCPT ); Mon, 13 Sep 2021 11:37:29 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWc6Q015211; Mon, 13 Sep 2021 08:36:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=syNRd9duDpPeKklJdbnxwr+32+jRVPBqI+cME300f6w=; b=UARm6ea2u7i7G+46sy1ckxv6Zdqd8FV6GevVB2RpSgn6tf4TNLdyr/YUJZ2CoGIUyWTg DiuK6SH7v709sBqlhLqB6Dr5+NYKkZHfm+CF8T+oxmGorSurwwu1CU5Nq877P0w0uKnc BlKNlwAaQubLQpn4tw5LRiwGd0E+w9It+t+W7ITar+lmyxtEkjJA6w0WoBXro0sc5WVH d0yn7AKPFOW7cfLGTAGWL+LsFq1i4oR9YHwRyjgQ5Bc3t6XRxfIdtO89ErWncbnbviTv Kh3DJahcTMRjcp909aCMBPPBXv5s6CWGWcEInC4KI+rA913vZkYTZ8oh3Cj8t7//186C Yg== Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2174.outbound.protection.outlook.com [104.47.59.174]) by mx0a-0064b401.pphosted.com with ESMTP id 3b1kfn0pm2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 08:36:02 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G2xJrElYesSfZJyc8tnjfEk4wzVGT7PkAdqSi6f6jONRaAynXsa48Yba8oKN8g7JUziXUWnZkChr/tNS0+jlwAqc231Ee9IrtDcOsvJFaLzfbjXE/GbRGGvovW74HGYnoPff620S0UHW4JbRHX0z3aMHySHl0zhX6emClodz0IpIPfIsP8FHwXM0iMB+rRUFvkRoq6ZE3O6bQepUVeaM+zcmWubimkar5YIm/PfHwj88llBNEDPwEyfVAEDb7CBnN3FWxIPgmT402OBeZkgxhK/fLSkMYBCRcuG+ohSD4ZOkLQv1t88/4WWEGP43tcTgd+2nAc3Qfiiuxp9iehPcEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=syNRd9duDpPeKklJdbnxwr+32+jRVPBqI+cME300f6w=; b=nct2RnkL7y61DDR7r4uWMlPeorIIHpbel33JpjRd8Bljg9Uc4CFLde9r4ZEA5/mg7MtsDRfqexLidbEx2a7CnTlt3PWjgQbkYnEDUAox4LfcFPVC4YnUJS2avsLqsJEAIny/dvDCqB03YGglPDzsoHMVFkYSyhOZPjn6EmTXSW0Yc6PYg4PnqVRCKjVzbJkZC4OwtNbqCQotMilEQ5CVNbHyo5uUh6n6SXH/TZ8pI3Z9fZ8KWDmOz7qxwua3PD4PA/1h30++Mkp/YYah3qBceoCOi6akomPtXP2jFMrzGmBtBQs4Njvjj2zDmwh6iS68bwqz75DcVw3mNyYwJM1jMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM4PR11MB5549.namprd11.prod.outlook.com (2603:10b6:5:388::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.17; Mon, 13 Sep 2021 15:36:01 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:36:01 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 04/13] bpf: Reject indirect var_off stack access in raw mode Date: Mon, 13 Sep 2021 18:35:28 +0300 Message-Id: <20210913153537.2162465-5-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:36:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 33722b57-f84f-4b80-695b-08d976cc3077 X-MS-TrafficTypeDiagnostic: DM4PR11MB5549: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:480; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: er7tCke17f6D8EiGJdny3o5tlf3m8P8kA58aKvuJ3LhpqzO+pnJ0aTwjn6u4NiIzOri3vLd422Sl2XW7s+7i+6Kj0apGs2OwplgEusgr2gQZjdLhf/jrIpUfvLDGBxY9c5zFqF2BcWaAP+aKIcbSN1Xg4hj0id7t6s6rpj7pfwOkXx83GbJQKXxECdA2IkmEHzbu8s8+cdwYUH3Sf7+xX/w54CnfUZTd2vp/nd30+lemo0c5pLMNw4l52twGFhT1ziHZ8JEGxNg9cJOJ9putO+7YxvWHtYAxh8R+Yk8Fn4qC5aGpmE+y+NVTouvBUtVXaX0fjoDq1T9OLpMTEGwWJPAjl9XMwyGCaPwBrHsxVUz1nqL+/UuA5hfKrXfIhfc8akfOqXBK6zYWuT6RzUR8AtnaXivhqmsc1NMquergbfkYB9TOLpGmXb3gQlAyV64cNsZRmNzUvnpfkwQmIJfaeX4Q0oVTaEPK1kFPtzy+QygXfmuY2XP5ayglllpyYshp337FlcTcAfSOf0raWpG+xdbdg0h1U5Z89h7kq1aYc6KaFAdVayAW113mLk+G6Dg1dNFa3QBgOOZyjrSSwN5qA/EdquGcUiBrSRIK0ygMJRr0ofcJYeAfoXWwUbegHR9BRD8oyjnJF+jF4r/eoa+rcGlzWQd8WVaioiwu24OqRYf3+uTV8qFSmysnzomm+ISOC7mQR8ZIrlMVKl4Gfs17aA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(366004)(39850400004)(136003)(376002)(478600001)(26005)(83380400001)(186003)(38350700002)(2616005)(8936002)(38100700002)(44832011)(52116002)(8676002)(6916009)(5660300002)(6506007)(956004)(86362001)(316002)(36756003)(2906002)(4326008)(1076003)(66946007)(6666004)(66476007)(66556008)(6512007)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: EPQHizXPfO+PLp5jdsptm1q38PX5QpwzCDdklt+Oz6m3V6z9nxknUiIhZbbSE5XZhBn4aUhEcTLWx4lBwYWKWxrfKxtLuseudKFZzrqS1GWMCW1t+8MHZXhxSJAB+SdD6KCbSC8AQ7TDphJlp+f2iVl87VL2RU6pvyvLAUpIFRBBCcxa8c/8tL1CZb0RpdXke7q4mBboQDytkFgt9E1PIKb4a/B2FnYSdsvuMErD8GtvIv1WX5noyNDVz4TgE8GUgV5pBxaHKOzJa505dmwcY2VPO70hVuQbFeQvKU5AArOZJ7/B2dz8M/X2eVhzAkxm/RuLr4mVg51lPLwUV6/vtiXZT8EJMmpMyiKNqPLiEnuJPZOhMKXrEWpvwfLuZAHoddB4jQCqjnpNo3qBt3etIgzzIC/5FBIgM8+xCijKKVLyMLwK4zRzgDeMKF5z5jdcNfykxynPe+JxmBNwczLp+s/uJbAx1Z4SMWnpb8OfgWN158Zgsl+5AxTXT6ZpLFyBIp5iY4cxnQNMWtEA0/9bguPaYQoJrJ3KqsUJ0FaFK5zcY4uwRuFydaZ5wbuT0oC+7QyDoph2vOaUlHu7nWjM1Kuu8iYRo9eYf9MfrHwg/dbOdY2D+xN5lkgXUq7Y/h1w+AQ3VKZjdZCMSGmAtYGCJyx64n7h1KkuTBBwHKswHcG0EVDVDSFTSPe9m4eTvGrBHRAp7XnYTojC/HLh3K8CXZQcmUFwhnCmRP7++W3DHIUQ1hUqyGDqyRdqTpe3VTmP2HMQCFSWn9UmjiZtPAZEjPjjwzGhDBN9B/ogjeN8MpvdkAZp9mq3VCc36M9xmkRXQgUQz4MhKBJLGW2So1Wkdpg+XL7i/3/T7S34vXihCsgeegfHc1LyzcgSnoXA+13Fh4i+GtrPFybOa0EAWvlWaOu8xkkjZUDxY+/wTyxKwPjwOa1CO02x7zjgYB5NlIQeKBSH5hYaGKVc52jtx1EsJ9owis8BXdIxgMntyi9Vceek5sf/74IhtdNtaVVTxCuaVru1HBfrBiWoAXBV+PxkXO7OrPh5ts4M30sQ5Ul+OjUxZLXRZ0Hwh+8SZkWpjph9yGD13REAczJZihSCbRVWw/703MhqNeB1cjQoEOVcax3uW85NzN1Wwdghj+VVdV6tuXM+dvvAXQLb1G4ynBNUR4GatdoUVmEiY2zolVxvJJYOYiIyUI9L854MgMkncPwiZveIDCx9eQbUZv5DkJo0sTf7DAXiVa/lK2DEkqu5AOCcGYejfx/UytCM0hNZURdLp4yBBsmH1K9ag1vjmw+PpjLjcabWz+RnOkW2I5edbY5k9jIH+BhTsSwoIF7dZeiG X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33722b57-f84f-4b80-695b-08d976cc3077 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:36:00.9508 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NOPECfbi+Tw1+E33GIawttz/3XNiVew7QrroodRnT9CTtmQzHxzaLun+o3pFUPxAZEvlSdtYwBLzyuMt+qkyiGJQMLz9rRJ3I3nyZjrz9oM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5549 X-Proofpoint-ORIG-GUID: Wj4NHe_pVUp7AMOvdCUGgmZ9BpvFVBqL X-Proofpoint-GUID: Wj4NHe_pVUp7AMOvdCUGgmZ9BpvFVBqL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andrey Ignatov commit f2bcd05ec7b839ff826d2008506ad2d2dff46a59 upstream. It's hard to guarantee that whole memory is marked as initialized on helper return if uninitialized stack is accessed with variable offset since specific bounds are unknown to verifier. This may cause uninitialized stack leaking. Reject such an access in check_stack_boundary to prevent possible leaking. There are no known use-cases for indirect uninitialized stack access with variable offset so it shouldn't break anything. Fixes: 2011fccfb61b ("bpf: Support variable offset stack access from helpers") Reported-by: Daniel Borkmann Signed-off-by: Andrey Ignatov Signed-off-by: Daniel Borkmann Signed-off-by: Ovidiu Panait --- kernel/bpf/verifier.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 47395fa40219..a5360b603e4c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1811,6 +1811,15 @@ static int check_stack_boundary(struct bpf_verifier_env *env, int regno, if (err) return err; } else { + /* Only initialized buffer on stack is allowed to be accessed + * with variable offset. With uninitialized buffer it's hard to + * guarantee that whole memory is marked as initialized on + * helper return since specific bounds are unknown what may + * cause uninitialized stack leaking. + */ + if (meta && meta->raw_mode) + meta = NULL; + min_off = reg->smin_value + reg->off; max_off = reg->umax_value + reg->off; err = __check_stack_boundary(env, regno, min_off, access_size, From patchwork Mon Sep 13 15:35:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5AB6C433EF for ; Mon, 13 Sep 2021 15:36:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 909086103B for ; Mon, 13 Sep 2021 15:36:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241480AbhIMPhi (ORCPT ); Mon, 13 Sep 2021 11:37:38 -0400 Received: from mx0a-0064b401.pphosted.com ([205.220.166.238]:5292 "EHLO mx0a-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233162AbhIMPhc (ORCPT ); Mon, 13 Sep 2021 11:37:32 -0400 Received: from pps.filterd (m0250810.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWXVu015200; Mon, 13 Sep 2021 08:36:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=w1WxVnzVRN8wrhvPxtQ3W4l2TE2xJo1JXBSp5co0y6o=; b=WFiCBnCXCIj9oniWTWOc/LmBugsN2/tngYtWg0orJpOu+iMX9bQM0eQywWKpy3pfzWkb kq0UHDDwiX/YQ+8iZgbDQ3It5HTlaZmb3LL63WzhPL4JUqftQ2EzxEIGFjMSf4HUNTbj vYdLSCFuFjCsPj5KFvHVTg8rPAeLJFSlwkharl+aJwOOJ8guiqQWlc2p4J3AnTvGpGpu YcdjcyJ98hqYxkGLDkv+PWClum9dxmElMjPx0zF2kJVNAEdfGPeYOObBttgp/+6hQfhA jO8/yZjx8bLS/eRGxpgrZvyFReEilee6BZNZUiiNB3CaXCSELKpP6mOxFKFWJfNiXEfu hw== Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2048.outbound.protection.outlook.com [104.47.66.48]) by mx0a-0064b401.pphosted.com with ESMTP id 3b1kfn0pm7-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 08:36:06 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Um4L5W+SUbDsxkIH8GdAg3qUJ620fj00usD58WFTYYqw68sQcfJ5ZDNEZLSqV3kYGJeQmodQZYQHeYxTtajOv/qhynHzAeUiF9xOFdq2JY0V20IOugVpQbeq5iuouXcmu1U+oZ9cXRagpK5346v918l++6sUsuPxG7J7JTsgXdGHkafkh0+e0mfLzNVgD7covivQaYcuZ04oOAwSbWETgNSQaQF7dZiD8/oAP7dJCiRfJBW46uu4duJfEuZRVjUcbdKb4uguxiw0LWncxvvnc7/EfCPV/GK6wqbaZ/NRPUTJLpx/oP2JanPM0hezSwZvX5T5gwvmvctOTSNtYJ22Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=w1WxVnzVRN8wrhvPxtQ3W4l2TE2xJo1JXBSp5co0y6o=; b=V2EhwLtpeKxDQNAhew6TrauGqGiWneEzlS8poFT1Zxrw6MUAXMC+ihOabi9zWNqcqvKjrodsQMUdMcb5yeklS3CkBYSShJLZachphJMD+DIIZkK9JvU0Hi15vkbRMrcx8pIAq64COiOaH2aE2iBuefMd44/pkLH6UwZa05Np716YXwsPJsAZXJeASlzgbklVISGTcDxNVpDRjxOZ970EfrQ8lhMybyo7IRRnu9iUq5yxmyHzoO7GJ/bxRJRtjeWO4HZV92Z0wb1jRViHW2qoOi/augUvyasB4zi8NnzGTFEbWrECO60a9KEOoKiPaWir9hKbjLbZujVQ4ALdOHpHoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM6PR11MB4657.namprd11.prod.outlook.com (2603:10b6:5:2a6::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Mon, 13 Sep 2021 15:36:04 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:36:04 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 07/13] selftests/bpf: Test variable offset stack access Date: Mon, 13 Sep 2021 18:35:31 +0300 Message-Id: <20210913153537.2162465-8-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:36:03 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 91289e02-1e06-42a0-80d2-08d976cc32a4 X-MS-TrafficTypeDiagnostic: DM6PR11MB4657: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:139; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IrVkyid1m+hlqDuxgA56t6l7KrcOvIZ4SlWLw8ptp7hNZmFkQmCgs9r34GDyqmUIizoEOoU2Kkb2QHzqn+hrgCNBG+zFrBF0U1/HrUBnvRWVrYWV/afNfn5q35T6vG/SjxWszyZ5KdqqHFx0jwxIrxXEcRMCmMVTCCXgfg3ciNE3WfdQquIhbF7uYe5hQhtAUs4F3JpbPZpVCpC5Kap/PXgwDAoTcqpaid2HDSa6N74p3iLVAJVyXkdCkdmXEj72HLtYbCSQx/V1OugnvjBd6ftEpizqdx1wrI6/D4xdYHDKemIRq6CXvz+JJs8EL+f6ZKgkobdqWUeUWikCNeQYgUp7aVPTtGTUH5Dti5PHXEI6LIpuTrH4sssaLr6IzNdc85j8VxEWe8AcLirTAhN9QyLc4frD2Xy+b4lx/ve9W24TvEZijX7knhx0ROFnAWv/gSLQRavA1HQQ0zt4zSNOiLxKLnnVjlFpCAyzyT0twZvRhO0No9ZbLPSTuV5cXSquF1mB6EuUXZImH7QfzhBBiFQmUj9+79ty7kWwLe1Tg6dSXTmgHyTDwfV/hJxpogct3yQ3m4eL+6eyccvtVFPT960hr8gEORwhjt/iy5+wpIwHQSkZye+8Ak1z6TqCS9L5reJ2SPquRyNCdLtBH6x0qeSV+PyWO2G3XXMJT2cASIcUfHLqXl4LoS/18u3+mWmb66sBGXVyyMzQrdY21Uln6A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(136003)(346002)(366004)(39850400004)(26005)(52116002)(478600001)(186003)(6506007)(1076003)(66556008)(66476007)(8676002)(86362001)(83380400001)(6666004)(38100700002)(44832011)(36756003)(8936002)(38350700002)(66946007)(6486002)(6512007)(6916009)(5660300002)(4326008)(956004)(316002)(2616005)(2906002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +Ojp+BhuZK2+SwfsmvWjFwy4t9hisLkD9GuKbKawDi+eJSHAZnGtIMPxd4oDnJzZkO24+/bXIoxrdCyuWZavinqJUinmc4jzeurSo7+faH3Z8iijk1VvlfFCE7g6/frrTmzGqndzYELkccnJdQB+1soAbulQzPwDpYkiu8KCcZKlMeVpe5pAfbKz8epufFBDhf4HEGXqvJeSWwMDPsWv6d2IeBQ02z+W08c3dIgT2Axq5nuZz3FnGzDUCSHty4w4avia7oOdrTFdglGivaX/6ffljoUCzBv+k9SgdOPBcNap7SQVcQKNPAiSnXZxQ3TMqebVHdcLtJXTS25ZzSNsK+Kpp45XnzUFMaBDBfLx1sDg3cCD4lf8M82tJRr1k1HiIzGky/IZ+7oankdBqYoPHWqF1kz0gRr3Y8slNk52+n3/KQ7QZyi/6alPchlvunW6dOqbK7ek/uMqjCRNVB3Zq9g4tUVMhH7Rit0NPqXQBuc0CX2LHKf/mYfegjFY5NJSAo4FCFbYlRigl+4gRu7IeuGCQ7Xulud4fQeotnuz8clPGgeUCkTFGsOHdrqpfL3r6g0w5WNpo4Cwhd0lFMZElXMBJa9KNq1L3YM/nF6qtdE8qgkbgMw59gkemgamE9Ofi0M+xusbDKZ7R/4JbnmaUwpesBnolKU5SCo5rc2cbjI4toyjqfJy+bYmQp0JPIg5jAZm0//sNn3t4L9OQ6T/hri/Dt4i0jnbJCWx3+/kXsZ3MBtFJytRnLi2TWHSsp48+dqX48T69SjPnuJuBtbdN2w+c63dCwJTmHMFbwr9CD9cn7P6awlmfurtcRv38eshXD1ImEXibxlnMvY//G2BrbfYYTw89APBLvEhupxtmRXrJx/4ypmm5Hrdr9c74+/27c2MMC8tfxw3lAGyoVop2YHp1n3Y5Qwa8wrIb3IVIgV+Iy7kg8/8gKwbvI7mLrrsJxE7yPOdS1DSxNVF8l4R13UJHzxZ/+1vovmJAhSPcX6B2/vRyjAzYmErKtsuvCpbRlSrywnWW/1B7w2absv4sphIqqxa4XtHfON9euSbP8XCU4Xu9y+Cjk/9MPlHQSBFUHO2xKM78l1HXdbuElrGWMtWETYePb+lOtDF9xh3sga/5UKU4Lk0ZBLmmOaLIyUhOea2FP4rsSREp6QAI39OCDK2MxLPgM/UFsR996sloeBv6ZvSqh+bUYT8Ldkro2yhnrk+Gs1IrmCwdqr7+elHV/XIiJbvXRu+l1M7XQmAaXZUUc15vc1h1NbnJw8nV44/mVi4EE4SHiO4L0OrFiBFqccnHe2c8rUs9XbjfQIZJH8zacuBPFsMwzQLx7YgGC2g X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 91289e02-1e06-42a0-80d2-08d976cc32a4 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:36:04.6097 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rpN5zDDvWTVwYEsTjXeNxDLixt0cDD3mcOjIRLuF/4gKu5PKSqi19NKLuk/muTARtZZxFjLbTNUJWGNV88D0URaKcXeF8YmqlIrcx9L2wVE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4657 X-Proofpoint-ORIG-GUID: ygx_0Wvk_95wVMZbj164wYNpncNkjtCC X-Proofpoint-GUID: ygx_0Wvk_95wVMZbj164wYNpncNkjtCC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 clxscore=1015 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Andrey Ignatov commit 8ff80e96e3ccea5ff0a890d4f18997e0344dbec2 upstream. Test different scenarios of indirect variable-offset stack access: out of bound access (>0), min_off below initialized part of the stack, max_off+size above initialized part of the stack, initialized stack. Example of output: ... #856/p indirect variable-offset stack access, out of bound OK #857/p indirect variable-offset stack access, max_off+size > max_initialized OK #858/p indirect variable-offset stack access, min_off < min_initialized OK #859/p indirect variable-offset stack access, ok OK ... Signed-off-by: Andrey Ignatov Signed-off-by: Alexei Starovoitov [OP: backport to 4.19] Signed-off-by: Ovidiu Panait --- tools/testing/selftests/bpf/test_verifier.c | 79 ++++++++++++++++++++- 1 file changed, 77 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 6b9ed915c6b0..1ded69b9fd77 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -8495,7 +8495,7 @@ static struct bpf_test tests[] = { .prog_type = BPF_PROG_TYPE_LWT_IN, }, { - "indirect variable-offset stack access", + "indirect variable-offset stack access, out of bound", .insns = { /* Fill the top 8 bytes of the stack */ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0), @@ -8516,10 +8516,85 @@ static struct bpf_test tests[] = { BPF_EXIT_INSN(), }, .fixup_map1 = { 5 }, - .errstr = "variable stack read R2", + .errstr = "invalid stack type R2 var_off", .result = REJECT, .prog_type = BPF_PROG_TYPE_LWT_IN, }, + { + "indirect variable-offset stack access, max_off+size > max_initialized", + .insns = { + /* Fill only the second from top 8 bytes of the stack. */ + BPF_ST_MEM(BPF_DW, BPF_REG_10, -16, 0), + /* Get an unknown value. */ + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), + /* Make it small and 4-byte aligned. */ + BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 4), + BPF_ALU64_IMM(BPF_SUB, BPF_REG_2, 16), + /* Add it to fp. We now have either fp-12 or fp-16, but we don't know + * which. fp-12 size 8 is partially uninitialized stack. + */ + BPF_ALU64_REG(BPF_ADD, BPF_REG_2, BPF_REG_10), + /* Dereference it indirectly. */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map1 = { 5 }, + .errstr = "invalid indirect read from stack var_off", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_LWT_IN, + }, + { + "indirect variable-offset stack access, min_off < min_initialized", + .insns = { + /* Fill only the top 8 bytes of the stack. */ + BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0), + /* Get an unknown value */ + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), + /* Make it small and 4-byte aligned. */ + BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 4), + BPF_ALU64_IMM(BPF_SUB, BPF_REG_2, 16), + /* Add it to fp. We now have either fp-12 or fp-16, but we don't know + * which. fp-16 size 8 is partially uninitialized stack. + */ + BPF_ALU64_REG(BPF_ADD, BPF_REG_2, BPF_REG_10), + /* Dereference it indirectly. */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map1 = { 5 }, + .errstr = "invalid indirect read from stack var_off", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_LWT_IN, + }, + { + "indirect variable-offset stack access, ok", + .insns = { + /* Fill the top 16 bytes of the stack. */ + BPF_ST_MEM(BPF_DW, BPF_REG_10, -16, 0), + BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0), + /* Get an unknown value. */ + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), + /* Make it small and 4-byte aligned. */ + BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 4), + BPF_ALU64_IMM(BPF_SUB, BPF_REG_2, 16), + /* Add it to fp. We now have either fp-12 or fp-16, we don't know + * which, but either way it points to initialized stack. + */ + BPF_ALU64_REG(BPF_ADD, BPF_REG_2, BPF_REG_10), + /* Dereference it indirectly. */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map1 = { 6 }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_LWT_IN, + }, { "direct stack access with 32-bit wraparound. test1", .insns = { From patchwork Mon Sep 13 15:35:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4D7BC433F5 for ; Mon, 13 Sep 2021 15:36:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D041B61051 for ; Mon, 13 Sep 2021 15:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243621AbhIMPho (ORCPT ); Mon, 13 Sep 2021 11:37:44 -0400 Received: from mx0b-0064b401.pphosted.com ([205.220.178.238]:40912 "EHLO mx0b-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242502AbhIMPhh (ORCPT ); Mon, 13 Sep 2021 11:37:37 -0400 Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWGwe016966; Mon, 13 Sep 2021 15:36:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=sPjU47ANvl7sP4znSE4Nyj47R6xAfyf2bv05+es4IQM=; b=quPv1IkncavUxSlD/EaCf7s2cBMhs8qd1eDhApeA2rnNxtIuNKl5Y1RpysYowYIkumSj zlP641xWJvVZvos/z4pnjaCmlfWRlrmv/Epuo+vcvGfA1IzYsKnhhFPGHu5vgndUt6ew WlEJlZKLlS8+uSgW8semq3+j6ByL55L1uFSQHjrkCJYPN6NCuKYX/liOZ3KWEMWAk03F 2qOAcM92i60AxeG1nPJD3OFbpXHgtR6yp8ubKdi3DH2VlBOZaY8U3Tgjr5kEFBClXagq Qb1f9xxh2j5AQhGOUVzUn8z7dNYp85QvevcFq9Pc+nz4F/9GgwuB998QcK3j8X2xOv0B vA== Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2040.outbound.protection.outlook.com [104.47.56.40]) by mx0a-0064b401.pphosted.com with ESMTP id 3b26m1854b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 15:36:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KEMPvp4qfxUpGRKuz7b1pI8BwE0PqK+kUJ2N4JuNSytrwyqiIR+FRrF+FtRF2kt6KkY83uCxIb305dTGqq5Ml02znW79R4XnSRhoArA3oPT6o+qwUxC5kJpin8Zqi9BCNz4USyWYOTWYZ+rotljWTIfbHCq+dDFLJkfqPNdaBOvnzzcIWZel9Soin2ieYFVFzxhoktkrEPB6CFjxeob/2q56VU9IDdyKaqKs++5Z2k8IzjvpKsqEguZGbfsP5KeY9I3ffez5WwjPsX2QRiEvi30j5PWgaU0Om7IoJxP4Dzxt+GSZR53gTtf1geNV5V+1g5Jl+5XR5ip6NOsGDhl7bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=sPjU47ANvl7sP4znSE4Nyj47R6xAfyf2bv05+es4IQM=; b=jIr9z9Ojr6dy4VdNESQHBZ6GsKCpKEaRLnSrYnaLHFEUG1XKUz7+BFJIn42FQ4GBy2eN2KSdzWvAlAh1dHmtfITmXNKfLjehUYnZldOf5ndz4t9djjpeACtN2ym+5GoJjNn+g/bj7ELuHZcboLTfNfh0zcaXrSQS0GJlrXqZTXzf0+d9zvFicE0DJG/xRxhQLWT/HoLMtyOCf0j9HLnZpmZqrkOLxTbvsuB39AMC+7BpZsYnqufX7CIuP9VySjgjLCj2KKH8mjkRUFrGoBeqK35CMu8pO3/9TscvQS10DpJuejI4n1hayEDYvQE6jj/00qKA/5r48+G/I16Uul1j9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM5PR1101MB2201.namprd11.prod.outlook.com (2603:10b6:4:51::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Mon, 13 Sep 2021 15:36:07 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:36:07 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 09/13] selftests/bpf: fix tests due to const spill/fill Date: Mon, 13 Sep 2021 18:35:33 +0300 Message-Id: <20210913153537.2162465-10-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:36:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9e471de9-06dc-4a81-c2e4-08d976cc342c X-MS-TrafficTypeDiagnostic: DM5PR1101MB2201: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:227; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aA4zVqZYFPfpFv8dukmntAs8J95RO9w0sVyBFLKvTCS8VBS/UwdKFpom0i+n3QxTThZdsfq4eVVAimvWwBPMVD3WA3ZDK8Imn3wACY6rGKKOAOHCh04XUKe1/y/Byy40+Njpu4cEfrDK2gZ2nsUMeYbMjk9MAOiCi+jhtQ2NlXRkb0t0A+BPb6QV6bsmjL80heX2+hacDeTD/7JyCR9shQ/jMbt1hgW+vCxEr82srW9xV6/eRo9bVz6+uFHVs+yctws1JxRJGYeXQENavpG0PlX4B0XdMTS+GENqtzuaZ6IeIqL4mFyudF5ksmA7q9e2UMVm2jtEyr2ldaI39SXb5cedhT/jh1h2cxKeHXVqsl7D0hP9wl/PYcokV/yiU/gLQ9ybhSGYEuua404B8PbGRegxs3Mpr2ST64Lmm+VxxfxO4WYmnoYNZIjBACg9/Mun9bmStT5MJt9Sn8ychUBQuFKgmPMb3rWJbq02SFAYeEKL6ghtKRCAieM5BVNFl02VhNvI83GVH89y2jOV1E7l2HGwQ7Mnw5QZ7t8SHoi9aQYnaLk+XcXg5O+ndi3k9QM9V5+sUzrE20eeStNJfacOYqvRX9YxEapwfcht3+lIA9/ULbFU766318bKeZILLcm0zAwaQ7jHGdOMChaSm6wpc7KHZ5Y0hxttcBDVCoReEbUPLQHzXwOxzCFMu6AoWoAwZ2t0FfcoRIzzaWY5/Hbcdw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(366004)(39850400004)(376002)(346002)(52116002)(6916009)(316002)(6486002)(38350700002)(6666004)(86362001)(5660300002)(2616005)(4326008)(8936002)(26005)(38100700002)(6512007)(6506007)(956004)(83380400001)(478600001)(1076003)(36756003)(186003)(44832011)(66476007)(2906002)(8676002)(66946007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gNSFJscMZwOqiBPnPh+F50tv0S3oQBVbcttKWbTl2kc1H3tKUjylNpfRnte66a3earDiIxB/yl33cfb7vknLqo/zEHYZ4ljI6AxPifJYgTzFWUnvtmmMlyFkWRzq5rGzSvtx12gu9BCuWf3304vePwz/+2j8epJokeBGIWab0F1uei74BrION3qsfmlxC5Ndmjm24ThT/6Pu2pMsIjdNV1SaLVvY7zvlZeKbIHySI+Oxe3dVsVZ4m+hkh4KSQ4qSYH7XDl/mmX6jiSzAZQMhLgCYKMXl/zfNc9tidPzpNIu1hKM6km+3FKWPceU88fPzV1grofevtxXUZOzhZAppjh8/BdVweI3U3Tl2CAKpYAqXTta6TDKSU2qXnkKXxGCl9ilo8gdv8Rp0TUloAdb9x98+OgwlhO31d//cd8/n50XAnKV1mADzCcJsNSQDEJeb38kBAho8sd0mR/5y8tI42onrIYhPdHVUexvGBKbV2bdkuTn/KxoYnVFZekW5R1v5CBY5KUML7M/q8iGLhGs0B8EaH+NHnx5RHp/VFAciDu79pvVJZIdVYJsWo+M6wia1HrzU26XlhOCFx6UZ/yq0UNJ06AWuCDnzi/i+XKJbXvLO/QjFtz2anp+ncDDtAhH32tR1p24OD/MwDzkIIoxPKb5Pjr/UVJeISQHaTMV7RJW+VyNnXztPPlRE7teUeSGFUtxw6QlVMzTh7lthfqYrlZea3wJqZHzm2FCbBwPo4HniL2TsJJYTEXWEiyxxfRrzYVSirnt9bYthw7dG44weG3CLdcW3M3Ot5vjHRYknw0ZUBOQiHYBRsR4A8pK/AXB+QRaAn6BhONWmGAh8Pe5YLSYdNb5ovLLZA8Uz8cn4fepb+/fUF3gcbDbMBxcO/0atrifg+VTL6Fl0qWBhYkPH1PeydOboIFGD4khXcZ5MnvlFqE5Vflarhne0R90pPOoXA0VrS3pe2GprPKY3lctGbuYgw/qAgCZlNTA+qOpiyywYI0UjAwgVjmA8kXFeZ7ZtpnlI6B6+gUjx20D8RBQ70AUjkjbf3AnvrVsV5ZhCgEIIAI7xBaAyDxJlL14xLdgdbWwoBOhbUxp2nqI/YXj6+esu0D5dRALU13e1zan0VJJk8HnFTahXzoLVdcsDOHAR2vy6wXOdH/JNyqyzXsEBy7sCYeH9Y5p85CI4KWozY1ekxsUEY11x4JuXGcmhK1oIWmA7j+Ii9X7o3hzHpt7KzcdVpfa0l8eQEH0gMV6lbUca9qDmjjOBzmPahpsul3GiavqYO2X4RzUQ0PluOA+7TjwuMocmgxLURU07Qk0WGXm4/BAauf6jHy9hK0oYVsR9 X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e471de9-06dc-4a81-c2e4-08d976cc342c X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:36:07.1734 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PT5VuN4lFSvJ9IbXu7vMJYYegr3/VCLYGMdNU8QzelRaRFuj7awwz+bWFaIcEGWvkmCEx8kz/9OIglDlVNLVE5GPDqSc4VK7SSphcz7KWc4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1101MB2201 X-Proofpoint-GUID: lgX_8f0NqrfQfO1sz3eGcJgXW5X7tZLB X-Proofpoint-ORIG-GUID: lgX_8f0NqrfQfO1sz3eGcJgXW5X7tZLB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 impostorscore=0 adultscore=0 malwarescore=0 mlxscore=0 suspectscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=667 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Alexei Starovoitov commit fc559a70d57c6ee5443f7a750858503e94cdc941 upstream. fix tests that incorrectly assumed that the verifier cannot track constants through stack. Signed-off-by: Alexei Starovoitov Acked-by: Andrii Nakryiko Signed-off-by: Daniel Borkmann [OP: backport to 4.19] Signed-off-by: Ovidiu Panait --- tools/testing/selftests/bpf/test_verifier.c | 31 +++++++++++---------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 1ded69b9fd77..858e55143233 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -3888,7 +3888,8 @@ static struct bpf_test tests[] = { offsetof(struct __sk_buff, data)), BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_1, offsetof(struct __sk_buff, data_end)), - BPF_MOV64_IMM(BPF_REG_0, 0xffffffff), + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct __sk_buff, mark)), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_0, -8), BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8), BPF_ALU64_IMM(BPF_AND, BPF_REG_0, 0xffff), @@ -6560,9 +6561,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, bitwise AND, zero included", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 64), @@ -6577,9 +6578,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, bitwise AND + JMP, wrong max", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 65), @@ -6653,9 +6654,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, JMP, bounds + offset", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_JMP_IMM(BPF_JGT, BPF_REG_2, 64, 5), @@ -6674,9 +6675,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, JMP, wrong max", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_JMP_IMM(BPF_JGT, BPF_REG_2, 65, 4), @@ -6694,9 +6695,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, JMP, no max check", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_MOV64_IMM(BPF_REG_4, 0), @@ -6714,9 +6715,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, JMP, no min check", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_JMP_IMM(BPF_JGT, BPF_REG_2, 64, 3), @@ -6732,9 +6733,9 @@ static struct bpf_test tests[] = { { "helper access to variable memory: stack, JMP (signed), no min check", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), - BPF_MOV64_IMM(BPF_REG_2, 16), BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, -128), BPF_JMP_IMM(BPF_JSGT, BPF_REG_2, 64, 3), @@ -6776,6 +6777,7 @@ static struct bpf_test tests[] = { { "helper access to variable memory: map, JMP, wrong max", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_6, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), @@ -6783,7 +6785,7 @@ static struct bpf_test tests[] = { BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem), BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 10), BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), - BPF_MOV64_IMM(BPF_REG_2, sizeof(struct test_val)), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_6), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_10, -128), BPF_JMP_IMM(BPF_JSGT, BPF_REG_2, @@ -6795,7 +6797,7 @@ static struct bpf_test tests[] = { BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN(), }, - .fixup_map2 = { 3 }, + .fixup_map2 = { 4 }, .errstr = "invalid access to map value, value_size=48 off=0 size=49", .result = REJECT, .prog_type = BPF_PROG_TYPE_TRACEPOINT, @@ -6830,6 +6832,7 @@ static struct bpf_test tests[] = { { "helper access to variable memory: map adjusted, JMP, wrong max", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_6, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8), BPF_ST_MEM(BPF_DW, BPF_REG_2, 0, 0), @@ -6838,7 +6841,7 @@ static struct bpf_test tests[] = { BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 11), BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 20), - BPF_MOV64_IMM(BPF_REG_2, sizeof(struct test_val)), + BPF_MOV64_REG(BPF_REG_2, BPF_REG_6), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_10, -128), BPF_JMP_IMM(BPF_JSGT, BPF_REG_2, @@ -6850,7 +6853,7 @@ static struct bpf_test tests[] = { BPF_MOV64_IMM(BPF_REG_0, 0), BPF_EXIT_INSN(), }, - .fixup_map2 = { 3 }, + .fixup_map2 = { 4 }, .errstr = "R1 min value is outside of the array range", .result = REJECT, .prog_type = BPF_PROG_TYPE_TRACEPOINT, @@ -6872,8 +6875,8 @@ static struct bpf_test tests[] = { { "helper access to variable memory: size > 0 not allowed on NULL (ARG_PTR_TO_MEM_OR_NULL)", .insns = { + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), BPF_MOV64_IMM(BPF_REG_1, 0), - BPF_MOV64_IMM(BPF_REG_2, 1), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_10, -128), BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 64), @@ -7100,6 +7103,7 @@ static struct bpf_test tests[] = { { "helper access to variable memory: 8 bytes leak", .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1, 8), BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -64), BPF_MOV64_IMM(BPF_REG_0, 0), @@ -7110,7 +7114,6 @@ static struct bpf_test tests[] = { BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_0, -24), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_0, -16), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_0, -8), - BPF_MOV64_IMM(BPF_REG_2, 1), BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_2, -128), BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_10, -128), BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 63), From patchwork Mon Sep 13 15:35:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ovidiu Panait X-Patchwork-Id: 510345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59F4DC433F5 for ; Mon, 13 Sep 2021 15:36:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3DD246103B for ; Mon, 13 Sep 2021 15:36:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235347AbhIMPhq (ORCPT ); Mon, 13 Sep 2021 11:37:46 -0400 Received: from mx0b-0064b401.pphosted.com ([205.220.178.238]:40888 "EHLO mx0b-0064b401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241059AbhIMPhh (ORCPT ); Mon, 13 Sep 2021 11:37:37 -0400 Received: from pps.filterd (m0250812.ppops.net [127.0.0.1]) by mx0a-0064b401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18DFWGwg016966; Mon, 13 Sep 2021 15:36:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=windriver.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=PPS06212021; bh=mteNhSHB5F2WrjqyGdyxY6XvhUEeuyEt61mitW6Sxhs=; b=b0f+EAd2lN1YXrhPOWLajyIJFDH1e+9LnJrLYF1N4ajSfRrZoS6+teR0hRU3zOMceQyd vAUBk61Yh/Zu3CWvZ+RfBclRARwvaX6kbs6H4+LIflCVGoNG791l4Yl1bMEtm9I6RPOe QicZRkzjwhcA5z/PeS0WkoaHcXJ92p0h9JvXL5yMAmCjgKEawoC2Anv2vW5dPBV5E2UJ rLP8bttxl/3jDnR6AOv4p35oLp+2ohgq++xBy3d+WJVOuZOH+XEF1vr/8HedPgU+cBfk jD0m4qA19q4kLCT0AqD2NwslSdR6XSYkVFmYcT2EvaU8SgVwi117wjBhcNUa7dCtv/5J yQ== Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2040.outbound.protection.outlook.com [104.47.56.40]) by mx0a-0064b401.pphosted.com with ESMTP id 3b26m1854b-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Sep 2021 15:36:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GSKX3B1wLt2Tqz8dEyml1GVFPWFcQh9LAU1hXEGnwG9AqtRgfd6PJ15BjQ3/oVU1OEXojWrBZWZQ6iYm05e8umP6JSDcp42J6qU310gdLfYBiAY8NEwT+d4LsTIxHF4wSbYujSEqNGUo6OhuYObquQjNp/+7DPZHYiFtKw+nIq0I9nTQ4ACyDPTwOovbeqUXnvI7HmLB+04uI2Ln4D7yTTGN3q09wJ4eapYnnp0aRuknJVdGRg/wFji70vNLeXANBbJhqYAXukgRjM8nR6S9iSbyh1yu1Z26cEjGS3vFzTGFJt+7fpR5QQN6KNwXHPvvG/2cYYTVWKFhU2x3SQIOrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mteNhSHB5F2WrjqyGdyxY6XvhUEeuyEt61mitW6Sxhs=; b=hc+1vFMTPd4AjzhSoCwkh7KTT5hn/D85EtIJFibrbPmRFGT0LCuC7XUE6jkiBqtglG0X8CGc4wli9HEARSsYdPy5XyZihi05AH7HEHP9M3DYmXG7cUJ9e6Z4hnHlKxzxJtgoWCVhA0FHNoaZGofpIntbwwd1fyrkeBn4XHT+E/0RyaEwJ8BnJ0m1pPXo8icDm6tS1wGuERJVF3OZ0Q8fY/MTxttzKKzAbBgxiulgTr8BtnDQNW3EAHmFE9cndUbacFJ4lHA2+jGInMK1hrbdXRkIZ8lb0dh07HXhnD9wwqMI2qLNfvXUxwkJlgwDK+wCIqurAo3I9DwXPrHengKNSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=windriver.com; dmarc=pass action=none header.from=windriver.com; dkim=pass header.d=windriver.com; arc=none Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=windriver.com; Received: from DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) by DM5PR1101MB2201.namprd11.prod.outlook.com (2603:10b6:4:51::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.15; Mon, 13 Sep 2021 15:36:08 +0000 Received: from DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb]) by DM4PR11MB5327.namprd11.prod.outlook.com ([fe80::952a:bdc6:20c3:94cb%7]) with mapi id 15.20.4500.019; Mon, 13 Sep 2021 15:36:08 +0000 From: Ovidiu Panait To: stable@vger.kernel.org Cc: bpf@vger.kernel.org, daniel@iogearbox.net Subject: [PATCH 4.19 10/13] bpf: Introduce BPF nospec instruction for mitigating Spectre v4 Date: Mon, 13 Sep 2021 18:35:34 +0300 Message-Id: <20210913153537.2162465-11-ovidiu.panait@windriver.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210913153537.2162465-1-ovidiu.panait@windriver.com> References: <20210913153537.2162465-1-ovidiu.panait@windriver.com> X-ClientProxiedBy: VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) To DM4PR11MB5327.namprd11.prod.outlook.com (2603:10b6:5:392::22) MIME-Version: 1.0 Received: from otp-linux03.wrs.com (46.97.150.20) by VI1PR10CA0095.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Mon, 13 Sep 2021 15:36:07 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4af51a2d-3b5e-4302-0088-08d976cc34e2 X-MS-TrafficTypeDiagnostic: DM5PR1101MB2201: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:229; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FtWp3/3grlsP7fiW8uibOM1jbTCOrySykhlPduedfUS5N6eqphRRaGTQsThoSxuQUrJKDTc7aC2GZk889iTIe+Wfa1VVPWwCQfkN6RYjDadNxj9fQYHUuBr02VSaJzmHMDR3HnQE0enQEDUPgQDiPo6id8ghtJcCR1CMSASAGMNpxxuvx7t+T/DRSPL43F7o1A12MXUYpf5a05FMdjaFEOGtT8pWWQADT3+TxQOrxU/iX03+Subqw1uM4ov10VeLzpzqA5DsPPyMg4f1K5zRH3pvey2AmydVTQKL9RrjIWB4ZjZK3B4DNQRnGS1YsGE3Tl2JiMIEkpl3yRRi2ZiR2WzLDjqF4N0FXBXKFlxwmlsSFnMPG6QBCgJFa8xUb9En7AL47/eMQ1cCKXM6hTykGhwjQ0gk0grMCJagYT+VbS32EdAbcv2Yf+0NQr36hvPazEO8cAbuebZqb25crsUay79Fbi5jZb2IpqvbuDPfUiMtmTtR8a9Dn0rl+4NalRx43OkMU0bRN1vIPZhgyPyc8XTSU1I+V5ABV641YZwn74rOZqUv69LWQDITJtOnoM0t6z0MZ3+YjygIvljOpbJAf56QkdhNS9P7YqU5+TlN4vzzhJtlqG8gph0gxTwCsb2Usv9CgX7YRbxjLMpacMUASKFwZXHRVuEEHgUufNi4jtUoV6b8m5BSg4JKGmLYa794uU/rChKriDre+icVGzEOKA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM4PR11MB5327.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(366004)(39850400004)(376002)(346002)(52116002)(6916009)(316002)(6486002)(38350700002)(6666004)(86362001)(5660300002)(2616005)(4326008)(8936002)(26005)(38100700002)(6512007)(6506007)(956004)(83380400001)(478600001)(1076003)(36756003)(186003)(44832011)(66476007)(2906002)(8676002)(66946007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 5OJeizZV6tUhQbNsHM+OPKO6iPeASsGb3NLbyOiyKrwkWJfh3Mj7JqblIt1n0tV5n+XruVvzfcjLSXpvSZL5p2BddxTK/+qfk9KMu3JZEY4DPxOo5h7uu7NhrstBCcVtIMPS8hNidME69UAdrjm6eOtQnaCNn7EtpMn+Bg6PPycS3A00FHvIYhunW3gsm61SPUGp2PCmrw+qrRd9baqwwazVYetRQZOnFCC/1AfDRRPcqObJ3khkm0Gk1zmU0f6vR+9S7BWFZGbW0jHMF9eMfzqeJcKWoUv5C+eMlTEqLUlaQJpzKDPWDB5G0rVVspHFKryFa674iottX9Y2ehgZlZXxAzl10+5ce7iyfbFJn5bOb1E0fbbXtnpy4cwu4DICHr3WNX8PBfp7kway6yBaT5kTMaTuUOfZRexGEws2Qy59zc2bUCxZg1DoGXKVVkcbCJd33K2t0L7pXT1nydM0lswhYpE9gtJpRrnmVcV052w9Q6gWoI0IzsD6rbuisw9v65ZX1jKmaM1re8MY79mDvRqB6L9wRHISB9xKjI4tfA/hd0c1rcFyS1lM0SkA4zqCcJf4A1QtC9nP/3bRum6T8tIuOa1R75q4QfMzLUA9/atDeSgZRlOe1iccCp8kwHH2qNq3XMvy1SO7wD2eBz9DVDIaNE6SHECxKN33be9PlwskYUgp1jzPOL5ggGCf2eb8RQsS1qAm6sfhedAYKIkE3XR8EhZUE7DeKiEokhZcEKrp51WyEyULIRlZPOdxLhLlahFSe+UBfpx9JNnmHV5dVaCF+AEEzKLv0l7vP2/KtKbdXejNQryHEpc5tr8CS4Tbsa4F8sQRoJ2pGwFvfaRLZ4HlcsGHtqSrGZ+2foccn7LJl8Zo1E09852yG7QUWhJAXL9UkOGFamSyqC7KsUCRhWeT7jCmHB4j689PL5VkUN2CVdlJhazKgcIV0rYbq2VJumCWw9p33Ay/l7J5bgXkby0/LgcQ+57F9PHyyDI5K5seLYz/Z/icWzo8DifdQagtrM1obedsUlBbh2yjquURDh+l81eoDuXhtqB5u2FFgG9nrPF37TZ8fAHvmXU9jZuCszIkGvsu98gykHVx5biO196LRgQQUZGL968FhwcRzdzyVl5ym0urIkp6ii7EKWeZgAqRQVE9Sjm/bxiy0ltBOVe2ly6c+DLZxb3n9s8G0IxTno0sx4bvKL7SUBLtsD5d/AOweO9JQajxbHJMlVeQBB/YqzaV0XXN4bXnOBg7Cu/kV2HXOtcyUnWZzIweERO/JtU44f+c7Nd0Rd4ERCxerun/wiUEZ8BZmG2S2JO81TddhdWLkhJPfl0VKAjmkGFu X-OriginatorOrg: windriver.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4af51a2d-3b5e-4302-0088-08d976cc34e2 X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5327.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2021 15:36:08.3821 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ddb2873-a1ad-4a18-ae4e-4644631433be X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zU5qcklWL43LMxe6B1f4KkuH05ZwNjwV432fNGqo8VSmgAJCY3JCIKOa9LSfcCj3JkQguimoEHF+vF68GW6naUFzdOMNnU5VVS//Jp2nLMU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1101MB2201 X-Proofpoint-GUID: n2qKqqFN5Ea6Gca2u8hEFZCe86tJbFLY X-Proofpoint-ORIG-GUID: n2qKqqFN5Ea6Gca2u8hEFZCe86tJbFLY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1, Aquarius:18.0.790, Hydra:6.0.391, FMLib:17.0.607.475 definitions=2021-09-13_07,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 impostorscore=0 adultscore=0 malwarescore=0 mlxscore=0 suspectscore=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130103 Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Daniel Borkmann commit f5e81d1117501546b7be050c5fbafa6efd2c722c upstream. In case of JITs, each of the JIT backends compiles the BPF nospec instruction /either/ to a machine instruction which emits a speculation barrier /or/ to /no/ machine instruction in case the underlying architecture is not affected by Speculative Store Bypass or has different mitigations in place already. This covers both x86 and (implicitly) arm64: In case of x86, we use 'lfence' instruction for mitigation. In case of arm64, we rely on the firmware mitigation as controlled via the ssbd kernel parameter. Whenever the mitigation is enabled, it works for all of the kernel code with no need to provide any additional instructions here (hence only comment in arm64 JIT). Other archs can follow as needed. The BPF nospec instruction is specifically targeting Spectre v4 since i) we don't use a serialization barrier for the Spectre v1 case, and ii) mitigation instructions for v1 and v4 might be different on some archs. The BPF nospec is required for a future commit, where the BPF verifier does annotate intermediate BPF programs with speculation barriers. Co-developed-by: Piotr Krysiuk Co-developed-by: Benedict Schlueter Signed-off-by: Daniel Borkmann Signed-off-by: Piotr Krysiuk Signed-off-by: Benedict Schlueter Acked-by: Alexei Starovoitov [OP: adjusted context for 4.19, drop riscv and ppc32 changes] Signed-off-by: Ovidiu Panait --- arch/arm/net/bpf_jit_32.c | 3 +++ arch/arm64/net/bpf_jit_comp.c | 13 +++++++++++++ arch/mips/net/ebpf_jit.c | 3 +++ arch/powerpc/net/bpf_jit_comp64.c | 6 ++++++ arch/s390/net/bpf_jit_comp.c | 5 +++++ arch/sparc/net/bpf_jit_comp_64.c | 3 +++ arch/x86/net/bpf_jit_comp.c | 7 +++++++ arch/x86/net/bpf_jit_comp32.c | 6 ++++++ include/linux/filter.h | 15 +++++++++++++++ kernel/bpf/core.c | 18 +++++++++++++++++- kernel/bpf/disasm.c | 16 +++++++++------- 11 files changed, 87 insertions(+), 8 deletions(-) diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c index 328ced7bfaf2..79b12e744537 100644 --- a/arch/arm/net/bpf_jit_32.c +++ b/arch/arm/net/bpf_jit_32.c @@ -1578,6 +1578,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx) rn = arm_bpf_get_reg32(src_lo, tmp2[1], ctx); emit_ldx_r(dst, rn, off, ctx, BPF_SIZE(code)); break; + /* speculation barrier */ + case BPF_ST | BPF_NOSPEC: + break; /* ST: *(size *)(dst + off) = imm */ case BPF_ST | BPF_MEM | BPF_W: case BPF_ST | BPF_MEM | BPF_H: diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 7f0258ed1f5f..6876e8205042 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -685,6 +685,19 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx) } break; + /* speculation barrier */ + case BPF_ST | BPF_NOSPEC: + /* + * Nothing required here. + * + * In case of arm64, we rely on the firmware mitigation of + * Speculative Store Bypass as controlled via the ssbd kernel + * parameter. Whenever the mitigation is enabled, it works + * for all of the kernel code with no need to provide any + * additional instructions. + */ + break; + /* ST: *(size *)(dst + off) = imm */ case BPF_ST | BPF_MEM | BPF_W: case BPF_ST | BPF_MEM | BPF_H: diff --git a/arch/mips/net/ebpf_jit.c b/arch/mips/net/ebpf_jit.c index 3832c4628608..947a7172c814 100644 --- a/arch/mips/net/ebpf_jit.c +++ b/arch/mips/net/ebpf_jit.c @@ -1282,6 +1282,9 @@ static int build_one_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, } break; + case BPF_ST | BPF_NOSPEC: /* speculation barrier */ + break; + case BPF_ST | BPF_B | BPF_MEM: case BPF_ST | BPF_H | BPF_MEM: case BPF_ST | BPF_W | BPF_MEM: diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c index 7e3ab477f67f..e7d56ddba43a 100644 --- a/arch/powerpc/net/bpf_jit_comp64.c +++ b/arch/powerpc/net/bpf_jit_comp64.c @@ -596,6 +596,12 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, } break; + /* + * BPF_ST NOSPEC (speculation barrier) + */ + case BPF_ST | BPF_NOSPEC: + break; + /* * BPF_ST(X) */ diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index e42354b15e0b..2a36845dcbc0 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -883,6 +883,11 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, int i break; } break; + /* + * BPF_NOSPEC (speculation barrier) + */ + case BPF_ST | BPF_NOSPEC: + break; /* * BPF_ST(X) */ diff --git a/arch/sparc/net/bpf_jit_comp_64.c b/arch/sparc/net/bpf_jit_comp_64.c index ec4da4dc98f1..1bb1e64d4377 100644 --- a/arch/sparc/net/bpf_jit_comp_64.c +++ b/arch/sparc/net/bpf_jit_comp_64.c @@ -1261,6 +1261,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx) emit(opcode | RS1(src) | rs2 | RD(dst), ctx); break; } + /* speculation barrier */ + case BPF_ST | BPF_NOSPEC: + break; /* ST: *(size *)(dst + off) = imm */ case BPF_ST | BPF_MEM | BPF_W: case BPF_ST | BPF_MEM | BPF_H: diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 924ca27a6139..81c3d4b4c7e2 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -731,6 +731,13 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, } break; + /* speculation barrier */ + case BPF_ST | BPF_NOSPEC: + if (boot_cpu_has(X86_FEATURE_XMM2)) + /* Emit 'lfence' */ + EMIT3(0x0F, 0xAE, 0xE8); + break; + /* ST: *(u8*)(dst_reg + off) = imm */ case BPF_ST | BPF_MEM | BPF_B: if (is_ereg(dst_reg)) diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index adee990abab1..f48300988bc2 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -1683,6 +1683,12 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, i++; break; } + /* speculation barrier */ + case BPF_ST | BPF_NOSPEC: + if (boot_cpu_has(X86_FEATURE_XMM2)) + /* Emit 'lfence' */ + EMIT3(0x0F, 0xAE, 0xE8); + break; /* ST: *(u8*)(dst_reg + off) = imm */ case BPF_ST | BPF_MEM | BPF_H: case BPF_ST | BPF_MEM | BPF_B: diff --git a/include/linux/filter.h b/include/linux/filter.h index 7c84762cb59e..e981bd92a4e3 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -64,6 +64,11 @@ struct sock_reuseport; /* unused opcode to mark call to interpreter with arguments */ #define BPF_CALL_ARGS 0xe0 +/* unused opcode to mark speculation barrier for mitigating + * Speculative Store Bypass + */ +#define BPF_NOSPEC 0xc0 + /* As per nm, we expose JITed images as text (code) section for * kallsyms. That way, tools like perf can find it to match * addresses. @@ -354,6 +359,16 @@ struct sock_reuseport; .off = 0, \ .imm = 0 }) +/* Speculation barrier */ + +#define BPF_ST_NOSPEC() \ + ((struct bpf_insn) { \ + .code = BPF_ST | BPF_NOSPEC, \ + .dst_reg = 0, \ + .src_reg = 0, \ + .off = 0, \ + .imm = 0 }) + /* Internal classic blocks for direct assignment */ #define __BPF_STMT(CODE, K) \ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index d2b6d2459aad..341402bc1202 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -33,6 +33,7 @@ #include #include +#include #include /* Registers */ @@ -1050,6 +1051,7 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) /* Non-UAPI available opcodes. */ [BPF_JMP | BPF_CALL_ARGS] = &&JMP_CALL_ARGS, [BPF_JMP | BPF_TAIL_CALL] = &&JMP_TAIL_CALL, + [BPF_ST | BPF_NOSPEC] = &&ST_NOSPEC, }; #undef BPF_INSN_3_LBL #undef BPF_INSN_2_LBL @@ -1356,7 +1358,21 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn, u64 *stack) JMP_EXIT: return BPF_R0; - /* STX and ST and LDX*/ + /* ST, STX and LDX*/ + ST_NOSPEC: + /* Speculation barrier for mitigating Speculative Store Bypass. + * In case of arm64, we rely on the firmware mitigation as + * controlled via the ssbd kernel parameter. Whenever the + * mitigation is enabled, it works for all of the kernel code + * with no need to provide any additional instructions here. + * In case of x86, we use 'lfence' insn for mitigation. We + * reuse preexisting logic from Spectre v1 mitigation that + * happens to produce the required code on x86 for v4 as well. + */ +#ifdef CONFIG_X86 + barrier_nospec(); +#endif + CONT; #define LDST(SIZEOP, SIZE) \ STX_MEM_##SIZEOP: \ *(SIZE *)(unsigned long) (DST + insn->off) = SRC; \ diff --git a/kernel/bpf/disasm.c b/kernel/bpf/disasm.c index d6b76377cb6e..cbd75dd5992e 100644 --- a/kernel/bpf/disasm.c +++ b/kernel/bpf/disasm.c @@ -171,15 +171,17 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs, else verbose(cbs->private_data, "BUG_%02x\n", insn->code); } else if (class == BPF_ST) { - if (BPF_MODE(insn->code) != BPF_MEM) { + if (BPF_MODE(insn->code) == BPF_MEM) { + verbose(cbs->private_data, "(%02x) *(%s *)(r%d %+d) = %d\n", + insn->code, + bpf_ldst_string[BPF_SIZE(insn->code) >> 3], + insn->dst_reg, + insn->off, insn->imm); + } else if (BPF_MODE(insn->code) == 0xc0 /* BPF_NOSPEC, no UAPI */) { + verbose(cbs->private_data, "(%02x) nospec\n", insn->code); + } else { verbose(cbs->private_data, "BUG_st_%02x\n", insn->code); - return; } - verbose(cbs->private_data, "(%02x) *(%s *)(r%d %+d) = %d\n", - insn->code, - bpf_ldst_string[BPF_SIZE(insn->code) >> 3], - insn->dst_reg, - insn->off, insn->imm); } else if (class == BPF_LDX) { if (BPF_MODE(insn->code) != BPF_MEM) { verbose(cbs->private_data, "BUG_ldx_%02x\n", insn->code);