From patchwork Mon Jul 2 18:11:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 140848 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp248954ljj; Mon, 2 Jul 2018 11:14:13 -0700 (PDT) X-Google-Smtp-Source: AAOMgpcz/TVoZ6RwrZN+tKHuT8TrFBHhu4KmMtb+U8mjU3pkS9kgyLC6IpcDUEN+ZSzDJWI9N5I1 X-Received: by 2002:a62:98d6:: with SMTP id d83-v6mr17842733pfk.186.1530555253005; Mon, 02 Jul 2018 11:14:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530555252; cv=none; d=google.com; s=arc-20160816; b=jH92wHNzIiqDxT8HaqvDAE9EcnDe7ktzm0byJCW/x6NNEMDrLFU697MQuTYzM5biiK ZBWAE2yWuE4cAEwv1fQrd2te7kKUPZb4UxnbH2qAXIyuhQMsEcW8FzF1OTr+ygu1URrP W0JfjR7b6Ie4xew0u26qCC7RfBK8xRfV4AMStGxz+BxMtdS4f/L2cDbmqAg1zz3ce20K wBCDnToZt540zuVuYwb3aAEb/A7Wgqn29ZXM8Bnj++QS5NI4B8+4qIWt2rvn8EoeFC57 yuZCvkB2Skbet9aM2e4vcLHPQbcxPYw17ng1ZXeRpHGxA3gLwqWO86RLVWp1s1UHcjAL dwAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=w4lxjj61CeIJtbmnFVjG2GWM9frrLYdO8XaMvDkK3nI=; b=cP6gwpWx1WHmhQgDil96AbuzZh0agWFs3qZ7vKw09mNxYB/jScVhI35oTawHInzgRd VkajA1BgpbkdlPn9mBhyDM4zs1oxr9GxYyHhnWdqciuNu3galMo5/61EqCqx9ADRYvqA Y6fy/7BJKPI+C5GJn4BSrP++Zyw1H8mw8qe6JskCoFI5y8etdZzxHfgsNxZEv1rUf6ls ROqLheG2L5W0e45YTRyYumyriQwbnWJdBx6O5vtJDYaIZM0fEep8ScAv9kZvECPre60N X5nSrZspFymCwihLbUV0FGlW44/D0tLQtT6kV8iCO8idOSAFFt9hAzLdYGsBq8sIDQSG yPxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GymrCStW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 6-v6si16787191plb.409.2018.07.02.11.14.12; Mon, 02 Jul 2018 11:14:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GymrCStW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932148AbeGBSOK (ORCPT + 31 others); Mon, 2 Jul 2018 14:14:10 -0400 Received: from mail-wr0-f194.google.com ([209.85.128.194]:44074 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753384AbeGBSME (ORCPT ); Mon, 2 Jul 2018 14:12:04 -0400 Received: by mail-wr0-f194.google.com with SMTP id p12-v6so16417898wrn.11 for ; Mon, 02 Jul 2018 11:12:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=w4lxjj61CeIJtbmnFVjG2GWM9frrLYdO8XaMvDkK3nI=; b=GymrCStWjyo2SvoZDUrwyeQXJkBuT4se9Fg+tlEcaSo8Aym536nvFRSHOd7XtAmMya 1cL8Wd/1/asNFKwEGUHVvxVPJM+2IlGnMiza3uAWHP5OVn+C/2H3QjewoJJxBO7xHlhd OpHAV/O6SWI/9zM3HEWvMLl+wwzB48KZ/wPpM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=w4lxjj61CeIJtbmnFVjG2GWM9frrLYdO8XaMvDkK3nI=; b=VJwSMMdzxJnEMMEYtkTPBIvXFqIgefpV3UKSGs2UPziUPE0l38sw+ynk4shM8bnmP6 XppepRi7mocWGhN3BD+7HsOD0oOl8csYRwpYvcksdXjcT9WVwg9O8e//MeagY3C2LW6g ClAM0oORmUx0pLV7ig+sOlfoxXM3uLb4W6CY60RhvFnX1koff4uhmBdDGxPyYSfV57Gm OP1gWv4XdgGqeHb80nx3htLmUaRGN63HzoYG7T3n5aDBLLto0oZo77Qr8kjRrCAs3eve zlEUb7l+od0+799vBVslKNzNxQhhJ6la5/A7lMGzo+2wlnisoD3I0ODGHx8JYDdrajcx HExQ== X-Gm-Message-State: APt69E0hCe+Uy6dHdJcNQg+I/WIL3aPc08lhJYrBwU3nqRCC/eWOTkQe zv7PNhbHEw7yPbnhVudwyOqVHQ== X-Received: by 2002:adf:8e49:: with SMTP id n67-v6mr19861946wrb.131.1530555123423; Mon, 02 Jul 2018 11:12:03 -0700 (PDT) Received: from localhost.localdomain (151.21.90.92.rev.sfr.net. [92.90.21.151]) by smtp.gmail.com with ESMTPSA id 189-v6sm10582822wmd.17.2018.07.02.11.12.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Jul 2018 11:12:02 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-arch@vger.kernel.org Cc: Ard Biesheuvel , Arnd Bergmann , Heiko Carstens , Kees Cook , Will Deacon , Thomas Gleixner , Catalin Marinas , Ingo Molnar , Steven Rostedt , Martin Schwidefsky , Jessica Yu , Peter Zijlstra Subject: [PATCH v2 5/8] x86: jump_label: switch to jump_entry accessors Date: Mon, 2 Jul 2018 20:11:42 +0200 Message-Id: <20180702181145.4799-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702181145.4799-1-ard.biesheuvel@linaro.org> References: <20180702181145.4799-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation of switching x86 to use place-relative references for the code, target and key members of struct jump_entry, replace direct references to the struct members with invocations of the new accessors. This will allow us to make the switch by modifying the accessors only. This incorporates a cleanup of __jump_label_transform() proposed by Peter. Cc: Peter Zijlstra Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/jump_label.c | 62 ++++++++------------ 1 file changed, 25 insertions(+), 37 deletions(-) -- 2.17.1 diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index e56c95be2808..023768222a94 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -42,52 +42,37 @@ static void __jump_label_transform(struct jump_entry *entry, void *(*poker)(void *, const void *, size_t), int init) { - union jump_code_union code; + union jump_code_union jmp; const unsigned char default_nop[] = { STATIC_KEY_INIT_NOP }; const unsigned char *ideal_nop = ideal_nops[NOP_ATOMIC5]; + const void *expect, *code; + int line; + + jmp.jump = 0xe9; + jmp.offset = jump_entry_target(entry) - + (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); if (type == JUMP_LABEL_JMP) { if (init) { - /* - * Jump label is enabled for the first time. - * So we expect a default_nop... - */ - if (unlikely(memcmp((void *)entry->code, default_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - /* - * ...otherwise expect an ideal_nop. Otherwise - * something went horribly wrong. - */ - if (unlikely(memcmp((void *)entry->code, ideal_nop, 5) - != 0)) - bug_at((void *)entry->code, __LINE__); + expect = ideal_nop; line = __LINE__; } - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); + code = &jmp.code; } else { - /* - * We are disabling this jump label. If it is not what - * we think it is, then something must have gone wrong. - * If this is the first initialization call, then we - * are converting the default nop to the ideal nop. - */ if (init) { - if (unlikely(memcmp((void *)entry->code, default_nop, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = default_nop; line = __LINE__; } else { - code.jump = 0xe9; - code.offset = entry->target - - (entry->code + JUMP_LABEL_NOP_SIZE); - if (unlikely(memcmp((void *)entry->code, &code, 5) != 0)) - bug_at((void *)entry->code, __LINE__); + expect = &jmp.code; line = __LINE__; } - memcpy(&code, ideal_nops[NOP_ATOMIC5], JUMP_LABEL_NOP_SIZE); + + code = ideal_nop; } + if (memcmp((void *)jump_entry_code(entry), expect, JUMP_LABEL_NOP_SIZE)) + bug_at((void *)jump_entry_code(entry), line); + /* * Make text_poke_bp() a default fallback poker. * @@ -96,11 +81,14 @@ static void __jump_label_transform(struct jump_entry *entry, * always nop being the 'currently valid' instruction * */ - if (poker) - (*poker)((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE); - else - text_poke_bp((void *)entry->code, &code, JUMP_LABEL_NOP_SIZE, - (void *)entry->code + JUMP_LABEL_NOP_SIZE); + if (poker) { + (*poker)((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); + return; + } + + text_poke_bp((void *)jump_entry_code(entry), code, JUMP_LABEL_NOP_SIZE, + (void *)jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); } void arch_jump_label_transform(struct jump_entry *entry,