From patchwork Mon Jul 2 18:11:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 140839 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp246954ljj; Mon, 2 Jul 2018 11:12:07 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfEs6XnALxqr1tWdYWZ5f7QaWsqiGLvP4DoO8jLo4+X1Kiz6mPwatLsHPlQfV6RSRdWaagQ X-Received: by 2002:a62:b29c:: with SMTP id z28-v6mr21753952pfl.8.1530555127366; Mon, 02 Jul 2018 11:12:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530555127; cv=none; d=google.com; s=arc-20160816; b=wnlt5v2s13mXAkR2u6uqFNZuRZvQ3fmJGDy22JGmURhG0dPwG4cZLuReObbDNs/Gqb DfB8H+xrNRdwP/ydlQyCE8UCGVpC0W7TdFWDPvNbh21v4Y4B5yFMQM8o6oHY52DBJayp K123QKpC4nIZBUeD1LyWPDQT30YqsM0KiFjUT0z203JH/fB8Wri/a7Pxl0qNWoBvpdAj vSmzxlu4H9/K8eSIKmHA4djbbREsSowZx0/NBP0zcwTBB5sSPH5jzo0qlsOF2jKAvbZe 5cpto767Bjyx1ZOQSSXmnyBn8hhm4Jp9GrHURyJw1FYXc8pNZBJGMJ0JLxhYlVss6Ubg uZVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=JouXMgpVKc7hntbzheCsZsEE/QB3z+v4ohQdygIzKCMaKXpTBYaBxrWNdkLNaqJDD2 5e8pogpVww+DsHlO0CPeMWDvQs195wd7d1SeK62vPioEETvOq2Po+XeVZEDQcmyzO17z HuT6YpBpSOLM6Sx/47SBJ6ba1mYH/Vd7YZKW7bEm1Rckcr7otUAc/IFx9IgrLDBxFjjR pp5oMvNTxS3c2123tiFA98h3CAsU/4uPWVbobaJyKo6E9CGnIftnFzMb2G4Z/z8jLd9a uZsNc4CvCJ4JMSPYeQ2fCi23cDnaTwx82ufFDjKDY2OHFHQuC26qFabUcN5CcnBfE8k4 B27w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L12Ji9yF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b18-v6si14440380pgs.417.2018.07.02.11.12.07; Mon, 02 Jul 2018 11:12:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L12Ji9yF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753383AbeGBSMC (ORCPT + 31 others); Mon, 2 Jul 2018 14:12:02 -0400 Received: from mail-wr0-f193.google.com ([209.85.128.193]:37072 "EHLO mail-wr0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753348AbeGBSL5 (ORCPT ); Mon, 2 Jul 2018 14:11:57 -0400 Received: by mail-wr0-f193.google.com with SMTP id q10-v6so1707126wrd.4 for ; Mon, 02 Jul 2018 11:11:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=L12Ji9yFkOczS8LxdqLIaOyOwP4xEPBvHP1UgNpl+z3lLalzVbfYOgTI0GKhzcnrls UZIePyE850IJ5e69rkXOVAhCKUpEBVKr0gkM64ZjPQ/ZdROpXtXWMMfaSm7qlG5/qQWo draOD5nv3m+K4RcQdV/waomYm1A6BDXnXjz2s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sN+HT57t+gSH0ShF8Th3JKe4zmSu8oRHrYwUlNvmdrc=; b=WbDaOSrD0F4PEPoi4BxQxX0zFVEfymZ95lJqVuBskyelnSeDDsip6ADDxu9u8wZMTA /HqV7+526jW+T8tCvHNCUr1eTE+qrFL3WhOExob7Uv8yokqmkNEjwvFjT9DWSi4sHa0O C9lYK62xEwGn2jfRWSetey2LHGdyT6BGTB4GhEoaqpFtN9QMYzSgDWzX+OOKc1YMNUBZ uoJSs0z59XK/HkL8kehWc2tEAD1KXiTS7WmWxzvnaWBItc0hFEQZZYBmZz6l3sMBurIK pA4twDVG1LQHtMI/NW6T5XmEyEt+4cgngeTg5g2V2kD1j6vZ9MpcoO27LWmxUznOzhky ZNDg== X-Gm-Message-State: APt69E3YdcBhETmui9dNqV4vPuTJJSTQM8YGVrpyTo1uYUK/ve7Mmbbr rRACgteOiglyOkY2xAqXs7oauw== X-Received: by 2002:adf:83a6:: with SMTP id 35-v6mr20405339wre.13.1530555116389; Mon, 02 Jul 2018 11:11:56 -0700 (PDT) Received: from localhost.localdomain (151.21.90.92.rev.sfr.net. [92.90.21.151]) by smtp.gmail.com with ESMTPSA id 189-v6sm10582822wmd.17.2018.07.02.11.11.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Jul 2018 11:11:55 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-arch@vger.kernel.org Cc: Ard Biesheuvel , Arnd Bergmann , Heiko Carstens , Kees Cook , Will Deacon , Thomas Gleixner , Catalin Marinas , Ingo Molnar , Steven Rostedt , Martin Schwidefsky , Jessica Yu , Peter Zijlstra Subject: [PATCH v2 2/8] kernel/jump_label: implement generic support for relative references Date: Mon, 2 Jul 2018 20:11:39 +0200 Message-Id: <20180702181145.4799-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702181145.4799-1-ard.biesheuvel@linaro.org> References: <20180702181145.4799-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To reduce the size taken up by absolute references in jump label entries themselves and the associated relocation records in the .init segment, add support for emitting them as relative references instead. Note that this requires some extra care in the sorting routine, given that the offsets change when entries are moved around in the jump_entry table. Signed-off-by: Ard Biesheuvel --- arch/Kconfig | 3 +++ include/linux/jump_label.h | 28 ++++++++++++++++++++ kernel/jump_label.c | 22 ++++++++++++++- 3 files changed, 52 insertions(+), 1 deletion(-) -- 2.17.1 diff --git a/arch/Kconfig b/arch/Kconfig index 2b8b70820002..22fa3792626e 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -348,6 +348,9 @@ config HAVE_PERF_USER_STACK_DUMP config HAVE_ARCH_JUMP_LABEL bool +config HAVE_ARCH_JUMP_LABEL_RELATIVE + bool + config HAVE_RCU_TABLE_FREE bool diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index 4603a1c88e48..871826fd0c3b 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -121,6 +121,32 @@ struct static_key { #include #ifndef __ASSEMBLY__ +#ifdef CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE + +struct jump_entry { + s32 code; + s32 target; + long key; // key may be far away from the core kernel under KASLR +}; + +static inline unsigned long jump_entry_code(const struct jump_entry *entry) +{ + return (unsigned long)&entry->code + entry->code; +} + +static inline unsigned long jump_entry_target(const struct jump_entry *entry) +{ + return (unsigned long)&entry->target + entry->target; +} + +static inline struct static_key *jump_entry_key(const struct jump_entry *entry) +{ + long offset = entry->key & ~1L; + + return (struct static_key *)((unsigned long)&entry->key + offset); +} + +#else static inline unsigned long jump_entry_code(const struct jump_entry *entry) { @@ -137,6 +163,8 @@ static inline struct static_key *jump_entry_key(const struct jump_entry *entry) return (struct static_key *)((unsigned long)entry->key & ~1UL); } +#endif + static inline bool jump_entry_is_branch(const struct jump_entry *entry) { return (unsigned long)entry->key & 1UL; diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 8a3ac4f5f490..d424e1d22d63 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -47,14 +47,34 @@ static int jump_label_cmp(const void *a, const void *b) return 0; } +static void jump_label_swap(void *a, void *b, int size) +{ + long delta = (unsigned long)a - (unsigned long)b; + struct jump_entry *jea = a; + struct jump_entry *jeb = b; + struct jump_entry tmp = *jea; + + jea->code = jeb->code - delta; + jea->target = jeb->target - delta; + jea->key = jeb->key - delta; + + jeb->code = tmp.code + delta; + jeb->target = tmp.target + delta; + jeb->key = tmp.key + delta; +} + static void jump_label_sort_entries(struct jump_entry *start, struct jump_entry *stop) { unsigned long size; + void *swapfn = NULL; + + if (IS_ENABLED(CONFIG_HAVE_ARCH_JUMP_LABEL_RELATIVE)) + swapfn = jump_label_swap; size = (((unsigned long)stop - (unsigned long)start) / sizeof(struct jump_entry)); - sort(start, size, sizeof(struct jump_entry), jump_label_cmp, NULL); + sort(start, size, sizeof(struct jump_entry), jump_label_cmp, swapfn); } static void jump_label_update(struct static_key *key);