From patchwork Thu Sep 17 19:38:55 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 53841 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f198.google.com (mail-wi0-f198.google.com [209.85.212.198]) by patches.linaro.org (Postfix) with ESMTPS id 6827922E57 for ; Thu, 17 Sep 2015 19:57:12 +0000 (UTC) Received: by wicmn1 with SMTP id mn1sf1333724wic.1 for ; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:cc:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list; bh=qXUSmewY0wHcFMoPRW9G+cnTjv6MI9KnW872IenRiEk=; b=cMiAMYKF/bzHbmv5orfH6U9muV1SEh+rL9SKilWNYI7IK8BHOMMFpHSg1chcgd4BHO JZsKk+q6zcOgxLOBQF1yuZnoRoCcWQSBC4MXD2E2AA2mx5KjQ9wnS9GyNT0PxSCddsTS t1QSi6t/xcftF1sRKcxC2u378Mwtld8Or/LysUJAqqsNlIpoJ1aMRZ08QHd3IH4yuQ1I J2h32460ayozeQxsgxMy98bcFqqcFWRWXiGkdWVdGlzwMwH6CjY/kKD4q6sUx2yeGBdN 0xKCpraXGJqpuV6VQg08PC8ajQ6BGs3HaP+JPUu3cXMGvGhwYF2j6MELZ6F62Ufcabf7 Potw== X-Gm-Message-State: ALoCoQnKvxTGf3scIVFo7mL9Iv7Gt2QUQpHpBl3P4i02+MS/IYGU+K3Jb+6uj+OQRMfZXSshQh+V X-Received: by 10.180.96.226 with SMTP id dv2mr1128017wib.2.1442519831721; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.204.200 with SMTP id la8ls124831lac.96.gmail; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) X-Received: by 10.112.53.231 with SMTP id e7mr1117720lbp.103.1442519831565; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) Received: from mail-lb0-f179.google.com (mail-lb0-f179.google.com. [209.85.217.179]) by mx.google.com with ESMTPS id s5si3316202lbw.89.2015.09.17.12.57.11 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 Sep 2015 12:57:11 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) client-ip=209.85.217.179; Received: by lbpo4 with SMTP id o4so15063341lbp.2 for ; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) X-Received: by 10.152.170.225 with SMTP id ap1mr1193303lac.72.1442519831442; Thu, 17 Sep 2015 12:57:11 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp3261755lbq; Thu, 17 Sep 2015 12:57:10 -0700 (PDT) X-Received: by 10.107.129.205 with SMTP id l74mr9509734ioi.181.1442519830378; Thu, 17 Sep 2015 12:57:10 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id c192si3898454ioe.50.2015.09.17.12.57.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 Sep 2015 12:57:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZcfH3-00074a-Fh; Thu, 17 Sep 2015 19:55:17 +0000 Received: from merlin.infradead.org ([2001:4978:20e::2]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Zcf1o-0004XV-Af for linux-arm-kernel@bombadil.infradead.org; Thu, 17 Sep 2015 19:39:32 +0000 Received: from mail-wi0-f171.google.com ([209.85.212.171]) by merlin.infradead.org with esmtps (Exim 4.85 #2 (Red Hat Linux)) id 1Zcf1m-0007ab-Ry for linux-arm-kernel@lists.infradead.org; Thu, 17 Sep 2015 19:39:31 +0000 Received: by wicfx3 with SMTP id fx3so37585873wic.1 for ; Thu, 17 Sep 2015 12:39:07 -0700 (PDT) X-Received: by 10.194.158.68 with SMTP id ws4mr1750196wjb.25.1442518747448; Thu, 17 Sep 2015 12:39:07 -0700 (PDT) Received: from localhost.localdomain (228.202.154.77.rev.sfr.net. [77.154.202.228]) by smtp.gmail.com with ESMTPSA id lf10sm4989873wjb.23.2015.09.17.12.39.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 17 Sep 2015 12:39:06 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, will.deacon@arm.com, catalin.marinas@arm.com Subject: [RFC PATCH 2/2] arm64: errata: add module load workaround for erratum #843419 Date: Thu, 17 Sep 2015 21:38:55 +0200 Message-Id: <1442518735-16625-3-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1442518735-16625-1-git-send-email-ard.biesheuvel@linaro.org> References: <1442518735-16625-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150917_153931_027154_425851A1 X-CRM114-Status: GOOD ( 25.47 ) X-Spam-Score: -2.6 (--) X-Spam-Report: SpamAssassin version 3.4.1 on merlin.infradead.org summary: Content analysis details: (-2.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.212.171 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.212.171 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Cc: andre.przywara@arm.com, Ard Biesheuvel , suzuki.poulose@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.179 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 In order to work around Cortex-A35 erratum #843419, this patch updates the module loading logic to either change potentially problematic adrp instructions into adr instructions (if the symbol turns out to be in range), or emit a veneer that is guaranteed to be at an offset that does not trigger the issue. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 17 ++++++++++ arch/arm64/include/asm/veneers.h | 19 +++++++++++ arch/arm64/kernel/module.c | 33 +++++++++++++++++++ arch/arm64/kernel/veneers.c | 34 ++++++++++++++++++++ 4 files changed, 103 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 115586d8299b..57e45e77d7e3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -338,6 +338,23 @@ config ARM64_MODULE_VENEERS bool depends on MODULES +config ARM64_ERRATUM_843419 + bool "Cortex-A53: 843419: A load or store might access an incorrect address" + depends on MODULES + select ARM64_MODULE_VENEERS + default y + help + This option builds kernel modules using the large memory model in + order to avoid the use of the ADRP instruction, which can cause + a subsequent memory access to use an incorrect address on Cortex-A53 + parts up to r0p4. + + Note that the kernel itself must be linked with a version of ld + which fixes potentially affected ADRP instructions through the + use of veneers. + + If unsure, say Y. + choice prompt "Page size" default ARM64_4K_PAGES diff --git a/arch/arm64/include/asm/veneers.h b/arch/arm64/include/asm/veneers.h new file mode 100644 index 000000000000..4ee6efe4f5a1 --- /dev/null +++ b/arch/arm64/include/asm/veneers.h @@ -0,0 +1,19 @@ + +#include + +struct veneer_erratum_843419 { + u32 adrp; + u32 branch; +}; + +static inline bool erratum_843419_affects_adrp_insn(void *addr) +{ + /* + * The workaround for erratum 843419 only needs to be + * applied if the adrp instruction appears in either of + * the last two instruction slots in the 4 KB page. + */ + return ((u64)addr % SZ_4K) >= (SZ_4K - 8); +} + +void *emit_erratum_843419_veneer(struct module *mod, u32 *insn); diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 67bf4107f6ef..5307d08f15e8 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -28,6 +28,7 @@ #include #include #include +#include #define AARCH64_INSN_IMM_MOVNZ AARCH64_INSN_IMM_MAX #define AARCH64_INSN_IMM_MOVK AARCH64_INSN_IMM_16 @@ -335,6 +336,38 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, case R_AARCH64_ADR_PREL_PG_HI21_NC: overflow_check = false; case R_AARCH64_ADR_PREL_PG_HI21: +#ifdef CONFIG_ARM64_ERRATUM_843419 + /* + * TODO check for presence of affected A53 cores + */ + if (erratum_843419_affects_adrp_insn(loc)) { + struct veneer_erratum_843419 *v; + + /* + * This adrp instruction appears at an offset + * that may be problematic on older Cortex-A53 + * cores. So first, try to convert it into a + * simple adr instruction. + */ + ovf = reloc_insn_imm(RELOC_OP_PREL, loc, + val & ~(SZ_4K - 1), 0, 21, + AARCH64_INSN_IMM_ADR); + if (ovf == 0) { + /* success! convert adrp -> adr */ + *(u32 *)loc &= 0x7fffffff; + break; + } else { + /* symbol out of range -> emit veneer */ + v = emit_erratum_843419_veneer(me, loc); + *(u32 *)loc = aarch64_insn_gen_branch_imm( + (unsigned long)loc, + (unsigned long)v, + AARCH64_INSN_BRANCH_NOLINK); + loc = &v->adrp; + } + /* fall through */ + } +#endif ovf = reloc_insn_imm(RELOC_OP_PAGE, loc, val, 12, 21, AARCH64_INSN_IMM_ADR); break; diff --git a/arch/arm64/kernel/veneers.c b/arch/arm64/kernel/veneers.c index 0a33a63a9b46..1b708d6a021a 100644 --- a/arch/arm64/kernel/veneers.c +++ b/arch/arm64/kernel/veneers.c @@ -10,6 +10,8 @@ #include #include +#include + static bool in_init(const struct module *mod, u64 addr) { return addr - (u64)mod->module_init < mod->init_size; @@ -31,6 +33,30 @@ static void __maybe_unused *alloc_veneer(struct module *mod, u64 loc, int size) return ret; } +#ifdef CONFIG_ARM64_ERRATUM_843419 +void *emit_erratum_843419_veneer(struct module *mod, u32 *insn) +{ + struct veneer_erratum_843419 *veneer; + + veneer = alloc_veneer(mod, (u64)insn, 2 * sizeof(*veneer)); + if (erratum_843419_affects_adrp_insn(&veneer->adrp)) + /* + * We allocated a veneer that is susceptible to the same problem + * as the original location. We allocated twice the space, so + * just advance to the next slot. + */ + veneer++; + + veneer->adrp = *insn; + veneer->branch = aarch64_insn_gen_branch_imm( + (unsigned long)&veneer->branch, + (unsigned long)(insn + 1), + AARCH64_INSN_BRANCH_NOLINK); + + return veneer; +} +#endif + /* estimate the maximum size of the veneer for this relocation */ static unsigned long get_veneers_size(Elf64_Addr base, const Elf64_Rel *rel, int num) @@ -40,6 +66,14 @@ static unsigned long get_veneers_size(Elf64_Addr base, const Elf64_Rel *rel, for (i = 0; i < num; i++) switch (ELF64_R_TYPE(rel[i].r_info)) { + case R_AARCH64_ADR_PREL_PG_HI21_NC: + case R_AARCH64_ADR_PREL_PG_HI21: +#ifdef CONFIG_ARM64_ERRATUM_843419 + if (erratum_843419_affects_adrp_insn((void *)base + + rel[i].r_offset)) + ret += 2 * sizeof(struct veneer_erratum_843419); +#endif + break; } return ret; }