From patchwork Tue Jun 5 15:22:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 137759 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp1098906lji; Tue, 5 Jun 2018 08:26:04 -0700 (PDT) X-Google-Smtp-Source: ADUXVKJowUArdvj+IFBZ4E9AqkxJvAEdak2XUCAotyhTZr2k1OSUmHtIjsXj7JXNqiksMgKo1++F X-Received: by 2002:a24:243:: with SMTP id 64-v6mr18467181itu.20.1528212364430; Tue, 05 Jun 2018 08:26:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528212364; cv=none; d=google.com; s=arc-20160816; b=BkzeunCim0z4idskGhFfClp0RtdYVS5mfEL5YcVRkbJl5zJJY7j2AQrwIJib5qVWY3 yvJ5IP+2qmPx+G7U9X1jKAja2JDvMcZiBQd1eJZpSOX39upwLzArKFY/CgwvBQ0IkkcG oZYuwWqOt00YmpSwjkQQwLPbLgHrYAyLO6OxOFrZu9UQOMLIM7Bgny2sBcsjWvqYkjlV 3QqV4pJg9zGNGhp0wwU9rW5Yr7vRQlKHuKLo750XsIkdziD93ty4aDm5i5Gi3e6ZYyp2 jzz5ZjE2G4fLQ8zHXXBgypwd3XfkdJ8yBEtvSxkL/UODUXtLeE/Dk2und1h4noObUo5H RvIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from :arc-authentication-results; bh=TOWGXZUFkWM4cifKOWg8lYWbqud3GPyZRBgJQT0NJ2A=; b=AjNr6mJkeedbtLdgMricvKFyT7QeAni4JwY4khxdSmYQyJpebQrrYQIqlvWUF8RuGA Ewm0ba18+Jm3pWlHFPRoNC5T31COciSgLRE8IvvAaHgMPXQ7k2Xp1VvgH8mXd8BiHjQW dyqd/phYMkGG1JQANOH5q+TQYGYZXfi1j9K73hubs8ET6+akDeaisTdrmo43Fr8aPiKi wF/g7CnqkgQpuZdPg7qcZYizX0EV/VUGKUQflag3FmG1yuHzJ/nX/x2IuVnLqmwg36UL esSAmHwVMQSarAUn2hWE+mKd2T+i5tpKtlXnKxiYTV4B1Kw7ky3Jz/Z6U+5vRuRhEoi/ 89IQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id 187-v6si1781332itm.133.2018.06.05.08.26.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 05 Jun 2018 08:26:04 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1fQDo3-0005Ch-6T; Tue, 05 Jun 2018 15:23:31 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1fQDo1-0005C8-HS for xen-devel@lists.xenproject.org; Tue, 05 Jun 2018 15:23:29 +0000 X-Inumbo-ID: 155bf193-68d4-11e8-9728-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 155bf193-68d4-11e8-9728-bc764e045a96; Tue, 05 Jun 2018 17:21:13 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B175715B2; Tue, 5 Jun 2018 08:23:27 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.206.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C61C23F557; Tue, 5 Jun 2018 08:23:26 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 5 Jun 2018 16:22:58 +0100 Message-Id: <20180605152303.14450-9-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180605152303.14450-1-julien.grall@arm.com> References: <20180605152303.14450-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v1 08/13] xen/arm: alternatives: Add dynamic patching feature X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andre.przywara@arm.com, Julien Grall , sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This is based on the Linux commit dea5e2a4c5bc "arm64: alternatives: Add dynamic patching feature" written by Marc Zyngier: We've so far relied on a patching infrastructure that only gave us a single alternative, without any way to provide a range of potential replacement instructions. For a single feature, this is an all or nothing thing. It would be interesting to have a more flexible grained way of patching the kernel though, where we could dynamically tune the code that gets injected. In order to achive this, let's introduce a new form of dynamic patching, assiciating a callback to a patching site. This callback gets source and target locations of the patching request, as well as the number of instructions to be patched. Dynamic patching is declared with the new ALTERNATIVE_CB and alternative_cb directives: asm volatile(ALTERNATIVE_CB("mov %0, #0\n", callback) : "r" (v)); or alternative_cb callback mov x0, #0 alternative_cb_end where callback is the C function computing the alternative. Reviewed-by: Christoffer Dall Reviewed-by: Catalin Marinas Signed-off-by: Marc Zyngier This is part of XSA-263. Signed-off-by: Julien Grall Acked-by: Stefano Stabellini --- Changes in v2: - Fix typo in the commit message - Add Stefano's acked-by --- xen/arch/arm/alternative.c | 48 +++++++++++++++++++++++++++++---------- xen/include/asm-arm/alternative.h | 44 +++++++++++++++++++++++++++++++---- 2 files changed, 75 insertions(+), 17 deletions(-) diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c index 936cf04956..52ed7edf69 100644 --- a/xen/arch/arm/alternative.c +++ b/xen/arch/arm/alternative.c @@ -30,6 +30,8 @@ #include #include #include +/* XXX: Move ARCH_PATCH_INSN_SIZE out of livepatch.h */ +#include #include /* Override macros from asm/page.h to make them work with mfn_t */ @@ -94,6 +96,23 @@ static u32 get_alt_insn(const struct alt_instr *alt, return insn; } +static void patch_alternative(const struct alt_instr *alt, + const uint32_t *origptr, + uint32_t *updptr, int nr_inst) +{ + const uint32_t *replptr; + unsigned int i; + + replptr = ALT_REPL_PTR(alt); + for ( i = 0; i < nr_inst; i++ ) + { + uint32_t insn; + + insn = get_alt_insn(alt, origptr + i, replptr + i); + updptr[i] = cpu_to_le32(insn); + } +} + /* * The region patched should be read-write to allow __apply_alternatives * to replacing the instructions when necessary. @@ -105,33 +124,38 @@ static int __apply_alternatives(const struct alt_region *region, paddr_t update_offset) { const struct alt_instr *alt; - const u32 *replptr, *origptr; + const u32 *origptr; u32 *updptr; + alternative_cb_t alt_cb; printk(XENLOG_INFO "alternatives: Patching with alt table %p -> %p\n", region->begin, region->end); for ( alt = region->begin; alt < region->end; alt++ ) { - u32 insn; - int i, nr_inst; + int nr_inst; - if ( !cpus_have_cap(alt->cpufeature) ) + /* Use ARM_CB_PATCH as an unconditional patch */ + if ( alt->cpufeature < ARM_CB_PATCH && + !cpus_have_cap(alt->cpufeature) ) continue; - BUG_ON(alt->alt_len != alt->orig_len); + if ( alt->cpufeature == ARM_CB_PATCH ) + BUG_ON(alt->alt_len != 0); + else + BUG_ON(alt->alt_len != alt->orig_len); origptr = ALT_ORIG_PTR(alt); updptr = (void *)origptr + update_offset; - replptr = ALT_REPL_PTR(alt); - nr_inst = alt->alt_len / sizeof(insn); + nr_inst = alt->orig_len / ARCH_PATCH_INSN_SIZE; - for ( i = 0; i < nr_inst; i++ ) - { - insn = get_alt_insn(alt, origptr + i, replptr + i); - *(updptr + i) = cpu_to_le32(insn); - } + if ( alt->cpufeature < ARM_CB_PATCH ) + alt_cb = patch_alternative; + else + alt_cb = ALT_REPL_PTR(alt); + + alt_cb(alt, origptr, updptr, nr_inst); /* Ensure the new instructions reached the memory and nuke */ clean_and_invalidate_dcache_va_range(origptr, diff --git a/xen/include/asm-arm/alternative.h b/xen/include/asm-arm/alternative.h index 4e33d1cdf7..9b4b02811b 100644 --- a/xen/include/asm-arm/alternative.h +++ b/xen/include/asm-arm/alternative.h @@ -3,6 +3,8 @@ #include +#define ARM_CB_PATCH ARM_NCAPS + #ifndef __ASSEMBLY__ #include @@ -18,16 +20,24 @@ struct alt_instr { }; /* Xen: helpers used by common code. */ -#define __ALT_PTR(a,f) ((u32 *)((void *)&(a)->f + (a)->f)) +#define __ALT_PTR(a,f) ((void *)&(a)->f + (a)->f) #define ALT_ORIG_PTR(a) __ALT_PTR(a, orig_offset) #define ALT_REPL_PTR(a) __ALT_PTR(a, alt_offset) +typedef void (*alternative_cb_t)(const struct alt_instr *alt, + const uint32_t *origptr, uint32_t *updptr, + int nr_inst); + void __init apply_alternatives_all(void); int apply_alternatives(const struct alt_instr *start, const struct alt_instr *end); -#define ALTINSTR_ENTRY(feature) \ +#define ALTINSTR_ENTRY(feature, cb) \ " .word 661b - .\n" /* label */ \ + " .if " __stringify(cb) " == 0\n" \ " .word 663f - .\n" /* new instruction */ \ + " .else\n" \ + " .word " __stringify(cb) "- .\n" /* callback */ \ + " .endif\n" \ " .hword " __stringify(feature) "\n" /* feature bit */ \ " .byte 662b-661b\n" /* source len */ \ " .byte 664f-663f\n" /* replacement len */ @@ -45,15 +55,18 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en * but most assemblers die if insn1 or insn2 have a .inst. This should * be fixed in a binutils release posterior to 2.25.51.0.2 (anything * containing commit 4e4d08cf7399b606 or c1baaddf8861). + * + * Alternatives with callbacks do not generate replacement instructions. */ -#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled) \ +#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled, cb) \ ".if "__stringify(cfg_enabled)" == 1\n" \ "661:\n\t" \ oldinstr "\n" \ "662:\n" \ ".pushsection .altinstructions,\"a\"\n" \ - ALTINSTR_ENTRY(feature) \ + ALTINSTR_ENTRY(feature,cb) \ ".popsection\n" \ + " .if " __stringify(cb) " == 0\n" \ ".pushsection .altinstr_replacement, \"a\"\n" \ "663:\n\t" \ newinstr "\n" \ @@ -61,11 +74,17 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en ".popsection\n\t" \ ".org . - (664b-663b) + (662b-661b)\n\t" \ ".org . - (662b-661b) + (664b-663b)\n" \ + ".else\n\t" \ + "663:\n\t" \ + "664:\n\t" \ + ".endif\n" \ ".endif\n" #define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \ - __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg)) + __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg), 0) +#define ALTERNATIVE_CB(oldinstr, cb) \ + __ALTERNATIVE_CFG(oldinstr, "NOT_AN_INSTRUCTION", ARM_CB_PATCH, 1, cb) #else #include @@ -126,6 +145,14 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en 663: .endm +.macro alternative_cb cb + .set .Lasm_alt_mode, 0 + .pushsection .altinstructions, "a" + altinstruction_entry 661f, \cb, ARM_CB_PATCH, 662f-661f, 0 + .popsection +661: +.endm + /* * Complete an alternative code sequence. */ @@ -135,6 +162,13 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en .org . - (662b-661b) + (664b-663b) .endm +/* + * Callback-based alternative epilogue + */ +.macro alternative_cb_end +662: +.endm + #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...) \ alternative_insn insn1, insn2, cap, IS_ENABLED(cfg)