From patchwork Thu Dec 6 15:57:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 153041 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10695909ljp; Thu, 6 Dec 2018 07:59:49 -0800 (PST) X-Google-Smtp-Source: AFSGD/Vw0lZlV0QKJFjig26SLHQuqlNTUA2Luz63P9sNdU+2rTZHC0mL2/h1cYVaJjWzpLjrbxkJ X-Received: by 2002:a63:e445:: with SMTP id i5mr24522639pgk.307.1544111989515; Thu, 06 Dec 2018 07:59:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544111989; cv=none; d=google.com; s=arc-20160816; b=S6H526yUyqgxxfFJx0n3lwnWgUagPKIiXgjOYev3fDrEuIRolj2507SMStzkGcI08a rTs2BvNg7vjbsFNOQ6S5l3q898zih1Hr/XjO7V7Y0qWCavqoombrDbIFczyD/FB8jmRy QpadwDmWzOYUpPeG+Z51YDUQjYxhAgZwKKPk40LBG/t52NknbdwQ5PRPa5xAe69tYcMa jnLIHQIkcUgCjoGqWFCmGBSOF2vfF6c0a2rMDtgL0EY9so/Zg0hxOs5eskJT1pHHWusn q4w1cCYB0M6DG8pfOGcLCco9H/xV37UVC6L0wQpL4NB7uyqlsEZS1u6oRsmyoCJqIGon u1eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ufilkC4KXT597IQ5zu2bEqK9cc/hF4PdgqYxBckPn68=; b=pnpwBau07Ia9bQ8b1dXQ3eC4sLLaHkBZb4k60VdPMt8O7Uan+sPVwiR53LDvBmJISQ j9kEVINuMHuOykhMsOpx/EWxodbmQBtIzA/1mKz9REKfdroyjuoD15iUgDuM/Hg8grHd 63haI78nyXttrz8K1CMzVoava/Ya9YhzYqhYH/ADdariluHRbsea0xVluPjh6RJJfMjo rhmgrA3T3Ch9c3SGwhKpeg+j1iE2NQhRJl79KtvaDaipA8fIuoyihF/PC0dWoYJleb2F tIhMzVez335bKQxtAnETzlrosby9GUfyV2Oz/QLowvRDL1U8dA79XAXcGFglaFyY6a4t e2Sg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QPZ0I44B; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u188si541179pfb.232.2018.12.06.07.59.49; Thu, 06 Dec 2018 07:59:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QPZ0I44B; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726276AbeLFP7s (ORCPT + 31 others); Thu, 6 Dec 2018 10:59:48 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:55192 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726065AbeLFP7p (ORCPT ); Thu, 6 Dec 2018 10:59:45 -0500 Received: by mail-wm1-f65.google.com with SMTP id z18so1492830wmc.4 for ; Thu, 06 Dec 2018 07:59:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ufilkC4KXT597IQ5zu2bEqK9cc/hF4PdgqYxBckPn68=; b=QPZ0I44ByTCjuk9ePqJIX5NJL9ZGtQMKSitalKgY/UpP53/+9EfqvnX2MxcuN5M1eZ oq5oh0fNXmM1AGdxRQIiea69X3ReBymxkcjUr+xacjQzoSXYvUFMVFF+y4/GgLJnNdov fsOZZX2NsvdRupkKw+hdtAqBCOTmpJ81eN47U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ufilkC4KXT597IQ5zu2bEqK9cc/hF4PdgqYxBckPn68=; b=Uw1GCcgP7POqq/PKE30QfS6s9zl0mEsvq8n3ek1NTOvo/58vIpd0gvsoxeXmf+JojP lLCoY7QafqZNk6vUtg7+tUEk06ilWWdQyCTiImPiZ5C24dFX0UvEPB8McX38/GT/jS25 1PKXbBkEs2HEsKvUpVHGKMzqie0SCM/Xn3Vigg6sHgH47VAVYbAqiML7c7wRjhSdYOIA LidXXPhOLHYdoUSHYn9cHS/Avow5/DNZ2ST0jiZqVd32TBg8HYY1sWhwZ8sejHO+4BhZ dy9gRjSSgxy73B/fzJKab1CoOkVj/4OwJxB/ZDOS3L3L1q7blDLxMOMxUL5XphcO3EtG k5SA== X-Gm-Message-State: AA+aEWagZmgRjwcCbHdnd3eWJ/ycEiUK4f/3ucfuZ0myoyQSsM3Ded41 qltGuQ40LrNGp2UzgUbn4oqUZIC3PKB47g== X-Received: by 2002:a7b:cb86:: with SMTP id m6mr20340663wmi.61.1544111982823; Thu, 06 Dec 2018 07:59:42 -0800 (PST) Received: from localhost.localdomain (laubervilliers-657-1-83-120.w92-154.abo.wanadoo.fr. [92.154.90.120]) by smtp.gmail.com with ESMTPSA id y34sm1525233wrd.68.2018.12.06.07.59.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 07:59:41 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Robin Murphy , Will Deacon , Catalin Marinas , Marc Zyngier , Suzuki Poulose , Dave Martin Subject: [PATCH 1/5] arm64/alternative_cb: move callback reference into replacements section Date: Thu, 6 Dec 2018 16:57:35 +0100 Message-Id: <20181206155739.20229-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181206155739.20229-1-ard.biesheuvel@linaro.org> References: <20181206155739.20229-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation of permitting callback type alternatives patching routines to use any number of alternative instructions, move the relative reference to the callback routine out of the primary 'struct alt_instr' data structure, where we are currently overloading the 'alt_offset' member depending on the feature id. Instead, put a relative reference to the callback at the start of the alternative sequence. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/alternative.h | 39 +++++++++----------- arch/arm64/kernel/alternative.c | 11 ++++-- 2 files changed, 25 insertions(+), 25 deletions(-) -- 2.19.2 diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 4b650ec1d7dd..77da798e888b 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -24,6 +24,11 @@ struct alt_instr { u8 alt_len; /* size of new instruction(s), <= orig_len */ }; +struct alt_instr_cb { + s32 cb_offset; /* offset to callback handler */ + __le32 insn[]; /* sequence of alternative instructions */ +}; + typedef void (*alternative_cb_t)(struct alt_instr *alt, __le32 *origptr, __le32 *updptr, int nr_inst); @@ -35,13 +40,9 @@ void apply_alternatives_module(void *start, size_t length); static inline void apply_alternatives_module(void *start, size_t length) { } #endif -#define ALTINSTR_ENTRY(feature,cb) \ +#define ALTINSTR_ENTRY(feature) \ " .word 661b - .\n" /* label */ \ - " .if " __stringify(cb) " == 0\n" \ " .word 663f - .\n" /* new instruction */ \ - " .else\n" \ - " .word " __stringify(cb) "- .\n" /* callback */ \ - " .endif\n" \ " .hword " __stringify(feature) "\n" /* feature bit */ \ " .byte 662b-661b\n" /* source len */ \ " .byte 664f-663f\n" /* replacement len */ @@ -59,36 +60,29 @@ static inline void apply_alternatives_module(void *start, size_t length) { } * but most assemblers die if insn1 or insn2 have a .inst. This should * be fixed in a binutils release posterior to 2.25.51.0.2 (anything * containing commit 4e4d08cf7399b606 or c1baaddf8861). - * - * Alternatives with callbacks do not generate replacement instructions. */ -#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled, cb) \ - ".if "__stringify(cfg_enabled)" == 1\n" \ +#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature) \ "661:\n\t" \ oldinstr "\n" \ "662:\n" \ ".pushsection .altinstructions,\"a\"\n" \ - ALTINSTR_ENTRY(feature,cb) \ + ALTINSTR_ENTRY(feature) \ ".popsection\n" \ - " .if " __stringify(cb) " == 0\n" \ ".pushsection .altinstr_replacement, \"a\"\n" \ "663:\n\t" \ newinstr "\n" \ "664:\n\t" \ - ".popsection\n\t" \ + ".popsection\n\t" + +#define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \ + ".if "__stringify(IS_ENABLED(cfg))" == 1\n" \ + __ALTERNATIVE_CFG(oldinstr, newinstr, feature) \ ".org . - (664b-663b) + (662b-661b)\n\t" \ ".org . - (662b-661b) + (664b-663b)\n" \ - ".else\n\t" \ - "663:\n\t" \ - "664:\n\t" \ - ".endif\n" \ ".endif\n" -#define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \ - __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg), 0) - #define ALTERNATIVE_CB(oldinstr, cb) \ - __ALTERNATIVE_CFG(oldinstr, "NOT_AN_INSTRUCTION", ARM64_CB_PATCH, 1, cb) + __ALTERNATIVE_CFG(oldinstr, ".word " __stringify(cb) " - .\n", ARM64_CB_PATCH) #else #include @@ -158,8 +152,11 @@ static inline void apply_alternatives_module(void *start, size_t length) { } .macro alternative_cb cb .set .Lasm_alt_mode, 0 .pushsection .altinstructions, "a" - altinstruction_entry 661f, \cb, ARM64_CB_PATCH, 662f-661f, 0 + altinstruction_entry 661f, 663f, ARM64_CB_PATCH, 662f-661f, 664f-663f .popsection + .pushsection .altinstr_replacement, "ax" +663: .word \cb - . +664: .popsection 661: .endm diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index b5d603992d40..a49930843784 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -151,6 +151,7 @@ static void __apply_alternatives(void *alt_region, bool is_module) struct alt_region *region = alt_region; __le32 *origptr, *updptr; alternative_cb_t alt_cb; + struct alt_instr_cb *alt_cb_insn; for (alt = region->begin; alt < region->end; alt++) { int nr_inst; @@ -161,7 +162,7 @@ static void __apply_alternatives(void *alt_region, bool is_module) continue; if (alt->cpufeature == ARM64_CB_PATCH) - BUG_ON(alt->alt_len != 0); + BUG_ON(alt->alt_len < sizeof(*alt_cb_insn)); else BUG_ON(alt->alt_len != alt->orig_len); @@ -171,10 +172,12 @@ static void __apply_alternatives(void *alt_region, bool is_module) updptr = is_module ? origptr : lm_alias(origptr); nr_inst = alt->orig_len / AARCH64_INSN_SIZE; - if (alt->cpufeature < ARM64_CB_PATCH) + if (alt->cpufeature < ARM64_CB_PATCH) { alt_cb = patch_alternative; - else - alt_cb = ALT_REPL_PTR(alt); + } else { + alt_cb_insn = ALT_REPL_PTR(alt); + alt_cb = offset_to_ptr(&alt_cb_insn->cb_offset); + } alt_cb(alt, origptr, updptr, nr_inst); From patchwork Thu Dec 6 15:57:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 153044 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10696110ljp; Thu, 6 Dec 2018 08:00:00 -0800 (PST) X-Google-Smtp-Source: AFSGD/WRjw+Q4K6CVjaC7jLZg9eE354vy1jOOyWzUKwP9jfgek/883DBjuGRDHZYaH4yPTwxhfpk X-Received: by 2002:a17:902:a50a:: with SMTP id s10mr27428900plq.278.1544112000782; Thu, 06 Dec 2018 08:00:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544112000; cv=none; d=google.com; s=arc-20160816; b=m3oWxqFHp3Smv9OyNN3EZsxXvEw+IfO2I0q+y3N0CxeAwZ5rggmeH4eAsovOxj/Vkv W5O08wlHmVnH6QFZe5BQZVb3bvN4N9yTQkSS1JH+LLwr0Q2BU23WWmVcu7JkffcjnzYV ZhcWMyjrNbQt6wgeQg2gAkbZ+4clyXMF0nRFW2MMq0Lom7HSlYrU+gPmVq+xIfI0SPOF mr8GiXTzT3KeOLvXvd+nNsHkCJWVpQh+f84eFCxiA5XvWxsSBZ0B/7Hr0l5VDv8I6bOb 6AkYSWz4ugp228w6tDsFWNsaIba8FV7bSG1S6WS/Rl9uiamppXhiy6PmVV8zD4xTbV0s sOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=dgSG9gChmYBffNKXbMNH/SpVS1I5j9iEUF32P6VsSNQ=; b=lAClaIqJ+IXxHTUL3WoTwz/DjiAHcoNw8R+KxD8QQ2WWd9TM+GKNDMz5zPr1cra7dG rp95AVjKQMDelXPOfx3Qnjm2QKBr78LoBhHiDvSVyDSoqgZ8lfmiBOzqsrdi9ExQTzPB VAAcisRpk3fr0XRYdGM5TSUv3muuSrrFRrgRJlxVIhOxX+0/YSYwdnck1C3hCxiS4+DS vLEIjDXFuvztgp2hSoA4bvXg5IZXeIfpM9kXoVuoxXEXbLTMq6vVdL63zH0hrJLcnb2S i3E+VZHOpuCFGfqd1PIip+iKodZZf8usIcQ7A9mSIv2qz0+S0KdbB0n5b/Ijb3YCfudf UIVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OeTiLmJb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p10si466188pgi.549.2018.12.06.08.00.00; Thu, 06 Dec 2018 08:00:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OeTiLmJb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726296AbeLFP7t (ORCPT + 31 others); Thu, 6 Dec 2018 10:59:49 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:37061 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725889AbeLFP7q (ORCPT ); Thu, 6 Dec 2018 10:59:46 -0500 Received: by mail-wr1-f68.google.com with SMTP id j10so1009842wru.4 for ; Thu, 06 Dec 2018 07:59:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dgSG9gChmYBffNKXbMNH/SpVS1I5j9iEUF32P6VsSNQ=; b=OeTiLmJb/vMVFqlou6bQKq9apgP2LUcCUWVMJEAQrfWdHhvbmur7qW30Qg1tVQ7d2L M8nEaUPwwxyHVyla64j+Pb81s2Trsf3uZB0cnLc2jPytOGg2K6MvthrZl/eUs3awgRtz BHW8jdTyEdBkCxpo+LKBm4JJ2hhbcz726b/kw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dgSG9gChmYBffNKXbMNH/SpVS1I5j9iEUF32P6VsSNQ=; b=irg4subCwWwBzsIARl0IKzVbOAP+EW61vOtcWRyc6OGx16a7Fn4iY0zE8ufRnEDkiR H356cHTBjEiN40At/qduB8fzYTX4UuNOFypHFPQQOYTJKmvG73KLq3sgB/5x8IZgM+jn PuZB8fxqKXhdrrFjlClNnWbeEoeiRtsKulcZBCotk4ZApDUOGo7820JrWunya7COtI3D N5dBM/z4v4uedP0Db9gHH1RlkVIL22HI7+Y49YoxttyKUhOkHS8wagbQqdTYpiManMP1 iAojqGq+WaariDMwCoekd+WB55z4u4xSQnUS4KAFEMiPeNgCekrc4IWUAtBk62EKWBQA 0tlw== X-Gm-Message-State: AA+aEWbQOlBol7EdrgphqJY1cbE5C5rS9IrblGT8QGBotTtwuwHrfe7Q O7wXbJi/wLaXXC8SKrA2YwKlgGjItb836g== X-Received: by 2002:adf:8342:: with SMTP id 60mr24344596wrd.212.1544111984136; Thu, 06 Dec 2018 07:59:44 -0800 (PST) Received: from localhost.localdomain (laubervilliers-657-1-83-120.w92-154.abo.wanadoo.fr. [92.154.90.120]) by smtp.gmail.com with ESMTPSA id y34sm1525233wrd.68.2018.12.06.07.59.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 07:59:43 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Robin Murphy , Will Deacon , Catalin Marinas , Marc Zyngier , Suzuki Poulose , Dave Martin Subject: [PATCH 2/5] arm64/alternative_cb: add nr_alts parameter to callback handlers Date: Thu, 6 Dec 2018 16:57:36 +0100 Message-Id: <20181206155739.20229-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181206155739.20229-1-ard.biesheuvel@linaro.org> References: <20181206155739.20229-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Callback handlers for alternative patching will shortly be able to access a set of alternative instructions provided at assembly time. So update the callback handler prototypes so we can pass this number. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/alternative.h | 4 ++-- arch/arm64/include/asm/kvm_mmu.h | 4 ++-- arch/arm64/kernel/alternative.c | 11 +++++++---- arch/arm64/kernel/cpu_errata.c | 10 ++++------ arch/arm64/kvm/va_layout.c | 8 ++++---- 5 files changed, 19 insertions(+), 18 deletions(-) -- 2.19.2 diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 77da798e888b..987c1514183a 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -29,8 +29,8 @@ struct alt_instr_cb { __le32 insn[]; /* sequence of alternative instructions */ }; -typedef void (*alternative_cb_t)(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst); +typedef void (*alternative_cb_t)(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts); void __init apply_alternatives_all(void); diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 658657367f2f..5e32e314b9f0 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -100,8 +100,8 @@ alternative_cb_end #include #include -void kvm_update_va_mask(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst); +void kvm_update_va_mask(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_insti, int nr_alts); static inline unsigned long __kern_hyp_va(unsigned long v) { diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index a49930843784..f55afa0bbaa4 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -107,8 +107,8 @@ static u32 get_alt_insn(struct alt_instr *alt, __le32 *insnptr, __le32 *altinsnp return insn; } -static void patch_alternative(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst) +static void patch_alternative(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts) { __le32 *replptr; int i; @@ -154,7 +154,7 @@ static void __apply_alternatives(void *alt_region, bool is_module) struct alt_instr_cb *alt_cb_insn; for (alt = region->begin; alt < region->end; alt++) { - int nr_inst; + int nr_inst, nr_alts; /* Use ARM64_CB_PATCH as an unconditional patch */ if (alt->cpufeature < ARM64_CB_PATCH && @@ -174,12 +174,15 @@ static void __apply_alternatives(void *alt_region, bool is_module) if (alt->cpufeature < ARM64_CB_PATCH) { alt_cb = patch_alternative; + nr_alts = alt->alt_len / AARCH64_INSN_SIZE; } else { alt_cb_insn = ALT_REPL_PTR(alt); alt_cb = offset_to_ptr(&alt_cb_insn->cb_offset); + nr_alts = (alt->alt_len - sizeof(*alt_cb_insn)) / AARCH64_INSN_SIZE; + } - alt_cb(alt, origptr, updptr, nr_inst); + alt_cb(alt, origptr, updptr, nr_inst, nr_alts); if (!is_module) { clean_dcache_range_nopatch((u64)origptr, diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 6ad715d67df8..c5489b4612c5 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -305,9 +305,8 @@ static int __init ssbd_cfg(char *buf) } early_param("ssbd", ssbd_cfg); -void __init arm64_update_smccc_conduit(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, - int nr_inst) +void __init arm64_update_smccc_conduit(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts) { u32 insn; @@ -327,9 +326,8 @@ void __init arm64_update_smccc_conduit(struct alt_instr *alt, *updptr = cpu_to_le32(insn); } -void __init arm64_enable_wa2_handling(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, - int nr_inst) +void __init arm64_enable_wa2_handling(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts) { BUG_ON(nr_inst != 1); /* diff --git a/arch/arm64/kvm/va_layout.c b/arch/arm64/kvm/va_layout.c index c712a7376bc1..db7ba73e306a 100644 --- a/arch/arm64/kvm/va_layout.c +++ b/arch/arm64/kvm/va_layout.c @@ -114,8 +114,8 @@ static u32 compute_instruction(int n, u32 rd, u32 rn) return insn; } -void __init kvm_update_va_mask(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst) +void __init kvm_update_va_mask(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts) { int i; @@ -154,8 +154,8 @@ void __init kvm_update_va_mask(struct alt_instr *alt, void *__kvm_bp_vect_base; int __kvm_harden_el2_vector_slot; -void kvm_patch_vector_branch(struct alt_instr *alt, - __le32 *origptr, __le32 *updptr, int nr_inst) +void kvm_patch_vector_branch(struct alt_instr *alt, __le32 *origptr, + __le32 *updptr, int nr_inst, int nr_alts) { u64 addr; u32 insn; From patchwork Thu Dec 6 15:57:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 153042 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10695957ljp; Thu, 6 Dec 2018 07:59:52 -0800 (PST) X-Google-Smtp-Source: AFSGD/XJIyq1TkTeiekl8dtPh3c5BaUovDOozdLXwgEwDCE4y9JpixpRgh5jbUu6334kxCHYpp0Z X-Received: by 2002:a17:902:d70b:: with SMTP id w11mr29235133ply.294.1544111991919; Thu, 06 Dec 2018 07:59:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544111991; cv=none; d=google.com; s=arc-20160816; b=Ww5Iy/MlqyCk840vyEImt+yEUg8owGRhUWXR627wrvHl4O+uNsaiUe6+hJKN4AlBWZ Y6OeXgLLcDpejUY3MK+lqbuCXaMiZSWmhpEQCeLgMekds0a6WJ+DaYilw50zL7g8sHR5 xBcUnEIvma+257/AeJhrNur34xz6DcJc+TQAT09a1VydyUU7Z55m9ULMKnFHhKr+d/1D wCJNOWhIWi/KWxzqLWZ9+XuzE/CNIgCI8quzE8FixyZEbBTEsXBZd2yuVlCCaZ+ayVBU JVOy3OTcRskCBo+iVTXiMId2a3jJT4DDAIr1+k1obvM6gYKlixw13+9aGmMwren6LYTq clhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=EBf2ddABzNmUATNTkk2NnIWCCRz0jpWPHKnmmJ2ur1E=; b=teuNMnJx0JZLScZhtiZtVu6XnegztAzZ7IDC5Wqq3qhDQOUsWxT5EUW0Lp4Sk8UhuD 3AyT7J7jCg9TGxGw5EJE79MoympGe++zLF6s2+fO8D5XsZq81etjF3yszoNRxHB0TTBN hbC7O5XBcydqJeRDYACchByS3Q9/SIto38DSfz9rsbd9jzFaFW87N+NO4bmhFNaK9PHQ 3W1Xj9c2Ac7eW0c4OuBK3VZgDGKEJzGenQYz8CcGvbravyHZZH1Olhw+GGhbwDF6dg0L lKafJkb5cQuZd9iKxFan7zzCnASwAprGQ1t5VHATgdjsU5jriBp4aPCtHDnzscR9Ylas DV3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cikb10eS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u188si541179pfb.232.2018.12.06.07.59.51; Thu, 06 Dec 2018 07:59:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cikb10eS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726310AbeLFP7u (ORCPT + 31 others); Thu, 6 Dec 2018 10:59:50 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:43950 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726090AbeLFP7r (ORCPT ); Thu, 6 Dec 2018 10:59:47 -0500 Received: by mail-wr1-f68.google.com with SMTP id r10so970406wrs.10 for ; Thu, 06 Dec 2018 07:59:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EBf2ddABzNmUATNTkk2NnIWCCRz0jpWPHKnmmJ2ur1E=; b=cikb10eScMgEpPy7mtJP4RXIN8T1QZq9WVgYZi95AR+i7To5iS+heCKoRM9014xRYT Ur1lV8CjLtLPaiHFc7wRUaDcfjjrmA9TvhtO503Qjkft00Bz8qad0RgXK+dBAKCNrjXj FchhmDN30ABh9EurnN30vGYzRdHREgSGwqJ1Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EBf2ddABzNmUATNTkk2NnIWCCRz0jpWPHKnmmJ2ur1E=; b=oruKmnO+eKiKzFTrrQg1A2P22rrb0TqGdFJQwprfDyAKCW4wZxbD+8X6SDxOPpVB4S rApT61DQa5CvAdGrsucXNtOuZS7qnRNyXEt2xqOk4sD5cjRIzbyk9XCpu7Sg2tcDcI0x DUAR7+GuvvdA1sp4phEtOI59p96Qq5zoFT+QJJJN/xvTQV5Cu6O93YUfvtHr2/QQ6T+X 9G1rl2iOhcot+QV6BWn0AEDksXmPd6ghQebvpRyFQ5I6hG+hyP+ckCOAgZ1VOedGVkTg Upp2lETQKTJ9PwkIgCaxBuvLI08Sy5x9nkqmKWn3GFLneOWwov6f39w16qvzPnQ0/SsS zrNQ== X-Gm-Message-State: AA+aEWYsR9llxRGitWQBUmxajL3/shxnnMk1Pul2dY5WrdTymVIJ7bff I/7OwM2WoRbinJ/LFj/oSj/Ly/4YG3PJ8w== X-Received: by 2002:adf:b201:: with SMTP id u1mr25976578wra.165.1544111985333; Thu, 06 Dec 2018 07:59:45 -0800 (PST) Received: from localhost.localdomain (laubervilliers-657-1-83-120.w92-154.abo.wanadoo.fr. [92.154.90.120]) by smtp.gmail.com with ESMTPSA id y34sm1525233wrd.68.2018.12.06.07.59.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 07:59:44 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Robin Murphy , Will Deacon , Catalin Marinas , Marc Zyngier , Suzuki Poulose , Dave Martin Subject: [PATCH 3/5] arm64/alternative_cb: add support for alternative sequences Date: Thu, 6 Dec 2018 16:57:37 +0100 Message-Id: <20181206155739.20229-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181206155739.20229-1-ard.biesheuvel@linaro.org> References: <20181206155739.20229-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Permit callback type alternatives to carry a sequence of alternative instructions, and leave it up to the callback handler to decide how to use them. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/alternative.h | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) -- 2.19.2 diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 987c1514183a..6b7726ee6b0a 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -156,7 +156,7 @@ static inline void apply_alternatives_module(void *start, size_t length) { } .popsection .pushsection .altinstr_replacement, "ax" 663: .word \cb - . -664: .popsection + .popsection 661: .endm @@ -185,10 +185,21 @@ static inline void apply_alternatives_module(void *start, size_t length) { } .org . - (662b-661b) + (664b-663b) .endm +.macro alternative_cb_alt + .set .Lasm_alt_mode, 1 + .pushsection .altinstr_replacement, "ax" + .org 663b + AARCH64_INSN_SIZE +.endm + /* * Callback-based alternative epilogue */ .macro alternative_cb_end + .if .Lasm_alt_mode==0 + .pushsection .altinstr_replacement, "ax" + .org 663b + AARCH64_INSN_SIZE + .endif +664: .popsection 662: .endm From patchwork Thu Dec 6 15:57:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 153045 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10696136ljp; Thu, 6 Dec 2018 08:00:02 -0800 (PST) X-Google-Smtp-Source: AFSGD/W9RP8reW/EWlw/jJY7kN3mltAiNKklpNfds88mToTirSsXmmgUX/9F4vZ6leIkbDySnxUC X-Received: by 2002:a63:c141:: with SMTP id p1mr24501178pgi.424.1544112002559; Thu, 06 Dec 2018 08:00:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544112002; cv=none; d=google.com; s=arc-20160816; b=Z2bKfdUGjxTE8ZMMvTwVfawRp/Doub4Edws4nV6XaTdpTTx/dDN69p3f/BWvzyVxl7 mF+gMxqUIjrA0noHqelaug0QM4su2fadDRY/cq4c+8C4nnbunFZe57rG7F3FnHgvJKzW JLRUfId69xd/gcqDibXQMMwWYrLLwbUFfdboe725UyxP2yLhn3HOPyB34eidFNyiTsHh /RADZwRl3nSrTQBvYzVt7CzQdN3hsizagBxTBVq3bMYSokzDflT7yzTKkkVHa6wVMaMF Lw1tzazizGc+HMD6Ce3JGEs+98+sDDbyfV7fbrYCCzqZQFRlOX/gj+UJIp7RimGlXnC0 AuGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Wpw2u+vd9657tht5bV1UNpmFvYbIAJm69uSezSpG9Ao=; b=E2EGquFJStj9I04uWEH4PUjZKGOM9pIADKh+F4bQx+if0tAUGn+S2uI4Bwz0l6DUSL lN9jQpeQKvmoj7+AfEcMhBY3q0cJ7bsu7tAHrHfVLH2YIcuqvO4pBv10XJYMtBiBu3eQ 0MKIxEMWCNfIt8eb8Qg66iH8g2XWgMCCu9RvXq+4MZWQWleOcuJRzkzu9mXQ5C8Rh8ZH Pp86bBoxfmoRO4FpYbNiXNV64Gi7VoMA4ODri8QSnwW1rCWkUxTabRZvWGFT+hfFxP93 0qcOB+DsH/xWPCMCvSDZRnDpg1lBvmXWPO+cF+LZnrz5RhRLCPw2YMljLsykEp+O6NUG qJLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fTafQJUW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l24si502266pgb.489.2018.12.06.08.00.02; Thu, 06 Dec 2018 08:00:02 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fTafQJUW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726341AbeLFQAA (ORCPT + 31 others); Thu, 6 Dec 2018 11:00:00 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:55514 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726211AbeLFP7t (ORCPT ); Thu, 6 Dec 2018 10:59:49 -0500 Received: by mail-wm1-f68.google.com with SMTP id y139so1473206wmc.5 for ; Thu, 06 Dec 2018 07:59:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Wpw2u+vd9657tht5bV1UNpmFvYbIAJm69uSezSpG9Ao=; b=fTafQJUWvloms3Jzq3/B4I3D4zX2ANxFQwunJyp5pR6sv3hRU4IVk1JpJ8FsIJKO2K QPb0bek0z0OHe6JFTaM5W+uS1oa2eopKgztXOzHIeeHhHXr+FgYIO/sXMi+vx5VCaGAP NGf1CTJ4GtXAbZq9WswRDOtXlym28H4oKCc+w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wpw2u+vd9657tht5bV1UNpmFvYbIAJm69uSezSpG9Ao=; b=X6GWvh+AIMwbwC47X7ZSdKhhDpspRBRPswPzDkG2zmE2Efd/u5ksQ8WoLs35VrOXnT 9t00tcJQYXpTtujF/qotWRiFUM3Nr0u0h797dcdvkbcPqgsfNoW8FDscI888T4ckhUob lq6TFgi3Vy4kSjPB3TttAmho9mtwQ5ruQg2w0KiJtEWVUQ8J8RJxUJJSdS818m94Ubxe up1mDxUbp0HgFo+pZdbxQRhWLkDoS+3zcHv/CATLcg/Ene4iBZeRC/OR9opG4U40cTUZ /sES8mN35HUYFMIDjxutBJqQiAPLk6RtdLedoU4EHA1sBgwGKGpal3zy0BD1ct5rffkC 0YzA== X-Gm-Message-State: AA+aEWZsjTAhFMIXIMtt4eJ7D/3NpTaZwhzhE0SLWMucHwNjfVc8FiSh gij9B+h7yMt8bQWZncOzA63cOjkLWNS2bA== X-Received: by 2002:a1c:f404:: with SMTP id z4mr20445248wma.68.1544111986633; Thu, 06 Dec 2018 07:59:46 -0800 (PST) Received: from localhost.localdomain (laubervilliers-657-1-83-120.w92-154.abo.wanadoo.fr. [92.154.90.120]) by smtp.gmail.com with ESMTPSA id y34sm1525233wrd.68.2018.12.06.07.59.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 07:59:45 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Robin Murphy , Will Deacon , Catalin Marinas , Marc Zyngier , Suzuki Poulose , Dave Martin Subject: [PATCH 4/5] arm64/assembler: use callback to 3-way alt-patch DC CVAP instructions Date: Thu, 6 Dec 2018 16:57:38 +0100 Message-Id: <20181206155739.20229-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181206155739.20229-1-ard.biesheuvel@linaro.org> References: <20181206155739.20229-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Use the enhanced alternative_cb implementation to reimplement the code patching logic for DC CVAP instructions so that we don't fall back to DV CVAC instructions on systems that require those to be upgraded to DC CIVAC to work around silicon errata. At the same time, we don't want to use DV CIVAC needlessly, since doing so may adversely affect performance. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 9 +++++---- arch/arm64/kernel/cpu_errata.c | 14 ++++++++++++++ 2 files changed, 19 insertions(+), 4 deletions(-) -- 2.19.2 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6142402c2eb4..09c5a5452f60 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -390,11 +390,12 @@ alternative_else dc civac, \kaddr alternative_endif .elseif (\op == cvap) -alternative_if ARM64_HAS_DCPOP - sys 3, c7, c12, 1, \kaddr // dc cvap -alternative_else +alternative_cb arm64_handle_dc_cvap + dc civac, \kaddr +alternative_cb_alt + sys 3, c7, c12, 1, \kaddr // dc cvap dc cvac, \kaddr -alternative_endif +alternative_cb_end .else dc \op, \kaddr .endif diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index c5489b4612c5..a63e362da307 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -755,3 +755,17 @@ const struct arm64_cpu_capabilities arm64_errata[] = { { } }; + +asmlinkage void __init arm64_handle_dc_cvap(struct alt_instr *alt, + __le32 *origptr, __le32 *updptr, + int nr_inst, int nr_alts) +{ + struct alt_instr_cb *alt_insn = offset_to_ptr(&alt->alt_offset); + + BUG_ON(nr_inst != 1 || nr_alts != 2); + + if (cpus_have_cap(ARM64_HAS_DCPOP)) + updptr[0] = alt_insn->insn[0]; + else if (!cpus_have_cap(ARM64_WORKAROUND_CLEAN_CACHE)) + updptr[0] = alt_insn->insn[1]; +} From patchwork Thu Dec 6 15:57:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 153043 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp10696005ljp; Thu, 6 Dec 2018 07:59:55 -0800 (PST) X-Google-Smtp-Source: AFSGD/WWAUwxU/FGgZM261jz7rRS8zKF1SqDoRdFl6QiLpvFsCriiDuFJiphRnjLloYViIDHS9iK X-Received: by 2002:a17:902:a83:: with SMTP id 3mr26920327plp.276.1544111995308; Thu, 06 Dec 2018 07:59:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544111995; cv=none; d=google.com; s=arc-20160816; b=YWH3uIRxCRPAxFVWHctRkRUJkNBat321x/9Wp8bjKxIb0+1AK8Wi5uqsihGHuoAP8I 991N4cueGzNTbIbY5gEeW5QaD60I9nPfX+xx/TDb5/XNhxBpBRGw70csVle1WvMb5hB/ cFbuLBh38R6sWuxHtpRQU3HDiTvMVonIG9d0p6YQCZH16TTeoL4FgGVKT1y9Bb2e6vWl zIyXUDdo6TeRPm/kV/g5hJnxmqeOcvNuivHuvQcI/vNmEFEuPCAmIYCrBaBJb2XLxLGP f2Ac9aDGWbyYuhyFLtPTciGiwd8llYgRdloo+LVSpDREeO8moM8PF7s1gE2ADB0k/REr foGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=YAJ/pSqxHadg9FAalxrP9XtBWK91iIUrnuOMWYKHTIQ=; b=G9r5UYvSnggeT0xAM8oOUFdCSgA7WZZwOy/jfcR6O29N/ua3Hg/20U7ZGrzFCevst0 2usrd4oWxhSn8BvezmBYc5a+at+ksTKdsuztV85K8VOW2j7IzEoDCr01rujsQs1hWGmb tp5vnvY6xTAHpRuHo0Jw4j/8TQWN97XbQbu97O5QOJvFNnojSrRKEqeyNMWXkzDxPg56 w7glAO4Di+lRHxoHUH/feYLvI+FHM3rQcqvRQ5+mwJ9nrQ9UXRG9i5IgmLnq9MdxbKYU dBct/RHU0JesR63MiD2rUhGuLX5KE9BURQC8gUJObBkvxjDTKRq3iAn2gVOXfAcUmtOh tOIQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bQGwIxIB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p10si466188pgi.549.2018.12.06.07.59.54; Thu, 06 Dec 2018 07:59:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bQGwIxIB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726323AbeLFP7y (ORCPT + 31 others); Thu, 6 Dec 2018 10:59:54 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35747 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726293AbeLFP7u (ORCPT ); Thu, 6 Dec 2018 10:59:50 -0500 Received: by mail-wm1-f68.google.com with SMTP id c126so1594601wmh.0 for ; Thu, 06 Dec 2018 07:59:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YAJ/pSqxHadg9FAalxrP9XtBWK91iIUrnuOMWYKHTIQ=; b=bQGwIxIBK0/KWy1ZsHZsbFC0w94wP01OxLh+vtw1bAeRhFTgOQJAMWKaoVcaUF7cZl lusOfufP8vuz4RrpEz6ugIwqp3RJ182W4P7mOp69ecMwFaYMJrPN5UzI1ZK73k+sd659 MvVrQK0ljOjwxFk4GJiWJRA/VvjgBOWLtupSg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YAJ/pSqxHadg9FAalxrP9XtBWK91iIUrnuOMWYKHTIQ=; b=jHhcVDDX+SRXIU/Vw/LJuwEmss1ebp0RD+pgpHGUdNLg5b8iAS6fypJCVOCMFnFye6 B/f7U0uXN2nYJtmFG+DwE1HfKlwDkCtjWmvbz7NyDv9xPHPmSBCNaRSvR4HwLpQF23fF g7Ihormz1DxWdPBauglbywZ4FVx3Via1b8T8lFPfC1uvAJd29zeNXgJRHIh7jle0mIHT R4+tcRLiJ9Suv/cbi1pJ6++rmwKfgcf2dFpPHTGVMNkPLQBQYdTmanRSQDmEz2LRj6gs JGiw2y9XYH8SO7M6u0Aj/xDjC6w/Pg5HFkgquizfx4Re3uQLSNNG/lTrDFiQZYVQCfQF zkDQ== X-Gm-Message-State: AA+aEWbzwJf5lB0gUjJ2HOVstY2LIg2xk/b68NT037ieH1kUlWokN+a0 FJRWAfDOkYQeoz3y5GMDZ/+dlidHjEn9/Q== X-Received: by 2002:a1c:770c:: with SMTP id t12mr20471294wmi.101.1544111987884; Thu, 06 Dec 2018 07:59:47 -0800 (PST) Received: from localhost.localdomain (laubervilliers-657-1-83-120.w92-154.abo.wanadoo.fr. [92.154.90.120]) by smtp.gmail.com with ESMTPSA id y34sm1525233wrd.68.2018.12.06.07.59.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Dec 2018 07:59:47 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Robin Murphy , Will Deacon , Catalin Marinas , Marc Zyngier , Suzuki Poulose , Dave Martin Subject: [PATCH 5/5] arm64/mm: use string comparisons in dcache_by_line_op Date: Thu, 6 Dec 2018 16:57:39 +0100 Message-Id: <20181206155739.20229-6-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.2 In-Reply-To: <20181206155739.20229-1-ard.biesheuvel@linaro.org> References: <20181206155739.20229-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The GAS directives that are currently being used in dcache_by_line_op rely on assembler behavior that is not documented, and probably not guaranteed to produce the correct behavior going forward. Currently, we end up with some undefined symbols in cache.o: $ nm arch/arm64/mm/cache.o ... U civac ... U cvac U cvap U cvau This is due to the fact that the comparisons used to select the operation type in the dcache_by_line_op macro are comparing symbols not strings, and even though it seems that GAS is doing the right thing here (undefined symbols by the same name are equal to each other), it seems unwise to rely on this. So let's switch to conditional directives that are documented as operating on strings. Since these cannot be combined like ordinary arithmetic expressions, invert the comparison logic. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) -- 2.19.2 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 09c5a5452f60..df3043e76e6a 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -383,21 +383,23 @@ alternative_endif sub \tmp2, \tmp1, #1 bic \kaddr, \kaddr, \tmp2 9998: - .if (\op == cvau || \op == cvac) + .ifnc \op, civac + .ifnc \op, cvap alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE dc \op, \kaddr alternative_else dc civac, \kaddr alternative_endif - .elseif (\op == cvap) + .else alternative_cb arm64_handle_dc_cvap dc civac, \kaddr alternative_cb_alt sys 3, c7, c12, 1, \kaddr // dc cvap dc cvac, \kaddr alternative_cb_end + .endif .else - dc \op, \kaddr + dc civac, \kaddr .endif add \kaddr, \kaddr, \tmp1 cmp \kaddr, \size