From patchwork Tue Feb 24 17:16:23 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 44955 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f70.google.com (mail-la0-f70.google.com [209.85.215.70]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2986120502 for ; Tue, 24 Feb 2015 17:18:49 +0000 (UTC) Received: by lams18 with SMTP id s18sf18189131lam.1 for ; Tue, 24 Feb 2015 09:18:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id:cc :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list; bh=FIeGlcdVh2OD80NRoh0+ioZ4CGkZrGjsqEuFozHbYRw=; b=M9zWjKWvcioQliAtPQeAlgtshK79hg0yny2MC8ekBRyKr/NEmFRTB2iPZ7pj1VeYSY ABHnTDL3YpTQ9JC+InqiclTPv1IQkTP+Uneu8V4hlCVfvC9+A4ezbTkiG0umOoGYZB9n YXpFXpXNyW3LPBsmjZVxbKkcj9ZoPKuwdOMK6reebV0OSX4ZZ2GjEZK6gLA7Ys/l1bGS IucvdhIWwVrlN9p2iB1YpPPIHUKzaM/wliiu51xTMFzCeB2BxvPPgV8NQlB0XCUk8Btp jPJaU4d7QgS001ls3QFWlAFH/f6EDpOfACQFAQZm9ufUnale5E1NYLb4nFukzshcsWUF Bsgg== X-Gm-Message-State: ALoCoQkSAKwMNxkxFCOHgpgpt504BVWY9EoXXM6Eh0F+KVaxfCu/y/QeXF9cbrWzTsfgZe/cPAWb X-Received: by 10.112.50.106 with SMTP id b10mr2299838lbo.8.1424798327727; Tue, 24 Feb 2015 09:18:47 -0800 (PST) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.121.1 with SMTP id lg1ls645139lab.34.gmail; Tue, 24 Feb 2015 09:18:47 -0800 (PST) X-Received: by 10.112.170.72 with SMTP id ak8mr14578718lbc.95.1424798327584; Tue, 24 Feb 2015 09:18:47 -0800 (PST) Received: from mail-la0-f53.google.com (mail-la0-f53.google.com. [209.85.215.53]) by mx.google.com with ESMTPS id is5si23506502lac.125.2015.02.24.09.18.46 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 09:18:46 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.53 as permitted sender) client-ip=209.85.215.53; Received: by labgf13 with SMTP id gf13so27140027lab.9 for ; Tue, 24 Feb 2015 09:18:46 -0800 (PST) X-Received: by 10.152.21.201 with SMTP id x9mr4514434lae.72.1424798326699; Tue, 24 Feb 2015 09:18:46 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp2050629lbj; Tue, 24 Feb 2015 09:18:45 -0800 (PST) X-Received: by 10.68.253.233 with SMTP id ad9mr5387878pbd.167.1424798324586; Tue, 24 Feb 2015 09:18:44 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id x8si12516430pdm.114.2015.02.24.09.18.43 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Feb 2015 09:18:44 -0800 (PST) Received-SPF: none (google.com: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org does not designate permitted sender hosts) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YQJ6Q-0005tp-2o; Tue, 24 Feb 2015 17:16:58 +0000 Received: from mail-wg0-f41.google.com ([74.125.82.41]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YQJ6J-0005o1-RV for linux-arm-kernel@lists.infradead.org; Tue, 24 Feb 2015 17:16:53 +0000 Received: by wghb13 with SMTP id b13so6701470wgh.0 for ; Tue, 24 Feb 2015 09:16:29 -0800 (PST) X-Received: by 10.194.61.161 with SMTP id q1mr25695302wjr.132.1424798189599; Tue, 24 Feb 2015 09:16:29 -0800 (PST) Received: from ards-macbook-pro.lan (bl11-65-113.dsl.telepac.pt. [85.244.65.113]) by mx.google.com with ESMTPSA id n1sm21372688wib.11.2015.02.24.09.16.27 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 09:16:28 -0800 (PST) From: Ard Biesheuvel To: marc.zyngier@arm.com, christoffer.dall@linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu Subject: [PATCH] ARM, arm64: kvm: get rid of the bounce page Date: Tue, 24 Feb 2015 17:16:23 +0000 Message-Id: <1424798183-3202-1-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.8.3.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150224_091652_039252_50436F80 X-CRM114-Status: GOOD ( 21.43 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [74.125.82.41 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.41 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders Cc: catalin.marinas@arm.com, will.deacon@arm.com, Ard Biesheuvel X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.53 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 The HYP init bounce page is a runtime construct that ensures that the HYP init code does not cross a page boundary. However, this is something we can do perfectly well at build time, by aligning the code appropriately. For arm64, we just align to PAGE_SIZE, the rationale being that you are either running with 4k page size, in which case the 2k vector table plus the code size don't leave much of the page unused, or you are running with 64k page size, and you don't care about wasting memory anyway. For ARM, the whole code is less than 256 bytes, so we tweak the linker script to align at a power of 2 upper bound of the code size Note that this also fixes a benign off-by-one error in the original bounce page code, where a bounce page would be allocated unnecessarily if the code was exactly 1 page in size. Signed-off-by: Ard Biesheuvel --- arch/arm/kernel/vmlinux.lds.S | 12 +++++++++--- arch/arm/kvm/init.S | 11 +++++++++++ arch/arm/kvm/mmu.c | 42 +++++------------------------------------ arch/arm64/kernel/vmlinux.lds.S | 18 ++++++++++++------ 4 files changed, 37 insertions(+), 46 deletions(-) diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S index b31aa73e8076..8179d3903dee 100644 --- a/arch/arm/kernel/vmlinux.lds.S +++ b/arch/arm/kernel/vmlinux.lds.S @@ -23,7 +23,7 @@ VMLINUX_SYMBOL(__idmap_text_start) = .; \ *(.idmap.text) \ VMLINUX_SYMBOL(__idmap_text_end) = .; \ - . = ALIGN(32); \ + . = ALIGN(1 << __hyp_idmap_align_order); \ VMLINUX_SYMBOL(__hyp_idmap_text_start) = .; \ *(.hyp.idmap.text) \ VMLINUX_SYMBOL(__hyp_idmap_text_end) = .; @@ -346,8 +346,14 @@ SECTIONS */ ASSERT((__proc_info_end - __proc_info_begin), "missing CPU support") ASSERT((__arch_info_end - __arch_info_begin), "no machine record defined") + /* - * The HYP init code can't be more than a page long. + * The HYP init code can't be more than a page long, + * and should not cross a page boundary. * The above comment applies as well. */ -ASSERT(((__hyp_idmap_text_end - __hyp_idmap_text_start) <= PAGE_SIZE), "HYP init code too big") +ASSERT(((__hyp_idmap_text_end - 1) & PAGE_MASK) - + (__hyp_idmap_text_start & PAGE_MASK) == 0, + "HYP init code too big or unaligned") +ASSERT(__hyp_idmap_size <= (1 << __hyp_idmap_align_order), + "__hyp_idmap_size should be <= (1 << __hyp_idmap_align_order)") diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S index 3988e72d16ff..7a279bc8e0e1 100644 --- a/arch/arm/kvm/init.S +++ b/arch/arm/kvm/init.S @@ -157,3 +157,14 @@ target: @ We're now in the trampoline code, switch page tables __kvm_hyp_init_end: .popsection + + /* + * When making changes to this file, make sure that the value of + * __hyp_idmap_align_order is updated if the size of the code ends up + * exceeding (1 << __hyp_idmap_align_order). This helps ensure that the + * code never crosses a page boundary, without wasting too much memory + * on aligning to PAGE_SIZE. + */ + .global __hyp_idmap_size, __hyp_idmap_align_order + .set __hyp_idmap_size, __kvm_hyp_init_end - __kvm_hyp_init + .set __hyp_idmap_align_order, 8 diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 3e6859bc3e11..42a24d6b003b 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -37,7 +37,6 @@ static pgd_t *boot_hyp_pgd; static pgd_t *hyp_pgd; static DEFINE_MUTEX(kvm_hyp_pgd_mutex); -static void *init_bounce_page; static unsigned long hyp_idmap_start; static unsigned long hyp_idmap_end; static phys_addr_t hyp_idmap_vector; @@ -405,9 +404,6 @@ void free_boot_hyp_pgd(void) if (hyp_pgd) unmap_range(NULL, hyp_pgd, TRAMPOLINE_VA, PAGE_SIZE); - free_page((unsigned long)init_bounce_page); - init_bounce_page = NULL; - mutex_unlock(&kvm_hyp_pgd_mutex); } @@ -1498,39 +1494,11 @@ int kvm_mmu_init(void) hyp_idmap_end = kvm_virt_to_phys(__hyp_idmap_text_end); hyp_idmap_vector = kvm_virt_to_phys(__kvm_hyp_init); - if ((hyp_idmap_start ^ hyp_idmap_end) & PAGE_MASK) { - /* - * Our init code is crossing a page boundary. Allocate - * a bounce page, copy the code over and use that. - */ - size_t len = __hyp_idmap_text_end - __hyp_idmap_text_start; - phys_addr_t phys_base; - - init_bounce_page = (void *)__get_free_page(GFP_KERNEL); - if (!init_bounce_page) { - kvm_err("Couldn't allocate HYP init bounce page\n"); - err = -ENOMEM; - goto out; - } - - memcpy(init_bounce_page, __hyp_idmap_text_start, len); - /* - * Warning: the code we just copied to the bounce page - * must be flushed to the point of coherency. - * Otherwise, the data may be sitting in L2, and HYP - * mode won't be able to observe it as it runs with - * caches off at that point. - */ - kvm_flush_dcache_to_poc(init_bounce_page, len); - - phys_base = kvm_virt_to_phys(init_bounce_page); - hyp_idmap_vector += phys_base - hyp_idmap_start; - hyp_idmap_start = phys_base; - hyp_idmap_end = phys_base + len; - - kvm_info("Using HYP init bounce page @%lx\n", - (unsigned long)phys_base); - } + /* + * We rely on the linker script to ensure at build time that the HYP + * init code does not cross a page boundary. + */ + BUG_ON((hyp_idmap_start ^ (hyp_idmap_end - 1)) & PAGE_MASK); hyp_pgd = (pgd_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, hyp_pgd_order); boot_hyp_pgd = (pgd_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, hyp_pgd_order); diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 5d9d2dca530d..a01eedbe59eb 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -23,10 +23,14 @@ jiffies = jiffies_64; #define HYPERVISOR_TEXT \ /* \ - * Force the alignment to be compatible with \ - * the vectors requirements \ + * Align to PAGE_SIZE so that \ + * a) the HYP vector table is at its minimum \ + * alignment of 2048 bytes \ + * b) the HYP init code will not cross a page \ + * boundary if its size does not exceed \ + * PAGE_SIZE (see related ASSERT() below) \ */ \ - . = ALIGN(2048); \ + . = ALIGN(PAGE_SIZE); \ VMLINUX_SYMBOL(__hyp_idmap_text_start) = .; \ *(.hyp.idmap.text) \ VMLINUX_SYMBOL(__hyp_idmap_text_end) = .; \ @@ -163,10 +167,12 @@ SECTIONS } /* - * The HYP init code can't be more than a page long. + * The HYP init code can't be more than a page long, + * and should not cross a page boundary. */ -ASSERT(((__hyp_idmap_text_start + PAGE_SIZE) > __hyp_idmap_text_end), - "HYP init code too big") +ASSERT(((__hyp_idmap_text_end - 1) & PAGE_MASK) - + (__hyp_idmap_text_start & PAGE_MASK) == 0, + "HYP init code too big or unaligned") /* * If padding is applied before .head.text, virt<->phys conversions will fail.