From patchwork Thu Apr 3 17:16:42 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 27699 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ig0-f197.google.com (mail-ig0-f197.google.com [209.85.213.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id E5B202036E for ; Thu, 3 Apr 2014 17:18:00 +0000 (UTC) Received: by mail-ig0-f197.google.com with SMTP id hn18sf4986266igb.0 for ; Thu, 03 Apr 2014 10:18:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :content-type:content-transfer-encoding; bh=TPsaGA18f9AzfT29fy1ldYj8MDI7UxQbBxQWTlvwRX8=; b=A+yr4miK0NaSOGurjOpgUn6qp23o0kfHT8JI1S3qlazUrJOXLGin23kOIQ5E5n+Ixn uPjQttiyCvuOk+NwUGKtgw7bjCZzemSLblSIiVmIX9qCyuPUdIZbBvEppWiarKjdhxGb 76XQNWBOnOIiCorLpoe1kncKzETF0JJB4FYVuG7CjbuoxcbxNi9DS0qVSdWdJPODE5I3 oaKIUz/a1hxe1yFssW2Up+FjyNA8zhjWORhs3triyZ85+rMv1GgWmQgOXc8r5vbT7r8N YgR52SlvXD169ugm/4BQQjv/eFf41lzB0KXRP71tDN8PwYIuRexwzDw5Y6o1/abhK2rd oMRw== X-Gm-Message-State: ALoCoQlzndV4rWfP1v1HVIEanmWU8fokuACSr/1JQ1mwALL1+Rfzl/XfI273M2vxCl3grDFqw/Fo X-Received: by 10.43.88.73 with SMTP id az9mr4298521icc.5.1396545480316; Thu, 03 Apr 2014 10:18:00 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.104.163 with SMTP id a32ls714201qgf.73.gmail; Thu, 03 Apr 2014 10:18:00 -0700 (PDT) X-Received: by 10.52.108.228 with SMTP id hn4mr7265373vdb.43.1396545480208; Thu, 03 Apr 2014 10:18:00 -0700 (PDT) Received: from mail-ve0-f182.google.com (mail-ve0-f182.google.com [209.85.128.182]) by mx.google.com with ESMTPS id i3si1325825vca.185.2014.04.03.10.18.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 03 Apr 2014 10:18:00 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.182; Received: by mail-ve0-f182.google.com with SMTP id jw12so507215veb.27 for ; Thu, 03 Apr 2014 10:18:00 -0700 (PDT) X-Received: by 10.221.28.202 with SMTP id rv10mr3889717vcb.10.1396545479947; Thu, 03 Apr 2014 10:17:59 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.12.8 with SMTP id v8csp46011vcv; Thu, 3 Apr 2014 10:17:58 -0700 (PDT) X-Received: by 10.180.90.50 with SMTP id bt18mr12889012wib.49.1396545478297; Thu, 03 Apr 2014 10:17:58 -0700 (PDT) Received: from casper.infradead.org (casper.infradead.org. [2001:770:15f::2]) by mx.google.com with ESMTPS id fu4si14323700wib.30.2014.04.03.10.17.57 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Apr 2014 10:17:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:770:15f::2 as permitted sender) client-ip=2001:770:15f::2; Received: from merlin.infradead.org ([2001:4978:20e::2]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WVlGf-00030e-5g; Thu, 03 Apr 2014 17:17:33 +0000 Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1WVlGc-00016U-Tx; Thu, 03 Apr 2014 17:17:30 +0000 Received: from fw-tnat.cambridge.arm.com ([217.140.96.21] helo=cam-smtp0.cambridge.arm.com) by merlin.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1WVlGa-00015d-08 for linux-arm-kernel@lists.infradead.org; Thu, 03 Apr 2014 17:17:28 +0000 Received: from e102109-lin.cambridge.arm.com (e102109-lin.cambridge.arm.com [10.1.203.182]) by cam-smtp0.cambridge.arm.com (8.13.8/8.13.8) with ESMTP id s33HH3bJ030524; Thu, 3 Apr 2014 18:17:03 +0100 From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] arm64: Relax the kernel cache requirements for boot Date: Thu, 3 Apr 2014 18:16:42 +0100 Message-Id: <1396545402-10274-1-git-send-email-catalin.marinas@arm.com> X-Mailer: git-send-email 1.7.12.3 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140403_131728_237503_82DDDFD1 X-CRM114-Status: GOOD ( 15.25 ) X-Spam-Score: -3.2 (---) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (-3.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [217.140.96.21 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.6 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: catalin.marinas@arm.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 With system caches for the host OS or architected caches for guest OS we cannot easily guarantee that there are no dirty or stale cache lines for the areas of memory written by the kernel during boot with the MMU off (therefore non-cacheable accesses). This patch adds the necessary cache maintenance during boot and relaxes the booting requirements. Signed-off-by: Catalin Marinas --- This patch is required at least for EFI_STUB on platforms with a system cache (and I really hope it's one that handles standard cache ops by VA) and for guests running UEFI or some boot-loader (with or without system caches) that disable the MMU during boot. Later on, probably useful for kexec as well. In addition, commit 82b2f495fba3 (arm64: virt: ensure visibility of __boot_cpu_mode) will be reverted as it is no longer needed after this patch. Documentation/arm64/booting.txt | 10 ++++++++-- arch/arm64/kernel/head.S | 30 ++++++++++++++++++++++++++++-- arch/arm64/mm/cache.S | 9 +++++++++ 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt index a9691cc48fe3..beb754e87c65 100644 --- a/Documentation/arm64/booting.txt +++ b/Documentation/arm64/booting.txt @@ -111,8 +111,14 @@ Before jumping into the kernel, the following conditions must be met: - Caches, MMUs The MMU must be off. Instruction cache may be on or off. - Data cache must be off and invalidated. - External caches (if present) must be configured and disabled. + The address range corresponding to the loaded kernel image must be + cleaned to the PoC. In the presence of a system cache or other + coherent masters with caches enabled, this will typically require + cache maintenance by VA rather than set/way operations. + System caches which respect the architected cache maintenance by VA + operations must be configured and may be enabled. + System caches which do not respect architected cache maintenance by VA + operations (not recommended) must be configured and disabled. - Architected timers CNTFRQ must be programmed with the timer frequency and CNTVOFF must diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 61035d6814cb..26109682d2fa 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include @@ -229,7 +230,11 @@ ENTRY(set_cpu_boot_mode_flag) cmp w20, #BOOT_CPU_MODE_EL2 b.ne 1f add x1, x1, #4 -1: str w20, [x1] // This CPU has booted in EL1 +1: dc cvac, x1 // Clean potentially dirty cache line + dsb sy + str w20, [x1] // This CPU has booted in EL1 + dc civac, x1 // Clean&invalidate potentially stale cache line + dsb sy ret ENDPROC(set_cpu_boot_mode_flag) @@ -240,8 +245,9 @@ ENDPROC(set_cpu_boot_mode_flag) * This is not in .bss, because we set it sufficiently early that the boot-time * zeroing of .bss would clobber it. */ - .pushsection .data + .pushsection .data..cacheline_aligned ENTRY(__boot_cpu_mode) + .align L1_CACHE_SHIFT .long BOOT_CPU_MODE_EL2 .long 0 .popsection @@ -408,6 +414,15 @@ ENDPROC(__calc_phys_offset) */ __create_page_tables: pgtbl x25, x26, x24 // idmap_pg_dir and swapper_pg_dir addresses + mov x27, lr + + /* + * Invalidate the idmap and swapper page tables to avoid potential + * dirty cache lines being evicted. + */ + mov x0, x25 + add x1, x26, #SWAPPER_DIR_SIZE + bl __inval_cache_range /* * Clear the idmap and swapper page tables. @@ -470,6 +485,17 @@ __create_page_tables: add x0, x26, #2 * PAGE_SIZE // section table address create_pgd_entry x26, x0, x5, x6, x7 #endif + + /* + * Since the page tables have been populated with non-cacheable + * accesses (MMU disabled), invalidate the idmap and swapper page + * tables again to remove any speculatively loaded cache lines. + */ + mov x0, x25 + add x1, x26, #SWAPPER_DIR_SIZE + bl __inval_cache_range + + mov lr, x27 ret ENDPROC(__create_page_tables) .ltorg diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index c46f48b33c14..e803a62e0e45 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -168,6 +168,14 @@ ENTRY(__flush_dcache_area) ENDPROC(__flush_dcache_area) /* + * __inval_cache_range(start, end) + * - start - start address of region + * - end - end address of region + */ +ENTRY(__inval_cache_range) + /* FALLTHROUGH */ + +/* * __dma_inv_range(start, end) * - start - virtual start address of region * - end - virtual end address of region @@ -183,6 +191,7 @@ __dma_inv_range: b.lo 1b dsb sy ret +ENDPROC(__inval_cache_range) ENDPROC(__dma_inv_range) /*