From patchwork Mon Mar 14 08:26:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25D52C433F5 for ; Mon, 14 Mar 2022 08:27:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236676AbiCNI2Q (ORCPT ); Mon, 14 Mar 2022 04:28:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231159AbiCNI2P (ORCPT ); Mon, 14 Mar 2022 04:28:15 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EAB61C90D for ; Mon, 14 Mar 2022 01:27:05 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1D69B61244 for ; Mon, 14 Mar 2022 08:27:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C53F9C340EE; Mon, 14 Mar 2022 08:27:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246423; bh=uEYMh97A01Bpv6vYFf1w8esIt7Uwp6A86+/ymKZ/QmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cFZIQ2VAIElAX2RcajF3ADu3sOABlNhYz7hm682nuUXpv0InwdA3y7ExTtkrn5aRm /u1+gq/3ialsb9lM03XQR43jWofOuVVdSEVlnX5JQt9eC8SIQlqDzwkv3Yr2FdOkbb aOFstDA5qNUIzOf5+299IAj3tSEHxw5ZwZQdsf3RzRTKpFUYqeBvytmp/1puEA/2Wm XVbCTMW5+/cEWbgTEJiThuIbw0J4ZIz3glL8dqS2niPkPlBqrQmMmAEq1XtAfygd/W H9ybQzqFKKzb5EaOYj1QmKPbzS6ooT3pEVWsd+WCdpay58O8RlM4iUjWzRTCWtzjgt nw923Aev7tCUw== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 1/6] Implement a bare metal Rust runtime on top of QEMU's mach-virt Date: Mon, 14 Mar 2022 09:26:39 +0100 Message-Id: <20220314082644.3436071-2-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel Implement the startup sequence to set up a runtime for Rust code, and populate it with a logger wired to the QEMU emulated PL011 UART, and a heap allocator. The .text and .rodata parts are in emulated NOR flash, and the executable pieces execute in place. The .data and .bss sections as well as the stack are disjoint from the flash image, and reside in DRAM. The assembler startup code sets up the stack pointer and initializes the writable sections. The startup code programs the MMU with a set of translation tables in NOR flash, which describe a 2M R-X region in flash, a 2 M R-- region covering the base of DRAM (which is where QEMU's mach-virt puts the device tree), and 2M RW- region used for the stack, .data/.bss and the heap. --- .cargo/config | 5 + .gitignore | 2 + Cargo.lock | 73 ++++++++++++ Cargo.toml | 10 ++ efilite.lds | 61 ++++++++++ src/console.rs | 57 +++++++++ src/cstring.rs | 9 ++ src/head.S | 121 ++++++++++++++++++++ src/main.rs | 49 +++++++- src/ttable.S | 37 ++++++ 10 files changed, 422 insertions(+), 2 deletions(-) diff --git a/.cargo/config b/.cargo/config new file mode 100644 index 000000000000..584568a162de --- /dev/null +++ b/.cargo/config @@ -0,0 +1,5 @@ +[target.aarch64-unknown-linux-gnu] +rustflags = ["-C", "relocation-model=static", "-C", "link-arg=-Wl,-Tefilite.lds,--orphan-handling=warn", "-C", "link-arg=-nostartfiles"] + +[build] +target = "aarch64-unknown-linux-gnu" diff --git a/.gitignore b/.gitignore index ea8c4bf7f35f..c5a7561a896d 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,3 @@ /target +*.bin +.*.swp diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 000000000000..617acc9c6086 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,73 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "cfg-if" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" + +[[package]] +name = "efilite" +version = "0.1.0" +dependencies = [ + "linked_list_allocator", + "log", + "mmio", + "rlibc", +] + +[[package]] +name = "linked_list_allocator" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "549ce1740e46b291953c4340adcd74c59bcf4308f4cac050fd33ba91b7168f4a" +dependencies = [ + "spinning_top", +] + +[[package]] +name = "lock_api" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "88943dd7ef4a2e5a4bfa2753aaab3013e34ce2533d1996fb18ef591e315e2b3b" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "51b9bbe6c47d51fc3e1a9b945965946b4c44142ab8792c50835a980d362c2710" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "mmio" +version = "2.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee857bfd0b37394f3507d78ee7bd4b712a2179a2ce50e47d36bbb481672f5408" + +[[package]] +name = "rlibc" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc874b127765f014d792f16763a81245ab80500e2ad921ed4ee9e82481ee08fe" + +[[package]] +name = "scopeguard" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd" + +[[package]] +name = "spinning_top" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75adad84ee84b521fb2cca2d4fd0f1dab1d8d026bda3c5bea4ca63b5f9f9293c" +dependencies = [ + "lock_api", +] diff --git a/Cargo.toml b/Cargo.toml index 07cf0efb7baf..9bc2b39f6e9b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -6,3 +6,13 @@ edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies] +rlibc = "1.0.0" +linked_list_allocator = "0.9.1" +log = "0.4.14" +mmio = "2.1.0" + +[profile.dev] +panic = "abort" + +[profile.release] +panic = "abort" diff --git a/efilite.lds b/efilite.lds new file mode 100644 index 000000000000..0632cbaf8e4e --- /dev/null +++ b/efilite.lds @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +MEMORY +{ + flash : ORIGIN = 0, LENGTH = 2M + ram : ORIGIN = 0x40000000, LENGTH = 4M +} + +PROVIDE(_init_base = 0x40000000); +PROVIDE(_init_size = 0x400000); + +ENTRY(__init) + +SECTIONS +{ + .text : { + *(.head) + *(.text .text*) + *(.rodata .rodata*) + *(.got .got.plt) + } >flash + + /* + * QEMU passes the DT blob by storing it at the base of DRAM + * before starting the guest + */ + .dtb (NOLOAD) : { + _dtb = .; + . += 0x200000; + } >ram + + /* + * put the stack first so we will notice if we overrun and + * hit the R/O mapping of the DT blob + */ + .stack (NOLOAD) : { + . += 0x4000; + _stack_end = .; + } >ram + + .data : ALIGN(32) { + _data = .; + *(.data .data*) + . = ALIGN(32); + _edata = .; + } >ram AT >flash + + data_lma = LOADADDR(.data); + + .bss : ALIGN (32) { + _bss_start = .; + *(.bss .bss*) + . = ALIGN(32); + _bss_end = .; + _end = .; + } >ram + + /DISCARD/ : { + *(.note*) + } +} diff --git a/src/console.rs b/src/console.rs new file mode 100644 index 000000000000..3841c6cb2dd0 --- /dev/null +++ b/src/console.rs @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use core::fmt::Write; +use log::{Level, Metadata, Record}; +use mmio::{Allow, Deny, VolBox}; + +pub struct QemuSerialConsole { + base: u64, +} + +struct QemuSerialConsoleWriter<'a> { + console: &'a QemuSerialConsole, +} + +impl QemuSerialConsole { + fn puts(&self, s: &str) { + // + // This is technically racy, as nothing is preventing concurrent accesses to the UART if we + // model it this way. However, this is a debug tool only, and we never read back the + // register value so any races cannot have any observeable side effects to the program + // itself. + // + let mut out = unsafe { VolBox::::new(self.base as *mut u32) }; + + for b in s.as_bytes().iter() { + if *b == b'\n' { + out.write(b'\r' as u32); + } + out.write(*b as u32) + } + } +} + +impl Write for QemuSerialConsoleWriter<'_> { + fn write_str(&mut self, s: &str) -> core::fmt::Result { + self.console.puts(s); + Ok(()) + } +} + +impl log::Log for QemuSerialConsole { + fn enabled(&self, metadata: &Metadata) -> bool { + metadata.level() <= Level::Info + } + + fn log(&self, record: &Record) { + if self.enabled(record.metadata()) { + let mut out = QemuSerialConsoleWriter { console: &self }; + write!(&mut out, "{} - {}", record.level(), record.args()).unwrap(); + } + } + + fn flush(&self) {} +} + +// The primary UART of QEMU's mach-virt +pub static OUT: QemuSerialConsole = QemuSerialConsole { base: 0x900_0000 }; diff --git a/src/cstring.rs b/src/cstring.rs new file mode 100644 index 000000000000..b6a5c4308067 --- /dev/null +++ b/src/cstring.rs @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +extern crate rlibc; +use rlibc::memcmp; + +#[no_mangle] +pub unsafe extern "C" fn bcmp(s1: *const u8, s2: *const u8, len: usize) -> i32 { + memcmp(s1, s2, len) +} diff --git a/src/head.S b/src/head.S new file mode 100644 index 000000000000..b82dca4325fa --- /dev/null +++ b/src/head.S @@ -0,0 +1,121 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + + .macro adr_l, reg:req, sym:req + adrp \reg, \sym + add \reg, \reg, :lo12:\sym + .endm + + .macro mov_i, reg:req, imm:req + movz \reg, :abs_g2:\imm + movk \reg, :abs_g1_nc:\imm + movk \reg, :abs_g0_nc:\imm + .endm + + .section ".head", "ax", %progbits + .globl __init +__init: + mov_i x0, mairval + mov_i x1, tcrval + adrp x2, idmap + mov_i x3, sctlrval + mov_i x4, cpacrval + adr_l x5, vector_table + + msr mair_el1, x0 // set up the 1:1 mapping + msr tcr_el1, x1 + msr ttbr0_el1, x2 + isb + + tlbi vmalle1 // invalidate any cached translations + ic iallu // invalidate the I-cache + dsb nsh + isb + + msr sctlr_el1, x3 // enable MMU and caches + msr cpacr_el1, x4 // enable FP/SIMD + msr vbar_el1, x5 // enable exception handling + isb + + adr_l x0, _data // initialize the .data section + adr_l x1, _edata + adr_l x2, data_lma +0: cmp x0, x1 + b.ge 1f + ldp q0, q1, [x2], #32 + stp q0, q1, [x0], #32 + b 0b + +1: adr_l x0, _bss_start // wipe the .bss section + adr_l x1, _bss_end + movi v0.16b, #0 +2: cmp x0, x1 + b.ge 3f + stp q0, q0, [x0], #32 + b 2b + +3: mov x29, xzr // initialize the frame pointer + adrp x0, _stack_end + mov sp, x0 + adrp x0, _init_base // initial DRAM base address + movz x1, :abs_g1:_init_size // initially mapped area + adr_l x2, _end // statically allocated by program + sub x2, x2, x0 + bl efilite_main + +4: mov_i x0, 0x84000008 // PSCI SYSTEM OFF + hvc #0 + wfi + b 4b + + .macro vector_entry + adrp x0, idmap + adrp x1, _stack_end + msr ttbr0_el1, x0 // switch back to the initial ID map + isb + mov sp, x1 // reset the stack pointer + mov x29, xzr + mrs x0, esr_el1 + mrs x1, elr_el1 + mrs x2, far_el1 + bl handle_exception + .endm + + .section ".text", "ax", %progbits + .align 11 +vector_table: + vector_entry + .org vector_table + 0x200 + vector_entry + .org vector_table + 0x400 + vector_entry + .org vector_table + 0x600 + vector_entry + + .set .L_MAIR_DEV_nGnRE, 0x04 + .set .L_MAIR_MEM_WBWA, 0xff + .set mairval, .L_MAIR_DEV_nGnRE | (.L_MAIR_MEM_WBWA << 8) + + .set .L_TCR_TG0_4KB, 0x0 << 14 + .set .L_TCR_TG1_4KB, 0x2 << 30 + .set .L_TCR_IPS_64GB, 0x1 << 32 + .set .L_TCR_EPD1, 0x1 << 23 + .set .L_TCR_SH_INNER, 0x3 << 12 + .set .L_TCR_RGN_OWB, 0x1 << 10 + .set .L_TCR_RGN_IWB, 0x1 << 8 + .set tcrval, .L_TCR_TG0_4KB | .L_TCR_TG1_4KB | .L_TCR_EPD1 | .L_TCR_IPS_64GB | .L_TCR_RGN_OWB + .set tcrval, tcrval | .L_TCR_RGN_IWB | .L_TCR_SH_INNER | (64 - 36) // TCR_T0SZ + + .set .L_SCTLR_ELx_I, 0x1 << 12 + .set .L_SCTLR_ELx_SA, 0x1 << 3 + .set .L_SCTLR_ELx_C, 0x1 << 2 + .set .L_SCTLR_ELx_M, 0x1 << 0 + .set .L_SCTLR_EL1_SPAN, 0x1 << 23 + .set .L_SCTLR_EL1_WXN, 0x1 << 19 + .set .L_SCTLR_EL1_SED, 0x1 << 8 + .set .L_SCTLR_EL1_ITD, 0x1 << 7 + .set .L_SCTLR_EL1_RES1, (0x1 << 11) | (0x1 << 20) | (0x1 << 22) | (0x1 << 28) | (0x1 << 29) + .set sctlrval, .L_SCTLR_ELx_M | .L_SCTLR_ELx_C | .L_SCTLR_ELx_SA | .L_SCTLR_EL1_ITD | .L_SCTLR_EL1_SED + .set sctlrval, sctlrval | .L_SCTLR_ELx_I | .L_SCTLR_EL1_WXN | .L_SCTLR_EL1_SPAN | .L_SCTLR_EL1_RES1 + + .set .L_CPACR_EL1_FPEN, 0x3 << 20 + .set cpacrval, .L_CPACR_EL1_FPEN diff --git a/src/main.rs b/src/main.rs index e7a11a969c03..698e9c5724bf 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,3 +1,48 @@ -fn main() { - println!("Hello, world!"); +/* SPDX-License-Identifier: GPL-2.0 */ + +#![no_std] +#![no_main] + +mod console; +mod cstring; + +use core::{arch::global_asm, panic::PanicInfo}; +use linked_list_allocator::LockedHeap; +use log::{error, info}; + +#[global_allocator] +pub static ALLOCATOR: LockedHeap = LockedHeap::empty(); + +#[no_mangle] +extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { + #[cfg(debug_assertions)] + log::set_logger(&console::OUT) + .map(|()| log::set_max_level(log::LevelFilter::Info)) + .unwrap(); + + // Give the mapped but unused memory to the heap allocator + info!( + "Heap allocator with {} KB of memory\n", + (mapped - used) / 1024 + ); + unsafe { + ALLOCATOR.lock().init(base + used, mapped - used); + } } + +#[no_mangle] +extern "C" fn handle_exception(esr: u64, elr: u64, far: u64) -> ! { + panic!( + "Unhandled exception: ESR = 0x{:X}, ELR = 0x{:X}, FAR = 0x{:X}.", + esr, elr, far + ); +} + +#[panic_handler] +fn panic(info: &PanicInfo) -> ! { + error!("{}\n", info); + loop {} +} + +global_asm!(include_str!("head.S")); +global_asm!(include_str!("ttable.S")); diff --git a/src/ttable.S b/src/ttable.S new file mode 100644 index 000000000000..b6fdadb6dbdc --- /dev/null +++ b/src/ttable.S @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + + .set .L_TT_TYPE_BLOCK, 0x1 + .set .L_TT_TYPE_TABLE, 0x3 + + .set .L_TT_AF, 0x1 << 10 + .set .L_TT_NG, 0x3 << 11 + .set .L_TT_RO, 0x2 << 6 + .set .L_TT_XN, 0x3 << 53 + + .set .L_TT_MT_DEV, 0x0 << 2 // MAIR #0 + .set .L_TT_MT_MEM, (0x1 << 2) | (0x3 << 8) // MAIR #1 + + .set BLOCK_XIP, .L_TT_TYPE_BLOCK | .L_TT_MT_MEM | .L_TT_AF | .L_TT_RO + .set BLOCK_DEV, .L_TT_TYPE_BLOCK | .L_TT_MT_DEV | .L_TT_AF | .L_TT_XN + .set BLOCK_MEM, .L_TT_TYPE_BLOCK | .L_TT_MT_MEM | .L_TT_AF | .L_TT_XN | .L_TT_NG + + .section ".rodata", "a", %progbits + .align 12 + /* level 2 */ +0: .quad BLOCK_XIP // 2 MB of R-X flash + .fill 63, 8, 0x0 // 126 MB of unused flash + .set idx, 64 + .rept 448 + .quad BLOCK_DEV | (idx << 21) // 896 MB of RW- device mappings + .set idx, idx + 1 + .endr +1: .quad BLOCK_XIP | 0x40000000 // DT provided by VMM + .quad BLOCK_MEM | 0x40200000 // 2 MB of DRAM + .fill 510, 8, 0x0 + + .globl idmap +idmap: + /* level 1 */ + .quad 0b + .L_TT_TYPE_TABLE // flash and device mappings + .quad 1b + .L_TT_TYPE_TABLE // up to 1 GB of DRAM + .fill 62, 8, 0x0 // 62 GB of remaining VA space From patchwork Mon Mar 14 08:26:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C05C433EF for ; Mon, 14 Mar 2022 08:27:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231159AbiCNI2R (ORCPT ); Mon, 14 Mar 2022 04:28:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237028AbiCNI2P (ORCPT ); Mon, 14 Mar 2022 04:28:15 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B72421CB0B for ; Mon, 14 Mar 2022 01:27:06 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4F6DC6125C for ; Mon, 14 Mar 2022 08:27:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F4005C340F6; Mon, 14 Mar 2022 08:27:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246425; bh=vzSBjJ1sxDwQUc1X0CmXZXicnEitrFeXZRaxKaqJLb8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hOmF36EguKA2IrnDnWkzFjcwzCUk5QZ6sGQF3A6qqs4LegPuTtWerBg3orBrpoQPk vJoxzdM/RZB6DRTJxyiBqvgI2A2JU7Irawf4F8sc1zuZeuVHNGj8Ms0bBksPsc0N1L WbCi1DiynEFA8DQXQcbhHwtEWWZVt1VkdeUy9NzDoC1in5gBTHOl0aEua5Viq7adEW J1Oxk9GC1h6mqj18JEbxV5W8fy6CQSoumoH7TcJXqsPnhFAG92X1NWyeYGLLHYPmvJ qYlrnKz0fB1LMtdykJ/Dr9CX1NX4hIEuKf80Q/+DXJ9/ROaYgKOayQSwWTh7wFidlw UrYbJkrMj1JFQ== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 2/6] Add DTB processing Date: Mon, 14 Mar 2022 09:26:40 +0100 Message-Id: <20220314082644.3436071-3-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel Add handling of the QEMU provided DTB, which we will need to consult to find the DRAM layout and the fwcfg device. Initially, just dump the /chosen/bootargs only, if one is provided. --- Cargo.lock | 7 +++++++ Cargo.toml | 1 + src/main.rs | 10 ++++++++++ 3 files changed, 18 insertions(+) diff --git a/Cargo.lock b/Cargo.lock index 617acc9c6086..2750d4a3937c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -12,12 +12,19 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" name = "efilite" version = "0.1.0" dependencies = [ + "fdt", "linked_list_allocator", "log", "mmio", "rlibc", ] +[[package]] +name = "fdt" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b643857cf70949306b81d7e92cb9d47add673868edac9863c4a49c42feaf3f1e" + [[package]] name = "linked_list_allocator" version = "0.9.1" diff --git a/Cargo.toml b/Cargo.toml index 9bc2b39f6e9b..b073376d9e16 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -10,6 +10,7 @@ rlibc = "1.0.0" linked_list_allocator = "0.9.1" log = "0.4.14" mmio = "2.1.0" +fdt = "0.1.3" [profile.dev] panic = "abort" diff --git a/src/main.rs b/src/main.rs index 698e9c5724bf..6d880732b469 100644 --- a/src/main.rs +++ b/src/main.rs @@ -13,6 +13,10 @@ use log::{error, info}; #[global_allocator] pub static ALLOCATOR: LockedHeap = LockedHeap::empty(); +extern "C" { + static _dtb: u8; +} + #[no_mangle] extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { #[cfg(debug_assertions)] @@ -28,6 +32,12 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { unsafe { ALLOCATOR.lock().init(base + used, mapped - used); } + + let fdt = unsafe { fdt::Fdt::from_ptr(&_dtb).expect("Failed to parse device tree") }; + + fdt.chosen() + .bootargs() + .map(|args| info!("/chosen/bootargs: {:?}\n", args)); } #[no_mangle] From patchwork Mon Mar 14 08:26:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAF7CC433EF for ; Mon, 14 Mar 2022 08:27:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237690AbiCNI2U (ORCPT ); Mon, 14 Mar 2022 04:28:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237028AbiCNI2T (ORCPT ); Mon, 14 Mar 2022 04:28:19 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 289CC1C90D for ; Mon, 14 Mar 2022 01:27:09 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A917D6124F for ; Mon, 14 Mar 2022 08:27:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F444C340EE; Mon, 14 Mar 2022 08:27:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246428; bh=ubbdt9lUOLVXhY8roFjfmBcF2EH6IX9ywQRSlISbxk4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jkT3yxu/Vdlndvx1q/NdeAIfYBwwKClIi13/NhQYfl+3+hlB10tlbIGp+Ps61uelG PJD0QiO5qtjy13I/7ecMwpN/2SQozdWtW0utEVVcS/eal2fVqXyCScxyanzGhcRYs8 LA+Gs7eYSUBqK3SJNzd4y8qQiQtVfYuui/8pU+eUtGq9oVn4HFAWnS2mkGT7MDcQbz w3XS2fBb2Kzbu3DcknId+OEtvR4Xn04gxaSFdcVDA3edmCSCyyX2zhNIvfeFdwQy5V qQfdzx6lXKtIs53h3/srMydxIVM9oe/gEh2dBSU4atwCavOoZiUjo5DtYCOExiX8j+ 36M21a1jEr0Aw== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 3/6] Add paging code to manage the full ID map Date: Mon, 14 Mar 2022 09:26:41 +0100 Message-Id: <20220314082644.3436071-4-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel We enter with a minimal ID map carried in NOR flash, but in order to load anything into DRAM and boot it, we need to map the memory first. So add a paging module loosely based on the libhermit project's code, but with a few tweaks and necessary fixes. (In particular, remove all the code that is irrelevant for 1:1 mappings) This code needs some more work: it currently greedily maps all memory ranges down to pages, which is unnecessary, and costly in terms of heap footprint. It also fails to deal with the need to split block mappings into table mappings. --- Cargo.lock | 7 + Cargo.toml | 1 + efilite.lds | 1 + src/main.rs | 44 ++ src/pagealloc.rs | 44 ++ src/paging.rs | 499 ++++++++++++++++++++ 6 files changed, 596 insertions(+) diff --git a/Cargo.lock b/Cargo.lock index 2750d4a3937c..8ad5db72fef6 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2,6 +2,12 @@ # It is not intended for manual editing. version = 3 +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + [[package]] name = "cfg-if" version = "1.0.0" @@ -12,6 +18,7 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd" name = "efilite" version = "0.1.0" dependencies = [ + "bitflags", "fdt", "linked_list_allocator", "log", diff --git a/Cargo.toml b/Cargo.toml index b073376d9e16..defa16ec4ab1 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -11,6 +11,7 @@ linked_list_allocator = "0.9.1" log = "0.4.14" mmio = "2.1.0" fdt = "0.1.3" +bitflags = "1.3" [profile.dev] panic = "abort" diff --git a/efilite.lds b/efilite.lds index 0632cbaf8e4e..e460f9e9b917 100644 --- a/efilite.lds +++ b/efilite.lds @@ -27,6 +27,7 @@ SECTIONS .dtb (NOLOAD) : { _dtb = .; . += 0x200000; + _dtb_end = .; } >ram /* diff --git a/src/main.rs b/src/main.rs index 6d880732b469..af58ccc0318d 100644 --- a/src/main.rs +++ b/src/main.rs @@ -2,19 +2,30 @@ #![no_std] #![no_main] +// needed by the paging code +#![allow(incomplete_features)] +#![feature(specialization)] mod console; mod cstring; +mod pagealloc; +mod paging; use core::{arch::global_asm, panic::PanicInfo}; use linked_list_allocator::LockedHeap; use log::{error, info}; +use crate::paging::{PageTableEntryFlags, VirtAddr}; + +#[macro_use] +extern crate bitflags; + #[global_allocator] pub static ALLOCATOR: LockedHeap = LockedHeap::empty(); extern "C" { static _dtb: u8; + static _dtb_end: u8; } #[no_mangle] @@ -38,6 +49,39 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { fdt.chosen() .bootargs() .map(|args| info!("/chosen/bootargs: {:?}\n", args)); + + paging::init(); + + let mut mem_flags = PageTableEntryFlags::empty(); + mem_flags.normal().non_global().execute_disable(); + + info!("Mapping all DRAM regions found in the DT:\n"); + for reg in fdt.memory().regions() { + if let Some(size) = reg.size { + paging::map_range(reg.starting_address as VirtAddr, size as u64, mem_flags); + } + } + + info!("Remapping initial DRAM regions:\n"); + + // Ensure that the initial DRAM region remains mapped + paging::map_range(base as VirtAddr, mapped as u64, mem_flags); + + // Ensure that the DT retains its global R/O mapping + let mut nor_flags = PageTableEntryFlags::empty(); + nor_flags.normal().read_only(); + paging::map_range( + unsafe { &_dtb as *const _ } as VirtAddr, + unsafe { &_dtb_end as *const _ as u64 - &_dtb as *const _ as u64 }, + nor_flags, + ); + + // Switch to the new ID map so we can use all of DRAM + paging::activate(); + + // Switch back to the initial ID map so we can remap + // the loaded kernel image with different permissions + paging::deactivate(); } #[no_mangle] diff --git a/src/pagealloc.rs b/src/pagealloc.rs new file mode 100644 index 000000000000..f91043000033 --- /dev/null +++ b/src/pagealloc.rs @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use core::alloc::GlobalAlloc; +use core::arch::asm; + +use crate::paging::{BasePageSize, PageSize}; +use crate::ALLOCATOR; + +const DCZID_BS_MASK: u64 = 0xf; + +pub fn get_zeroed_page() -> u64 { + let layout = + core::alloc::Layout::from_size_align(BasePageSize::SIZE, BasePageSize::SIZE).unwrap(); + let page = unsafe { ALLOCATOR.alloc(layout) }; + if page.is_null() { + panic!("Out of memory!"); + } + + let dczid = unsafe { + let mut l: u64; + asm!("mrs {reg}, dczid_el0", + reg = out(reg) l, + options(pure, nomem, nostack, preserves_flags), + ); + l + }; + + let line_shift = 2 + (dczid & DCZID_BS_MASK); + let line_size: isize = 1 << line_shift; + let num_lines = BasePageSize::SIZE >> line_shift; + let mut offset: isize = 0; + + for _ in 1..=num_lines { + unsafe { + asm!( + "dc zva, {line}", + line = in(reg) page.offset(offset), + options(nostack, preserves_flags), + ); + } + offset += line_size; + } + page as u64 +} diff --git a/src/paging.rs b/src/paging.rs new file mode 100644 index 000000000000..222c40e47c78 --- /dev/null +++ b/src/paging.rs @@ -0,0 +1,499 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +// Mostly derived from +// https://github.com/hermitcore/libhermit-rs/blob/master/src/arch/aarch64/mm/paging.rs + +use core::arch::asm; +use core::marker::PhantomData; +use core::mem; +use log::info; + +use crate::pagealloc; + +// Use a different ASID for the full ID map, as well as non-global attributes +// for all its DRAM mappings. This way, we can ignore break-before-make rules +// entirely when breaking down block mappings, as long as we don't do so while +// the full ID map is active. +const ASID: u64 = 1; + +// Number of Offset bits of a virtual address for a 4 KiB page, which are shifted away to get its Page Frame Number (PFN). +const PAGE_BITS: usize = 12; + +// Number of bits of the index in each table (L0Table, L1Table, L2Table, L3Table). +const PAGE_MAP_BITS: usize = 9; + +// A mask where PAGE_MAP_BITS are set to calculate a table index. +const PAGE_MAP_MASK: usize = 0x1FF; + +#[repr(C, align(512))] +struct IdMap { + entries: [u64; 64], +} + +extern "C" { + // Root level of the initial ID map in NOR flash + static idmap: IdMap; +} + +// Root level of the full ID map managed by this code +static mut IDMAP: IdMap = IdMap { + entries: [0u64; 64], +}; + +macro_rules! align_down { + ($value:expr, $alignment:expr) => { + ($value) & !($alignment - 1) + }; +} + +pub fn init() { + // Clone the first entry, which covers the first 1 GB of PA space + // It contains the NOR flash and most peripherals + unsafe { + IDMAP.entries[0] = idmap.entries[0]; + } +} + +pub type VirtAddr = u64; +pub type PhysAddr = u64; + +// A memory page of the size given by S. +#[derive(Clone, Copy)] +struct Page { + // Virtual memory address of this page. + // This is rounded to a page size boundary on creation. + virtual_address: VirtAddr, + + // Required by Rust to support the S parameter. + size: PhantomData, +} + +impl Page { + // Returns a PageIter to iterate from the given first Page to the given last Page (inclusive). + fn range(first: Self, last: Self) -> PageIter { + assert!(first.virtual_address <= last.virtual_address); + PageIter { + current: first, + last: last, + } + } + + // Returns a Page including the given virtual address. + // That means, the address is rounded down to a page size boundary. + fn including_address(virtual_address: VirtAddr) -> Self { + Self { + virtual_address: align_down!(virtual_address, S::SIZE as u64), + size: PhantomData, + } + } + + // Returns the index of this page in the table given by L. + fn table_index(&self) -> usize { + assert!(L::LEVEL <= S::MAP_LEVEL); + self.virtual_address as usize >> PAGE_BITS >> (3 - L::LEVEL) * PAGE_MAP_BITS & PAGE_MAP_MASK + } +} + +struct PageIter { + current: Page, + last: Page, +} + +impl Iterator for PageIter { + type Item = Page; + + fn next(&mut self) -> Option> { + if self.current.virtual_address <= self.last.virtual_address { + let p = self.current; + self.current.virtual_address += S::SIZE as u64; + Some(p) + } else { + None + } + } +} + +fn get_page_range(virtual_address: VirtAddr, count: u64) -> PageIter { + let first_page = Page::::including_address(virtual_address); + let last_page = Page::::including_address(virtual_address + (count - 1) * S::SIZE as u64); + Page::range(first_page, last_page) +} + +fn map(base: VirtAddr, size: u64, flags: PageTableEntryFlags) { + info!( + "Mapping memory at [0x{:X} - 0x{:X}] {:?}\n", + base, + base + size - 1, + flags + ); + + let range = get_page_range::(base, size / S::SIZE as u64); + let root_pagetable = unsafe { + &mut *mem::transmute::<*mut u64, *mut PageTable>(&IDMAP as *const _ as *mut u64) + }; + root_pagetable.map_pages(range, base, flags); +} + +pub fn map_range(base: VirtAddr, size: u64, flags: PageTableEntryFlags) { + map::(base, size, flags); +} + +pub fn activate() { + unsafe { + asm!( + "msr ttbr0_el1, {ttbrval}", + "isb", + ttbrval = in(reg) &IDMAP as *const _ as u64 | (ASID << 48), + options(preserves_flags), + ); + } +} + +pub fn deactivate() { + unsafe { + asm!( + "msr ttbr0_el1, {ttbrval}", + "isb", + "tlbi aside1, {asid}", + "dsb nsh", + "isb", + asid = in(reg) ASID << 48, + ttbrval = in(reg) &idmap as *const _ as u64, + options(preserves_flags), + ); + } +} + +bitflags! { + // Useful flags for an entry in either table (L0Table, L1Table, L2Table, L3Table). + // + // See ARM Architecture Reference Manual, ARMv8, for ARMv8-A Reference Profile, Issue C.a, Chapter D4.3.3 + pub struct PageTableEntryFlags: u64 { + // Set if this entry is valid. + const PRESENT = 1 << 0; + + // Set if this entry points to a table or a 4 KiB page. + const TABLE_OR_4KIB_PAGE = 1 << 1; + + // Set if this entry points to device memory (non-gathering, non-reordering, no early write acknowledgement) + const DEVICE_NGNRE = 0 << 4 | 0 << 3 | 0 << 2; + + // Set if this entry points to normal memory (cacheable) + const NORMAL = 0 << 4 | 0 << 3 | 1 << 2; + + // Set if memory referenced by this entry shall be read-only. + const READ_ONLY = 1 << 7; + + // Set if this entry shall be shared between all cores of the system. + const INNER_SHAREABLE = 1 << 8 | 1 << 9; + + // Set if software has accessed this entry (for memory access or address translation). + const ACCESSED = 1 << 10; + + // Set if this entry is scoped by ASID + const NON_GLOBAL = 1 << 11; + + // Set if code execution shall be disabled for memory referenced by this entry in privileged mode. + const PRIVILEGED_EXECUTE_NEVER = 1 << 53; + + // Set if code execution shall be disabled for memory referenced by this entry in unprivileged mode. + const UNPRIVILEGED_EXECUTE_NEVER = 1 << 54; + + // Self-reference to the Level 0 page table + const SELF = 1 << 55; + } +} + +impl PageTableEntryFlags { + // An empty set of flags for unused/zeroed table entries. + // Needed as long as empty() is no const function. + const BLANK: PageTableEntryFlags = PageTableEntryFlags { bits: 0 }; + + // pub fn device(&mut self) -> &mut Self { + // self.insert(PageTableEntryFlags::DEVICE_NGNRE); + // self + // } + + pub fn normal(&mut self) -> &mut Self { + self.insert(PageTableEntryFlags::NORMAL); + self + } + + pub fn read_only(&mut self) -> &mut Self { + self.insert(PageTableEntryFlags::READ_ONLY); + self + } + + // pub fn writable(&mut self) -> &mut Self { + // self.remove(PageTableEntryFlags::READ_ONLY); + // self + // } + + pub fn non_global(&mut self) -> &mut Self { + self.insert(PageTableEntryFlags::NON_GLOBAL); + self + } + + pub fn execute_disable(&mut self) -> &mut Self { + self.insert(PageTableEntryFlags::PRIVILEGED_EXECUTE_NEVER); + self.insert(PageTableEntryFlags::UNPRIVILEGED_EXECUTE_NEVER); + self + } +} + +// An interface to allow for a generic implementation of struct PageTable for all 4 page tables. +// Must be implemented by all page tables. +trait PageTableLevel { + // Numeric page table level + const LEVEL: usize; +} + +trait PageTableLevelWithSubtables: PageTableLevel { + type SubtableLevel; +} + +// The Level 1 Table (can map 1 GiB pages) +enum L1Table {} +impl PageTableLevel for L1Table { + const LEVEL: usize = 1; +} + +impl PageTableLevelWithSubtables for L1Table { + type SubtableLevel = L2Table; +} + +// The Level 2 Table (can map 2 MiB pages) +enum L2Table {} +impl PageTableLevel for L2Table { + const LEVEL: usize = 2; +} + +impl PageTableLevelWithSubtables for L2Table { + type SubtableLevel = L3Table; +} + +// The Level 3 Table (can map 4 KiB pages) +enum L3Table {} +impl PageTableLevel for L3Table { + const LEVEL: usize = 3; +} + +// Representation of any page table in memory. +// Parameter L supplies information for Rust's typing system to distinguish between the different tables. +#[repr(align(4096))] +struct PageTable { + // Each page table has at most 512 entries (can be calculated using PAGE_MAP_BITS). + entries: [PageTableEntry; 1 << PAGE_MAP_BITS], + + // Required by Rust to support the L parameter. + level: PhantomData, +} + +// A trait defining methods every page table has to implement. +// This additional trait is necessary to make use of Rust's specialization feature and provide a default +// implementation of some methods. +trait PageTableMethods { + fn map_page_in_this_table( + &mut self, + page: Page, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ); + fn map_page( + &mut self, + page: Page, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ); +} + +// An entry in either table +#[derive(Clone, Copy)] +pub struct PageTableEntry { + // Physical memory address this entry refers, combined with flags from PageTableEntryFlags. + physical_address_and_flags: PhysAddr, +} + +impl PageTableEntry { + // Returns whether this entry is valid (present). + fn is_present(&self) -> bool { + (self.physical_address_and_flags & PageTableEntryFlags::PRESENT.bits()) != 0 + } + + // Mark this as a valid (present) entry and set address translation and flags. + // + // # Arguments + // + // * `physical_address` - The physical memory address this entry shall translate to + // * `flags` - Flags from PageTableEntryFlags (note that the PRESENT, INNER_SHAREABLE, and ACCESSED flags are set automatically) + fn set(&mut self, physical_address: PhysAddr, flags: PageTableEntryFlags) { + // Verify that the offset bits for a 4 KiB page are zero. + assert_eq!( + physical_address % BasePageSize::SIZE as u64, + 0, + "Physical address is not on a 4 KiB page boundary (physical_address = {:#X})", + physical_address + ); + + let mut flags_to_set = flags; + flags_to_set.insert(PageTableEntryFlags::PRESENT); + self.physical_address_and_flags = physical_address | flags_to_set.bits(); + } +} + +impl PageTableMethods for PageTable { + // Maps a single page in this table to the given physical address. + // + // Must only be called if a page of this size is mapped at this page table level! + fn map_page_in_this_table( + &mut self, + page: Page, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ) { + assert_eq!(L::LEVEL, S::MAP_LEVEL); + let index = page.table_index::(); + + if flags == PageTableEntryFlags::BLANK { + // in this case we unmap the pages + self.entries[index].set(physical_address, flags); + } else { + let mut flags_to_set = flags; + flags_to_set.insert(PageTableEntryFlags::INNER_SHAREABLE); + flags_to_set.insert(PageTableEntryFlags::ACCESSED); + self.entries[index].set(physical_address, S::MAP_EXTRA_FLAG | flags_to_set); + } + } + + // Maps a single page to the given physical address. + // + // This is the default implementation that just calls the map_page_in_this_table method. + // It is overridden by a specialized implementation for all tables with sub tables (all except L3Table). + default fn map_page( + &mut self, + page: Page, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ) { + self.map_page_in_this_table::(page, physical_address, flags) + } +} + +impl PageTable +where + L::SubtableLevel: PageTableLevel, +{ + // Returns the next subtable for the given page in the page table hierarchy. + // + // Must only be called if a page of this size is mapped in a subtable! + fn subtable(&self, page: Page) -> &mut PageTable { + assert!(L::LEVEL < S::MAP_LEVEL); + + // Calculate the address of the subtable. + let index = page.table_index::(); + let subtable_address = + self.entries[index].physical_address_and_flags & !((1 << PAGE_BITS) - 1); + unsafe { &mut *(subtable_address as *mut PageTable) } + } + + // Maps a continuous range of pages. + // + // # Arguments + // + // * `range` - The range of pages of size S + // * `physical_address` - First physical address to map these pages to + // * `flags` - Flags from PageTableEntryFlags to set for the page table entry (e.g. WRITABLE or EXECUTE_DISABLE). + // The PRESENT and ACCESSED are already set automatically. + fn map_pages( + &mut self, + range: PageIter, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ) { + let mut current_physical_address = physical_address; + + for page in range { + self.map_page::(page, current_physical_address, flags); + current_physical_address += S::SIZE as u64; + } + } +} + +impl PageTableMethods for PageTable +where + L::SubtableLevel: PageTableLevel, +{ + // Maps a single page to the given physical address. + // + // This is the implementation for all tables with subtables (L0Table, L1Table, L2Table). + // It overrides the default implementation above. + fn map_page( + &mut self, + page: Page, + physical_address: PhysAddr, + flags: PageTableEntryFlags, + ) { + assert!(L::LEVEL <= S::MAP_LEVEL); + + if L::LEVEL < S::MAP_LEVEL { + let index = page.table_index::(); + + // Does the table exist yet? + if !self.entries[index].is_present() { + // Allocate a single 4 KiB page for the new entry and mark it as a valid, writable subtable. + let physical_address = pagealloc::get_zeroed_page(); + self.entries[index].set(physical_address, PageTableEntryFlags::TABLE_OR_4KIB_PAGE); + } + + let subtable = self.subtable::(page); + subtable.map_page::(page, physical_address, flags) + } else { + // Calling the default implementation from a specialized one is not supported (yet), + // so we have to resort to an extra function. + self.map_page_in_this_table::(page, physical_address, flags) + } + } +} + +// A generic interface to support all possible page sizes. +// +// This is defined as a subtrait of Copy to enable #[derive(Clone, Copy)] for Page. +// Currently, deriving implementations for these traits only works if all dependent types implement it as well. +pub trait PageSize: Copy { + // The page size in bytes. + const SIZE: usize; + + // The page table level at which a page of this size is mapped + const MAP_LEVEL: usize; + + // Any extra flag that needs to be set to map a page of this size. + // For example: PageTableEntryFlags::TABLE_OR_4KIB_PAGE. + const MAP_EXTRA_FLAG: PageTableEntryFlags; +} + +// A 4 KiB page mapped in the L3Table. +#[derive(Clone, Copy)] +pub enum BasePageSize {} +impl PageSize for BasePageSize { + const SIZE: usize = 4096; + const MAP_LEVEL: usize = 3; + const MAP_EXTRA_FLAG: PageTableEntryFlags = PageTableEntryFlags::TABLE_OR_4KIB_PAGE; +} + +// A 2 MiB page mapped in the L2Table. +#[derive(Clone, Copy)] +pub enum LargePageSize {} +impl PageSize for LargePageSize { + const SIZE: usize = 2 * 1024 * 1024; + const MAP_LEVEL: usize = 2; + const MAP_EXTRA_FLAG: PageTableEntryFlags = PageTableEntryFlags::BLANK; +} + +// A 1 GiB page mapped in the L1Table. +#[derive(Clone, Copy)] +pub enum HugePageSize {} +impl PageSize for HugePageSize { + const SIZE: usize = 1024 * 1024 * 1024; + const MAP_LEVEL: usize = 1; + const MAP_EXTRA_FLAG: PageTableEntryFlags = PageTableEntryFlags::BLANK; +} From patchwork Mon Mar 14 08:26:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFD6DC433F5 for ; Mon, 14 Mar 2022 08:27:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237084AbiCNI2X (ORCPT ); Mon, 14 Mar 2022 04:28:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237028AbiCNI2W (ORCPT ); Mon, 14 Mar 2022 04:28:22 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA2AE1C90D for ; Mon, 14 Mar 2022 01:27:12 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 7CF84B80B01 for ; Mon, 14 Mar 2022 08:27:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 88578C340F6; Mon, 14 Mar 2022 08:27:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246430; bh=z/YGAz9LQB8pb8wT8N++FTB8PEZI7HV+rq1+GaYfjpU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fNzdZjkDs+iRFcUzGzvBd/xrLOEyuqcphlEFIf8vsyckpcgWaxbEkgxDOX83ST/vc ruvWX7GX9uiAOxi1xKAGgVC3y2L8Tg66VAVURVUSUiTa8b8Sws/JWwXNUH0Ngbz4YC HQ8R9NJgt1QSzaJvKI2czyaZucn57gboovH03Zah9JrAWj03I9UgpCxJuSJstUvPA1 Gi+rBJ3B7r/q9f9qPBEvSIDKAk5uQAJ8Kv+B6gcqaL6lwrkxM/Tk1950prPMvaLELF 3LDOymK8RirWsaKmd0Tn+3Z3EuZxeVbaXGbZGTUmmontSOS656NqbHciWybR1bYOEl CqWvaLBWjyO8g== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 4/6] Discover QEMU fwcfg device and use it to load the kernel Date: Mon, 14 Mar 2022 09:26:42 +0100 Message-Id: <20220314082644.3436071-5-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel QEMU exposes a paravirtualized interface to load various items exposed by the host into the guest. Implement a minimal driver for it, and use it to load the kernel image into DRAM. --- src/fwcfg.rs | 85 ++++++++++++++++++++ src/main.rs | 18 +++++ 2 files changed, 103 insertions(+) diff --git a/src/fwcfg.rs b/src/fwcfg.rs new file mode 100644 index 000000000000..57f405df174b --- /dev/null +++ b/src/fwcfg.rs @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use mmio::{Allow, Deny, VolBox}; + +pub struct FwCfg { + // read-only data register + data: VolBox, + + // write-only selector register + selector: VolBox, + + // write-only DMA register + dmacontrol: VolBox, +} + +const CFG_KERNEL_SIZE: u16 = 0x08; +const CFG_KERNEL_DATA: u16 = 0x11; + +const CFG_DMACTL_DONE: u32 = 0; +const CFG_DMACTL_ERROR: u32 = 1; +const CFG_DMACTL_READ: u32 = 2; + +#[repr(C)] +struct DmaTransfer { + control: u32, + length: u32, + address: u64, +} + +impl FwCfg { + pub fn from_fdt_node(node: fdt::node::FdtNode) -> Option { + if let Some(mut iter) = node.reg() { + iter.next().map(|reg| { + let addr = reg.starting_address; + unsafe { + FwCfg { + data: VolBox::::new(addr as *mut u64), + selector: VolBox::::new(addr.offset(8) as *mut u16), + dmacontrol: VolBox::::new(addr.offset(16) as *mut u64), + } + } + }) + } else { + None + } + } + + unsafe fn dma_transfer( + &mut self, + load_address: *mut u8, + size: usize, + config_item: u16, + ) -> Result<(), &str> { + let xfer = DmaTransfer { + control: u32::to_be(CFG_DMACTL_READ), + length: u32::to_be(size as u32), + address: u64::to_be(load_address as u64), + }; + self.selector.write(u16::to_be(config_item)); + self.dmacontrol.write(u64::to_be(&xfer as *const _ as u64)); + + let control = VolBox::::new(&xfer.control as *const _ as *mut u32); + loop { + match control.read() { + CFG_DMACTL_DONE => return Ok(()), + CFG_DMACTL_ERROR => return Err("fwcfg DMA error"), + _ => (), // keep polling + } + } + } + + pub fn get_kernel_size(&mut self) -> usize { + self.selector.write(u16::to_be(CFG_KERNEL_SIZE)); + self.data.read() as usize + } + + pub fn load_kernel_image(&mut self, load_address: *mut u8) -> Result<(), &str> { + let size = self.get_kernel_size(); + if size > 0 { + unsafe { self.dma_transfer(load_address, size, CFG_KERNEL_DATA) } + } else { + Err("No kernel image provided by fwcfg") + } + } +} diff --git a/src/main.rs b/src/main.rs index af58ccc0318d..048d1b4842cb 100644 --- a/src/main.rs +++ b/src/main.rs @@ -8,6 +8,7 @@ mod console; mod cstring; +mod fwcfg; mod pagealloc; mod paging; @@ -28,6 +29,8 @@ extern "C" { static _dtb_end: u8; } +const LOAD_ADDRESS: *mut u8 = 0x43210000 as _; + #[no_mangle] extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { #[cfg(debug_assertions)] @@ -79,6 +82,21 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { // Switch to the new ID map so we can use all of DRAM paging::activate(); + let compat = ["qemu,fw-cfg-mmio"]; + let fwcfg_node = fdt + .find_compatible(&compat) + .expect("QEMU fwcfg node not found"); + + info!("QEMU fwcfg node found: {}\n", fwcfg_node.name); + + let mut fwcfg = fwcfg::FwCfg::from_fdt_node(fwcfg_node).expect("Failed to open fwcfg device"); + + // TODO allocate fwcfg.get_kernel_size() bytes here instead of using a fixed address + + fwcfg + .load_kernel_image(LOAD_ADDRESS) + .expect("Failed to load kernel image"); + // Switch back to the initial ID map so we can remap // the loaded kernel image with different permissions paging::deactivate(); From patchwork Mon Mar 14 08:26:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46F21C433FE for ; Mon, 14 Mar 2022 08:27:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237028AbiCNI2X (ORCPT ); Mon, 14 Mar 2022 04:28:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237016AbiCNI2W (ORCPT ); Mon, 14 Mar 2022 04:28:22 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73BFE1CFDF for ; Mon, 14 Mar 2022 01:27:13 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 10A6861244 for ; Mon, 14 Mar 2022 08:27:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B64B8C340E9; Mon, 14 Mar 2022 08:27:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246432; bh=06tI/T1GvhECF+IatJT8s2EXaWx5xagNfQnNrKsZMMY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ELoP4rvEy7ZpK5RCFCWN1lim32r836rjlT8rAYg4g9TJH3n0m1EMjGwglMKTf/F0G aeiYg5AG3NDaL3A7Cq5aOL35wwRe3wOW1SOt+sd39dJZlzaGW1bQKqjUm27aBV6ZFW SVG0S0V54AaP/kOmJqYMT2zlqZZYzQiXrbr0nddd5exHerKfjvn29LshIqohEfJBDf JzR5A2fInJFBVDfuJuVfFLvZ1+bi1esL0OxEAAOTEP+t5zxN3UmLlw7o2wliW37pG0 1OjM92y9IurC19jGDE24PFcNGcAhdnTq9D/B+cnzQZnKcrgEOyh3YFWdVMV1xCQzC4 uR0qO1gzDAaxQ== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 5/6] Remap code section of loaded kernel and boot it Date: Mon, 14 Mar 2022 09:26:43 +0100 Message-Id: <20220314082644.3436071-6-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel Implement the bare minimum needed to discover the size of the text/rodata region of the loaded image, and use it to remap this region read-only so that we can execute it while WXN protections are enabled. Then, boot the loaded image by jumping to the start of it. --- src/cmo.rs | 37 ++++++++++++++++++++ src/main.rs | 22 ++++++++++++ src/pecoff.rs | 23 ++++++++++++ 3 files changed, 82 insertions(+) diff --git a/src/cmo.rs b/src/cmo.rs new file mode 100644 index 000000000000..49456222c705 --- /dev/null +++ b/src/cmo.rs @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +use core::arch::asm; + +const CTR_IDC: u64 = 1 << 28; + +const CTR_DMINLINE_SHIFT: u64 = 16; +const CTR_DMINLINE_MASK: u64 = 0xf; + +pub fn dcache_clean_to_pou(base: *const u8, size: isize) { + let ctr = unsafe { + let mut l: u64; + asm!("mrs {reg}, ctr_el0", // CTR: cache type register + reg = out(reg) l, + options(pure, nomem, nostack, preserves_flags), + ); + l + }; + + // Perform the clean only if needed for coherency with the I side + if (ctr & CTR_IDC) == 0 { + let line_shift = 2 + ((ctr >> CTR_DMINLINE_SHIFT) & CTR_DMINLINE_MASK); + let line_size: isize = 1 << line_shift; + let num_lines = (size + line_size - 1) >> line_shift; + let mut offset: isize = 0; + + for _ in 1..=num_lines { + unsafe { + asm!("dc cvau, {reg}", + reg = in(reg) base.offset(offset), + options(nomem, nostack, preserves_flags), + ); + } + offset += line_size; + } + } +} diff --git a/src/main.rs b/src/main.rs index 048d1b4842cb..81208c18d094 100644 --- a/src/main.rs +++ b/src/main.rs @@ -6,11 +6,13 @@ #![allow(incomplete_features)] #![feature(specialization)] +mod cmo; mod console; mod cstring; mod fwcfg; mod pagealloc; mod paging; +mod pecoff; use core::{arch::global_asm, panic::PanicInfo}; use linked_list_allocator::LockedHeap; @@ -29,6 +31,8 @@ extern "C" { static _dtb_end: u8; } +type EntryFn = unsafe extern "C" fn(*const u8, u64, u64, u64) -> !; + const LOAD_ADDRESS: *mut u8 = 0x43210000 as _; #[no_mangle] @@ -97,9 +101,27 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { .load_kernel_image(LOAD_ADDRESS) .expect("Failed to load kernel image"); + let pe_image = pecoff::Parser::from_ptr(LOAD_ADDRESS); + + // Clean the code region of the loaded image to the PoU so we + // can safely fetch instructions from it once the PXN/UXN + // attributes are cleared + let code_size = pe_image.get_code_size(); + cmo::dcache_clean_to_pou(LOAD_ADDRESS, code_size as isize); + // Switch back to the initial ID map so we can remap // the loaded kernel image with different permissions paging::deactivate(); + + // Remap the text/rodata part of the image read-only so we will + // be able to execute it with WXN protections enabled + paging::map_range(LOAD_ADDRESS as u64, code_size, nor_flags); + paging::activate(); + + unsafe { + let entrypoint: EntryFn = core::mem::transmute(LOAD_ADDRESS); + entrypoint(&_dtb as *const _, 0, 0, 0); + } } #[no_mangle] diff --git a/src/pecoff.rs b/src/pecoff.rs new file mode 100644 index 000000000000..b9b82fc5cc53 --- /dev/null +++ b/src/pecoff.rs @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +pub struct Parser { + base_of_code: u64, + size_of_code: u64, +} + +impl Parser { + pub fn from_ptr(ptr: *const u8) -> Parser { + // TODO check magic number, arch, etc + // TODO deal with variable PE header offset + let pehdr_offset = 64; + + Parser { + base_of_code: unsafe { *(ptr.offset(pehdr_offset + 28) as *const u32) } as u64, + size_of_code: unsafe { *(ptr.offset(pehdr_offset + 44) as *const u32) } as u64, + } + } + + pub fn get_code_size(&self) -> u64 { + return self.base_of_code + self.size_of_code; + } +} From patchwork Mon Mar 14 08:26:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 551491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97553C433EF for ; Mon, 14 Mar 2022 08:27:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237116AbiCNI22 (ORCPT ); Mon, 14 Mar 2022 04:28:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237016AbiCNI20 (ORCPT ); Mon, 14 Mar 2022 04:28:26 -0400 Received: from sin.source.kernel.org (sin.source.kernel.org [IPv6:2604:1380:40e1:4800::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF0931C90D for ; Mon, 14 Mar 2022 01:27:17 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 3C8E9CE10C0 for ; Mon, 14 Mar 2022 08:27:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3F30C340F4; Mon, 14 Mar 2022 08:27:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1647246434; bh=0eiMuQnYJxpXHSlilUSyfpYbrccsw17ywT8caWbUXDg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tOmeaEfApmiNG90U4mupIzm/QCVpL53n1TmrFHyHLIMK/ugWHJ+aEGLjUPpsPqX4v RJdy5fpLcoJTrNTMKaVTn+gd4wAL0057h25yErKXo+2SEnIMs3z73l5bWwgq1ngE5S zafNp8V//Aw/2X+gkWM+TWnYu+Y7FGviTWti3Ss6YLSJfeGlDLmIwnM8uVCY6QUxhP tOKUWZ6cghXKzgGGlbgMe1i7JgtkqCO4Lfd5/kguDG6Zel7PJ3hXU/vtpaVq66Whtg gEPMk99mBgVvF5NY4AoBRc889uTe4KJdzBuqLxIGlWH54WHQ0eY1aXGl/QEREKjmBp E6kIUwAbXRvHQ== From: ardb@kernel.org To: linux-efi@vger.kernel.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Quentin Perret , David Brazdil , Fuad Tabba , Kees Cook Subject: [RFC PATCH v0 6/6] Temporarily pass the kaslr seed via register X1 Date: Mon, 14 Mar 2022 09:26:44 +0100 Message-Id: <20220314082644.3436071-7-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220314082644.3436071-1-ardb@kernel.org> References: <20220314082644.3436071-1-ardb@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org From: Ard Biesheuvel Currently, we boot the kernel via its 'bare metal' entry point, rather than via the EFI entry point, as we haven't implemented EFI yet. Booting with the MMU enabled requires that the KASLR seed is known before setting up the page tables, as we will do so only once, rather than twice when reading the seed from the DT. For this reason, the EFI stub passes the KASLR seed via register X1 as well as the kaslr-seed property in chosen, and those values need to be in sync. So as long as we are not using the EFI entry point, pass the DT's kaslr-seed value via register X1 at boot. --- src/main.rs | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/src/main.rs b/src/main.rs index 81208c18d094..ad12e069372f 100644 --- a/src/main.rs +++ b/src/main.rs @@ -118,9 +118,21 @@ extern "C" fn efilite_main(base: usize, mapped: usize, used: usize) { paging::map_range(LOAD_ADDRESS as u64, code_size, nor_flags); paging::activate(); + // TODO remove this once we boot via the EFI entry point + // passing the kaslr seed via x1 is part of the stub's internal boot protocol + let kaslr_seed: u64 = { + let mut seed: u64 = 0; + let chosen = fdt.find_node("/chosen").unwrap(); + if let Some(prop) = chosen.property("kaslr-seed") { + seed = prop.as_usize().unwrap() as _; + info!("/chosen/kaslr-seed: {:#x}\n", seed); + }; + seed + }; + unsafe { let entrypoint: EntryFn = core::mem::transmute(LOAD_ADDRESS); - entrypoint(&_dtb as *const _, 0, 0, 0); + entrypoint(&_dtb as *const _, kaslr_seed, 0, 0); } }