From patchwork Mon Feb 26 08:19:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129565 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356100lja; Mon, 26 Feb 2018 00:22:21 -0800 (PST) X-Google-Smtp-Source: AH8x226lDyaqoHCcVPvdGsnJ/UyrxMEoxZtw3Le6U7gH1UsGBEHNjkSxe0USYbroUai+WWvwkd4g X-Received: by 10.99.103.69 with SMTP id b66mr7760843pgc.233.1519633341118; Mon, 26 Feb 2018 00:22:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633341; cv=none; d=google.com; s=arc-20160816; b=P5XYF4zbBk32z7LqWvUKytC2vXg+LQgODjjLzlXp0jDn9hTzl9JIcLq4GF/Drvtf2L 7Pv5c3yilFfvMAijJ+Zaz7xp5X30kUhd5PP85BeS+Y05r5En0BsGZLXwBwYzENyxASEU YMfBEGu2+pv5YAA6PtfiBAGuh6AGROIqe+IztB/7dKjxpAgJmwv4tYqdMS4v8CHhgFAd V3X+18WNCqp9iUg5/U4uiXhDqwMe6WrL9dU5cTFfj5LSqr7AF6ulns9FpE906jCaA5y8 tHiIWU2kyEMEAU7O1bbHoEWQ+Hws72CleKXWeet4JJczM75Cf2Iak3GKclQhsnQQOX9R l3EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=kLvfmXnWmAcTLHO4SHsgjaJnxGwZiMZ6wnkD36Xy92o9GPt9+CfE818VRSpQXwpaBq 1kWtrGDWak2O7dQCvP9Y58YHgqUo0d0icIzvcG4venuFVN83xM5sYkx7iUen3Jdj7q/O Mxz1qdAn0dEAboWiExFOh9IzyNHSCXp4UD2+O+Vw8nZS2hyLn/8yfnSv7nv+ot+98+2o US6qKtHawI5mcPzSdI4eIW6CasuXtVKkp0PUNF3friDuyKH/s6ikklbBo+T8ViGphLTm Mh5eSkQbRPPxM1vLpvGdaAx7NYqCHGI5bFD2qtTwVM9EJt7OwJR9q6CdKboNOiaIwuij CSNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=khZZHeON; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si5260385pgc.75.2018.02.26.00.22.20; Mon, 26 Feb 2018 00:22:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=khZZHeON; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752400AbeBZIWR (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:17 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:38381 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751530AbeBZIWN (ORCPT ); Mon, 26 Feb 2018 03:22:13 -0500 Received: by mail-pf0-f193.google.com with SMTP id d26so6219694pfn.5 for ; Mon, 26 Feb 2018 00:22:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=khZZHeONaEEupHkBSAr5hTKGNyxnUDb3a2umThtM0LhaYWZOKFVGPinL6RU3KO3BRi 2F0QIoG0h5jOWaIn3WjeHBbgGDrKPbyJm4b1eu2Qf0Fd/+cAcY8Jthvfu8d2JrLaKTpT MJwuW/6sKG15JgkTAGO8lhbbXlI6eB4nMuWdM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=AHB/GNf+oLVhT35ZGWxXGHwP1ajqRw2Ptsf9VKEsdB/cNogdYtmwop1yCuLp5JkDmu 8R6+IOH1JRDPt0xdS9D9dAwW6wpZiC2ZRB2PDlmY0q0YcwS1FgixKQrcEKz7BazCINoI e7UBenqKwZ8Soav6G5P9TTkoRXReHRvYq2ZSa9bRgpk9qjerIf/JSLH/AUUQZM7XTp0S gmGpdJReQzVa2CiSspTw8Gxa7B5sXjbvFc5WrMWpk+CEiHNqNjMjtxgQ9xO8SwXDv0vG yRc9GPolqFfChUV7lZNkfDRVhHUhPUw30m/ixe1Eiec1atRij9ygXKNVRUnsUoe8OpZX ekkg== X-Gm-Message-State: APf1xPAusCeiHYmzxu2lq71ZVhHz4tK5+xRPSDZtIUzVlPY0ch/uPDeV ps82wu1Jc91H+CciyuFlw9Cd7Q== X-Received: by 10.98.98.194 with SMTP id w185mr9862265pfb.9.1519633332837; Mon, 26 Feb 2018 00:22:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Robin Murphy Subject: [PATCH 09/52] arm64: Make USER_DS an inclusive limit Date: Mon, 26 Feb 2018 16:19:43 +0800 Message-Id: <1519633227-29832-10-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 51369e398d0d upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas (cherry picked from commit 83b20dff71ea949431cf57c6aebaaf7ebd5c1991) Signed-off-by: Alex Shi Conflicts: force replace __range_ok and add asm/processor.h in arch/arm64/include/asm/uaccess.h using old macro TI_ADDR_LIMIT instead of TSK_TI_ADDR_LIMIT in arch/arm64/kernel/entry.S manual change USER_DS to TASK_SIZE in arch/arm64/mm/fault.c --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/uaccess.h | 46 ++++++++++++++++++++++---------------- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/fault.c | 2 +- 4 files changed, 32 insertions(+), 21 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 4258f4d..5917147 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 09c9b59..7b1eb49 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -36,6 +36,7 @@ #include #include #include +#include #define VERIFY_READ 0 #define VERIFY_WRITE 1 @@ -62,10 +63,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -90,22 +88,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -114,7 +122,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs #define _ASM_EXTABLE(from, to) \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 478f0fe..6915697 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -103,7 +103,7 @@ /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] /* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */ .endif /* \el == 0 */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 403fe9e..4df70c9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -332,7 +332,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (is_permission_fault(esr) && (addr < USER_DS)) { + if (is_permission_fault(esr) && (addr < TASK_SIZE)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die("Accessing user space memory with fs=KERNEL_DS", regs, esr);