From patchwork Tue Nov 18 06:32:53 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 40995 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2A367246D4 for ; Tue, 18 Nov 2014 06:33:06 +0000 (UTC) Received: by mail-la0-f69.google.com with SMTP id gq15sf1647409lab.0 for ; Mon, 17 Nov 2014 22:33:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=m1jL2Bd+sKfRgDtgBNwzNo2sWLt+USaBl5hrhq9m00M=; b=VOI8dl7THa3x6PvQqUZlcfK6DYL5iUTPnobzzY1oRT18s6KyQ1QTB4uCYICq6SiiFG LckjGM1CrUxu88pcyWRcGb6mZZnXV/fusd8DPyFQmEf/sqSON8nAdtZKZXL2iEGGMpKH ePvi1ShnofCy0CHdqfe23KtVijb8cCvzVs4lb58YYQUQowAmicrohR7U2mrBVjp3emtJ jebKsck14OHe0Hb6cXo39NT7gwJ5QW5YGkh9ItQ3xnYrboih7bxQdpk19gGjO8fauxqt 7ACOKOdgxkrcboN4lFgiMRlJOQN9tEoqwlJX8mhmxYjwbsvw6cu/7CPCm3hmn8ys2UJW NhvA== X-Gm-Message-State: ALoCoQlQMRa0RK554bh2AgFStbqUrIbeKRb6SIDc7h3HveHzjO1lWPKaXcij6Wf9cEpqIhCMgcDZ X-Received: by 10.112.151.38 with SMTP id un6mr10936273lbb.0.1416292385045; Mon, 17 Nov 2014 22:33:05 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.36.201 with SMTP id s9ls946978laj.100.gmail; Mon, 17 Nov 2014 22:33:04 -0800 (PST) X-Received: by 10.112.62.166 with SMTP id z6mr16064793lbr.74.1416292384740; Mon, 17 Nov 2014 22:33:04 -0800 (PST) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com. [209.85.215.50]) by mx.google.com with ESMTPS id xk6si54530148lbb.22.2014.11.17.22.33.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Nov 2014 22:33:04 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) client-ip=209.85.215.50; Received: by mail-la0-f50.google.com with SMTP id pv20so811418lab.37 for ; Mon, 17 Nov 2014 22:33:04 -0800 (PST) X-Received: by 10.112.45.102 with SMTP id l6mr4309032lbm.46.1416292384578; Mon, 17 Nov 2014 22:33:04 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.112.184.201 with SMTP id ew9csp1282130lbc; Mon, 17 Nov 2014 22:33:03 -0800 (PST) X-Received: by 10.224.167.132 with SMTP id q4mr40255952qay.48.1416292381967; Mon, 17 Nov 2014 22:33:01 -0800 (PST) Received: from mail-qg0-f52.google.com (mail-qg0-f52.google.com. [209.85.192.52]) by mx.google.com with ESMTPS id k8si40466788qad.19.2014.11.17.22.33.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Nov 2014 22:33:01 -0800 (PST) Received-SPF: pass (google.com: domain of dave.long@linaro.org designates 209.85.192.52 as permitted sender) client-ip=209.85.192.52; Received: by mail-qg0-f52.google.com with SMTP id a108so4531514qge.25 for ; Mon, 17 Nov 2014 22:33:01 -0800 (PST) X-Received: by 10.140.29.9 with SMTP id a9mr40643329qga.41.1416292381434; Mon, 17 Nov 2014 22:33:01 -0800 (PST) Received: from localhost.localdomain (pool-72-71-243-249.cncdnh.fast00.myfairpoint.net. [72.71.243.249]) by mx.google.com with ESMTPSA id 9sm36315651qau.37.2014.11.17.22.33.00 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Nov 2014 22:33:00 -0800 (PST) From: David Long To: linux-arm-kernel@lists.infradead.org, Russell King Cc: Sandeepa Prabhu , William Cohen , Catalin Marinas , Will Deacon , "Jon Medhurst (Tixy)" , Masami Hiramatsu , Ananth N Mavinakayanahalli , Anil S Keshavamurthy , , linux-kernel@vger.kernel.org Subject: [PATCH v3 3/5] arm64: Add kernel return probes support(kretprobes) Date: Tue, 18 Nov 2014 01:32:53 -0500 Message-Id: <1416292375-29560-4-git-send-email-dave.long@linaro.org> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1416292375-29560-1-git-send-email-dave.long@linaro.org> References: <1416292375-29560-1-git-send-email-dave.long@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: dave.long@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , From: Sandeepa Prabhu AArch64 ISA does not instructions to pop PC register value from stack(like ARM v7 has ldmia {...,pc}) without using one of the general purpose registers. This means return probes cannot return to the actual return address directly without modifying register context, and without trapping into debug exception. So like many other architectures, we prepare a global routine with NOPs, which serve as trampoline to hack away the function return address, by placing an extra kprobe on the trampoline entry. The pre-handler of this special trampoline' kprobe execute return probe handler functions and restore original return address in ELR_EL1, this way, saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu Signed-off-by: David A. Long --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kprobes.h | 1 + arch/arm64/kernel/kprobes.c | 114 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 115 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 256ef90..02f31b7 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -61,6 +61,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES if !XIP_KERNEL + select HAVE_KRETPROBES if HAVE_KPROBES select IRQ_DOMAIN select MODULES_USE_ELF_RELA select NO_BOOTMEM diff --git a/arch/arm64/include/asm/kprobes.h b/arch/arm64/include/asm/kprobes.h index b35d3b9..a2de3b8 100644 --- a/arch/arm64/include/asm/kprobes.h +++ b/arch/arm64/include/asm/kprobes.h @@ -56,5 +56,6 @@ void arch_remove_kprobe(struct kprobe *); int kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr); int kprobe_exceptions_notify(struct notifier_block *self, unsigned long val, void *data); +void kretprobe_trampoline(void); #endif /* _ARM_KPROBES_H */ diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index 789ab61..be7c330 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -569,6 +569,117 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) return 0; } +/* + * Kretprobes: kernel return probes handling + * + * AArch64 mode does not support popping the PC value from the + * stack like on ARM 32-bit (ldmia {..,pc}), so atleast one + * register need to be used to achieve branching/return. + * It means return probes cannot return back to the original + * return address directly without modifying the register context. + * + * So like other architectures, we prepare a global routine + * with NOPs, which serve as trampoline address that hack away the + * function return, with the exact register context. + * Placing a kprobe on trampoline routine entry will trap again to + * execute return probe handlers and restore original return address + * in ELR_EL1, this way saved pt_regs still hold the original + * register values to be carried back to the caller. + */ +static void __used kretprobe_trampoline_holder(void) +{ + asm volatile (".global kretprobe_trampoline\n" + "kretprobe_trampoline:\n" + "NOP\n\t" + "NOP\n\t"); +} + +static int __kprobes +trampoline_probe_handler(struct kprobe *p, struct pt_regs *regs) +{ + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + kprobes_restore_local_irqflag(regs); + + /* return 1 so that post handlers not called */ + return 1; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +static struct kprobe trampoline = { + .addr = (kprobe_opcode_t *) &kretprobe_trampoline, + .pre_handler = trampoline_probe_handler +}; + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return p->addr == (kprobe_opcode_t *) &kretprobe_trampoline; +} + /* Break Handler hook */ static struct break_hook kprobes_break_hook = { .esr_mask = BRK64_ESR_MASK, @@ -586,5 +697,6 @@ int __init arch_init_kprobes(void) register_break_hook(&kprobes_break_hook); register_step_hook(&kprobes_step_hook); - return 0; + /* register trampoline for kret probe */ + return register_kprobe(&trampoline); }