From patchwork Tue Oct 1 15:58:00 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sandeepa Prabhu X-Patchwork-Id: 20724 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id C0DC525E05 for ; Tue, 1 Oct 2013 15:59:53 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id j7sf6145892qaq.11 for ; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:x-gm-message-state:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=e1hvJAiC0T6sjOwm0tyKC1ODxS8KYVpfaqXmdl2jCsY=; b=YDcyyYKgnkvLkX6II9SHVBSneJCsAoyNUPE4hvT0WX+KgKMnhX+sIpoe+TLYj+ea7h zgbF/VsgjJiw+MvkEK3SB9bphs4pV/q9YTN/Utll+QjuLC4LuA9jDylJw/DaZiQV+Qh0 O5LnPs1F3HdqVoA0kGopW4niEFtqInab8T4aFlYUHUCH0MIPvOzKYNXMLv0B7dfS4R5v iZ3ihccbVyfbaa6n0hL7Fv3n5OI4u7jj19AukrNHZ4Xu4KqCO76ZnCIgWfPVhHac8C5R bDuqtv07HcDUTTPDREnqSn7+IgvCU2cBylv1+yYQZ3ZDUDFAtEWzf06MHpW5tI8qnHgR MwnQ== X-Received: by 10.236.53.70 with SMTP id f46mr9058564yhc.17.1380643193601; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.18.199 with SMTP id y7ls123449qed.78.gmail; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) X-Received: by 10.58.19.233 with SMTP id i9mr349841vee.36.1380643193360; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) Received: from mail-vb0-f49.google.com (mail-vb0-f49.google.com [209.85.212.49]) by mx.google.com with ESMTPS id z6si1459916vcu.145.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 08:59:53 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.212.49 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.49; Received: by mail-vb0-f49.google.com with SMTP id w16so4852112vbb.8 for ; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) X-Gm-Message-State: ALoCoQkUOCzufu9WZyKwXMkUrQYlMChfWNVKQFWaLD4Oehy8fEqERZHKKZrlw3s9WC/Xzo9QKOP5 X-Received: by 10.220.94.206 with SMTP id a14mr28139180vcn.19.1380643193260; Tue, 01 Oct 2013 08:59:53 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp28288vcz; Tue, 1 Oct 2013 08:59:52 -0700 (PDT) X-Received: by 10.68.4.197 with SMTP id m5mr30301534pbm.46.1380643192335; Tue, 01 Oct 2013 08:59:52 -0700 (PDT) Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) by mx.google.com with ESMTPS id iv2si5298796pac.293.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 08:59:52 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.192.175 is neither permitted nor denied by best guess record for domain of sandeepa.prabhu@linaro.org) client-ip=209.85.192.175; Received: by mail-pd0-f175.google.com with SMTP id q10so7453740pdj.34 for ; Tue, 01 Oct 2013 08:59:51 -0700 (PDT) X-Received: by 10.68.252.33 with SMTP id zp1mr30448436pbc.95.1380643191888; Tue, 01 Oct 2013 08:59:51 -0700 (PDT) Received: from linaro-workstation.ban.broadcom.com ([202.122.18.226]) by mx.google.com with ESMTPSA id nj9sm7593660pbc.13.1969.12.31.16.00.00 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Oct 2013 08:59:51 -0700 (PDT) From: Sandeepa Prabhu To: linux-arm-kernel@lists.infradead.org Cc: deepak.saxena@linaro.org, catalin.marinas@arm.com, will.deacon@arm.com, tixy@linaro.org, linaro-kernel@lists.linaro.org, patches@linaro.org, Sandeepa Prabhu Subject: [PATCH RFC v1 5/5] AArch64: Support kretprobe support for ARM v8 Date: Tue, 1 Oct 2013 21:28:00 +0530 Message-Id: <1380643080-8984-6-git-send-email-sandeepa.prabhu@linaro.org> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1380643080-8984-1-git-send-email-sandeepa.prabhu@linaro.org> References: <1380643080-8984-1-git-send-email-sandeepa.prabhu@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: sandeepa.prabhu@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.49 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Unlike ARM v7(ldmia {.., pc} ) ARM v8 ISA does not support popping the PC value from stack or absolute addr without using one of the general purpose registers. This means return probes cannot return to the original return address directly without modifying register context, without trapping into debug exception. So like many other architectures, we prepare a global routine with NOPs, which serve as trampoline that hacks away the function return address, by placing an extra kprobe on the trampoline entry. The pre-handler of this special trampoline' kprobe execute return probe handlers and restore original return address in ELR_EL1, this way, saved pt_regs still hold the original register values to be carried back to the caller. Signed-off-by: Sandeepa Prabhu --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/kprobes.h | 1 + arch/arm64/kernel/kprobes.c | 115 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 116 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 8cf5cde..9ca71b0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -25,6 +25,7 @@ config ARM64 select HAVE_GENERIC_DMA_COHERENT select HAVE_HW_BREAKPOINT if PERF_EVENTS select HAVE_KPROBES if !XIP_KERNEL + select HAVE_KRETPROBES if (HAVE_KPROBES) select HAVE_MEMBLOCK select HAVE_PERF_EVENTS select IRQ_DOMAIN diff --git a/arch/arm64/include/asm/kprobes.h b/arch/arm64/include/asm/kprobes.h index a43f74f..c2a1ff1 100644 --- a/arch/arm64/include/asm/kprobes.h +++ b/arch/arm64/include/asm/kprobes.h @@ -53,5 +53,6 @@ void arch_remove_kprobe(struct kprobe *); int kprobe_fault_handler(struct pt_regs *regs, unsigned int fsr); int kprobe_exceptions_notify(struct notifier_block *self, unsigned long val, void *data); +void kretprobe_trampoline(void); #endif /* _ARM_KPROBES_H */ diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index 4840433..8c0f32d 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -508,6 +508,118 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) return 0; } +/* + * Kretprobes: kernel return probes handling + * + * ARM v8 ISA does not support popping the PC value from the + * stack like on v7(ldmia {..,pc}), so atleast one register need + * to be used for achieving branch. It means return probes cannot + * return back to the original return address directly without + * modifying the register context. + * + * So like many other architectures, we prepare a global routine + * with NOPs, which serve as trampoline address that hack away the + * function return, with the exact register context. + * + * We place a kprobe on trampoline routine entry to trap again and + * execute return probe handlers and restore original return address + * in ELR_EL1, this way saved pt_regs still hold the original + * register values to be carried back to the caller. + */ +static void __used kretprobe_trampoline_holder(void) +{ + asm volatile (".global kretprobe_trampoline\n" + "kretprobe_trampoline:\n" + "NOP\n\t" + "NOP\n\t"); +} + +static int __kprobes +trampoline_probe_handler(struct kprobe *p, struct pt_regs *regs) +{ + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __get_cpu_var(current_kprobe) = &ri->rp->kp; + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __get_cpu_var(current_kprobe) = NULL; + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + preempt_enable_no_resched(); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + /* return 1 so that post handlers not called */ + return 1; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *) + (instruction_pointer(regs) + sizeof(kprobe_opcode_t)); + + /* Replace the return addr with trampoline addr */ + instruction_pointer(regs) = (unsigned long)&kretprobe_trampoline; +} + +static struct kprobe trampoline = { + .addr = (kprobe_opcode_t *) &kretprobe_trampoline, + .pre_handler = trampoline_probe_handler +}; + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return p->addr == (kprobe_opcode_t *) &kretprobe_trampoline; +} + /* Break Handler hook */ static struct break_hook kprobes_break_hook = { .esr_mask = BRK64_ESR_MASK, @@ -525,5 +637,6 @@ int __init arch_init_kprobes() register_break_hook(&kprobes_break_hook); register_step_hook(&kprobes_step_hook); - return 0; + /* register trampoline for kret probe */ + return register_kprobe(&trampoline); }