From patchwork Thu Dec 21 08:20:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Leo Yan X-Patchwork-Id: 122515 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp580422qgn; Thu, 21 Dec 2017 00:21:01 -0800 (PST) X-Google-Smtp-Source: ACJfBou4cPiJJJ1+7eLW9nRni0ngkbKxngLBuQIV6Jv+wroymuauM8THleixMZlkOqkPG5Fgky+0 X-Received: by 10.99.159.2 with SMTP id g2mr8910540pge.240.1513844461426; Thu, 21 Dec 2017 00:21:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1513844461; cv=none; d=google.com; s=arc-20160816; b=IwDvIhHKdn1UuJrrBQoADtBIFCGjVov95T79xdaOOPn6Yj8jfJfq/FsCpabMCnPXtj mP3ZXNW0Ru2Zrk6pX8XPkThmd5ZYkCOeUZGv3wCZ2dxLGoDEbfZNIVdsiF1NHUNcGkM3 qpki02GqXoueCjSdnXD2AmkycCg4wS5snrMVPdrDkC+yEwYi2Wef1PzEzgj/mdVThHhA vcjHfc+MkfdUw4rYDvaHAEyUd8oPuRzuQsz9NYohhyAnFaAGzBg2edDoHmjp7DCHxfbL 2DcqtvpQuUKu49VdfqAsptH1WeNA+fM/rfcSEfj+L/6XHGShcfSE7dc5L0qbLVyd0s8D 2dBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=24EyWKlM2VrmVqohn9HBsDzFsotx0DFZspF9D9RbzmM=; b=ayopL3GK2txuA63fN6b6Wm53fp3m9ukpPe1Oz5ItHO2vk2wnOhHfreLPGOC0iqxbdV 3G/ZULap3ZhFGoDZUjVb2D6vU9roE5DnVZIvrRc9fLjbMZFH8yT86yqfSDFaVD6RygTX S6+89bWhYSyhTP032dskJ+pbnPgZEed+DhhosXIw88wJBQ2x6tYZ3tb9MJoei7Co+gYi avZnuVbeaE5h7gh2TDfOIVtbi5hXU6W4Y3rP0lRpvbx+Qn/cvh16WzscOUZla6zvagze m29j2dw7ZLQLfSFY/KLaRw7RHf6b4U02PedilkIoLvMovMG2+gs/9e3dbS+sIMaSy3sA YHFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TOws2kjk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b3si14397869plb.41.2017.12.21.00.21.01; Thu, 21 Dec 2017 00:21:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TOws2kjk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751777AbdLUIU5 (ORCPT + 28 others); Thu, 21 Dec 2017 03:20:57 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:41716 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750782AbdLUIUs (ORCPT ); Thu, 21 Dec 2017 03:20:48 -0500 Received: by mail-pl0-f68.google.com with SMTP id g2so10587476pli.8 for ; Thu, 21 Dec 2017 00:20:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=24EyWKlM2VrmVqohn9HBsDzFsotx0DFZspF9D9RbzmM=; b=TOws2kjkk+942kH42zRnlN+mIBwAHZz/8gnff8GR0vRHGebx0K8c6EXcTuzYdJct9x 1l/T1WjXSkI03SI7yK+VdVr78GqxAeiMafspWDTl4v0SA+P9arUujQPJ1hMSDDYP/0kh 7XqqUq1P8yLbeQGrRhPqBR1u/zTVz9DTZVcV8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=24EyWKlM2VrmVqohn9HBsDzFsotx0DFZspF9D9RbzmM=; b=bZ3NcPVAx4CbBmVH4Il4P1nX0sjx1c55S1Id7fNxSzspxLeRbW6yTE/nnG7UQ52uOD ZR/h1eP37MUBjcwf9/F50uweht/Yhd/5cOkQTNFkzydJyVRlvnCTVgr5oTic8GMonieD WGIPM2/T5dAUr3dlG0WtZ4iAsUjlEu+Qr5AAFFZo0FKOXqqLAsU5/33GY20Rnr3PVkoO xvY/hGi66s2da3AXSat4b3L96LxoZ35+17P0MB6y1sloSyuRDDog9pbptNmbkYelmUNT AxLgcHTb8TlH8wV2YxuABPI2RZPZYGZLfLLNttp717PZGoVJE6jyd0/2rnbJ4U+LjcCA MOjA== X-Gm-Message-State: AKGB3mJigeZwFnL37VSh8IkfgditHA3dvWg4qPNo9qcxKOryEFd2OVqD msApkCzl/BprX50FnLCrD6g7Kg== X-Received: by 10.84.172.195 with SMTP id n61mr9798441plb.49.1513844448043; Thu, 21 Dec 2017 00:20:48 -0800 (PST) Received: from localhost.localdomain (li159-223.members.linode.com. [173.230.149.223]) by smtp.gmail.com with ESMTPSA id r86sm43699720pfk.114.2017.12.21.00.20.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 21 Dec 2017 00:20:46 -0800 (PST) From: Leo Yan To: Jonathan Corbet , Mathieu Poirier , Greg Kroah-Hartman , Will Deacon , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, coresight@lists.linaro.org Cc: Leo Yan Subject: [PATCH v3 2/6] doc: Add documentation for Coresight panic kdump Date: Thu, 21 Dec 2017 16:20:11 +0800 Message-Id: <1513844415-11427-3-git-send-email-leo.yan@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1513844415-11427-1-git-send-email-leo.yan@linaro.org> References: <1513844415-11427-1-git-send-email-leo.yan@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add detailed documentation for Coresight panic kdump, which contains the idea for why need this and introduce the framework implementation and usage. Signed-off-by: Leo Yan --- .../trace/coresight/coresight-panic-kdump.txt | 91 ++++++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 92 insertions(+) create mode 100644 Documentation/trace/coresight/coresight-panic-kdump.txt -- 2.7.4 diff --git a/Documentation/trace/coresight/coresight-panic-kdump.txt b/Documentation/trace/coresight/coresight-panic-kdump.txt new file mode 100644 index 0000000..6bf9cac --- /dev/null +++ b/Documentation/trace/coresight/coresight-panic-kdump.txt @@ -0,0 +1,91 @@ + Coresight Panic Kdump + ===================== + + Author: Leo Yan + Date: Dec 21th, 2017 + +Introduction +------------ + +Coresight has different sinks for trace data, the trace data is quite useful +for postmortem debugging. Embedded Trace Buffer (ETB) is one type sink which +provides on-chip storage of trace data, usually uses SRAM as buffer with +several KBs size; if the SoC designs to support 'Local ETF' (ARM DDI 0461B, +chapter 1.2.7), every CPU has one local ETB buffer so the per CPU trace data +can avoid to be overwritted by other CPUs. Trace Memory Controller (TMC) is +another kind sink designed as a successor to the CoreSight ETB to capture trace +into DRAM. + +After Linux kernel trigger panic, the trace data keeps the last execution flow +before issues happen. We could consider the trace data is quite useful for +postmortem debugging, especially when we can record trace data into DRAM and rely +on kdump to save them into vmcore file; at the end we can retrieve trace data +from vmcore file and "offline" to analyze the execution flow. + + +Implementation +-------------- + +Coresight panic kdump is a simple framework to support dump functionality, +it maintains dump list with every node recording the dump buffer base address +and buffer size. Coresight panic kdump provides the general APIs +{coresight_kdump_add|coresight_kdump_del} as helper functions so any coresight +device can add itself into dump list or delete as needed. + +Generally coresight device set its 'panic_cb' in the ops structure, the panic +notifier iterates dump list and invokes callback function to dump device specific +info. + +For easily used by offline analysis, we also record tracer metadata so can +retrieve tracer IDs and configuration, in this case the node records CPU number so +can create connection between the metadata and specific CPU. The tracer +driver uses helper function coresight_kdump_update() to update the dump +buffer base address and buffer size; so the tracer can save metadata at runtime +and these info can be prepared well pre panic happening. + + +Usage +----- + +Build Linux kernel with enabling 'CONFIG_CORESIGHT_PANIC_KDUMP' configuration. + +After system booting up, we need firstly prepare dump-capture kernel, this can +refer doc [1] chapter 'Load the Dump-capture Kernel' for detailed steps. Then +we need enable the coresight tracer, this can use either perf framework method +or sysFS interface, please refer doc [2] chapter 'How to use the tracer modules' +for detailed steps. + +When kernel panic happens, the panic kdump records trace data and launches +dump-capture kernel, we can utilize the dump-capture kernel to save kernel dump +file, this can refer doc [1] chapter 'Write Out the Dump File'. + +After get kernel dump file, we can use 'crash' tool + csdump.so extension to +extract trace data and generate 'perf.data' file: + + ./crash vmcore vmlinux + crash> extend csdump.so + crash> csdump output_dir + + We can see in the 'output_dir' there will generate out three files: + output_dir/ + ├── cstrace.bin -> trace raw data + ├── metadata.bin -> meta data + └── perf.data -> 'perf' format compatible file + +Finally use 'perf' tool for offline analysis: + + ./perf script -v -F cpu,event,ip,sym,symoff -i perf.data -k vmlinux --kallsyms /proc/kallsyms + [001] instructions: ffff000008559ad0 pl011_console_write+0x90 + [001] instructions: ffff000008559230 pl011_read+0x0 + [001] instructions: ffff00000855924c pl011_read+0x1c + [001] instructions: ffff000008559ae0 pl011_console_write+0xa0 + [001] instructions: ffff000008559ad0 pl011_console_write+0x90 + [001] instructions: ffff000008559230 pl011_read+0x0 + [001] instructions: ffff00000855924c pl011_read+0x1c + [001] instructions: ffff000008559ae0 pl011_console_write+0xa0 + [001] instructions: ffff000008559ad0 pl011_console_write+0x90 + [001] instructions: ffff000008559230 pl011_read+0x0 + [001] instructions: ffff00000855924c pl011_read+0x1c + +[1] Documentation/kdump/kdump.txt +[2] Documentation/trace/coresight/coresight.txt diff --git a/MAINTAINERS b/MAINTAINERS index d7a6fc7..26276e0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1309,6 +1309,7 @@ S: Maintained F: drivers/hwtracing/coresight/* F: Documentation/trace/coresight/coresight.txt F: Documentation/trace/coresight/coresight-cpu-debug.txt +F: Documentation/trace/coresight/coresight-panic-kdump.txt F: Documentation/devicetree/bindings/arm/coresight.txt F: Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt F: Documentation/ABI/testing/sysfs-bus-coresight-devices-*