From patchwork Tue May 14 14:28:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ranjan Kumar X-Patchwork-Id: 797644 Received: from mail-oo1-f44.google.com (mail-oo1-f44.google.com [209.85.161.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5555A16FF47 for ; Tue, 14 May 2024 14:32:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715697132; cv=none; b=E+f7fT/I7joGFDICDPFCKbFQfdjTOO4bwOt1be2cCJ3Omo1qXhsclQnWw2c8KYw8o4oRoC6cn2+8A8aytr1ngfqKCMG6I37yCa1SoXDQ1rCyU/cREeHS9n7mnwgVuNqQfX/prEKa5z0AxD/jX5R08Xp8aPHMGnUZeiA7FQ9mBu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715697132; c=relaxed/simple; bh=n9BT01ZSf/XoWW4BG6H30H7hCJIJHytb6ta7Z9NPR2E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=mOuhFfUulq5bsWNSQKpaQSIIQorD2Hv60Gu4pu19m/L+I6Dk6nbY2rIW29wmcp4xtptGRpXq5xgIhuJKKJZrXnHbNTCIgSFoKluKvaI9xPdO89L4gnqXRrD7Ewi1yAK1DHU6SeU9yx2+7XawFXTVtSoT5ao1uCiB69/FIHh4RPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=E/U8Nvbz; arc=none smtp.client-ip=209.85.161.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="E/U8Nvbz" Received: by mail-oo1-f44.google.com with SMTP id 006d021491bc7-5b278cdfad6so2248737eaf.3 for ; Tue, 14 May 2024 07:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1715697128; x=1716301928; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=fhaci3C+4PZV9nVC5+nEGBZH1HKRylihuvj6kFrAXBs=; b=E/U8NvbzBMZio6jjAOLXP92NOKJt8lof8agvguVJLk10qYsrr3I56t97jiqa5a6wNd rxNa5ebdBXZqb3D1vDTNGTbJxqG/w+oRXT8YBRm8blifp773wiTHSONrA2pkZ+MHF2bN EqR8MSETtwyhxr1C7SUIBTcnW5/AUS1tCnhk4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715697128; x=1716301928; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fhaci3C+4PZV9nVC5+nEGBZH1HKRylihuvj6kFrAXBs=; b=huu6EGYPhAVJSONKxjqr98W8Dg3gbgZe4MIFDeSrJdsHlVJu8jWIfZZqW9FYanU0Jo xsvV+WX8Ltpfpccu9WKHqwhGxz3u1vTA+B0RGCEi6Jbj8MKNQqcU+p/wX20zceLqv6Gt NbrTUsGZA+4oDO2O7DzGQOoqs1/KoyToEzsvEMioqglB711eLfkj6jdND2RzLilJO+6c z5F8qH2UYFgT3Z10XyVhGUSexGD/yOMaB5MwXh/7I+2RpSBbaTyw8mIUH/WFm9DG38nD FXMXy87B9Ab+mAFfOgnPjX8qOcP1geFTojacgAClVLvV3N/71/Umuli0gRQr1kL6xeRW 12YQ== X-Gm-Message-State: AOJu0YxmBwasxFkSK1hIhnyChaJ7Cgl3IO63t5gWxJy8+mZ6jqtrSLEX C3+1xecAo8KTVUZSqIhLBi+/3GArg6WDzXhWRBvaWgCC3HokO6MVLc2epp3VRnE+4B/aD6eVMfd SzGPUH5CRKc8w3sGqUBapBZnvzmV5XVGNwYmQd9xI2TqgTZC8FcmCi0a+ljF/NTNxG9y11oBVhG T8SvOio64zy/cis6nzkiHtzPkqWZ8MxrezIx00HFTvOlF3Hw== X-Google-Smtp-Source: AGHT+IGclM28gj1o4C2fkaG2Ywuzi4tmPFepRtAi72ShI5KknGXCu4XZx+QwB81vq3Zva7BpHU+15g== X-Received: by 2002:a05:6359:5a8c:b0:192:5bd6:74d7 with SMTP id e5c5f4694b2df-193bb518d17mr1384874555d.13.1715697128227; Tue, 14 May 2024 07:32:08 -0700 (PDT) Received: from localhost.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-63560731046sm7849104a12.67.2024.05.14.07.32.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 May 2024 07:32:07 -0700 (PDT) From: Ranjan Kumar To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: rajsekhar.chundru@broadcom.com, sathya.prakash@broadcom.com, sumit.saxena@broadcom.com, chandrakanth.patil@broadcom.com, prayas.patel@broadcom.com, Ranjan Kumar Subject: [PATCH v1 3/6] mpi3mr: Dump driver and dmesg logs into driver diag buffer Date: Tue, 14 May 2024 19:58:55 +0530 Message-Id: <20240514142858.51992-4-ranjan.kumar@broadcom.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240514142858.51992-1-ranjan.kumar@broadcom.com> References: <20240514142858.51992-1-ranjan.kumar@broadcom.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch adds support for collecting the kernel messages based on the driver buffer capture level set in the module parameter and copy the pertinent information to the driver diagnostic buffer posted to the controller. The buffer capture and copy will be executed when the driver detected the controller in the fault state. Signed-off-by: Sathya Prakash Signed-off-by: Ranjan Kumar --- drivers/scsi/mpi3mr/mpi/mpi30_tool.h | 2 + drivers/scsi/mpi3mr/mpi3mr.h | 5 ++ drivers/scsi/mpi3mr/mpi3mr_fw.c | 126 +++++++++++++++++++++++++++ 3 files changed, 133 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h index 495933856006..8b8b6ba00c7b 100644 --- a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h +++ b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h @@ -10,6 +10,8 @@ #define MPI3_DIAG_BUFFER_TYPE_DRIVER (0x10) #define MPI3_DIAG_BUFFER_ACTION_RELEASE (0x01) +#define MPI3_DRIVER_DIAG_BUFFER_HEADER_FLAGS_CIRCULAR_BUF_FORMAT_ASCII (0x00000000) + struct mpi3_diag_buffer_post_request { __le16 host_tag; u8 ioc_use_only02; diff --git a/drivers/scsi/mpi3mr/mpi3mr.h b/drivers/scsi/mpi3mr/mpi3mr.h index dc7e8f461826..b6030a665ec8 100644 --- a/drivers/scsi/mpi3mr/mpi3mr.h +++ b/drivers/scsi/mpi3mr/mpi3mr.h @@ -226,6 +226,10 @@ extern atomic64_t event_counter; #define MPI3MR_WRITE_SAME_MAX_LEN_256_BLKS 256 #define MPI3MR_WRITE_SAME_MAX_LEN_2048_BLKS 2048 +struct mpi3mr_kmsg_dumper { + struct kmsg_dump_iter kdumper; +}; + /* Driver diag buffer levels */ enum mpi3mr_drv_db_level { MRIOC_DRV_DB_DISABLED = 0, @@ -1331,6 +1335,7 @@ struct mpi3mr_ioc { void *drv_diag_buffer; dma_addr_t drv_diag_buffer_dma; u32 drv_diag_buffer_sz; + struct mpi3mr_kmsg_dumper dump; }; /** diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index 5937054b3cdb..35143d2b7fe4 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -1200,6 +1200,128 @@ static void mpi3mr_alloc_ioctl_dma_memory(struct mpi3mr_ioc *mrioc) mpi3mr_free_ioctl_dma_memory(mrioc); } +/** + * mpi3mr_do_mini_dump - copy system logs associated with mrioc. + * @mrioc: Adapter instance reference + * @prev_offset: offset returned from previous operation + * + * Read system logs and search for pattern mpi3mr%d and copy the lines + * into driver diag buffer + * + * Return: next available location in driver diag buffer. + */ +static int mpi3mr_do_mini_dump(struct mpi3mr_ioc *mrioc) +{ + int n = 0, lines, pos_mini_dump = 0; + struct mpi3mr_kmsg_dumper dumper; + size_t len; + char buf[201]; + char *mini_start = "<6> Minidump start\n"; + char *mini_end = "<6> Minidump end\n"; + + struct mpi3_driver_buffer_header *drv_buff_header = NULL; + + dumper = mrioc->dump; + + kmsg_dump_rewind(&dumper.kdumper); + while (kmsg_dump_get_line(&dumper.kdumper, 1, NULL, 0, NULL)) + n++; + + lines = n; + kmsg_dump_rewind(&dumper.kdumper); + + drv_buff_header = (struct mpi3_driver_buffer_header *)mrioc->drv_diag_buffer; + drv_buff_header->signature = 0x43495243; + drv_buff_header->logical_buffer_start = 0; + drv_buff_header->circular_buffer_size = + mrioc->drv_diag_buffer_sz - sizeof(struct mpi3_driver_buffer_header); + drv_buff_header->flags = + MPI3_DRIVER_DIAG_BUFFER_HEADER_FLAGS_CIRCULAR_BUF_FORMAT_ASCII; + + if ((pos_mini_dump + strlen(mini_start) + < mrioc->drv_diag_buffer_sz)) { + sprintf((char *)mrioc->drv_diag_buffer + pos_mini_dump, + "%s\n", mini_start); + pos_mini_dump += strlen(mini_start); + } else { + ioc_info(mrioc, "driver diag buffer is full. minidump is not started\n"); + goto out; + } + + while (kmsg_dump_get_line(&dumper.kdumper, 1, buf, sizeof(buf), &len)) { + if (!lines--) + break; + if (strstr(buf, mrioc->name) && + ((pos_mini_dump + len + strlen(mini_end)) + < mrioc->drv_diag_buffer_sz)) { + sprintf((char *)mrioc->drv_diag_buffer + + pos_mini_dump, "%s", buf); + pos_mini_dump += len; + } + } + + if ((pos_mini_dump + strlen(mini_end) + < mrioc->drv_diag_buffer_sz)) { + sprintf((char *)mrioc->drv_diag_buffer + pos_mini_dump, + "%s\n", mini_end); + pos_mini_dump += strlen(mini_end); + } + +out: + drv_buff_header->logical_buffer_end = + pos_mini_dump - sizeof(struct mpi3_driver_buffer_header); + + ioc_info(mrioc, "driver diag buffer base_address(including 4K header) 0x%016llx, end_address 0x%016llx\n", + (unsigned long long)mrioc->drv_diag_buffer_dma, + (unsigned long long)mrioc->drv_diag_buffer_dma + + mrioc->drv_diag_buffer_sz); + ioc_info(mrioc, "logical_buffer end_address 0x%016llx, logical_buffer_end 0x%08x\n", + (unsigned long long)mrioc->drv_diag_buffer_dma + + drv_buff_header->logical_buffer_end, + drv_buff_header->logical_buffer_end); + + return pos_mini_dump; +} + +/** + * mpi3mr_do_dump - copy system logs into driver diag buffer. + * @mrioc: Adapter instance reference + * + * Return: Nothing. + */ +static void mpi3mr_do_dump(struct mpi3mr_ioc *mrioc) +{ + int offset = 0; + size_t dump_size; + struct mpi3_driver_buffer_header *drv_buff_header = NULL; + + if (!mrioc->drv_diag_buffer) + return; + + memset(mrioc->drv_diag_buffer, 0, mrioc->drv_diag_buffer_sz); + + if (drv_db_level == MRIOC_DRV_DB_DISABLED) + return; + + /* Copy controller specific logs */ + offset += mpi3mr_do_mini_dump(mrioc); + if (drv_db_level != MRIOC_DRV_DB_FULL) + return; + + kmsg_dump_rewind(&mrioc->dump.kdumper); + kmsg_dump_get_buffer(&mrioc->dump.kdumper, true, + mrioc->drv_diag_buffer + offset, + mrioc->drv_diag_buffer_sz - offset, &dump_size); + + drv_buff_header = (struct mpi3_driver_buffer_header *) + mrioc->drv_diag_buffer; + drv_buff_header->logical_buffer_end += dump_size; + ioc_info(mrioc, "logical_buffer end_address(0x%016llx), logical_buffer_end(0x%08x)\n", + (unsigned long long)mrioc->drv_diag_buffer_dma + + drv_buff_header->logical_buffer_end, + drv_buff_header->logical_buffer_end); +} + /** * mpi3mr_clear_reset_history - clear reset history * @mrioc: Adapter instance reference @@ -2767,6 +2889,7 @@ static void mpi3mr_watchdog_work(struct work_struct *work) if (!mrioc->diagsave_timeout) { mpi3mr_print_fault_info(mrioc); ioc_warn(mrioc, "diag save in progress\n"); + mpi3mr_do_dump(mrioc); } if ((mrioc->diagsave_timeout++) <= MPI3_SYSIF_DIAG_SAVE_TIMEOUT) goto schedule_work; @@ -5311,6 +5434,9 @@ int mpi3mr_soft_reset_handler(struct mpi3mr_ioc *mrioc, mpi3mr_ioc_disable_intr(mrioc); if (snapdump) { + dprint_reset(mrioc, + "soft_reset_handler: saving snapdump\n"); + mpi3mr_do_dump(mrioc); mpi3mr_set_diagsave(mrioc); retval = mpi3mr_issue_reset(mrioc, MPI3_SYSIF_HOST_DIAG_RESET_ACTION_DIAG_FAULT, reset_reason);