From patchwork Wed Feb 10 17:54:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17812C433DB for ; Wed, 10 Feb 2021 17:57:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D702D64E6F for ; Wed, 10 Feb 2021 17:57:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232369AbhBJR5A (ORCPT ); Wed, 10 Feb 2021 12:57:00 -0500 Received: from mga01.intel.com ([192.55.52.88]:60436 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232315AbhBJR4q (ORCPT ); Wed, 10 Feb 2021 12:56:46 -0500 IronPort-SDR: vG2jqzvz9JdXwIhckjspVj/Pdn7q5zshE6UUCRo3THv+MkmQA2O2f2dTmfmU7ffVyoGvQ+4L8+ IEmRwVO7DHew== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236009" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236009" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:05 -0800 IronPort-SDR: pNUue1vAJeKBFSVaDGzM508+GA2UMrOul4Y5bazKWSjopBjoZsvSutRztyblZzSTtXF9hIVwiN jcINzHbRP92Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235706" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:05 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 02/20] dlb: initialize device Date: Wed, 10 Feb 2021 11:54:05 -0600 Message-Id: <20210210175423.1873-3-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Assign the physical function device 'ops' callbacks, map the PCI BAR space, create a char device, set the DMA API mask for 64-bit addressing. Add the corresponding undo operations to the remove callback. The ops callbacks are implement behavior that differs depending on device type -- physical function (PF, as implemented here) or virtual function/device (support to be added later). Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/Makefile | 1 + drivers/misc/dlb/dlb_hw_types.h | 23 +++++++++++++ drivers/misc/dlb/dlb_main.c | 59 +++++++++++++++++++++++++++++++++ drivers/misc/dlb/dlb_main.h | 14 ++++++++ drivers/misc/dlb/dlb_pf_ops.c | 50 ++++++++++++++++++++++++++++ 5 files changed, 147 insertions(+) create mode 100644 drivers/misc/dlb/dlb_pf_ops.c diff --git a/drivers/misc/dlb/Makefile b/drivers/misc/dlb/Makefile index 8911375effd2..a33bf774e6a8 100644 --- a/drivers/misc/dlb/Makefile +++ b/drivers/misc/dlb/Makefile @@ -7,3 +7,4 @@ obj-$(CONFIG_INTEL_DLB) := dlb.o dlb-objs := dlb_main.o +dlb-objs += dlb_pf_ops.o diff --git a/drivers/misc/dlb/dlb_hw_types.h b/drivers/misc/dlb/dlb_hw_types.h index 778ec8665ea0..a4ce28c157de 100644 --- a/drivers/misc/dlb/dlb_hw_types.h +++ b/drivers/misc/dlb/dlb_hw_types.h @@ -4,6 +4,20 @@ #ifndef __DLB_HW_TYPES_H #define __DLB_HW_TYPES_H +#include + +/* Read/write register 'reg' in the CSR BAR space */ +#define DLB_CSR_REG_ADDR(a, reg) ((a)->csr_kva + (reg)) +#define DLB_CSR_RD(hw, reg) ioread32(DLB_CSR_REG_ADDR((hw), (reg))) +#define DLB_CSR_WR(hw, reg, value) iowrite32((value), \ + DLB_CSR_REG_ADDR((hw), (reg))) + +/* Read/write register 'reg' in the func BAR space */ +#define DLB_FUNC_REG_ADDR(a, reg) ((a)->func_kva + (reg)) +#define DLB_FUNC_RD(hw, reg) ioread32(DLB_FUNC_REG_ADDR((hw), (reg))) +#define DLB_FUNC_WR(hw, reg, value) iowrite32((value), \ + DLB_FUNC_REG_ADDR((hw), (reg))) + #define DLB_MAX_NUM_VDEVS 16 #define DLB_MAX_NUM_DOMAINS 32 #define DLB_MAX_NUM_LDB_QUEUES 32 /* LDB == load-balanced */ @@ -29,4 +43,13 @@ #define PCI_DEVICE_ID_INTEL_DLB_PF 0x2710 +struct dlb_hw { + /* BAR 0 address */ + void __iomem *csr_kva; + unsigned long csr_phys_addr; + /* BAR 2 address */ + void __iomem *func_kva; + unsigned long func_phys_addr; +}; + #endif /* __DLB_HW_TYPES_H */ diff --git a/drivers/misc/dlb/dlb_main.c b/drivers/misc/dlb/dlb_main.c index fbd77203b398..7fb6e9c360c8 100644 --- a/drivers/misc/dlb/dlb_main.c +++ b/drivers/misc/dlb/dlb_main.c @@ -16,10 +16,38 @@ MODULE_LICENSE("GPL v2"); MODULE_DESCRIPTION("Intel(R) Dynamic Load Balancer (DLB) Driver"); static struct class *dlb_class; +static struct cdev dlb_cdev; static dev_t dlb_devt; static DEFINE_IDR(dlb_ids); static DEFINE_SPINLOCK(dlb_ids_lock); +static int dlb_device_create(struct dlb *dlb, struct pci_dev *pdev) +{ + /* + * Create a new device in order to create a /dev/dlb node. This device + * is a child of the DLB PCI device. + */ + dlb->dev_number = MKDEV(MAJOR(dlb_devt), dlb->id); + dlb->dev = device_create(dlb_class, &pdev->dev, dlb->dev_number, dlb, + "dlb%d", dlb->id); + if (IS_ERR(dlb->dev)) { + dev_err(dlb->dev, "device_create() returned %ld\n", + PTR_ERR(dlb->dev)); + + return PTR_ERR(dlb->dev); + } + + return 0; +} + +/********************************/ +/****** Char dev callbacks ******/ +/********************************/ + +static const struct file_operations dlb_fops = { + .owner = THIS_MODULE, +}; + /**********************************/ /****** PCI driver callbacks ******/ /**********************************/ @@ -33,6 +61,8 @@ static int dlb_probe(struct pci_dev *pdev, const struct pci_device_id *pdev_id) if (!dlb) return -ENOMEM; + dlb->ops = &dlb_pf_ops; + pci_set_drvdata(pdev, dlb); dlb->pdev = pdev; @@ -71,8 +101,24 @@ static int dlb_probe(struct pci_dev *pdev, const struct pci_device_id *pdev_id) if (ret != 0) dev_info(&pdev->dev, "Failed to enable AER %d\n", ret); + ret = dlb->ops->map_pci_bar_space(dlb, pdev); + if (ret) + goto map_pci_bar_fail; + + ret = dlb_device_create(dlb, pdev); + if (ret) + goto map_pci_bar_fail; + + ret = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); + if (ret) + goto dma_set_mask_fail; + return 0; +dma_set_mask_fail: + device_destroy(dlb_class, dlb->dev_number); +map_pci_bar_fail: + pci_disable_pcie_error_reporting(pdev); pci_enable_device_fail: spin_lock(&dlb_ids_lock); idr_remove(&dlb_ids, dlb->id); @@ -85,6 +131,8 @@ static void dlb_remove(struct pci_dev *pdev) { struct dlb *dlb = pci_get_drvdata(pdev); + device_destroy(dlb_class, dlb->dev_number); + pci_disable_pcie_error_reporting(pdev); spin_lock(&dlb_ids_lock); @@ -107,6 +155,7 @@ static struct pci_driver dlb_pci_driver = { static int __init dlb_init_module(void) { + int dlb_major; int err; dlb_class = class_create(THIS_MODULE, "dlb"); @@ -126,6 +175,12 @@ static int __init dlb_init_module(void) goto alloc_chrdev_fail; } + dlb_major = MAJOR(dlb_devt); + cdev_init(&dlb_cdev, &dlb_fops); + err = cdev_add(&dlb_cdev, MKDEV(dlb_major, 0), DLB_MAX_NUM_DEVICES); + if (err) + goto cdev_add_fail; + err = pci_register_driver(&dlb_pci_driver); if (err < 0) { pr_err("dlb: pci_register_driver() returned %d\n", err); @@ -136,6 +191,8 @@ static int __init dlb_init_module(void) return 0; pci_register_fail: + cdev_del(&dlb_cdev); +cdev_add_fail: unregister_chrdev_region(dlb_devt, DLB_MAX_NUM_DEVICES); alloc_chrdev_fail: class_destroy(dlb_class); @@ -147,6 +204,8 @@ static void __exit dlb_exit_module(void) { pci_unregister_driver(&dlb_pci_driver); + cdev_del(&dlb_cdev); + unregister_chrdev_region(dlb_devt, DLB_MAX_NUM_DEVICES); class_destroy(dlb_class); diff --git a/drivers/misc/dlb/dlb_main.h b/drivers/misc/dlb/dlb_main.h index ab2a2014bafd..21570b206419 100644 --- a/drivers/misc/dlb/dlb_main.h +++ b/drivers/misc/dlb/dlb_main.h @@ -29,9 +29,23 @@ enum dlb_device_type { DLB_PF, }; +struct dlb; + +struct dlb_device_ops { + int (*map_pci_bar_space)(struct dlb *dlb, struct pci_dev *pdev); + void (*unmap_pci_bar_space)(struct dlb *dlb, struct pci_dev *pdev); +}; + +extern struct dlb_device_ops dlb_pf_ops; + struct dlb { struct pci_dev *pdev; + struct dlb_hw hw; + struct dlb_device_ops *ops; + struct device *dev; + enum dlb_device_type type; int id; + dev_t dev_number; }; #endif /* __DLB_MAIN_H */ diff --git a/drivers/misc/dlb/dlb_pf_ops.c b/drivers/misc/dlb/dlb_pf_ops.c new file mode 100644 index 000000000000..0951c99f6183 --- /dev/null +++ b/drivers/misc/dlb/dlb_pf_ops.c @@ -0,0 +1,50 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(C) 2016-2020 Intel Corporation. All rights reserved. */ + +#include "dlb_main.h" + +/********************************/ +/****** PCI BAR management ******/ +/********************************/ + +static void +dlb_pf_unmap_pci_bar_space(struct dlb *dlb, struct pci_dev *pdev) +{ + pcim_iounmap(pdev, dlb->hw.csr_kva); + pcim_iounmap(pdev, dlb->hw.func_kva); +} + +static int +dlb_pf_map_pci_bar_space(struct dlb *dlb, struct pci_dev *pdev) +{ + dlb->hw.func_kva = pcim_iomap_table(pdev)[DLB_FUNC_BAR]; + dlb->hw.func_phys_addr = pci_resource_start(pdev, DLB_FUNC_BAR); + + if (!dlb->hw.func_kva) { + dev_err(&pdev->dev, "Cannot iomap BAR 0 (size %llu)\n", + pci_resource_len(pdev, 0)); + + return -EIO; + } + + dlb->hw.csr_kva = pcim_iomap_table(pdev)[DLB_CSR_BAR]; + dlb->hw.csr_phys_addr = pci_resource_start(pdev, DLB_CSR_BAR); + + if (!dlb->hw.csr_kva) { + dev_err(&pdev->dev, "Cannot iomap BAR 2 (size %llu)\n", + pci_resource_len(pdev, 2)); + + return -EIO; + } + + return 0; +} + +/********************************/ +/****** DLB PF Device Ops ******/ +/********************************/ + +struct dlb_device_ops dlb_pf_ops = { + .map_pci_bar_space = dlb_pf_map_pci_bar_space, + .unmap_pci_bar_space = dlb_pf_unmap_pci_bar_space, +}; From patchwork Wed Feb 10 17:54:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C98EC433E0 for ; Wed, 10 Feb 2021 17:57:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B633864E6F for ; Wed, 10 Feb 2021 17:57:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233189AbhBJR5Y (ORCPT ); Wed, 10 Feb 2021 12:57:24 -0500 Received: from mga01.intel.com ([192.55.52.88]:60442 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232734AbhBJR4z (ORCPT ); Wed, 10 Feb 2021 12:56:55 -0500 IronPort-SDR: tQL10W59T+e6BvFap6RdI1/btlAzo/D6B4v569/5A4IrDPbpTwSlb0qB3vGavF3lOtMfgCHtdm 0OtbJeXaPM6Q== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236015" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236015" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:06 -0800 IronPort-SDR: +1GA0qKxBkldCO6AIIkXR2iNu6x6SqlHnQ/q7wPjxw7NyvX96h4AAUCo85xG8BiqtRYLX2GeLp UgJEZrboblRA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235720" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:06 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 04/20] dlb: add device ioctl layer and first three ioctls Date: Wed, 10 Feb 2021 11:54:07 -0600 Message-Id: <20210210175423.1873-5-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Introduce the dlb device ioctl layer and the first three ioctls: query device version, query available resources, and create a scheduling domain. Also introduce the user-space interface file dlb_user.h. The device version query is designed to allow each DLB device version/type to have its own unique ioctl API through the /dev/dlb%d node. Each such API would share in common the device version command as its first command, and all subsequent commands can be unique to the particular device. The hardware operation for scheduling domain creation will be added in a subsequent commit. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- .../userspace-api/ioctl/ioctl-number.rst | 1 + drivers/misc/dlb/Makefile | 2 +- drivers/misc/dlb/dlb_bitmap.h | 32 ++++ drivers/misc/dlb/dlb_ioctl.c | 103 +++++++++++ drivers/misc/dlb/dlb_main.c | 1 + drivers/misc/dlb/dlb_main.h | 10 ++ drivers/misc/dlb/dlb_pf_ops.c | 22 +++ drivers/misc/dlb/dlb_resource.c | 62 +++++++ drivers/misc/dlb/dlb_resource.h | 4 + include/uapi/linux/dlb.h | 167 ++++++++++++++++++ 10 files changed, 403 insertions(+), 1 deletion(-) create mode 100644 drivers/misc/dlb/dlb_ioctl.c create mode 100644 include/uapi/linux/dlb.h diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst index a4c75a28c839..747b48b141c8 100644 --- a/Documentation/userspace-api/ioctl/ioctl-number.rst +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst @@ -300,6 +300,7 @@ Code Seq# Include File Comments 'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict! '|' 00-7F linux/media.h 0x80 00-1F linux/fb.h +0x81 00-1F uapi/linux/dlb.h 0x89 00-06 arch/x86/include/asm/sockios.h 0x89 0B-DF linux/sockios.h 0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range diff --git a/drivers/misc/dlb/Makefile b/drivers/misc/dlb/Makefile index 8a49ea5fd752..aaafb3086d8d 100644 --- a/drivers/misc/dlb/Makefile +++ b/drivers/misc/dlb/Makefile @@ -7,4 +7,4 @@ obj-$(CONFIG_INTEL_DLB) := dlb.o dlb-objs := dlb_main.o -dlb-objs += dlb_pf_ops.o dlb_resource.o +dlb-objs += dlb_pf_ops.o dlb_resource.o dlb_ioctl.o diff --git a/drivers/misc/dlb/dlb_bitmap.h b/drivers/misc/dlb/dlb_bitmap.h index fb3ef52a306d..3ea78b42c79f 100644 --- a/drivers/misc/dlb/dlb_bitmap.h +++ b/drivers/misc/dlb/dlb_bitmap.h @@ -73,4 +73,36 @@ static inline void dlb_bitmap_free(struct dlb_bitmap *bitmap) kfree(bitmap); } +/** + * dlb_bitmap_longest_set_range() - returns longest contiguous range of set + * bits + * @bitmap: pointer to dlb_bitmap structure. + * + * Return: + * Returns the bitmap's longest contiguous range of set bits upon success, + * <0 otherwise. + * + * Errors: + * EINVAL - bitmap is NULL or is uninitialized. + */ +static inline int dlb_bitmap_longest_set_range(struct dlb_bitmap *bitmap) +{ + int max_len, len; + int start, end; + + if (!bitmap || !bitmap->map) + return -EINVAL; + + if (bitmap_weight(bitmap->map, bitmap->len) == 0) + return 0; + + max_len = 0; + bitmap_for_each_set_region(bitmap->map, start, end, 0, bitmap->len) { + len = end - start; + if (max_len < len) + max_len = len; + } + return max_len; +} + #endif /* __DLB_OSDEP_BITMAP_H */ diff --git a/drivers/misc/dlb/dlb_ioctl.c b/drivers/misc/dlb/dlb_ioctl.c new file mode 100644 index 000000000000..47d6cab773d4 --- /dev/null +++ b/drivers/misc/dlb/dlb_ioctl.c @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(C) 2016-2020 Intel Corporation. All rights reserved. */ + +#include + +#include + +#include "dlb_main.h" + +/* [7:0]: device revision, [15:8]: device version */ +#define DLB_SET_DEVICE_VERSION(ver, rev) (((ver) << 8) | (rev)) + +static int dlb_ioctl_get_device_version(unsigned long user_arg) +{ + struct dlb_get_device_version_args arg; + u8 revision; + + switch (boot_cpu_data.x86_stepping) { + case 0: + revision = DLB_REV_A0; + break; + case 1: + revision = DLB_REV_A1; + break; + case 2: + revision = DLB_REV_A2; + break; + default: + /* Treat all revisions >= 3 as B0 */ + revision = DLB_REV_B0; + break; + } + + arg.response.status = 0; + arg.response.id = DLB_SET_DEVICE_VERSION(2, revision); + + if (copy_to_user((void __user *)user_arg, &arg, sizeof(arg))) + return -EFAULT; + + return 0; +} + +static int dlb_ioctl_create_sched_domain(struct dlb *dlb, unsigned long user_arg) +{ + struct dlb_create_sched_domain_args __user *uarg; + struct dlb_create_sched_domain_args arg; + struct dlb_cmd_response response = {0}; + int ret; + + uarg = (void __user *)user_arg; + if (copy_from_user(&arg, uarg, sizeof(arg))) + return -EFAULT; + + mutex_lock(&dlb->resource_mutex); + + ret = dlb->ops->create_sched_domain(&dlb->hw, &arg, &response); + + mutex_unlock(&dlb->resource_mutex); + + response.status = ret; + BUILD_BUG_ON(offsetof(typeof(arg), response) != 0); + + if (copy_to_user((void __user *)&uarg->response, &response, sizeof(response))) + return -EFAULT; + + return ret; +} + +static int dlb_ioctl_get_num_resources(struct dlb *dlb, unsigned long user_arg) +{ + struct dlb_get_num_resources_args arg = {0}; + int ret; + + mutex_lock(&dlb->resource_mutex); + + ret = dlb->ops->get_num_resources(&dlb->hw, &arg); + + mutex_unlock(&dlb->resource_mutex); + + BUILD_BUG_ON(offsetof(typeof(arg), response) != 0); + arg.response.status = ret; + + if (copy_to_user((void __user *)user_arg, &arg, sizeof(arg))) + return -EFAULT; + + return ret; +} + +long dlb_ioctl(struct file *f, unsigned int cmd, unsigned long arg) +{ + struct dlb *dlb = f->private_data; + + switch (cmd) { + case DLB_IOC_GET_DEVICE_VERSION: + return dlb_ioctl_get_device_version(arg); + case DLB_IOC_CREATE_SCHED_DOMAIN: + return dlb_ioctl_create_sched_domain(dlb, arg); + case DLB_IOC_GET_NUM_RESOURCES: + return dlb_ioctl_get_num_resources(dlb, arg); + default: + return -ENOTTY; + } +} diff --git a/drivers/misc/dlb/dlb_main.c b/drivers/misc/dlb/dlb_main.c index 12707b23ab3e..d92956b1643d 100644 --- a/drivers/misc/dlb/dlb_main.c +++ b/drivers/misc/dlb/dlb_main.c @@ -64,6 +64,7 @@ static int dlb_device_create(struct dlb *dlb, struct pci_dev *pdev) static const struct file_operations dlb_fops = { .owner = THIS_MODULE, + .unlocked_ioctl = dlb_ioctl, }; /**********************************/ diff --git a/drivers/misc/dlb/dlb_main.h b/drivers/misc/dlb/dlb_main.h index ec5eb7bd8f54..3089a66a3560 100644 --- a/drivers/misc/dlb/dlb_main.h +++ b/drivers/misc/dlb/dlb_main.h @@ -12,6 +12,8 @@ #include #include +#include + #include "dlb_hw_types.h" /* @@ -37,6 +39,11 @@ struct dlb_device_ops { int (*init_driver_state)(struct dlb *dlb); void (*enable_pm)(struct dlb *dlb); int (*wait_for_device_ready)(struct dlb *dlb, struct pci_dev *pdev); + int (*create_sched_domain)(struct dlb_hw *hw, + struct dlb_create_sched_domain_args *args, + struct dlb_cmd_response *resp); + int (*get_num_resources)(struct dlb_hw *hw, + struct dlb_get_num_resources_args *args); }; extern struct dlb_device_ops dlb_pf_ops; @@ -56,4 +63,7 @@ struct dlb { dev_t dev_number; }; +/* Prototypes for dlb_ioctl.c */ +long dlb_ioctl(struct file *f, unsigned int cmd, unsigned long arg); + #endif /* __DLB_MAIN_H */ diff --git a/drivers/misc/dlb/dlb_pf_ops.c b/drivers/misc/dlb/dlb_pf_ops.c index 124b4fee8564..125ef6fe6c70 100644 --- a/drivers/misc/dlb/dlb_pf_ops.c +++ b/drivers/misc/dlb/dlb_pf_ops.c @@ -95,6 +95,26 @@ static int dlb_pf_wait_for_device_ready(struct dlb *dlb, struct pci_dev *pdev) return 0; } +/*****************************/ +/****** IOCTL callbacks ******/ +/*****************************/ + +static int dlb_pf_create_sched_domain(struct dlb_hw *hw, + struct dlb_create_sched_domain_args *args, + struct dlb_cmd_response *resp) +{ + resp->id = 0; + resp->status = 0; + + return 0; +} + +static int dlb_pf_get_num_resources(struct dlb_hw *hw, + struct dlb_get_num_resources_args *args) +{ + return dlb_hw_get_num_resources(hw, args, false, 0); +} + /********************************/ /****** DLB PF Device Ops ******/ /********************************/ @@ -105,4 +125,6 @@ struct dlb_device_ops dlb_pf_ops = { .init_driver_state = dlb_pf_init_driver_state, .enable_pm = dlb_pf_enable_pm, .wait_for_device_ready = dlb_pf_wait_for_device_ready, + .create_sched_domain = dlb_pf_create_sched_domain, + .get_num_resources = dlb_pf_get_num_resources, }; diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index fca444c46aca..9d75b12eb793 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -200,6 +200,68 @@ int dlb_resource_init(struct dlb_hw *hw) return ret; } +/** + * dlb_hw_get_num_resources() - query the PCI function's available resources + * @hw: dlb_hw handle for a particular device. + * @arg: pointer to resource counts. + * @vdev_req: indicates whether this request came from a vdev. + * @vdev_id: If vdev_req is true, this contains the vdev's ID. + * + * This function returns the number of available resources for the PF or for a + * VF. + * + * A vdev can be either an SR-IOV virtual function or a Scalable IOV virtual + * device. + * + * Return: + * Returns 0 upon success, -EINVAL if vdev_req is true and vdev_id is + * invalid. + */ +int dlb_hw_get_num_resources(struct dlb_hw *hw, + struct dlb_get_num_resources_args *arg, + bool vdev_req, unsigned int vdev_id) +{ + struct dlb_function_resources *rsrcs; + struct dlb_bitmap *map; + int i; + + if (vdev_req && vdev_id >= DLB_MAX_NUM_VDEVS) + return -EINVAL; + + if (vdev_req) + rsrcs = &hw->vdev[vdev_id]; + else + rsrcs = &hw->pf; + + arg->num_sched_domains = rsrcs->num_avail_domains; + + arg->num_ldb_queues = rsrcs->num_avail_ldb_queues; + + arg->num_ldb_ports = 0; + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) + arg->num_ldb_ports += rsrcs->num_avail_ldb_ports[i]; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) + arg->num_cos_ldb_ports[i] = rsrcs->num_avail_ldb_ports[i]; + + arg->num_dir_ports = rsrcs->num_avail_dir_pq_pairs; + + arg->num_atomic_inflights = rsrcs->num_avail_aqed_entries; + + map = rsrcs->avail_hist_list_entries; + + arg->num_hist_list_entries = bitmap_weight(map->map, map->len); + + arg->max_contiguous_hist_list_entries = + dlb_bitmap_longest_set_range(map); + + arg->num_ldb_credits = rsrcs->num_avail_qed_entries; + + arg->num_dir_credits = rsrcs->num_avail_dqed_entries; + + return 0; +} + /** * dlb_clr_pmcsr_disable() - power on bulk of DLB 2.0 logic * @hw: dlb_hw handle for a particular device. diff --git a/drivers/misc/dlb/dlb_resource.h b/drivers/misc/dlb/dlb_resource.h index 2229813d9c45..3e6d419796bc 100644 --- a/drivers/misc/dlb/dlb_resource.h +++ b/drivers/misc/dlb/dlb_resource.h @@ -12,6 +12,10 @@ int dlb_resource_init(struct dlb_hw *hw); void dlb_resource_free(struct dlb_hw *hw); +int dlb_hw_get_num_resources(struct dlb_hw *hw, + struct dlb_get_num_resources_args *arg, + bool vdev_req, unsigned int vdev_id); + void dlb_clr_pmcsr_disable(struct dlb_hw *hw); #endif /* __DLB_RESOURCE_H */ diff --git a/include/uapi/linux/dlb.h b/include/uapi/linux/dlb.h new file mode 100644 index 000000000000..87ba1bfa75ed --- /dev/null +++ b/include/uapi/linux/dlb.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* Copyright(C) 2016-2020 Intel Corporation. All rights reserved. */ + +#ifndef __DLB_H +#define __DLB_H + +#include + +struct dlb_cmd_response { + __u32 status; /* Interpret using enum dlb_error */ + __u32 id; +}; + +/********************************/ +/* 'dlb' device file commands */ +/********************************/ + +#define DLB_DEVICE_VERSION(x) (((x) >> 8) & 0xFF) +#define DLB_DEVICE_REVISION(x) ((x) & 0xFF) + +enum dlb_revisions { + DLB_REV_A0 = 0, + DLB_REV_A1, + DLB_REV_A2, + DLB_REV_B0, +}; + +/* + * DLB_CMD_GET_DEVICE_VERSION: Query the DLB device version. + * + * All DLB device versions have the same ioctl API. Each version may have + * different resource and feature set. The device revision is provided + * in case of any hardware errata. + * + * Output parameters: + * @response.status: Detailed error code. In certain cases, such as if the + * ioctl request arg is invalid, the driver won't set status. + * @response.id[7:0]: Device revision. + * @response.id[15:8]: Device version. + */ + +struct dlb_get_device_version_args { + /* Output parameters */ + struct dlb_cmd_response response; +}; + +/* + * DLB_CMD_CREATE_SCHED_DOMAIN: Create a DLB 2.0 scheduling domain and reserve + * its hardware resources. This command returns the newly created domain + * ID and a file descriptor for accessing the domain. + * + * Output parameters: + * @response.status: Detailed error code. In certain cases, such as if the + * ioctl request arg is invalid, the driver won't set status. + * @response.id: domain ID. + * @domain_fd: file descriptor for performing the domain's ioctl operations + * @padding0: Reserved for future use. + * + * Input parameters: + * @num_ldb_queues: Number of load-balanced queues. + * @num_ldb_ports: Number of load-balanced ports that can be allocated from + * any class-of-service with available ports. + * @num_cos_ldb_ports[4]: Number of load-balanced ports from + * classes-of-service 0-3. + * @num_dir_ports: Number of directed ports. A directed port has one directed + * queue, so no num_dir_queues argument is necessary. + * @num_atomic_inflights: This specifies the amount of temporary atomic QE + * storage for the domain. This storage is divided among the domain's + * load-balanced queues that are configured for atomic scheduling. + * @num_hist_list_entries: Amount of history list storage. This is divided + * among the domain's CQs. + * @num_ldb_credits: Amount of load-balanced QE storage (QED). QEs occupy this + * space until they are scheduled to a load-balanced CQ. One credit + * represents the storage for one QE. + * @num_dir_credits: Amount of directed QE storage (DQED). QEs occupy this + * space until they are scheduled to a directed CQ. One credit represents + * the storage for one QE. + * @cos_strict: If set, return an error if there are insufficient ports in + * class-of-service N to satisfy the num_ldb_ports_cosN argument. If + * unset, attempt to fulfill num_ldb_ports_cosN arguments from other + * classes-of-service if class N does not contain enough free ports. + * @padding1: Reserved for future use. + */ +struct dlb_create_sched_domain_args { + /* Output parameters */ + struct dlb_cmd_response response; + __u32 domain_fd; + __u32 padding0; + /* Input parameters */ + __u32 num_ldb_queues; + __u32 num_ldb_ports; + __u32 num_cos_ldb_ports[4]; + __u32 num_dir_ports; + __u32 num_atomic_inflights; + __u32 num_hist_list_entries; + __u32 num_ldb_credits; + __u32 num_dir_credits; + __u8 cos_strict; + __u8 padding1[3]; +}; + +/* + * DLB_CMD_GET_NUM_RESOURCES: Return the number of available resources + * (queues, ports, etc.) that this device owns. + * + * Output parameters: + * @response.status: Detailed error code. In certain cases, such as if the + * ioctl request arg is invalid, the driver won't set status. + * @num_domains: Number of available scheduling domains. + * @num_ldb_queues: Number of available load-balanced queues. + * @num_ldb_ports: Total number of available load-balanced ports. + * @num_cos_ldb_ports[4]: Number of available load-balanced ports from + * classes-of-service 0-3. + * @num_dir_ports: Number of available directed ports. There is one directed + * queue for every directed port. + * @num_atomic_inflights: Amount of available temporary atomic QE storage. + * @num_hist_list_entries: Amount of history list storage. + * @max_contiguous_hist_list_entries: History list storage is allocated in + * a contiguous chunk, and this return value is the longest available + * contiguous range of history list entries. + * @num_ldb_credits: Amount of available load-balanced QE storage. + * @num_dir_credits: Amount of available directed QE storage. + */ +struct dlb_get_num_resources_args { + /* Output parameters */ + struct dlb_cmd_response response; + __u32 num_sched_domains; + __u32 num_ldb_queues; + __u32 num_ldb_ports; + __u32 num_cos_ldb_ports[4]; + __u32 num_dir_ports; + __u32 num_atomic_inflights; + __u32 num_hist_list_entries; + __u32 max_contiguous_hist_list_entries; + __u32 num_ldb_credits; + __u32 num_dir_credits; +}; + +enum dlb_user_interface_commands { + DLB_CMD_GET_DEVICE_VERSION, + DLB_CMD_CREATE_SCHED_DOMAIN, + DLB_CMD_GET_NUM_RESOURCES, + + /* NUM_DLB_CMD must be last */ + NUM_DLB_CMD, +}; + +/********************/ +/* dlb ioctl codes */ +/********************/ + +#define DLB_IOC_MAGIC 0x81 + +#define DLB_IOC_GET_DEVICE_VERSION \ + _IOR(DLB_IOC_MAGIC, \ + DLB_CMD_GET_DEVICE_VERSION, \ + struct dlb_get_device_version_args) +#define DLB_IOC_CREATE_SCHED_DOMAIN \ + _IOWR(DLB_IOC_MAGIC, \ + DLB_CMD_CREATE_SCHED_DOMAIN, \ + struct dlb_create_sched_domain_args) +#define DLB_IOC_GET_NUM_RESOURCES \ + _IOR(DLB_IOC_MAGIC, \ + DLB_CMD_GET_NUM_RESOURCES, \ + struct dlb_get_num_resources_args) + +#endif /* __DLB_H */ From patchwork Wed Feb 10 17:54:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 194D7C433E9 for ; Wed, 10 Feb 2021 17:58:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E448A64EE6 for ; Wed, 10 Feb 2021 17:58:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233501AbhBJR56 (ORCPT ); Wed, 10 Feb 2021 12:57:58 -0500 Received: from mga01.intel.com ([192.55.52.88]:60436 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232918AbhBJR5E (ORCPT ); Wed, 10 Feb 2021 12:57:04 -0500 IronPort-SDR: XyI0voq2kAjWsG02HXwSI12Fjufnx04dl23HnBGOOBJa91v4luk7t83tcAcFw85vu+vqSFT+Cf yXC1LZrymPsg== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236018" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236018" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:07 -0800 IronPort-SDR: zSixV8+/D1xCxmJnjd0M5Tz7zsei2ozPVflk5yQrwKciyLu5t4jiYpWj79AEIOgSQCcyv3KY1Y iBKW+SLfXPEw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235735" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:07 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 06/20] dlb: add domain software reset Date: Wed, 10 Feb 2021 11:54:09 -0600 Message-Id: <20210210175423.1873-7-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add operation to reset a domain's resource's software state when its reference count reaches zero, and re-inserts those resources in their respective available-resources linked lists, for use by future scheduling domains. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_bitmap.h | 28 +++++ drivers/misc/dlb/dlb_hw_types.h | 6 + drivers/misc/dlb/dlb_ioctl.c | 10 +- drivers/misc/dlb/dlb_main.c | 10 +- drivers/misc/dlb/dlb_main.h | 2 + drivers/misc/dlb/dlb_pf_ops.c | 7 ++ drivers/misc/dlb/dlb_resource.c | 217 ++++++++++++++++++++++++++++++++ drivers/misc/dlb/dlb_resource.h | 3 + include/uapi/linux/dlb.h | 1 + 9 files changed, 282 insertions(+), 2 deletions(-) diff --git a/drivers/misc/dlb/dlb_bitmap.h b/drivers/misc/dlb/dlb_bitmap.h index 5cebf833fab4..332135689dd9 100644 --- a/drivers/misc/dlb/dlb_bitmap.h +++ b/drivers/misc/dlb/dlb_bitmap.h @@ -73,6 +73,34 @@ static inline void dlb_bitmap_free(struct dlb_bitmap *bitmap) kfree(bitmap); } +/** + * dlb_bitmap_set_range() - set a range of bitmap entries + * @bitmap: pointer to dlb_bitmap structure. + * @bit: starting bit index. + * @len: length of the range. + * + * Return: + * Returns 0 upon success, < 0 otherwise. + * + * Errors: + * EINVAL - bitmap is NULL or is uninitialized, or the range exceeds the bitmap + * length. + */ +static inline int dlb_bitmap_set_range(struct dlb_bitmap *bitmap, + unsigned int bit, + unsigned int len) +{ + if (!bitmap || !bitmap->map) + return -EINVAL; + + if (bitmap->len <= bit) + return -EINVAL; + + bitmap_set(bitmap->map, bit, len); + + return 0; +} + /** * dlb_bitmap_clear_range() - clear a range of bitmap entries * @bitmap: pointer to dlb_bitmap structure. diff --git a/drivers/misc/dlb/dlb_hw_types.h b/drivers/misc/dlb/dlb_hw_types.h index 3e03b061d5ff..c486ea344292 100644 --- a/drivers/misc/dlb/dlb_hw_types.h +++ b/drivers/misc/dlb/dlb_hw_types.h @@ -150,6 +150,12 @@ struct dlb_sn_group { u32 id; }; +static inline void +dlb_sn_group_free_slot(struct dlb_sn_group *group, int slot) +{ + group->slot_use_bitmap &= ~(BIT(slot)); +} + struct dlb_hw_domain { struct dlb_function_resources *parent_func; struct list_head func_list; diff --git a/drivers/misc/dlb/dlb_ioctl.c b/drivers/misc/dlb/dlb_ioctl.c index 7871d0cea118..75892966f061 100644 --- a/drivers/misc/dlb/dlb_ioctl.c +++ b/drivers/misc/dlb/dlb_ioctl.c @@ -57,13 +57,21 @@ static int dlb_ioctl_create_sched_domain(struct dlb *dlb, unsigned long user_arg mutex_lock(&dlb->resource_mutex); + if (dlb->domain_reset_failed) { + response.status = DLB_ST_DOMAIN_RESET_FAILED; + ret = -EINVAL; + goto unlock; + } + ret = dlb->ops->create_sched_domain(&dlb->hw, &arg, &response); if (ret) goto unlock; ret = dlb_init_domain(dlb, response.id); - if (ret) + if (ret) { + dlb->ops->reset_domain(&dlb->hw, response.id); goto unlock; + } domain = dlb->sched_domains[response.id]; diff --git a/drivers/misc/dlb/dlb_main.c b/drivers/misc/dlb/dlb_main.c index a4ed413eee2f..70030d779033 100644 --- a/drivers/misc/dlb/dlb_main.c +++ b/drivers/misc/dlb/dlb_main.c @@ -103,12 +103,20 @@ int dlb_init_domain(struct dlb *dlb, u32 domain_id) static int __dlb_free_domain(struct dlb_domain *domain) { struct dlb *dlb = domain->dlb; + int ret; + + ret = dlb->ops->reset_domain(&dlb->hw, domain->id); + if (ret) { + dlb->domain_reset_failed = true; + dev_err(dlb->dev, + "Internal error: Domain reset failed. To recover, reset the device.\n"); + } dlb->sched_domains[domain->id] = NULL; kfree(domain); - return 0; + return ret; } void dlb_free_domain(struct kref *kref) diff --git a/drivers/misc/dlb/dlb_main.h b/drivers/misc/dlb/dlb_main.h index 824416e6cdcf..ecfda11b297b 100644 --- a/drivers/misc/dlb/dlb_main.h +++ b/drivers/misc/dlb/dlb_main.h @@ -44,6 +44,7 @@ struct dlb_device_ops { struct dlb_cmd_response *resp); int (*get_num_resources)(struct dlb_hw *hw, struct dlb_get_num_resources_args *args); + int (*reset_domain)(struct dlb_hw *hw, u32 domain_id); }; extern struct dlb_device_ops dlb_pf_ops; @@ -70,6 +71,7 @@ struct dlb { enum dlb_device_type type; int id; dev_t dev_number; + u8 domain_reset_failed; }; /* Prototypes for dlb_ioctl.c */ diff --git a/drivers/misc/dlb/dlb_pf_ops.c b/drivers/misc/dlb/dlb_pf_ops.c index b59e9eaa600d..5dea0037d14b 100644 --- a/drivers/misc/dlb/dlb_pf_ops.c +++ b/drivers/misc/dlb/dlb_pf_ops.c @@ -112,6 +112,12 @@ static int dlb_pf_get_num_resources(struct dlb_hw *hw, return dlb_hw_get_num_resources(hw, args, false, 0); } +static int +dlb_pf_reset_domain(struct dlb_hw *hw, u32 id) +{ + return dlb_reset_domain(hw, id, false, 0); +} + /********************************/ /****** DLB PF Device Ops ******/ /********************************/ @@ -124,4 +130,5 @@ struct dlb_device_ops dlb_pf_ops = { .wait_for_device_ready = dlb_pf_wait_for_device_ready, .create_sched_domain = dlb_pf_create_sched_domain, .get_num_resources = dlb_pf_get_num_resources, + .reset_domain = dlb_pf_reset_domain, }; diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index b7df23c6a158..6d73c2479819 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -201,6 +201,29 @@ int dlb_resource_init(struct dlb_hw *hw) return ret; } +static struct dlb_hw_domain *dlb_get_domain_from_id(struct dlb_hw *hw, u32 id, + bool vdev_req, + unsigned int vdev_id) +{ + struct dlb_function_resources *rsrcs; + struct dlb_hw_domain *domain; + + if (id >= DLB_MAX_NUM_DOMAINS) + return NULL; + + if (!vdev_req) + return &hw->domains[id]; + + rsrcs = &hw->vdev[vdev_id]; + + list_for_each_entry(domain, &rsrcs->used_domains, func_list) { + if (domain->id.virt_id == id) + return domain; + } + + return NULL; +} + static int dlb_attach_ldb_queues(struct dlb_hw *hw, struct dlb_function_resources *rsrcs, struct dlb_hw_domain *domain, u32 num_queues, @@ -805,6 +828,200 @@ int dlb_hw_create_sched_domain(struct dlb_hw *hw, return 0; } +static int dlb_domain_reset_software_state(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb *dlb = container_of(hw, struct dlb, hw); + struct dlb_dir_pq_pair *tmp_dir_port; + struct dlb_function_resources *rsrcs; + struct dlb_ldb_queue *tmp_ldb_queue; + struct dlb_ldb_port *tmp_ldb_port; + struct dlb_dir_pq_pair *dir_port; + struct dlb_ldb_queue *ldb_queue; + struct dlb_ldb_port *ldb_port; + int ret, i; + + lockdep_assert_held(&dlb->resource_mutex); + + rsrcs = domain->parent_func; + + /* Move the domain's ldb queues to the function's avail list */ + list_for_each_entry_safe(ldb_queue, tmp_ldb_queue, + &domain->used_ldb_queues, domain_list) { + if (ldb_queue->sn_cfg_valid) { + struct dlb_sn_group *grp; + + grp = &hw->rsrcs.sn_groups[ldb_queue->sn_group]; + + dlb_sn_group_free_slot(grp, ldb_queue->sn_slot); + ldb_queue->sn_cfg_valid = false; + } + + ldb_queue->owned = false; + ldb_queue->num_mappings = 0; + ldb_queue->num_pending_additions = 0; + + list_del(&ldb_queue->domain_list); + list_add(&ldb_queue->func_list, &rsrcs->avail_ldb_queues); + rsrcs->num_avail_ldb_queues++; + } + + list_for_each_entry_safe(ldb_queue, tmp_ldb_queue, + &domain->avail_ldb_queues, domain_list) { + ldb_queue->owned = false; + + list_del(&ldb_queue->domain_list); + list_add(&ldb_queue->func_list, &rsrcs->avail_ldb_queues); + rsrcs->num_avail_ldb_queues++; + } + + /* Move the domain's ldb ports to the function's avail list */ + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry_safe(ldb_port, tmp_ldb_port, + &domain->used_ldb_ports[i], domain_list) { + int j; + + ldb_port->owned = false; + ldb_port->configured = false; + ldb_port->num_pending_removals = 0; + ldb_port->num_mappings = 0; + ldb_port->init_tkn_cnt = 0; + for (j = 0; j < DLB_MAX_NUM_QIDS_PER_LDB_CQ; j++) + ldb_port->qid_map[j].state = + DLB_QUEUE_UNMAPPED; + + list_del(&ldb_port->domain_list); + list_add(&ldb_port->func_list, + &rsrcs->avail_ldb_ports[i]); + rsrcs->num_avail_ldb_ports[i]++; + } + + list_for_each_entry_safe(ldb_port, tmp_ldb_port, + &domain->avail_ldb_ports[i], domain_list) { + ldb_port->owned = false; + + list_del(&ldb_port->domain_list); + list_add(&ldb_port->func_list, + &rsrcs->avail_ldb_ports[i]); + rsrcs->num_avail_ldb_ports[i]++; + } + } + + /* Move the domain's dir ports to the function's avail list */ + list_for_each_entry_safe(dir_port, tmp_dir_port, + &domain->used_dir_pq_pairs, domain_list) { + dir_port->owned = false; + dir_port->port_configured = false; + dir_port->init_tkn_cnt = 0; + + list_del(&dir_port->domain_list); + + list_add(&dir_port->func_list, &rsrcs->avail_dir_pq_pairs); + rsrcs->num_avail_dir_pq_pairs++; + } + + list_for_each_entry_safe(dir_port, tmp_dir_port, + &domain->avail_dir_pq_pairs, domain_list) { + dir_port->owned = false; + + list_del(&dir_port->domain_list); + + list_add(&dir_port->func_list, &rsrcs->avail_dir_pq_pairs); + rsrcs->num_avail_dir_pq_pairs++; + } + + /* Return hist list entries to the function */ + ret = dlb_bitmap_set_range(rsrcs->avail_hist_list_entries, + domain->hist_list_entry_base, + domain->total_hist_list_entries); + if (ret) { + DLB_HW_ERR(hw, + "[%s()] Internal error: domain hist list base doesn't match the function's bitmap.\n", + __func__); + return ret; + } + + domain->total_hist_list_entries = 0; + domain->avail_hist_list_entries = 0; + domain->hist_list_entry_base = 0; + domain->hist_list_entry_offset = 0; + + rsrcs->num_avail_qed_entries += domain->num_ldb_credits; + domain->num_ldb_credits = 0; + + rsrcs->num_avail_dqed_entries += domain->num_dir_credits; + domain->num_dir_credits = 0; + + rsrcs->num_avail_aqed_entries += domain->num_avail_aqed_entries; + rsrcs->num_avail_aqed_entries += domain->num_used_aqed_entries; + domain->num_avail_aqed_entries = 0; + domain->num_used_aqed_entries = 0; + + domain->num_pending_removals = 0; + domain->num_pending_additions = 0; + domain->configured = false; + domain->started = false; + + /* + * Move the domain out of the used_domains list and back to the + * function's avail_domains list. + */ + list_del(&domain->func_list); + list_add(&domain->func_list, &rsrcs->avail_domains); + rsrcs->num_avail_domains++; + + return 0; +} + +static void dlb_log_reset_domain(struct dlb_hw *hw, u32 domain_id, + bool vdev_req, unsigned int vdev_id) +{ + DLB_HW_DBG(hw, "DLB reset domain:\n"); + if (vdev_req) + DLB_HW_DBG(hw, "(Request from vdev %d)\n", vdev_id); + DLB_HW_DBG(hw, "\tDomain ID: %d\n", domain_id); +} + +/** + * dlb_reset_domain() - reset a scheduling domain + * @hw: dlb_hw handle for a particular device. + * @domain_id: domain ID. + * @vdev_req: indicates whether this request came from a vdev. + * @vdev_id: If vdev_req is true, this contains the vdev's ID. + * + * This function resets and frees a DLB 2.0 scheduling domain and its associated + * resources. + * + * Pre-condition: the driver must ensure software has stopped sending QEs + * through this domain's producer ports before invoking this function, or + * undefined behavior will result. + * + * A vdev can be either an SR-IOV virtual function or a Scalable IOV virtual + * device. + * + * Return: + * Returns 0 upon success, -1 otherwise. + * + * EINVAL - Invalid domain ID, or the domain is not configured. + * EFAULT - Internal error. (Possibly caused if software is the pre-condition + * is not met.) + * ETIMEDOUT - Hardware component didn't reset in the expected time. + */ +int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, + unsigned int vdev_id) +{ + struct dlb_hw_domain *domain; + + dlb_log_reset_domain(hw, domain_id, vdev_req, vdev_id); + + domain = dlb_get_domain_from_id(hw, domain_id, vdev_req, vdev_id); + + if (!domain || !domain->configured) + return -EINVAL; + + return dlb_domain_reset_software_state(hw, domain); +} + /** * dlb_hw_get_num_resources() - query the PCI function's available resources * @hw: dlb_hw handle for a particular device. diff --git a/drivers/misc/dlb/dlb_resource.h b/drivers/misc/dlb/dlb_resource.h index efc5140970cd..8c50f449cb9b 100644 --- a/drivers/misc/dlb/dlb_resource.h +++ b/drivers/misc/dlb/dlb_resource.h @@ -19,6 +19,9 @@ int dlb_hw_create_sched_domain(struct dlb_hw *hw, struct dlb_cmd_response *resp, bool vdev_req, unsigned int vdev_id); +int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, + unsigned int vdev_id); + int dlb_hw_get_num_resources(struct dlb_hw *hw, struct dlb_get_num_resources_args *arg, bool vdev_req, unsigned int vdev_id); diff --git a/include/uapi/linux/dlb.h b/include/uapi/linux/dlb.h index 0b152d29f9e4..0513116072a7 100644 --- a/include/uapi/linux/dlb.h +++ b/include/uapi/linux/dlb.h @@ -18,6 +18,7 @@ enum dlb_error { DLB_ST_ATOMIC_INFLIGHTS_UNAVAILABLE, DLB_ST_HIST_LIST_ENTRIES_UNAVAILABLE, DLB_ST_LDB_PORT_REQUIRED_FOR_LDB_QUEUES, + DLB_ST_DOMAIN_RESET_FAILED, }; struct dlb_cmd_response { From patchwork Wed Feb 10 17:54:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UPPERCASE_50_75, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A2D6C433DB for ; Wed, 10 Feb 2021 17:58:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD45064EDF for ; Wed, 10 Feb 2021 17:58:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232492AbhBJR6p (ORCPT ); Wed, 10 Feb 2021 12:58:45 -0500 Received: from mga01.intel.com ([192.55.52.88]:60424 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232116AbhBJR5e (ORCPT ); Wed, 10 Feb 2021 12:57:34 -0500 IronPort-SDR: 0rZt/pEYpE3PDa/R1qgZEdM4nLGCtqjMrc958STEoVFQHWvQDVSjhvJO71RqZZkrfhNdoCBCUk SRivRmsQkwSA== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236021" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236021" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:08 -0800 IronPort-SDR: aGD0yomOGK7czxJuNjAWWKTG2Yy6PIL//QYA5yan223melAON5FX9x4Zi0Fqd8FefuBbDVQTN7 rTQt87DEx71Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235744" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:07 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 07/20] dlb: add low-level register reset operations Date: Wed, 10 Feb 2021 11:54:10 -0600 Message-Id: <20210210175423.1873-8-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Program all registers used to configure the domain's resources back to their reset values during scheduling domain reset. This ensures the device is in a known good state if/when it is configured again in the future. Additional work is required if a resource is in-use (e.g. a queue is non-empty) at that time. Support for these cases will be added in subsequent commits. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_regs.h | 3527 ++++++++++++++++++++++++++++++- drivers/misc/dlb/dlb_resource.c | 387 ++++ 2 files changed, 3902 insertions(+), 12 deletions(-) diff --git a/drivers/misc/dlb/dlb_regs.h b/drivers/misc/dlb/dlb_regs.h index 0fd499f384de..5e4609504a01 100644 --- a/drivers/misc/dlb/dlb_regs.h +++ b/drivers/misc/dlb/dlb_regs.h @@ -4,25 +4,3475 @@ #ifndef __DLB_REGS_H #define __DLB_REGS_H -#include +#define PF_VF2PF_MAILBOX_BYTES 256 +#define PF_VF2PF_MAILBOX(vf_id, x) \ + (0x1000 + 0x4 * (x) + (vf_id) * 0x10000) +#define PF_VF2PF_MAILBOX_RST 0x0 + +#define PF_VF2PF_MAILBOX_MSG 0xFFFFFFFF +#define PF_VF2PF_MAILBOX_MSG_LOC 0 + +#define PF_VF2PF_MAILBOX_ISR(vf_id) \ + (0x1f00 + (vf_id) * 0x10000) +#define PF_VF2PF_MAILBOX_ISR_RST 0x0 + +#define PF_VF2PF_MAILBOX_ISR_VF0_ISR 0x00000001 +#define PF_VF2PF_MAILBOX_ISR_VF1_ISR 0x00000002 +#define PF_VF2PF_MAILBOX_ISR_VF2_ISR 0x00000004 +#define PF_VF2PF_MAILBOX_ISR_VF3_ISR 0x00000008 +#define PF_VF2PF_MAILBOX_ISR_VF4_ISR 0x00000010 +#define PF_VF2PF_MAILBOX_ISR_VF5_ISR 0x00000020 +#define PF_VF2PF_MAILBOX_ISR_VF6_ISR 0x00000040 +#define PF_VF2PF_MAILBOX_ISR_VF7_ISR 0x00000080 +#define PF_VF2PF_MAILBOX_ISR_VF8_ISR 0x00000100 +#define PF_VF2PF_MAILBOX_ISR_VF9_ISR 0x00000200 +#define PF_VF2PF_MAILBOX_ISR_VF10_ISR 0x00000400 +#define PF_VF2PF_MAILBOX_ISR_VF11_ISR 0x00000800 +#define PF_VF2PF_MAILBOX_ISR_VF12_ISR 0x00001000 +#define PF_VF2PF_MAILBOX_ISR_VF13_ISR 0x00002000 +#define PF_VF2PF_MAILBOX_ISR_VF14_ISR 0x00004000 +#define PF_VF2PF_MAILBOX_ISR_VF15_ISR 0x00008000 +#define PF_VF2PF_MAILBOX_ISR_RSVD0 0xFFFF0000 +#define PF_VF2PF_MAILBOX_ISR_VF0_ISR_LOC 0 +#define PF_VF2PF_MAILBOX_ISR_VF1_ISR_LOC 1 +#define PF_VF2PF_MAILBOX_ISR_VF2_ISR_LOC 2 +#define PF_VF2PF_MAILBOX_ISR_VF3_ISR_LOC 3 +#define PF_VF2PF_MAILBOX_ISR_VF4_ISR_LOC 4 +#define PF_VF2PF_MAILBOX_ISR_VF5_ISR_LOC 5 +#define PF_VF2PF_MAILBOX_ISR_VF6_ISR_LOC 6 +#define PF_VF2PF_MAILBOX_ISR_VF7_ISR_LOC 7 +#define PF_VF2PF_MAILBOX_ISR_VF8_ISR_LOC 8 +#define PF_VF2PF_MAILBOX_ISR_VF9_ISR_LOC 9 +#define PF_VF2PF_MAILBOX_ISR_VF10_ISR_LOC 10 +#define PF_VF2PF_MAILBOX_ISR_VF11_ISR_LOC 11 +#define PF_VF2PF_MAILBOX_ISR_VF12_ISR_LOC 12 +#define PF_VF2PF_MAILBOX_ISR_VF13_ISR_LOC 13 +#define PF_VF2PF_MAILBOX_ISR_VF14_ISR_LOC 14 +#define PF_VF2PF_MAILBOX_ISR_VF15_ISR_LOC 15 +#define PF_VF2PF_MAILBOX_ISR_RSVD0_LOC 16 + +#define PF_VF2PF_FLR_ISR(vf_id) \ + (0x1f04 + (vf_id) * 0x10000) +#define PF_VF2PF_FLR_ISR_RST 0x0 + +#define PF_VF2PF_FLR_ISR_VF0_ISR 0x00000001 +#define PF_VF2PF_FLR_ISR_VF1_ISR 0x00000002 +#define PF_VF2PF_FLR_ISR_VF2_ISR 0x00000004 +#define PF_VF2PF_FLR_ISR_VF3_ISR 0x00000008 +#define PF_VF2PF_FLR_ISR_VF4_ISR 0x00000010 +#define PF_VF2PF_FLR_ISR_VF5_ISR 0x00000020 +#define PF_VF2PF_FLR_ISR_VF6_ISR 0x00000040 +#define PF_VF2PF_FLR_ISR_VF7_ISR 0x00000080 +#define PF_VF2PF_FLR_ISR_VF8_ISR 0x00000100 +#define PF_VF2PF_FLR_ISR_VF9_ISR 0x00000200 +#define PF_VF2PF_FLR_ISR_VF10_ISR 0x00000400 +#define PF_VF2PF_FLR_ISR_VF11_ISR 0x00000800 +#define PF_VF2PF_FLR_ISR_VF12_ISR 0x00001000 +#define PF_VF2PF_FLR_ISR_VF13_ISR 0x00002000 +#define PF_VF2PF_FLR_ISR_VF14_ISR 0x00004000 +#define PF_VF2PF_FLR_ISR_VF15_ISR 0x00008000 +#define PF_VF2PF_FLR_ISR_RSVD0 0xFFFF0000 +#define PF_VF2PF_FLR_ISR_VF0_ISR_LOC 0 +#define PF_VF2PF_FLR_ISR_VF1_ISR_LOC 1 +#define PF_VF2PF_FLR_ISR_VF2_ISR_LOC 2 +#define PF_VF2PF_FLR_ISR_VF3_ISR_LOC 3 +#define PF_VF2PF_FLR_ISR_VF4_ISR_LOC 4 +#define PF_VF2PF_FLR_ISR_VF5_ISR_LOC 5 +#define PF_VF2PF_FLR_ISR_VF6_ISR_LOC 6 +#define PF_VF2PF_FLR_ISR_VF7_ISR_LOC 7 +#define PF_VF2PF_FLR_ISR_VF8_ISR_LOC 8 +#define PF_VF2PF_FLR_ISR_VF9_ISR_LOC 9 +#define PF_VF2PF_FLR_ISR_VF10_ISR_LOC 10 +#define PF_VF2PF_FLR_ISR_VF11_ISR_LOC 11 +#define PF_VF2PF_FLR_ISR_VF12_ISR_LOC 12 +#define PF_VF2PF_FLR_ISR_VF13_ISR_LOC 13 +#define PF_VF2PF_FLR_ISR_VF14_ISR_LOC 14 +#define PF_VF2PF_FLR_ISR_VF15_ISR_LOC 15 +#define PF_VF2PF_FLR_ISR_RSVD0_LOC 16 + +#define PF_VF2PF_ISR_PEND(vf_id) \ + (0x1f10 + (vf_id) * 0x10000) +#define PF_VF2PF_ISR_PEND_RST 0x0 + +#define PF_VF2PF_ISR_PEND_ISR_PEND 0x00000001 +#define PF_VF2PF_ISR_PEND_RSVD0 0xFFFFFFFE +#define PF_VF2PF_ISR_PEND_ISR_PEND_LOC 0 +#define PF_VF2PF_ISR_PEND_RSVD0_LOC 1 + +#define PF_PF2VF_MAILBOX_BYTES 64 +#define PF_PF2VF_MAILBOX(vf_id, x) \ + (0x2000 + 0x4 * (x) + (vf_id) * 0x10000) +#define PF_PF2VF_MAILBOX_RST 0x0 + +#define PF_PF2VF_MAILBOX_MSG 0xFFFFFFFF +#define PF_PF2VF_MAILBOX_MSG_LOC 0 + +#define PF_PF2VF_MAILBOX_ISR(vf_id) \ + (0x2f00 + (vf_id) * 0x10000) +#define PF_PF2VF_MAILBOX_ISR_RST 0x0 + +#define PF_PF2VF_MAILBOX_ISR_VF0_ISR 0x00000001 +#define PF_PF2VF_MAILBOX_ISR_VF1_ISR 0x00000002 +#define PF_PF2VF_MAILBOX_ISR_VF2_ISR 0x00000004 +#define PF_PF2VF_MAILBOX_ISR_VF3_ISR 0x00000008 +#define PF_PF2VF_MAILBOX_ISR_VF4_ISR 0x00000010 +#define PF_PF2VF_MAILBOX_ISR_VF5_ISR 0x00000020 +#define PF_PF2VF_MAILBOX_ISR_VF6_ISR 0x00000040 +#define PF_PF2VF_MAILBOX_ISR_VF7_ISR 0x00000080 +#define PF_PF2VF_MAILBOX_ISR_VF8_ISR 0x00000100 +#define PF_PF2VF_MAILBOX_ISR_VF9_ISR 0x00000200 +#define PF_PF2VF_MAILBOX_ISR_VF10_ISR 0x00000400 +#define PF_PF2VF_MAILBOX_ISR_VF11_ISR 0x00000800 +#define PF_PF2VF_MAILBOX_ISR_VF12_ISR 0x00001000 +#define PF_PF2VF_MAILBOX_ISR_VF13_ISR 0x00002000 +#define PF_PF2VF_MAILBOX_ISR_VF14_ISR 0x00004000 +#define PF_PF2VF_MAILBOX_ISR_VF15_ISR 0x00008000 +#define PF_PF2VF_MAILBOX_ISR_RSVD0 0xFFFF0000 +#define PF_PF2VF_MAILBOX_ISR_VF0_ISR_LOC 0 +#define PF_PF2VF_MAILBOX_ISR_VF1_ISR_LOC 1 +#define PF_PF2VF_MAILBOX_ISR_VF2_ISR_LOC 2 +#define PF_PF2VF_MAILBOX_ISR_VF3_ISR_LOC 3 +#define PF_PF2VF_MAILBOX_ISR_VF4_ISR_LOC 4 +#define PF_PF2VF_MAILBOX_ISR_VF5_ISR_LOC 5 +#define PF_PF2VF_MAILBOX_ISR_VF6_ISR_LOC 6 +#define PF_PF2VF_MAILBOX_ISR_VF7_ISR_LOC 7 +#define PF_PF2VF_MAILBOX_ISR_VF8_ISR_LOC 8 +#define PF_PF2VF_MAILBOX_ISR_VF9_ISR_LOC 9 +#define PF_PF2VF_MAILBOX_ISR_VF10_ISR_LOC 10 +#define PF_PF2VF_MAILBOX_ISR_VF11_ISR_LOC 11 +#define PF_PF2VF_MAILBOX_ISR_VF12_ISR_LOC 12 +#define PF_PF2VF_MAILBOX_ISR_VF13_ISR_LOC 13 +#define PF_PF2VF_MAILBOX_ISR_VF14_ISR_LOC 14 +#define PF_PF2VF_MAILBOX_ISR_VF15_ISR_LOC 15 +#define PF_PF2VF_MAILBOX_ISR_RSVD0_LOC 16 + +#define PF_VF_RESET_IN_PROGRESS(vf_id) \ + (0x3000 + (vf_id) * 0x10000) +#define PF_VF_RESET_IN_PROGRESS_RST 0xffff + +#define PF_VF_RESET_IN_PROGRESS_VF0_RESET_IN_PROGRESS 0x00000001 +#define PF_VF_RESET_IN_PROGRESS_VF1_RESET_IN_PROGRESS 0x00000002 +#define PF_VF_RESET_IN_PROGRESS_VF2_RESET_IN_PROGRESS 0x00000004 +#define PF_VF_RESET_IN_PROGRESS_VF3_RESET_IN_PROGRESS 0x00000008 +#define PF_VF_RESET_IN_PROGRESS_VF4_RESET_IN_PROGRESS 0x00000010 +#define PF_VF_RESET_IN_PROGRESS_VF5_RESET_IN_PROGRESS 0x00000020 +#define PF_VF_RESET_IN_PROGRESS_VF6_RESET_IN_PROGRESS 0x00000040 +#define PF_VF_RESET_IN_PROGRESS_VF7_RESET_IN_PROGRESS 0x00000080 +#define PF_VF_RESET_IN_PROGRESS_VF8_RESET_IN_PROGRESS 0x00000100 +#define PF_VF_RESET_IN_PROGRESS_VF9_RESET_IN_PROGRESS 0x00000200 +#define PF_VF_RESET_IN_PROGRESS_VF10_RESET_IN_PROGRESS 0x00000400 +#define PF_VF_RESET_IN_PROGRESS_VF11_RESET_IN_PROGRESS 0x00000800 +#define PF_VF_RESET_IN_PROGRESS_VF12_RESET_IN_PROGRESS 0x00001000 +#define PF_VF_RESET_IN_PROGRESS_VF13_RESET_IN_PROGRESS 0x00002000 +#define PF_VF_RESET_IN_PROGRESS_VF14_RESET_IN_PROGRESS 0x00004000 +#define PF_VF_RESET_IN_PROGRESS_VF15_RESET_IN_PROGRESS 0x00008000 +#define PF_VF_RESET_IN_PROGRESS_RSVD0 0xFFFF0000 +#define PF_VF_RESET_IN_PROGRESS_VF0_RESET_IN_PROGRESS_LOC 0 +#define PF_VF_RESET_IN_PROGRESS_VF1_RESET_IN_PROGRESS_LOC 1 +#define PF_VF_RESET_IN_PROGRESS_VF2_RESET_IN_PROGRESS_LOC 2 +#define PF_VF_RESET_IN_PROGRESS_VF3_RESET_IN_PROGRESS_LOC 3 +#define PF_VF_RESET_IN_PROGRESS_VF4_RESET_IN_PROGRESS_LOC 4 +#define PF_VF_RESET_IN_PROGRESS_VF5_RESET_IN_PROGRESS_LOC 5 +#define PF_VF_RESET_IN_PROGRESS_VF6_RESET_IN_PROGRESS_LOC 6 +#define PF_VF_RESET_IN_PROGRESS_VF7_RESET_IN_PROGRESS_LOC 7 +#define PF_VF_RESET_IN_PROGRESS_VF8_RESET_IN_PROGRESS_LOC 8 +#define PF_VF_RESET_IN_PROGRESS_VF9_RESET_IN_PROGRESS_LOC 9 +#define PF_VF_RESET_IN_PROGRESS_VF10_RESET_IN_PROGRESS_LOC 10 +#define PF_VF_RESET_IN_PROGRESS_VF11_RESET_IN_PROGRESS_LOC 11 +#define PF_VF_RESET_IN_PROGRESS_VF12_RESET_IN_PROGRESS_LOC 12 +#define PF_VF_RESET_IN_PROGRESS_VF13_RESET_IN_PROGRESS_LOC 13 +#define PF_VF_RESET_IN_PROGRESS_VF14_RESET_IN_PROGRESS_LOC 14 +#define PF_VF_RESET_IN_PROGRESS_VF15_RESET_IN_PROGRESS_LOC 15 +#define PF_VF_RESET_IN_PROGRESS_RSVD0_LOC 16 + +#define MSIX_VECTOR_CTRL(x) \ + (0x100000c + (x) * 0x10) +#define MSIX_VECTOR_CTRL_RST 0x1 + +#define MSIX_VECTOR_CTRL_VEC_MASK 0x00000001 +#define MSIX_VECTOR_CTRL_RSVD0 0xFFFFFFFE +#define MSIX_VECTOR_CTRL_VEC_MASK_LOC 0 +#define MSIX_VECTOR_CTRL_RSVD0_LOC 1 + +#define IOSF_SMON_COMP_MASK1(x) \ + (0x8002024 + (x) * 0x40) +#define IOSF_SMON_COMP_MASK1_RST 0xffffffff + +#define IOSF_SMON_COMP_MASK1_COMP_MASK1 0xFFFFFFFF +#define IOSF_SMON_COMP_MASK1_COMP_MASK1_LOC 0 + +#define IOSF_SMON_COMP_MASK0(x) \ + (0x8002020 + (x) * 0x40) +#define IOSF_SMON_COMP_MASK0_RST 0xffffffff + +#define IOSF_SMON_COMP_MASK0_COMP_MASK0 0xFFFFFFFF +#define IOSF_SMON_COMP_MASK0_COMP_MASK0_LOC 0 + +#define IOSF_SMON_MAX_TMR(x) \ + (0x800201c + (x) * 0x40) +#define IOSF_SMON_MAX_TMR_RST 0x0 + +#define IOSF_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define IOSF_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define IOSF_SMON_TMR(x) \ + (0x8002018 + (x) * 0x40) +#define IOSF_SMON_TMR_RST 0x0 + +#define IOSF_SMON_TMR_TIMER_VAL 0xFFFFFFFF +#define IOSF_SMON_TMR_TIMER_VAL_LOC 0 + +#define IOSF_SMON_ACTIVITYCNTR1(x) \ + (0x8002014 + (x) * 0x40) +#define IOSF_SMON_ACTIVITYCNTR1_RST 0x0 + +#define IOSF_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define IOSF_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define IOSF_SMON_ACTIVITYCNTR0(x) \ + (0x8002010 + (x) * 0x40) +#define IOSF_SMON_ACTIVITYCNTR0_RST 0x0 + +#define IOSF_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define IOSF_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define IOSF_SMON_COMPARE1(x) \ + (0x800200c + (x) * 0x40) +#define IOSF_SMON_COMPARE1_RST 0x0 + +#define IOSF_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define IOSF_SMON_COMPARE1_COMPARE1_LOC 0 + +#define IOSF_SMON_COMPARE0(x) \ + (0x8002008 + (x) * 0x40) +#define IOSF_SMON_COMPARE0_RST 0x0 + +#define IOSF_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define IOSF_SMON_COMPARE0_COMPARE0_LOC 0 + +#define IOSF_SMON_CFG1(x) \ + (0x8002004 + (x) * 0x40) +#define IOSF_SMON_CFG1_RST 0x0 + +#define IOSF_SMON_CFG1_MODE0 0x000000FF +#define IOSF_SMON_CFG1_MODE1 0x0000FF00 +#define IOSF_SMON_CFG1_RSVD 0xFFFF0000 +#define IOSF_SMON_CFG1_MODE0_LOC 0 +#define IOSF_SMON_CFG1_MODE1_LOC 8 +#define IOSF_SMON_CFG1_RSVD_LOC 16 + +#define IOSF_SMON_CFG0(x) \ + (0x8002000 + (x) * 0x40) +#define IOSF_SMON_CFG0_RST 0x40000000 + +#define IOSF_SMON_CFG0_SMON_ENABLE 0x00000001 +#define IOSF_SMON_CFG0_RSVD2 0x0000000E +#define IOSF_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define IOSF_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define IOSF_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define IOSF_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define IOSF_SMON_CFG0_SMON_MODE 0x0000F000 +#define IOSF_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define IOSF_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define IOSF_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define IOSF_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define IOSF_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define IOSF_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define IOSF_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define IOSF_SMON_CFG0_RSVD1 0x00800000 +#define IOSF_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define IOSF_SMON_CFG0_RSVD0 0x20000000 +#define IOSF_SMON_CFG0_VERSION 0xC0000000 +#define IOSF_SMON_CFG0_SMON_ENABLE_LOC 0 +#define IOSF_SMON_CFG0_RSVD2_LOC 1 +#define IOSF_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define IOSF_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define IOSF_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define IOSF_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define IOSF_SMON_CFG0_SMON_MODE_LOC 12 +#define IOSF_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define IOSF_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define IOSF_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define IOSF_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define IOSF_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define IOSF_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define IOSF_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define IOSF_SMON_CFG0_RSVD1_LOC 23 +#define IOSF_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define IOSF_SMON_CFG0_RSVD0_LOC 29 +#define IOSF_SMON_CFG0_VERSION_LOC 30 + +#define IOSF_FUNC_VF_BAR_DSBL(x) \ + (0x20 + (x) * 0x4) +#define IOSF_FUNC_VF_BAR_DSBL_RST 0x0 + +#define IOSF_FUNC_VF_BAR_DSBL_FUNC_VF_BAR_DIS 0x00000001 +#define IOSF_FUNC_VF_BAR_DSBL_RSVD0 0xFFFFFFFE +#define IOSF_FUNC_VF_BAR_DSBL_FUNC_VF_BAR_DIS_LOC 0 +#define IOSF_FUNC_VF_BAR_DSBL_RSVD0_LOC 1 + +#define SYS_TOTAL_VAS 0x1000011c +#define SYS_TOTAL_VAS_RST 0x20 + +#define SYS_TOTAL_VAS_TOTAL_VAS 0xFFFFFFFF +#define SYS_TOTAL_VAS_TOTAL_VAS_LOC 0 + +#define SYS_TOTAL_DIR_PORTS 0x10000118 +#define SYS_TOTAL_DIR_PORTS_RST 0x40 + +#define SYS_TOTAL_DIR_PORTS_TOTAL_DIR_PORTS 0xFFFFFFFF +#define SYS_TOTAL_DIR_PORTS_TOTAL_DIR_PORTS_LOC 0 + +#define SYS_TOTAL_LDB_PORTS 0x10000114 +#define SYS_TOTAL_LDB_PORTS_RST 0x40 + +#define SYS_TOTAL_LDB_PORTS_TOTAL_LDB_PORTS 0xFFFFFFFF +#define SYS_TOTAL_LDB_PORTS_TOTAL_LDB_PORTS_LOC 0 + +#define SYS_TOTAL_DIR_QID 0x10000110 +#define SYS_TOTAL_DIR_QID_RST 0x40 + +#define SYS_TOTAL_DIR_QID_TOTAL_DIR_QID 0xFFFFFFFF +#define SYS_TOTAL_DIR_QID_TOTAL_DIR_QID_LOC 0 + +#define SYS_TOTAL_LDB_QID 0x1000010c +#define SYS_TOTAL_LDB_QID_RST 0x20 + +#define SYS_TOTAL_LDB_QID_TOTAL_LDB_QID 0xFFFFFFFF +#define SYS_TOTAL_LDB_QID_TOTAL_LDB_QID_LOC 0 + +#define SYS_TOTAL_DIR_CRDS 0x10000108 +#define SYS_TOTAL_DIR_CRDS_RST 0x1000 + +#define SYS_TOTAL_DIR_CRDS_TOTAL_DIR_CREDITS 0xFFFFFFFF +#define SYS_TOTAL_DIR_CRDS_TOTAL_DIR_CREDITS_LOC 0 + +#define SYS_TOTAL_LDB_CRDS 0x10000104 +#define SYS_TOTAL_LDB_CRDS_RST 0x2000 + +#define SYS_TOTAL_LDB_CRDS_TOTAL_LDB_CREDITS 0xFFFFFFFF +#define SYS_TOTAL_LDB_CRDS_TOTAL_LDB_CREDITS_LOC 0 + +#define SYS_ALARM_PF_SYND2 0x10000508 +#define SYS_ALARM_PF_SYND2_RST 0x0 + +#define SYS_ALARM_PF_SYND2_LOCK_ID 0x0000FFFF +#define SYS_ALARM_PF_SYND2_MEAS 0x00010000 +#define SYS_ALARM_PF_SYND2_DEBUG 0x00FE0000 +#define SYS_ALARM_PF_SYND2_CQ_POP 0x01000000 +#define SYS_ALARM_PF_SYND2_QE_UHL 0x02000000 +#define SYS_ALARM_PF_SYND2_QE_ORSP 0x04000000 +#define SYS_ALARM_PF_SYND2_QE_VALID 0x08000000 +#define SYS_ALARM_PF_SYND2_CQ_INT_REARM 0x10000000 +#define SYS_ALARM_PF_SYND2_DSI_ERROR 0x20000000 +#define SYS_ALARM_PF_SYND2_RSVD0 0xC0000000 +#define SYS_ALARM_PF_SYND2_LOCK_ID_LOC 0 +#define SYS_ALARM_PF_SYND2_MEAS_LOC 16 +#define SYS_ALARM_PF_SYND2_DEBUG_LOC 17 +#define SYS_ALARM_PF_SYND2_CQ_POP_LOC 24 +#define SYS_ALARM_PF_SYND2_QE_UHL_LOC 25 +#define SYS_ALARM_PF_SYND2_QE_ORSP_LOC 26 +#define SYS_ALARM_PF_SYND2_QE_VALID_LOC 27 +#define SYS_ALARM_PF_SYND2_CQ_INT_REARM_LOC 28 +#define SYS_ALARM_PF_SYND2_DSI_ERROR_LOC 29 +#define SYS_ALARM_PF_SYND2_RSVD0_LOC 30 + +#define SYS_ALARM_PF_SYND1 0x10000504 +#define SYS_ALARM_PF_SYND1_RST 0x0 + +#define SYS_ALARM_PF_SYND1_DSI 0x0000FFFF +#define SYS_ALARM_PF_SYND1_QID 0x00FF0000 +#define SYS_ALARM_PF_SYND1_QTYPE 0x03000000 +#define SYS_ALARM_PF_SYND1_QPRI 0x1C000000 +#define SYS_ALARM_PF_SYND1_MSG_TYPE 0xE0000000 +#define SYS_ALARM_PF_SYND1_DSI_LOC 0 +#define SYS_ALARM_PF_SYND1_QID_LOC 16 +#define SYS_ALARM_PF_SYND1_QTYPE_LOC 24 +#define SYS_ALARM_PF_SYND1_QPRI_LOC 26 +#define SYS_ALARM_PF_SYND1_MSG_TYPE_LOC 29 + +#define SYS_ALARM_PF_SYND0 0x10000500 +#define SYS_ALARM_PF_SYND0_RST 0x0 + +#define SYS_ALARM_PF_SYND0_SYNDROME 0x000000FF +#define SYS_ALARM_PF_SYND0_RTYPE 0x00000300 +#define SYS_ALARM_PF_SYND0_RSVD0 0x00001C00 +#define SYS_ALARM_PF_SYND0_IS_LDB 0x00002000 +#define SYS_ALARM_PF_SYND0_CLS 0x0000C000 +#define SYS_ALARM_PF_SYND0_AID 0x003F0000 +#define SYS_ALARM_PF_SYND0_UNIT 0x03C00000 +#define SYS_ALARM_PF_SYND0_SOURCE 0x3C000000 +#define SYS_ALARM_PF_SYND0_MORE 0x40000000 +#define SYS_ALARM_PF_SYND0_VALID 0x80000000 +#define SYS_ALARM_PF_SYND0_SYNDROME_LOC 0 +#define SYS_ALARM_PF_SYND0_RTYPE_LOC 8 +#define SYS_ALARM_PF_SYND0_RSVD0_LOC 10 +#define SYS_ALARM_PF_SYND0_IS_LDB_LOC 13 +#define SYS_ALARM_PF_SYND0_CLS_LOC 14 +#define SYS_ALARM_PF_SYND0_AID_LOC 16 +#define SYS_ALARM_PF_SYND0_UNIT_LOC 22 +#define SYS_ALARM_PF_SYND0_SOURCE_LOC 26 +#define SYS_ALARM_PF_SYND0_MORE_LOC 30 +#define SYS_ALARM_PF_SYND0_VALID_LOC 31 + +#define SYS_VF_LDB_VPP_V(x) \ + (0x10000f00 + (x) * 0x1000) +#define SYS_VF_LDB_VPP_V_RST 0x0 + +#define SYS_VF_LDB_VPP_V_VPP_V 0x00000001 +#define SYS_VF_LDB_VPP_V_RSVD0 0xFFFFFFFE +#define SYS_VF_LDB_VPP_V_VPP_V_LOC 0 +#define SYS_VF_LDB_VPP_V_RSVD0_LOC 1 + +#define SYS_VF_LDB_VPP2PP(x) \ + (0x10000f04 + (x) * 0x1000) +#define SYS_VF_LDB_VPP2PP_RST 0x0 + +#define SYS_VF_LDB_VPP2PP_PP 0x0000003F +#define SYS_VF_LDB_VPP2PP_RSVD0 0xFFFFFFC0 +#define SYS_VF_LDB_VPP2PP_PP_LOC 0 +#define SYS_VF_LDB_VPP2PP_RSVD0_LOC 6 + +#define SYS_VF_DIR_VPP_V(x) \ + (0x10000f08 + (x) * 0x1000) +#define SYS_VF_DIR_VPP_V_RST 0x0 + +#define SYS_VF_DIR_VPP_V_VPP_V 0x00000001 +#define SYS_VF_DIR_VPP_V_RSVD0 0xFFFFFFFE +#define SYS_VF_DIR_VPP_V_VPP_V_LOC 0 +#define SYS_VF_DIR_VPP_V_RSVD0_LOC 1 + +#define SYS_VF_DIR_VPP2PP(x) \ + (0x10000f0c + (x) * 0x1000) +#define SYS_VF_DIR_VPP2PP_RST 0x0 + +#define SYS_VF_DIR_VPP2PP_PP 0x0000003F +#define SYS_VF_DIR_VPP2PP_RSVD0 0xFFFFFFC0 +#define SYS_VF_DIR_VPP2PP_PP_LOC 0 +#define SYS_VF_DIR_VPP2PP_RSVD0_LOC 6 + +#define SYS_VF_LDB_VQID_V(x) \ + (0x10000f10 + (x) * 0x1000) +#define SYS_VF_LDB_VQID_V_RST 0x0 + +#define SYS_VF_LDB_VQID_V_VQID_V 0x00000001 +#define SYS_VF_LDB_VQID_V_RSVD0 0xFFFFFFFE +#define SYS_VF_LDB_VQID_V_VQID_V_LOC 0 +#define SYS_VF_LDB_VQID_V_RSVD0_LOC 1 + +#define SYS_VF_LDB_VQID2QID(x) \ + (0x10000f14 + (x) * 0x1000) +#define SYS_VF_LDB_VQID2QID_RST 0x0 + +#define SYS_VF_LDB_VQID2QID_QID 0x0000001F +#define SYS_VF_LDB_VQID2QID_RSVD0 0xFFFFFFE0 +#define SYS_VF_LDB_VQID2QID_QID_LOC 0 +#define SYS_VF_LDB_VQID2QID_RSVD0_LOC 5 + +#define SYS_LDB_QID2VQID(x) \ + (0x10000f18 + (x) * 0x1000) +#define SYS_LDB_QID2VQID_RST 0x0 + +#define SYS_LDB_QID2VQID_VQID 0x0000001F +#define SYS_LDB_QID2VQID_RSVD0 0xFFFFFFE0 +#define SYS_LDB_QID2VQID_VQID_LOC 0 +#define SYS_LDB_QID2VQID_RSVD0_LOC 5 + +#define SYS_VF_DIR_VQID_V(x) \ + (0x10000f1c + (x) * 0x1000) +#define SYS_VF_DIR_VQID_V_RST 0x0 + +#define SYS_VF_DIR_VQID_V_VQID_V 0x00000001 +#define SYS_VF_DIR_VQID_V_RSVD0 0xFFFFFFFE +#define SYS_VF_DIR_VQID_V_VQID_V_LOC 0 +#define SYS_VF_DIR_VQID_V_RSVD0_LOC 1 + +#define SYS_VF_DIR_VQID2QID(x) \ + (0x10000f20 + (x) * 0x1000) +#define SYS_VF_DIR_VQID2QID_RST 0x0 + +#define SYS_VF_DIR_VQID2QID_QID 0x0000003F +#define SYS_VF_DIR_VQID2QID_RSVD0 0xFFFFFFC0 +#define SYS_VF_DIR_VQID2QID_QID_LOC 0 +#define SYS_VF_DIR_VQID2QID_RSVD0_LOC 6 + +#define SYS_LDB_VASQID_V(x) \ + (0x10000f24 + (x) * 0x1000) +#define SYS_LDB_VASQID_V_RST 0x0 + +#define SYS_LDB_VASQID_V_VASQID_V 0x00000001 +#define SYS_LDB_VASQID_V_RSVD0 0xFFFFFFFE +#define SYS_LDB_VASQID_V_VASQID_V_LOC 0 +#define SYS_LDB_VASQID_V_RSVD0_LOC 1 + +#define SYS_DIR_VASQID_V(x) \ + (0x10000f28 + (x) * 0x1000) +#define SYS_DIR_VASQID_V_RST 0x0 + +#define SYS_DIR_VASQID_V_VASQID_V 0x00000001 +#define SYS_DIR_VASQID_V_RSVD0 0xFFFFFFFE +#define SYS_DIR_VASQID_V_VASQID_V_LOC 0 +#define SYS_DIR_VASQID_V_RSVD0_LOC 1 + +#define SYS_ALARM_VF_SYND2(x) \ + (0x10000f48 + (x) * 0x1000) +#define SYS_ALARM_VF_SYND2_RST 0x0 + +#define SYS_ALARM_VF_SYND2_LOCK_ID 0x0000FFFF +#define SYS_ALARM_VF_SYND2_DEBUG 0x00FF0000 +#define SYS_ALARM_VF_SYND2_CQ_POP 0x01000000 +#define SYS_ALARM_VF_SYND2_QE_UHL 0x02000000 +#define SYS_ALARM_VF_SYND2_QE_ORSP 0x04000000 +#define SYS_ALARM_VF_SYND2_QE_VALID 0x08000000 +#define SYS_ALARM_VF_SYND2_ISZ 0x10000000 +#define SYS_ALARM_VF_SYND2_DSI_ERROR 0x20000000 +#define SYS_ALARM_VF_SYND2_DLBRSVD 0xC0000000 +#define SYS_ALARM_VF_SYND2_LOCK_ID_LOC 0 +#define SYS_ALARM_VF_SYND2_DEBUG_LOC 16 +#define SYS_ALARM_VF_SYND2_CQ_POP_LOC 24 +#define SYS_ALARM_VF_SYND2_QE_UHL_LOC 25 +#define SYS_ALARM_VF_SYND2_QE_ORSP_LOC 26 +#define SYS_ALARM_VF_SYND2_QE_VALID_LOC 27 +#define SYS_ALARM_VF_SYND2_ISZ_LOC 28 +#define SYS_ALARM_VF_SYND2_DSI_ERROR_LOC 29 +#define SYS_ALARM_VF_SYND2_DLBRSVD_LOC 30 + +#define SYS_ALARM_VF_SYND1(x) \ + (0x10000f44 + (x) * 0x1000) +#define SYS_ALARM_VF_SYND1_RST 0x0 + +#define SYS_ALARM_VF_SYND1_DSI 0x0000FFFF +#define SYS_ALARM_VF_SYND1_QID 0x00FF0000 +#define SYS_ALARM_VF_SYND1_QTYPE 0x03000000 +#define SYS_ALARM_VF_SYND1_QPRI 0x1C000000 +#define SYS_ALARM_VF_SYND1_MSG_TYPE 0xE0000000 +#define SYS_ALARM_VF_SYND1_DSI_LOC 0 +#define SYS_ALARM_VF_SYND1_QID_LOC 16 +#define SYS_ALARM_VF_SYND1_QTYPE_LOC 24 +#define SYS_ALARM_VF_SYND1_QPRI_LOC 26 +#define SYS_ALARM_VF_SYND1_MSG_TYPE_LOC 29 + +#define SYS_ALARM_VF_SYND0(x) \ + (0x10000f40 + (x) * 0x1000) +#define SYS_ALARM_VF_SYND0_RST 0x0 + +#define SYS_ALARM_VF_SYND0_SYNDROME 0x000000FF +#define SYS_ALARM_VF_SYND0_RTYPE 0x00000300 +#define SYS_ALARM_VF_SYND0_VF_SYND0_PARITY 0x00000400 +#define SYS_ALARM_VF_SYND0_VF_SYND1_PARITY 0x00000800 +#define SYS_ALARM_VF_SYND0_VF_SYND2_PARITY 0x00001000 +#define SYS_ALARM_VF_SYND0_IS_LDB 0x00002000 +#define SYS_ALARM_VF_SYND0_CLS 0x0000C000 +#define SYS_ALARM_VF_SYND0_AID 0x003F0000 +#define SYS_ALARM_VF_SYND0_UNIT 0x03C00000 +#define SYS_ALARM_VF_SYND0_SOURCE 0x3C000000 +#define SYS_ALARM_VF_SYND0_MORE 0x40000000 +#define SYS_ALARM_VF_SYND0_VALID 0x80000000 +#define SYS_ALARM_VF_SYND0_SYNDROME_LOC 0 +#define SYS_ALARM_VF_SYND0_RTYPE_LOC 8 +#define SYS_ALARM_VF_SYND0_VF_SYND0_PARITY_LOC 10 +#define SYS_ALARM_VF_SYND0_VF_SYND1_PARITY_LOC 11 +#define SYS_ALARM_VF_SYND0_VF_SYND2_PARITY_LOC 12 +#define SYS_ALARM_VF_SYND0_IS_LDB_LOC 13 +#define SYS_ALARM_VF_SYND0_CLS_LOC 14 +#define SYS_ALARM_VF_SYND0_AID_LOC 16 +#define SYS_ALARM_VF_SYND0_UNIT_LOC 22 +#define SYS_ALARM_VF_SYND0_SOURCE_LOC 26 +#define SYS_ALARM_VF_SYND0_MORE_LOC 30 +#define SYS_ALARM_VF_SYND0_VALID_LOC 31 + +#define SYS_LDB_QID_CFG_V(x) \ + (0x10000f58 + (x) * 0x1000) +#define SYS_LDB_QID_CFG_V_RST 0x0 + +#define SYS_LDB_QID_CFG_V_SN_CFG_V 0x00000001 +#define SYS_LDB_QID_CFG_V_FID_CFG_V 0x00000002 +#define SYS_LDB_QID_CFG_V_RSVD0 0xFFFFFFFC +#define SYS_LDB_QID_CFG_V_SN_CFG_V_LOC 0 +#define SYS_LDB_QID_CFG_V_FID_CFG_V_LOC 1 +#define SYS_LDB_QID_CFG_V_RSVD0_LOC 2 + +#define SYS_LDB_QID_ITS(x) \ + (0x10000f54 + (x) * 0x1000) +#define SYS_LDB_QID_ITS_RST 0x0 + +#define SYS_LDB_QID_ITS_QID_ITS 0x00000001 +#define SYS_LDB_QID_ITS_RSVD0 0xFFFFFFFE +#define SYS_LDB_QID_ITS_QID_ITS_LOC 0 +#define SYS_LDB_QID_ITS_RSVD0_LOC 1 + +#define SYS_LDB_QID_V(x) \ + (0x10000f50 + (x) * 0x1000) +#define SYS_LDB_QID_V_RST 0x0 + +#define SYS_LDB_QID_V_QID_V 0x00000001 +#define SYS_LDB_QID_V_RSVD0 0xFFFFFFFE +#define SYS_LDB_QID_V_QID_V_LOC 0 +#define SYS_LDB_QID_V_RSVD0_LOC 1 + +#define SYS_DIR_QID_ITS(x) \ + (0x10000f64 + (x) * 0x1000) +#define SYS_DIR_QID_ITS_RST 0x0 + +#define SYS_DIR_QID_ITS_QID_ITS 0x00000001 +#define SYS_DIR_QID_ITS_RSVD0 0xFFFFFFFE +#define SYS_DIR_QID_ITS_QID_ITS_LOC 0 +#define SYS_DIR_QID_ITS_RSVD0_LOC 1 + +#define SYS_DIR_QID_V(x) \ + (0x10000f60 + (x) * 0x1000) +#define SYS_DIR_QID_V_RST 0x0 + +#define SYS_DIR_QID_V_QID_V 0x00000001 +#define SYS_DIR_QID_V_RSVD0 0xFFFFFFFE +#define SYS_DIR_QID_V_QID_V_LOC 0 +#define SYS_DIR_QID_V_RSVD0_LOC 1 + +#define SYS_LDB_CQ_AI_DATA(x) \ + (0x10000fa8 + (x) * 0x1000) +#define SYS_LDB_CQ_AI_DATA_RST 0x0 + +#define SYS_LDB_CQ_AI_DATA_CQ_AI_DATA 0xFFFFFFFF +#define SYS_LDB_CQ_AI_DATA_CQ_AI_DATA_LOC 0 + +#define SYS_LDB_CQ_AI_ADDR(x) \ + (0x10000fa4 + (x) * 0x1000) +#define SYS_LDB_CQ_AI_ADDR_RST 0x0 + +#define SYS_LDB_CQ_AI_ADDR_RSVD1 0x00000003 +#define SYS_LDB_CQ_AI_ADDR_CQ_AI_ADDR 0x000FFFFC +#define SYS_LDB_CQ_AI_ADDR_RSVD0 0xFFF00000 +#define SYS_LDB_CQ_AI_ADDR_RSVD1_LOC 0 +#define SYS_LDB_CQ_AI_ADDR_CQ_AI_ADDR_LOC 2 +#define SYS_LDB_CQ_AI_ADDR_RSVD0_LOC 20 + +#define SYS_LDB_CQ_PASID(x) \ + (0x10000fa0 + (x) * 0x1000) +#define SYS_LDB_CQ_PASID_RST 0x0 + +#define SYS_LDB_CQ_PASID_PASID 0x000FFFFF +#define SYS_LDB_CQ_PASID_EXE_REQ 0x00100000 +#define SYS_LDB_CQ_PASID_PRIV_REQ 0x00200000 +#define SYS_LDB_CQ_PASID_FMT2 0x00400000 +#define SYS_LDB_CQ_PASID_RSVD0 0xFF800000 +#define SYS_LDB_CQ_PASID_PASID_LOC 0 +#define SYS_LDB_CQ_PASID_EXE_REQ_LOC 20 +#define SYS_LDB_CQ_PASID_PRIV_REQ_LOC 21 +#define SYS_LDB_CQ_PASID_FMT2_LOC 22 +#define SYS_LDB_CQ_PASID_RSVD0_LOC 23 + +#define SYS_LDB_CQ_AT(x) \ + (0x10000f9c + (x) * 0x1000) +#define SYS_LDB_CQ_AT_RST 0x0 + +#define SYS_LDB_CQ_AT_CQ_AT 0x00000003 +#define SYS_LDB_CQ_AT_RSVD0 0xFFFFFFFC +#define SYS_LDB_CQ_AT_CQ_AT_LOC 0 +#define SYS_LDB_CQ_AT_RSVD0_LOC 2 + +#define SYS_LDB_CQ_ISR(x) \ + (0x10000f98 + (x) * 0x1000) +#define SYS_LDB_CQ_ISR_RST 0x0 +/* CQ Interrupt Modes */ +#define DLB_CQ_ISR_MODE_DIS 0 +#define DLB_CQ_ISR_MODE_MSI 1 +#define DLB_CQ_ISR_MODE_MSIX 2 +#define DLB_CQ_ISR_MODE_ADI 3 + +#define SYS_LDB_CQ_ISR_VECTOR 0x0000003F +#define SYS_LDB_CQ_ISR_VF 0x000003C0 +#define SYS_LDB_CQ_ISR_EN_CODE 0x00000C00 +#define SYS_LDB_CQ_ISR_RSVD0 0xFFFFF000 +#define SYS_LDB_CQ_ISR_VECTOR_LOC 0 +#define SYS_LDB_CQ_ISR_VF_LOC 6 +#define SYS_LDB_CQ_ISR_EN_CODE_LOC 10 +#define SYS_LDB_CQ_ISR_RSVD0_LOC 12 + +#define SYS_LDB_CQ2VF_PF_RO(x) \ + (0x10000f94 + (x) * 0x1000) +#define SYS_LDB_CQ2VF_PF_RO_RST 0x0 + +#define SYS_LDB_CQ2VF_PF_RO_VF 0x0000000F +#define SYS_LDB_CQ2VF_PF_RO_IS_PF 0x00000010 +#define SYS_LDB_CQ2VF_PF_RO_RO 0x00000020 +#define SYS_LDB_CQ2VF_PF_RO_RSVD0 0xFFFFFFC0 +#define SYS_LDB_CQ2VF_PF_RO_VF_LOC 0 +#define SYS_LDB_CQ2VF_PF_RO_IS_PF_LOC 4 +#define SYS_LDB_CQ2VF_PF_RO_RO_LOC 5 +#define SYS_LDB_CQ2VF_PF_RO_RSVD0_LOC 6 + +#define SYS_LDB_PP_V(x) \ + (0x10000f90 + (x) * 0x1000) +#define SYS_LDB_PP_V_RST 0x0 + +#define SYS_LDB_PP_V_PP_V 0x00000001 +#define SYS_LDB_PP_V_RSVD0 0xFFFFFFFE +#define SYS_LDB_PP_V_PP_V_LOC 0 +#define SYS_LDB_PP_V_RSVD0_LOC 1 + +#define SYS_LDB_PP2VDEV(x) \ + (0x10000f8c + (x) * 0x1000) +#define SYS_LDB_PP2VDEV_RST 0x0 + +#define SYS_LDB_PP2VDEV_VDEV 0x0000000F +#define SYS_LDB_PP2VDEV_RSVD0 0xFFFFFFF0 +#define SYS_LDB_PP2VDEV_VDEV_LOC 0 +#define SYS_LDB_PP2VDEV_RSVD0_LOC 4 + +#define SYS_LDB_PP2VAS(x) \ + (0x10000f88 + (x) * 0x1000) +#define SYS_LDB_PP2VAS_RST 0x0 + +#define SYS_LDB_PP2VAS_VAS 0x0000001F +#define SYS_LDB_PP2VAS_RSVD0 0xFFFFFFE0 +#define SYS_LDB_PP2VAS_VAS_LOC 0 +#define SYS_LDB_PP2VAS_RSVD0_LOC 5 + +#define SYS_LDB_CQ_ADDR_U(x) \ + (0x10000f84 + (x) * 0x1000) +#define SYS_LDB_CQ_ADDR_U_RST 0x0 + +#define SYS_LDB_CQ_ADDR_U_ADDR_U 0xFFFFFFFF +#define SYS_LDB_CQ_ADDR_U_ADDR_U_LOC 0 + +#define SYS_LDB_CQ_ADDR_L(x) \ + (0x10000f80 + (x) * 0x1000) +#define SYS_LDB_CQ_ADDR_L_RST 0x0 + +#define SYS_LDB_CQ_ADDR_L_RSVD0 0x0000003F +#define SYS_LDB_CQ_ADDR_L_ADDR_L 0xFFFFFFC0 +#define SYS_LDB_CQ_ADDR_L_RSVD0_LOC 0 +#define SYS_LDB_CQ_ADDR_L_ADDR_L_LOC 6 + +#define SYS_DIR_CQ_FMT(x) \ + (0x10000fec + (x) * 0x1000) +#define SYS_DIR_CQ_FMT_RST 0x0 + +#define SYS_DIR_CQ_FMT_KEEP_PF_PPID 0x00000001 +#define SYS_DIR_CQ_FMT_RSVD0 0xFFFFFFFE +#define SYS_DIR_CQ_FMT_KEEP_PF_PPID_LOC 0 +#define SYS_DIR_CQ_FMT_RSVD0_LOC 1 + +#define SYS_DIR_CQ_AI_DATA(x) \ + (0x10000fe8 + (x) * 0x1000) +#define SYS_DIR_CQ_AI_DATA_RST 0x0 + +#define SYS_DIR_CQ_AI_DATA_CQ_AI_DATA 0xFFFFFFFF +#define SYS_DIR_CQ_AI_DATA_CQ_AI_DATA_LOC 0 + +#define SYS_DIR_CQ_AI_ADDR(x) \ + (0x10000fe4 + (x) * 0x1000) +#define SYS_DIR_CQ_AI_ADDR_RST 0x0 + +#define SYS_DIR_CQ_AI_ADDR_RSVD1 0x00000003 +#define SYS_DIR_CQ_AI_ADDR_CQ_AI_ADDR 0x000FFFFC +#define SYS_DIR_CQ_AI_ADDR_RSVD0 0xFFF00000 +#define SYS_DIR_CQ_AI_ADDR_RSVD1_LOC 0 +#define SYS_DIR_CQ_AI_ADDR_CQ_AI_ADDR_LOC 2 +#define SYS_DIR_CQ_AI_ADDR_RSVD0_LOC 20 + +#define SYS_DIR_CQ_PASID(x) \ + (0x10000fe0 + (x) * 0x1000) +#define SYS_DIR_CQ_PASID_RST 0x0 + +#define SYS_DIR_CQ_PASID_PASID 0x000FFFFF +#define SYS_DIR_CQ_PASID_EXE_REQ 0x00100000 +#define SYS_DIR_CQ_PASID_PRIV_REQ 0x00200000 +#define SYS_DIR_CQ_PASID_FMT2 0x00400000 +#define SYS_DIR_CQ_PASID_RSVD0 0xFF800000 +#define SYS_DIR_CQ_PASID_PASID_LOC 0 +#define SYS_DIR_CQ_PASID_EXE_REQ_LOC 20 +#define SYS_DIR_CQ_PASID_PRIV_REQ_LOC 21 +#define SYS_DIR_CQ_PASID_FMT2_LOC 22 +#define SYS_DIR_CQ_PASID_RSVD0_LOC 23 + +#define SYS_DIR_CQ_AT(x) \ + (0x10000fdc + (x) * 0x1000) +#define SYS_DIR_CQ_AT_RST 0x0 + +#define SYS_DIR_CQ_AT_CQ_AT 0x00000003 +#define SYS_DIR_CQ_AT_RSVD0 0xFFFFFFFC +#define SYS_DIR_CQ_AT_CQ_AT_LOC 0 +#define SYS_DIR_CQ_AT_RSVD0_LOC 2 + +#define SYS_DIR_CQ_ISR(x) \ + (0x10000fd8 + (x) * 0x1000) +#define SYS_DIR_CQ_ISR_RST 0x0 + +#define SYS_DIR_CQ_ISR_VECTOR 0x0000003F +#define SYS_DIR_CQ_ISR_VF 0x000003C0 +#define SYS_DIR_CQ_ISR_EN_CODE 0x00000C00 +#define SYS_DIR_CQ_ISR_RSVD0 0xFFFFF000 +#define SYS_DIR_CQ_ISR_VECTOR_LOC 0 +#define SYS_DIR_CQ_ISR_VF_LOC 6 +#define SYS_DIR_CQ_ISR_EN_CODE_LOC 10 +#define SYS_DIR_CQ_ISR_RSVD0_LOC 12 + +#define SYS_DIR_CQ2VF_PF_RO(x) \ + (0x10000fd4 + (x) * 0x1000) +#define SYS_DIR_CQ2VF_PF_RO_RST 0x0 + +#define SYS_DIR_CQ2VF_PF_RO_VF 0x0000000F +#define SYS_DIR_CQ2VF_PF_RO_IS_PF 0x00000010 +#define SYS_DIR_CQ2VF_PF_RO_RO 0x00000020 +#define SYS_DIR_CQ2VF_PF_RO_RSVD0 0xFFFFFFC0 +#define SYS_DIR_CQ2VF_PF_RO_VF_LOC 0 +#define SYS_DIR_CQ2VF_PF_RO_IS_PF_LOC 4 +#define SYS_DIR_CQ2VF_PF_RO_RO_LOC 5 +#define SYS_DIR_CQ2VF_PF_RO_RSVD0_LOC 6 + +#define SYS_DIR_PP_V(x) \ + (0x10000fd0 + (x) * 0x1000) +#define SYS_DIR_PP_V_RST 0x0 + +#define SYS_DIR_PP_V_PP_V 0x00000001 +#define SYS_DIR_PP_V_RSVD0 0xFFFFFFFE +#define SYS_DIR_PP_V_PP_V_LOC 0 +#define SYS_DIR_PP_V_RSVD0_LOC 1 + +#define SYS_DIR_PP2VDEV(x) \ + (0x10000fcc + (x) * 0x1000) +#define SYS_DIR_PP2VDEV_RST 0x0 + +#define SYS_DIR_PP2VDEV_VDEV 0x0000000F +#define SYS_DIR_PP2VDEV_RSVD0 0xFFFFFFF0 +#define SYS_DIR_PP2VDEV_VDEV_LOC 0 +#define SYS_DIR_PP2VDEV_RSVD0_LOC 4 + +#define SYS_DIR_PP2VAS(x) \ + (0x10000fc8 + (x) * 0x1000) +#define SYS_DIR_PP2VAS_RST 0x0 + +#define SYS_DIR_PP2VAS_VAS 0x0000001F +#define SYS_DIR_PP2VAS_RSVD0 0xFFFFFFE0 +#define SYS_DIR_PP2VAS_VAS_LOC 0 +#define SYS_DIR_PP2VAS_RSVD0_LOC 5 + +#define SYS_DIR_CQ_ADDR_U(x) \ + (0x10000fc4 + (x) * 0x1000) +#define SYS_DIR_CQ_ADDR_U_RST 0x0 + +#define SYS_DIR_CQ_ADDR_U_ADDR_U 0xFFFFFFFF +#define SYS_DIR_CQ_ADDR_U_ADDR_U_LOC 0 + +#define SYS_DIR_CQ_ADDR_L(x) \ + (0x10000fc0 + (x) * 0x1000) +#define SYS_DIR_CQ_ADDR_L_RST 0x0 + +#define SYS_DIR_CQ_ADDR_L_RSVD0 0x0000003F +#define SYS_DIR_CQ_ADDR_L_ADDR_L 0xFFFFFFC0 +#define SYS_DIR_CQ_ADDR_L_RSVD0_LOC 0 +#define SYS_DIR_CQ_ADDR_L_ADDR_L_LOC 6 + +#define SYS_PM_SMON_COMP_MASK1 0x10003024 +#define SYS_PM_SMON_COMP_MASK1_RST 0xffffffff + +#define SYS_PM_SMON_COMP_MASK1_COMP_MASK1 0xFFFFFFFF +#define SYS_PM_SMON_COMP_MASK1_COMP_MASK1_LOC 0 + +#define SYS_PM_SMON_COMP_MASK0 0x10003020 +#define SYS_PM_SMON_COMP_MASK0_RST 0xffffffff + +#define SYS_PM_SMON_COMP_MASK0_COMP_MASK0 0xFFFFFFFF +#define SYS_PM_SMON_COMP_MASK0_COMP_MASK0_LOC 0 + +#define SYS_PM_SMON_MAX_TMR 0x1000301c +#define SYS_PM_SMON_MAX_TMR_RST 0x0 + +#define SYS_PM_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define SYS_PM_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define SYS_PM_SMON_TMR 0x10003018 +#define SYS_PM_SMON_TMR_RST 0x0 + +#define SYS_PM_SMON_TMR_TIMER_VAL 0xFFFFFFFF +#define SYS_PM_SMON_TMR_TIMER_VAL_LOC 0 + +#define SYS_PM_SMON_ACTIVITYCNTR1 0x10003014 +#define SYS_PM_SMON_ACTIVITYCNTR1_RST 0x0 + +#define SYS_PM_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define SYS_PM_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define SYS_PM_SMON_ACTIVITYCNTR0 0x10003010 +#define SYS_PM_SMON_ACTIVITYCNTR0_RST 0x0 + +#define SYS_PM_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define SYS_PM_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define SYS_PM_SMON_COMPARE1 0x1000300c +#define SYS_PM_SMON_COMPARE1_RST 0x0 + +#define SYS_PM_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define SYS_PM_SMON_COMPARE1_COMPARE1_LOC 0 + +#define SYS_PM_SMON_COMPARE0 0x10003008 +#define SYS_PM_SMON_COMPARE0_RST 0x0 + +#define SYS_PM_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define SYS_PM_SMON_COMPARE0_COMPARE0_LOC 0 + +#define SYS_PM_SMON_CFG1 0x10003004 +#define SYS_PM_SMON_CFG1_RST 0x0 + +#define SYS_PM_SMON_CFG1_MODE0 0x000000FF +#define SYS_PM_SMON_CFG1_MODE1 0x0000FF00 +#define SYS_PM_SMON_CFG1_RSVD 0xFFFF0000 +#define SYS_PM_SMON_CFG1_MODE0_LOC 0 +#define SYS_PM_SMON_CFG1_MODE1_LOC 8 +#define SYS_PM_SMON_CFG1_RSVD_LOC 16 + +#define SYS_PM_SMON_CFG0 0x10003000 +#define SYS_PM_SMON_CFG0_RST 0x40000000 + +#define SYS_PM_SMON_CFG0_SMON_ENABLE 0x00000001 +#define SYS_PM_SMON_CFG0_RSVD2 0x0000000E +#define SYS_PM_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define SYS_PM_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define SYS_PM_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define SYS_PM_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define SYS_PM_SMON_CFG0_SMON_MODE 0x0000F000 +#define SYS_PM_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define SYS_PM_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define SYS_PM_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define SYS_PM_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define SYS_PM_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define SYS_PM_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define SYS_PM_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define SYS_PM_SMON_CFG0_RSVD1 0x00800000 +#define SYS_PM_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define SYS_PM_SMON_CFG0_RSVD0 0x20000000 +#define SYS_PM_SMON_CFG0_VERSION 0xC0000000 +#define SYS_PM_SMON_CFG0_SMON_ENABLE_LOC 0 +#define SYS_PM_SMON_CFG0_RSVD2_LOC 1 +#define SYS_PM_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define SYS_PM_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define SYS_PM_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define SYS_PM_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define SYS_PM_SMON_CFG0_SMON_MODE_LOC 12 +#define SYS_PM_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define SYS_PM_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define SYS_PM_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define SYS_PM_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define SYS_PM_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define SYS_PM_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define SYS_PM_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define SYS_PM_SMON_CFG0_RSVD1_LOC 23 +#define SYS_PM_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define SYS_PM_SMON_CFG0_RSVD0_LOC 29 +#define SYS_PM_SMON_CFG0_VERSION_LOC 30 + +#define SYS_SMON_COMP_MASK1(x) \ + (0x18002024 + (x) * 0x40) +#define SYS_SMON_COMP_MASK1_RST 0xffffffff + +#define SYS_SMON_COMP_MASK1_COMP_MASK1 0xFFFFFFFF +#define SYS_SMON_COMP_MASK1_COMP_MASK1_LOC 0 + +#define SYS_SMON_COMP_MASK0(x) \ + (0x18002020 + (x) * 0x40) +#define SYS_SMON_COMP_MASK0_RST 0xffffffff + +#define SYS_SMON_COMP_MASK0_COMP_MASK0 0xFFFFFFFF +#define SYS_SMON_COMP_MASK0_COMP_MASK0_LOC 0 + +#define SYS_SMON_MAX_TMR(x) \ + (0x1800201c + (x) * 0x40) +#define SYS_SMON_MAX_TMR_RST 0x0 + +#define SYS_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define SYS_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define SYS_SMON_TMR(x) \ + (0x18002018 + (x) * 0x40) +#define SYS_SMON_TMR_RST 0x0 + +#define SYS_SMON_TMR_TIMER_VAL 0xFFFFFFFF +#define SYS_SMON_TMR_TIMER_VAL_LOC 0 + +#define SYS_SMON_ACTIVITYCNTR1(x) \ + (0x18002014 + (x) * 0x40) +#define SYS_SMON_ACTIVITYCNTR1_RST 0x0 + +#define SYS_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define SYS_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define SYS_SMON_ACTIVITYCNTR0(x) \ + (0x18002010 + (x) * 0x40) +#define SYS_SMON_ACTIVITYCNTR0_RST 0x0 + +#define SYS_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define SYS_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define SYS_SMON_COMPARE1(x) \ + (0x1800200c + (x) * 0x40) +#define SYS_SMON_COMPARE1_RST 0x0 + +#define SYS_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define SYS_SMON_COMPARE1_COMPARE1_LOC 0 + +#define SYS_SMON_COMPARE0(x) \ + (0x18002008 + (x) * 0x40) +#define SYS_SMON_COMPARE0_RST 0x0 + +#define SYS_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define SYS_SMON_COMPARE0_COMPARE0_LOC 0 + +#define SYS_SMON_CFG1(x) \ + (0x18002004 + (x) * 0x40) +#define SYS_SMON_CFG1_RST 0x0 + +#define SYS_SMON_CFG1_MODE0 0x000000FF +#define SYS_SMON_CFG1_MODE1 0x0000FF00 +#define SYS_SMON_CFG1_RSVD 0xFFFF0000 +#define SYS_SMON_CFG1_MODE0_LOC 0 +#define SYS_SMON_CFG1_MODE1_LOC 8 +#define SYS_SMON_CFG1_RSVD_LOC 16 + +#define SYS_SMON_CFG0(x) \ + (0x18002000 + (x) * 0x40) +#define SYS_SMON_CFG0_RST 0x40000000 + +#define SYS_SMON_CFG0_SMON_ENABLE 0x00000001 +#define SYS_SMON_CFG0_RSVD2 0x0000000E +#define SYS_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define SYS_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define SYS_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define SYS_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define SYS_SMON_CFG0_SMON_MODE 0x0000F000 +#define SYS_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define SYS_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define SYS_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define SYS_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define SYS_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define SYS_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define SYS_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define SYS_SMON_CFG0_RSVD1 0x00800000 +#define SYS_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define SYS_SMON_CFG0_RSVD0 0x20000000 +#define SYS_SMON_CFG0_VERSION 0xC0000000 +#define SYS_SMON_CFG0_SMON_ENABLE_LOC 0 +#define SYS_SMON_CFG0_RSVD2_LOC 1 +#define SYS_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define SYS_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define SYS_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define SYS_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define SYS_SMON_CFG0_SMON_MODE_LOC 12 +#define SYS_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define SYS_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define SYS_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define SYS_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define SYS_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define SYS_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define SYS_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define SYS_SMON_CFG0_RSVD1_LOC 23 +#define SYS_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define SYS_SMON_CFG0_RSVD0_LOC 29 +#define SYS_SMON_CFG0_VERSION_LOC 30 + +#define SYS_INGRESS_ALARM_ENBL 0x10000300 +#define SYS_INGRESS_ALARM_ENBL_RST 0x0 + +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_HCW 0x00000001 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_PP 0x00000002 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_PASID 0x00000004 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_QID 0x00000008 +#define SYS_INGRESS_ALARM_ENBL_DISABLED_QID 0x00000010 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_LDB_QID_CFG 0x00000020 +#define SYS_INGRESS_ALARM_ENBL_RSVD0 0xFFFFFFC0 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_HCW_LOC 0 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_PP_LOC 1 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_PASID_LOC 2 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_QID_LOC 3 +#define SYS_INGRESS_ALARM_ENBL_DISABLED_QID_LOC 4 +#define SYS_INGRESS_ALARM_ENBL_ILLEGAL_LDB_QID_CFG_LOC 5 +#define SYS_INGRESS_ALARM_ENBL_RSVD0_LOC 6 + +#define SYS_MSIX_ACK 0x10000400 +#define SYS_MSIX_ACK_RST 0x0 + +#define SYS_MSIX_ACK_MSIX_0_ACK 0x00000001 +#define SYS_MSIX_ACK_MSIX_1_ACK 0x00000002 +#define SYS_MSIX_ACK_RSVD0 0xFFFFFFFC +#define SYS_MSIX_ACK_MSIX_0_ACK_LOC 0 +#define SYS_MSIX_ACK_MSIX_1_ACK_LOC 1 +#define SYS_MSIX_ACK_RSVD0_LOC 2 + +#define SYS_MSIX_PASSTHRU 0x10000404 +#define SYS_MSIX_PASSTHRU_RST 0x0 + +#define SYS_MSIX_PASSTHRU_MSIX_0_PASSTHRU 0x00000001 +#define SYS_MSIX_PASSTHRU_MSIX_1_PASSTHRU 0x00000002 +#define SYS_MSIX_PASSTHRU_RSVD0 0xFFFFFFFC +#define SYS_MSIX_PASSTHRU_MSIX_0_PASSTHRU_LOC 0 +#define SYS_MSIX_PASSTHRU_MSIX_1_PASSTHRU_LOC 1 +#define SYS_MSIX_PASSTHRU_RSVD0_LOC 2 + +#define SYS_MSIX_MODE 0x10000408 +#define SYS_MSIX_MODE_RST 0x0 +/* MSI-X Modes */ +#define DLB_MSIX_MODE_PACKED 0 +#define DLB_MSIX_MODE_COMPRESSED 1 + +#define SYS_MSIX_MODE_MODE 0x00000001 +#define SYS_MSIX_MODE_POLL_MODE 0x00000002 +#define SYS_MSIX_MODE_POLL_MASK 0x00000004 +#define SYS_MSIX_MODE_POLL_LOCK 0x00000008 +#define SYS_MSIX_MODE_RSVD0 0xFFFFFFF0 +#define SYS_MSIX_MODE_MODE_LOC 0 +#define SYS_MSIX_MODE_POLL_MODE_LOC 1 +#define SYS_MSIX_MODE_POLL_MASK_LOC 2 +#define SYS_MSIX_MODE_POLL_LOCK_LOC 3 +#define SYS_MSIX_MODE_RSVD0_LOC 4 + +#define SYS_DIR_CQ_31_0_OCC_INT_STS 0x10000440 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_RST 0x0 + +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_0_OCC_INT 0x00000001 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_1_OCC_INT 0x00000002 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_2_OCC_INT 0x00000004 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_3_OCC_INT 0x00000008 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_4_OCC_INT 0x00000010 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_5_OCC_INT 0x00000020 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_6_OCC_INT 0x00000040 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_7_OCC_INT 0x00000080 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_8_OCC_INT 0x00000100 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_9_OCC_INT 0x00000200 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_10_OCC_INT 0x00000400 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_11_OCC_INT 0x00000800 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_12_OCC_INT 0x00001000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_13_OCC_INT 0x00002000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_14_OCC_INT 0x00004000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_15_OCC_INT 0x00008000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_16_OCC_INT 0x00010000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_17_OCC_INT 0x00020000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_18_OCC_INT 0x00040000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_19_OCC_INT 0x00080000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_20_OCC_INT 0x00100000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_21_OCC_INT 0x00200000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_22_OCC_INT 0x00400000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_23_OCC_INT 0x00800000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_24_OCC_INT 0x01000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_25_OCC_INT 0x02000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_26_OCC_INT 0x04000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_27_OCC_INT 0x08000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_28_OCC_INT 0x10000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_29_OCC_INT 0x20000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_30_OCC_INT 0x40000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_31_OCC_INT 0x80000000 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_0_OCC_INT_LOC 0 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_1_OCC_INT_LOC 1 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_2_OCC_INT_LOC 2 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_3_OCC_INT_LOC 3 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_4_OCC_INT_LOC 4 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_5_OCC_INT_LOC 5 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_6_OCC_INT_LOC 6 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_7_OCC_INT_LOC 7 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_8_OCC_INT_LOC 8 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_9_OCC_INT_LOC 9 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_10_OCC_INT_LOC 10 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_11_OCC_INT_LOC 11 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_12_OCC_INT_LOC 12 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_13_OCC_INT_LOC 13 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_14_OCC_INT_LOC 14 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_15_OCC_INT_LOC 15 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_16_OCC_INT_LOC 16 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_17_OCC_INT_LOC 17 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_18_OCC_INT_LOC 18 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_19_OCC_INT_LOC 19 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_20_OCC_INT_LOC 20 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_21_OCC_INT_LOC 21 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_22_OCC_INT_LOC 22 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_23_OCC_INT_LOC 23 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_24_OCC_INT_LOC 24 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_25_OCC_INT_LOC 25 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_26_OCC_INT_LOC 26 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_27_OCC_INT_LOC 27 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_28_OCC_INT_LOC 28 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_29_OCC_INT_LOC 29 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_30_OCC_INT_LOC 30 +#define SYS_DIR_CQ_31_0_OCC_INT_STS_CQ_31_OCC_INT_LOC 31 + +#define SYS_DIR_CQ_63_32_OCC_INT_STS 0x10000444 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_RST 0x0 + +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_32_OCC_INT 0x00000001 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_33_OCC_INT 0x00000002 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_34_OCC_INT 0x00000004 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_35_OCC_INT 0x00000008 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_36_OCC_INT 0x00000010 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_37_OCC_INT 0x00000020 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_38_OCC_INT 0x00000040 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_39_OCC_INT 0x00000080 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_40_OCC_INT 0x00000100 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_41_OCC_INT 0x00000200 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_42_OCC_INT 0x00000400 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_43_OCC_INT 0x00000800 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_44_OCC_INT 0x00001000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_45_OCC_INT 0x00002000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_46_OCC_INT 0x00004000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_47_OCC_INT 0x00008000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_48_OCC_INT 0x00010000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_49_OCC_INT 0x00020000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_50_OCC_INT 0x00040000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_51_OCC_INT 0x00080000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_52_OCC_INT 0x00100000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_53_OCC_INT 0x00200000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_54_OCC_INT 0x00400000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_55_OCC_INT 0x00800000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_56_OCC_INT 0x01000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_57_OCC_INT 0x02000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_58_OCC_INT 0x04000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_59_OCC_INT 0x08000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_60_OCC_INT 0x10000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_61_OCC_INT 0x20000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_62_OCC_INT 0x40000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_63_OCC_INT 0x80000000 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_32_OCC_INT_LOC 0 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_33_OCC_INT_LOC 1 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_34_OCC_INT_LOC 2 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_35_OCC_INT_LOC 3 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_36_OCC_INT_LOC 4 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_37_OCC_INT_LOC 5 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_38_OCC_INT_LOC 6 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_39_OCC_INT_LOC 7 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_40_OCC_INT_LOC 8 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_41_OCC_INT_LOC 9 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_42_OCC_INT_LOC 10 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_43_OCC_INT_LOC 11 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_44_OCC_INT_LOC 12 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_45_OCC_INT_LOC 13 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_46_OCC_INT_LOC 14 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_47_OCC_INT_LOC 15 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_48_OCC_INT_LOC 16 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_49_OCC_INT_LOC 17 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_50_OCC_INT_LOC 18 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_51_OCC_INT_LOC 19 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_52_OCC_INT_LOC 20 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_53_OCC_INT_LOC 21 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_54_OCC_INT_LOC 22 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_55_OCC_INT_LOC 23 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_56_OCC_INT_LOC 24 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_57_OCC_INT_LOC 25 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_58_OCC_INT_LOC 26 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_59_OCC_INT_LOC 27 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_60_OCC_INT_LOC 28 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_61_OCC_INT_LOC 29 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_62_OCC_INT_LOC 30 +#define SYS_DIR_CQ_63_32_OCC_INT_STS_CQ_63_OCC_INT_LOC 31 + +#define SYS_LDB_CQ_31_0_OCC_INT_STS 0x10000460 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_RST 0x0 + +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_0_OCC_INT 0x00000001 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_1_OCC_INT 0x00000002 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_2_OCC_INT 0x00000004 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_3_OCC_INT 0x00000008 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_4_OCC_INT 0x00000010 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_5_OCC_INT 0x00000020 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_6_OCC_INT 0x00000040 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_7_OCC_INT 0x00000080 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_8_OCC_INT 0x00000100 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_9_OCC_INT 0x00000200 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_10_OCC_INT 0x00000400 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_11_OCC_INT 0x00000800 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_12_OCC_INT 0x00001000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_13_OCC_INT 0x00002000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_14_OCC_INT 0x00004000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_15_OCC_INT 0x00008000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_16_OCC_INT 0x00010000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_17_OCC_INT 0x00020000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_18_OCC_INT 0x00040000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_19_OCC_INT 0x00080000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_20_OCC_INT 0x00100000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_21_OCC_INT 0x00200000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_22_OCC_INT 0x00400000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_23_OCC_INT 0x00800000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_24_OCC_INT 0x01000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_25_OCC_INT 0x02000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_26_OCC_INT 0x04000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_27_OCC_INT 0x08000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_28_OCC_INT 0x10000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_29_OCC_INT 0x20000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_30_OCC_INT 0x40000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_31_OCC_INT 0x80000000 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_0_OCC_INT_LOC 0 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_1_OCC_INT_LOC 1 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_2_OCC_INT_LOC 2 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_3_OCC_INT_LOC 3 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_4_OCC_INT_LOC 4 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_5_OCC_INT_LOC 5 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_6_OCC_INT_LOC 6 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_7_OCC_INT_LOC 7 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_8_OCC_INT_LOC 8 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_9_OCC_INT_LOC 9 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_10_OCC_INT_LOC 10 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_11_OCC_INT_LOC 11 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_12_OCC_INT_LOC 12 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_13_OCC_INT_LOC 13 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_14_OCC_INT_LOC 14 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_15_OCC_INT_LOC 15 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_16_OCC_INT_LOC 16 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_17_OCC_INT_LOC 17 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_18_OCC_INT_LOC 18 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_19_OCC_INT_LOC 19 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_20_OCC_INT_LOC 20 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_21_OCC_INT_LOC 21 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_22_OCC_INT_LOC 22 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_23_OCC_INT_LOC 23 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_24_OCC_INT_LOC 24 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_25_OCC_INT_LOC 25 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_26_OCC_INT_LOC 26 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_27_OCC_INT_LOC 27 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_28_OCC_INT_LOC 28 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_29_OCC_INT_LOC 29 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_30_OCC_INT_LOC 30 +#define SYS_LDB_CQ_31_0_OCC_INT_STS_CQ_31_OCC_INT_LOC 31 + +#define SYS_LDB_CQ_63_32_OCC_INT_STS 0x10000464 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_RST 0x0 + +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_32_OCC_INT 0x00000001 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_33_OCC_INT 0x00000002 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_34_OCC_INT 0x00000004 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_35_OCC_INT 0x00000008 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_36_OCC_INT 0x00000010 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_37_OCC_INT 0x00000020 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_38_OCC_INT 0x00000040 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_39_OCC_INT 0x00000080 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_40_OCC_INT 0x00000100 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_41_OCC_INT 0x00000200 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_42_OCC_INT 0x00000400 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_43_OCC_INT 0x00000800 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_44_OCC_INT 0x00001000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_45_OCC_INT 0x00002000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_46_OCC_INT 0x00004000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_47_OCC_INT 0x00008000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_48_OCC_INT 0x00010000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_49_OCC_INT 0x00020000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_50_OCC_INT 0x00040000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_51_OCC_INT 0x00080000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_52_OCC_INT 0x00100000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_53_OCC_INT 0x00200000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_54_OCC_INT 0x00400000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_55_OCC_INT 0x00800000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_56_OCC_INT 0x01000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_57_OCC_INT 0x02000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_58_OCC_INT 0x04000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_59_OCC_INT 0x08000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_60_OCC_INT 0x10000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_61_OCC_INT 0x20000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_62_OCC_INT 0x40000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_63_OCC_INT 0x80000000 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_32_OCC_INT_LOC 0 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_33_OCC_INT_LOC 1 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_34_OCC_INT_LOC 2 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_35_OCC_INT_LOC 3 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_36_OCC_INT_LOC 4 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_37_OCC_INT_LOC 5 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_38_OCC_INT_LOC 6 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_39_OCC_INT_LOC 7 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_40_OCC_INT_LOC 8 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_41_OCC_INT_LOC 9 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_42_OCC_INT_LOC 10 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_43_OCC_INT_LOC 11 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_44_OCC_INT_LOC 12 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_45_OCC_INT_LOC 13 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_46_OCC_INT_LOC 14 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_47_OCC_INT_LOC 15 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_48_OCC_INT_LOC 16 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_49_OCC_INT_LOC 17 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_50_OCC_INT_LOC 18 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_51_OCC_INT_LOC 19 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_52_OCC_INT_LOC 20 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_53_OCC_INT_LOC 21 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_54_OCC_INT_LOC 22 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_55_OCC_INT_LOC 23 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_56_OCC_INT_LOC 24 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_57_OCC_INT_LOC 25 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_58_OCC_INT_LOC 26 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_59_OCC_INT_LOC 27 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_60_OCC_INT_LOC 28 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_61_OCC_INT_LOC 29 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_62_OCC_INT_LOC 30 +#define SYS_LDB_CQ_63_32_OCC_INT_STS_CQ_63_OCC_INT_LOC 31 + +#define SYS_DIR_CQ_OPT_CLR 0x100004c0 +#define SYS_DIR_CQ_OPT_CLR_RST 0x0 + +#define SYS_DIR_CQ_OPT_CLR_CQ 0x0000003F +#define SYS_DIR_CQ_OPT_CLR_RSVD0 0xFFFFFFC0 +#define SYS_DIR_CQ_OPT_CLR_CQ_LOC 0 +#define SYS_DIR_CQ_OPT_CLR_RSVD0_LOC 6 + +#define SYS_ALARM_HW_SYND 0x1000050c +#define SYS_ALARM_HW_SYND_RST 0x0 + +#define SYS_ALARM_HW_SYND_SYNDROME 0x000000FF +#define SYS_ALARM_HW_SYND_RTYPE 0x00000300 +#define SYS_ALARM_HW_SYND_ALARM 0x00000400 +#define SYS_ALARM_HW_SYND_CWD 0x00000800 +#define SYS_ALARM_HW_SYND_VF_PF_MB 0x00001000 +#define SYS_ALARM_HW_SYND_RSVD0 0x00002000 +#define SYS_ALARM_HW_SYND_CLS 0x0000C000 +#define SYS_ALARM_HW_SYND_AID 0x003F0000 +#define SYS_ALARM_HW_SYND_UNIT 0x03C00000 +#define SYS_ALARM_HW_SYND_SOURCE 0x3C000000 +#define SYS_ALARM_HW_SYND_MORE 0x40000000 +#define SYS_ALARM_HW_SYND_VALID 0x80000000 +#define SYS_ALARM_HW_SYND_SYNDROME_LOC 0 +#define SYS_ALARM_HW_SYND_RTYPE_LOC 8 +#define SYS_ALARM_HW_SYND_ALARM_LOC 10 +#define SYS_ALARM_HW_SYND_CWD_LOC 11 +#define SYS_ALARM_HW_SYND_VF_PF_MB_LOC 12 +#define SYS_ALARM_HW_SYND_RSVD0_LOC 13 +#define SYS_ALARM_HW_SYND_CLS_LOC 14 +#define SYS_ALARM_HW_SYND_AID_LOC 16 +#define SYS_ALARM_HW_SYND_UNIT_LOC 22 +#define SYS_ALARM_HW_SYND_SOURCE_LOC 26 +#define SYS_ALARM_HW_SYND_MORE_LOC 30 +#define SYS_ALARM_HW_SYND_VALID_LOC 31 + +#define AQED_QID_FID_LIM(x) \ + (0x20000000 + (x) * 0x1000) +#define AQED_QID_FID_LIM_RST 0x7ff + +#define AQED_QID_FID_LIM_QID_FID_LIMIT 0x00001FFF +#define AQED_QID_FID_LIM_RSVD0 0xFFFFE000 +#define AQED_QID_FID_LIM_QID_FID_LIMIT_LOC 0 +#define AQED_QID_FID_LIM_RSVD0_LOC 13 + +#define AQED_QID_HID_WIDTH(x) \ + (0x20080000 + (x) * 0x1000) +#define AQED_QID_HID_WIDTH_RST 0x0 + +#define AQED_QID_HID_WIDTH_COMPRESS_CODE 0x00000007 +#define AQED_QID_HID_WIDTH_RSVD0 0xFFFFFFF8 +#define AQED_QID_HID_WIDTH_COMPRESS_CODE_LOC 0 +#define AQED_QID_HID_WIDTH_RSVD0_LOC 3 + +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0 0x24000004 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_RST 0xfefcfaf8 + +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI0 0x000000FF +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI1 0x0000FF00 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI2 0x00FF0000 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI3 0xFF000000 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI0_LOC 0 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI1_LOC 8 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI2_LOC 16 +#define AQED_CFG_ARB_WEIGHTS_TQPRI_ATM_0_PRI3_LOC 24 + +#define AQED_SMON_ACTIVITYCNTR0 0x2c00004c +#define AQED_SMON_ACTIVITYCNTR0_RST 0x0 + +#define AQED_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define AQED_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define AQED_SMON_ACTIVITYCNTR1 0x2c000050 +#define AQED_SMON_ACTIVITYCNTR1_RST 0x0 + +#define AQED_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define AQED_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define AQED_SMON_COMPARE0 0x2c000054 +#define AQED_SMON_COMPARE0_RST 0x0 + +#define AQED_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define AQED_SMON_COMPARE0_COMPARE0_LOC 0 + +#define AQED_SMON_COMPARE1 0x2c000058 +#define AQED_SMON_COMPARE1_RST 0x0 + +#define AQED_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define AQED_SMON_COMPARE1_COMPARE1_LOC 0 + +#define AQED_SMON_CFG0 0x2c00005c +#define AQED_SMON_CFG0_RST 0x40000000 + +#define AQED_SMON_CFG0_SMON_ENABLE 0x00000001 +#define AQED_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define AQED_SMON_CFG0_RSVZ0 0x0000000C +#define AQED_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define AQED_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define AQED_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define AQED_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define AQED_SMON_CFG0_SMON_MODE 0x0000F000 +#define AQED_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define AQED_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define AQED_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define AQED_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define AQED_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define AQED_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define AQED_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define AQED_SMON_CFG0_RSVZ1 0x00800000 +#define AQED_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define AQED_SMON_CFG0_RSVZ2 0x20000000 +#define AQED_SMON_CFG0_VERSION 0xC0000000 +#define AQED_SMON_CFG0_SMON_ENABLE_LOC 0 +#define AQED_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define AQED_SMON_CFG0_RSVZ0_LOC 2 +#define AQED_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define AQED_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define AQED_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define AQED_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define AQED_SMON_CFG0_SMON_MODE_LOC 12 +#define AQED_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define AQED_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define AQED_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define AQED_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define AQED_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define AQED_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define AQED_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define AQED_SMON_CFG0_RSVZ1_LOC 23 +#define AQED_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define AQED_SMON_CFG0_RSVZ2_LOC 29 +#define AQED_SMON_CFG0_VERSION_LOC 30 + +#define AQED_SMON_CFG1 0x2c000060 +#define AQED_SMON_CFG1_RST 0x0 + +#define AQED_SMON_CFG1_MODE0 0x000000FF +#define AQED_SMON_CFG1_MODE1 0x0000FF00 +#define AQED_SMON_CFG1_RSVZ0 0xFFFF0000 +#define AQED_SMON_CFG1_MODE0_LOC 0 +#define AQED_SMON_CFG1_MODE1_LOC 8 +#define AQED_SMON_CFG1_RSVZ0_LOC 16 + +#define AQED_SMON_MAX_TMR 0x2c000064 +#define AQED_SMON_MAX_TMR_RST 0x0 + +#define AQED_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define AQED_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define AQED_SMON_TMR 0x2c000068 +#define AQED_SMON_TMR_RST 0x0 + +#define AQED_SMON_TMR_TIMER 0xFFFFFFFF +#define AQED_SMON_TMR_TIMER_LOC 0 + +#define ATM_QID2CQIDIX_00(x) \ + (0x30080000 + (x) * 0x1000) +#define ATM_QID2CQIDIX_00_RST 0x0 +#define ATM_QID2CQIDIX(x, y) \ + (ATM_QID2CQIDIX_00(x) + 0x80000 * (y)) +#define ATM_QID2CQIDIX_NUM 16 + +#define ATM_QID2CQIDIX_00_CQ_P0 0x000000FF +#define ATM_QID2CQIDIX_00_CQ_P1 0x0000FF00 +#define ATM_QID2CQIDIX_00_CQ_P2 0x00FF0000 +#define ATM_QID2CQIDIX_00_CQ_P3 0xFF000000 +#define ATM_QID2CQIDIX_00_CQ_P0_LOC 0 +#define ATM_QID2CQIDIX_00_CQ_P1_LOC 8 +#define ATM_QID2CQIDIX_00_CQ_P2_LOC 16 +#define ATM_QID2CQIDIX_00_CQ_P3_LOC 24 + +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN 0x34000004 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_RST 0xfffefdfc + +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN0 0x000000FF +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN1 0x0000FF00 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN2 0x00FF0000 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN3 0xFF000000 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN0_LOC 0 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN1_LOC 8 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN2_LOC 16 +#define ATM_CFG_ARB_WEIGHTS_RDY_BIN_BIN3_LOC 24 + +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN 0x34000008 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_RST 0xfffefdfc + +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN0 0x000000FF +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN1 0x0000FF00 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN2 0x00FF0000 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN3 0xFF000000 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN0_LOC 0 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN1_LOC 8 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN2_LOC 16 +#define ATM_CFG_ARB_WEIGHTS_SCHED_BIN_BIN3_LOC 24 + +#define ATM_SMON_ACTIVITYCNTR0 0x3c000050 +#define ATM_SMON_ACTIVITYCNTR0_RST 0x0 + +#define ATM_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define ATM_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define ATM_SMON_ACTIVITYCNTR1 0x3c000054 +#define ATM_SMON_ACTIVITYCNTR1_RST 0x0 + +#define ATM_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define ATM_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define ATM_SMON_COMPARE0 0x3c000058 +#define ATM_SMON_COMPARE0_RST 0x0 + +#define ATM_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define ATM_SMON_COMPARE0_COMPARE0_LOC 0 + +#define ATM_SMON_COMPARE1 0x3c00005c +#define ATM_SMON_COMPARE1_RST 0x0 + +#define ATM_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define ATM_SMON_COMPARE1_COMPARE1_LOC 0 + +#define ATM_SMON_CFG0 0x3c000060 +#define ATM_SMON_CFG0_RST 0x40000000 + +#define ATM_SMON_CFG0_SMON_ENABLE 0x00000001 +#define ATM_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define ATM_SMON_CFG0_RSVZ0 0x0000000C +#define ATM_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define ATM_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define ATM_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define ATM_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define ATM_SMON_CFG0_SMON_MODE 0x0000F000 +#define ATM_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define ATM_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define ATM_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define ATM_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define ATM_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define ATM_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define ATM_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define ATM_SMON_CFG0_RSVZ1 0x00800000 +#define ATM_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define ATM_SMON_CFG0_RSVZ2 0x20000000 +#define ATM_SMON_CFG0_VERSION 0xC0000000 +#define ATM_SMON_CFG0_SMON_ENABLE_LOC 0 +#define ATM_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define ATM_SMON_CFG0_RSVZ0_LOC 2 +#define ATM_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define ATM_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define ATM_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define ATM_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define ATM_SMON_CFG0_SMON_MODE_LOC 12 +#define ATM_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define ATM_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define ATM_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define ATM_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define ATM_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define ATM_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define ATM_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define ATM_SMON_CFG0_RSVZ1_LOC 23 +#define ATM_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define ATM_SMON_CFG0_RSVZ2_LOC 29 +#define ATM_SMON_CFG0_VERSION_LOC 30 + +#define ATM_SMON_CFG1 0x3c000064 +#define ATM_SMON_CFG1_RST 0x0 + +#define ATM_SMON_CFG1_MODE0 0x000000FF +#define ATM_SMON_CFG1_MODE1 0x0000FF00 +#define ATM_SMON_CFG1_RSVZ0 0xFFFF0000 +#define ATM_SMON_CFG1_MODE0_LOC 0 +#define ATM_SMON_CFG1_MODE1_LOC 8 +#define ATM_SMON_CFG1_RSVZ0_LOC 16 + +#define ATM_SMON_MAX_TMR 0x3c000068 +#define ATM_SMON_MAX_TMR_RST 0x0 + +#define ATM_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define ATM_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define ATM_SMON_TMR 0x3c00006c +#define ATM_SMON_TMR_RST 0x0 + +#define ATM_SMON_TMR_TIMER 0xFFFFFFFF +#define ATM_SMON_TMR_TIMER_LOC 0 #define CHP_CFG_DIR_VAS_CRD(x) \ (0x40000000 + (x) * 0x1000) #define CHP_CFG_DIR_VAS_CRD_RST 0x0 -#define CHP_CFG_DIR_VAS_CRD_COUNT 0x00003FFF -#define CHP_CFG_DIR_VAS_CRD_RSVD0 0xFFFFC000 -#define CHP_CFG_DIR_VAS_CRD_COUNT_LOC 0 -#define CHP_CFG_DIR_VAS_CRD_RSVD0_LOC 14 +#define CHP_CFG_DIR_VAS_CRD_COUNT 0x00003FFF +#define CHP_CFG_DIR_VAS_CRD_RSVD0 0xFFFFC000 +#define CHP_CFG_DIR_VAS_CRD_COUNT_LOC 0 +#define CHP_CFG_DIR_VAS_CRD_RSVD0_LOC 14 + +#define CHP_CFG_LDB_VAS_CRD(x) \ + (0x40080000 + (x) * 0x1000) +#define CHP_CFG_LDB_VAS_CRD_RST 0x0 + +#define CHP_CFG_LDB_VAS_CRD_COUNT 0x00007FFF +#define CHP_CFG_LDB_VAS_CRD_RSVD0 0xFFFF8000 +#define CHP_CFG_LDB_VAS_CRD_COUNT_LOC 0 +#define CHP_CFG_LDB_VAS_CRD_RSVD0_LOC 15 + +#define CHP_ORD_QID_SN(x) \ + (0x40100000 + (x) * 0x1000) +#define CHP_ORD_QID_SN_RST 0x0 + +#define CHP_ORD_QID_SN_SN 0x000003FF +#define CHP_ORD_QID_SN_RSVD0 0xFFFFFC00 +#define CHP_ORD_QID_SN_SN_LOC 0 +#define CHP_ORD_QID_SN_RSVD0_LOC 10 + +#define CHP_ORD_QID_SN_MAP(x) \ + (0x40180000 + (x) * 0x1000) +#define CHP_ORD_QID_SN_MAP_RST 0x0 + +#define CHP_ORD_QID_SN_MAP_MODE 0x00000007 +#define CHP_ORD_QID_SN_MAP_SLOT 0x00000078 +#define CHP_ORD_QID_SN_MAP_RSVZ0 0x00000080 +#define CHP_ORD_QID_SN_MAP_GRP 0x00000100 +#define CHP_ORD_QID_SN_MAP_RSVZ1 0x00000200 +#define CHP_ORD_QID_SN_MAP_RSVD0 0xFFFFFC00 +#define CHP_ORD_QID_SN_MAP_MODE_LOC 0 +#define CHP_ORD_QID_SN_MAP_SLOT_LOC 3 +#define CHP_ORD_QID_SN_MAP_RSVZ0_LOC 7 +#define CHP_ORD_QID_SN_MAP_GRP_LOC 8 +#define CHP_ORD_QID_SN_MAP_RSVZ1_LOC 9 +#define CHP_ORD_QID_SN_MAP_RSVD0_LOC 10 + +#define CHP_SN_CHK_ENBL(x) \ + (0x40200000 + (x) * 0x1000) +#define CHP_SN_CHK_ENBL_RST 0x0 + +#define CHP_SN_CHK_ENBL_EN 0x00000001 +#define CHP_SN_CHK_ENBL_RSVD0 0xFFFFFFFE +#define CHP_SN_CHK_ENBL_EN_LOC 0 +#define CHP_SN_CHK_ENBL_RSVD0_LOC 1 + +#define CHP_DIR_CQ_DEPTH(x) \ + (0x40280000 + (x) * 0x1000) +#define CHP_DIR_CQ_DEPTH_RST 0x0 + +#define CHP_DIR_CQ_DEPTH_DEPTH 0x00001FFF +#define CHP_DIR_CQ_DEPTH_RSVD0 0xFFFFE000 +#define CHP_DIR_CQ_DEPTH_DEPTH_LOC 0 +#define CHP_DIR_CQ_DEPTH_RSVD0_LOC 13 + +#define CHP_DIR_CQ_INT_DEPTH_THRSH(x) \ + (0x40300000 + (x) * 0x1000) +#define CHP_DIR_CQ_INT_DEPTH_THRSH_RST 0x0 + +#define CHP_DIR_CQ_INT_DEPTH_THRSH_DEPTH_THRESHOLD 0x00001FFF +#define CHP_DIR_CQ_INT_DEPTH_THRSH_RSVD0 0xFFFFE000 +#define CHP_DIR_CQ_INT_DEPTH_THRSH_DEPTH_THRESHOLD_LOC 0 +#define CHP_DIR_CQ_INT_DEPTH_THRSH_RSVD0_LOC 13 + +#define CHP_DIR_CQ_INT_ENB(x) \ + (0x40380000 + (x) * 0x1000) +#define CHP_DIR_CQ_INT_ENB_RST 0x0 + +#define CHP_DIR_CQ_INT_ENB_EN_TIM 0x00000001 +#define CHP_DIR_CQ_INT_ENB_EN_DEPTH 0x00000002 +#define CHP_DIR_CQ_INT_ENB_RSVD0 0xFFFFFFFC +#define CHP_DIR_CQ_INT_ENB_EN_TIM_LOC 0 +#define CHP_DIR_CQ_INT_ENB_EN_DEPTH_LOC 1 +#define CHP_DIR_CQ_INT_ENB_RSVD0_LOC 2 + +#define CHP_DIR_CQ_TMR_THRSH(x) \ + (0x40480000 + (x) * 0x1000) +#define CHP_DIR_CQ_TMR_THRSH_RST 0x1 + +#define CHP_DIR_CQ_TMR_THRSH_THRSH_0 0x00000001 +#define CHP_DIR_CQ_TMR_THRSH_THRSH_13_1 0x00003FFE +#define CHP_DIR_CQ_TMR_THRSH_RSVD0 0xFFFFC000 +#define CHP_DIR_CQ_TMR_THRSH_THRSH_0_LOC 0 +#define CHP_DIR_CQ_TMR_THRSH_THRSH_13_1_LOC 1 +#define CHP_DIR_CQ_TMR_THRSH_RSVD0_LOC 14 + +#define CHP_DIR_CQ_TKN_DEPTH_SEL(x) \ + (0x40500000 + (x) * 0x1000) +#define CHP_DIR_CQ_TKN_DEPTH_SEL_RST 0x0 + +#define CHP_DIR_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT 0x0000000F +#define CHP_DIR_CQ_TKN_DEPTH_SEL_RSVD0 0xFFFFFFF0 +#define CHP_DIR_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT_LOC 0 +#define CHP_DIR_CQ_TKN_DEPTH_SEL_RSVD0_LOC 4 + +#define CHP_DIR_CQ_WD_ENB(x) \ + (0x40580000 + (x) * 0x1000) +#define CHP_DIR_CQ_WD_ENB_RST 0x0 + +#define CHP_DIR_CQ_WD_ENB_WD_ENABLE 0x00000001 +#define CHP_DIR_CQ_WD_ENB_RSVD0 0xFFFFFFFE +#define CHP_DIR_CQ_WD_ENB_WD_ENABLE_LOC 0 +#define CHP_DIR_CQ_WD_ENB_RSVD0_LOC 1 + +#define CHP_DIR_CQ_WPTR(x) \ + (0x40600000 + (x) * 0x1000) +#define CHP_DIR_CQ_WPTR_RST 0x0 + +#define CHP_DIR_CQ_WPTR_WRITE_POINTER 0x00001FFF +#define CHP_DIR_CQ_WPTR_RSVD0 0xFFFFE000 +#define CHP_DIR_CQ_WPTR_WRITE_POINTER_LOC 0 +#define CHP_DIR_CQ_WPTR_RSVD0_LOC 13 + +#define CHP_DIR_CQ2VAS(x) \ + (0x40680000 + (x) * 0x1000) +#define CHP_DIR_CQ2VAS_RST 0x0 + +#define CHP_DIR_CQ2VAS_CQ2VAS 0x0000001F +#define CHP_DIR_CQ2VAS_RSVD0 0xFFFFFFE0 +#define CHP_DIR_CQ2VAS_CQ2VAS_LOC 0 +#define CHP_DIR_CQ2VAS_RSVD0_LOC 5 + +#define CHP_HIST_LIST_BASE(x) \ + (0x40700000 + (x) * 0x1000) +#define CHP_HIST_LIST_BASE_RST 0x0 + +#define CHP_HIST_LIST_BASE_BASE 0x00001FFF +#define CHP_HIST_LIST_BASE_RSVD0 0xFFFFE000 +#define CHP_HIST_LIST_BASE_BASE_LOC 0 +#define CHP_HIST_LIST_BASE_RSVD0_LOC 13 + +#define CHP_HIST_LIST_LIM(x) \ + (0x40780000 + (x) * 0x1000) +#define CHP_HIST_LIST_LIM_RST 0x0 + +#define CHP_HIST_LIST_LIM_LIMIT 0x00001FFF +#define CHP_HIST_LIST_LIM_RSVD0 0xFFFFE000 +#define CHP_HIST_LIST_LIM_LIMIT_LOC 0 +#define CHP_HIST_LIST_LIM_RSVD0_LOC 13 + +#define CHP_HIST_LIST_POP_PTR(x) \ + (0x40800000 + (x) * 0x1000) +#define CHP_HIST_LIST_POP_PTR_RST 0x0 + +#define CHP_HIST_LIST_POP_PTR_POP_PTR 0x00001FFF +#define CHP_HIST_LIST_POP_PTR_GENERATION 0x00002000 +#define CHP_HIST_LIST_POP_PTR_RSVD0 0xFFFFC000 +#define CHP_HIST_LIST_POP_PTR_POP_PTR_LOC 0 +#define CHP_HIST_LIST_POP_PTR_GENERATION_LOC 13 +#define CHP_HIST_LIST_POP_PTR_RSVD0_LOC 14 + +#define CHP_HIST_LIST_PUSH_PTR(x) \ + (0x40880000 + (x) * 0x1000) +#define CHP_HIST_LIST_PUSH_PTR_RST 0x0 + +#define CHP_HIST_LIST_PUSH_PTR_PUSH_PTR 0x00001FFF +#define CHP_HIST_LIST_PUSH_PTR_GENERATION 0x00002000 +#define CHP_HIST_LIST_PUSH_PTR_RSVD0 0xFFFFC000 +#define CHP_HIST_LIST_PUSH_PTR_PUSH_PTR_LOC 0 +#define CHP_HIST_LIST_PUSH_PTR_GENERATION_LOC 13 +#define CHP_HIST_LIST_PUSH_PTR_RSVD0_LOC 14 + +#define CHP_LDB_CQ_DEPTH(x) \ + (0x40900000 + (x) * 0x1000) +#define CHP_LDB_CQ_DEPTH_RST 0x0 + +#define CHP_LDB_CQ_DEPTH_DEPTH 0x000007FF +#define CHP_LDB_CQ_DEPTH_RSVD0 0xFFFFF800 +#define CHP_LDB_CQ_DEPTH_DEPTH_LOC 0 +#define CHP_LDB_CQ_DEPTH_RSVD0_LOC 11 + +#define CHP_LDB_CQ_INT_DEPTH_THRSH(x) \ + (0x40980000 + (x) * 0x1000) +#define CHP_LDB_CQ_INT_DEPTH_THRSH_RST 0x0 + +#define CHP_LDB_CQ_INT_DEPTH_THRSH_DEPTH_THRESHOLD 0x000007FF +#define CHP_LDB_CQ_INT_DEPTH_THRSH_RSVD0 0xFFFFF800 +#define CHP_LDB_CQ_INT_DEPTH_THRSH_DEPTH_THRESHOLD_LOC 0 +#define CHP_LDB_CQ_INT_DEPTH_THRSH_RSVD0_LOC 11 + +#define CHP_LDB_CQ_INT_ENB(x) \ + (0x40a00000 + (x) * 0x1000) +#define CHP_LDB_CQ_INT_ENB_RST 0x0 + +#define CHP_LDB_CQ_INT_ENB_EN_TIM 0x00000001 +#define CHP_LDB_CQ_INT_ENB_EN_DEPTH 0x00000002 +#define CHP_LDB_CQ_INT_ENB_RSVD0 0xFFFFFFFC +#define CHP_LDB_CQ_INT_ENB_EN_TIM_LOC 0 +#define CHP_LDB_CQ_INT_ENB_EN_DEPTH_LOC 1 +#define CHP_LDB_CQ_INT_ENB_RSVD0_LOC 2 + +#define CHP_LDB_CQ_TMR_THRSH(x) \ + (0x40b00000 + (x) * 0x1000) +#define CHP_LDB_CQ_TMR_THRSH_RST 0x1 + +#define CHP_LDB_CQ_TMR_THRSH_THRSH_0 0x00000001 +#define CHP_LDB_CQ_TMR_THRSH_THRSH_13_1 0x00003FFE +#define CHP_LDB_CQ_TMR_THRSH_RSVD0 0xFFFFC000 +#define CHP_LDB_CQ_TMR_THRSH_THRSH_0_LOC 0 +#define CHP_LDB_CQ_TMR_THRSH_THRSH_13_1_LOC 1 +#define CHP_LDB_CQ_TMR_THRSH_RSVD0_LOC 14 + +#define CHP_LDB_CQ_TKN_DEPTH_SEL(x) \ + (0x40b80000 + (x) * 0x1000) +#define CHP_LDB_CQ_TKN_DEPTH_SEL_RST 0x0 + +#define CHP_LDB_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT 0x0000000F +#define CHP_LDB_CQ_TKN_DEPTH_SEL_RSVD0 0xFFFFFFF0 +#define CHP_LDB_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT_LOC 0 +#define CHP_LDB_CQ_TKN_DEPTH_SEL_RSVD0_LOC 4 + +#define CHP_LDB_CQ_WD_ENB(x) \ + (0x40c00000 + (x) * 0x1000) +#define CHP_LDB_CQ_WD_ENB_RST 0x0 + +#define CHP_LDB_CQ_WD_ENB_WD_ENABLE 0x00000001 +#define CHP_LDB_CQ_WD_ENB_RSVD0 0xFFFFFFFE +#define CHP_LDB_CQ_WD_ENB_WD_ENABLE_LOC 0 +#define CHP_LDB_CQ_WD_ENB_RSVD0_LOC 1 + +#define CHP_LDB_CQ_WPTR(x) \ + (0x40c80000 + (x) * 0x1000) +#define CHP_LDB_CQ_WPTR_RST 0x0 + +#define CHP_LDB_CQ_WPTR_WRITE_POINTER 0x000007FF +#define CHP_LDB_CQ_WPTR_RSVD0 0xFFFFF800 +#define CHP_LDB_CQ_WPTR_WRITE_POINTER_LOC 0 +#define CHP_LDB_CQ_WPTR_RSVD0_LOC 11 + +#define CHP_LDB_CQ2VAS(x) \ + (0x40d00000 + (x) * 0x1000) +#define CHP_LDB_CQ2VAS_RST 0x0 + +#define CHP_LDB_CQ2VAS_CQ2VAS 0x0000001F +#define CHP_LDB_CQ2VAS_RSVD0 0xFFFFFFE0 +#define CHP_LDB_CQ2VAS_CQ2VAS_LOC 0 +#define CHP_LDB_CQ2VAS_RSVD0_LOC 5 + +#define CHP_CFG_CHP_CSR_CTRL 0x44000008 +#define CHP_CFG_CHP_CSR_CTRL_RST 0x180002 + +#define CHP_CFG_CHP_CSR_CTRL_INT_COR_ALARM_DIS 0x00000001 +#define CHP_CFG_CHP_CSR_CTRL_INT_COR_SYND_DIS 0x00000002 +#define CHP_CFG_CHP_CSR_CTRL_INT_UNCR_ALARM_DIS 0x00000004 +#define CHP_CFG_CHP_CSR_CTRL_INT_UNC_SYND_DIS 0x00000008 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF0_ALARM_DIS 0x00000010 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF0_SYND_DIS 0x00000020 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF1_ALARM_DIS 0x00000040 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF1_SYND_DIS 0x00000080 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF2_ALARM_DIS 0x00000100 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF2_SYND_DIS 0x00000200 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF3_ALARM_DIS 0x00000400 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF3_SYND_DIS 0x00000800 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF4_ALARM_DIS 0x00001000 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF4_SYND_DIS 0x00002000 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF5_ALARM_DIS 0x00004000 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF5_SYND_DIS 0x00008000 +#define CHP_CFG_CHP_CSR_CTRL_DLB_COR_ALARM_ENABLE 0x00010000 +#define CHP_CFG_CHP_CSR_CTRL_CFG_64BYTES_QE_LDB_CQ_MODE 0x00020000 +#define CHP_CFG_CHP_CSR_CTRL_CFG_64BYTES_QE_DIR_CQ_MODE 0x00040000 +#define CHP_CFG_CHP_CSR_CTRL_PAD_WRITE_LDB 0x00080000 +#define CHP_CFG_CHP_CSR_CTRL_PAD_WRITE_DIR 0x00100000 +#define CHP_CFG_CHP_CSR_CTRL_PAD_FIRST_WRITE_LDB 0x00200000 +#define CHP_CFG_CHP_CSR_CTRL_PAD_FIRST_WRITE_DIR 0x00400000 +#define CHP_CFG_CHP_CSR_CTRL_RSVZ0 0xFF800000 +#define CHP_CFG_CHP_CSR_CTRL_INT_COR_ALARM_DIS_LOC 0 +#define CHP_CFG_CHP_CSR_CTRL_INT_COR_SYND_DIS_LOC 1 +#define CHP_CFG_CHP_CSR_CTRL_INT_UNCR_ALARM_DIS_LOC 2 +#define CHP_CFG_CHP_CSR_CTRL_INT_UNC_SYND_DIS_LOC 3 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF0_ALARM_DIS_LOC 4 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF0_SYND_DIS_LOC 5 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF1_ALARM_DIS_LOC 6 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF1_SYND_DIS_LOC 7 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF2_ALARM_DIS_LOC 8 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF2_SYND_DIS_LOC 9 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF3_ALARM_DIS_LOC 10 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF3_SYND_DIS_LOC 11 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF4_ALARM_DIS_LOC 12 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF4_SYND_DIS_LOC 13 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF5_ALARM_DIS_LOC 14 +#define CHP_CFG_CHP_CSR_CTRL_INT_INF5_SYND_DIS_LOC 15 +#define CHP_CFG_CHP_CSR_CTRL_DLB_COR_ALARM_ENABLE_LOC 16 +#define CHP_CFG_CHP_CSR_CTRL_CFG_64BYTES_QE_LDB_CQ_MODE_LOC 17 +#define CHP_CFG_CHP_CSR_CTRL_CFG_64BYTES_QE_DIR_CQ_MODE_LOC 18 +#define CHP_CFG_CHP_CSR_CTRL_PAD_WRITE_LDB_LOC 19 +#define CHP_CFG_CHP_CSR_CTRL_PAD_WRITE_DIR_LOC 20 +#define CHP_CFG_CHP_CSR_CTRL_PAD_FIRST_WRITE_LDB_LOC 21 +#define CHP_CFG_CHP_CSR_CTRL_PAD_FIRST_WRITE_DIR_LOC 22 +#define CHP_CFG_CHP_CSR_CTRL_RSVZ0_LOC 23 + +#define CHP_DIR_CQ_INTR_ARMED0 0x4400005c +#define CHP_DIR_CQ_INTR_ARMED0_RST 0x0 + +#define CHP_DIR_CQ_INTR_ARMED0_ARMED 0xFFFFFFFF +#define CHP_DIR_CQ_INTR_ARMED0_ARMED_LOC 0 + +#define CHP_DIR_CQ_INTR_ARMED1 0x44000060 +#define CHP_DIR_CQ_INTR_ARMED1_RST 0x0 + +#define CHP_DIR_CQ_INTR_ARMED1_ARMED 0xFFFFFFFF +#define CHP_DIR_CQ_INTR_ARMED1_ARMED_LOC 0 + +#define CHP_CFG_DIR_CQ_TIMER_CTL 0x44000084 +#define CHP_CFG_DIR_CQ_TIMER_CTL_RST 0x0 + +#define CHP_CFG_DIR_CQ_TIMER_CTL_SAMPLE_INTERVAL 0x000000FF +#define CHP_CFG_DIR_CQ_TIMER_CTL_ENB 0x00000100 +#define CHP_CFG_DIR_CQ_TIMER_CTL_RSVZ0 0xFFFFFE00 +#define CHP_CFG_DIR_CQ_TIMER_CTL_SAMPLE_INTERVAL_LOC 0 +#define CHP_CFG_DIR_CQ_TIMER_CTL_ENB_LOC 8 +#define CHP_CFG_DIR_CQ_TIMER_CTL_RSVZ0_LOC 9 + +#define CHP_CFG_DIR_WDTO_0 0x44000088 +#define CHP_CFG_DIR_WDTO_0_RST 0x0 + +#define CHP_CFG_DIR_WDTO_0_WDTO 0xFFFFFFFF +#define CHP_CFG_DIR_WDTO_0_WDTO_LOC 0 + +#define CHP_CFG_DIR_WDTO_1 0x4400008c +#define CHP_CFG_DIR_WDTO_1_RST 0x0 + +#define CHP_CFG_DIR_WDTO_1_WDTO 0xFFFFFFFF +#define CHP_CFG_DIR_WDTO_1_WDTO_LOC 0 + +#define CHP_CFG_DIR_WD_DISABLE0 0x44000098 +#define CHP_CFG_DIR_WD_DISABLE0_RST 0xffffffff + +#define CHP_CFG_DIR_WD_DISABLE0_WD_DISABLE 0xFFFFFFFF +#define CHP_CFG_DIR_WD_DISABLE0_WD_DISABLE_LOC 0 + +#define CHP_CFG_DIR_WD_DISABLE1 0x4400009c +#define CHP_CFG_DIR_WD_DISABLE1_RST 0xffffffff + +#define CHP_CFG_DIR_WD_DISABLE1_WD_DISABLE 0xFFFFFFFF +#define CHP_CFG_DIR_WD_DISABLE1_WD_DISABLE_LOC 0 + +#define CHP_CFG_DIR_WD_ENB_INTERVAL 0x440000a0 +#define CHP_CFG_DIR_WD_ENB_INTERVAL_RST 0x0 + +#define CHP_CFG_DIR_WD_ENB_INTERVAL_SAMPLE_INTERVAL 0x0FFFFFFF +#define CHP_CFG_DIR_WD_ENB_INTERVAL_ENB 0x10000000 +#define CHP_CFG_DIR_WD_ENB_INTERVAL_RSVZ0 0xE0000000 +#define CHP_CFG_DIR_WD_ENB_INTERVAL_SAMPLE_INTERVAL_LOC 0 +#define CHP_CFG_DIR_WD_ENB_INTERVAL_ENB_LOC 28 +#define CHP_CFG_DIR_WD_ENB_INTERVAL_RSVZ0_LOC 29 + +#define CHP_CFG_DIR_WD_THRESHOLD 0x440000ac +#define CHP_CFG_DIR_WD_THRESHOLD_RST 0x0 + +#define CHP_CFG_DIR_WD_THRESHOLD_WD_THRESHOLD 0x000000FF +#define CHP_CFG_DIR_WD_THRESHOLD_RSVZ0 0xFFFFFF00 +#define CHP_CFG_DIR_WD_THRESHOLD_WD_THRESHOLD_LOC 0 +#define CHP_CFG_DIR_WD_THRESHOLD_RSVZ0_LOC 8 + +#define CHP_LDB_CQ_INTR_ARMED0 0x440000b0 +#define CHP_LDB_CQ_INTR_ARMED0_RST 0x0 + +#define CHP_LDB_CQ_INTR_ARMED0_ARMED 0xFFFFFFFF +#define CHP_LDB_CQ_INTR_ARMED0_ARMED_LOC 0 + +#define CHP_LDB_CQ_INTR_ARMED1 0x440000b4 +#define CHP_LDB_CQ_INTR_ARMED1_RST 0x0 + +#define CHP_LDB_CQ_INTR_ARMED1_ARMED 0xFFFFFFFF +#define CHP_LDB_CQ_INTR_ARMED1_ARMED_LOC 0 + +#define CHP_CFG_LDB_CQ_TIMER_CTL 0x440000d8 +#define CHP_CFG_LDB_CQ_TIMER_CTL_RST 0x0 + +#define CHP_CFG_LDB_CQ_TIMER_CTL_SAMPLE_INTERVAL 0x000000FF +#define CHP_CFG_LDB_CQ_TIMER_CTL_ENB 0x00000100 +#define CHP_CFG_LDB_CQ_TIMER_CTL_RSVZ0 0xFFFFFE00 +#define CHP_CFG_LDB_CQ_TIMER_CTL_SAMPLE_INTERVAL_LOC 0 +#define CHP_CFG_LDB_CQ_TIMER_CTL_ENB_LOC 8 +#define CHP_CFG_LDB_CQ_TIMER_CTL_RSVZ0_LOC 9 + +#define CHP_CFG_LDB_WDTO_0 0x440000dc +#define CHP_CFG_LDB_WDTO_0_RST 0x0 + +#define CHP_CFG_LDB_WDTO_0_WDTO 0xFFFFFFFF +#define CHP_CFG_LDB_WDTO_0_WDTO_LOC 0 + +#define CHP_CFG_LDB_WDTO_1 0x440000e0 +#define CHP_CFG_LDB_WDTO_1_RST 0x0 + +#define CHP_CFG_LDB_WDTO_1_WDTO 0xFFFFFFFF +#define CHP_CFG_LDB_WDTO_1_WDTO_LOC 0 + +#define CHP_CFG_LDB_WD_DISABLE0 0x440000ec +#define CHP_CFG_LDB_WD_DISABLE0_RST 0xffffffff + +#define CHP_CFG_LDB_WD_DISABLE0_WD_DISABLE 0xFFFFFFFF +#define CHP_CFG_LDB_WD_DISABLE0_WD_DISABLE_LOC 0 + +#define CHP_CFG_LDB_WD_DISABLE1 0x440000f0 +#define CHP_CFG_LDB_WD_DISABLE1_RST 0xffffffff + +#define CHP_CFG_LDB_WD_DISABLE1_WD_DISABLE 0xFFFFFFFF +#define CHP_CFG_LDB_WD_DISABLE1_WD_DISABLE_LOC 0 + +#define CHP_CFG_LDB_WD_ENB_INTERVAL 0x440000f4 +#define CHP_CFG_LDB_WD_ENB_INTERVAL_RST 0x0 + +#define CHP_CFG_LDB_WD_ENB_INTERVAL_SAMPLE_INTERVAL 0x0FFFFFFF +#define CHP_CFG_LDB_WD_ENB_INTERVAL_ENB 0x10000000 +#define CHP_CFG_LDB_WD_ENB_INTERVAL_RSVZ0 0xE0000000 +#define CHP_CFG_LDB_WD_ENB_INTERVAL_SAMPLE_INTERVAL_LOC 0 +#define CHP_CFG_LDB_WD_ENB_INTERVAL_ENB_LOC 28 +#define CHP_CFG_LDB_WD_ENB_INTERVAL_RSVZ0_LOC 29 + +#define CHP_CFG_LDB_WD_THRESHOLD 0x44000100 +#define CHP_CFG_LDB_WD_THRESHOLD_RST 0x0 + +#define CHP_CFG_LDB_WD_THRESHOLD_WD_THRESHOLD 0x000000FF +#define CHP_CFG_LDB_WD_THRESHOLD_RSVZ0 0xFFFFFF00 +#define CHP_CFG_LDB_WD_THRESHOLD_WD_THRESHOLD_LOC 0 +#define CHP_CFG_LDB_WD_THRESHOLD_RSVZ0_LOC 8 + +#define CHP_SMON_COMPARE0 0x4c000000 +#define CHP_SMON_COMPARE0_RST 0x0 + +#define CHP_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define CHP_SMON_COMPARE0_COMPARE0_LOC 0 + +#define CHP_SMON_COMPARE1 0x4c000004 +#define CHP_SMON_COMPARE1_RST 0x0 + +#define CHP_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define CHP_SMON_COMPARE1_COMPARE1_LOC 0 + +#define CHP_SMON_CFG0 0x4c000008 +#define CHP_SMON_CFG0_RST 0x40000000 + +#define CHP_SMON_CFG0_SMON_ENABLE 0x00000001 +#define CHP_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define CHP_SMON_CFG0_RSVZ0 0x0000000C +#define CHP_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define CHP_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define CHP_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define CHP_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define CHP_SMON_CFG0_SMON_MODE 0x0000F000 +#define CHP_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define CHP_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define CHP_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define CHP_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define CHP_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define CHP_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define CHP_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define CHP_SMON_CFG0_RSVZ1 0x00800000 +#define CHP_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define CHP_SMON_CFG0_RSVZ2 0x20000000 +#define CHP_SMON_CFG0_VERSION 0xC0000000 +#define CHP_SMON_CFG0_SMON_ENABLE_LOC 0 +#define CHP_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define CHP_SMON_CFG0_RSVZ0_LOC 2 +#define CHP_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define CHP_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define CHP_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define CHP_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define CHP_SMON_CFG0_SMON_MODE_LOC 12 +#define CHP_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define CHP_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define CHP_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define CHP_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define CHP_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define CHP_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define CHP_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define CHP_SMON_CFG0_RSVZ1_LOC 23 +#define CHP_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define CHP_SMON_CFG0_RSVZ2_LOC 29 +#define CHP_SMON_CFG0_VERSION_LOC 30 + +#define CHP_SMON_CFG1 0x4c00000c +#define CHP_SMON_CFG1_RST 0x0 + +#define CHP_SMON_CFG1_MODE0 0x000000FF +#define CHP_SMON_CFG1_MODE1 0x0000FF00 +#define CHP_SMON_CFG1_RSVZ0 0xFFFF0000 +#define CHP_SMON_CFG1_MODE0_LOC 0 +#define CHP_SMON_CFG1_MODE1_LOC 8 +#define CHP_SMON_CFG1_RSVZ0_LOC 16 + +#define CHP_SMON_ACTIVITYCNTR0 0x4c000010 +#define CHP_SMON_ACTIVITYCNTR0_RST 0x0 + +#define CHP_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define CHP_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define CHP_SMON_ACTIVITYCNTR1 0x4c000014 +#define CHP_SMON_ACTIVITYCNTR1_RST 0x0 -#define CHP_CFG_LDB_VAS_CRD(x) \ - (0x40080000 + (x) * 0x1000) -#define CHP_CFG_LDB_VAS_CRD_RST 0x0 +#define CHP_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define CHP_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 -#define CHP_CFG_LDB_VAS_CRD_COUNT 0x00007FFF -#define CHP_CFG_LDB_VAS_CRD_RSVD0 0xFFFF8000 -#define CHP_CFG_LDB_VAS_CRD_COUNT_LOC 0 -#define CHP_CFG_LDB_VAS_CRD_RSVD0_LOC 15 +#define CHP_SMON_MAX_TMR 0x4c000018 +#define CHP_SMON_MAX_TMR_RST 0x0 + +#define CHP_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define CHP_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define CHP_SMON_TMR 0x4c00001c +#define CHP_SMON_TMR_RST 0x0 + +#define CHP_SMON_TMR_TIMER 0xFFFFFFFF +#define CHP_SMON_TMR_TIMER_LOC 0 + +#define CHP_CTRL_DIAG_02 0x4c000028 +#define CHP_CTRL_DIAG_02_RST 0x1555 + +#define CHP_CTRL_DIAG_02_EGRESS_CREDIT_STATUS_EMPTY 0x00000001 +#define CHP_CTRL_DIAG_02_EGRESS_CREDIT_STATUS_AFULL 0x00000002 +#define CHP_CTRL_DIAG_02_CHP_OUTBOUND_HCW_PIPE_CREDIT_STATUS_EMPTY 0x00000004 +#define CHP_CTRL_DIAG_02_CHP_OUTBOUND_HCW_PIPE_CREDIT_STATUS_AFULL 0x00000008 +#define CHP_CTRL_DIAG_02_CHP_LSP_AP_CMP_PIPE_CREDIT_STATUS_EMPTY 0x00000010 +#define CHP_CTRL_DIAG_02_CHP_LSP_AP_CMP_PIPE_CREDIT_STATUS_AFULL 0x00000020 +#define CHP_CTRL_DIAG_02_CHP_LSP_TOK_PIPE_CREDIT_STATUS_EMPTY 0x00000040 +#define CHP_CTRL_DIAG_02_CHP_LSP_TOK_PIPE_CREDIT_STATUS_AFULL 0x00000080 +#define CHP_CTRL_DIAG_02_CHP_ROP_PIPE_CREDIT_STATUS_EMPTY 0x00000100 +#define CHP_CTRL_DIAG_02_CHP_ROP_PIPE_CREDIT_STATUS_AFULL 0x00000200 +#define CHP_CTRL_DIAG_02_QED_TO_CQ_PIPE_CREDIT_STATUS_EMPTY 0x00000400 +#define CHP_CTRL_DIAG_02_QED_TO_CQ_PIPE_CREDIT_STATUS_AFULL 0x00000800 +#define CHP_CTRL_DIAG_02_EGRESS_LSP_TOKEN_CREDIT_STATUS_EMPTY 0x00001000 +#define CHP_CTRL_DIAG_02_EGRESS_LSP_TOKEN_CREDIT_STATUS_AFULL 0x00002000 +#define CHP_CTRL_DIAG_02_RSVD0 0xFFFFC000 +#define CHP_CTRL_DIAG_02_EGRESS_CREDIT_STATUS_EMPTY_LOC 0 +#define CHP_CTRL_DIAG_02_EGRESS_CREDIT_STATUS_AFULL_LOC 1 +#define CHP_CTRL_DIAG_02_CHP_OUTBOUND_HCW_PIPE_CREDIT_STATUS_EMPTY_LOC 2 +#define CHP_CTRL_DIAG_02_CHP_OUTBOUND_HCW_PIPE_CREDIT_STATUS_AFULL_LOC 3 +#define CHP_CTRL_DIAG_02_CHP_LSP_AP_CMP_PIPE_CREDIT_STATUS_EMPTY_LOC 4 +#define CHP_CTRL_DIAG_02_CHP_LSP_AP_CMP_PIPE_CREDIT_STATUS_AFULL_LOC 5 +#define CHP_CTRL_DIAG_02_CHP_LSP_TOK_PIPE_CREDIT_STATUS_EMPTY_LOC 6 +#define CHP_CTRL_DIAG_02_CHP_LSP_TOK_PIPE_CREDIT_STATUS_AFULL_LOC 7 +#define CHP_CTRL_DIAG_02_CHP_ROP_PIPE_CREDIT_STATUS_EMPTY_LOC 8 +#define CHP_CTRL_DIAG_02_CHP_ROP_PIPE_CREDIT_STATUS_AFULL_LOC 9 +#define CHP_CTRL_DIAG_02_QED_TO_CQ_PIPE_CREDIT_STATUS_EMPTY_LOC 10 +#define CHP_CTRL_DIAG_02_QED_TO_CQ_PIPE_CREDIT_STATUS_AFULL_LOC 11 +#define CHP_CTRL_DIAG_02_EGRESS_LSP_TOKEN_CREDIT_STATUS_EMPTY_LOC 12 +#define CHP_CTRL_DIAG_02_EGRESS_LSP_TOKEN_CREDIT_STATUS_AFULL_LOC 13 +#define CHP_CTRL_DIAG_02_RSVD0_LOC 14 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0 0x54000000 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_RST 0xfefcfaf8 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI0 0x000000FF +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI1 0x0000FF00 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI2 0x00FF0000 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI3 0xFF000000 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI0_LOC 0 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI1_LOC 8 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI2_LOC 16 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_0_PRI3_LOC 24 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_1 0x54000004 +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_1_RST 0x0 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_1_RSVZ0 0xFFFFFFFF +#define DP_CFG_ARB_WEIGHTS_TQPRI_DIR_1_RSVZ0_LOC 0 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0 0x54000008 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_RST 0xfefcfaf8 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI0 0x000000FF +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI1 0x0000FF00 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI2 0x00FF0000 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI3 0xFF000000 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI0_LOC 0 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI1_LOC 8 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI2_LOC 16 +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI3_LOC 24 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1 0x5400000c +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RST 0x0 + +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RSVZ0 0xFFFFFFFF +#define DP_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RSVZ0_LOC 0 + +#define DP_DIR_CSR_CTRL 0x54000010 +#define DP_DIR_CSR_CTRL_RST 0x0 + +#define DP_DIR_CSR_CTRL_INT_COR_ALARM_DIS 0x00000001 +#define DP_DIR_CSR_CTRL_INT_COR_SYND_DIS 0x00000002 +#define DP_DIR_CSR_CTRL_INT_UNCR_ALARM_DIS 0x00000004 +#define DP_DIR_CSR_CTRL_INT_UNC_SYND_DIS 0x00000008 +#define DP_DIR_CSR_CTRL_INT_INF0_ALARM_DIS 0x00000010 +#define DP_DIR_CSR_CTRL_INT_INF0_SYND_DIS 0x00000020 +#define DP_DIR_CSR_CTRL_INT_INF1_ALARM_DIS 0x00000040 +#define DP_DIR_CSR_CTRL_INT_INF1_SYND_DIS 0x00000080 +#define DP_DIR_CSR_CTRL_INT_INF2_ALARM_DIS 0x00000100 +#define DP_DIR_CSR_CTRL_INT_INF2_SYND_DIS 0x00000200 +#define DP_DIR_CSR_CTRL_INT_INF3_ALARM_DIS 0x00000400 +#define DP_DIR_CSR_CTRL_INT_INF3_SYND_DIS 0x00000800 +#define DP_DIR_CSR_CTRL_INT_INF4_ALARM_DIS 0x00001000 +#define DP_DIR_CSR_CTRL_INT_INF4_SYND_DIS 0x00002000 +#define DP_DIR_CSR_CTRL_INT_INF5_ALARM_DIS 0x00004000 +#define DP_DIR_CSR_CTRL_INT_INF5_SYND_DIS 0x00008000 +#define DP_DIR_CSR_CTRL_RSVZ0 0xFFFF0000 +#define DP_DIR_CSR_CTRL_INT_COR_ALARM_DIS_LOC 0 +#define DP_DIR_CSR_CTRL_INT_COR_SYND_DIS_LOC 1 +#define DP_DIR_CSR_CTRL_INT_UNCR_ALARM_DIS_LOC 2 +#define DP_DIR_CSR_CTRL_INT_UNC_SYND_DIS_LOC 3 +#define DP_DIR_CSR_CTRL_INT_INF0_ALARM_DIS_LOC 4 +#define DP_DIR_CSR_CTRL_INT_INF0_SYND_DIS_LOC 5 +#define DP_DIR_CSR_CTRL_INT_INF1_ALARM_DIS_LOC 6 +#define DP_DIR_CSR_CTRL_INT_INF1_SYND_DIS_LOC 7 +#define DP_DIR_CSR_CTRL_INT_INF2_ALARM_DIS_LOC 8 +#define DP_DIR_CSR_CTRL_INT_INF2_SYND_DIS_LOC 9 +#define DP_DIR_CSR_CTRL_INT_INF3_ALARM_DIS_LOC 10 +#define DP_DIR_CSR_CTRL_INT_INF3_SYND_DIS_LOC 11 +#define DP_DIR_CSR_CTRL_INT_INF4_ALARM_DIS_LOC 12 +#define DP_DIR_CSR_CTRL_INT_INF4_SYND_DIS_LOC 13 +#define DP_DIR_CSR_CTRL_INT_INF5_ALARM_DIS_LOC 14 +#define DP_DIR_CSR_CTRL_INT_INF5_SYND_DIS_LOC 15 +#define DP_DIR_CSR_CTRL_RSVZ0_LOC 16 + +#define DP_SMON_ACTIVITYCNTR0 0x5c000058 +#define DP_SMON_ACTIVITYCNTR0_RST 0x0 + +#define DP_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define DP_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define DP_SMON_ACTIVITYCNTR1 0x5c00005c +#define DP_SMON_ACTIVITYCNTR1_RST 0x0 + +#define DP_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define DP_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define DP_SMON_COMPARE0 0x5c000060 +#define DP_SMON_COMPARE0_RST 0x0 + +#define DP_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define DP_SMON_COMPARE0_COMPARE0_LOC 0 + +#define DP_SMON_COMPARE1 0x5c000064 +#define DP_SMON_COMPARE1_RST 0x0 + +#define DP_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define DP_SMON_COMPARE1_COMPARE1_LOC 0 + +#define DP_SMON_CFG0 0x5c000068 +#define DP_SMON_CFG0_RST 0x40000000 + +#define DP_SMON_CFG0_SMON_ENABLE 0x00000001 +#define DP_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define DP_SMON_CFG0_RSVZ0 0x0000000C +#define DP_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define DP_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define DP_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define DP_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define DP_SMON_CFG0_SMON_MODE 0x0000F000 +#define DP_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define DP_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define DP_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define DP_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define DP_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define DP_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define DP_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define DP_SMON_CFG0_RSVZ1 0x00800000 +#define DP_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define DP_SMON_CFG0_RSVZ2 0x20000000 +#define DP_SMON_CFG0_VERSION 0xC0000000 +#define DP_SMON_CFG0_SMON_ENABLE_LOC 0 +#define DP_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define DP_SMON_CFG0_RSVZ0_LOC 2 +#define DP_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define DP_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define DP_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define DP_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define DP_SMON_CFG0_SMON_MODE_LOC 12 +#define DP_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define DP_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define DP_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define DP_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define DP_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define DP_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define DP_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define DP_SMON_CFG0_RSVZ1_LOC 23 +#define DP_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define DP_SMON_CFG0_RSVZ2_LOC 29 +#define DP_SMON_CFG0_VERSION_LOC 30 + +#define DP_SMON_CFG1 0x5c00006c +#define DP_SMON_CFG1_RST 0x0 + +#define DP_SMON_CFG1_MODE0 0x000000FF +#define DP_SMON_CFG1_MODE1 0x0000FF00 +#define DP_SMON_CFG1_RSVZ0 0xFFFF0000 +#define DP_SMON_CFG1_MODE0_LOC 0 +#define DP_SMON_CFG1_MODE1_LOC 8 +#define DP_SMON_CFG1_RSVZ0_LOC 16 + +#define DP_SMON_MAX_TMR 0x5c000070 +#define DP_SMON_MAX_TMR_RST 0x0 + +#define DP_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define DP_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define DP_SMON_TMR 0x5c000074 +#define DP_SMON_TMR_RST 0x0 + +#define DP_SMON_TMR_TIMER 0xFFFFFFFF +#define DP_SMON_TMR_TIMER_LOC 0 + +#define DQED_SMON_ACTIVITYCNTR0 0x6c000024 +#define DQED_SMON_ACTIVITYCNTR0_RST 0x0 + +#define DQED_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define DQED_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define DQED_SMON_ACTIVITYCNTR1 0x6c000028 +#define DQED_SMON_ACTIVITYCNTR1_RST 0x0 + +#define DQED_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define DQED_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define DQED_SMON_COMPARE0 0x6c00002c +#define DQED_SMON_COMPARE0_RST 0x0 + +#define DQED_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define DQED_SMON_COMPARE0_COMPARE0_LOC 0 + +#define DQED_SMON_COMPARE1 0x6c000030 +#define DQED_SMON_COMPARE1_RST 0x0 + +#define DQED_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define DQED_SMON_COMPARE1_COMPARE1_LOC 0 + +#define DQED_SMON_CFG0 0x6c000034 +#define DQED_SMON_CFG0_RST 0x40000000 + +#define DQED_SMON_CFG0_SMON_ENABLE 0x00000001 +#define DQED_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define DQED_SMON_CFG0_RSVZ0 0x0000000C +#define DQED_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define DQED_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define DQED_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define DQED_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define DQED_SMON_CFG0_SMON_MODE 0x0000F000 +#define DQED_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define DQED_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define DQED_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define DQED_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define DQED_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define DQED_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define DQED_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define DQED_SMON_CFG0_RSVZ1 0x00800000 +#define DQED_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define DQED_SMON_CFG0_RSVZ2 0x20000000 +#define DQED_SMON_CFG0_VERSION 0xC0000000 +#define DQED_SMON_CFG0_SMON_ENABLE_LOC 0 +#define DQED_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define DQED_SMON_CFG0_RSVZ0_LOC 2 +#define DQED_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define DQED_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define DQED_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define DQED_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define DQED_SMON_CFG0_SMON_MODE_LOC 12 +#define DQED_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define DQED_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define DQED_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define DQED_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define DQED_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define DQED_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define DQED_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define DQED_SMON_CFG0_RSVZ1_LOC 23 +#define DQED_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define DQED_SMON_CFG0_RSVZ2_LOC 29 +#define DQED_SMON_CFG0_VERSION_LOC 30 + +#define DQED_SMON_CFG1 0x6c000038 +#define DQED_SMON_CFG1_RST 0x0 + +#define DQED_SMON_CFG1_MODE0 0x000000FF +#define DQED_SMON_CFG1_MODE1 0x0000FF00 +#define DQED_SMON_CFG1_RSVZ0 0xFFFF0000 +#define DQED_SMON_CFG1_MODE0_LOC 0 +#define DQED_SMON_CFG1_MODE1_LOC 8 +#define DQED_SMON_CFG1_RSVZ0_LOC 16 + +#define DQED_SMON_MAX_TMR 0x6c00003c +#define DQED_SMON_MAX_TMR_RST 0x0 + +#define DQED_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define DQED_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define DQED_SMON_TMR 0x6c000040 +#define DQED_SMON_TMR_RST 0x0 + +#define DQED_SMON_TMR_TIMER 0xFFFFFFFF +#define DQED_SMON_TMR_TIMER_LOC 0 + +#define QED_SMON_ACTIVITYCNTR0 0x7c000024 +#define QED_SMON_ACTIVITYCNTR0_RST 0x0 + +#define QED_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define QED_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define QED_SMON_ACTIVITYCNTR1 0x7c000028 +#define QED_SMON_ACTIVITYCNTR1_RST 0x0 + +#define QED_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define QED_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define QED_SMON_COMPARE0 0x7c00002c +#define QED_SMON_COMPARE0_RST 0x0 + +#define QED_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define QED_SMON_COMPARE0_COMPARE0_LOC 0 + +#define QED_SMON_COMPARE1 0x7c000030 +#define QED_SMON_COMPARE1_RST 0x0 + +#define QED_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define QED_SMON_COMPARE1_COMPARE1_LOC 0 + +#define QED_SMON_CFG0 0x7c000034 +#define QED_SMON_CFG0_RST 0x40000000 + +#define QED_SMON_CFG0_SMON_ENABLE 0x00000001 +#define QED_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define QED_SMON_CFG0_RSVZ0 0x0000000C +#define QED_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define QED_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define QED_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define QED_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define QED_SMON_CFG0_SMON_MODE 0x0000F000 +#define QED_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define QED_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define QED_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define QED_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define QED_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define QED_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define QED_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define QED_SMON_CFG0_RSVZ1 0x00800000 +#define QED_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define QED_SMON_CFG0_RSVZ2 0x20000000 +#define QED_SMON_CFG0_VERSION 0xC0000000 +#define QED_SMON_CFG0_SMON_ENABLE_LOC 0 +#define QED_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define QED_SMON_CFG0_RSVZ0_LOC 2 +#define QED_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define QED_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define QED_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define QED_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define QED_SMON_CFG0_SMON_MODE_LOC 12 +#define QED_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define QED_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define QED_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define QED_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define QED_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define QED_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define QED_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define QED_SMON_CFG0_RSVZ1_LOC 23 +#define QED_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define QED_SMON_CFG0_RSVZ2_LOC 29 +#define QED_SMON_CFG0_VERSION_LOC 30 + +#define QED_SMON_CFG1 0x7c000038 +#define QED_SMON_CFG1_RST 0x0 + +#define QED_SMON_CFG1_MODE0 0x000000FF +#define QED_SMON_CFG1_MODE1 0x0000FF00 +#define QED_SMON_CFG1_RSVZ0 0xFFFF0000 +#define QED_SMON_CFG1_MODE0_LOC 0 +#define QED_SMON_CFG1_MODE1_LOC 8 +#define QED_SMON_CFG1_RSVZ0_LOC 16 + +#define QED_SMON_MAX_TMR 0x7c00003c +#define QED_SMON_MAX_TMR_RST 0x0 + +#define QED_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define QED_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define QED_SMON_TMR 0x7c000040 +#define QED_SMON_TMR_RST 0x0 + +#define QED_SMON_TMR_TIMER 0xFFFFFFFF +#define QED_SMON_TMR_TIMER_LOC 0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0 0x84000000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_RST 0xfefcfaf8 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI0 0x000000FF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI1 0x0000FF00 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI2 0x00FF0000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI3 0xFF000000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI0_LOC 0 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI1_LOC 8 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI2_LOC 16 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_0_PRI3_LOC 24 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_1 0x84000004 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_1_RST 0x0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_1_RSVZ0 0xFFFFFFFF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_ATQ_1_RSVZ0_LOC 0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0 0x84000008 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_RST 0xfefcfaf8 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI0 0x000000FF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI1 0x0000FF00 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI2 0x00FF0000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI3 0xFF000000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI0_LOC 0 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI1_LOC 8 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI2_LOC 16 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_0_PRI3_LOC 24 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_1 0x8400000c +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_1_RST 0x0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_1_RSVZ0 0xFFFFFFFF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_NALB_1_RSVZ0_LOC 0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0 0x84000010 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_RST 0xfefcfaf8 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI0 0x000000FF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI1 0x0000FF00 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI2 0x00FF0000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI3 0xFF000000 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI0_LOC 0 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI1_LOC 8 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI2_LOC 16 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_0_PRI3_LOC 24 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1 0x84000014 +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RST 0x0 + +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RSVZ0 0xFFFFFFFF +#define NALB_CFG_ARB_WEIGHTS_TQPRI_REPLAY_1_RSVZ0_LOC 0 + +#define NALB_SMON_ACTIVITYCNTR0 0x8c000064 +#define NALB_SMON_ACTIVITYCNTR0_RST 0x0 + +#define NALB_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define NALB_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define NALB_SMON_ACTIVITYCNTR1 0x8c000068 +#define NALB_SMON_ACTIVITYCNTR1_RST 0x0 + +#define NALB_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define NALB_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define NALB_SMON_COMPARE0 0x8c00006c +#define NALB_SMON_COMPARE0_RST 0x0 + +#define NALB_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define NALB_SMON_COMPARE0_COMPARE0_LOC 0 + +#define NALB_SMON_COMPARE1 0x8c000070 +#define NALB_SMON_COMPARE1_RST 0x0 + +#define NALB_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define NALB_SMON_COMPARE1_COMPARE1_LOC 0 + +#define NALB_SMON_CFG0 0x8c000074 +#define NALB_SMON_CFG0_RST 0x40000000 + +#define NALB_SMON_CFG0_SMON_ENABLE 0x00000001 +#define NALB_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define NALB_SMON_CFG0_RSVZ0 0x0000000C +#define NALB_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define NALB_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define NALB_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define NALB_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define NALB_SMON_CFG0_SMON_MODE 0x0000F000 +#define NALB_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define NALB_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define NALB_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define NALB_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define NALB_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define NALB_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define NALB_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define NALB_SMON_CFG0_RSVZ1 0x00800000 +#define NALB_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define NALB_SMON_CFG0_RSVZ2 0x20000000 +#define NALB_SMON_CFG0_VERSION 0xC0000000 +#define NALB_SMON_CFG0_SMON_ENABLE_LOC 0 +#define NALB_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define NALB_SMON_CFG0_RSVZ0_LOC 2 +#define NALB_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define NALB_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define NALB_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define NALB_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define NALB_SMON_CFG0_SMON_MODE_LOC 12 +#define NALB_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define NALB_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define NALB_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define NALB_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define NALB_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define NALB_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define NALB_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define NALB_SMON_CFG0_RSVZ1_LOC 23 +#define NALB_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define NALB_SMON_CFG0_RSVZ2_LOC 29 +#define NALB_SMON_CFG0_VERSION_LOC 30 + +#define NALB_SMON_CFG1 0x8c000078 +#define NALB_SMON_CFG1_RST 0x0 + +#define NALB_SMON_CFG1_MODE0 0x000000FF +#define NALB_SMON_CFG1_MODE1 0x0000FF00 +#define NALB_SMON_CFG1_RSVZ0 0xFFFF0000 +#define NALB_SMON_CFG1_MODE0_LOC 0 +#define NALB_SMON_CFG1_MODE1_LOC 8 +#define NALB_SMON_CFG1_RSVZ0_LOC 16 + +#define NALB_SMON_MAX_TMR 0x8c00007c +#define NALB_SMON_MAX_TMR_RST 0x0 + +#define NALB_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define NALB_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define NALB_SMON_TMR 0x8c000080 +#define NALB_SMON_TMR_RST 0x0 + +#define NALB_SMON_TMR_TIMER 0xFFFFFFFF +#define NALB_SMON_TMR_TIMER_LOC 0 + +#define RO_GRP_0_SLT_SHFT(x) \ + (0x96000000 + (x) * 0x4) +#define RO_GRP_0_SLT_SHFT_RST 0x0 + +#define RO_GRP_0_SLT_SHFT_CHANGE 0x000003FF +#define RO_GRP_0_SLT_SHFT_RSVD0 0xFFFFFC00 +#define RO_GRP_0_SLT_SHFT_CHANGE_LOC 0 +#define RO_GRP_0_SLT_SHFT_RSVD0_LOC 10 + +#define RO_GRP_1_SLT_SHFT(x) \ + (0x96010000 + (x) * 0x4) +#define RO_GRP_1_SLT_SHFT_RST 0x0 + +#define RO_GRP_1_SLT_SHFT_CHANGE 0x000003FF +#define RO_GRP_1_SLT_SHFT_RSVD0 0xFFFFFC00 +#define RO_GRP_1_SLT_SHFT_CHANGE_LOC 0 +#define RO_GRP_1_SLT_SHFT_RSVD0_LOC 10 + +#define RO_GRP_SN_MODE 0x94000000 +#define RO_GRP_SN_MODE_RST 0x0 + +#define RO_GRP_SN_MODE_SN_MODE_0 0x00000007 +#define RO_GRP_SN_MODE_RSZV0 0x000000F8 +#define RO_GRP_SN_MODE_SN_MODE_1 0x00000700 +#define RO_GRP_SN_MODE_RSZV1 0xFFFFF800 +#define RO_GRP_SN_MODE_SN_MODE_0_LOC 0 +#define RO_GRP_SN_MODE_RSZV0_LOC 3 +#define RO_GRP_SN_MODE_SN_MODE_1_LOC 8 +#define RO_GRP_SN_MODE_RSZV1_LOC 11 + +#define RO_CFG_CTRL_GENERAL_0 0x9c000000 +#define RO_CFG_CTRL_GENERAL_0_RST 0x0 + +#define RO_CFG_CTRL_GENERAL_0_UNIT_SINGLE_STEP_MODE 0x00000001 +#define RO_CFG_CTRL_GENERAL_0_RR_EN 0x00000002 +#define RO_CFG_CTRL_GENERAL_0_RSZV0 0xFFFFFFFC +#define RO_CFG_CTRL_GENERAL_0_UNIT_SINGLE_STEP_MODE_LOC 0 +#define RO_CFG_CTRL_GENERAL_0_RR_EN_LOC 1 +#define RO_CFG_CTRL_GENERAL_0_RSZV0_LOC 2 + +#define RO_SMON_ACTIVITYCNTR0 0x9c000030 +#define RO_SMON_ACTIVITYCNTR0_RST 0x0 + +#define RO_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define RO_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define RO_SMON_ACTIVITYCNTR1 0x9c000034 +#define RO_SMON_ACTIVITYCNTR1_RST 0x0 + +#define RO_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define RO_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define RO_SMON_COMPARE0 0x9c000038 +#define RO_SMON_COMPARE0_RST 0x0 + +#define RO_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define RO_SMON_COMPARE0_COMPARE0_LOC 0 + +#define RO_SMON_COMPARE1 0x9c00003c +#define RO_SMON_COMPARE1_RST 0x0 + +#define RO_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define RO_SMON_COMPARE1_COMPARE1_LOC 0 + +#define RO_SMON_CFG0 0x9c000040 +#define RO_SMON_CFG0_RST 0x40000000 + +#define RO_SMON_CFG0_SMON_ENABLE 0x00000001 +#define RO_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define RO_SMON_CFG0_RSVZ0 0x0000000C +#define RO_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define RO_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define RO_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define RO_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define RO_SMON_CFG0_SMON_MODE 0x0000F000 +#define RO_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define RO_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define RO_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define RO_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define RO_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define RO_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define RO_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define RO_SMON_CFG0_RSVZ1 0x00800000 +#define RO_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define RO_SMON_CFG0_RSVZ2 0x20000000 +#define RO_SMON_CFG0_VERSION 0xC0000000 +#define RO_SMON_CFG0_SMON_ENABLE_LOC 0 +#define RO_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define RO_SMON_CFG0_RSVZ0_LOC 2 +#define RO_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define RO_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define RO_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define RO_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define RO_SMON_CFG0_SMON_MODE_LOC 12 +#define RO_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define RO_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define RO_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define RO_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define RO_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define RO_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define RO_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define RO_SMON_CFG0_RSVZ1_LOC 23 +#define RO_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define RO_SMON_CFG0_RSVZ2_LOC 29 +#define RO_SMON_CFG0_VERSION_LOC 30 + +#define RO_SMON_CFG1 0x9c000044 +#define RO_SMON_CFG1_RST 0x0 + +#define RO_SMON_CFG1_MODE0 0x000000FF +#define RO_SMON_CFG1_MODE1 0x0000FF00 +#define RO_SMON_CFG1_RSVZ0 0xFFFF0000 +#define RO_SMON_CFG1_MODE0_LOC 0 +#define RO_SMON_CFG1_MODE1_LOC 8 +#define RO_SMON_CFG1_RSVZ0_LOC 16 + +#define RO_SMON_MAX_TMR 0x9c000048 +#define RO_SMON_MAX_TMR_RST 0x0 + +#define RO_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define RO_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define RO_SMON_TMR 0x9c00004c +#define RO_SMON_TMR_RST 0x0 + +#define RO_SMON_TMR_TIMER 0xFFFFFFFF +#define RO_SMON_TMR_TIMER_LOC 0 + +#define LSP_CQ2PRIOV(x) \ + (0xa0000000 + (x) * 0x1000) +#define LSP_CQ2PRIOV_RST 0x0 + +#define LSP_CQ2PRIOV_PRIO 0x00FFFFFF +#define LSP_CQ2PRIOV_V 0xFF000000 +#define LSP_CQ2PRIOV_PRIO_LOC 0 +#define LSP_CQ2PRIOV_V_LOC 24 + +#define LSP_CQ2QID0(x) \ + (0xa0080000 + (x) * 0x1000) +#define LSP_CQ2QID0_RST 0x0 + +#define LSP_CQ2QID0_QID_P0 0x0000007F +#define LSP_CQ2QID0_RSVD3 0x00000080 +#define LSP_CQ2QID0_QID_P1 0x00007F00 +#define LSP_CQ2QID0_RSVD2 0x00008000 +#define LSP_CQ2QID0_QID_P2 0x007F0000 +#define LSP_CQ2QID0_RSVD1 0x00800000 +#define LSP_CQ2QID0_QID_P3 0x7F000000 +#define LSP_CQ2QID0_RSVD0 0x80000000 +#define LSP_CQ2QID0_QID_P0_LOC 0 +#define LSP_CQ2QID0_RSVD3_LOC 7 +#define LSP_CQ2QID0_QID_P1_LOC 8 +#define LSP_CQ2QID0_RSVD2_LOC 15 +#define LSP_CQ2QID0_QID_P2_LOC 16 +#define LSP_CQ2QID0_RSVD1_LOC 23 +#define LSP_CQ2QID0_QID_P3_LOC 24 +#define LSP_CQ2QID0_RSVD0_LOC 31 + +#define LSP_CQ2QID1(x) \ + (0xa0100000 + (x) * 0x1000) +#define LSP_CQ2QID1_RST 0x0 + +#define LSP_CQ2QID1_QID_P4 0x0000007F +#define LSP_CQ2QID1_RSVD3 0x00000080 +#define LSP_CQ2QID1_QID_P5 0x00007F00 +#define LSP_CQ2QID1_RSVD2 0x00008000 +#define LSP_CQ2QID1_QID_P6 0x007F0000 +#define LSP_CQ2QID1_RSVD1 0x00800000 +#define LSP_CQ2QID1_QID_P7 0x7F000000 +#define LSP_CQ2QID1_RSVD0 0x80000000 +#define LSP_CQ2QID1_QID_P4_LOC 0 +#define LSP_CQ2QID1_RSVD3_LOC 7 +#define LSP_CQ2QID1_QID_P5_LOC 8 +#define LSP_CQ2QID1_RSVD2_LOC 15 +#define LSP_CQ2QID1_QID_P6_LOC 16 +#define LSP_CQ2QID1_RSVD1_LOC 23 +#define LSP_CQ2QID1_QID_P7_LOC 24 +#define LSP_CQ2QID1_RSVD0_LOC 31 + +#define LSP_CQ_DIR_DSBL(x) \ + (0xa0180000 + (x) * 0x1000) +#define LSP_CQ_DIR_DSBL_RST 0x1 + +#define LSP_CQ_DIR_DSBL_DISABLED 0x00000001 +#define LSP_CQ_DIR_DSBL_RSVD0 0xFFFFFFFE +#define LSP_CQ_DIR_DSBL_DISABLED_LOC 0 +#define LSP_CQ_DIR_DSBL_RSVD0_LOC 1 + +#define LSP_CQ_DIR_TKN_CNT(x) \ + (0xa0200000 + (x) * 0x1000) +#define LSP_CQ_DIR_TKN_CNT_RST 0x0 + +#define LSP_CQ_DIR_TKN_CNT_COUNT 0x00001FFF +#define LSP_CQ_DIR_TKN_CNT_RSVD0 0xFFFFE000 +#define LSP_CQ_DIR_TKN_CNT_COUNT_LOC 0 +#define LSP_CQ_DIR_TKN_CNT_RSVD0_LOC 13 + +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI(x) \ + (0xa0280000 + (x) * 0x1000) +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_RST 0x0 + +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_TOKEN_DEPTH_SELECT 0x0000000F +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_DISABLE_WB_OPT 0x00000010 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_IGNORE_DEPTH 0x00000020 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_RSVD0 0xFFFFFFC0 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_TOKEN_DEPTH_SELECT_LOC 0 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_DISABLE_WB_OPT_LOC 4 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_IGNORE_DEPTH_LOC 5 +#define LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_RSVD0_LOC 6 + +#define LSP_CQ_DIR_TOT_SCH_CNTL(x) \ + (0xa0300000 + (x) * 0x1000) +#define LSP_CQ_DIR_TOT_SCH_CNTL_RST 0x0 + +#define LSP_CQ_DIR_TOT_SCH_CNTL_COUNT 0xFFFFFFFF +#define LSP_CQ_DIR_TOT_SCH_CNTL_COUNT_LOC 0 + +#define LSP_CQ_DIR_TOT_SCH_CNTH(x) \ + (0xa0380000 + (x) * 0x1000) +#define LSP_CQ_DIR_TOT_SCH_CNTH_RST 0x0 + +#define LSP_CQ_DIR_TOT_SCH_CNTH_COUNT 0xFFFFFFFF +#define LSP_CQ_DIR_TOT_SCH_CNTH_COUNT_LOC 0 + +#define LSP_CQ_LDB_DSBL(x) \ + (0xa0400000 + (x) * 0x1000) +#define LSP_CQ_LDB_DSBL_RST 0x1 + +#define LSP_CQ_LDB_DSBL_DISABLED 0x00000001 +#define LSP_CQ_LDB_DSBL_RSVD0 0xFFFFFFFE +#define LSP_CQ_LDB_DSBL_DISABLED_LOC 0 +#define LSP_CQ_LDB_DSBL_RSVD0_LOC 1 + +#define LSP_CQ_LDB_INFL_CNT(x) \ + (0xa0480000 + (x) * 0x1000) +#define LSP_CQ_LDB_INFL_CNT_RST 0x0 + +#define LSP_CQ_LDB_INFL_CNT_COUNT 0x00000FFF +#define LSP_CQ_LDB_INFL_CNT_RSVD0 0xFFFFF000 +#define LSP_CQ_LDB_INFL_CNT_COUNT_LOC 0 +#define LSP_CQ_LDB_INFL_CNT_RSVD0_LOC 12 + +#define LSP_CQ_LDB_INFL_LIM(x) \ + (0xa0500000 + (x) * 0x1000) +#define LSP_CQ_LDB_INFL_LIM_RST 0x0 + +#define LSP_CQ_LDB_INFL_LIM_LIMIT 0x00000FFF +#define LSP_CQ_LDB_INFL_LIM_RSVD0 0xFFFFF000 +#define LSP_CQ_LDB_INFL_LIM_LIMIT_LOC 0 +#define LSP_CQ_LDB_INFL_LIM_RSVD0_LOC 12 + +#define LSP_CQ_LDB_TKN_CNT(x) \ + (0xa0580000 + (x) * 0x1000) +#define LSP_CQ_LDB_TKN_CNT_RST 0x0 + +#define LSP_CQ_LDB_TKN_CNT_TOKEN_COUNT 0x000007FF +#define LSP_CQ_LDB_TKN_CNT_RSVD0 0xFFFFF800 +#define LSP_CQ_LDB_TKN_CNT_TOKEN_COUNT_LOC 0 +#define LSP_CQ_LDB_TKN_CNT_RSVD0_LOC 11 + +#define LSP_CQ_LDB_TKN_DEPTH_SEL(x) \ + (0xa0600000 + (x) * 0x1000) +#define LSP_CQ_LDB_TKN_DEPTH_SEL_RST 0x0 + +#define LSP_CQ_LDB_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT 0x0000000F +#define LSP_CQ_LDB_TKN_DEPTH_SEL_IGNORE_DEPTH 0x00000010 +#define LSP_CQ_LDB_TKN_DEPTH_SEL_RSVD0 0xFFFFFFE0 +#define LSP_CQ_LDB_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT_LOC 0 +#define LSP_CQ_LDB_TKN_DEPTH_SEL_IGNORE_DEPTH_LOC 4 +#define LSP_CQ_LDB_TKN_DEPTH_SEL_RSVD0_LOC 5 + +#define LSP_CQ_LDB_TOT_SCH_CNTL(x) \ + (0xa0680000 + (x) * 0x1000) +#define LSP_CQ_LDB_TOT_SCH_CNTL_RST 0x0 + +#define LSP_CQ_LDB_TOT_SCH_CNTL_COUNT 0xFFFFFFFF +#define LSP_CQ_LDB_TOT_SCH_CNTL_COUNT_LOC 0 + +#define LSP_CQ_LDB_TOT_SCH_CNTH(x) \ + (0xa0700000 + (x) * 0x1000) +#define LSP_CQ_LDB_TOT_SCH_CNTH_RST 0x0 + +#define LSP_CQ_LDB_TOT_SCH_CNTH_COUNT 0xFFFFFFFF +#define LSP_CQ_LDB_TOT_SCH_CNTH_COUNT_LOC 0 + +#define LSP_QID_DIR_MAX_DEPTH(x) \ + (0xa0780000 + (x) * 0x1000) +#define LSP_QID_DIR_MAX_DEPTH_RST 0x0 + +#define LSP_QID_DIR_MAX_DEPTH_DEPTH 0x00001FFF +#define LSP_QID_DIR_MAX_DEPTH_RSVD0 0xFFFFE000 +#define LSP_QID_DIR_MAX_DEPTH_DEPTH_LOC 0 +#define LSP_QID_DIR_MAX_DEPTH_RSVD0_LOC 13 + +#define LSP_QID_DIR_TOT_ENQ_CNTL(x) \ + (0xa0800000 + (x) * 0x1000) +#define LSP_QID_DIR_TOT_ENQ_CNTL_RST 0x0 + +#define LSP_QID_DIR_TOT_ENQ_CNTL_COUNT 0xFFFFFFFF +#define LSP_QID_DIR_TOT_ENQ_CNTL_COUNT_LOC 0 + +#define LSP_QID_DIR_TOT_ENQ_CNTH(x) \ + (0xa0880000 + (x) * 0x1000) +#define LSP_QID_DIR_TOT_ENQ_CNTH_RST 0x0 + +#define LSP_QID_DIR_TOT_ENQ_CNTH_COUNT 0xFFFFFFFF +#define LSP_QID_DIR_TOT_ENQ_CNTH_COUNT_LOC 0 + +#define LSP_QID_DIR_ENQUEUE_CNT(x) \ + (0xa0900000 + (x) * 0x1000) +#define LSP_QID_DIR_ENQUEUE_CNT_RST 0x0 + +#define LSP_QID_DIR_ENQUEUE_CNT_COUNT 0x00001FFF +#define LSP_QID_DIR_ENQUEUE_CNT_RSVD0 0xFFFFE000 +#define LSP_QID_DIR_ENQUEUE_CNT_COUNT_LOC 0 +#define LSP_QID_DIR_ENQUEUE_CNT_RSVD0_LOC 13 + +#define LSP_QID_DIR_DEPTH_THRSH(x) \ + (0xa0980000 + (x) * 0x1000) +#define LSP_QID_DIR_DEPTH_THRSH_RST 0x0 + +#define LSP_QID_DIR_DEPTH_THRSH_THRESH 0x00001FFF +#define LSP_QID_DIR_DEPTH_THRSH_RSVD0 0xFFFFE000 +#define LSP_QID_DIR_DEPTH_THRSH_THRESH_LOC 0 +#define LSP_QID_DIR_DEPTH_THRSH_RSVD0_LOC 13 + +#define LSP_QID_AQED_ACTIVE_CNT(x) \ + (0xa0a00000 + (x) * 0x1000) +#define LSP_QID_AQED_ACTIVE_CNT_RST 0x0 + +#define LSP_QID_AQED_ACTIVE_CNT_COUNT 0x00000FFF +#define LSP_QID_AQED_ACTIVE_CNT_RSVD0 0xFFFFF000 +#define LSP_QID_AQED_ACTIVE_CNT_COUNT_LOC 0 +#define LSP_QID_AQED_ACTIVE_CNT_RSVD0_LOC 12 + +#define LSP_QID_AQED_ACTIVE_LIM(x) \ + (0xa0a80000 + (x) * 0x1000) +#define LSP_QID_AQED_ACTIVE_LIM_RST 0x0 + +#define LSP_QID_AQED_ACTIVE_LIM_LIMIT 0x00000FFF +#define LSP_QID_AQED_ACTIVE_LIM_RSVD0 0xFFFFF000 +#define LSP_QID_AQED_ACTIVE_LIM_LIMIT_LOC 0 +#define LSP_QID_AQED_ACTIVE_LIM_RSVD0_LOC 12 + +#define LSP_QID_ATM_TOT_ENQ_CNTL(x) \ + (0xa0b00000 + (x) * 0x1000) +#define LSP_QID_ATM_TOT_ENQ_CNTL_RST 0x0 + +#define LSP_QID_ATM_TOT_ENQ_CNTL_COUNT 0xFFFFFFFF +#define LSP_QID_ATM_TOT_ENQ_CNTL_COUNT_LOC 0 + +#define LSP_QID_ATM_TOT_ENQ_CNTH(x) \ + (0xa0b80000 + (x) * 0x1000) +#define LSP_QID_ATM_TOT_ENQ_CNTH_RST 0x0 + +#define LSP_QID_ATM_TOT_ENQ_CNTH_COUNT 0xFFFFFFFF +#define LSP_QID_ATM_TOT_ENQ_CNTH_COUNT_LOC 0 + +#define LSP_QID_ATQ_ENQUEUE_CNT(x) \ + (0xa0c00000 + (x) * 0x1000) +#define LSP_QID_ATQ_ENQUEUE_CNT_RST 0x0 + +#define LSP_QID_ATQ_ENQUEUE_CNT_COUNT 0x00003FFF +#define LSP_QID_ATQ_ENQUEUE_CNT_RSVD0 0xFFFFC000 +#define LSP_QID_ATQ_ENQUEUE_CNT_COUNT_LOC 0 +#define LSP_QID_ATQ_ENQUEUE_CNT_RSVD0_LOC 14 + +#define LSP_QID_LDB_ENQUEUE_CNT(x) \ + (0xa0c80000 + (x) * 0x1000) +#define LSP_QID_LDB_ENQUEUE_CNT_RST 0x0 + +#define LSP_QID_LDB_ENQUEUE_CNT_COUNT 0x00003FFF +#define LSP_QID_LDB_ENQUEUE_CNT_RSVD0 0xFFFFC000 +#define LSP_QID_LDB_ENQUEUE_CNT_COUNT_LOC 0 +#define LSP_QID_LDB_ENQUEUE_CNT_RSVD0_LOC 14 + +#define LSP_QID_LDB_INFL_CNT(x) \ + (0xa0d00000 + (x) * 0x1000) +#define LSP_QID_LDB_INFL_CNT_RST 0x0 + +#define LSP_QID_LDB_INFL_CNT_COUNT 0x00000FFF +#define LSP_QID_LDB_INFL_CNT_RSVD0 0xFFFFF000 +#define LSP_QID_LDB_INFL_CNT_COUNT_LOC 0 +#define LSP_QID_LDB_INFL_CNT_RSVD0_LOC 12 + +#define LSP_QID_LDB_INFL_LIM(x) \ + (0xa0d80000 + (x) * 0x1000) +#define LSP_QID_LDB_INFL_LIM_RST 0x0 + +#define LSP_QID_LDB_INFL_LIM_LIMIT 0x00000FFF +#define LSP_QID_LDB_INFL_LIM_RSVD0 0xFFFFF000 +#define LSP_QID_LDB_INFL_LIM_LIMIT_LOC 0 +#define LSP_QID_LDB_INFL_LIM_RSVD0_LOC 12 + +#define LSP_QID2CQIDIX_00(x) \ + (0xa0e00000 + (x) * 0x1000) +#define LSP_QID2CQIDIX_00_RST 0x0 +#define LSP_QID2CQIDIX(x, y) \ + (LSP_QID2CQIDIX_00(x) + 0x80000 * (y)) +#define LSP_QID2CQIDIX_NUM 16 + +#define LSP_QID2CQIDIX_00_CQ_P0 0x000000FF +#define LSP_QID2CQIDIX_00_CQ_P1 0x0000FF00 +#define LSP_QID2CQIDIX_00_CQ_P2 0x00FF0000 +#define LSP_QID2CQIDIX_00_CQ_P3 0xFF000000 +#define LSP_QID2CQIDIX_00_CQ_P0_LOC 0 +#define LSP_QID2CQIDIX_00_CQ_P1_LOC 8 +#define LSP_QID2CQIDIX_00_CQ_P2_LOC 16 +#define LSP_QID2CQIDIX_00_CQ_P3_LOC 24 + +#define LSP_QID2CQIDIX2_00(x) \ + (0xa1600000 + (x) * 0x1000) +#define LSP_QID2CQIDIX2_00_RST 0x0 +#define LSP_QID2CQIDIX2(x, y) \ + (LSP_QID2CQIDIX2_00(x) + 0x80000 * (y)) +#define LSP_QID2CQIDIX2_NUM 16 + +#define LSP_QID2CQIDIX2_00_CQ_P0 0x000000FF +#define LSP_QID2CQIDIX2_00_CQ_P1 0x0000FF00 +#define LSP_QID2CQIDIX2_00_CQ_P2 0x00FF0000 +#define LSP_QID2CQIDIX2_00_CQ_P3 0xFF000000 +#define LSP_QID2CQIDIX2_00_CQ_P0_LOC 0 +#define LSP_QID2CQIDIX2_00_CQ_P1_LOC 8 +#define LSP_QID2CQIDIX2_00_CQ_P2_LOC 16 +#define LSP_QID2CQIDIX2_00_CQ_P3_LOC 24 + +#define LSP_QID_LDB_REPLAY_CNT(x) \ + (0xa1e00000 + (x) * 0x1000) +#define LSP_QID_LDB_REPLAY_CNT_RST 0x0 + +#define LSP_QID_LDB_REPLAY_CNT_COUNT 0x00003FFF +#define LSP_QID_LDB_REPLAY_CNT_RSVD0 0xFFFFC000 +#define LSP_QID_LDB_REPLAY_CNT_COUNT_LOC 0 +#define LSP_QID_LDB_REPLAY_CNT_RSVD0_LOC 14 + +#define LSP_QID_NALDB_MAX_DEPTH(x) \ + (0xa1f00000 + (x) * 0x1000) +#define LSP_QID_NALDB_MAX_DEPTH_RST 0x0 + +#define LSP_QID_NALDB_MAX_DEPTH_DEPTH 0x00003FFF +#define LSP_QID_NALDB_MAX_DEPTH_RSVD0 0xFFFFC000 +#define LSP_QID_NALDB_MAX_DEPTH_DEPTH_LOC 0 +#define LSP_QID_NALDB_MAX_DEPTH_RSVD0_LOC 14 + +#define LSP_QID_NALDB_TOT_ENQ_CNTL(x) \ + (0xa1f80000 + (x) * 0x1000) +#define LSP_QID_NALDB_TOT_ENQ_CNTL_RST 0x0 + +#define LSP_QID_NALDB_TOT_ENQ_CNTL_COUNT 0xFFFFFFFF +#define LSP_QID_NALDB_TOT_ENQ_CNTL_COUNT_LOC 0 + +#define LSP_QID_NALDB_TOT_ENQ_CNTH(x) \ + (0xa2000000 + (x) * 0x1000) +#define LSP_QID_NALDB_TOT_ENQ_CNTH_RST 0x0 + +#define LSP_QID_NALDB_TOT_ENQ_CNTH_COUNT 0xFFFFFFFF +#define LSP_QID_NALDB_TOT_ENQ_CNTH_COUNT_LOC 0 + +#define LSP_QID_ATM_DEPTH_THRSH(x) \ + (0xa2080000 + (x) * 0x1000) +#define LSP_QID_ATM_DEPTH_THRSH_RST 0x0 + +#define LSP_QID_ATM_DEPTH_THRSH_THRESH 0x00003FFF +#define LSP_QID_ATM_DEPTH_THRSH_RSVD0 0xFFFFC000 +#define LSP_QID_ATM_DEPTH_THRSH_THRESH_LOC 0 +#define LSP_QID_ATM_DEPTH_THRSH_RSVD0_LOC 14 + +#define LSP_QID_NALDB_DEPTH_THRSH(x) \ + (0xa2100000 + (x) * 0x1000) +#define LSP_QID_NALDB_DEPTH_THRSH_RST 0x0 + +#define LSP_QID_NALDB_DEPTH_THRSH_THRESH 0x00003FFF +#define LSP_QID_NALDB_DEPTH_THRSH_RSVD0 0xFFFFC000 +#define LSP_QID_NALDB_DEPTH_THRSH_THRESH_LOC 0 +#define LSP_QID_NALDB_DEPTH_THRSH_RSVD0_LOC 14 + +#define LSP_QID_ATM_ACTIVE(x) \ + (0xa2180000 + (x) * 0x1000) +#define LSP_QID_ATM_ACTIVE_RST 0x0 + +#define LSP_QID_ATM_ACTIVE_COUNT 0x00003FFF +#define LSP_QID_ATM_ACTIVE_RSVD0 0xFFFFC000 +#define LSP_QID_ATM_ACTIVE_COUNT_LOC 0 +#define LSP_QID_ATM_ACTIVE_RSVD0_LOC 14 + +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0 0xa4000008 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_RST 0x0 + +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI0_WEIGHT 0x000000FF +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI1_WEIGHT 0x0000FF00 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI2_WEIGHT 0x00FF0000 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI3_WEIGHT 0xFF000000 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI0_WEIGHT_LOC 0 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI1_WEIGHT_LOC 8 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI2_WEIGHT_LOC 16 +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_0_PRI3_WEIGHT_LOC 24 + +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_1 0xa400000c +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_1_RST 0x0 + +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_1_RSVZ0 0xFFFFFFFF +#define LSP_CFG_ARB_WEIGHT_ATM_NALB_QID_1_RSVZ0_LOC 0 + +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0 0xa4000014 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_RST 0x0 + +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI0_WEIGHT 0x000000FF +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI1_WEIGHT 0x0000FF00 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI2_WEIGHT 0x00FF0000 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI3_WEIGHT 0xFF000000 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI0_WEIGHT_LOC 0 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI1_WEIGHT_LOC 8 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI2_WEIGHT_LOC 16 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_0_PRI3_WEIGHT_LOC 24 + +#define LSP_CFG_ARB_WEIGHT_LDB_QID_1 0xa4000018 +#define LSP_CFG_ARB_WEIGHT_LDB_QID_1_RST 0x0 + +#define LSP_CFG_ARB_WEIGHT_LDB_QID_1_RSVZ0 0xFFFFFFFF +#define LSP_CFG_ARB_WEIGHT_LDB_QID_1_RSVZ0_LOC 0 + +#define LSP_LDB_SCHED_CTRL 0xa400002c +#define LSP_LDB_SCHED_CTRL_RST 0x0 + +#define LSP_LDB_SCHED_CTRL_CQ 0x000000FF +#define LSP_LDB_SCHED_CTRL_QIDIX 0x00000700 +#define LSP_LDB_SCHED_CTRL_VALUE 0x00000800 +#define LSP_LDB_SCHED_CTRL_NALB_HASWORK_V 0x00001000 +#define LSP_LDB_SCHED_CTRL_RLIST_HASWORK_V 0x00002000 +#define LSP_LDB_SCHED_CTRL_SLIST_HASWORK_V 0x00004000 +#define LSP_LDB_SCHED_CTRL_INFLIGHT_OK_V 0x00008000 +#define LSP_LDB_SCHED_CTRL_AQED_NFULL_V 0x00010000 +#define LSP_LDB_SCHED_CTRL_RSVZ0 0xFFFE0000 +#define LSP_LDB_SCHED_CTRL_CQ_LOC 0 +#define LSP_LDB_SCHED_CTRL_QIDIX_LOC 8 +#define LSP_LDB_SCHED_CTRL_VALUE_LOC 11 +#define LSP_LDB_SCHED_CTRL_NALB_HASWORK_V_LOC 12 +#define LSP_LDB_SCHED_CTRL_RLIST_HASWORK_V_LOC 13 +#define LSP_LDB_SCHED_CTRL_SLIST_HASWORK_V_LOC 14 +#define LSP_LDB_SCHED_CTRL_INFLIGHT_OK_V_LOC 15 +#define LSP_LDB_SCHED_CTRL_AQED_NFULL_V_LOC 16 +#define LSP_LDB_SCHED_CTRL_RSVZ0_LOC 17 + +#define LSP_DIR_SCH_CNT_L 0xa4000034 +#define LSP_DIR_SCH_CNT_L_RST 0x0 + +#define LSP_DIR_SCH_CNT_L_COUNT 0xFFFFFFFF +#define LSP_DIR_SCH_CNT_L_COUNT_LOC 0 + +#define LSP_DIR_SCH_CNT_H 0xa4000038 +#define LSP_DIR_SCH_CNT_H_RST 0x0 + +#define LSP_DIR_SCH_CNT_H_COUNT 0xFFFFFFFF +#define LSP_DIR_SCH_CNT_H_COUNT_LOC 0 + +#define LSP_LDB_SCH_CNT_L 0xa400003c +#define LSP_LDB_SCH_CNT_L_RST 0x0 + +#define LSP_LDB_SCH_CNT_L_COUNT 0xFFFFFFFF +#define LSP_LDB_SCH_CNT_L_COUNT_LOC 0 + +#define LSP_LDB_SCH_CNT_H 0xa4000040 +#define LSP_LDB_SCH_CNT_H_RST 0x0 + +#define LSP_LDB_SCH_CNT_H_COUNT 0xFFFFFFFF +#define LSP_LDB_SCH_CNT_H_COUNT_LOC 0 + +#define LSP_CFG_SHDW_CTRL 0xa4000070 +#define LSP_CFG_SHDW_CTRL_RST 0x0 + +#define LSP_CFG_SHDW_CTRL_TRANSFER 0x00000001 +#define LSP_CFG_SHDW_CTRL_RSVD0 0xFFFFFFFE +#define LSP_CFG_SHDW_CTRL_TRANSFER_LOC 0 +#define LSP_CFG_SHDW_CTRL_RSVD0_LOC 1 + +#define LSP_CFG_SHDW_RANGE_COS(x) \ + (0xa4000074 + (x) * 4) +#define LSP_CFG_SHDW_RANGE_COS_RST 0x40 + +#define LSP_CFG_SHDW_RANGE_COS_BW_RANGE 0x000001FF +#define LSP_CFG_SHDW_RANGE_COS_RSVZ0 0x7FFFFE00 +#define LSP_CFG_SHDW_RANGE_COS_NO_EXTRA_CREDIT 0x80000000 +#define LSP_CFG_SHDW_RANGE_COS_BW_RANGE_LOC 0 +#define LSP_CFG_SHDW_RANGE_COS_RSVZ0_LOC 9 +#define LSP_CFG_SHDW_RANGE_COS_NO_EXTRA_CREDIT_LOC 31 + +#define LSP_CFG_CTRL_GENERAL_0 0xac000000 +#define LSP_CFG_CTRL_GENERAL_0_RST 0x0 + +#define LSP_CFG_CTRL_GENERAL_0_DISAB_ATQ_EMPTY_ARB 0x00000001 +#define LSP_CFG_CTRL_GENERAL_0_INC_TOK_UNIT_IDLE 0x00000002 +#define LSP_CFG_CTRL_GENERAL_0_DISAB_RLIST_PRI 0x00000004 +#define LSP_CFG_CTRL_GENERAL_0_INC_CMP_UNIT_IDLE 0x00000008 +#define LSP_CFG_CTRL_GENERAL_0_RSVZ0 0x00000030 +#define LSP_CFG_CTRL_GENERAL_0_DIR_SINGLE_OP 0x00000040 +#define LSP_CFG_CTRL_GENERAL_0_DIR_HALF_BW 0x00000080 +#define LSP_CFG_CTRL_GENERAL_0_DIR_SINGLE_OUT 0x00000100 +#define LSP_CFG_CTRL_GENERAL_0_DIR_DISAB_MULTI 0x00000200 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_SINGLE_OP 0x00000400 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_HALF_BW 0x00000800 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_SINGLE_OUT 0x00001000 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_DISAB_MULTI 0x00002000 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_SINGLE_OP 0x00004000 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_HALF_BW 0x00008000 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_SINGLE_OUT 0x00010000 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_SINGLE_OP 0x00020000 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_HALF_BW 0x00040000 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_SINGLE_OUT 0x00080000 +#define LSP_CFG_CTRL_GENERAL_0_LDB_SINGLE_OP 0x00100000 +#define LSP_CFG_CTRL_GENERAL_0_LDB_HALF_BW 0x00200000 +#define LSP_CFG_CTRL_GENERAL_0_LDB_DISAB_MULTI 0x00400000 +#define LSP_CFG_CTRL_GENERAL_0_ATM_SINGLE_SCH 0x00800000 +#define LSP_CFG_CTRL_GENERAL_0_ATM_SINGLE_CMP 0x01000000 +#define LSP_CFG_CTRL_GENERAL_0_LDB_CE_TOG_ARB 0x02000000 +#define LSP_CFG_CTRL_GENERAL_0_RSVZ1 0x04000000 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_VALID_SEL 0x18000000 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_VALUE_SEL 0x20000000 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_COMPARE_SEL 0xC0000000 +#define LSP_CFG_CTRL_GENERAL_0_DISAB_ATQ_EMPTY_ARB_LOC 0 +#define LSP_CFG_CTRL_GENERAL_0_INC_TOK_UNIT_IDLE_LOC 1 +#define LSP_CFG_CTRL_GENERAL_0_DISAB_RLIST_PRI_LOC 2 +#define LSP_CFG_CTRL_GENERAL_0_INC_CMP_UNIT_IDLE_LOC 3 +#define LSP_CFG_CTRL_GENERAL_0_RSVZ0_LOC 4 +#define LSP_CFG_CTRL_GENERAL_0_DIR_SINGLE_OP_LOC 6 +#define LSP_CFG_CTRL_GENERAL_0_DIR_HALF_BW_LOC 7 +#define LSP_CFG_CTRL_GENERAL_0_DIR_SINGLE_OUT_LOC 8 +#define LSP_CFG_CTRL_GENERAL_0_DIR_DISAB_MULTI_LOC 9 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_SINGLE_OP_LOC 10 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_HALF_BW_LOC 11 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_SINGLE_OUT_LOC 12 +#define LSP_CFG_CTRL_GENERAL_0_ATQ_DISAB_MULTI_LOC 13 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_SINGLE_OP_LOC 14 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_HALF_BW_LOC 15 +#define LSP_CFG_CTRL_GENERAL_0_DIRRPL_SINGLE_OUT_LOC 16 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_SINGLE_OP_LOC 17 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_HALF_BW_LOC 18 +#define LSP_CFG_CTRL_GENERAL_0_LBRPL_SINGLE_OUT_LOC 19 +#define LSP_CFG_CTRL_GENERAL_0_LDB_SINGLE_OP_LOC 20 +#define LSP_CFG_CTRL_GENERAL_0_LDB_HALF_BW_LOC 21 +#define LSP_CFG_CTRL_GENERAL_0_LDB_DISAB_MULTI_LOC 22 +#define LSP_CFG_CTRL_GENERAL_0_ATM_SINGLE_SCH_LOC 23 +#define LSP_CFG_CTRL_GENERAL_0_ATM_SINGLE_CMP_LOC 24 +#define LSP_CFG_CTRL_GENERAL_0_LDB_CE_TOG_ARB_LOC 25 +#define LSP_CFG_CTRL_GENERAL_0_RSVZ1_LOC 26 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_VALID_SEL_LOC 27 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_VALUE_SEL_LOC 29 +#define LSP_CFG_CTRL_GENERAL_0_SMON0_COMPARE_SEL_LOC 30 + +#define LSP_SMON_COMPARE0 0xac000048 +#define LSP_SMON_COMPARE0_RST 0x0 + +#define LSP_SMON_COMPARE0_COMPARE0 0xFFFFFFFF +#define LSP_SMON_COMPARE0_COMPARE0_LOC 0 + +#define LSP_SMON_COMPARE1 0xac00004c +#define LSP_SMON_COMPARE1_RST 0x0 + +#define LSP_SMON_COMPARE1_COMPARE1 0xFFFFFFFF +#define LSP_SMON_COMPARE1_COMPARE1_LOC 0 + +#define LSP_SMON_CFG0 0xac000050 +#define LSP_SMON_CFG0_RST 0x40000000 + +#define LSP_SMON_CFG0_SMON_ENABLE 0x00000001 +#define LSP_SMON_CFG0_SMON_0TRIGGER_ENABLE 0x00000002 +#define LSP_SMON_CFG0_RSVZ0 0x0000000C +#define LSP_SMON_CFG0_SMON0_FUNCTION 0x00000070 +#define LSP_SMON_CFG0_SMON0_FUNCTION_COMPARE 0x00000080 +#define LSP_SMON_CFG0_SMON1_FUNCTION 0x00000700 +#define LSP_SMON_CFG0_SMON1_FUNCTION_COMPARE 0x00000800 +#define LSP_SMON_CFG0_SMON_MODE 0x0000F000 +#define LSP_SMON_CFG0_STOPCOUNTEROVFL 0x00010000 +#define LSP_SMON_CFG0_INTCOUNTEROVFL 0x00020000 +#define LSP_SMON_CFG0_STATCOUNTER0OVFL 0x00040000 +#define LSP_SMON_CFG0_STATCOUNTER1OVFL 0x00080000 +#define LSP_SMON_CFG0_STOPTIMEROVFL 0x00100000 +#define LSP_SMON_CFG0_INTTIMEROVFL 0x00200000 +#define LSP_SMON_CFG0_STATTIMEROVFL 0x00400000 +#define LSP_SMON_CFG0_RSVZ1 0x00800000 +#define LSP_SMON_CFG0_TIMER_PRESCALE 0x1F000000 +#define LSP_SMON_CFG0_RSVZ2 0x20000000 +#define LSP_SMON_CFG0_VERSION 0xC0000000 +#define LSP_SMON_CFG0_SMON_ENABLE_LOC 0 +#define LSP_SMON_CFG0_SMON_0TRIGGER_ENABLE_LOC 1 +#define LSP_SMON_CFG0_RSVZ0_LOC 2 +#define LSP_SMON_CFG0_SMON0_FUNCTION_LOC 4 +#define LSP_SMON_CFG0_SMON0_FUNCTION_COMPARE_LOC 7 +#define LSP_SMON_CFG0_SMON1_FUNCTION_LOC 8 +#define LSP_SMON_CFG0_SMON1_FUNCTION_COMPARE_LOC 11 +#define LSP_SMON_CFG0_SMON_MODE_LOC 12 +#define LSP_SMON_CFG0_STOPCOUNTEROVFL_LOC 16 +#define LSP_SMON_CFG0_INTCOUNTEROVFL_LOC 17 +#define LSP_SMON_CFG0_STATCOUNTER0OVFL_LOC 18 +#define LSP_SMON_CFG0_STATCOUNTER1OVFL_LOC 19 +#define LSP_SMON_CFG0_STOPTIMEROVFL_LOC 20 +#define LSP_SMON_CFG0_INTTIMEROVFL_LOC 21 +#define LSP_SMON_CFG0_STATTIMEROVFL_LOC 22 +#define LSP_SMON_CFG0_RSVZ1_LOC 23 +#define LSP_SMON_CFG0_TIMER_PRESCALE_LOC 24 +#define LSP_SMON_CFG0_RSVZ2_LOC 29 +#define LSP_SMON_CFG0_VERSION_LOC 30 + +#define LSP_SMON_CFG1 0xac000054 +#define LSP_SMON_CFG1_RST 0x0 + +#define LSP_SMON_CFG1_MODE0 0x000000FF +#define LSP_SMON_CFG1_MODE1 0x0000FF00 +#define LSP_SMON_CFG1_RSVZ0 0xFFFF0000 +#define LSP_SMON_CFG1_MODE0_LOC 0 +#define LSP_SMON_CFG1_MODE1_LOC 8 +#define LSP_SMON_CFG1_RSVZ0_LOC 16 + +#define LSP_SMON_ACTIVITYCNTR0 0xac000058 +#define LSP_SMON_ACTIVITYCNTR0_RST 0x0 + +#define LSP_SMON_ACTIVITYCNTR0_COUNTER0 0xFFFFFFFF +#define LSP_SMON_ACTIVITYCNTR0_COUNTER0_LOC 0 + +#define LSP_SMON_ACTIVITYCNTR1 0xac00005c +#define LSP_SMON_ACTIVITYCNTR1_RST 0x0 + +#define LSP_SMON_ACTIVITYCNTR1_COUNTER1 0xFFFFFFFF +#define LSP_SMON_ACTIVITYCNTR1_COUNTER1_LOC 0 + +#define LSP_SMON_MAX_TMR 0xac000060 +#define LSP_SMON_MAX_TMR_RST 0x0 + +#define LSP_SMON_MAX_TMR_MAXVALUE 0xFFFFFFFF +#define LSP_SMON_MAX_TMR_MAXVALUE_LOC 0 + +#define LSP_SMON_TMR 0xac000064 +#define LSP_SMON_TMR_RST 0x0 + +#define LSP_SMON_TMR_TIMER 0xFFFFFFFF +#define LSP_SMON_TMR_TIMER_LOC 0 + +#define CM_DIAG_RESET_STS 0xb4000000 +#define CM_DIAG_RESET_STS_RST 0x80000bff + +#define CM_DIAG_RESET_STS_CHP_PF_RESET_DONE 0x00000001 +#define CM_DIAG_RESET_STS_ROP_PF_RESET_DONE 0x00000002 +#define CM_DIAG_RESET_STS_LSP_PF_RESET_DONE 0x00000004 +#define CM_DIAG_RESET_STS_NALB_PF_RESET_DONE 0x00000008 +#define CM_DIAG_RESET_STS_AP_PF_RESET_DONE 0x00000010 +#define CM_DIAG_RESET_STS_DP_PF_RESET_DONE 0x00000020 +#define CM_DIAG_RESET_STS_QED_PF_RESET_DONE 0x00000040 +#define CM_DIAG_RESET_STS_DQED_PF_RESET_DONE 0x00000080 +#define CM_DIAG_RESET_STS_AQED_PF_RESET_DONE 0x00000100 +#define CM_DIAG_RESET_STS_SYS_PF_RESET_DONE 0x00000200 +#define CM_DIAG_RESET_STS_PF_RESET_ACTIVE 0x00000400 +#define CM_DIAG_RESET_STS_FLRSM_STATE 0x0003F800 +#define CM_DIAG_RESET_STS_RSVD0 0x7FFC0000 +#define CM_DIAG_RESET_STS_DLB_PROC_RESET_DONE 0x80000000 +#define CM_DIAG_RESET_STS_CHP_PF_RESET_DONE_LOC 0 +#define CM_DIAG_RESET_STS_ROP_PF_RESET_DONE_LOC 1 +#define CM_DIAG_RESET_STS_LSP_PF_RESET_DONE_LOC 2 +#define CM_DIAG_RESET_STS_NALB_PF_RESET_DONE_LOC 3 +#define CM_DIAG_RESET_STS_AP_PF_RESET_DONE_LOC 4 +#define CM_DIAG_RESET_STS_DP_PF_RESET_DONE_LOC 5 +#define CM_DIAG_RESET_STS_QED_PF_RESET_DONE_LOC 6 +#define CM_DIAG_RESET_STS_DQED_PF_RESET_DONE_LOC 7 +#define CM_DIAG_RESET_STS_AQED_PF_RESET_DONE_LOC 8 +#define CM_DIAG_RESET_STS_SYS_PF_RESET_DONE_LOC 9 +#define CM_DIAG_RESET_STS_PF_RESET_ACTIVE_LOC 10 +#define CM_DIAG_RESET_STS_FLRSM_STATE_LOC 11 +#define CM_DIAG_RESET_STS_RSVD0_LOC 18 +#define CM_DIAG_RESET_STS_DLB_PROC_RESET_DONE_LOC 31 #define CM_CFG_DIAGNOSTIC_IDLE_STATUS 0xb4000004 #define CM_CFG_DIAGNOSTIC_IDLE_STATUS_RST 0x9d0fffff @@ -134,4 +3584,57 @@ #define CM_CFG_PM_PMCSR_DISABLE_DISABLE_LOC 0 #define CM_CFG_PM_PMCSR_DISABLE_RSVZ0_LOC 1 +#define VF_VF2PF_MAILBOX_BYTES 256 +#define VF_VF2PF_MAILBOX(x) \ + (0x1000 + (x) * 0x4) +#define VF_VF2PF_MAILBOX_RST 0x0 + +#define VF_VF2PF_MAILBOX_MSG 0xFFFFFFFF +#define VF_VF2PF_MAILBOX_MSG_LOC 0 + +#define VF_VF2PF_MAILBOX_ISR 0x1f00 +#define VF_VF2PF_MAILBOX_ISR_RST 0x0 +#define VF_SIOV_MBOX_ISR_TRIGGER 0x8000 + +#define VF_VF2PF_MAILBOX_ISR_ISR 0x00000001 +#define VF_VF2PF_MAILBOX_ISR_RSVD0 0xFFFFFFFE +#define VF_VF2PF_MAILBOX_ISR_ISR_LOC 0 +#define VF_VF2PF_MAILBOX_ISR_RSVD0_LOC 1 + +#define VF_PF2VF_MAILBOX_BYTES 64 +#define VF_PF2VF_MAILBOX(x) \ + (0x2000 + (x) * 0x4) +#define VF_PF2VF_MAILBOX_RST 0x0 + +#define VF_PF2VF_MAILBOX_MSG 0xFFFFFFFF +#define VF_PF2VF_MAILBOX_MSG_LOC 0 + +#define VF_PF2VF_MAILBOX_ISR 0x2f00 +#define VF_PF2VF_MAILBOX_ISR_RST 0x0 + +#define VF_PF2VF_MAILBOX_ISR_PF_ISR 0x00000001 +#define VF_PF2VF_MAILBOX_ISR_RSVD0 0xFFFFFFFE +#define VF_PF2VF_MAILBOX_ISR_PF_ISR_LOC 0 +#define VF_PF2VF_MAILBOX_ISR_RSVD0_LOC 1 + +#define VF_VF_MSI_ISR_PEND 0x2f10 +#define VF_VF_MSI_ISR_PEND_RST 0x0 + +#define VF_VF_MSI_ISR_PEND_ISR_PEND 0xFFFFFFFF +#define VF_VF_MSI_ISR_PEND_ISR_PEND_LOC 0 + +#define VF_VF_RESET_IN_PROGRESS 0x3000 +#define VF_VF_RESET_IN_PROGRESS_RST 0x1 + +#define VF_VF_RESET_IN_PROGRESS_RESET_IN_PROGRESS 0x00000001 +#define VF_VF_RESET_IN_PROGRESS_RSVD0 0xFFFFFFFE +#define VF_VF_RESET_IN_PROGRESS_RESET_IN_PROGRESS_LOC 0 +#define VF_VF_RESET_IN_PROGRESS_RSVD0_LOC 1 + +#define VF_VF_MSI_ISR 0x4000 +#define VF_VF_MSI_ISR_RST 0x0 + +#define VF_VF_MSI_ISR_VF_MSI_ISR 0xFFFFFFFF +#define VF_VF_MSI_ISR_VF_MSI_ISR_LOC 0 + #endif /* __DLB_REGS_H */ diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index 6d73c2479819..26b7d9ea94e3 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -973,6 +973,390 @@ static int dlb_domain_reset_software_state(struct dlb_hw *hw, return 0; } +static void __dlb_domain_reset_ldb_port_registers(struct dlb_hw *hw, + struct dlb_ldb_port *port) +{ + DLB_CSR_WR(hw, + SYS_LDB_PP2VAS(port->id.phys_id), + SYS_LDB_PP2VAS_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ2VAS(port->id.phys_id), + CHP_LDB_CQ2VAS_RST); + + DLB_CSR_WR(hw, + SYS_LDB_PP2VDEV(port->id.phys_id), + SYS_LDB_PP2VDEV_RST); + + DLB_CSR_WR(hw, + SYS_LDB_PP_V(port->id.phys_id), + SYS_LDB_PP_V_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_DSBL(port->id.phys_id), + LSP_CQ_LDB_DSBL_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_DEPTH(port->id.phys_id), + CHP_LDB_CQ_DEPTH_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_INFL_LIM(port->id.phys_id), + LSP_CQ_LDB_INFL_LIM_RST); + + DLB_CSR_WR(hw, + CHP_HIST_LIST_LIM(port->id.phys_id), + CHP_HIST_LIST_LIM_RST); + + DLB_CSR_WR(hw, + CHP_HIST_LIST_BASE(port->id.phys_id), + CHP_HIST_LIST_BASE_RST); + + DLB_CSR_WR(hw, + CHP_HIST_LIST_POP_PTR(port->id.phys_id), + CHP_HIST_LIST_POP_PTR_RST); + + DLB_CSR_WR(hw, + CHP_HIST_LIST_PUSH_PTR(port->id.phys_id), + CHP_HIST_LIST_PUSH_PTR_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_INT_DEPTH_THRSH(port->id.phys_id), + CHP_LDB_CQ_INT_DEPTH_THRSH_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_TMR_THRSH(port->id.phys_id), + CHP_LDB_CQ_TMR_THRSH_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_INT_ENB(port->id.phys_id), + CHP_LDB_CQ_INT_ENB_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ_ISR(port->id.phys_id), + SYS_LDB_CQ_ISR_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_TKN_DEPTH_SEL(port->id.phys_id), + LSP_CQ_LDB_TKN_DEPTH_SEL_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_TKN_DEPTH_SEL(port->id.phys_id), + CHP_LDB_CQ_TKN_DEPTH_SEL_RST); + + DLB_CSR_WR(hw, + CHP_LDB_CQ_WPTR(port->id.phys_id), + CHP_LDB_CQ_WPTR_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_TKN_CNT(port->id.phys_id), + LSP_CQ_LDB_TKN_CNT_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ_ADDR_L(port->id.phys_id), + SYS_LDB_CQ_ADDR_L_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ_ADDR_U(port->id.phys_id), + SYS_LDB_CQ_ADDR_U_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ_AT(port->id.phys_id), + SYS_LDB_CQ_AT_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ_PASID(port->id.phys_id), + SYS_LDB_CQ_PASID_RST); + + DLB_CSR_WR(hw, + SYS_LDB_CQ2VF_PF_RO(port->id.phys_id), + SYS_LDB_CQ2VF_PF_RO_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_TOT_SCH_CNTL(port->id.phys_id), + LSP_CQ_LDB_TOT_SCH_CNTL_RST); + + DLB_CSR_WR(hw, + LSP_CQ_LDB_TOT_SCH_CNTH(port->id.phys_id), + LSP_CQ_LDB_TOT_SCH_CNTH_RST); + + DLB_CSR_WR(hw, + LSP_CQ2QID0(port->id.phys_id), + LSP_CQ2QID0_RST); + + DLB_CSR_WR(hw, + LSP_CQ2QID1(port->id.phys_id), + LSP_CQ2QID1_RST); + + DLB_CSR_WR(hw, + LSP_CQ2PRIOV(port->id.phys_id), + LSP_CQ2PRIOV_RST); +} + +static void dlb_domain_reset_ldb_port_registers(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) + __dlb_domain_reset_ldb_port_registers(hw, port); + } +} + +static void +__dlb_domain_reset_dir_port_registers(struct dlb_hw *hw, + struct dlb_dir_pq_pair *port) +{ + DLB_CSR_WR(hw, + CHP_DIR_CQ2VAS(port->id.phys_id), + CHP_DIR_CQ2VAS_RST); + + DLB_CSR_WR(hw, + LSP_CQ_DIR_DSBL(port->id.phys_id), + LSP_CQ_DIR_DSBL_RST); + + DLB_CSR_WR(hw, SYS_DIR_CQ_OPT_CLR, port->id.phys_id); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_DEPTH(port->id.phys_id), + CHP_DIR_CQ_DEPTH_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_INT_DEPTH_THRSH(port->id.phys_id), + CHP_DIR_CQ_INT_DEPTH_THRSH_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_TMR_THRSH(port->id.phys_id), + CHP_DIR_CQ_TMR_THRSH_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_INT_ENB(port->id.phys_id), + CHP_DIR_CQ_INT_ENB_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_ISR(port->id.phys_id), + SYS_DIR_CQ_ISR_RST); + + DLB_CSR_WR(hw, + LSP_CQ_DIR_TKN_DEPTH_SEL_DSI(port->id.phys_id), + LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_TKN_DEPTH_SEL(port->id.phys_id), + CHP_DIR_CQ_TKN_DEPTH_SEL_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ_WPTR(port->id.phys_id), + CHP_DIR_CQ_WPTR_RST); + + DLB_CSR_WR(hw, + LSP_CQ_DIR_TKN_CNT(port->id.phys_id), + LSP_CQ_DIR_TKN_CNT_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_ADDR_L(port->id.phys_id), + SYS_DIR_CQ_ADDR_L_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_ADDR_U(port->id.phys_id), + SYS_DIR_CQ_ADDR_U_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_AT(port->id.phys_id), + SYS_DIR_CQ_AT_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_PASID(port->id.phys_id), + SYS_DIR_CQ_PASID_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ_FMT(port->id.phys_id), + SYS_DIR_CQ_FMT_RST); + + DLB_CSR_WR(hw, + SYS_DIR_CQ2VF_PF_RO(port->id.phys_id), + SYS_DIR_CQ2VF_PF_RO_RST); + + DLB_CSR_WR(hw, + LSP_CQ_DIR_TOT_SCH_CNTL(port->id.phys_id), + LSP_CQ_DIR_TOT_SCH_CNTL_RST); + + DLB_CSR_WR(hw, + LSP_CQ_DIR_TOT_SCH_CNTH(port->id.phys_id), + LSP_CQ_DIR_TOT_SCH_CNTH_RST); + + DLB_CSR_WR(hw, + SYS_DIR_PP2VAS(port->id.phys_id), + SYS_DIR_PP2VAS_RST); + + DLB_CSR_WR(hw, + CHP_DIR_CQ2VAS(port->id.phys_id), + CHP_DIR_CQ2VAS_RST); + + DLB_CSR_WR(hw, + SYS_DIR_PP2VDEV(port->id.phys_id), + SYS_DIR_PP2VDEV_RST); + + DLB_CSR_WR(hw, + SYS_DIR_PP_V(port->id.phys_id), + SYS_DIR_PP_V_RST); +} + +static void dlb_domain_reset_dir_port_registers(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *port; + + list_for_each_entry(port, &domain->used_dir_pq_pairs, domain_list) + __dlb_domain_reset_dir_port_registers(hw, port); +} + +static void dlb_domain_reset_ldb_queue_registers(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_queue *queue; + + list_for_each_entry(queue, &domain->used_ldb_queues, domain_list) { + unsigned int queue_id = queue->id.phys_id; + int i; + + DLB_CSR_WR(hw, + LSP_QID_NALDB_TOT_ENQ_CNTL(queue_id), + LSP_QID_NALDB_TOT_ENQ_CNTL_RST); + + DLB_CSR_WR(hw, + LSP_QID_NALDB_TOT_ENQ_CNTH(queue_id), + LSP_QID_NALDB_TOT_ENQ_CNTH_RST); + + DLB_CSR_WR(hw, + LSP_QID_ATM_TOT_ENQ_CNTL(queue_id), + LSP_QID_ATM_TOT_ENQ_CNTL_RST); + + DLB_CSR_WR(hw, + LSP_QID_ATM_TOT_ENQ_CNTH(queue_id), + LSP_QID_ATM_TOT_ENQ_CNTH_RST); + + DLB_CSR_WR(hw, + LSP_QID_NALDB_MAX_DEPTH(queue_id), + LSP_QID_NALDB_MAX_DEPTH_RST); + + DLB_CSR_WR(hw, + LSP_QID_LDB_INFL_LIM(queue_id), + LSP_QID_LDB_INFL_LIM_RST); + + DLB_CSR_WR(hw, + LSP_QID_AQED_ACTIVE_LIM(queue_id), + LSP_QID_AQED_ACTIVE_LIM_RST); + + DLB_CSR_WR(hw, + LSP_QID_ATM_DEPTH_THRSH(queue_id), + LSP_QID_ATM_DEPTH_THRSH_RST); + + DLB_CSR_WR(hw, + LSP_QID_NALDB_DEPTH_THRSH(queue_id), + LSP_QID_NALDB_DEPTH_THRSH_RST); + + DLB_CSR_WR(hw, + SYS_LDB_QID_ITS(queue_id), + SYS_LDB_QID_ITS_RST); + + DLB_CSR_WR(hw, + CHP_ORD_QID_SN(queue_id), + CHP_ORD_QID_SN_RST); + + DLB_CSR_WR(hw, + CHP_ORD_QID_SN_MAP(queue_id), + CHP_ORD_QID_SN_MAP_RST); + + DLB_CSR_WR(hw, + SYS_LDB_QID_V(queue_id), + SYS_LDB_QID_V_RST); + + DLB_CSR_WR(hw, + SYS_LDB_QID_CFG_V(queue_id), + SYS_LDB_QID_CFG_V_RST); + + if (queue->sn_cfg_valid) { + u32 offs[2]; + + offs[0] = RO_GRP_0_SLT_SHFT(queue->sn_slot); + offs[1] = RO_GRP_1_SLT_SHFT(queue->sn_slot); + + DLB_CSR_WR(hw, + offs[queue->sn_group], + RO_GRP_0_SLT_SHFT_RST); + } + + for (i = 0; i < LSP_QID2CQIDIX_NUM; i++) { + DLB_CSR_WR(hw, + LSP_QID2CQIDIX(queue_id, i), + LSP_QID2CQIDIX_00_RST); + + DLB_CSR_WR(hw, + LSP_QID2CQIDIX2(queue_id, i), + LSP_QID2CQIDIX2_00_RST); + + DLB_CSR_WR(hw, + ATM_QID2CQIDIX(queue_id, i), + ATM_QID2CQIDIX_00_RST); + } + } +} + +static void dlb_domain_reset_dir_queue_registers(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *queue; + + list_for_each_entry(queue, &domain->used_dir_pq_pairs, domain_list) { + DLB_CSR_WR(hw, + LSP_QID_DIR_MAX_DEPTH(queue->id.phys_id), + LSP_QID_DIR_MAX_DEPTH_RST); + + DLB_CSR_WR(hw, + LSP_QID_DIR_TOT_ENQ_CNTL(queue->id.phys_id), + LSP_QID_DIR_TOT_ENQ_CNTL_RST); + + DLB_CSR_WR(hw, + LSP_QID_DIR_TOT_ENQ_CNTH(queue->id.phys_id), + LSP_QID_DIR_TOT_ENQ_CNTH_RST); + + DLB_CSR_WR(hw, + LSP_QID_DIR_DEPTH_THRSH(queue->id.phys_id), + LSP_QID_DIR_DEPTH_THRSH_RST); + + DLB_CSR_WR(hw, + SYS_DIR_QID_ITS(queue->id.phys_id), + SYS_DIR_QID_ITS_RST); + + DLB_CSR_WR(hw, + SYS_DIR_QID_V(queue->id.phys_id), + SYS_DIR_QID_V_RST); + } +} + +static void dlb_domain_reset_registers(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + dlb_domain_reset_ldb_port_registers(hw, domain); + + dlb_domain_reset_dir_port_registers(hw, domain); + + dlb_domain_reset_ldb_queue_registers(hw, domain); + + dlb_domain_reset_dir_queue_registers(hw, domain); + + DLB_CSR_WR(hw, + CHP_CFG_LDB_VAS_CRD(domain->id.phys_id), + CHP_CFG_LDB_VAS_CRD_RST); + + DLB_CSR_WR(hw, + CHP_CFG_DIR_VAS_CRD(domain->id.phys_id), + CHP_CFG_DIR_VAS_CRD_RST); +} + static void dlb_log_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, unsigned int vdev_id) { @@ -1019,6 +1403,9 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, if (!domain || !domain->configured) return -EINVAL; + /* Reset the QID and port state. */ + dlb_domain_reset_registers(hw, domain); + return dlb_domain_reset_software_state(hw, domain); } From patchwork Wed Feb 10 17:54:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FA83C433E6 for ; Wed, 10 Feb 2021 17:59:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 413F664EDC for ; Wed, 10 Feb 2021 17:59:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233675AbhBJR7N (ORCPT ); Wed, 10 Feb 2021 12:59:13 -0500 Received: from mga01.intel.com ([192.55.52.88]:60436 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233352AbhBJR5g (ORCPT ); Wed, 10 Feb 2021 12:57:36 -0500 IronPort-SDR: hapCCa2lGUMCKx1Eitu4LfyTkpYM8snMO06QqVve7mMyEF1ZlFymuwxBKTvHyCCXgSymsWqKxP +uAwgV2/d1VQ== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236028" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236028" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:10 -0800 IronPort-SDR: 0lDAPcsCWLKcVpdr0wXTxgzhzJzkdDyo0cjVHuk3Rd4luAc+IADQmKE+sEQJSkwRozwckdPx4o sVHTPbTZXS+A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235762" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:09 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 10/20] dlb: add register operations for queue management Date: Wed, 10 Feb 2021 11:54:13 -0600 Message-Id: <20210210175423.1873-11-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add the low-level code for configuring a new queue and querying its depth. When configuring a queue, program the device based on the user-supplied queue configuration ioctl arguments. Add low-level code for resetting (draining) a non-empty queue during scheduling domain reset. Draining a queue is an iterative process of checking if the queue is empty, and if not then selecting a linked 'victim' port and dequeueing the queue's events through this port. A port can only receive a small number of events at a time, usually much fewer than the queue depth, so draining a queue typically takes multiple iterations. This process is finite since software cannot enqueue new events to the DLB's (finite) on-device storage. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_hw_types.h | 46 +++ drivers/misc/dlb/dlb_resource.c | 610 +++++++++++++++++++++++++++++++- 2 files changed, 654 insertions(+), 2 deletions(-) diff --git a/drivers/misc/dlb/dlb_hw_types.h b/drivers/misc/dlb/dlb_hw_types.h index d382c414e2b0..c7827defa66a 100644 --- a/drivers/misc/dlb/dlb_hw_types.h +++ b/drivers/misc/dlb/dlb_hw_types.h @@ -50,6 +50,29 @@ #define PCI_DEVICE_ID_INTEL_DLB_PF 0x2710 +/* + * Hardware-defined base addresses. Those prefixed 'DLB_DRV' are only used by + * the PF driver. + */ +#define DLB_DRV_LDB_PP_BASE 0x2300000 +#define DLB_DRV_LDB_PP_STRIDE 0x1000 +#define DLB_DRV_LDB_PP_BOUND (DLB_DRV_LDB_PP_BASE + \ + DLB_DRV_LDB_PP_STRIDE * DLB_MAX_NUM_LDB_PORTS) +#define DLB_DRV_DIR_PP_BASE 0x2200000 +#define DLB_DRV_DIR_PP_STRIDE 0x1000 +#define DLB_DRV_DIR_PP_BOUND (DLB_DRV_DIR_PP_BASE + \ + DLB_DRV_DIR_PP_STRIDE * DLB_MAX_NUM_DIR_PORTS) +#define DLB_LDB_PP_BASE 0x2100000 +#define DLB_LDB_PP_STRIDE 0x1000 +#define DLB_LDB_PP_BOUND (DLB_LDB_PP_BASE + \ + DLB_LDB_PP_STRIDE * DLB_MAX_NUM_LDB_PORTS) +#define DLB_LDB_PP_OFFS(id) (DLB_LDB_PP_BASE + (id) * DLB_PP_SIZE) +#define DLB_DIR_PP_BASE 0x2000000 +#define DLB_DIR_PP_STRIDE 0x1000 +#define DLB_DIR_PP_BOUND (DLB_DIR_PP_BASE + \ + DLB_DIR_PP_STRIDE * DLB_MAX_NUM_DIR_PORTS) +#define DLB_DIR_PP_OFFS(id) (DLB_DIR_PP_BASE + (id) * DLB_PP_SIZE) + struct dlb_resource_id { u32 phys_id; u32 virt_id; @@ -68,6 +91,29 @@ static inline u32 dlb_freelist_count(struct dlb_freelist *list) return list->bound - list->base - list->offset; } +struct dlb_hcw { + u64 data; + /* Word 3 */ + u16 opaque; + u8 qid; + u8 sched_type:2; + u8 priority:3; + u8 msg_type:3; + /* Word 4 */ + u16 lock_id; + u8 ts_flag:1; + u8 rsvd1:2; + u8 no_dec:1; + u8 cmp_id:4; + u8 cq_token:1; + u8 qe_comp:1; + u8 qe_frag:1; + u8 qe_valid:1; + u8 int_arm:1; + u8 error:1; + u8 rsvd:2; +}; + struct dlb_ldb_queue { struct list_head domain_list; struct list_head func_list; diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index b36f14a661fa..3c4f4c4af2ac 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -1,12 +1,24 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(C) 2016-2020 Intel Corporation. All rights reserved. */ +#include #include "dlb_bitmap.h" #include "dlb_hw_types.h" #include "dlb_main.h" #include "dlb_regs.h" #include "dlb_resource.h" +/* + * The PF driver cannot assume that a register write will affect subsequent HCW + * writes. To ensure a write completes, the driver must read back a CSR. This + * function only need be called for configuration that can occur after the + * domain has started; prior to starting, applications can't send HCWs. + */ +static inline void dlb_flush_csr(struct dlb_hw *hw) +{ + DLB_CSR_RD(hw, SYS_TOTAL_VAS); +} + static void dlb_init_fn_rsrc_lists(struct dlb_function_resources *rsrc) { int i; @@ -844,6 +856,148 @@ dlb_verify_create_dir_queue_args(struct dlb_hw *hw, u32 domain_id, return 0; } +static void dlb_configure_ldb_queue(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_queue *queue, + struct dlb_create_ldb_queue_args *args, + bool vdev_req, unsigned int vdev_id) +{ + struct dlb_sn_group *sn_group; + unsigned int offs; + u32 reg = 0; + u32 alimit; + u32 level; + + /* QID write permissions are turned on when the domain is started */ + offs = domain->id.phys_id * DLB_MAX_NUM_LDB_QUEUES + queue->id.phys_id; + + DLB_CSR_WR(hw, SYS_LDB_VASQID_V(offs), reg); + + /* + * Unordered QIDs get 4K inflights, ordered get as many as the number + * of sequence numbers. + */ + BITS_SET(reg, args->num_qid_inflights, LSP_QID_LDB_INFL_LIM_LIMIT); + DLB_CSR_WR(hw, LSP_QID_LDB_INFL_LIM(queue->id.phys_id), reg); + + alimit = queue->aqed_limit; + + if (alimit > DLB_MAX_NUM_AQED_ENTRIES) + alimit = DLB_MAX_NUM_AQED_ENTRIES; + + reg = 0; + BITS_SET(reg, alimit, LSP_QID_AQED_ACTIVE_LIM_LIMIT); + DLB_CSR_WR(hw, LSP_QID_AQED_ACTIVE_LIM(queue->id.phys_id), reg); + + level = args->lock_id_comp_level; + if (level >= 64 && level <= 4096) + BITS_SET(reg, ilog2(level) - 5, AQED_QID_HID_WIDTH_COMPRESS_CODE); + else + reg = 0; + + DLB_CSR_WR(hw, AQED_QID_HID_WIDTH(queue->id.phys_id), reg); + + reg = 0; + /* Don't timestamp QEs that pass through this queue */ + DLB_CSR_WR(hw, SYS_LDB_QID_ITS(queue->id.phys_id), reg); + + BITS_SET(reg, args->depth_threshold, LSP_QID_ATM_DEPTH_THRSH_THRESH); + DLB_CSR_WR(hw, LSP_QID_ATM_DEPTH_THRSH(queue->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, args->depth_threshold, LSP_QID_NALDB_DEPTH_THRSH_THRESH); + DLB_CSR_WR(hw, LSP_QID_NALDB_DEPTH_THRSH(queue->id.phys_id), reg); + + /* + * This register limits the number of inflight flows a queue can have + * at one time. It has an upper bound of 2048, but can be + * over-subscribed. 512 is chosen so that a single queue doesn't use + * the entire atomic storage, but can use a substantial portion if + * needed. + */ + reg = 0; + BITS_SET(reg, 512, AQED_QID_FID_LIM_QID_FID_LIMIT); + DLB_CSR_WR(hw, AQED_QID_FID_LIM(queue->id.phys_id), reg); + + /* Configure SNs */ + reg = 0; + sn_group = &hw->rsrcs.sn_groups[queue->sn_group]; + BITS_SET(reg, sn_group->mode, CHP_ORD_QID_SN_MAP_MODE); + BITS_SET(reg, queue->sn_slot, CHP_ORD_QID_SN_MAP_SLOT); + BITS_SET(reg, sn_group->id, CHP_ORD_QID_SN_MAP_GRP); + + DLB_CSR_WR(hw, CHP_ORD_QID_SN_MAP(queue->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, (u32)(args->num_sequence_numbers != 0), + SYS_LDB_QID_CFG_V_SN_CFG_V); + BITS_SET(reg, (u32)(args->num_atomic_inflights != 0), + SYS_LDB_QID_CFG_V_FID_CFG_V); + + DLB_CSR_WR(hw, SYS_LDB_QID_CFG_V(queue->id.phys_id), reg); + + if (vdev_req) { + offs = vdev_id * DLB_MAX_NUM_LDB_QUEUES + queue->id.virt_id; + + reg = 0; + reg |= SYS_VF_LDB_VQID_V_VQID_V; + DLB_CSR_WR(hw, SYS_VF_LDB_VQID_V(offs), reg); + + reg = 0; + BITS_SET(reg, queue->id.phys_id, SYS_VF_LDB_VQID2QID_QID); + DLB_CSR_WR(hw, SYS_VF_LDB_VQID2QID(offs), reg); + + reg = 0; + BITS_SET(reg, queue->id.virt_id, SYS_LDB_QID2VQID_VQID); + DLB_CSR_WR(hw, SYS_LDB_QID2VQID(queue->id.phys_id), reg); + } + + reg = 0; + reg |= SYS_LDB_QID_V_QID_V; + DLB_CSR_WR(hw, SYS_LDB_QID_V(queue->id.phys_id), reg); +} + +static void dlb_configure_dir_queue(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_dir_pq_pair *queue, + struct dlb_create_dir_queue_args *args, + bool vdev_req, unsigned int vdev_id) +{ + unsigned int offs; + u32 reg = 0; + + /* QID write permissions are turned on when the domain is started */ + offs = domain->id.phys_id * DLB_MAX_NUM_DIR_QUEUES + + queue->id.phys_id; + + DLB_CSR_WR(hw, SYS_DIR_VASQID_V(offs), reg); + + /* Don't timestamp QEs that pass through this queue */ + DLB_CSR_WR(hw, SYS_DIR_QID_ITS(queue->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, args->depth_threshold, LSP_QID_DIR_DEPTH_THRSH_THRESH); + DLB_CSR_WR(hw, LSP_QID_DIR_DEPTH_THRSH(queue->id.phys_id), reg); + + if (vdev_req) { + offs = vdev_id * DLB_MAX_NUM_DIR_QUEUES + queue->id.virt_id; + + reg = 0; + reg |= SYS_VF_DIR_VQID_V_VQID_V; + DLB_CSR_WR(hw, SYS_VF_DIR_VQID_V(offs), reg); + + reg = 0; + BITS_SET(reg, queue->id.phys_id, SYS_VF_DIR_VQID2QID_QID); + DLB_CSR_WR(hw, SYS_VF_DIR_VQID2QID(offs), reg); + } + + reg = 0; + reg |= SYS_DIR_QID_V_QID_V; + DLB_CSR_WR(hw, SYS_DIR_QID_V(queue->id.phys_id), reg); + + queue->queue_configured = true; +} + static void dlb_configure_domain_credits(struct dlb_hw *hw, struct dlb_hw_domain *domain) { @@ -971,6 +1125,56 @@ dlb_ldb_queue_attach_resources(struct dlb_hw *hw, return 0; } +static void dlb_ldb_port_cq_enable(struct dlb_hw *hw, + struct dlb_ldb_port *port) +{ + u32 reg = 0; + + /* + * Don't re-enable the port if a removal is pending. The caller should + * mark this port as enabled (if it isn't already), and when the + * removal completes the port will be enabled. + */ + if (port->num_pending_removals) + return; + + DLB_CSR_WR(hw, LSP_CQ_LDB_DSBL(port->id.phys_id), reg); + + dlb_flush_csr(hw); +} + +static void dlb_ldb_port_cq_disable(struct dlb_hw *hw, + struct dlb_ldb_port *port) +{ + u32 reg = 0; + + reg |= LSP_CQ_LDB_DSBL_DISABLED; + DLB_CSR_WR(hw, LSP_CQ_LDB_DSBL(port->id.phys_id), reg); + + dlb_flush_csr(hw); +} + +static void dlb_dir_port_cq_enable(struct dlb_hw *hw, + struct dlb_dir_pq_pair *port) +{ + u32 reg = 0; + + DLB_CSR_WR(hw, LSP_CQ_DIR_DSBL(port->id.phys_id), reg); + + dlb_flush_csr(hw); +} + +static void dlb_dir_port_cq_disable(struct dlb_hw *hw, + struct dlb_dir_pq_pair *port) +{ + u32 reg = 0; + + reg |= LSP_CQ_DIR_DSBL_DISABLED; + DLB_CSR_WR(hw, LSP_CQ_DIR_DSBL(port->id.phys_id), reg); + + dlb_flush_csr(hw); +} + static void dlb_log_create_sched_domain_args(struct dlb_hw *hw, struct dlb_create_sched_domain_args *args, @@ -1147,6 +1351,8 @@ int dlb_hw_create_ldb_queue(struct dlb_hw *hw, u32 domain_id, return ret; } + dlb_configure_ldb_queue(hw, domain, queue, args, vdev_req, vdev_id); + queue->num_mappings = 0; queue->configured = true; @@ -1223,6 +1429,8 @@ int dlb_hw_create_dir_queue(struct dlb_hw *hw, u32 domain_id, if (ret) return ret; + dlb_configure_dir_queue(hw, domain, queue, args, vdev_req, vdev_id); + /* * Configuration succeeded, so move the resource from the 'avail' to * the 'used' list (if it's not already there). @@ -1240,6 +1448,92 @@ int dlb_hw_create_dir_queue(struct dlb_hw *hw, u32 domain_id, return 0; } +static u32 dlb_ldb_cq_inflight_count(struct dlb_hw *hw, + struct dlb_ldb_port *port) +{ + u32 cnt; + + cnt = DLB_CSR_RD(hw, LSP_CQ_LDB_INFL_CNT(port->id.phys_id)); + + return BITS_GET(cnt, LSP_CQ_LDB_INFL_CNT_COUNT); +} + +static u32 dlb_ldb_cq_token_count(struct dlb_hw *hw, struct dlb_ldb_port *port) +{ + u32 cnt; + + cnt = DLB_CSR_RD(hw, LSP_CQ_LDB_TKN_CNT(port->id.phys_id)); + + /* + * Account for the initial token count, which is used in order to + * provide a CQ with depth less than 8. + */ + + return BITS_GET(cnt, LSP_CQ_LDB_TKN_CNT_TOKEN_COUNT) - port->init_tkn_cnt; +} + +static void __iomem *dlb_producer_port_addr(struct dlb_hw *hw, u8 port_id, + bool is_ldb) +{ + struct dlb *dlb = container_of(hw, struct dlb, hw); + uintptr_t address = (uintptr_t)dlb->hw.func_kva; + unsigned long size; + + if (is_ldb) { + size = DLB_LDB_PP_STRIDE; + address += DLB_DRV_LDB_PP_BASE + size * port_id; + } else { + size = DLB_DIR_PP_STRIDE; + address += DLB_DRV_DIR_PP_BASE + size * port_id; + } + + return (void __iomem *)address; +} + +static void dlb_drain_ldb_cq(struct dlb_hw *hw, struct dlb_ldb_port *port) +{ + u32 infl_cnt, tkn_cnt; + unsigned int i; + + infl_cnt = dlb_ldb_cq_inflight_count(hw, port); + tkn_cnt = dlb_ldb_cq_token_count(hw, port); + + if (infl_cnt || tkn_cnt) { + struct dlb_hcw hcw_mem[8], *hcw; + void __iomem *pp_addr; + + pp_addr = dlb_producer_port_addr(hw, port->id.phys_id, true); + + /* Point hcw to a 64B-aligned location */ + hcw = (struct dlb_hcw *)((uintptr_t)&hcw_mem[4] & ~0x3F); + + /* + * Program the first HCW for a completion and token return and + * the other HCWs as NOOPS + */ + + memset(hcw, 0, 4 * sizeof(*hcw)); + hcw->qe_comp = (infl_cnt > 0); + hcw->cq_token = (tkn_cnt > 0); + hcw->lock_id = tkn_cnt - 1; + + /* Return tokens in the first HCW */ + iosubmit_cmds512(pp_addr, hcw, 1); + + hcw->cq_token = 0; + + /* Issue remaining completions (if any) */ + for (i = 1; i < infl_cnt; i++) + iosubmit_cmds512(pp_addr, hcw, 1); + + /* + * To ensure outstanding HCWs reach the device before subsequent device + * accesses, fence them. + */ + mb(); + } +} + static int dlb_domain_reset_software_state(struct dlb_hw *hw, struct dlb_hw_domain *domain) { @@ -1385,6 +1679,21 @@ static int dlb_domain_reset_software_state(struct dlb_hw *hw, return 0; } +static u32 dlb_dir_queue_depth(struct dlb_hw *hw, struct dlb_dir_pq_pair *queue) +{ + u32 cnt; + + cnt = DLB_CSR_RD(hw, LSP_QID_DIR_ENQUEUE_CNT(queue->id.phys_id)); + + return BITS_GET(cnt, LSP_QID_DIR_ENQUEUE_CNT_COUNT); +} + +static bool dlb_dir_queue_is_empty(struct dlb_hw *hw, + struct dlb_dir_pq_pair *queue) +{ + return dlb_dir_queue_depth(hw, queue) == 0; +} + static void dlb_log_get_dir_queue_depth(struct dlb_hw *hw, u32 domain_id, u32 queue_id, bool vdev_req, unsigned int vf_id) @@ -1446,7 +1755,7 @@ int dlb_hw_get_dir_queue_depth(struct dlb_hw *hw, u32 domain_id, return -EINVAL; } - resp->id = 0; + resp->id = dlb_dir_queue_depth(hw, queue); return 0; } @@ -1525,7 +1834,7 @@ int dlb_hw_get_ldb_queue_depth(struct dlb_hw *hw, u32 domain_id, return -EINVAL; } - resp->id = 0; + resp->id = dlb_ldb_queue_depth(hw, queue); return 0; } @@ -1894,6 +2203,21 @@ static void dlb_domain_reset_dir_queue_registers(struct dlb_hw *hw, } } +static u32 dlb_dir_cq_token_count(struct dlb_hw *hw, + struct dlb_dir_pq_pair *port) +{ + u32 cnt; + + cnt = DLB_CSR_RD(hw, LSP_CQ_DIR_TKN_CNT(port->id.phys_id)); + + /* + * Account for the initial token count, which is used in order to + * provide a CQ with depth less than 8. + */ + + return BITS_GET(cnt, LSP_CQ_DIR_TKN_CNT_COUNT) - port->init_tkn_cnt; +} + static int dlb_domain_verify_reset_success(struct dlb_hw *hw, struct dlb_hw_domain *domain) { @@ -1935,6 +2259,270 @@ static void dlb_domain_reset_registers(struct dlb_hw *hw, CHP_CFG_DIR_VAS_CRD_RST); } +static void dlb_domain_drain_ldb_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + bool toggle_port) +{ + struct dlb_ldb_port *port; + int i; + + /* If the domain hasn't been started, there's no traffic to drain */ + if (!domain->started) + return; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + if (toggle_port) + dlb_ldb_port_cq_disable(hw, port); + + dlb_drain_ldb_cq(hw, port); + + if (toggle_port) + dlb_ldb_port_cq_enable(hw, port); + } + } +} + +static bool dlb_domain_mapped_queues_empty(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_queue *queue; + + list_for_each_entry(queue, &domain->used_ldb_queues, domain_list) { + if (queue->num_mappings == 0) + continue; + + if (!dlb_ldb_queue_is_empty(hw, queue)) + return false; + } + + return true; +} + +static int dlb_domain_drain_mapped_queues(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + int i; + + /* If the domain hasn't been started, there's no traffic to drain */ + if (!domain->started) + return 0; + + if (domain->num_pending_removals > 0) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to unmap domain queues\n", + __func__); + return -EFAULT; + } + + for (i = 0; i < DLB_MAX_QID_EMPTY_CHECK_LOOPS; i++) { + dlb_domain_drain_ldb_cqs(hw, domain, true); + + if (dlb_domain_mapped_queues_empty(hw, domain)) + break; + } + + if (i == DLB_MAX_QID_EMPTY_CHECK_LOOPS) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to empty queues\n", + __func__); + return -EFAULT; + } + + /* + * Drain the CQs one more time. For the queues to go empty, they would + * have scheduled one or more QEs. + */ + dlb_domain_drain_ldb_cqs(hw, domain, true); + + return 0; +} + +static int dlb_drain_dir_cq(struct dlb_hw *hw, struct dlb_dir_pq_pair *port) +{ + unsigned int port_id = port->id.phys_id; + u32 cnt; + + /* Return any outstanding tokens */ + cnt = dlb_dir_cq_token_count(hw, port); + + if (cnt != 0) { + struct dlb_hcw hcw_mem[8], *hcw; + void __iomem *pp_addr; + + pp_addr = dlb_producer_port_addr(hw, port_id, false); + + /* Point hcw to a 64B-aligned location */ + hcw = (struct dlb_hcw *)((uintptr_t)&hcw_mem[4] & ~0x3F); + + /* + * Program the first HCW for a batch token return and + * the rest as NOOPS + */ + memset(hcw, 0, 4 * sizeof(*hcw)); + hcw->cq_token = 1; + hcw->lock_id = cnt - 1; + + iosubmit_cmds512(pp_addr, hcw, 1); + + /* + * To ensure outstanding HCWs reach the device before subsequent device + * accesses, fence them. + */ + mb(); + } + + return 0; +} + +static int dlb_domain_drain_dir_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + bool toggle_port) +{ + struct dlb_dir_pq_pair *port; + int ret; + + list_for_each_entry(port, &domain->used_dir_pq_pairs, domain_list) { + /* + * Can't drain a port if it's not configured, and there's + * nothing to drain if its queue is unconfigured. + */ + if (!port->port_configured || !port->queue_configured) + continue; + + if (toggle_port) + dlb_dir_port_cq_disable(hw, port); + + ret = dlb_drain_dir_cq(hw, port); + if (ret) + return ret; + + if (toggle_port) + dlb_dir_port_cq_enable(hw, port); + } + + return 0; +} + +static bool dlb_domain_dir_queues_empty(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *queue; + + list_for_each_entry(queue, &domain->used_dir_pq_pairs, domain_list) { + if (!dlb_dir_queue_is_empty(hw, queue)) + return false; + } + + return true; +} + +static int dlb_domain_drain_dir_queues(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + int i, ret; + + /* If the domain hasn't been started, there's no traffic to drain */ + if (!domain->started) + return 0; + + for (i = 0; i < DLB_MAX_QID_EMPTY_CHECK_LOOPS; i++) { + ret = dlb_domain_drain_dir_cqs(hw, domain, true); + if (ret) + return ret; + + if (dlb_domain_dir_queues_empty(hw, domain)) + break; + } + + if (i == DLB_MAX_QID_EMPTY_CHECK_LOOPS) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to empty queues\n", + __func__); + return -EFAULT; + } + + /* + * Drain the CQs one more time. For the queues to go empty, they would + * have scheduled one or more QEs. + */ + ret = dlb_domain_drain_dir_cqs(hw, domain, true); + if (ret) + return ret; + + return 0; +} + +static void +dlb_domain_disable_ldb_queue_write_perms(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + int domain_offset = domain->id.phys_id * DLB_MAX_NUM_LDB_QUEUES; + struct dlb_ldb_queue *queue; + + list_for_each_entry(queue, &domain->used_ldb_queues, domain_list) { + int idx = domain_offset + queue->id.phys_id; + + DLB_CSR_WR(hw, SYS_LDB_VASQID_V(idx), 0); + } +} + +static void +dlb_domain_disable_dir_queue_write_perms(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + int domain_offset = domain->id.phys_id * DLB_MAX_NUM_DIR_PORTS; + struct dlb_dir_pq_pair *queue; + + list_for_each_entry(queue, &domain->used_dir_pq_pairs, domain_list) { + int idx = domain_offset + queue->id.phys_id; + + DLB_CSR_WR(hw, SYS_DIR_VASQID_V(idx), 0); + } +} + +static void dlb_domain_disable_dir_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *port; + + list_for_each_entry(port, &domain->used_dir_pq_pairs, domain_list) { + port->enabled = false; + + dlb_dir_port_cq_disable(hw, port); + } +} + +static void dlb_domain_disable_ldb_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + port->enabled = false; + + dlb_ldb_port_cq_disable(hw, port); + } + } +} + +static void dlb_domain_enable_ldb_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + port->enabled = true; + + dlb_ldb_port_cq_enable(hw, port); + } + } +} + static void dlb_log_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, unsigned int vdev_id) { @@ -1987,6 +2575,24 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, * cause any traffic sent to it to be dropped. Well-behaved software * should not be sending QEs at this point. */ + dlb_domain_disable_dir_queue_write_perms(hw, domain); + + dlb_domain_disable_ldb_queue_write_perms(hw, domain); + + /* Re-enable the CQs in order to drain the mapped queues. */ + dlb_domain_enable_ldb_cqs(hw, domain); + + ret = dlb_domain_drain_mapped_queues(hw, domain); + if (ret) + return ret; + + /* Done draining LDB QEs, so disable the CQs. */ + dlb_domain_disable_ldb_cqs(hw, domain); + + dlb_domain_drain_dir_queues(hw, domain); + + /* Done draining DIR QEs, so disable the CQs. */ + dlb_domain_disable_dir_cqs(hw, domain); ret = dlb_domain_verify_reset_success(hw, domain); if (ret) From patchwork Wed Feb 10 17:54:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E915C433DB for ; Wed, 10 Feb 2021 18:00:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2034E64E6F for ; Wed, 10 Feb 2021 18:00:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233722AbhBJR7l (ORCPT ); Wed, 10 Feb 2021 12:59:41 -0500 Received: from mga01.intel.com ([192.55.52.88]:60424 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233617AbhBJR6Q (ORCPT ); Wed, 10 Feb 2021 12:58:16 -0500 IronPort-SDR: rnmL6cBUKxTe+ZlWQ4gVoV4uw+1qqfRzc0wHpl95zeRj+6rLKtHpCSk9oUKeo3qOM9M4GgQQPq PVt5UBICqCPA== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236034" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236034" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:11 -0800 IronPort-SDR: BF+cynANtCBo+V0nbTEifyYEfmX29G4MNEytVv3ihQnC9sOjU1aWxqQwiegxQZjL5tCHo5ulUF Rt0CpS1DnwoA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235781" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:10 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 12/20] dlb: add register operations for port management Date: Wed, 10 Feb 2021 11:54:15 -0600 Message-Id: <20210210175423.1873-13-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add the low-level code for configuring a new port, programming the device-wide poll mode setting, and resetting a port. The low-level port configuration functions program the device based on the user-supplied ioctl arguments. These arguments are first verified, e.g. to ensure that the port's CQ base address is properly cache-line aligned. During domain reset, each port is drained until its inflight count and owed-token count reaches 0, reflecting an empty CQ. Once the ports are drained, the domain reset operation disables them from being candidates for future scheduling decisions -- until they are re-assigned to a new scheduling domain in the future and re-enabled. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_resource.c | 448 +++++++++++++++++++++++++++++++- 1 file changed, 443 insertions(+), 5 deletions(-) diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index ac6c5889c435..822c1f4f7849 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -890,7 +890,7 @@ static void dlb_configure_ldb_queue(struct dlb_hw *hw, DLB_CSR_WR(hw, LSP_QID_AQED_ACTIVE_LIM(queue->id.phys_id), reg); level = args->lock_id_comp_level; - if (level >= 64 && level <= 4096) + if (level >= 64 && level <= 4096 && is_power_of_2(level)) BITS_SET(reg, ilog2(level) - 5, AQED_QID_HID_WIDTH_COMPRESS_CODE); else reg = 0; @@ -1001,12 +1001,10 @@ static void dlb_configure_dir_queue(struct dlb_hw *hw, static bool dlb_cq_depth_is_valid(u32 depth) { - u32 n = ilog2(depth); - /* Valid values for depth are * 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, and 1024. */ - if (depth > 1024 || ((1U << n) != depth)) + if (!is_power_of_2(depth) || depth > 1024) return false; return true; @@ -1347,6 +1345,320 @@ static void dlb_dir_port_cq_disable(struct dlb_hw *hw, dlb_flush_csr(hw); } +static void dlb_ldb_port_configure_pp(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_port *port) +{ + u32 reg = 0; + + BITS_SET(reg, domain->id.phys_id, SYS_LDB_PP2VAS_VAS); + DLB_CSR_WR(hw, SYS_LDB_PP2VAS(port->id.phys_id), reg); + + reg = 0; + reg |= SYS_LDB_PP_V_PP_V; + DLB_CSR_WR(hw, SYS_LDB_PP_V(port->id.phys_id), reg); +} + +static int dlb_ldb_port_configure_cq(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_port *port, + uintptr_t cq_dma_base, + struct dlb_create_ldb_port_args *args, + bool vdev_req, unsigned int vdev_id) +{ + u32 hl_base = 0; + u32 reg = 0; + u32 ds = 0; + u32 n; + + /* The CQ address is 64B-aligned, and the DLB only wants bits [63:6] */ + BITS_SET(reg, cq_dma_base >> 6, SYS_LDB_CQ_ADDR_L_ADDR_L); + DLB_CSR_WR(hw, SYS_LDB_CQ_ADDR_L(port->id.phys_id), reg); + + reg = cq_dma_base >> 32; + DLB_CSR_WR(hw, SYS_LDB_CQ_ADDR_U(port->id.phys_id), reg); + + /* + * 'ro' == relaxed ordering. This setting allows DLB to write + * cache lines out-of-order (but QEs within a cache line are always + * updated in-order). + */ + reg = 0; + BITS_SET(reg, vdev_id, SYS_LDB_CQ2VF_PF_RO_VF); + BITS_SET(reg, (u32)(!vdev_req), SYS_LDB_CQ2VF_PF_RO_IS_PF); + reg |= SYS_LDB_CQ2VF_PF_RO_RO; + + DLB_CSR_WR(hw, SYS_LDB_CQ2VF_PF_RO(port->id.phys_id), reg); + + if (!dlb_cq_depth_is_valid(args->cq_depth)) { + DLB_HW_ERR(hw, + "[%s():%d] Internal error: invalid CQ depth\n", + __func__, __LINE__); + return -EINVAL; + } + + if (args->cq_depth <= 8) { + ds = 1; + } else { + n = ilog2(args->cq_depth); + ds = n - 2; + } + + reg = 0; + BITS_SET(reg, ds, CHP_LDB_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT); + DLB_CSR_WR(hw, CHP_LDB_CQ_TKN_DEPTH_SEL(port->id.phys_id), reg); + + /* + * To support CQs with depth less than 8, program the token count + * register with a non-zero initial value. Operations such as domain + * reset must take this initial value into account when quiescing the + * CQ. + */ + port->init_tkn_cnt = 0; + + if (args->cq_depth < 8) { + reg = 0; + port->init_tkn_cnt = 8 - args->cq_depth; + + BITS_SET(reg, port->init_tkn_cnt, LSP_CQ_LDB_TKN_CNT_TOKEN_COUNT); + DLB_CSR_WR(hw, LSP_CQ_LDB_TKN_CNT(port->id.phys_id), reg); + } else { + DLB_CSR_WR(hw, + LSP_CQ_LDB_TKN_CNT(port->id.phys_id), + LSP_CQ_LDB_TKN_CNT_RST); + } + + reg = 0; + BITS_SET(reg, ds, LSP_CQ_LDB_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT); + DLB_CSR_WR(hw, LSP_CQ_LDB_TKN_DEPTH_SEL(port->id.phys_id), reg); + + /* Reset the CQ write pointer */ + DLB_CSR_WR(hw, + CHP_LDB_CQ_WPTR(port->id.phys_id), + CHP_LDB_CQ_WPTR_RST); + + reg = 0; + BITS_SET(reg, port->hist_list_entry_limit - 1, CHP_HIST_LIST_LIM_LIMIT); + DLB_CSR_WR(hw, CHP_HIST_LIST_LIM(port->id.phys_id), reg); + + BITS_SET(hl_base, port->hist_list_entry_base, CHP_HIST_LIST_BASE_BASE); + DLB_CSR_WR(hw, CHP_HIST_LIST_BASE(port->id.phys_id), hl_base); + + /* + * The inflight limit sets a cap on the number of QEs for which this CQ + * can owe completions at one time. + */ + reg = 0; + BITS_SET(reg, args->cq_history_list_size, LSP_CQ_LDB_INFL_LIM_LIMIT); + DLB_CSR_WR(hw, LSP_CQ_LDB_INFL_LIM(port->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, BITS_GET(hl_base, CHP_HIST_LIST_BASE_BASE), + CHP_HIST_LIST_PUSH_PTR_PUSH_PTR); + DLB_CSR_WR(hw, CHP_HIST_LIST_PUSH_PTR(port->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, BITS_GET(hl_base, CHP_HIST_LIST_BASE_BASE), + CHP_HIST_LIST_POP_PTR_POP_PTR); + DLB_CSR_WR(hw, CHP_HIST_LIST_POP_PTR(port->id.phys_id), reg); + + /* + * Address translation (AT) settings: 0: untranslated, 2: translated + * (see ATS spec regarding Address Type field for more details) + */ + + reg = 0; + DLB_CSR_WR(hw, SYS_LDB_CQ_AT(port->id.phys_id), reg); + DLB_CSR_WR(hw, SYS_LDB_CQ_PASID(port->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, domain->id.phys_id, CHP_LDB_CQ2VAS_CQ2VAS); + DLB_CSR_WR(hw, CHP_LDB_CQ2VAS(port->id.phys_id), reg); + + /* Disable the port's QID mappings */ + reg = 0; + DLB_CSR_WR(hw, LSP_CQ2PRIOV(port->id.phys_id), reg); + + return 0; +} + +static int dlb_configure_ldb_port(struct dlb_hw *hw, struct dlb_hw_domain *domain, + struct dlb_ldb_port *port, + uintptr_t cq_dma_base, + struct dlb_create_ldb_port_args *args, + bool vdev_req, unsigned int vdev_id) +{ + int ret, i; + + port->hist_list_entry_base = domain->hist_list_entry_base + + domain->hist_list_entry_offset; + port->hist_list_entry_limit = port->hist_list_entry_base + + args->cq_history_list_size; + + domain->hist_list_entry_offset += args->cq_history_list_size; + domain->avail_hist_list_entries -= args->cq_history_list_size; + + ret = dlb_ldb_port_configure_cq(hw, + domain, + port, + cq_dma_base, + args, + vdev_req, + vdev_id); + if (ret) + return ret; + + dlb_ldb_port_configure_pp(hw, domain, port); + + dlb_ldb_port_cq_enable(hw, port); + + for (i = 0; i < DLB_MAX_NUM_QIDS_PER_LDB_CQ; i++) + port->qid_map[i].state = DLB_QUEUE_UNMAPPED; + port->num_mappings = 0; + + port->enabled = true; + + port->configured = true; + + return 0; +} + +static void dlb_dir_port_configure_pp(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_dir_pq_pair *port) +{ + u32 reg = 0; + + BITS_SET(reg, domain->id.phys_id, SYS_DIR_PP2VAS_VAS); + DLB_CSR_WR(hw, SYS_DIR_PP2VAS(port->id.phys_id), reg); + + reg = 0; + reg |= SYS_DIR_PP_V_PP_V; + DLB_CSR_WR(hw, SYS_DIR_PP_V(port->id.phys_id), reg); +} + +static int dlb_dir_port_configure_cq(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_dir_pq_pair *port, + uintptr_t cq_dma_base, + struct dlb_create_dir_port_args *args, + bool vdev_req, unsigned int vdev_id) +{ + u32 reg = 0; + u32 ds = 0; + u32 n; + + /* The CQ address is 64B-aligned, and the DLB only wants bits [63:6] */ + BITS_SET(reg, cq_dma_base >> 6, SYS_DIR_CQ_ADDR_L_ADDR_L); + DLB_CSR_WR(hw, SYS_DIR_CQ_ADDR_L(port->id.phys_id), reg); + + reg = cq_dma_base >> 32; + DLB_CSR_WR(hw, SYS_DIR_CQ_ADDR_U(port->id.phys_id), reg); + + /* + * 'ro' == relaxed ordering. This setting allows DLB to write + * cache lines out-of-order (but QEs within a cache line are always + * updated in-order). + */ + reg = 0; + BITS_SET(reg, vdev_id, SYS_DIR_CQ2VF_PF_RO_VF); + BITS_SET(reg, (u32)(!vdev_req), SYS_DIR_CQ2VF_PF_RO_IS_PF); + reg |= SYS_DIR_CQ2VF_PF_RO_RO; + + DLB_CSR_WR(hw, SYS_DIR_CQ2VF_PF_RO(port->id.phys_id), reg); + + if (!dlb_cq_depth_is_valid(args->cq_depth)) { + DLB_HW_ERR(hw, + "[%s():%d] Internal error: invalid CQ depth\n", + __func__, __LINE__); + return -EINVAL; + } + + if (args->cq_depth <= 8) { + ds = 1; + } else { + n = ilog2(args->cq_depth); + ds = n - 2; + } + + reg = 0; + BITS_SET(reg, ds, CHP_DIR_CQ_TKN_DEPTH_SEL_TOKEN_DEPTH_SELECT); + DLB_CSR_WR(hw, CHP_DIR_CQ_TKN_DEPTH_SEL(port->id.phys_id), reg); + + /* + * To support CQs with depth less than 8, program the token count + * register with a non-zero initial value. Operations such as domain + * reset must take this initial value into account when quiescing the + * CQ. + */ + port->init_tkn_cnt = 0; + + if (args->cq_depth < 8) { + reg = 0; + port->init_tkn_cnt = 8 - args->cq_depth; + + BITS_SET(reg, port->init_tkn_cnt, LSP_CQ_DIR_TKN_CNT_COUNT); + DLB_CSR_WR(hw, LSP_CQ_DIR_TKN_CNT(port->id.phys_id), reg); + } else { + DLB_CSR_WR(hw, + LSP_CQ_DIR_TKN_CNT(port->id.phys_id), + LSP_CQ_DIR_TKN_CNT_RST); + } + + reg = 0; + BITS_SET(reg, ds, LSP_CQ_DIR_TKN_DEPTH_SEL_DSI_TOKEN_DEPTH_SELECT); + DLB_CSR_WR(hw, LSP_CQ_DIR_TKN_DEPTH_SEL_DSI(port->id.phys_id), reg); + + /* Reset the CQ write pointer */ + DLB_CSR_WR(hw, + CHP_DIR_CQ_WPTR(port->id.phys_id), + CHP_DIR_CQ_WPTR_RST); + + /* Virtualize the PPID */ + reg = 0; + DLB_CSR_WR(hw, SYS_DIR_CQ_FMT(port->id.phys_id), reg); + + /* + * Address translation (AT) settings: 0: untranslated, 2: translated + * (see ATS spec regarding Address Type field for more details) + */ + reg = 0; + DLB_CSR_WR(hw, SYS_DIR_CQ_AT(port->id.phys_id), reg); + + DLB_CSR_WR(hw, SYS_DIR_CQ_PASID(port->id.phys_id), reg); + + reg = 0; + BITS_SET(reg, domain->id.phys_id, CHP_DIR_CQ2VAS_CQ2VAS); + DLB_CSR_WR(hw, CHP_DIR_CQ2VAS(port->id.phys_id), reg); + + return 0; +} + +static int dlb_configure_dir_port(struct dlb_hw *hw, struct dlb_hw_domain *domain, + struct dlb_dir_pq_pair *port, + uintptr_t cq_dma_base, + struct dlb_create_dir_port_args *args, + bool vdev_req, unsigned int vdev_id) +{ + int ret; + + ret = dlb_dir_port_configure_cq(hw, domain, port, cq_dma_base, + args, vdev_req, vdev_id); + + if (ret) + return ret; + + dlb_dir_port_configure_pp(hw, domain, port); + + dlb_dir_port_cq_enable(hw, port); + + port->enabled = true; + + port->port_configured = true; + + return 0; +} + static void dlb_log_create_sched_domain_args(struct dlb_hw *hw, struct dlb_create_sched_domain_args *args, @@ -1693,6 +2005,11 @@ int dlb_hw_create_ldb_port(struct dlb_hw *hw, u32 domain_id, if (ret) return ret; + ret = dlb_configure_ldb_port(hw, domain, port, cq_dma_base, + args, vdev_req, vdev_id); + if (ret) + return ret; + /* * Configuration succeeded, so move the resource from the 'avail' to * the 'used' list. @@ -1775,6 +2092,11 @@ int dlb_hw_create_dir_port(struct dlb_hw *hw, u32 domain_id, if (ret) return ret; + ret = dlb_configure_dir_port(hw, domain, port, cq_dma_base, + args, vdev_req, vdev_id); + if (ret) + return ret; + /* * Configuration succeeded, so move the resource from the 'avail' to * the 'used' list (if it's not already there). @@ -1877,6 +2199,33 @@ static void dlb_drain_ldb_cq(struct dlb_hw *hw, struct dlb_ldb_port *port) } } +static int dlb_domain_wait_for_ldb_cqs_to_empty(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + int j; + + for (j = 0; j < DLB_MAX_CQ_COMP_CHECK_LOOPS; j++) { + if (dlb_ldb_cq_inflight_count(hw, port) == 0) + break; + } + + if (j == DLB_MAX_CQ_COMP_CHECK_LOOPS) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to flush load-balanced port %d's completions.\n", + __func__, port->id.phys_id); + return -EFAULT; + } + } + } + + return 0; +} + static int dlb_domain_reset_software_state(struct dlb_hw *hw, struct dlb_hw_domain *domain) { @@ -2564,7 +2913,10 @@ static u32 dlb_dir_cq_token_count(struct dlb_hw *hw, static int dlb_domain_verify_reset_success(struct dlb_hw *hw, struct dlb_hw_domain *domain) { + struct dlb_dir_pq_pair *dir_port; + struct dlb_ldb_port *ldb_port; struct dlb_ldb_queue *queue; + int i; /* * Confirm that all the domain's queue's inflight counts and AQED @@ -2579,6 +2931,35 @@ static int dlb_domain_verify_reset_success(struct dlb_hw *hw, } } + /* Confirm that all the domain's CQs inflight and token counts are 0. */ + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(ldb_port, &domain->used_ldb_ports[i], domain_list) { + if (dlb_ldb_cq_inflight_count(hw, ldb_port) || + dlb_ldb_cq_token_count(hw, ldb_port)) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to empty ldb port %d\n", + __func__, ldb_port->id.phys_id); + return -EFAULT; + } + } + } + + list_for_each_entry(dir_port, &domain->used_dir_pq_pairs, domain_list) { + if (!dlb_dir_queue_is_empty(hw, dir_port)) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to empty dir queue %d\n", + __func__, dir_port->id.phys_id); + return -EFAULT; + } + + if (dlb_dir_cq_token_count(hw, dir_port)) { + DLB_HW_ERR(hw, + "[%s()] Internal error: failed to empty dir port %d\n", + __func__, dir_port->id.phys_id); + return -EFAULT; + } + } + return 0; } @@ -2796,6 +3177,51 @@ static int dlb_domain_drain_dir_queues(struct dlb_hw *hw, return 0; } +static void +dlb_domain_disable_dir_producer_ports(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *port; + u32 pp_v = 0; + + list_for_each_entry(port, &domain->used_dir_pq_pairs, domain_list) { + DLB_CSR_WR(hw, SYS_DIR_PP_V(port->id.phys_id), pp_v); + } +} + +static void +dlb_domain_disable_ldb_producer_ports(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + u32 pp_v = 0; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + DLB_CSR_WR(hw, + SYS_LDB_PP_V(port->id.phys_id), + pp_v); + } + } +} + +static void dlb_domain_disable_ldb_seq_checks(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + u32 chk_en = 0; + int i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + DLB_CSR_WR(hw, + CHP_SN_CHK_ENBL(port->id.phys_id), + chk_en); + } + } +} + static void dlb_domain_disable_ldb_queue_write_perms(struct dlb_hw *hw, struct dlb_hw_domain *domain) @@ -2922,6 +3348,9 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, dlb_domain_disable_ldb_queue_write_perms(hw, domain); + /* Turn off completion tracking on all the domain's PPs. */ + dlb_domain_disable_ldb_seq_checks(hw, domain); + /* * Disable the LDB CQs and drain them in order to complete the map and * unmap procedures, which require zero CQ inflights and zero QID @@ -2931,6 +3360,10 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, dlb_domain_drain_ldb_cqs(hw, domain, false); + ret = dlb_domain_wait_for_ldb_cqs_to_empty(hw, domain); + if (ret) + return ret; + /* Re-enable the CQs in order to drain the mapped queues. */ dlb_domain_enable_ldb_cqs(hw, domain); @@ -2946,6 +3379,11 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, /* Done draining DIR QEs, so disable the CQs. */ dlb_domain_disable_dir_cqs(hw, domain); + /* Disable PPs */ + dlb_domain_disable_dir_producer_ports(hw, domain); + + dlb_domain_disable_ldb_producer_ports(hw, domain); + ret = dlb_domain_verify_reset_success(hw, domain); if (ret) return ret; @@ -3039,7 +3477,7 @@ void dlb_clr_pmcsr_disable(struct dlb_hw *hw) /** * dlb_hw_enable_sparse_ldb_cq_mode() - enable sparse mode for load-balanced - * ports. + * ports. * @hw: dlb_hw handle for a particular device. * * This function must be called prior to configuring scheduling domains. From patchwork Wed Feb 10 17:54:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D9D7C433E0 for ; Wed, 10 Feb 2021 18:00:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E4DF64D9E for ; Wed, 10 Feb 2021 18:00:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233168AbhBJR7s (ORCPT ); Wed, 10 Feb 2021 12:59:48 -0500 Received: from mga01.intel.com ([192.55.52.88]:60432 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233621AbhBJR6S (ORCPT ); Wed, 10 Feb 2021 12:58:18 -0500 IronPort-SDR: EyRaYEUCBf03b5/fMyWfXaoxiqxKVipQ/QZFnEx9Df0DzI1yz1zAQpjWezDrL7wKOkBtJ89UmS iZ/S26aRT0dg== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236036" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236036" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:11 -0800 IronPort-SDR: 3yqbSiJspnpnJ0LNy9GsOQQ50U4lpQ+hXF+9QCzKi2IZ1yMP1znZ5bZXxRP47hEtMG4w0buGnC RuD7v/3QxeYw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235789" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:11 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 13/20] dlb: add port mmap support Date: Wed, 10 Feb 2021 11:54:16 -0600 Message-Id: <20210210175423.1873-14-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Once a port is created, the application can mmap the corresponding DMA memory and MMIO into user-space. This allows user-space applications to do (performance-sensitive) enqueue and dequeue independent of the kernel driver. The mmap callback is only available through special port files: a producer port (PP) file and a consumer queue (CQ) file. User-space gets an fd for these files by calling a new ioctl, DLB_DOMAIN_CMD_GET_{LDB, DIR}_PORT_{PP, CQ}_FD, and passing in a port ID. If the ioctl succeeds, the returned fd can be used to mmap that port's PP/CQ. Device reset requires first unmapping all user-space mappings, to prevent applications from interfering with the reset operation. To this end, the driver uses a single inode -- allocated when the first PP/CQ file is created, and freed when the last such file is closed -- and attaches all port files to this common inode, as done elsewhere in Linux (e.g. cxl, dax). Allocating this inode requires creating a pseudo-filesystem. The driver initializes this FS when the inode is allocated, and frees the FS after the inode is freed. The driver doesn't use anon_inode_getfd() for these port mmap files because the anon inode layer uses a single inode that is shared with other kernel components -- calling unmap_mapping_range() on that shared inode would likely break the kernel. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Magnus Karlsson Reviewed-by: Dan Williams --- drivers/misc/dlb/Makefile | 1 + drivers/misc/dlb/dlb_file.c | 149 ++++++++++++++++++++++++++++++++ drivers/misc/dlb/dlb_hw_types.h | 4 +- drivers/misc/dlb/dlb_ioctl.c | 143 ++++++++++++++++++++++++++++++ drivers/misc/dlb/dlb_main.c | 118 +++++++++++++++++++++++++ drivers/misc/dlb/dlb_main.h | 24 +++++ drivers/misc/dlb/dlb_pf_ops.c | 18 ++++ drivers/misc/dlb/dlb_resource.c | 131 ++++++++++++++++++++++++++++ drivers/misc/dlb/dlb_resource.h | 6 ++ include/uapi/linux/dlb.h | 59 +++++++++++++ 10 files changed, 651 insertions(+), 2 deletions(-) create mode 100644 drivers/misc/dlb/dlb_file.c diff --git a/drivers/misc/dlb/Makefile b/drivers/misc/dlb/Makefile index aaafb3086d8d..66676222ca07 100644 --- a/drivers/misc/dlb/Makefile +++ b/drivers/misc/dlb/Makefile @@ -8,3 +8,4 @@ obj-$(CONFIG_INTEL_DLB) := dlb.o dlb-objs := dlb_main.o dlb-objs += dlb_pf_ops.o dlb_resource.o dlb_ioctl.o +dlb-objs += dlb_file.o diff --git a/drivers/misc/dlb/dlb_file.c b/drivers/misc/dlb/dlb_file.c new file mode 100644 index 000000000000..310b86735353 --- /dev/null +++ b/drivers/misc/dlb/dlb_file.c @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(C) 2016-2020 Intel Corporation. All rights reserved. */ + +#include +#include +#include +#include +#include + +#include "dlb_main.h" + +/* + * dlb tracks its memory mappings so it can revoke them when an FLR is + * requested and user-space cannot be allowed to access the device. To achieve + * that, the driver creates a single inode through which all driver-created + * files can share a struct address_space, and unmaps the inode's address space + * during the reset preparation phase. Since the anon inode layer shares its + * inode with multiple kernel components, we cannot use that here. + * + * Doing so requires a custom pseudo-filesystem to allocate the inode. The FS + * and the inode are allocated on demand when a file is created, and both are + * freed when the last such file is closed. + * + * This is inspired by other drivers (cxl, dax, mem) and the anon inode layer. + */ +static int dlb_fs_cnt; +static struct vfsmount *dlb_vfs_mount; + +#define DLBFS_MAGIC 0x444C4232 /* ASCII for DLB */ +static int dlb_init_fs_context(struct fs_context *fc) +{ + return init_pseudo(fc, DLBFS_MAGIC) ? 0 : -ENOMEM; +} + +static struct file_system_type dlb_fs_type = { + .name = "dlb", + .owner = THIS_MODULE, + .init_fs_context = dlb_init_fs_context, + .kill_sb = kill_anon_super, +}; + +/* Allocate an anonymous inode. Must hold the resource mutex while calling. */ +static struct inode *dlb_alloc_inode(struct dlb *dlb) +{ + struct inode *inode; + int ret; + + /* Increment the pseudo-FS's refcnt and (if not already) mount it. */ + ret = simple_pin_fs(&dlb_fs_type, &dlb_vfs_mount, &dlb_fs_cnt); + if (ret < 0) { + dev_err(dlb->dev, + "[%s()] Cannot mount pseudo filesystem: %d\n", + __func__, ret); + return ERR_PTR(ret); + } + + dlb->inode_cnt++; + + if (dlb->inode_cnt > 1) { + /* + * Return the previously allocated inode. In this case, there + * is guaranteed >= 1 reference and so ihold() is safe to call. + */ + ihold(dlb->inode); + return dlb->inode; + } + + inode = alloc_anon_inode(dlb_vfs_mount->mnt_sb); + if (IS_ERR(inode)) { + dev_err(dlb->dev, + "[%s()] Cannot allocate inode: %ld\n", + __func__, PTR_ERR(inode)); + dlb->inode_cnt = 0; + simple_release_fs(&dlb_vfs_mount, &dlb_fs_cnt); + } + + dlb->inode = inode; + + return inode; +} + +/* + * Decrement the inode reference count and release the FS. Intended for + * unwinding dlb_alloc_inode(). Must hold the resource mutex while calling. + */ +static void dlb_free_inode(struct inode *inode) +{ + iput(inode); + simple_release_fs(&dlb_vfs_mount, &dlb_fs_cnt); +} + +/* + * Release the FS. Intended for use in a file_operations release callback, + * which decrements the inode reference count separately. Must hold the + * resource mutex while calling. + */ +void dlb_release_fs(struct dlb *dlb) +{ + mutex_lock(&dlb_driver_mutex); + + simple_release_fs(&dlb_vfs_mount, &dlb_fs_cnt); + + dlb->inode_cnt--; + + /* When the fs refcnt reaches zero, the inode has been freed */ + if (dlb->inode_cnt == 0) + dlb->inode = NULL; + + mutex_unlock(&dlb_driver_mutex); +} + +/* + * Allocate a file with the requested flags, file operations, and name that + * uses the device's shared inode. Must hold the resource mutex while calling. + * + * Caller must separately allocate an fd and install the file in that fd. + */ +struct file *dlb_getfile(struct dlb *dlb, + int flags, + const struct file_operations *fops, + const char *name) +{ + struct inode *inode; + struct file *f; + + if (!try_module_get(THIS_MODULE)) + return ERR_PTR(-ENOENT); + + mutex_lock(&dlb_driver_mutex); + + inode = dlb_alloc_inode(dlb); + if (IS_ERR(inode)) { + mutex_unlock(&dlb_driver_mutex); + module_put(THIS_MODULE); + return ERR_CAST(inode); + } + + f = alloc_file_pseudo(inode, dlb_vfs_mount, name, flags, fops); + if (IS_ERR(f)) { + dlb_free_inode(inode); + mutex_unlock(&dlb_driver_mutex); + module_put(THIS_MODULE); + return f; + } + + mutex_unlock(&dlb_driver_mutex); + + return f; +} diff --git a/drivers/misc/dlb/dlb_hw_types.h b/drivers/misc/dlb/dlb_hw_types.h index c7827defa66a..b892a9dd172c 100644 --- a/drivers/misc/dlb/dlb_hw_types.h +++ b/drivers/misc/dlb/dlb_hw_types.h @@ -66,12 +66,12 @@ #define DLB_LDB_PP_STRIDE 0x1000 #define DLB_LDB_PP_BOUND (DLB_LDB_PP_BASE + \ DLB_LDB_PP_STRIDE * DLB_MAX_NUM_LDB_PORTS) -#define DLB_LDB_PP_OFFS(id) (DLB_LDB_PP_BASE + (id) * DLB_PP_SIZE) +#define DLB_LDB_PP_OFFSET(id) (DLB_LDB_PP_BASE + (id) * DLB_PP_SIZE) #define DLB_DIR_PP_BASE 0x2000000 #define DLB_DIR_PP_STRIDE 0x1000 #define DLB_DIR_PP_BOUND (DLB_DIR_PP_BASE + \ DLB_DIR_PP_STRIDE * DLB_MAX_NUM_DIR_PORTS) -#define DLB_DIR_PP_OFFS(id) (DLB_DIR_PP_BASE + (id) * DLB_PP_SIZE) +#define DLB_DIR_PP_OFFSET(id) (DLB_DIR_PP_BASE + (id) * DLB_PP_SIZE) struct dlb_resource_id { u32 phys_id; diff --git a/drivers/misc/dlb/dlb_ioctl.c b/drivers/misc/dlb/dlb_ioctl.c index 84bf833631bd..6a311b969643 100644 --- a/drivers/misc/dlb/dlb_ioctl.c +++ b/drivers/misc/dlb/dlb_ioctl.c @@ -161,6 +161,141 @@ static int dlb_domain_ioctl_create_dir_port(struct dlb *dlb, return ret; } +static int dlb_create_port_fd(struct dlb *dlb, const char *prefix, u32 id, + const struct file_operations *fops, + int *fd, struct file **f) +{ + char *name; + int ret; + + ret = get_unused_fd_flags(O_RDWR); + if (ret < 0) + return ret; + + *fd = ret; + + name = kasprintf(GFP_KERNEL, "%s:%d", prefix, id); + if (!name) { + put_unused_fd(*fd); + return -ENOMEM; + } + + *f = dlb_getfile(dlb, O_RDWR | O_CLOEXEC, fops, name); + + kfree(name); + + if (IS_ERR(*f)) { + put_unused_fd(*fd); + return PTR_ERR(*f); + } + + return 0; +} + +static int dlb_domain_get_port_fd(struct dlb *dlb, struct dlb_domain *domain, + unsigned long user_arg, const char *name, + const struct file_operations *fops, + bool is_ldb) +{ + struct dlb_get_port_fd_args __user *uarg; + struct dlb_cmd_response response = {0}; + struct dlb_get_port_fd_args arg; + struct dlb_port *port; + struct file *file; + int ret, fd; + + uarg = (void __user *)user_arg; + if (copy_from_user(&arg, uarg, sizeof(arg))) + return -EFAULT; + + mutex_lock(&dlb->resource_mutex); + + if ((is_ldb && + dlb->ops->ldb_port_owned_by_domain(&dlb->hw, domain->id, + arg.port_id) != 1)) { + response.status = DLB_ST_INVALID_PORT_ID; + ret = -EINVAL; + goto end; + } + + if (!is_ldb && + dlb->ops->dir_port_owned_by_domain(&dlb->hw, domain->id, + arg.port_id) != 1) { + response.status = DLB_ST_INVALID_PORT_ID; + ret = -EINVAL; + goto end; + } + + port = (is_ldb) ? &dlb->ldb_port[arg.port_id] : + &dlb->dir_port[arg.port_id]; + + if (!port->valid) { + response.status = DLB_ST_INVALID_PORT_ID; + ret = -EINVAL; + goto end; + } + + ret = dlb_create_port_fd(dlb, name, arg.port_id, fops, &fd, &file); + if (ret < 0) + goto end; + + file->private_data = port; + + response.id = fd; + +end: + BUILD_BUG_ON(offsetof(typeof(arg), response) != 0); + + if (copy_to_user((void __user *)&uarg->response, &response, sizeof(response))) + ret = -EFAULT; + + /* + * Save fd_install() until after the last point of failure. The domain + * refcnt is decremented in the close callback. + */ + if (ret == 0) { + kref_get(&domain->refcnt); + + fd_install(fd, file); + } + + mutex_unlock(&dlb->resource_mutex); + + return ret; +} + +static int dlb_domain_ioctl_get_ldb_port_pp_fd(struct dlb *dlb, + struct dlb_domain *domain, + unsigned long user_arg) +{ + return dlb_domain_get_port_fd(dlb, domain, user_arg, + "dlb_ldb_pp:", &dlb_pp_fops, true); +} + +static int dlb_domain_ioctl_get_ldb_port_cq_fd(struct dlb *dlb, + struct dlb_domain *domain, + unsigned long user_arg) +{ + return dlb_domain_get_port_fd(dlb, domain, user_arg, + "dlb_ldb_cq:", &dlb_cq_fops, true); +} + +static int dlb_domain_ioctl_get_dir_port_pp_fd(struct dlb *dlb, + struct dlb_domain *domain, + unsigned long user_arg) +{ + return dlb_domain_get_port_fd(dlb, domain, user_arg, + "dlb_dir_pp:", &dlb_pp_fops, false); +} + +static int dlb_domain_ioctl_get_dir_port_cq_fd(struct dlb *dlb, + struct dlb_domain *domain, + unsigned long user_arg) +{ + return dlb_domain_get_port_fd(dlb, domain, user_arg, + "dlb_dir_cq:", &dlb_cq_fops, false); +} + long dlb_domain_ioctl(struct file *f, unsigned int cmd, unsigned long arg) { struct dlb_domain *dom = f->private_data; @@ -179,6 +314,14 @@ long dlb_domain_ioctl(struct file *f, unsigned int cmd, unsigned long arg) return dlb_domain_ioctl_create_ldb_port(dlb, dom, arg); case DLB_IOC_CREATE_DIR_PORT: return dlb_domain_ioctl_create_dir_port(dlb, dom, arg); + case DLB_IOC_GET_LDB_PORT_PP_FD: + return dlb_domain_ioctl_get_ldb_port_pp_fd(dlb, dom, arg); + case DLB_IOC_GET_LDB_PORT_CQ_FD: + return dlb_domain_ioctl_get_ldb_port_cq_fd(dlb, dom, arg); + case DLB_IOC_GET_DIR_PORT_PP_FD: + return dlb_domain_ioctl_get_dir_port_pp_fd(dlb, dom, arg); + case DLB_IOC_GET_DIR_PORT_CQ_FD: + return dlb_domain_ioctl_get_dir_port_cq_fd(dlb, dom, arg); default: return -ENOTTY; } diff --git a/drivers/misc/dlb/dlb_main.c b/drivers/misc/dlb/dlb_main.c index e4c19714f1c4..69ab9b532ed4 100644 --- a/drivers/misc/dlb/dlb_main.c +++ b/drivers/misc/dlb/dlb_main.c @@ -17,6 +17,9 @@ MODULE_LICENSE("GPL v2"); MODULE_DESCRIPTION("Intel(R) Dynamic Load Balancer (DLB) Driver"); +/* The driver mutex protects data structures that used by multiple devices. */ +DEFINE_MUTEX(dlb_driver_mutex); + static struct class *dlb_class; static struct cdev dlb_cdev; static dev_t dlb_devt; @@ -233,6 +236,121 @@ const struct file_operations dlb_domain_fops = { .compat_ioctl = compat_ptr_ioctl, }; +static unsigned long dlb_get_pp_addr(struct dlb *dlb, struct dlb_port *port) +{ + unsigned long pgoff = dlb->hw.func_phys_addr; + + if (port->is_ldb) + pgoff += DLB_LDB_PP_OFFSET(port->id); + else + pgoff += DLB_DIR_PP_OFFSET(port->id); + + return pgoff; +} + +static int dlb_pp_mmap(struct file *f, struct vm_area_struct *vma) +{ + struct dlb_port *port = f->private_data; + struct dlb_domain *domain = port->domain; + struct dlb *dlb = domain->dlb; + unsigned long pgoff; + pgprot_t pgprot; + int ret; + + mutex_lock(&dlb->resource_mutex); + + if ((vma->vm_end - vma->vm_start) != DLB_PP_SIZE) { + ret = -EINVAL; + goto end; + } + + pgprot = pgprot_noncached(vma->vm_page_prot); + + pgoff = dlb_get_pp_addr(dlb, port); + ret = io_remap_pfn_range(vma, + vma->vm_start, + pgoff >> PAGE_SHIFT, + vma->vm_end - vma->vm_start, + pgprot); + +end: + mutex_unlock(&dlb->resource_mutex); + + return ret; +} + +static int dlb_cq_mmap(struct file *f, struct vm_area_struct *vma) +{ + struct dlb_port *port = f->private_data; + struct dlb_domain *domain = port->domain; + struct dlb *dlb = domain->dlb; + struct page *page; + int ret; + + mutex_lock(&dlb->resource_mutex); + + if ((vma->vm_end - vma->vm_start) != DLB_CQ_SIZE) { + ret = -EINVAL; + goto end; + } + + page = virt_to_page(port->cq_base); + + ret = remap_pfn_range(vma, + vma->vm_start, + page_to_pfn(page), + vma->vm_end - vma->vm_start, + vma->vm_page_prot); +end: + mutex_unlock(&dlb->resource_mutex); + + return ret; +} + +static void dlb_port_unmap(struct dlb *dlb, struct dlb_port *port) +{ + if (!port->cq_base) { + unmap_mapping_range(dlb->inode->i_mapping, + (unsigned long)port->cq_base, + DLB_CQ_SIZE, 1); + } else { + unmap_mapping_range(dlb->inode->i_mapping, + dlb_get_pp_addr(dlb, port), + DLB_PP_SIZE, 1); + } +} + +static int dlb_port_close(struct inode *i, struct file *f) +{ + struct dlb_port *port = f->private_data; + struct dlb_domain *domain = port->domain; + struct dlb *dlb = domain->dlb; + + mutex_lock(&dlb->resource_mutex); + + kref_put(&domain->refcnt, dlb_free_domain); + + dlb_port_unmap(dlb, port); + /* Decrement the refcnt of the pseudo-FS used to allocate the inode */ + dlb_release_fs(dlb); + + mutex_unlock(&dlb->resource_mutex); + + return 0; +} + +const struct file_operations dlb_pp_fops = { + .owner = THIS_MODULE, + .release = dlb_port_close, + .mmap = dlb_pp_mmap, +}; + +const struct file_operations dlb_cq_fops = { + .owner = THIS_MODULE, + .release = dlb_port_close, + .mmap = dlb_cq_mmap, +}; + /**********************************/ /****** PCI driver callbacks ******/ /**********************************/ diff --git a/drivers/misc/dlb/dlb_main.h b/drivers/misc/dlb/dlb_main.h index 08dead13fb11..477974e1a178 100644 --- a/drivers/misc/dlb/dlb_main.h +++ b/drivers/misc/dlb/dlb_main.h @@ -11,6 +11,7 @@ #include #include #include +#include #include @@ -27,6 +28,8 @@ #define DLB_NUM_FUNCS_PER_DEVICE (1 + DLB_MAX_NUM_VDEVS) #define DLB_MAX_NUM_DEVICES (DLB_MAX_NUM_PFS * DLB_NUM_FUNCS_PER_DEVICE) +extern struct mutex dlb_driver_mutex; + enum dlb_device_type { DLB_PF, }; @@ -63,6 +66,12 @@ struct dlb_device_ops { int (*get_num_resources)(struct dlb_hw *hw, struct dlb_get_num_resources_args *args); int (*reset_domain)(struct dlb_hw *hw, u32 domain_id); + int (*ldb_port_owned_by_domain)(struct dlb_hw *hw, + u32 domain_id, + u32 port_id); + int (*dir_port_owned_by_domain)(struct dlb_hw *hw, + u32 domain_id, + u32 port_id); int (*get_ldb_queue_depth)(struct dlb_hw *hw, u32 domain_id, struct dlb_get_ldb_queue_depth_args *args, @@ -78,6 +87,8 @@ struct dlb_device_ops { extern struct dlb_device_ops dlb_pf_ops; extern const struct file_operations dlb_domain_fops; +extern const struct file_operations dlb_pp_fops; +extern const struct file_operations dlb_cq_fops; struct dlb_port { void *cq_base; @@ -103,6 +114,11 @@ struct dlb { struct file *f; struct dlb_port ldb_port[DLB_MAX_NUM_LDB_PORTS]; struct dlb_port dir_port[DLB_MAX_NUM_DIR_PORTS]; + /* + * Anonymous inode used to share an address_space for all domain + * device file mappings. + */ + struct inode *inode; /* * The resource mutex serializes access to driver data structures and * hardware registers. @@ -110,6 +126,7 @@ struct dlb { struct mutex resource_mutex; enum dlb_device_type type; int id; + u32 inode_cnt; dev_t dev_number; u8 domain_reset_failed; }; @@ -118,6 +135,13 @@ struct dlb { long dlb_ioctl(struct file *f, unsigned int cmd, unsigned long arg); long dlb_domain_ioctl(struct file *f, unsigned int cmd, unsigned long arg); +/* Prototypes for dlb_file.c */ +void dlb_release_fs(struct dlb *dlb); +struct file *dlb_getfile(struct dlb *dlb, + int flags, + const struct file_operations *fops, + const char *name); + int dlb_init_domain(struct dlb *dlb, u32 domain_id); void dlb_free_domain(struct kref *kref); diff --git a/drivers/misc/dlb/dlb_pf_ops.c b/drivers/misc/dlb/dlb_pf_ops.c index c2ce03114f8b..02a188aa5a60 100644 --- a/drivers/misc/dlb/dlb_pf_ops.c +++ b/drivers/misc/dlb/dlb_pf_ops.c @@ -201,6 +201,22 @@ dlb_pf_query_cq_poll_mode(struct dlb *dlb, struct dlb_cmd_response *user_resp) return 0; } +/**************************************/ +/****** Resource query callbacks ******/ +/**************************************/ + +static int +dlb_pf_ldb_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id) +{ + return dlb_ldb_port_owned_by_domain(hw, domain_id, port_id, false, 0); +} + +static int +dlb_pf_dir_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id) +{ + return dlb_dir_port_owned_by_domain(hw, domain_id, port_id, false, 0); +} + /********************************/ /****** DLB PF Device Ops ******/ /********************************/ @@ -218,6 +234,8 @@ struct dlb_device_ops dlb_pf_ops = { .create_dir_port = dlb_pf_create_dir_port, .get_num_resources = dlb_pf_get_num_resources, .reset_domain = dlb_pf_reset_domain, + .ldb_port_owned_by_domain = dlb_pf_ldb_port_owned_by_domain, + .dir_port_owned_by_domain = dlb_pf_dir_port_owned_by_domain, .get_ldb_queue_depth = dlb_pf_get_ldb_queue_depth, .get_dir_queue_depth = dlb_pf_get_dir_queue_depth, .init_hardware = dlb_pf_init_hardware, diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index 822c1f4f7849..2659190527a7 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -236,6 +236,32 @@ static struct dlb_hw_domain *dlb_get_domain_from_id(struct dlb_hw *hw, u32 id, return NULL; } +static struct dlb_ldb_port * +dlb_get_domain_ldb_port(u32 id, bool vdev_req, struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + if (id >= DLB_MAX_NUM_LDB_PORTS) + return NULL; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + if ((!vdev_req && port->id.phys_id == id) || + (vdev_req && port->id.virt_id == id)) + return port; + } + + list_for_each_entry(port, &domain->avail_ldb_ports[i], domain_list) { + if ((!vdev_req && port->id.phys_id == id) || + (vdev_req && port->id.virt_id == id)) + return port; + } + } + + return NULL; +} + static struct dlb_dir_pq_pair * dlb_get_domain_used_dir_pq(u32 id, bool vdev_req, struct dlb_hw_domain *domain) { @@ -253,6 +279,29 @@ dlb_get_domain_used_dir_pq(u32 id, bool vdev_req, struct dlb_hw_domain *domain) return NULL; } +static struct dlb_dir_pq_pair * +dlb_get_domain_dir_pq(u32 id, bool vdev_req, struct dlb_hw_domain *domain) +{ + struct dlb_dir_pq_pair *port; + + if (id >= DLB_MAX_NUM_DIR_PORTS) + return NULL; + + list_for_each_entry(port, &domain->used_dir_pq_pairs, domain_list) { + if ((!vdev_req && port->id.phys_id == id) || + (vdev_req && port->id.virt_id == id)) + return port; + } + + list_for_each_entry(port, &domain->avail_dir_pq_pairs, domain_list) { + if ((!vdev_req && port->id.phys_id == id) || + (vdev_req && port->id.virt_id == id)) + return port; + } + + return NULL; +} + static struct dlb_ldb_queue * dlb_get_domain_ldb_queue(u32 id, bool vdev_req, struct dlb_hw_domain *domain) { @@ -3394,6 +3443,88 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, return dlb_domain_reset_software_state(hw, domain); } +/** + * dlb_ldb_port_owned_by_domain() - query whether a port is owned by a domain + * @hw: dlb_hw handle for a particular device. + * @domain_id: domain ID. + * @port_id: port ID. + * @vdev_req: indicates whether this request came from a vdev. + * @vdev_id: If vdev_req is true, this contains the vdev's ID. + * + * This function returns whether a load-balanced port is owned by a specified + * domain. + * + * A vdev can be either an SR-IOV virtual function or a Scalable IOV virtual + * device. + * + * Return: + * Returns 0 if false, 1 if true, <0 otherwise. + * + * EINVAL - Invalid domain or port ID, or the domain is not configured. + */ +int dlb_ldb_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id, + bool vdev_req, unsigned int vdev_id) +{ + struct dlb_hw_domain *domain; + struct dlb_ldb_port *port; + + if (vdev_req && vdev_id >= DLB_MAX_NUM_VDEVS) + return -EINVAL; + + domain = dlb_get_domain_from_id(hw, domain_id, vdev_req, vdev_id); + + if (!domain || !domain->configured) + return -EINVAL; + + port = dlb_get_domain_ldb_port(port_id, vdev_req, domain); + + if (!port) + return -EINVAL; + + return port->domain_id.phys_id == domain->id.phys_id; +} + +/** + * dlb_dir_port_owned_by_domain() - query whether a port is owned by a domain + * @hw: dlb_hw handle for a particular device. + * @domain_id: domain ID. + * @port_id: port ID. + * @vdev_req: indicates whether this request came from a vdev. + * @vdev_id: If vdev_req is true, this contains the vdev's ID. + * + * This function returns whether a directed port is owned by a specified + * domain. + * + * A vdev can be either an SR-IOV virtual function or a Scalable IOV virtual + * device. + * + * Return: + * Returns 0 if false, 1 if true, <0 otherwise. + * + * EINVAL - Invalid domain or port ID, or the domain is not configured. + */ +int dlb_dir_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id, + bool vdev_req, unsigned int vdev_id) +{ + struct dlb_dir_pq_pair *port; + struct dlb_hw_domain *domain; + + if (vdev_req && vdev_id >= DLB_MAX_NUM_VDEVS) + return -EINVAL; + + domain = dlb_get_domain_from_id(hw, domain_id, vdev_req, vdev_id); + + if (!domain || !domain->configured) + return -EINVAL; + + port = dlb_get_domain_dir_pq(port_id, vdev_req, domain); + + if (!port) + return -EINVAL; + + return port->domain_id.phys_id == domain->id.phys_id; +} + /** * dlb_hw_get_num_resources() - query the PCI function's available resources * @hw: dlb_hw handle for a particular device. diff --git a/drivers/misc/dlb/dlb_resource.h b/drivers/misc/dlb/dlb_resource.h index bbe25a417cd4..8a3c37b6ab92 100644 --- a/drivers/misc/dlb/dlb_resource.h +++ b/drivers/misc/dlb/dlb_resource.h @@ -44,6 +44,12 @@ int dlb_hw_create_ldb_port(struct dlb_hw *hw, u32 domain_id, int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, unsigned int vdev_id); +int dlb_ldb_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id, + bool vdev_req, unsigned int vdev_id); + +int dlb_dir_port_owned_by_domain(struct dlb_hw *hw, u32 domain_id, u32 port_id, + bool vdev_req, unsigned int vdev_id); + int dlb_hw_get_num_resources(struct dlb_hw *hw, struct dlb_get_num_resources_args *arg, bool vdev_req, unsigned int vdev_id); diff --git a/include/uapi/linux/dlb.h b/include/uapi/linux/dlb.h index 9578d8f1c03b..6b7eceecae8a 100644 --- a/include/uapi/linux/dlb.h +++ b/include/uapi/linux/dlb.h @@ -374,6 +374,40 @@ struct dlb_create_dir_port_args { __s32 queue_id; }; +/* + * DLB_CMD_GET_LDB_PORT_PP_FD: Get file descriptor to mmap a load-balanced + * port's producer port (PP). + * DLB_CMD_GET_LDB_PORT_CQ_FD: Get file descriptor to mmap a load-balanced + * port's consumer queue (CQ). + * + * The load-balanced port must have been previously created with the ioctl + * DLB_CMD_CREATE_LDB_PORT. The fd is used to mmap the PP/CQ region. + * + * DLB_CMD_GET_DIR_PORT_PP_FD: Get file descriptor to mmap a directed port's + * producer port (PP). + * DLB_CMD_GET_DIR_PORT_CQ_FD: Get file descriptor to mmap a directed port's + * consumer queue (CQ). + * + * The directed port must have been previously created with the ioctl + * DLB_CMD_CREATE_DIR_PORT. The fd is used to mmap PP/CQ region. + * + * Output parameters: + * @response.status: Detailed error code. In certain cases, such as if the + * ioctl request arg is invalid, the driver won't set status. + * @response.id: fd. + * + * Input parameters: + * @port_id: port ID. + * @padding0: Reserved for future use. + */ +struct dlb_get_port_fd_args { + /* Output parameters */ + struct dlb_cmd_response response; + /* Input parameters */ + __u32 port_id; + __u32 padding0; +}; + enum dlb_domain_user_interface_commands { DLB_DOMAIN_CMD_CREATE_LDB_QUEUE, DLB_DOMAIN_CMD_CREATE_DIR_QUEUE, @@ -381,12 +415,21 @@ enum dlb_domain_user_interface_commands { DLB_DOMAIN_CMD_GET_DIR_QUEUE_DEPTH, DLB_DOMAIN_CMD_CREATE_LDB_PORT, DLB_DOMAIN_CMD_CREATE_DIR_PORT, + DLB_DOMAIN_CMD_GET_LDB_PORT_PP_FD, + DLB_DOMAIN_CMD_GET_LDB_PORT_CQ_FD, + DLB_DOMAIN_CMD_GET_DIR_PORT_PP_FD, + DLB_DOMAIN_CMD_GET_DIR_PORT_CQ_FD, /* NUM_DLB_DOMAIN_CMD must be last */ NUM_DLB_DOMAIN_CMD, }; +/* + * Mapping sizes for memory mapping the consumer queue (CQ) memory space, and + * producer port (PP) MMIO space. + */ #define DLB_CQ_SIZE 65536 +#define DLB_PP_SIZE 4096 /********************/ /* dlb ioctl codes */ @@ -434,5 +477,21 @@ enum dlb_domain_user_interface_commands { _IOWR(DLB_IOC_MAGIC, \ DLB_DOMAIN_CMD_CREATE_DIR_PORT, \ struct dlb_create_dir_port_args) +#define DLB_IOC_GET_LDB_PORT_PP_FD \ + _IOWR(DLB_IOC_MAGIC, \ + DLB_DOMAIN_CMD_GET_LDB_PORT_PP_FD, \ + struct dlb_get_port_fd_args) +#define DLB_IOC_GET_LDB_PORT_CQ_FD \ + _IOWR(DLB_IOC_MAGIC, \ + DLB_DOMAIN_CMD_GET_LDB_PORT_CQ_FD, \ + struct dlb_get_port_fd_args) +#define DLB_IOC_GET_DIR_PORT_PP_FD \ + _IOWR(DLB_IOC_MAGIC, \ + DLB_DOMAIN_CMD_GET_DIR_PORT_PP_FD, \ + struct dlb_get_port_fd_args) +#define DLB_IOC_GET_DIR_PORT_CQ_FD \ + _IOWR(DLB_IOC_MAGIC, \ + DLB_DOMAIN_CMD_GET_DIR_PORT_CQ_FD, \ + struct dlb_get_port_fd_args) #endif /* __DLB_H */ From patchwork Wed Feb 10 17:54:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9846C433E9 for ; Wed, 10 Feb 2021 18:03:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 913C664EE6 for ; Wed, 10 Feb 2021 18:03:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233764AbhBJSCl (ORCPT ); Wed, 10 Feb 2021 13:02:41 -0500 Received: from mga01.intel.com ([192.55.52.88]:60432 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233711AbhBJR70 (ORCPT ); Wed, 10 Feb 2021 12:59:26 -0500 IronPort-SDR: ZBr1bWhMF3VTCrotjr4HY7Hv9sG6KRZDrR5A6Vo2RRQ5ig/KwJrNmIwzZtUMZ2r95VYmXUviyl +rbcs0Cpe/Vg== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236044" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236044" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:13 -0800 IronPort-SDR: zJ6XmATtoSSz2EdaU2ZHFPbIDLvCXNlVenBhVqOU0pgOoGQ2UWfTjItWD9XBn0m0uhrKHBmJTL NhBD3kYPbcGA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235813" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:13 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 17/20] dlb: add static queue map register operations Date: Wed, 10 Feb 2021 11:54:20 -0600 Message-Id: <20210210175423.1873-18-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add the register accesses that implement the static queue map operation and handle an unmap request when a queue map operation is in progress. If a queue map operation is requested before the domain is started, it is a synchronous procedure on "static"/unchanging hardware. (The "dynamic" operation, when traffic is flowing in the device, will be added in a later commit.) Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Björn Töpel Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_resource.c | 144 +++++++++++++++++++++++++++++++- 1 file changed, 141 insertions(+), 3 deletions(-) diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index a830b547dadf..95ccb7eddb8b 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -2122,19 +2122,150 @@ static int dlb_configure_dir_port(struct dlb_hw *hw, struct dlb_hw_domain *domai return 0; } +static int dlb_ldb_port_map_qid_static(struct dlb_hw *hw, struct dlb_ldb_port *p, + struct dlb_ldb_queue *q, u8 priority) +{ + enum dlb_qid_map_state state; + u32 lsp_qid2cq2; + u32 lsp_qid2cq; + u32 atm_qid2cq; + u32 cq2priov; + u32 cq2qid; + int i; + + /* Look for a pending or already mapped slot, else an unused slot */ + if (!dlb_port_find_slot_queue(p, DLB_QUEUE_MAP_IN_PROG, q, &i) && + !dlb_port_find_slot_queue(p, DLB_QUEUE_MAPPED, q, &i) && + !dlb_port_find_slot(p, DLB_QUEUE_UNMAPPED, &i)) { + DLB_HW_ERR(hw, + "[%s():%d] Internal error: CQ has no available QID mapping slots\n", + __func__, __LINE__); + return -EFAULT; + } + + /* Read-modify-write the priority and valid bit register */ + cq2priov = DLB_CSR_RD(hw, LSP_CQ2PRIOV(p->id.phys_id)); + + cq2priov |= (1U << (i + LSP_CQ2PRIOV_V_LOC)) & LSP_CQ2PRIOV_V; + cq2priov |= ((priority & 0x7) << (i + LSP_CQ2PRIOV_PRIO_LOC) * 3) + & LSP_CQ2PRIOV_PRIO; + + DLB_CSR_WR(hw, LSP_CQ2PRIOV(p->id.phys_id), cq2priov); + + /* Read-modify-write the QID map register */ + if (i < 4) + cq2qid = DLB_CSR_RD(hw, LSP_CQ2QID0(p->id.phys_id)); + else + cq2qid = DLB_CSR_RD(hw, LSP_CQ2QID1(p->id.phys_id)); + + if (i == 0 || i == 4) + BITS_SET(cq2qid, q->id.phys_id, LSP_CQ2QID0_QID_P0); + if (i == 1 || i == 5) + BITS_SET(cq2qid, q->id.phys_id, LSP_CQ2QID0_QID_P1); + if (i == 2 || i == 6) + BITS_SET(cq2qid, q->id.phys_id, LSP_CQ2QID0_QID_P2); + if (i == 3 || i == 7) + BITS_SET(cq2qid, q->id.phys_id, LSP_CQ2QID0_QID_P3); + + if (i < 4) + DLB_CSR_WR(hw, LSP_CQ2QID0(p->id.phys_id), cq2qid); + else + DLB_CSR_WR(hw, LSP_CQ2QID1(p->id.phys_id), cq2qid); + + atm_qid2cq = DLB_CSR_RD(hw, + ATM_QID2CQIDIX(q->id.phys_id, + p->id.phys_id / 4)); + + lsp_qid2cq = DLB_CSR_RD(hw, + LSP_QID2CQIDIX(q->id.phys_id, + p->id.phys_id / 4)); + + lsp_qid2cq2 = DLB_CSR_RD(hw, + LSP_QID2CQIDIX2(q->id.phys_id, + p->id.phys_id / 4)); + + switch (p->id.phys_id % 4) { + case 0: + atm_qid2cq |= (1 << (i + ATM_QID2CQIDIX_00_CQ_P0_LOC)); + lsp_qid2cq |= (1 << (i + LSP_QID2CQIDIX_00_CQ_P0_LOC)); + lsp_qid2cq2 |= (1 << (i + LSP_QID2CQIDIX2_00_CQ_P0_LOC)); + break; + + case 1: + atm_qid2cq |= (1 << (i + ATM_QID2CQIDIX_00_CQ_P1_LOC)); + lsp_qid2cq |= (1 << (i + LSP_QID2CQIDIX_00_CQ_P1_LOC)); + lsp_qid2cq2 |= (1 << (i + LSP_QID2CQIDIX2_00_CQ_P1_LOC)); + break; + + case 2: + atm_qid2cq |= (1 << (i + ATM_QID2CQIDIX_00_CQ_P2_LOC)); + lsp_qid2cq |= (1 << (i + LSP_QID2CQIDIX_00_CQ_P2_LOC)); + lsp_qid2cq2 |= (1 << (i + LSP_QID2CQIDIX2_00_CQ_P2_LOC)); + break; + + case 3: + atm_qid2cq |= (1 << (i + ATM_QID2CQIDIX_00_CQ_P3_LOC)); + lsp_qid2cq |= (1 << (i + LSP_QID2CQIDIX_00_CQ_P3_LOC)); + lsp_qid2cq2 |= (1 << (i + LSP_QID2CQIDIX2_00_CQ_P3_LOC)); + break; + } + + DLB_CSR_WR(hw, + ATM_QID2CQIDIX(q->id.phys_id, p->id.phys_id / 4), + atm_qid2cq); + + DLB_CSR_WR(hw, + LSP_QID2CQIDIX(q->id.phys_id, p->id.phys_id / 4), + lsp_qid2cq); + + DLB_CSR_WR(hw, + LSP_QID2CQIDIX2(q->id.phys_id, p->id.phys_id / 4), + lsp_qid2cq2); + + dlb_flush_csr(hw); + + p->qid_map[i].qid = q->id.phys_id; + p->qid_map[i].priority = priority; + + state = DLB_QUEUE_MAPPED; + + return dlb_port_slot_state_transition(hw, p, q, i, state); +} + static void dlb_ldb_port_change_qid_priority(struct dlb_hw *hw, struct dlb_ldb_port *port, int slot, struct dlb_map_qid_args *args) { - /* Placeholder */ + u32 cq2priov; + + /* Read-modify-write the priority and valid bit register */ + cq2priov = DLB_CSR_RD(hw, LSP_CQ2PRIOV(port->id.phys_id)); + + cq2priov |= (1 << (slot + LSP_CQ2PRIOV_V_LOC)) & LSP_CQ2PRIOV_V; + cq2priov |= ((args->priority & 0x7) << slot * 3) & LSP_CQ2PRIOV_PRIO; + + DLB_CSR_WR(hw, LSP_CQ2PRIOV(port->id.phys_id), cq2priov); + + dlb_flush_csr(hw); + + port->qid_map[slot].priority = args->priority; +} + +static void dlb_ldb_queue_set_inflight_limit(struct dlb_hw *hw, + struct dlb_ldb_queue *queue) +{ + u32 infl_lim = 0; + + BITS_SET(infl_lim, queue->num_qid_inflights, LSP_QID_LDB_INFL_LIM_LIMIT); + + DLB_CSR_WR(hw, LSP_QID_LDB_INFL_LIM(queue->id.phys_id), infl_lim); } static int dlb_ldb_port_map_qid(struct dlb_hw *hw, struct dlb_hw_domain *domain, struct dlb_ldb_port *port, struct dlb_ldb_queue *queue, u8 prio) { - /* Placeholder */ - return 0; + return dlb_ldb_port_map_qid_static(hw, port, queue, prio); } static void @@ -2872,6 +3003,13 @@ int dlb_hw_unmap_qid(struct dlb_hw *hw, u32 domain_id, */ st = DLB_QUEUE_MAP_IN_PROG; if (dlb_port_find_slot_queue(port, st, queue, &i)) { + /* + * Since the in-progress map was aborted, re-enable the QID's + * inflights. + */ + if (queue->num_pending_additions == 0) + dlb_ldb_queue_set_inflight_limit(hw, queue); + st = DLB_QUEUE_UNMAPPED; ret = dlb_port_slot_state_transition(hw, port, queue, i, st); if (ret) From patchwork Wed Feb 10 17:54:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Chen, Mike Ximing" X-Patchwork-Id: 380698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D00EC433E0 for ; Wed, 10 Feb 2021 18:02:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D6A064EDF for ; Wed, 10 Feb 2021 18:02:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232149AbhBJSCB (ORCPT ); Wed, 10 Feb 2021 13:02:01 -0500 Received: from mga01.intel.com ([192.55.52.88]:60436 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232732AbhBJR72 (ORCPT ); Wed, 10 Feb 2021 12:59:28 -0500 IronPort-SDR: 8P4cz92t4RWG5slcnYM0BlNzlnm0t3ndDV/46FnBh/OGrujGAKmQ1f+NMjmKZHzkSlXpBy/QDj meW6CN1+uRGg== X-IronPort-AV: E=McAfee;i="6000,8403,9891"; a="201236048" X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="201236048" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 09:56:14 -0800 IronPort-SDR: zoTi9p+HQMlNlyThxik+9u3uKhzsmO+OVWEfwUDgVk90qKzCQpBmr7YeLZ0U8O1ISu6Z7dmO4L 0DeC+dHQMm6Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,168,1610438400"; d="scan'208";a="380235823" Received: from txasoft-yocto.an.intel.com ([10.123.72.192]) by fmsmga008.fm.intel.com with ESMTP; 10 Feb 2021 09:56:13 -0800 From: Mike Ximing Chen To: netdev@vger.kernel.org Cc: davem@davemloft.net, kuba@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, pierre-louis.bossart@linux.intel.com, Gage Eads Subject: [PATCH v10 18/20] dlb: add dynamic queue map register operations Date: Wed, 10 Feb 2021 11:54:21 -0600 Message-Id: <20210210175423.1873-19-mike.ximing.chen@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20210210175423.1873-1-mike.ximing.chen@intel.com> References: <20210210175423.1873-1-mike.ximing.chen@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Adds the "dynamic" map procedure and register operations. If a queue map is requested after the domain is started, the driver must disable the requested queue and wait for it to quiesce before mapping it to the requested port. Signed-off-by: Gage Eads Signed-off-by: Mike Ximing Chen Reviewed-by: Björn Töpel Reviewed-by: Dan Williams --- drivers/misc/dlb/dlb_resource.c | 393 +++++++++++++++++++++++++++++++- 1 file changed, 392 insertions(+), 1 deletion(-) diff --git a/drivers/misc/dlb/dlb_resource.c b/drivers/misc/dlb/dlb_resource.c index 95ccb7eddb8b..93a3de642024 100644 --- a/drivers/misc/dlb/dlb_resource.c +++ b/drivers/misc/dlb/dlb_resource.c @@ -328,6 +328,37 @@ dlb_get_domain_dir_pq(u32 id, bool vdev_req, struct dlb_hw_domain *domain) return NULL; } +static struct dlb_ldb_queue * +dlb_get_ldb_queue_from_id(struct dlb_hw *hw, u32 id, bool vdev_req, + unsigned int vdev_id) +{ + struct dlb_function_resources *rsrcs; + struct dlb_hw_domain *domain; + struct dlb_ldb_queue *queue; + + if (id >= DLB_MAX_NUM_LDB_QUEUES) + return NULL; + + rsrcs = (vdev_req) ? &hw->vdev[vdev_id] : &hw->pf; + + if (!vdev_req) + return &hw->rsrcs.ldb_queues[id]; + + list_for_each_entry(domain, &rsrcs->used_domains, func_list) { + list_for_each_entry(queue, &domain->used_ldb_queues, domain_list) { + if (queue->id.virt_id == id) + return queue; + } + } + + list_for_each_entry(queue, &rsrcs->avail_ldb_queues, func_list) { + if (queue->id.virt_id == id) + return queue; + } + + return NULL; +} + static struct dlb_ldb_queue * dlb_get_domain_ldb_queue(u32 id, bool vdev_req, struct dlb_hw_domain *domain) { @@ -2251,6 +2282,75 @@ static void dlb_ldb_port_change_qid_priority(struct dlb_hw *hw, port->qid_map[slot].priority = args->priority; } +static int dlb_ldb_port_set_has_work_bits(struct dlb_hw *hw, + struct dlb_ldb_port *port, + struct dlb_ldb_queue *queue, int slot) +{ + u32 ctrl = 0; + u32 active; + u32 enq; + + /* Set the atomic scheduling haswork bit */ + active = DLB_CSR_RD(hw, LSP_QID_AQED_ACTIVE_CNT(queue->id.phys_id)); + + BITS_SET(ctrl, port->id.phys_id, LSP_LDB_SCHED_CTRL_CQ); + BITS_SET(ctrl, slot, LSP_LDB_SCHED_CTRL_QIDIX); + ctrl |= LSP_LDB_SCHED_CTRL_VALUE; + BITS_SET(ctrl, (u32)(BITS_GET(active, LSP_QID_AQED_ACTIVE_CNT_COUNT) > 0), + LSP_LDB_SCHED_CTRL_RLIST_HASWORK_V); + + /* Set the non-atomic scheduling haswork bit */ + DLB_CSR_WR(hw, LSP_LDB_SCHED_CTRL, ctrl); + + enq = DLB_CSR_RD(hw, + LSP_QID_LDB_ENQUEUE_CNT(queue->id.phys_id)); + + memset(&ctrl, 0, sizeof(ctrl)); + + BITS_SET(ctrl, port->id.phys_id, LSP_LDB_SCHED_CTRL_CQ); + BITS_SET(ctrl, slot, LSP_LDB_SCHED_CTRL_QIDIX); + ctrl |= LSP_LDB_SCHED_CTRL_VALUE; + BITS_SET(ctrl, (u32)(BITS_GET(enq, LSP_QID_LDB_ENQUEUE_CNT_COUNT) > 0), + LSP_LDB_SCHED_CTRL_NALB_HASWORK_V); + + DLB_CSR_WR(hw, LSP_LDB_SCHED_CTRL, ctrl); + + dlb_flush_csr(hw); + + return 0; +} + +static void dlb_ldb_port_clear_queue_if_status(struct dlb_hw *hw, + struct dlb_ldb_port *port, + int slot) +{ + u32 ctrl = 0; + + BITS_SET(ctrl, port->id.phys_id, LSP_LDB_SCHED_CTRL_CQ); + BITS_SET(ctrl, slot, LSP_LDB_SCHED_CTRL_QIDIX); + ctrl |= LSP_LDB_SCHED_CTRL_INFLIGHT_OK_V; + + DLB_CSR_WR(hw, LSP_LDB_SCHED_CTRL, ctrl); + + dlb_flush_csr(hw); +} + +static void dlb_ldb_port_set_queue_if_status(struct dlb_hw *hw, + struct dlb_ldb_port *port, + int slot) +{ + u32 ctrl = 0; + + BITS_SET(ctrl, port->id.phys_id, LSP_LDB_SCHED_CTRL_CQ); + BITS_SET(ctrl, slot, LSP_LDB_SCHED_CTRL_QIDIX); + ctrl |= LSP_LDB_SCHED_CTRL_VALUE; + ctrl |= LSP_LDB_SCHED_CTRL_INFLIGHT_OK_V; + + DLB_CSR_WR(hw, LSP_LDB_SCHED_CTRL, ctrl); + + dlb_flush_csr(hw); +} + static void dlb_ldb_queue_set_inflight_limit(struct dlb_hw *hw, struct dlb_ldb_queue *queue) { @@ -2261,11 +2361,222 @@ static void dlb_ldb_queue_set_inflight_limit(struct dlb_hw *hw, DLB_CSR_WR(hw, LSP_QID_LDB_INFL_LIM(queue->id.phys_id), infl_lim); } +static void dlb_ldb_queue_clear_inflight_limit(struct dlb_hw *hw, + struct dlb_ldb_queue *queue) +{ + DLB_CSR_WR(hw, + LSP_QID_LDB_INFL_LIM(queue->id.phys_id), + LSP_QID_LDB_INFL_LIM_RST); +} + +/* + * dlb_ldb_queue_{enable, disable}_mapped_cqs() don't operate exactly as + * their function names imply, and should only be called by the dynamic CQ + * mapping code. + */ +static void dlb_ldb_queue_disable_mapped_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_queue *queue) +{ + struct dlb_ldb_port *port; + int slot, i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + enum dlb_qid_map_state state = DLB_QUEUE_MAPPED; + + if (!dlb_port_find_slot_queue(port, state, + queue, &slot)) + continue; + + if (port->enabled) + dlb_ldb_port_cq_disable(hw, port); + } + } +} + +static void dlb_ldb_queue_enable_mapped_cqs(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_queue *queue) +{ + struct dlb_ldb_port *port; + int slot, i; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + enum dlb_qid_map_state state = DLB_QUEUE_MAPPED; + + if (!dlb_port_find_slot_queue(port, state, + queue, &slot)) + continue; + + if (port->enabled) + dlb_ldb_port_cq_enable(hw, port); + } + } +} + +static int dlb_ldb_port_finish_map_qid_dynamic(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_port *port, + struct dlb_ldb_queue *queue) +{ + enum dlb_qid_map_state state; + int slot, ret, i; + u32 infl_cnt; + u8 prio; + + infl_cnt = DLB_CSR_RD(hw, LSP_QID_LDB_INFL_CNT(queue->id.phys_id)); + + if (BITS_GET(infl_cnt, LSP_QID_LDB_INFL_CNT_COUNT)) { + DLB_HW_ERR(hw, + "[%s()] Internal error: non-zero QID inflight count\n", + __func__); + return -EINVAL; + } + + /* + * Static map the port and set its corresponding has_work bits. + */ + state = DLB_QUEUE_MAP_IN_PROG; + if (!dlb_port_find_slot_queue(port, state, queue, &slot)) + return -EINVAL; + + prio = port->qid_map[slot].priority; + + /* + * Update the CQ2QID, CQ2PRIOV, and QID2CQIDX registers, and + * the port's qid_map state. + */ + ret = dlb_ldb_port_map_qid_static(hw, port, queue, prio); + if (ret) + return ret; + + ret = dlb_ldb_port_set_has_work_bits(hw, port, queue, slot); + if (ret) + return ret; + + /* + * Ensure IF_status(cq,qid) is 0 before enabling the port to + * prevent spurious schedules to cause the queue's inflight + * count to increase. + */ + dlb_ldb_port_clear_queue_if_status(hw, port, slot); + + /* Reset the queue's inflight status */ + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) { + state = DLB_QUEUE_MAPPED; + if (!dlb_port_find_slot_queue(port, state, + queue, &slot)) + continue; + + dlb_ldb_port_set_queue_if_status(hw, port, slot); + } + } + + dlb_ldb_queue_set_inflight_limit(hw, queue); + + /* Re-enable CQs mapped to this queue */ + dlb_ldb_queue_enable_mapped_cqs(hw, domain, queue); + + /* If this queue has other mappings pending, clear its inflight limit */ + if (queue->num_pending_additions > 0) + dlb_ldb_queue_clear_inflight_limit(hw, queue); + + return 0; +} + +/** + * dlb_ldb_port_map_qid_dynamic() - perform a "dynamic" QID->CQ mapping + * @hw: dlb_hw handle for a particular device. + * @port: load-balanced port + * @queue: load-balanced queue + * @priority: queue servicing priority + * + * Returns 0 if the queue was mapped, 1 if the mapping is scheduled to occur + * at a later point, and <0 if an error occurred. + */ +static int dlb_ldb_port_map_qid_dynamic(struct dlb_hw *hw, + struct dlb_ldb_port *port, + struct dlb_ldb_queue *queue, + u8 priority) +{ + enum dlb_qid_map_state state; + struct dlb_hw_domain *domain; + int domain_id, slot, ret; + u32 infl_cnt; + + domain_id = port->domain_id.phys_id; + + domain = dlb_get_domain_from_id(hw, domain_id, false, 0); + if (!domain) { + DLB_HW_ERR(hw, + "[%s()] Internal error: unable to find domain %d\n", + __func__, port->domain_id.phys_id); + return -EINVAL; + } + + /* + * Set the QID inflight limit to 0 to prevent further scheduling of the + * queue. + */ + DLB_CSR_WR(hw, LSP_QID_LDB_INFL_LIM(queue->id.phys_id), 0); + + if (!dlb_port_find_slot(port, DLB_QUEUE_UNMAPPED, &slot)) { + DLB_HW_ERR(hw, + "Internal error: No available unmapped slots\n"); + return -EFAULT; + } + + port->qid_map[slot].qid = queue->id.phys_id; + port->qid_map[slot].priority = priority; + + state = DLB_QUEUE_MAP_IN_PROG; + ret = dlb_port_slot_state_transition(hw, port, queue, slot, state); + if (ret) + return ret; + + infl_cnt = DLB_CSR_RD(hw, LSP_QID_LDB_INFL_CNT(queue->id.phys_id)); + + if (BITS_GET(infl_cnt, LSP_QID_LDB_INFL_CNT_COUNT)) + return 1; + + /* + * Disable the affected CQ, and the CQs already mapped to the QID, + * before reading the QID's inflight count a second time. There is an + * unlikely race in which the QID may schedule one more QE after we + * read an inflight count of 0, and disabling the CQs guarantees that + * the race will not occur after a re-read of the inflight count + * register. + */ + if (port->enabled) + dlb_ldb_port_cq_disable(hw, port); + + dlb_ldb_queue_disable_mapped_cqs(hw, domain, queue); + + infl_cnt = DLB_CSR_RD(hw, LSP_QID_LDB_INFL_CNT(queue->id.phys_id)); + + if (BITS_GET(infl_cnt, LSP_QID_LDB_INFL_CNT_COUNT)) { + if (port->enabled) + dlb_ldb_port_cq_enable(hw, port); + + dlb_ldb_queue_enable_mapped_cqs(hw, domain, queue); + + return 1; + } + + return dlb_ldb_port_finish_map_qid_dynamic(hw, domain, port, queue); +} + static int dlb_ldb_port_map_qid(struct dlb_hw *hw, struct dlb_hw_domain *domain, struct dlb_ldb_port *port, struct dlb_ldb_queue *queue, u8 prio) { - return dlb_ldb_port_map_qid_static(hw, port, queue, prio); + if (domain->started) + return dlb_ldb_port_map_qid_dynamic(hw, port, queue, prio); + else + return dlb_ldb_port_map_qid_static(hw, port, queue, prio); } static void @@ -2722,6 +3033,82 @@ int dlb_hw_create_dir_port(struct dlb_hw *hw, u32 domain_id, return 0; } +static void dlb_domain_finish_map_port(struct dlb_hw *hw, + struct dlb_hw_domain *domain, + struct dlb_ldb_port *port) +{ + int i; + + for (i = 0; i < DLB_MAX_NUM_QIDS_PER_LDB_CQ; i++) { + struct dlb_ldb_queue *queue; + u32 infl_cnt; + int qid; + + if (port->qid_map[i].state != DLB_QUEUE_MAP_IN_PROG) + continue; + + qid = port->qid_map[i].qid; + + queue = dlb_get_ldb_queue_from_id(hw, qid, false, 0); + + if (!queue) { + DLB_HW_ERR(hw, + "[%s()] Internal error: unable to find queue %d\n", + __func__, qid); + continue; + } + + infl_cnt = DLB_CSR_RD(hw, LSP_QID_LDB_INFL_CNT(qid)); + + if (BITS_GET(infl_cnt, LSP_QID_LDB_INFL_CNT_COUNT)) + continue; + + /* + * Disable the affected CQ, and the CQs already mapped to the + * QID, before reading the QID's inflight count a second time. + * There is an unlikely race in which the QID may schedule one + * more QE after we read an inflight count of 0, and disabling + * the CQs guarantees that the race will not occur after a + * re-read of the inflight count register. + */ + if (port->enabled) + dlb_ldb_port_cq_disable(hw, port); + + dlb_ldb_queue_disable_mapped_cqs(hw, domain, queue); + + infl_cnt = DLB_CSR_RD(hw, LSP_QID_LDB_INFL_CNT(qid)); + + if (BITS_GET(infl_cnt, LSP_QID_LDB_INFL_CNT_COUNT)) { + if (port->enabled) + dlb_ldb_port_cq_enable(hw, port); + + dlb_ldb_queue_enable_mapped_cqs(hw, domain, queue); + + continue; + } + + dlb_ldb_port_finish_map_qid_dynamic(hw, domain, port, queue); + } +} + +static unsigned int +dlb_domain_finish_map_qid_procedures(struct dlb_hw *hw, + struct dlb_hw_domain *domain) +{ + struct dlb_ldb_port *port; + int i; + + if (!domain->configured || domain->num_pending_additions == 0) + return 0; + + for (i = 0; i < DLB_NUM_COS_DOMAINS; i++) { + list_for_each_entry(port, &domain->used_ldb_ports[i], domain_list) + dlb_domain_finish_map_port(hw, domain, port); + } + + return domain->num_pending_additions; +} + static void dlb_log_map_qid(struct dlb_hw *hw, u32 domain_id, struct dlb_map_qid_args *args, bool vdev_req, unsigned int vdev_id) @@ -4454,6 +4841,10 @@ int dlb_reset_domain(struct dlb_hw *hw, u32 domain_id, bool vdev_req, if (ret) return ret; + ret = dlb_domain_finish_map_qid_procedures(hw, domain); + if (ret) + return ret; + /* Re-enable the CQs in order to drain the mapped queues. */ dlb_domain_enable_ldb_cqs(hw, domain);