From patchwork Thu Sep 10 16:11:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EAB6C43461 for ; Thu, 10 Sep 2020 19:04:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08B1820758 for ; Thu, 10 Sep 2020 19:04:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rMuYKT/n" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726805AbgIJTEv (ORCPT ); Thu, 10 Sep 2020 15:04:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725983AbgIJQLv (ORCPT ); Thu, 10 Sep 2020 12:11:51 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1959FC061796; Thu, 10 Sep 2020 09:11:51 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id q13so9520280ejo.9; Thu, 10 Sep 2020 09:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HqUXASV8/J3adrJxXgdebB7pk8ZpozkrNnLZPnpPNz4=; b=rMuYKT/nO+bCs0/PjntzsIL3yMzZX12bky9TbQC+p43luWertUNdPLHCmcEs5/s7lp aMpTMrWua4TtNULcw07fDXqnBINgH1XrvxHJIWztJiFx9CIAoONLTv76jjyb+KOeLXL4 3qmjjJrxKDsUmixe4O0e3mLWt4BvDxNO1+n8YOYjPOKMM0yfFlv4kSV1DlHu3Z0QPt7G Sxrsc/6AClS5S8cVFamdUGYaYYcHJHUvI6NA0fp98W23kR+wAlUGCJVjXUX2jHRNMIGQ TLbSysOhH3k1zkQVt0nz4NR3HuPfLYLXBInwj2Z1XkO9opu57YWwgd5Y6odJPo4mqKpI qwhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HqUXASV8/J3adrJxXgdebB7pk8ZpozkrNnLZPnpPNz4=; b=S30Y7hc+qClbrgkk1YPhBWzYQo9Q27AYjcpcQMTquqF4s4If7L5Q2Nh0Ew19OV4bXj 6TyOXdq3jSJuYAOzfzmDt+MzHKvStGJlLUJ2yvGnSHPuh4GBMdO9gem7Y5UQgaA+YmdU 0wFxGbBNsDODTI8/pMkW2Bu8GwAv6PV3j8tfbq9Md+GGagbAF3zDtKQnhpHVf8nwxYFF 6IgGFGRXGWjf2JAhMy7D/WNQLAEoIOiwJm8kJHfsRBu3qcpvGiwhThdWej2OQkmCyhFH Pxv5XDbQ+35W96+AK5Ijmukb+/7xrOEITMh/O3PWFi78/hcAfSmY9GHrCJcjok6TnaNS qRBg== X-Gm-Message-State: AOAM532ldCcj00tjlj9jZT+ZY+cOb1dK7cgdDjKLo9aDG+fhEX27oO54 0fTQWKV0VIlXb/bD1VNJ1tSP8XSE/aw= X-Google-Smtp-Source: ABdhPJwGOMYluoHBIFuyJ+aF+8vL+XdnHF/jHh1znvtHTiq4P26wtwfyKLhzusCwoGLiDFzFFEzN4Q== X-Received: by 2002:a17:906:8c8:: with SMTP id o8mr9460993eje.91.1599754308990; Thu, 10 Sep 2020 09:11:48 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.11.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:11:47 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 02/15] habanalabs/gaudi: add NIC firmware-related definitions Date: Thu, 10 Sep 2020 19:11:13 +0300 Message-Id: <20200910161126.30948-3-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Add new structures and messages that the driver use to interact with the firmware to receive information and events (errors) about GAUDI's NIC. Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- .../misc/habanalabs/include/common/cpucp_if.h | 34 ++++++++++++++++--- .../habanalabs/include/gaudi/gaudi_fw_if.h | 24 +++++++++++++ 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/drivers/misc/habanalabs/include/common/cpucp_if.h b/drivers/misc/habanalabs/include/common/cpucp_if.h index dcde440427b4..ace746bb206e 100644 --- a/drivers/misc/habanalabs/include/common/cpucp_if.h +++ b/drivers/misc/habanalabs/include/common/cpucp_if.h @@ -9,6 +9,7 @@ #define CPUCP_IF_H #include +#include /* * EVENT QUEUE @@ -199,6 +200,11 @@ enum pq_init_status { * CpuCP to write to the structure, to prevent data corruption in case of * mismatched driver/FW versions. * + * CPUCP_PACKET_NIC_INFO_GET - + * Fetch information from the device regarding the NIC. the host's driver + * passes the max size it allows the CpuCP to write to the structure, to + * prevent data corruption in case of mismatched driver/FW versions. + * * CPUCP_PACKET_TEMPERATURE_SET - * Set the value of the offset property of a specified thermal sensor. * The packet's arguments specify the desired sensor and the field to @@ -238,12 +244,12 @@ enum cpucp_packet_id { CPUCP_PACKET_MAX_POWER_GET, /* sysfs */ CPUCP_PACKET_MAX_POWER_SET, /* sysfs */ CPUCP_PACKET_EEPROM_DATA_GET, /* sysfs */ - CPUCP_RESERVED, + CPUCP_PACKET_NIC_INFO_GET, /* internal */ CPUCP_PACKET_TEMPERATURE_SET, /* sysfs */ CPUCP_PACKET_VOLTAGE_SET, /* sysfs */ CPUCP_PACKET_CURRENT_SET, /* sysfs */ - CPUCP_PACKET_PCIE_THROUGHPUT_GET, /* internal */ - CPUCP_PACKET_PCIE_REPLAY_CNT_GET, /* internal */ + CPUCP_PACKET_PCIE_THROUGHPUT_GET, /* internal */ + CPUCP_PACKET_PCIE_REPLAY_CNT_GET, /* internal */ CPUCP_PACKET_TOTAL_ENERGY_GET, /* internal */ }; @@ -288,7 +294,7 @@ struct cpucp_packet { /* For led set */ __le32 led_index; - /* For get CpuCP info/EEPROM data */ + /* For get CpuCP info/EEPROM data/NIC info */ __le32 data_max_size; }; @@ -367,6 +373,12 @@ struct eq_generic_event { #define CARD_NAME_MAX_LEN 16 #define VERSION_MAX_LEN 128 #define CPUCP_MAX_SENSORS 128 +#define CPUCP_MAX_NICS 128 +#define CPUCP_LANES_PER_NIC 4 +#define CPUCP_NIC_QSFP_EEPROM_MAX_LEN 1024 +#define CPUCP_MAX_NIC_LANES (CPUCP_MAX_NICS * CPUCP_LANES_PER_NIC) +#define CPUCP_NIC_MASK_ARR_LEN ((CPUCP_MAX_NICS + 63) / 64) +#define CPUCP_NIC_POLARITY_ARR_LEN ((CPUCP_MAX_NIC_LANES + 63) / 64) struct cpucp_sensor { __le32 type; @@ -415,4 +427,18 @@ struct cpucp_info { char card_name[CARD_NAME_MAX_LEN]; }; +struct cpucp_mac_addr { + __u8 mac_addr[ETH_ALEN]; +}; + +struct cpucp_nic_info { + struct cpucp_mac_addr mac_addrs[CPUCP_MAX_NICS]; + __le64 link_mask[CPUCP_NIC_MASK_ARR_LEN]; + __le64 pol_tx_mask[CPUCP_NIC_POLARITY_ARR_LEN]; + __le64 pol_rx_mask[CPUCP_NIC_POLARITY_ARR_LEN]; + __le64 link_ext_mask[CPUCP_NIC_MASK_ARR_LEN]; + __u8 qsfp_eeprom[CPUCP_NIC_QSFP_EEPROM_MAX_LEN]; + __le64 auto_neg_mask[CPUCP_NIC_MASK_ARR_LEN]; +}; + #endif /* CPUCP_IF_H */ diff --git a/drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h b/drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h index 8aadc6357da1..d61a4c87b765 100644 --- a/drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h +++ b/drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h @@ -8,6 +8,8 @@ #ifndef GAUDI_FW_IF_H #define GAUDI_FW_IF_H +#include + #define GAUDI_EVENT_QUEUE_MSI_IDX 8 #define GAUDI_NIC_PORT1_MSI_IDX 10 #define GAUDI_NIC_PORT3_MSI_IDX 12 @@ -31,6 +33,28 @@ enum gaudi_pll_index { IF_PLL }; +enum gaudi_nic_axi_error { + RXB, + RXE, + TXS, + TXE, + QPC_RESP, + NON_AXI_ERR, +}; + +/* + * struct eq_nic_sei_event - describes an AXI error cause. + * @axi_error_cause: one of the events defined in enum gaudi_nic_axi_error. + * @id: can be either 0 or 1, to further describe unit with interrupt cause + * (i.e. TXE0 or TXE1). + * @pad[6]: padding structure to 64bit. + */ +struct eq_nic_sei_event { + __u8 axi_error_cause; + __u8 id; + __u8 pad[6]; +}; + #define GAUDI_PLL_FREQ_LOW 200000000 /* 200 MHz */ #endif /* GAUDI_FW_IF_H */ From patchwork Thu Sep 10 16:11:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5A9FC43461 for ; Thu, 10 Sep 2020 18:53:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 512DA214F1 for ; Thu, 10 Sep 2020 18:53:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pHHpf50T" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726789AbgIJSuh (ORCPT ); Thu, 10 Sep 2020 14:50:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726535AbgIJQL5 (ORCPT ); Thu, 10 Sep 2020 12:11:57 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7164CC06179A; Thu, 10 Sep 2020 09:11:56 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id lo4so9516703ejb.8; Thu, 10 Sep 2020 09:11:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JrSu93bX5OMY9+zzwZ8lpGmMDg5QSi5IMVFkw3vHrJs=; b=pHHpf50TPQSfAAR0YJFLjdPTFGP1HtQ7t03aECWAz1hCouY8uMt9oFRM+ld0oFxolf p5tbkBJlDNy9SJlLSyVVXvvknWgQbdM+tG4PcK89ebLP48EiEUug6XDhnSDutnBvBRw+ a5oVS3Ftw0JLMVPLge+qMsjf/dWGcg293czRArux2Uns3aV/dbv2LosNsykWM8dBwJcz N+RhnakcxR9IvtneCUqnYpA4+nxolYwGahZo/s2OnU2POKqq7sTKcCr5wws2U7xbkFuW 2j6TAAf/2AaBnFeTQF4SepVEbDkJ3Xu0chE4uQrT3/rMOLnRTvQwALoIOWUevGAPqmKX yjcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JrSu93bX5OMY9+zzwZ8lpGmMDg5QSi5IMVFkw3vHrJs=; b=QMLm5Fs1kIWVUMEFMKmJqRxx8NznauipbakfvHL3Xnq1RdJ5GEJ26BdxDBz00Vu6T8 HCgBvW9oMpoBeQ5LWASS/wII1uJVSgkWYzvW3k8uK0sSpOGf51R3i4dwyu5yeC0kIXWm X8034VKZYIF7BFv5IhSw2/OBWiWZMIk8KAoNmqc7IKk6fEfxUb9W6mfcjQwkdGn29rLA w9QoPAgFAw/IK7iAA+7SE4eoNSNq6UiSuC2jlIXp+0q6EMA5zHixa7umnrPPLsipeX3w Rp1aySTKBWd4BBHPyz6JAAqz5iQ5FYgfIaL3qLmmlWmNXSd6nsKao494jOrbkSPus0vC QJeQ== X-Gm-Message-State: AOAM533asB2IGF27D/8dU8h/gYow7vMZbGPh+UlNU/AOI2ficrxXoHod 4L3iVFjV+8SEwQkqfmY39fRaYi93NbI= X-Google-Smtp-Source: ABdhPJynMAlyyvKEh4PtBLQHIWklNPU/+q+f/6GqvjVW8q4eKg/epd0upvOj1hVRir+d4AUnZj9klQ== X-Received: by 2002:a17:906:ae8f:: with SMTP id md15mr9177862ejb.131.1599754313851; Thu, 10 Sep 2020 09:11:53 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.11.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:11:52 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 04/15] habanalabs/gaudi: add support for NIC QMANs Date: Thu, 10 Sep 2020 19:11:15 +0300 Message-Id: <20200910161126.30948-5-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Initialize the QMANs that are responsible to submit doorbells to the NIC engines. Add support for stopping and disabling them, and reset them as part of the hard-reset procedure of GAUDI. This will allow the user to submit work to the NICs. Add support for receiving events on QMAN errors from the firmware. However, the nic_ports_mask is still initialized to 0. That means this code won't initialize the QMANs just yet. That will be in a later patch. Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/common/habanalabs.h | 3 +- drivers/misc/habanalabs/gaudi/gaudi.c | 741 ++++++++++++++++++-- drivers/misc/habanalabs/gaudi/gaudiP.h | 32 + 3 files changed, 731 insertions(+), 45 deletions(-) diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index ec765320159a..3be39d8b0563 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -1516,8 +1516,6 @@ struct hl_device_idle_busy_ts { * @pmmu_huge_range: is a different virtual addresses range used for PMMU with * huge pages. * @init_done: is the initialization of the device done. - * @mmu_enable: is MMU enabled. - * @mmu_huge_page_opt: is MMU huge pages optimization enabled. * @device_cpu_disabled: is the device CPU disabled (due to timeouts) * @dma_mask: the dma mask that was set for this device * @in_debug: is device under debug. This, together with fpriv_list, enforces @@ -1630,6 +1628,7 @@ struct hl_device { u8 supports_soft_reset; /* Parameters for bring-up */ + u64 nic_ports_mask; u8 mmu_enable; u8 mmu_huge_page_opt; u8 cpu_enable; diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/misc/habanalabs/gaudi/gaudi.c index 483989500863..2159e14be4ef 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/misc/habanalabs/gaudi/gaudi.c @@ -301,46 +301,46 @@ static enum hl_queue_type gaudi_queue_type[GAUDI_QUEUE_ID_SIZE] = { QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_TPC_7_1 */ QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_TPC_7_2 */ QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_TPC_7_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_0_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_0_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_0_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_0_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_1_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_1_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_1_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_1_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_2_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_2_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_2_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_2_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_3_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_3_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_3_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_3_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_4_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_4_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_4_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_4_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_5_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_5_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_5_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_5_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_6_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_6_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_6_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_6_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_7_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_7_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_7_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_7_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_8_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_8_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_8_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_8_3 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_9_0 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_9_1 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_9_2 */ - QUEUE_TYPE_NA, /* GAUDI_QUEUE_ID_NIC_9_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_0_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_0_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_0_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_0_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_1_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_1_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_1_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_1_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_2_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_2_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_2_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_2_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_3_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_3_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_3_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_3_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_4_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_4_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_4_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_4_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_5_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_5_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_5_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_5_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_6_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_6_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_6_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_6_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_7_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_7_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_7_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_7_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_8_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_8_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_8_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_8_3 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_9_0 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_9_1 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_9_2 */ + QUEUE_TYPE_INT, /* GAUDI_QUEUE_ID_NIC_9_3 */ }; struct ecc_info_extract_params { @@ -792,6 +792,27 @@ static int gaudi_late_init(struct hl_device *hdev) return rc; } + if ((hdev->card_type == cpucp_card_type_pci) && + (hdev->nic_ports_mask & 0x3)) { + dev_info(hdev->dev, + "PCI card detected, only 8 ports are enabled\n"); + hdev->nic_ports_mask &= ~0x3; + + /* Stop and disable unused NIC QMANs */ + WREG32(mmNIC0_QM0_GLBL_CFG1, NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + WREG32(mmNIC0_QM1_GLBL_CFG1, NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + WREG32(mmNIC0_QM0_GLBL_CFG0, 0); + WREG32(mmNIC0_QM1_GLBL_CFG0, 0); + + gaudi->hw_cap_initialized &= ~(HW_CAP_NIC0 | HW_CAP_NIC1); + } + rc = hl_fw_send_pci_access_msg(hdev, CPUCP_PACKET_ENABLE_PCI_ACCESS); if (rc) { dev_err(hdev->dev, "Failed to enable PCI access from CPU\n"); @@ -938,6 +959,9 @@ static int gaudi_alloc_internal_qmans_pq_mem(struct hl_device *hdev) case GAUDI_QUEUE_ID_TPC_0_0 ... GAUDI_QUEUE_ID_TPC_7_3: q->pq_size = TPC_QMAN_SIZE_IN_BYTES; break; + case GAUDI_QUEUE_ID_NIC_0_0 ... GAUDI_QUEUE_ID_NIC_9_3: + q->pq_size = NIC_QMAN_SIZE_IN_BYTES; + break; default: dev_err(hdev->dev, "Bad internal queue index %d", i); rc = -EINVAL; @@ -2332,6 +2356,133 @@ static void gaudi_init_tpc_qmans(struct hl_device *hdev) } } +static void gaudi_init_nic_qman(struct hl_device *hdev, u32 nic_offset, + int qman_id, u64 qman_base_addr, int nic_id) +{ + u32 mtr_base_lo, mtr_base_hi; + u32 so_base_lo, so_base_hi; + u32 q_off; + u32 nic_qm_err_cfg; + + mtr_base_lo = lower_32_bits(CFG_BASE + + mmSYNC_MNGR_E_N_SYNC_MNGR_OBJS_MON_PAY_ADDRL_0); + mtr_base_hi = upper_32_bits(CFG_BASE + + mmSYNC_MNGR_E_N_SYNC_MNGR_OBJS_MON_PAY_ADDRL_0); + so_base_lo = lower_32_bits(CFG_BASE + + mmSYNC_MNGR_E_N_SYNC_MNGR_OBJS_SOB_OBJ_0); + so_base_hi = upper_32_bits(CFG_BASE + + mmSYNC_MNGR_E_N_SYNC_MNGR_OBJS_SOB_OBJ_0); + + q_off = nic_offset + qman_id * 4; + + WREG32(mmNIC0_QM0_PQ_BASE_LO_0 + q_off, lower_32_bits(qman_base_addr)); + WREG32(mmNIC0_QM0_PQ_BASE_HI_0 + q_off, upper_32_bits(qman_base_addr)); + + WREG32(mmNIC0_QM0_PQ_SIZE_0 + q_off, ilog2(NIC_QMAN_LENGTH)); + WREG32(mmNIC0_QM0_PQ_PI_0 + q_off, 0); + WREG32(mmNIC0_QM0_PQ_CI_0 + q_off, 0); + + WREG32(mmNIC0_QM0_CP_LDMA_TSIZE_OFFSET_0 + q_off, 0x74); + WREG32(mmNIC0_QM0_CP_LDMA_SRC_BASE_LO_OFFSET_0 + q_off, 0x14); + WREG32(mmNIC0_QM0_CP_LDMA_DST_BASE_LO_OFFSET_0 + q_off, 0x1C); + + WREG32(mmNIC0_QM0_CP_MSG_BASE0_ADDR_LO_0 + q_off, mtr_base_lo); + WREG32(mmNIC0_QM0_CP_MSG_BASE0_ADDR_HI_0 + q_off, mtr_base_hi); + WREG32(mmNIC0_QM0_CP_MSG_BASE1_ADDR_LO_0 + q_off, so_base_lo); + WREG32(mmNIC0_QM0_CP_MSG_BASE1_ADDR_HI_0 + q_off, so_base_hi); + + if (qman_id == 0) { + /* Configure RAZWI IRQ */ + nic_qm_err_cfg = NIC_QMAN_GLBL_ERR_CFG_MSG_EN_MASK; + if (hdev->stop_on_err) { + nic_qm_err_cfg |= + NIC_QMAN_GLBL_ERR_CFG_STOP_ON_ERR_EN_MASK; + } + + WREG32(mmNIC0_QM0_GLBL_ERR_CFG + nic_offset, nic_qm_err_cfg); + WREG32(mmNIC0_QM0_GLBL_ERR_ADDR_LO + nic_offset, + lower_32_bits(CFG_BASE + + mmGIC_DISTRIBUTOR__5_GICD_SETSPI_NSR)); + WREG32(mmNIC0_QM0_GLBL_ERR_ADDR_HI + nic_offset, + upper_32_bits(CFG_BASE + + mmGIC_DISTRIBUTOR__5_GICD_SETSPI_NSR)); + WREG32(mmNIC0_QM0_GLBL_ERR_WDATA + nic_offset, + gaudi_irq_map_table[GAUDI_EVENT_NIC0_QM0].cpu_id + + nic_id); + + WREG32(mmNIC0_QM0_ARB_ERR_MSG_EN + nic_offset, + QM_ARB_ERR_MSG_EN_MASK); + + /* Increase ARB WDT to support streams architecture */ + WREG32(mmNIC0_QM0_ARB_SLV_CHOISE_WDT + nic_offset, + GAUDI_ARB_WDT_TIMEOUT); + + WREG32(mmNIC0_QM0_GLBL_CFG1 + nic_offset, 0); + WREG32(mmNIC0_QM0_GLBL_PROT + nic_offset, + QMAN_INTERNAL_MAKE_TRUSTED); + } +} + +/** + * gaudi_init_nic_qmans - Initialize NIC QMAN registers + * + * @hdev: pointer to hl_device structure + * + * Initialize the H/W registers of the NIC QMANs + * + */ +void gaudi_init_nic_qmans(struct hl_device *hdev) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + struct gaudi_internal_qman_info *q; + u64 qman_base_addr; + u32 nic_offset = 0; + u32 nic_delta_between_qmans = + mmNIC0_QM1_GLBL_CFG0 - mmNIC0_QM0_GLBL_CFG0; + u32 nic_delta_between_nics = + mmNIC1_QM0_GLBL_CFG0 - mmNIC0_QM0_GLBL_CFG0; + int i, nic_id, internal_q_index; + + if (!hdev->nic_ports_mask) + return; + + if (gaudi->hw_cap_initialized & HW_CAP_NIC_MASK) + return; + + dev_dbg(hdev->dev, "Initializing NIC QMANs\n"); + + for (nic_id = 0 ; nic_id < NIC_NUMBER_OF_ENGINES ; nic_id++) { + if (!(hdev->nic_ports_mask & (1 << nic_id))) { + nic_offset += nic_delta_between_qmans; + if (nic_id & 1) { + nic_offset -= (nic_delta_between_qmans * 2); + nic_offset += nic_delta_between_nics; + } + continue; + } + + for (i = 0 ; i < QMAN_STREAMS ; i++) { + internal_q_index = GAUDI_QUEUE_ID_NIC_0_0 + + nic_id * QMAN_STREAMS + i; + q = &gaudi->internal_qmans[internal_q_index]; + qman_base_addr = (u64) q->pq_dma_addr; + gaudi_init_nic_qman(hdev, nic_offset, (i & 0x3), + qman_base_addr, nic_id); + } + + /* Enable the QMAN */ + WREG32(mmNIC0_QM0_GLBL_CFG0 + nic_offset, NIC_QMAN_ENABLE); + + nic_offset += nic_delta_between_qmans; + if (nic_id & 1) { + nic_offset -= (nic_delta_between_qmans * 2); + nic_offset += nic_delta_between_nics; + } + + gaudi->hw_cap_initialized |= 1 << (HW_CAP_NIC_SHIFT + nic_id); + } +} + static void gaudi_disable_pci_dma_qmans(struct hl_device *hdev) { struct gaudi_device *gaudi = hdev->asic_specific; @@ -2384,6 +2535,30 @@ static void gaudi_disable_tpc_qmans(struct hl_device *hdev) } } +static void gaudi_disable_nic_qmans(struct hl_device *hdev) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + u32 nic_mask, nic_offset = 0; + u32 nic_delta_between_qmans = + mmNIC0_QM1_GLBL_CFG0 - mmNIC0_QM0_GLBL_CFG0; + u32 nic_delta_between_nics = + mmNIC1_QM0_GLBL_CFG0 - mmNIC0_QM0_GLBL_CFG0; + int nic_id; + + for (nic_id = 0 ; nic_id < NIC_NUMBER_OF_ENGINES ; nic_id++) { + nic_mask = 1 << (HW_CAP_NIC_SHIFT + nic_id); + + if (gaudi->hw_cap_initialized & nic_mask) + WREG32(mmNIC0_QM0_GLBL_CFG0 + nic_offset, 0); + + nic_offset += nic_delta_between_qmans; + if (nic_id & 1) { + nic_offset -= (nic_delta_between_qmans * 2); + nic_offset += nic_delta_between_nics; + } + } +} + static void gaudi_stop_pci_dma_qmans(struct hl_device *hdev) { struct gaudi_device *gaudi = hdev->asic_specific; @@ -2442,6 +2617,73 @@ static void gaudi_stop_tpc_qmans(struct hl_device *hdev) WREG32(mmTPC7_QM_GLBL_CFG1, 0x1F << TPC0_QM_GLBL_CFG1_CP_STOP_SHIFT); } +static void gaudi_stop_nic_qmans(struct hl_device *hdev) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + + /* Stop upper CPs of QMANs */ + + if (gaudi->hw_cap_initialized & HW_CAP_NIC0) + WREG32(mmNIC0_QM0_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC1) + WREG32(mmNIC0_QM1_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC2) + WREG32(mmNIC1_QM0_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC3) + WREG32(mmNIC1_QM1_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC4) + WREG32(mmNIC2_QM0_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC5) + WREG32(mmNIC2_QM1_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC6) + WREG32(mmNIC3_QM0_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC7) + WREG32(mmNIC3_QM1_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC8) + WREG32(mmNIC4_QM0_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); + + if (gaudi->hw_cap_initialized & HW_CAP_NIC9) + WREG32(mmNIC4_QM1_GLBL_CFG1, + NIC0_QM0_GLBL_CFG1_PQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CQF_STOP_MASK | + NIC0_QM0_GLBL_CFG1_CP_STOP_MASK); +} + static void gaudi_pci_dma_stall(struct hl_device *hdev) { struct gaudi_device *gaudi = hdev->asic_specific; @@ -2631,6 +2873,7 @@ static void gaudi_halt_engines(struct hl_device *hdev, bool hard_reset) else wait_timeout_ms = GAUDI_RESET_WAIT_MSEC; + gaudi_stop_nic_qmans(hdev); gaudi_stop_mme_qmans(hdev); gaudi_stop_tpc_qmans(hdev); @@ -2648,6 +2891,7 @@ static void gaudi_halt_engines(struct hl_device *hdev, bool hard_reset) msleep(wait_timeout_ms); + gaudi_disable_nic_qmans(hdev); gaudi_disable_mme_qmans(hdev); gaudi_disable_tpc_qmans(hdev); gaudi_disable_hbm_dma_qmans(hdev); @@ -2963,11 +3207,13 @@ static int gaudi_hw_init(struct hl_device *hdev) gaudi_init_tpc_qmans(hdev); + gaudi_init_nic_qmans(hdev); + hdev->asic_funcs->set_clock_gating(hdev); gaudi_enable_timestamp(hdev); - /* MSI must be enabled before CPU queues are initialized */ + /* MSI must be enabled before CPU queues and NIC are initialized */ rc = gaudi_enable_msi(hdev); if (rc) goto disable_queues; @@ -3066,7 +3312,7 @@ static void gaudi_hw_fini(struct hl_device *hdev, bool hard_reset) HW_CAP_HBM | HW_CAP_PCI_DMA | HW_CAP_MME | HW_CAP_TPC_MASK | HW_CAP_HBM_DMA | HW_CAP_PLL | - HW_CAP_MMU | + HW_CAP_NIC_MASK | HW_CAP_MMU | HW_CAP_SRAM_SCRAMBLER | HW_CAP_HBM_SCRAMBLER | HW_CAP_CLK_GATE); @@ -3336,6 +3582,166 @@ static void gaudi_ring_doorbell(struct hl_device *hdev, u32 hw_queue_id, u32 pi) db_reg_offset = mmTPC7_QM_PQ_PI_3; break; + case GAUDI_QUEUE_ID_NIC_0_0: + db_reg_offset = mmNIC0_QM0_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_0_1: + db_reg_offset = mmNIC0_QM0_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_0_2: + db_reg_offset = mmNIC0_QM0_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_0_3: + db_reg_offset = mmNIC0_QM0_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_1_0: + db_reg_offset = mmNIC0_QM1_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_1_1: + db_reg_offset = mmNIC0_QM1_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_1_2: + db_reg_offset = mmNIC0_QM1_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_1_3: + db_reg_offset = mmNIC0_QM1_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_2_0: + db_reg_offset = mmNIC1_QM0_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_2_1: + db_reg_offset = mmNIC1_QM0_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_2_2: + db_reg_offset = mmNIC1_QM0_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_2_3: + db_reg_offset = mmNIC1_QM0_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_3_0: + db_reg_offset = mmNIC1_QM1_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_3_1: + db_reg_offset = mmNIC1_QM1_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_3_2: + db_reg_offset = mmNIC1_QM1_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_3_3: + db_reg_offset = mmNIC1_QM1_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_4_0: + db_reg_offset = mmNIC2_QM0_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_4_1: + db_reg_offset = mmNIC2_QM0_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_4_2: + db_reg_offset = mmNIC2_QM0_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_4_3: + db_reg_offset = mmNIC2_QM0_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_5_0: + db_reg_offset = mmNIC2_QM1_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_5_1: + db_reg_offset = mmNIC2_QM1_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_5_2: + db_reg_offset = mmNIC2_QM1_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_5_3: + db_reg_offset = mmNIC2_QM1_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_6_0: + db_reg_offset = mmNIC3_QM0_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_6_1: + db_reg_offset = mmNIC3_QM0_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_6_2: + db_reg_offset = mmNIC3_QM0_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_6_3: + db_reg_offset = mmNIC3_QM0_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_7_0: + db_reg_offset = mmNIC3_QM1_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_7_1: + db_reg_offset = mmNIC3_QM1_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_7_2: + db_reg_offset = mmNIC3_QM1_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_7_3: + db_reg_offset = mmNIC3_QM1_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_8_0: + db_reg_offset = mmNIC4_QM0_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_8_1: + db_reg_offset = mmNIC4_QM0_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_8_2: + db_reg_offset = mmNIC4_QM0_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_8_3: + db_reg_offset = mmNIC4_QM0_PQ_PI_3; + break; + + case GAUDI_QUEUE_ID_NIC_9_0: + db_reg_offset = mmNIC4_QM1_PQ_PI_0; + break; + + case GAUDI_QUEUE_ID_NIC_9_1: + db_reg_offset = mmNIC4_QM1_PQ_PI_1; + break; + + case GAUDI_QUEUE_ID_NIC_9_2: + db_reg_offset = mmNIC4_QM1_PQ_PI_2; + break; + + case GAUDI_QUEUE_ID_NIC_9_3: + db_reg_offset = mmNIC4_QM1_PQ_PI_3; + break; + default: invalid_queue = true; } @@ -4230,6 +4636,17 @@ static int gaudi_parse_cb_no_ext_queue(struct hl_device *hdev, struct hl_cs_parser *parser) { struct asic_fixed_properties *asic_prop = &hdev->asic_prop; + struct gaudi_device *gaudi = hdev->asic_specific; + u32 nic_mask_q_id = 1 << (HW_CAP_NIC_SHIFT + + ((parser->hw_queue_id - GAUDI_QUEUE_ID_NIC_0_0) >> 2)); + + if ((parser->hw_queue_id >= GAUDI_QUEUE_ID_NIC_0_0) && + (parser->hw_queue_id <= GAUDI_QUEUE_ID_NIC_9_3) && + (!(gaudi->hw_cap_initialized & nic_mask_q_id))) { + dev_err(hdev->dev, "h/w queue %d is disabled\n", + parser->hw_queue_id); + return -EINVAL; + } /* For internal queue jobs just check if CB address is valid */ if (hl_mem_area_inside_range((u64) (uintptr_t) parser->user_cb, @@ -4463,6 +4880,12 @@ static void gaudi_restore_qm_registers(struct hl_device *hdev) qman_offset = i * TPC_QMAN_OFFSET; WREG32(mmTPC0_QM_ARB_CFG_0 + qman_offset, 0); } + + for (i = 0 ; i < NIC_NUMBER_OF_ENGINES ; i++) { + qman_offset = (i >> 1) * NIC_MACRO_QMAN_OFFSET + + (i & 0x1) * NIC_ENGINE_QMAN_OFFSET; + WREG32(mmNIC0_QM0_ARB_CFG_0 + qman_offset, 0); + } } static void gaudi_restore_user_registers(struct hl_device *hdev) @@ -4897,6 +5320,136 @@ static void gaudi_mmu_prepare(struct hl_device *hdev, u32 asid) gaudi_mmu_prepare_reg(hdev, mmMME2_ACC_WBC, asid); gaudi_mmu_prepare_reg(hdev, mmMME3_ACC_WBC, asid); + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC0) { + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM0_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM0_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM0_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM0_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM0_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC1) { + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM1_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM1_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM1_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM1_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC0_QM1_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC2) { + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM0_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM0_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM0_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM0_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM0_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC3) { + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM1_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM1_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM1_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM1_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC1_QM1_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC4) { + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM0_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM0_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM0_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM0_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM0_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC5) { + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM1_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM1_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM1_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM1_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC2_QM1_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC6) { + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM0_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM0_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM0_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM0_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM0_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC7) { + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM1_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM1_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM1_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM1_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC3_QM1_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC8) { + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM0_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM0_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM0_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM0_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM0_GLBL_NON_SECURE_PROPS_4, + asid); + } + + if (hdev->nic_ports_mask & GAUDI_NIC_MASK_NIC9) { + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM1_GLBL_NON_SECURE_PROPS_0, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM1_GLBL_NON_SECURE_PROPS_1, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM1_GLBL_NON_SECURE_PROPS_2, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM1_GLBL_NON_SECURE_PROPS_3, + asid); + gaudi_mmu_prepare_reg(hdev, mmNIC4_QM1_GLBL_NON_SECURE_PROPS_4, + asid); + } + gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_ARUSER, asid); gaudi_mmu_prepare_reg(hdev, mmPSOC_GLOBAL_CONF_TRACE_AWUSER, asid); @@ -5426,6 +5979,8 @@ static void gaudi_handle_ecc_event(struct hl_device *hdev, u16 event_type, params.num_memories = 33; params.derr = true; params.disable_clock_gating = true; + extract_info_from_fw = false; + break; default: return; } @@ -5477,6 +6032,56 @@ static void gaudi_handle_qman_err(struct hl_device *hdev, u16 event_type) mmDMA0_QM_ARB_ERR_CAUSE + index * DMA_QMAN_OFFSET; snprintf(desc, ARRAY_SIZE(desc), "%s%d", "DMA_QM", index); break; + case GAUDI_EVENT_NIC0_QM0: + glbl_sts_addr = mmNIC0_QM0_GLBL_STS1_0; + arb_err_addr = mmNIC0_QM0_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC0_QM0"); + break; + case GAUDI_EVENT_NIC0_QM1: + glbl_sts_addr = mmNIC0_QM1_GLBL_STS1_0; + arb_err_addr = mmNIC0_QM1_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC0_QM1"); + break; + case GAUDI_EVENT_NIC1_QM0: + glbl_sts_addr = mmNIC1_QM0_GLBL_STS1_0; + arb_err_addr = mmNIC1_QM0_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC1_QM0"); + break; + case GAUDI_EVENT_NIC1_QM1: + glbl_sts_addr = mmNIC1_QM1_GLBL_STS1_0; + arb_err_addr = mmNIC1_QM1_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC1_QM1"); + break; + case GAUDI_EVENT_NIC2_QM0: + glbl_sts_addr = mmNIC2_QM0_GLBL_STS1_0; + arb_err_addr = mmNIC2_QM0_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC2_QM0"); + break; + case GAUDI_EVENT_NIC2_QM1: + glbl_sts_addr = mmNIC2_QM1_GLBL_STS1_0; + arb_err_addr = mmNIC2_QM1_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC2_QM1"); + break; + case GAUDI_EVENT_NIC3_QM0: + glbl_sts_addr = mmNIC3_QM0_GLBL_STS1_0; + arb_err_addr = mmNIC3_QM0_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC3_QM0"); + break; + case GAUDI_EVENT_NIC3_QM1: + glbl_sts_addr = mmNIC3_QM1_GLBL_STS1_0; + arb_err_addr = mmNIC3_QM1_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC3_QM1"); + break; + case GAUDI_EVENT_NIC4_QM0: + glbl_sts_addr = mmNIC4_QM0_GLBL_STS1_0; + arb_err_addr = mmNIC4_QM0_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC4_QM0"); + break; + case GAUDI_EVENT_NIC4_QM1: + glbl_sts_addr = mmNIC4_QM1_GLBL_STS1_0; + arb_err_addr = mmNIC4_QM1_ARB_ERR_CAUSE; + snprintf(desc, ARRAY_SIZE(desc), "NIC4_QM1"); + break; default: return; } @@ -5854,6 +6459,16 @@ static void gaudi_handle_eqe(struct hl_device *hdev, case GAUDI_EVENT_MME0_QM ... GAUDI_EVENT_MME2_QM: case GAUDI_EVENT_DMA0_QM ... GAUDI_EVENT_DMA7_QM: fallthrough; + case GAUDI_EVENT_NIC0_QM0: + case GAUDI_EVENT_NIC0_QM1: + case GAUDI_EVENT_NIC1_QM0: + case GAUDI_EVENT_NIC1_QM1: + case GAUDI_EVENT_NIC2_QM0: + case GAUDI_EVENT_NIC2_QM1: + case GAUDI_EVENT_NIC3_QM0: + case GAUDI_EVENT_NIC3_QM1: + case GAUDI_EVENT_NIC4_QM0: + case GAUDI_EVENT_NIC4_QM1: case GAUDI_EVENT_DMA0_CORE ... GAUDI_EVENT_DMA7_CORE: gaudi_print_irq_info(hdev, event_type, true); gaudi_handle_qman_err(hdev, event_type); @@ -6087,10 +6702,11 @@ static bool gaudi_is_device_idle(struct hl_device *hdev, u64 *mask, struct gaudi_device *gaudi = hdev->asic_specific; const char *fmt = "%-5d%-9s%#-14x%#-12x%#x\n"; const char *mme_slave_fmt = "%-5d%-9s%-14s%-12s%#x\n"; + const char *nic_fmt = "%-5d%-9s%#-14x%#x\n"; u32 qm_glbl_sts0, qm_cgm_sts, dma_core_sts0, tpc_cfg_sts, mme_arch_sts; bool is_idle = true, is_eng_idle, is_slave; u64 offset; - int i, dma_id; + int i, dma_id, port; mutex_lock(&gaudi->clk_gate_mutex); @@ -6179,6 +6795,45 @@ static bool gaudi_is_device_idle(struct hl_device *hdev, u64 *mask, } } + if (s) + seq_puts(s, "\nNIC is_idle QM_GLBL_STS0 QM_CGM_STS\n" + "--- ------- ------------ ----------\n"); + + for (i = 0 ; i < (NIC_NUMBER_OF_ENGINES / 2) ; i++) { + offset = i * NIC_MACRO_QMAN_OFFSET; + port = 2 * i; + if (hdev->nic_ports_mask & BIT(port)) { + qm_glbl_sts0 = RREG32(mmNIC0_QM0_GLBL_STS0 + offset); + qm_cgm_sts = RREG32(mmNIC0_QM0_CGM_STS + offset); + is_eng_idle = IS_QM_IDLE(qm_glbl_sts0, qm_cgm_sts); + is_idle &= is_eng_idle; + + if (mask) + *mask |= ((u64) !is_eng_idle) << + (GAUDI_ENGINE_ID_NIC_0 + port); + if (s) + seq_printf(s, nic_fmt, port, + is_eng_idle ? "Y" : "N", + qm_glbl_sts0, qm_cgm_sts); + } + + port = 2 * i + 1; + if (hdev->nic_ports_mask & BIT(port)) { + qm_glbl_sts0 = RREG32(mmNIC0_QM1_GLBL_STS0 + offset); + qm_cgm_sts = RREG32(mmNIC0_QM1_CGM_STS + offset); + is_eng_idle = IS_QM_IDLE(qm_glbl_sts0, qm_cgm_sts); + is_idle &= is_eng_idle; + + if (mask) + *mask |= ((u64) !is_eng_idle) << + (GAUDI_ENGINE_ID_NIC_0 + port); + if (s) + seq_printf(s, nic_fmt, port, + is_eng_idle ? "Y" : "N", + qm_glbl_sts0, qm_cgm_sts); + } + } + if (s) seq_puts(s, "\n"); diff --git a/drivers/misc/habanalabs/gaudi/gaudiP.h b/drivers/misc/habanalabs/gaudi/gaudiP.h index dd222bc128f9..2ccf7e5a97c7 100644 --- a/drivers/misc/habanalabs/gaudi/gaudiP.h +++ b/drivers/misc/habanalabs/gaudi/gaudiP.h @@ -79,6 +79,7 @@ #define TPC_QMAN_OFFSET (mmTPC1_QM_BASE - mmTPC0_QM_BASE) #define MME_QMAN_OFFSET (mmMME1_QM_BASE - mmMME0_QM_BASE) #define NIC_MACRO_QMAN_OFFSET (mmNIC1_QM0_BASE - mmNIC0_QM0_BASE) +#define NIC_ENGINE_QMAN_OFFSET (mmNIC0_QM1_BASE - mmNIC0_QM0_BASE) #define TPC_CFG_OFFSET (mmTPC1_CFG_BASE - mmTPC0_CFG_BASE) @@ -132,6 +133,10 @@ #define TPC_QMAN_LENGTH 1024 #define TPC_QMAN_SIZE_IN_BYTES (TPC_QMAN_LENGTH * QMAN_PQ_ENTRY_SIZE) +#define NIC_QMAN_LENGTH 1024 +#define NIC_QMAN_SIZE_IN_BYTES (NIC_QMAN_LENGTH * QMAN_PQ_ENTRY_SIZE) + + #define SRAM_USER_BASE_OFFSET GAUDI_DRIVER_SRAM_RESERVED_SIZE_FROM_START /* Virtual address space */ @@ -153,6 +158,19 @@ #define HW_CAP_SRAM_SCRAMBLER BIT(10) #define HW_CAP_HBM_SCRAMBLER BIT(11) +#define HW_CAP_NIC0 BIT(14) +#define HW_CAP_NIC1 BIT(15) +#define HW_CAP_NIC2 BIT(16) +#define HW_CAP_NIC3 BIT(17) +#define HW_CAP_NIC4 BIT(18) +#define HW_CAP_NIC5 BIT(19) +#define HW_CAP_NIC6 BIT(20) +#define HW_CAP_NIC7 BIT(21) +#define HW_CAP_NIC8 BIT(22) +#define HW_CAP_NIC9 BIT(23) +#define HW_CAP_NIC_MASK GENMASK(23, 14) +#define HW_CAP_NIC_SHIFT 14 + #define HW_CAP_TPC0 BIT(24) #define HW_CAP_TPC1 BIT(25) #define HW_CAP_TPC2 BIT(26) @@ -200,6 +218,20 @@ enum gaudi_tpc_mask { GAUDI_TPC_MASK_ALL = 0xFF }; +enum gaudi_nic_mask { + GAUDI_NIC_MASK_NIC0 = 0x01, + GAUDI_NIC_MASK_NIC1 = 0x02, + GAUDI_NIC_MASK_NIC2 = 0x04, + GAUDI_NIC_MASK_NIC3 = 0x08, + GAUDI_NIC_MASK_NIC4 = 0x10, + GAUDI_NIC_MASK_NIC5 = 0x20, + GAUDI_NIC_MASK_NIC6 = 0x40, + GAUDI_NIC_MASK_NIC7 = 0x80, + GAUDI_NIC_MASK_NIC8 = 0x100, + GAUDI_NIC_MASK_NIC9 = 0x200, + GAUDI_NIC_MASK_ALL = 0x3FF +}; + /** * struct gaudi_internal_qman_info - Internal QMAN information. * @pq_kernel_addr: Kernel address of the PQ memory area in the host. From patchwork Thu Sep 10 16:11:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6FDEC433E2 for ; Thu, 10 Sep 2020 18:55:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1EAA42087C for ; Thu, 10 Sep 2020 18:55:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZSjvSrDX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727794AbgIJStk (ORCPT ); Thu, 10 Sep 2020 14:49:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726588AbgIJQMD (ORCPT ); Thu, 10 Sep 2020 12:12:03 -0400 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1A0DC06179B; Thu, 10 Sep 2020 09:12:01 -0700 (PDT) Received: by mail-ej1-x644.google.com with SMTP id z22so9531567ejl.7; Thu, 10 Sep 2020 09:12:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=njdLSF9HfunD8Ysv98Lm8fe5YzMWYTRaPRpyApNG6O4=; b=ZSjvSrDX+vrgtPU2k9WD1azypX4aJbdKf69jAJNpH+rBQV107rSQxLfb++ctCvghpP vPjsehSJYiQ3wz/wisnsm6WBIJUNDRmdsWwjmJ6RVgTtf8O8xr6AlR44Pjc3Glnygglr SY/Vif7ubDcVtxTMUeOkblGwESRzuXtrotoSuywvt5j+W7R8yQ9QWGxW/+UxuLyznMlf Jwj5+uKCaNxtke6zlqzDttugilVnzbjLni853grTJehinCr9zIdTz55vKr1KiwL0Y93t ISMQNOxx9SxsOMmpPxzxevHQlql5O1N944T+qtzSUOIzglEp9QbiXtdeEH/cqKxbonuv C3cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=njdLSF9HfunD8Ysv98Lm8fe5YzMWYTRaPRpyApNG6O4=; b=MQWWvwnseG+WrAsT+h6x1qcjAQ2lxary89E6pUHFyYNJxY76HtR+ZqVsPQtvgDvZuq QHGQfDgyYvmpjk0UP1O+garZqceCx7YWj6lSBq1aeGWGQc1QrgrqTXck0lWSJxgqmTg/ 10AkOYXpD2m7OqGvNQ0YpHGZwdLpOl00mREPZCGuM/bql3l98K32pH1vVGM4fRUmhUiH w24UOzZr6MBWLfImwjVznZqz2RXIYFKdjk78RYeV9oJRtNUVDzqnz3R0FVaxL7y7WvGL jlhp1Xlmu7ukGk+H9zuCP2tF1Rema0jjewYMgklpSzUURvhfqFon/3rmKebhXzPRd9Ey VJIQ== X-Gm-Message-State: AOAM533rEz2IR3Apf64CbDgxe8YrqZonfIh0CpBQSl1BZwCYAGZH79y7 1YiospBlvTFviobNlQzyg5bapoJfklk= X-Google-Smtp-Source: ABdhPJwBkS5rJMy07vajJsRDM6DmyjV3M+N/vs6MCbYHOE/U8U26haRbkfINt9wlDHxUqpZn5OzlEw== X-Received: by 2002:a17:907:213b:: with SMTP id qo27mr9284571ejb.441.1599754318785; Thu, 10 Sep 2020 09:11:58 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.11.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:11:57 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 06/15] habanalabs/gaudi: add NIC PHY code Date: Thu, 10 Sep 2020 19:11:17 +0300 Message-Id: <20200910161126.30948-7-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Configure the NIC PHY (physical layer). The PHY is configured with the correct polarity and Tx taps depending on the card type. After the initial configuration, the PHY flow contains the following: - Auto-negotiation (if enabled) - PHY F/W tuning - Physical Coding Sublayer (PCS) link check After acquiring the initial PCS link, it is checked periodically. Once we detect that there is no link, we fall to PHY F/W tuning or even Auto-negotiation to re-acquire the link. Currently we use Auto-negotiation only because it is a prerequisite for link training (physical layer quality improvement) and not for setting the transmission parameters. As a result, the Auto-negotiation is currently supported only between Gaudi cards. Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/gaudi/Makefile | 2 +- drivers/misc/habanalabs/gaudi/gaudi_nic.c | 454 +++++++- drivers/misc/habanalabs/gaudi/gaudi_nic.h | 17 + drivers/misc/habanalabs/gaudi/gaudi_phy.c | 1272 +++++++++++++++++++++ 4 files changed, 1742 insertions(+), 3 deletions(-) create mode 100644 drivers/misc/habanalabs/gaudi/gaudi_phy.c diff --git a/drivers/misc/habanalabs/gaudi/Makefile b/drivers/misc/habanalabs/gaudi/Makefile index 24e14cff563d..c5143cf6f025 100644 --- a/drivers/misc/habanalabs/gaudi/Makefile +++ b/drivers/misc/habanalabs/gaudi/Makefile @@ -2,4 +2,4 @@ HL_GAUDI_FILES := gaudi/gaudi.o gaudi/gaudi_hwmgr.o gaudi/gaudi_security.o \ gaudi/gaudi_coresight.o -HL_GAUDI_FILES += gaudi/gaudi_nic.o +HL_GAUDI_FILES += gaudi/gaudi_nic.o gaudi/gaudi_phy.o diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic.c b/drivers/misc/habanalabs/gaudi/gaudi_nic.c index df41de95ba58..ff08cfc81e69 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_nic.c +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic.c @@ -687,13 +687,26 @@ static void config_port_mac(struct gaudi_nic_device *gaudi_nic) } } +static void phy_start_stop(struct gaudi_nic_device *gaudi_nic, bool is_start) +{ + int i; + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->power_up_mask & BIT(i))) + continue; + + gaudi_nic_phy_start_stop(gaudi_nic, i, is_start); + } +} + static int hw_config(struct gaudi_nic_device *gaudi_nic) { struct hl_device *hdev = gaudi_nic->hdev; struct gaudi_device *gaudi = hdev->asic_specific; u64 mac_addr = 0, tmr_addr; u32 port = gaudi_nic->port, data_rate, speed = gaudi_nic->speed; - int i; + int i, rc; + bool do_auto_neg; for (i = 0 ; i < ETH_ALEN ; i++) { mac_addr <<= 8; @@ -729,6 +742,26 @@ static int hw_config(struct gaudi_nic_device *gaudi_nic) gaudi_nic->data_rate = data_rate; + if (gaudi->nic_phy_config_fw && !gaudi_nic->mac_loopback) { + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->power_up_mask & BIT(i))) + continue; + + do_auto_neg = gaudi_nic->auto_neg_enable && + (gaudi_nic->auto_neg_mask & BIT(i)); + + rc = gaudi_nic_phy_power_up(gaudi_nic, i, do_auto_neg); + if (rc) { + dev_err(hdev->dev, + "PHY power up failed for port %d\n", + port); + return rc; + } + } + + phy_start_stop(gaudi_nic, true); + } + /* if no need in macro configuration, do only port configuration */ if (gaudi_nic->do_macro_cfg) { config_port_mac(gaudi_nic); @@ -1191,6 +1224,364 @@ static void port_reset_state(struct gaudi_nic_device *gaudi_nic) gaudi_nic->uncorrectable_errors_cnt = 0; } +static void phy_reconfig(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = gaudi_nic->hdev->asic_specific; + u32 port = gaudi_nic->port; + int i, rc; + + if (!gaudi->nic_phy_config_fw) + return; + + dev_dbg(hdev->dev, "reconfiguring PHY, port %d\n", port); + + if (gaudi_nic->auto_neg_enable) { + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->auto_neg_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_fw_config_auto_neg(gaudi_nic, i); + if (rc) + dev_dbg(hdev->dev, + "F/W reconfig autoneg failed, port: %d, lane: %d\n", + port, i); + } + } else { + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->power_up_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_power_up(gaudi_nic, i, false); + if (rc) { + dev_err(hdev->dev, + "PHY reconfig power up failed for port %d\n", + port); + break; + } + } + } + + port_reset_state(gaudi_nic); +} + +static enum link_status update_pcs_link_failure( + struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = gaudi_nic->hdev->asic_specific; + struct kfifo *pcs_fifo = &gaudi_nic->pcs_fail_fifo; + ktime_t now, before; + u32 port = gaudi_nic->port; + int count; + + if (!gaudi_nic->auto_neg_enable) + return PCS_DOWN; + + now = ktime_get(); + + count = kfifo_in(pcs_fifo, &now, sizeof(now)); + if (count != sizeof(now)) { + dev_err(hdev->dev, + "Failed to push to PCS fifo, size: %d, count: %d, port: %d\n", + gaudi_nic->pcs_fail_cnt, count, port); + return PCS_DOWN; + } + + gaudi_nic->pcs_fail_cnt++; + + if (gaudi_nic->pcs_fail_cnt < gaudi->nic_pcs_fail_threshold) + return PCS_DOWN; + + /* + * Here we reached the threshold count of failures to reconfigure the + * link. Now need to check if all of the failure are in the needed time + * frame. It is sufficient to check the first item in the queue as it is + * the earliest failure and if it is in the needed time frame, all the + * rest if failures are in it too. + */ + count = kfifo_out_peek(pcs_fifo, &before, sizeof(before)); + if (count != sizeof(before)) + dev_err(hdev->dev, + "Failed to peek in PCS fifo, size: %d, count: %d, port: %d\n", + gaudi_nic->pcs_fail_cnt, count, port); + + if (ktime_ms_delta(now, before) <= + (gaudi->nic_pcs_fail_time_frame * MSEC_PER_SEC)) { + dev_dbg(hdev->dev, + "PHY reconfig due to PCS link failure cnt, port: %d\n", + port); + return FAIL_RECONFIG; + } + + /* + * The earliest failure is not in the needed time frame, hence + * we can remove it. + */ + count = kfifo_out(pcs_fifo, &before, sizeof(before)); + if (count != sizeof(before)) + dev_err(hdev->dev, + "Failed to pop from PCS fifo, size: %d, count: %d, port: %d\n", + gaudi_nic->pcs_fail_cnt, count, port); + + gaudi_nic->pcs_fail_cnt--; + + return PCS_DOWN; +} + +static void reset_tx(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + int i; + + /* This temporary WA is only for HLS external ports */ + if ((hdev->card_type != cpucp_card_type_pmc) || + (BIT(gaudi_nic->port) & ~hdev->nic_ports_ext_mask)) + return; + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) + if (gaudi_nic->fw_tuning_mask & BIT(i)) + gaudi_nic_phy_reset_tx(gaudi_nic, i); +} + +static enum link_status _check_pcs_link(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port, pcs_val, mac_val, + start_lane = __ffs(gaudi_nic->fw_tuning_mask); + int i, rc; + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->fw_tuning_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_check_link_status(gaudi_nic, i); + if (rc) + return PHY_DOWN; + } + + /* need to check the first lane only */ + mac_val = gaudi_nic_mac_read(gaudi_nic, start_lane, "mac", 0x40); + + if (mac_val & 1) + gaudi_nic->pcs_local_fault_cnt++; + else if (gaudi_nic->pcs_local_fault_cnt) + gaudi_nic->pcs_local_fault_cnt--; + + if (mac_val & 2) + gaudi_nic->pcs_remote_fault_cnt++; + else if (gaudi_nic->pcs_remote_fault_cnt) + gaudi_nic->pcs_remote_fault_cnt--; + + if (gaudi_nic->pcs_remote_fault_cnt == PCS_FAULT_THRESHOLD) { + dev_dbg(hdev->dev, + "PHY reconfig due to PCS remote fault cnt, port: %d\n", + port); + return FAULT_RECONFIG; + } + + /* need to check the first lane only */ + pcs_val = gaudi_nic_mac_read(gaudi_nic, start_lane, "xpcs", 0x20); + + if ((pcs_val >> 12) & 1) + return LINK_UP; + + return PCS_DOWN; +} + +static void check_pcs_link(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = gaudi_nic->hdev->asic_specific; + u32 port = gaudi_nic->port; + enum link_status link_status; + + if (!gaudi->nic_check_link) + return; + + link_status = _check_pcs_link(gaudi_nic); + if ((link_status == PCS_DOWN) || (link_status == PHY_DOWN)) { + /* Try again to overcome a momentary glitch */ + msleep(PCS_LINK_RETRY_MSEC); + + link_status = _check_pcs_link(gaudi_nic); + + if (link_status == LINK_UP) + dev_info(hdev->dev, "PCS link restored, port %d\n", + port); + } + + if (link_status == LINK_UP) + return; + + set_port_status(gaudi_nic, false); + gaudi_nic->pcs_link = false; + gaudi_nic->last_pcs_link_drop_ts = ktime_get(); + + dev_info(hdev->dev, "%s lost signal, port %d\n", + (link_status == PHY_DOWN) ? "PHY" : "PCS", port); + + /* TODO: fix the bug in the retimer to remove this Tx reset WA */ + /* + * No need to update about the PCS failure if we already need to + * reconfigure the PHY. + */ + if (link_status == FAULT_RECONFIG) + reset_tx(gaudi_nic); + else + link_status = update_pcs_link_failure(gaudi_nic); + + if ((link_status == FAULT_RECONFIG) || + (link_status == FAIL_RECONFIG)) + phy_reconfig(gaudi_nic); +} + +static void acquire_pcs_link(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port, pcs_val, + start_lane = __ffs(gaudi_nic->fw_tuning_mask); + + /* need to check the first lane only */ + pcs_val = gaudi_nic_mac_read(gaudi_nic, start_lane, "xpcs", 0x20); + gaudi_nic->pcs_link = (pcs_val >> 12) & 1; + gaudi_nic->retry_cnt++; + + if (gaudi_nic->pcs_link) { + dev_info(hdev->dev, "PCS link up, port %d\n", port); + set_port_status(gaudi_nic, true); + gaudi_nic->retry_cnt = 0; + } else if (gaudi_nic->retry_cnt == PCS_LINK_CNT) { + if (ktime_after(gaudi_nic->last_fw_tuning_ts, + gaudi_nic->last_pcs_link_drop_ts)) + dev_dbg(hdev->dev, + "PHY_reconfig due to PCS link down after F/W tuning, port %d\n", + port); + else + dev_dbg(hdev->dev, + "PHY reconfig due to PCS link cnt, port %d\n", + port); + phy_reconfig(gaudi_nic); + } +} + +static void do_fw_tuning(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port; + int i, rc = 0; + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->fw_tuning_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_fw_tuning(gaudi_nic, i, true); + if (rc) { + if (rc == -EAGAIN) { + if (gaudi_nic->retry_cnt++ == FW_TUNING_CNT) { + dev_dbg(hdev->dev, + "PHY reconfig due to F/W tuning cnt, port %d, lane %d\n", + port, i); + phy_reconfig(gaudi_nic); + } + } else { + dev_dbg(hdev->dev, + "PHY F/W tuning failed for port %d, lane %d, rc %d\n", + port, i, rc); + phy_reconfig(gaudi_nic); + } + break; + } + } + + if (!rc) { + gaudi_nic->phy_fw_tuned = true; + gaudi_nic->retry_cnt = 0; + gaudi_nic->last_fw_tuning_ts = ktime_get(); + } +} + +static void do_fw_tuning_auto_neg(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port; + int i, rc; + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->auto_neg_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_fw_tuning(gaudi_nic, i, false); + if (rc) { + if (rc != -EAGAIN) + dev_dbg(hdev->dev, + "PHY auto neg F/W tuning failed, port %d, lane %d, rc %d\n", + port, i, rc); + return; + } + } + + for (i = NIC_MAC_LANES_START ; i < NIC_MAC_NUM_OF_LANES ; i++) { + if (!(gaudi_nic->fw_tuning_mask & BIT(i))) + continue; + + rc = gaudi_nic_phy_config_pam4_link_training(gaudi_nic, i); + if (rc) { + if (rc == -EAGAIN) { + if (gaudi_nic->retry_cnt++ == + FW_LINK_TRAINING_CNT) { + dev_dbg(hdev->dev, + "PHY reconfig due to PAM4 cnt, port: %d, lane: %d\n", + port, i); + phy_reconfig(gaudi_nic); + } + } else { + dev_dbg(hdev->dev, + "PHY auto neg F/W speed config failed, port %d, lane %d, rc %d\n", + port, i, rc); + phy_reconfig(gaudi_nic); + } + + return; + } + } + + dev_dbg(hdev->dev, "auto neg done, port: %d\n", port); + gaudi_nic->auto_neg_resolved = true; + gaudi_nic->retry_cnt = 0; + do_fw_tuning(gaudi_nic); +} + +static void check_link_status(struct work_struct *work) +{ + struct gaudi_nic_device *gaudi_nic = container_of(work, + struct gaudi_nic_device, + link_status_work.work); + u32 timeout_ms; + + if (gaudi_nic->phy_fw_tuned) { + if (gaudi_nic->pcs_link) + check_pcs_link(gaudi_nic); + else + acquire_pcs_link(gaudi_nic); + } else { + if (gaudi_nic->auto_neg_enable && !gaudi_nic->auto_neg_resolved) + do_fw_tuning_auto_neg(gaudi_nic); + else + do_fw_tuning(gaudi_nic); + } + + if (gaudi_nic->pcs_link) + timeout_ms = 1000; + else if (gaudi_nic->phy_fw_tuned) + timeout_ms = 500; + else + timeout_ms = 1; + + schedule_delayed_work(&gaudi_nic->link_status_work, + msecs_to_jiffies(timeout_ms)); +} + static int _gaudi_nic_sw_init(struct gaudi_nic_device *gaudi_nic) { struct hl_device *hdev = gaudi_nic->hdev; @@ -1576,7 +1967,13 @@ static int port_open(struct gaudi_nic_device *gaudi_nic) napi_enable(&gaudi_nic->napi); } - set_port_status(gaudi_nic, true); + if (gaudi->nic_phy_config_fw && !gaudi_nic->mac_loopback) { + INIT_DELAYED_WORK(&gaudi_nic->link_status_work, + check_link_status); + schedule_delayed_work(&gaudi_nic->link_status_work, 0); + } else { + set_port_status(gaudi_nic, true); + } gaudi_nic->port_open = true; @@ -1628,10 +2025,17 @@ static void port_close(struct gaudi_nic_device *gaudi_nic) gaudi_nic->port_open = false; gaudi_nic->active = false; + if (gaudi->nic_phy_config_fw && !gaudi_nic->mac_loopback) + cancel_delayed_work_sync(&gaudi_nic->link_status_work); + /* Print if not in hard reset flow e.g. from ifconfig */ if (gaudi_nic->pcs_link && !hdev->hard_reset_pending) dev_info(hdev->dev, "port %d was closed\n", port); + /* stop F/W so the peer port will also lose link */ + if (gaudi->nic_phy_config_fw && !gaudi_nic->mac_loopback) + phy_start_stop(gaudi_nic, false); + port_reset_state(gaudi_nic); kfifo_free(&gaudi_nic->pcs_fail_fifo); @@ -1882,6 +2286,19 @@ static int port_register(struct hl_device *hdev, int port) ether_addr_copy(ndev->dev_addr, hdev->asic_prop.cpucp_nic_info.mac_addrs[port].mac_addr); + /* + * Reset the NIC macro PHY before the PHY configuration by each port. + * This function resets all the 4 lanes in the PHY macro, therefore only + * one of the two ports should call it. + */ + if (gaudi->nic_phy_config_fw && gaudi_nic->do_macro_cfg) { + rc = gaudi_nic_phy_reset_macro(gaudi_nic); + if (rc) + dev_err(hdev->dev, + "PHY power up 1 failed for port %d\n", + port); + } + if (register_netdev(ndev)) { dev_err(hdev->dev, "Could not register netdevice, port: %d\n", port); @@ -2051,6 +2468,24 @@ int gaudi_nic_ports_init(struct hl_device *hdev) cpu_to_le32((card_location >> 22) & 0x7); } + if (gaudi->nic_phy_load_fw) { + rc = gaudi_nic_phy_has_fw(hdev); + if (rc) { + dev_err(hdev->dev, "NIC F/W file was not found\n"); + return rc; + } + + rc = gaudi_nic_phy_fw_load_all(hdev); + if (rc) { + dev_err(hdev->dev, "NIC F/W load for all failed\n"); + return rc; + } + } + + if (gaudi->nic_phy_config_fw) + dev_dbg(hdev->dev, "NIC F/W CRC: 0x%x\n", + gaudi_nic_phy_get_crc(hdev)); + for (i = 0 ; i < NIC_NUMBER_OF_MACROS ; i++) { gaudi->nic_macros[i].idx = i; gaudi->nic_macros[i].num_of_lanes = NIC_LANES_2; @@ -2272,6 +2707,21 @@ void gaudi_nic_ports_reopen(struct hl_device *hdev) gaudi_nic = &gaudi->nic_devices[i]; port = gaudi_nic->port; + /* + * Reset the NIC macro PHY before the PHY configuration by each + * port. This function resets all the 4 lanes in the PHY macro, + * therefore only one of the two ports should call it. + * This must be called before we check if the port is enabled, + * as the PHY reset should be called anyway. + */ + if (gaudi->nic_phy_config_fw && gaudi_nic->do_macro_cfg) { + rc = gaudi_nic_phy_reset_macro(gaudi_nic); + if (rc) + dev_err(hdev->dev, + "PHY power up 1 failed for port %d\n", + port); + } + /* * It could be that the port was shutdown by 'ifconfig down', * and there is no need in reopening it. diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic.h b/drivers/misc/habanalabs/gaudi/gaudi_nic.h index 34bcf0514d30..1b2c42fb927c 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_nic.h +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic.h @@ -333,5 +333,22 @@ int gaudi_nic_port_reset(struct gaudi_nic_device *gaudi_nic); bool disabled_or_in_reset(struct gaudi_nic_device *gaudi_nic); u64 gaudi_nic_read_mac_stat_counter(struct hl_device *hdev, u32 port, int idx, bool is_rx); +int gaudi_nic_phy_reset_macro(struct gaudi_nic_device *gaudi_nic); +int gaudi_nic_phy_power_up(struct gaudi_nic_device *gaudi_nic, int lane, + bool do_auto_neg); +int gaudi_nic_phy_has_fw(struct hl_device *hdev); +int gaudi_nic_phy_fw_tuning(struct gaudi_nic_device *gaudi_nic, int lane, + bool check_status); +int gaudi_nic_phy_fw_load_all(struct hl_device *hdev); +int gaudi_nic_phy_check_link_status(struct gaudi_nic_device *gaudi_nic, + int lane); +int gaudi_nic_phy_config_pam4_link_training(struct gaudi_nic_device *gaudi_nic, + int lane); +int gaudi_nic_phy_fw_config_auto_neg(struct gaudi_nic_device *gaudi_nic, + int lane); +u16 gaudi_nic_phy_get_crc(struct hl_device *hdev); +void gaudi_nic_phy_reset_tx(struct gaudi_nic_device *gaudi_nic, int lane); +void gaudi_nic_phy_start_stop(struct gaudi_nic_device *gaudi_nic, int lane, + bool is_start); #endif /* GAUDI_NIC_DRV_H_ */ diff --git a/drivers/misc/habanalabs/gaudi/gaudi_phy.c b/drivers/misc/habanalabs/gaudi/gaudi_phy.c new file mode 100644 index 000000000000..e96188d9e47f --- /dev/null +++ b/drivers/misc/habanalabs/gaudi/gaudi_phy.c @@ -0,0 +1,1272 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright 2019 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +#include "gaudi_nic.h" +#include "../include/gaudi/asic_reg/gaudi_regs.h" + +#include +#include +#include + +#define HL_PHY_DEBUG 0 + +#define GAUDI_PHY_FW_FILE "habanalabs/gaudi/gaudi_nic_fw.bin" + +#define PHY_READ_COUNTS_PER_MS 1000 +#define PHY_FW_SIZE 0x1020 +#define PHY_FW_FINISHED (1 << 2) +#define PHY_FW_ERROR (1 << 3) + +static void phy_write_all(struct hl_device *hdev, u32 addr, u32 data) +{ + int lane, port; + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + + for (port = 0 ; port < 10 ; port += 2) + for (lane = 0 ; lane < 4 ; lane++) { + NIC_MACRO_WREG32_PORT(phy_base + 0xF60 + lane * 4, addr, + port); + /* only the lower 16 bits are in use */ + NIC_MACRO_WREG32_PORT(phy_base - 0x8000 + 0x2000 * lane, + data & 0xFFFF, port); + } +} + +static void phy_write_port(struct hl_device *hdev, int port, int lane, u32 addr, + u32 data) +{ + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + + NIC_MACRO_WREG32_PORT(phy_base + 0xF60 + lane * 4, addr, port); + /* only the lower 16 bits are in use */ + NIC_MACRO_WREG32_PORT(phy_base - 0x8000 + 0x2000 * lane, data & 0xFFFF, + port); +} + +static void phy_write(struct gaudi_nic_device *gaudi_nic, int lane, u32 addr, + u32 data) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + + NIC_MACRO_WREG32(phy_base + 0xF60 + lane * 4, addr); + /* only the lower 16 bits are in use */ + NIC_MACRO_WREG32(phy_base - 0x8000 + 0x2000 * lane, data & 0xFFFF); +} + +static u32 phy_read_port(struct hl_device *hdev, int port, int lane, u32 addr) +{ + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + + NIC_MACRO_WREG32_PORT(phy_base + 0xF60 + lane * 4, addr, port); + /* only the lower 16 bits are in use */ + return NIC_MACRO_RREG32_PORT(phy_base - 0x8000 + 0x2000 * lane, port) & + 0xFFFF; +} + +static u32 phy_read(struct gaudi_nic_device *gaudi_nic, int lane, u32 addr) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + + NIC_MACRO_WREG32(phy_base + 0xF60 + lane * 4, addr); + + /* only the lower 16 bits are in use */ + return NIC_MACRO_RREG32(phy_base - 0x8000 + 0x2000 * lane) & 0xFFFF; +} + +static void phy_write_mask(struct gaudi_nic_device *gaudi_nic, int lane, + u32 addr, u32 raw_data, u32 mask) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 phy_base = mmNIC0_PHY_BASE - CFG_BASE; + u32 data; + + NIC_MACRO_WREG32(phy_base + 0xF60 + lane * 4, addr); + + data = (NIC_MACRO_RREG32(phy_base - 0x8000 + 0x2000 * lane)) & 0xFFFF; + data = (data & ~mask) | (((raw_data << (__ffs(mask) % 32))) & 0xFFFF); + + NIC_MACRO_WREG32(phy_base - 0x8000 + 0x2000 * lane, data); +} + +static u32 twos_to_int(s32 twos_val, u32 bitWidth) +{ + return (u32) ((s32) (twos_val) - + ((s32) ((twos_val << 1) & (1 << bitWidth)))); +} + +static int fw_cmd_port(struct hl_device *hdev, int port, int lane, u32 cmd, + u32 detail, u32 expected_res, u32 *res_ptr) +{ + u32 res, val; + int checks; + + if (detail) + phy_write_port(hdev, port, lane, 0x9816, detail); + + phy_write_port(hdev, port, lane, 0x9815, cmd); + + checks = 0; + do { + usleep_range(1000, 2000); + res = phy_read_port(hdev, port, lane, 0x9815); + if (checks++ > PHY_READ_COUNTS_PER_MS) { + dev_err(hdev->dev, "timeout for PHY cmd 0x%x\n", cmd); + return -ETIMEDOUT; + } + } while (res == cmd); + + val = (res >> 8) & 0xF; + if (val != expected_res) { + dev_err(hdev->dev, "cmd 0x%x returned error 0x%x\n", cmd, val); + return -EFAULT; + } + + *res_ptr = res; + + return 0; +} + +static int fw_cmd(struct gaudi_nic_device *gaudi_nic, int lane, u32 cmd, + u32 detail, u32 expected_res, u32 *res_ptr) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port; + u32 res, val; + int checks; + + if (detail) + phy_write(gaudi_nic, lane, 0x9816, detail); + + phy_write(gaudi_nic, lane, 0x9815, cmd); + + checks = 0; + do { + usleep_range(1000, 2000); + res = phy_read(gaudi_nic, lane, 0x9815); + if (checks++ > PHY_READ_COUNTS_PER_MS) { + dev_dbg(hdev->dev, + "timeout for PHY cmd 0x%x port %d lane %d\n", + cmd, port, lane); + return -ETIMEDOUT; + } + } while (res == cmd); + + val = (res >> 8) & 0xF; + if (val != expected_res) { + dev_dbg(hdev->dev, + "cmd 0x%x returned error 0x%x port %d lane %d\n", cmd, + val, port, lane); + return -EFAULT; + } + + *res_ptr = res; + + return 0; +} + +static int fw_hash_port(struct hl_device *hdev, int port, int lane, u32 *hash) +{ + u32 res, low_word; + int rc; + + rc = fw_cmd_port(hdev, port, lane, 0xF000, 0, 0xF, &res); + if (rc) { + dev_err(hdev->dev, "F/W hash failed for port %d lane %d\n", + port, lane); + return rc; + } + + low_word = phy_read_port(hdev, port, lane, 0x9816); + + *hash = ((res & 0xFF) << 16) | low_word; + + return 0; +} + +static void set_pll(struct gaudi_nic_device *gaudi_nic, int lane, u32 data_rate, + bool pam4) +{ + u32 pll_n_val = 0, pll_cap_val = 0; + bool div4 = 1; /* for easy debug in the future */ + + phy_write_mask(gaudi_nic, lane, 0xFF, 1, 1 << 5); + + if (!pam4) + phy_write_mask(gaudi_nic, lane, 0x179, data_rate == NIC_DR_10, + 1); + + if (data_rate == NIC_DR_50) { + if (div4) + pll_n_val = 170; + else + pll_n_val = 42; + + pll_cap_val = 10; + } else if (data_rate == NIC_DR_25) { + if (div4) + pll_n_val = 165; + else + pll_n_val = 41; + + pll_cap_val = 12; + } else if (data_rate == NIC_DR_10) { + if (div4) + pll_n_val = 132; + else + pll_n_val = 33; + + pll_cap_val = 34; + } + + phy_write_mask(gaudi_nic, lane, 0xFD, pll_n_val, 0xFF80); + phy_write_mask(gaudi_nic, lane, 0xFC, pll_cap_val, 0xFC00); +} + +static void set_tx_taps(struct gaudi_nic_device *gaudi_nic, int lane, + s32 tx_pre2, s32 tx_pre1, s32 tx_main, s32 tx_post1, + s32 tx_post2) +{ + phy_write_mask(gaudi_nic, lane, 0xAD, twos_to_int(tx_pre2, 8), 0xFF00); + phy_write_mask(gaudi_nic, lane, 0xAB, twos_to_int(tx_pre1, 8), 0xFF00); + phy_write_mask(gaudi_nic, lane, 0xA9, twos_to_int(tx_main, 8), 0xFF00); + phy_write_mask(gaudi_nic, lane, 0xA7, twos_to_int(tx_post1, 8), 0xFF00); + phy_write_mask(gaudi_nic, lane, 0xA5, twos_to_int(tx_post2, 8), 0xFF00); +} + +static void config_nrz_tx(struct gaudi_nic_device *gaudi_nic, int lane, + bool half_rate) +{ + phy_write(gaudi_nic, lane, 0xAF, 0xF83E); + phy_write(gaudi_nic, lane, 0xB0, 0x4802); + phy_write_mask(gaudi_nic, lane, 0xB0, half_rate ? 1 : 0, 1); + phy_write_mask(gaudi_nic, lane, 0xB0, 0, 0x800); + phy_write_mask(gaudi_nic, lane, 0xB0, 1, 0x800); + phy_write(gaudi_nic, lane, 0xA0, 0xE300); + set_tx_taps(gaudi_nic, lane, 0, -4, 25, 0, 0); +} + +static void config_pam4_tx(struct gaudi_nic_device *gaudi_nic, int lane) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = hdev->asic_specific; + u32 lane_idx = (gaudi_nic->port >> 1) * NIC_MAC_NUM_OF_LANES + lane; + s32 *taps; + + taps = gaudi->nic_pam4_tx_taps[lane_idx].taps; + + phy_write(gaudi_nic, lane, 0xAF, 0xF83E); + phy_write(gaudi_nic, lane, 0xB0, 0); + phy_write(gaudi_nic, lane, 0xB0, 0x800); + phy_write(gaudi_nic, lane, 0xB0, 0); + phy_write(gaudi_nic, lane, 0xA0, 0xEF00); + set_tx_taps(gaudi_nic, lane, taps[0], taps[1], taps[2], taps[3], + taps[4]); +} + +static void pol(struct gaudi_nic_device *gaudi_nic, int lane, bool pam4, + u32 tx_pol, u32 rx_pol) +{ + phy_write_mask(gaudi_nic, lane, 0xA0, tx_pol, 0x20); + phy_write_mask(gaudi_nic, lane, 0x161, rx_pol, 0x4000); /* nrz */ + phy_write_mask(gaudi_nic, lane, 0x43, rx_pol, 0x80); /* pam4 */ +} + +static void msblsb(struct gaudi_nic_device *gaudi_nic, int lane, u32 tx_msblsb, + u32 rx_msblsb) +{ + phy_write_mask(gaudi_nic, lane, 0xAF, tx_msblsb, 0x400); + phy_write_mask(gaudi_nic, lane, 0x43, rx_msblsb, 0x8000); +} + +static void gc(struct gaudi_nic_device *gaudi_nic, int lane, u32 tx_gc, + u32 rx_gc) +{ + phy_write_mask(gaudi_nic, lane, 0xAF, tx_gc, 0x200); + phy_write_mask(gaudi_nic, lane, 0x42, rx_gc, 1); +} + +static void pc(struct gaudi_nic_device *gaudi_nic, int lane, u32 tx_pc, + u32 rx_pc) +{ + phy_write_mask(gaudi_nic, lane, 0xAF, tx_pc, 0x100); + phy_write_mask(gaudi_nic, lane, 0x42, rx_pc, 2); +} + +static void set_prbs_type(struct gaudi_nic_device *gaudi_nic, int lane, + bool pam4, char *pat) +{ + u32 prbs_mode_sel_addr; + u32 prbs_mode_sel_mask; + u32 pat_sel = 0; + + if (pam4) { + prbs_mode_sel_addr = 0x43; + prbs_mode_sel_mask = 0x60; + } else { + prbs_mode_sel_addr = 0x161; + prbs_mode_sel_mask = 0x3000; + } + + if (pam4) { + if (!strncmp(pat, "PRBS9", strlen(pat))) + pat_sel = 0; + else if (!strncmp(pat, "PRBS13", strlen(pat))) + pat_sel = 1; + else if (!strncmp(pat, "PRBS15", strlen(pat))) + pat_sel = 2; + else if (!strncmp(pat, "PRBS31", strlen(pat))) + pat_sel = 3; + } else { + if (!strncmp(pat, "PRBS9", strlen(pat))) + pat_sel = 0; + else if (!strncmp(pat, "PRBS15", strlen(pat))) + pat_sel = 1; + else if (!strncmp(pat, "PRBS23", strlen(pat))) + pat_sel = 2; + else if (!strncmp(pat, "PRBS31", strlen(pat))) + pat_sel = 3; + } + + phy_write_mask(gaudi_nic, lane, 0xA0, pat_sel, 0x300); + phy_write_mask(gaudi_nic, lane, prbs_mode_sel_addr, pat_sel, + prbs_mode_sel_mask); +} + +static void get_pol_tx_rx(struct gaudi_nic_device *gaudi_nic, u32 lane_idx, + u32 *pol_tx, u32 *pol_rx) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 card_location = + le32_to_cpu(hdev->asic_prop.cpucp_info.card_location); + + switch (hdev->card_type) { + case cpucp_card_type_pci: + switch (lane_idx) { + case 0 ... 3: + case 10 ... 11: + *pol_tx = 0; + *pol_rx = 0; + break; + case 5 ... 8: + case 12: + case 16: + *pol_tx = 0; + *pol_rx = 1; + break; + case 15: + case 19: + *pol_tx = 1; + *pol_rx = 0; + break; + case 4: + case 9: + case 13 ... 14: + case 17 ... 18: + *pol_tx = 1; + *pol_rx = 1; + break; + default: + dev_err(hdev->dev, "PCI NIC %d wrong lane idx %d\n", + gaudi_nic->port, lane_idx); + break; + } + break; + + case cpucp_card_type_pmc: + *pol_tx = *pol_rx = 0; + switch (card_location) { + case 0: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 1: + switch (lane_idx) { + case 0 ... 1: + case 3 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 2: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 3: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 4: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 5: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 10: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 6: + switch (lane_idx) { + case 0 ... 1: + case 3: + case 5 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + case 7: + switch (lane_idx) { + case 0 ... 1: + case 3 ... 6: + case 8 ... 9: + case 12 ... 15: + fallthrough; + case 17: + case 19: + *pol_rx = 1; + break; + case 2: + case 16: + case 18: + *pol_tx = 1; + break; + default: + break; + } + break; + } + break; + default: + dev_err(hdev->dev, "wrong card type %d\n", hdev->card_type); + break; + } +} + +static void config_connection(struct gaudi_nic_device *gaudi_nic, int lane, + bool pam4, bool do_auto_neg) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = hdev->asic_specific; + struct cpucp_nic_info *nic_info = &hdev->asic_prop.cpucp_nic_info; + char *prbs = "PRBS31"; + u32 pol_tx = 0; + u32 pol_rx = 0; + u32 msblsb_tx = 0; + u32 msblsb_rx = 0; + u32 gc_tx = 1; + u32 gc_rx = 1; + u32 pc_tx = 0; + u32 pc_rx = 0; + u32 lane_idx = (gaudi_nic->port >> 1) * NIC_MAC_NUM_OF_LANES + lane; + + if (!pam4) + gc_tx = gc_rx = 0; + + if (gaudi->nic_use_fw_polarity) { + pol_tx = + (le64_to_cpu(nic_info->pol_tx_mask[0]) >> lane_idx) & 1; + pol_rx = + (le64_to_cpu(nic_info->pol_rx_mask[0]) >> lane_idx) & 1; + } else { + get_pol_tx_rx(gaudi_nic, lane_idx, &pol_tx, &pol_rx); + } + + phy_write_mask(gaudi_nic, lane, 0xF7, 1, 0x1000); + pol(gaudi_nic, lane, pam4, pol_tx, pol_rx); + msblsb(gaudi_nic, lane, msblsb_tx, msblsb_rx); + gc(gaudi_nic, lane, gc_tx, gc_rx); + pc(gaudi_nic, lane, pc_tx, pc_rx); + + set_prbs_type(gaudi_nic, lane, pam4, prbs); +} + +static void functional_mode(struct gaudi_nic_device *gaudi_nic, int lane, + bool pam4) +{ + if (!pam4) { + phy_write_mask(gaudi_nic, lane, 0xA0, 0, 0x2000); + phy_write_mask(gaudi_nic, lane, 0x161, 0, 0x400); + } else { + phy_write_mask(gaudi_nic, lane, 0xA0, 0, 0x2000); + phy_write_mask(gaudi_nic, lane, 0x43, 0, 0x10); + } +} + +static u32 get_fw_reg(struct gaudi_nic_device *gaudi_nic, int lane, u32 fw_addr) +{ + u32 ignore; + + fw_cmd(gaudi_nic, lane, 0xE010, fw_addr, 0xE, &ignore); + + return phy_read(gaudi_nic, lane, 0x9812); +} + +static void config_pam4_fw_rx(struct gaudi_nic_device *gaudi_nic, int lane) +{ + phy_write_mask(gaudi_nic, lane, 0x980F, 0x1, 0x1000); + phy_write_mask(gaudi_nic, lane, 0x980F, 0x1, 0x0400); + phy_write_mask(gaudi_nic, lane, 0x980F, 0x1, 0x0800); + phy_write_mask(gaudi_nic, lane, 0x980F, 0x1, 0x0200); + + phy_write(gaudi_nic, lane, 0x43, 0x8CFA); + phy_write(gaudi_nic, lane, 0x44, 0x1035); + phy_write(gaudi_nic, lane, 0x45, 0x1008); +} + +static int fw_config_speed_nrz(struct gaudi_nic_device *gaudi_nic, int lane, + u32 data_rate, u32 speed, bool half_rate, + bool fmode, bool pam4) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 ignore; + int rc, i; + + /* clear go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 0, 0x8000); + + rc = fw_cmd(gaudi_nic, lane, 0x80C0, speed, 0x8, &ignore); + if (rc) { + dev_err(hdev->dev, + "F/W cmd failed for speed nrz configuration of lane %d\n", + lane); + return rc; + } + + config_nrz_tx(gaudi_nic, lane, half_rate); + phy_write_mask(gaudi_nic, lane, 0x0161, 0x1D, 0xFC00); + config_connection(gaudi_nic, lane, pam4, false); + functional_mode(gaudi_nic, lane, pam4); + + /* clock configuration */ + for (i = 0 ; i < 4 ; i++) + if (i == 0) + phy_write(gaudi_nic, i, 0x00C9, 0x390); + else + phy_write(gaudi_nic, i, 0x00C9, 0x310); + + set_pll(gaudi_nic, lane, data_rate, pam4); + phy_write_mask(gaudi_nic, lane, 0x980F, 1, 0x8000); + + return 0; +} + +int gaudi_nic_phy_fw_config_auto_neg(struct gaudi_nic_device *gaudi_nic, + int lane) +{ + struct hl_device *hdev = gaudi_nic->hdev; + struct gaudi_device *gaudi = hdev->asic_specific; + u32 ignore; + u64 basepage = 0x000080000001; + int rc; + + usleep_range(500, 1000); + + /* clear go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 0, 0x8000); + + set_pll(gaudi_nic, lane, NIC_DR_25, false); + + /* Disable AN/LT lane swapping */ + phy_write_mask(gaudi_nic, lane, 0x8440, 0, 0x8000); + config_nrz_tx(gaudi_nic, lane, 0); + + /* config_nrz_fw_rx */ + phy_write_mask(gaudi_nic, lane, 0x0161, 0x1D, 0x0); + config_connection(gaudi_nic, lane, false, true); + + phy_write_mask(gaudi_nic, lane, 0x8300, 7, 0xE000); + + /* AN mode */ + phy_write(gaudi_nic, lane, 0x8010, basepage & 0xffff); + phy_write(gaudi_nic, lane, 0x8011, (basepage >> 16) & 0xffff); + phy_write(gaudi_nic, lane, 0x8012, (basepage >> 32) & 0xffff); + + /* IEEE */ + phy_write_mask(gaudi_nic, lane, 0x8300, 1, 0x1000); + + if (gaudi->nic_phy_auto_neg_lpbk) + phy_write_mask(gaudi_nic, lane, 0x8300, 1, 0x400); + + /* set FW to start AN */ + rc = fw_cmd(gaudi_nic, lane, 0x8000, 0, 8, &ignore); + if (rc) { + dev_err(hdev->dev, + "F/W cmd 0x8000 failed for auto neg, port %d, lane %d\n", + gaudi_nic->port, lane); + return rc; + } + + /* set go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 1, 0x8000); + + return 0; +} + +static int fw_config_speed_pam4(struct gaudi_nic_device *gaudi_nic, int lane, + u32 data_rate, u32 speed, bool fmode, bool pam4) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 ignore; + int rc; + + dev_dbg(hdev->dev, + "port: %d, lane: %d, data rate: %d, pam4: %d, speed: %d\n", + gaudi_nic->port, lane, data_rate, pam4, speed); + + /* clear go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 0, 0x8000); + + phy_write_mask(gaudi_nic, lane, 0x8440, 0, 0x8000); + + rc = fw_cmd(gaudi_nic, lane, 0x80D0, speed, 0x8, &ignore); + if (rc) { + dev_err(hdev->dev, + "F/W cmd failed for speed pam4 configuration of lane %d\n", + lane); + return rc; + } + + config_pam4_tx(gaudi_nic, lane); + config_pam4_fw_rx(gaudi_nic, lane); + config_connection(gaudi_nic, lane, pam4, false); + functional_mode(gaudi_nic, lane, pam4); + + /* set go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 1, 0x8000); + + return 0; +} + +int gaudi_nic_phy_config_pam4_link_training(struct gaudi_nic_device *gaudi_nic, + int lane) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 port = gaudi_nic->port; + u32 ignore, speed = 9; + int rc; + +#if HL_PHY_DEBUG + dev_dbg(hdev->dev, "NIC %d lane: %d, speed: %d\n", port, lane, speed); +#endif + + /* clear go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 0, 0x8000); + + /* Disable lane swapping */ + phy_write_mask(gaudi_nic, lane, 0x8440, 0, 0x8000); + + /* Enable Link Training */ + speed |= 0x100; + + config_pam4_tx(gaudi_nic, lane); + phy_write_mask(gaudi_nic, lane, 0xA0, 0, 0x2000); + config_pam4_fw_rx(gaudi_nic, lane); + config_connection(gaudi_nic, lane, true, false); + + rc = fw_cmd(gaudi_nic, lane, 0x80D0, speed, 0x8, &ignore); + if (rc) { + dev_err(hdev->dev, + "F/W cmd failed for speed pam4 configuration of port %d lane %d\n", + port, lane); + return rc; + } + + phy_write_mask(gaudi_nic, lane, 0xAF, 0, 0x200); + phy_write_mask(gaudi_nic, lane, 0xAF, 0, 0x100); + phy_write_mask(gaudi_nic, lane, 0x42, 0, 0x2); + phy_write_mask(gaudi_nic, lane, 0x42, 0, 0x1); + + /* set go bit */ + phy_write_mask(gaudi_nic, lane, 0x980F, 1, 0x8000); + + return 0; +} + +static int fw_config(struct gaudi_nic_device *gaudi_nic, int lane, + u32 data_rate, bool fmode, bool pam4) +{ + struct hl_device *hdev = gaudi_nic->hdev; + + set_pll(gaudi_nic, lane, data_rate, pam4); + + if (data_rate == NIC_DR_10) + return fw_config_speed_nrz(gaudi_nic, lane, data_rate, 1, 1, + fmode, pam4); + else if (data_rate == NIC_DR_25 || data_rate == NIC_DR_26) + return fw_config_speed_nrz(gaudi_nic, lane, data_rate, 3, 0, + fmode, pam4); + else if (data_rate == NIC_DR_50) + return fw_config_speed_pam4(gaudi_nic, lane, data_rate, 9, + fmode, pam4); + + dev_err(hdev->dev, "invalid data_rate %d\n", data_rate); + + return -EFAULT; +} + +static int fw_crc_port(struct hl_device *hdev, int port, int lane, u16 *crc) +{ + u32 res; + int rc; + + rc = fw_cmd_port(hdev, port, lane, 0xF001, 0, 0xF, &res); + if (rc) { + dev_err(hdev->dev, "F/W crc failed for port %d lane %d\n", port, + lane); + return rc; + } + + *crc = phy_read_port(hdev, port, lane, 0x9816) & 0xFFFF; + + return 0; +} + +int gaudi_nic_phy_has_fw(struct hl_device *hdev) +{ + const struct firmware *fw; + int rc; + + rc = request_firmware(&fw, GAUDI_PHY_FW_FILE, hdev->dev); + if (rc) { + dev_err(hdev->dev, "Firmware file %s is not found!\n", + GAUDI_PHY_FW_FILE); + return rc; + } + + if (fw->size < PHY_FW_SIZE) { + dev_err(hdev->dev, "Illegal %s firmware size %zu\n", + GAUDI_PHY_FW_FILE, fw->size); + rc = -EFAULT; + } + + release_firmware(fw); + + return rc; +} + +static void fw_unload_all(struct hl_device *hdev, bool pam4) +{ + phy_write_all(hdev, 0x9814, 0xFFF0); + phy_write_all(hdev, 0x980D, 0xAAA); + phy_write_all(hdev, 0x980D, 0); + + msleep(100); + + phy_write_all(hdev, 0x9814, 0); + + if (pam4) + phy_write_all(hdev, 0x11, 0); + else + phy_write_all(hdev, 0x10B, 0); +} + +u16 gaudi_nic_phy_get_crc(struct hl_device *hdev) +{ + u16 crc = 0; + + fw_crc_port(hdev, 0, 0, &crc); + + return crc; +} + +int gaudi_nic_phy_fw_load_all(struct hl_device *hdev) +{ + const struct firmware *fw; + const void *fw_data; + u32 entry_point, length, ram_addr, sections, status, checks, hash = 0, + checksum = 0x800C, fw0 = 0x9F00, fw1 = 0x980D, fw2 = 0x9814; + u16 mdio_data, crc = 0; + int rc, i, j, port, data_ptr = 0, lane = 0; + bool pam4 = true; /* for debug */ + + fw_unload_all(hdev, pam4); + + rc = request_firmware(&fw, GAUDI_PHY_FW_FILE, hdev->dev); + if (rc) { + dev_err(hdev->dev, "Firmware file %s is not found!\n", + GAUDI_PHY_FW_FILE); + return rc; + } + + if (fw->size < PHY_FW_SIZE) { + dev_err(hdev->dev, "Illegal %s firmware size %zu\n", + GAUDI_PHY_FW_FILE, fw->size); + release_firmware(fw); + return -EFAULT; + } + + fw_data = (const void *) fw->data; + fw_data += 0x1000; + + /* skip hash, crc and date */ + entry_point = get_unaligned_be32(fw_data + 8); + length = get_unaligned_be32(fw_data + 12); + ram_addr = get_unaligned_be32(fw_data + 16); + + dev_dbg(hdev->dev, "entry_point: 0x%x\n", entry_point); + dev_dbg(hdev->dev, "length: 0x%x\n", length); + + fw_data += 20; + + sections = DIV_ROUND_UP(length, 24); + + dev_dbg(hdev->dev, "sections: %d\n", sections); + + phy_write_all(hdev, fw2, 0xFFF0); + phy_write_all(hdev, fw1, 0x0AAA); + phy_write_all(hdev, fw1, 0); + + msleep(500); + + checks = 0; + do { + usleep_range(10000, 20000); + status = phy_read_port(hdev, 0, 0, fw2); + dev_dbg(hdev->dev, "lane: %d, status: 0x%x\n", lane, status); + if (checks++ > PHY_READ_COUNTS_PER_MS) { + dev_err(hdev->dev, + "failed to load NIC F/W, fw2 timeout 0x%x\n", + status); + release_firmware(fw); + return -ETIMEDOUT; + } + } while (status); + + phy_write_all(hdev, fw2, 0); + + for (i = 0 ; i <= sections ; i++) { + checksum = 0x800C; + phy_write_all(hdev, fw0 + 12, ram_addr >> 16); + phy_write_all(hdev, fw0 + 13, ram_addr & 0xFFFF); + checksum += (ram_addr >> 16) + (ram_addr & 0xFFFF); + for (j = 0 ; j < 12 ; j++) { + if (data_ptr >= length) + mdio_data = 0; + else + mdio_data = + get_unaligned_be16(fw_data + data_ptr); + + phy_write_all(hdev, fw0 + j, mdio_data); + checksum += mdio_data; + data_ptr += 2; + ram_addr += 2; + } + + phy_write_all(hdev, fw0 + 14, (~checksum + 1) & 0xFFFF); + phy_write_all(hdev, fw0 + 15, 0x800C); + + checks = 0; + + do { + usleep_range(1000, 2000); + status = phy_read_port(hdev, 0, 0, fw0 + 15); + if (checks++ > PHY_READ_COUNTS_PER_MS) { + dev_err(hdev->dev, + "failed to load NIC F/W, fw0 timeout 0x%x\n", + status); + release_firmware(fw); + return -ETIMEDOUT; + } + } while (status == 0x800C); + } + + phy_write_all(hdev, fw0 + 12, entry_point >> 16); + phy_write_all(hdev, fw0 + 13, entry_point & 0xFFFF); + checksum = (entry_point >> 16) + (entry_point & 0xFFFF) + 0x4000; + phy_write_all(hdev, fw0 + 14, (~checksum + 1) & 0xFFFF); + phy_write_all(hdev, fw0 + 15, 0x4000); + + for (port = 0 ; port < 1 ; port += 2) + for (lane = 0 ; lane < 1 ; lane++) { + fw_crc_port(hdev, port, lane, &crc); + dev_dbg(hdev->dev, "port: %d lane: %d crc: 0x%x\n", + port, lane, crc); + fw_hash_port(hdev, port, lane, &hash); + dev_dbg(hdev->dev, "port: %d lane: %d hash: 0x%x\n", + port, lane, hash); + } + + return 0; +} + +static u32 fw_tuning_counter(struct gaudi_nic_device *gaudi_nic, int lane) +{ + return get_fw_reg(gaudi_nic, lane, 5); +} + +static u32 fw_reset_counter(struct gaudi_nic_device *gaudi_nic, int lane) +{ + return get_fw_reg(gaudi_nic, lane, 4); +} + +static void print_eye(struct gaudi_nic_device *gaudi_nic, int lane, bool pam4) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 dac, eye, mask, val1, val2; + s32 plus_margin, minus_margin, result, diff; + int pam4_eye[3], eye_index, i, sel; + + if (pam4) { + dac = (phy_read(gaudi_nic, lane, 0x28) & 0x1E0) >> 5; + for (eye_index = 0; eye_index < 3; eye_index++) { + result = 0xffff; + for (i = 0; i < 3; i++) { + sel = 3 * i + eye_index; + phy_write_mask(gaudi_nic, lane, 0x88, sel, + 0xF00); + phy_write_mask(gaudi_nic, lane, 0x88, sel, + 0xF000); + + msleep(100); + + val1 = phy_read(gaudi_nic, lane, 0x32); + plus_margin = (val1 & 0xFFF0) >> 4; + if (plus_margin > 0x7ff) + plus_margin = plus_margin - 0x1000; + + val1 = phy_read(gaudi_nic, lane, 0x32); + val2 = phy_read(gaudi_nic, lane, 0x33); + minus_margin = ((val1 & 0xF) << 8) + + ((val2 & 0xFF00) >> 8); + if (minus_margin > 0x7ff) + minus_margin = minus_margin - 0x1000; + + diff = plus_margin - minus_margin; + if (diff < result) + result = diff; + } + + pam4_eye[eye_index] = + (result * (100 + (50 * dac))) / 2048; + } + + dev_dbg(hdev->dev, + "NIC PAM4 dac: %d eye0: %d eye1: %d eye2: %d\n", dac, + pam4_eye[0], pam4_eye[1], pam4_eye[2]); + } else { + mask = 0xF000; + dac = (phy_read(gaudi_nic, lane, 0x17F) & mask) >> __ffs(mask); + mask = 0xFFF; + eye = (phy_read(gaudi_nic, lane, 0x12A) & mask) >> __ffs(mask); + + dev_dbg(hdev->dev, "dac: %d, eye: %d\n", dac, eye); + + if (eye > 0) + dev_dbg(hdev->dev, + "NIC port %d lane %d: F/W eye is %d\n", + gaudi_nic->port, lane, + (eye * (200 + 50 * dac)) / 2048); + else + dev_err(hdev->dev, + "NIC port %d lane %d: F/W got no eye\n", + gaudi_nic->port, lane); + } +} + +int gaudi_nic_phy_check_link_status(struct gaudi_nic_device *gaudi_nic, + int lane) +{ + u32 phy_status; +#if HL_PHY_DEBUG + bool signal_detect; +#endif + bool phy_ready, pam4 = gaudi_nic->data_rate == NIC_DR_50; + + if (pam4) { + phy_status = phy_read(gaudi_nic, lane, 0x6A); + phy_ready = ((phy_status & 0x8000) >> 15) & 1; +#if HL_PHY_DEBUG + signal_detect = ((phy_status & 0x80) >> 7) & 1; +#endif + } else { + phy_status = phy_read(gaudi_nic, lane, 0x12E); + phy_ready = ((phy_status & 0x4) >> 2) & 1; +#if HL_PHY_DEBUG + signal_detect = ((phy_status & 0x8) >> 3) & 1; +#endif + } + +#if HL_PHY_DEBUG + { + struct hl_device *hdev = gaudi_nic->hdev; + + dev_dbg_ratelimited(hdev->dev, + "port: %d, lane, %d, phy ready: %d, signal detect: %d\n", + gaudi_nic->port, lane, phy_ready, signal_detect); + } +#endif + + return phy_ready ? 0 : -EFAULT; +} + +int gaudi_nic_phy_fw_tuning(struct gaudi_nic_device *gaudi_nic, int lane, + bool check_status) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 status, port = gaudi_nic->port; + bool pam4 = gaudi_nic->data_rate == NIC_DR_50; + + fw_tuning_counter(gaudi_nic, lane); + fw_reset_counter(gaudi_nic, lane); + status = phy_read(gaudi_nic, lane, 0x9811); + + if (status & PHY_FW_FINISHED) { + if (status & PHY_FW_ERROR) { + dev_dbg(hdev->dev, "NIC %d lane %d F/W tuning failed\n", + port, lane); + return -EFAULT; + } +#if HL_PHY_DEBUG + dev_dbg(hdev->dev, + "NIC %d lane %d F/W Tuning is done\n", port, lane); +#endif + } else { + return -EAGAIN; + } + + if (!gaudi_nic->auto_neg_enable) { + phy_write_mask(gaudi_nic, lane, 0x14D, 1, 1 << 15); + print_eye(gaudi_nic, lane, pam4); + } else if (!check_status) { + return 0; + } + + return gaudi_nic_phy_check_link_status(gaudi_nic, lane); +} + +int gaudi_nic_phy_power_up(struct gaudi_nic_device *gaudi_nic, int lane, + bool do_auto_neg) +{ + struct hl_device *hdev = gaudi_nic->hdev; + u32 data_rate = gaudi_nic->data_rate; + bool pam4 = data_rate == NIC_DR_50, fmode = 0; + int rc; + + dev_dbg(hdev->dev, "PHY power up port %d lane %d auto_neg: %d\n", + gaudi_nic->port, lane, do_auto_neg); + + /* F/W configurations */ + if (gaudi_nic->auto_neg_enable) { + if (do_auto_neg) { + rc = gaudi_nic_phy_fw_config_auto_neg(gaudi_nic, lane); + if (rc) { + dev_err(hdev->dev, + "F/W configuration failed for NIC PHY\n"); + return rc; + } + } + } else { + rc = fw_config(gaudi_nic, lane, data_rate, fmode, pam4); + if (rc) { + dev_err(hdev->dev, + "F/W configuration failed for NIC PHY\n"); + return rc; + } + } + + return 0; +} + +int gaudi_nic_phy_reset_macro(struct gaudi_nic_device *gaudi_nic) +{ + struct hl_device *hdev = gaudi_nic->hdev; + s32 chip_reset_addr = 0x980D; + bool fmode = 0; + int rc, i; + + dev_dbg(hdev->dev, "PHY reset macro, port %d\n", gaudi_nic->port); + + /* soft reset */ + for (i = 0 ; i < 4 ; i++) + phy_write(gaudi_nic, i, chip_reset_addr, 0x888); + + usleep_range(500, 1000); + + /* clock configuration */ + for (i = 0 ; i < 4 ; i++) + if (i == 0) + phy_write(gaudi_nic, i, 0x00C9, 0x390); + else + phy_write(gaudi_nic, i, 0x00C9, 0x310); + + for (i = 0 ; i < 4 ; i++) { + phy_write(gaudi_nic, i, 0x8000, 0xC000); + phy_write(gaudi_nic, i, 0x8210, 0); + phy_write(gaudi_nic, i, 0x8100, 0); + } + + /* PHY controller reset - to force F/W to start from pointer 0 */ + for (i = 0 ; i < 4 ; i++) { + phy_write(gaudi_nic, i, chip_reset_addr, 0xAAA); + phy_write(gaudi_nic, i, chip_reset_addr, 0); + } + + /* force the lane pll to run in PAM4 before logical reset */ + for (i = 0 ; i < 4 ; i++) { + rc = fw_config(gaudi_nic, i, NIC_DR_50, fmode, true); + if (rc) { + dev_err(hdev->dev, + "F/W configuration failed for NIC PHY\n"); + return rc; + } + } + + /* logic reset */ + for (i = 0 ; i < 4 ; i++) { + phy_write(gaudi_nic, i, chip_reset_addr, 0x777); + phy_write(gaudi_nic, i, chip_reset_addr, 0); + } + + usleep_range(500, 1000); + + return 0; +} + +void gaudi_nic_phy_reset_tx(struct gaudi_nic_device *gaudi_nic, int lane) +{ + u32 val; + + /* disable TX */ + val = phy_read(gaudi_nic, lane, 0xA0); + /* set bit 13 to 1 */ + val |= 0x2000; + /* set bit 11 to 0 */ + val &= ~0x800; + phy_write(gaudi_nic, lane, 0xA0, val); + + msleep(500); + + /* enable TX */ + val = phy_read(gaudi_nic, lane, 0xA0); + /* set bit 13 to 0 */ + val &= ~0x2000; + phy_write(gaudi_nic, lane, 0xA0, val); +} + +void gaudi_nic_phy_start_stop(struct gaudi_nic_device *gaudi_nic, int lane, + bool is_start) +{ + if (is_start) { + /* Enable TX driver in SerDes */ + phy_write_mask(gaudi_nic, lane, 0xE3, 1, 0x2000); + /* Enable F/W Rx tuning is done during power up sequence */ + } else { + /* Disable TX driver in SerDes */ + phy_write_mask(gaudi_nic, lane, 0xE3, 0, 0x2000); + /* Silence F/W Rx tuning */ + phy_write(gaudi_nic, lane, 0x9815, 0x9000); + } +} From patchwork Thu Sep 10 16:11:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E3A4C43461 for ; Thu, 10 Sep 2020 16:14:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2CB56214F1 for ; Thu, 10 Sep 2020 16:14:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d0dh+hUT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726760AbgIJQOO (ORCPT ); Thu, 10 Sep 2020 12:14:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726660AbgIJQMn (ORCPT ); Thu, 10 Sep 2020 12:12:43 -0400 Received: from mail-ej1-x642.google.com (mail-ej1-x642.google.com [IPv6:2a00:1450:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BA04C0617A0; Thu, 10 Sep 2020 09:12:05 -0700 (PDT) Received: by mail-ej1-x642.google.com with SMTP id u21so9551914eja.2; Thu, 10 Sep 2020 09:12:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PPUHmhNkM3clI/FRO8oFz2HVwSTGOnBMXUedMDl1l20=; b=d0dh+hUTpmyuI+yHzRxM4NeQjjik8gILIxcc74eoFlFzgxKG6hIxjkY+yys6NYr2JJ uccWHvTgJsCKiSmzhsQK6hWKpWbWfMT5cEEpbQy+RuG4Fmd5pH62lJIO2+RkNxPyJqLf n2egQxfLXPx6V0c+pX6Sor+AQrYRBq5Xcr+aQqfxWxeS0fPsPBURPCtD883p84SXlmYq 63VHNp7bJZ1FIYvQ+k9aT1pGK5e01n/3pksTHTsimWuii6uRnEjfqxC+p+UWGrhCj8oL GslunbupA3n8qWI08Vix1gUdIeMsv0JqqJna8EeqIK+5FJ5k6skVwl/Gw/NltBB46I9T ssqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PPUHmhNkM3clI/FRO8oFz2HVwSTGOnBMXUedMDl1l20=; b=pKO+gizXhC3qfWWI0hckL8POEHN+1sjwyFwwpnYnBmC0vVRrrf3q/uuI+Bt2xk+NPw p0rND3pt8VVlN328YK6fDIHzkmpCR7yYR5KGzi4Yx0YbshzyQpWM4DtwHlBvOp4xpfxe yJHFUhGsBrXJDNA5DhlWnPSOdSDQWQSJqBbgnZUYOapzhF81i9zNg2VOW1z6yBZr7LvO Ms5w0MdEIV6UMcG6LD0Tf36KZI0Bke83lvXiNlVlte2D5LtOQ+OmCJQJjiPDyRFyg8MH pLDysDM2nmtVs26t7VzHPYWFL5717rjccRo5/L2D6xriOdxFqsUFyqKdo88yb1JmgVrU PwTA== X-Gm-Message-State: AOAM533MhOox7eXCu/dYS4nCdkGrGaQxR0VCpFNSP+6R8/p7Z7AulaGl NwjY9Cmvyu5mQVjU83wDq0dBtHCzor0= X-Google-Smtp-Source: ABdhPJzzxYcX+rjrX+Gpc322klSdJtPAJxDZ98qdNwB0NJ02js96AFcQVP7iKLBKipNZCTG+Zh9qWQ== X-Received: by 2002:a17:906:3ad0:: with SMTP id z16mr10026141ejd.193.1599754322600; Thu, 10 Sep 2020 09:12:02 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.11.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:11:59 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 07/15] habanalabs/gaudi: allow user to get MAC addresses in INFO IOCTL Date: Thu, 10 Sep 2020 19:11:18 +0300 Message-Id: <20200910161126.30948-8-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman The user needs this information when working in a distributed environment with master/slave configuration. All the slaves get their MAC addresses from the driver and send them to the master. Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/common/habanalabs.h | 5 +++ .../misc/habanalabs/common/habanalabs_ioctl.c | 31 +++++++++++++++++++ drivers/misc/habanalabs/gaudi/gaudi.c | 1 + drivers/misc/habanalabs/gaudi/gaudiP.h | 2 ++ drivers/misc/habanalabs/gaudi/gaudi_nic.c | 27 ++++++++++++++++ drivers/misc/habanalabs/goya/goya.c | 9 ++++++ include/uapi/misc/habanalabs.h | 20 +++++++++++- 7 files changed, 94 insertions(+), 1 deletion(-) diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index f99db3483ba4..6bfef3da6e61 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -602,6 +602,8 @@ enum div_select_defs { DIV_SEL_DIVIDED_PLL = 3, }; +struct hl_info_mac_addr; + /** * struct hl_asic_funcs - ASIC specific functions that are can be called from * common code. @@ -679,6 +681,7 @@ enum div_select_defs { * @get_hw_state: retrieve the H/W state * @pci_bars_map: Map PCI BARs. * @init_iatu: Initialize the iATU unit inside the PCI controller. + * @get_mac_addr: Get list of MAC addresses. * @rreg: Read a register. Needed for simulator support. * @wreg: Write a register. Needed for simulator support. * @halt_coresight: stop the ETF and ETR traces. @@ -782,6 +785,8 @@ struct hl_asic_funcs { enum hl_device_hw_state (*get_hw_state)(struct hl_device *hdev); int (*pci_bars_map)(struct hl_device *hdev); int (*init_iatu)(struct hl_device *hdev); + int (*get_mac_addr)(struct hl_device *hdev, + struct hl_info_mac_addr *mac_addr); u32 (*rreg)(struct hl_device *hdev, u32 reg); void (*wreg)(struct hl_device *hdev, u32 reg, u32 val); void (*halt_coresight)(struct hl_device *hdev); diff --git a/drivers/misc/habanalabs/common/habanalabs_ioctl.c b/drivers/misc/habanalabs/common/habanalabs_ioctl.c index 07317ea49129..01425b821828 100644 --- a/drivers/misc/habanalabs/common/habanalabs_ioctl.c +++ b/drivers/misc/habanalabs/common/habanalabs_ioctl.c @@ -203,6 +203,33 @@ static int debug_coresight(struct hl_device *hdev, struct hl_debug_args *args) return rc; } +static int mac_addr_info(struct hl_device *hdev, struct hl_info_args *args) +{ + struct hl_info_mac_addr *mac_addr; + u32 max_size = args->return_size; + void __user *out = (void __user *) (uintptr_t) args->return_pointer; + int rc; + + if (!max_size || !out) + return -EINVAL; + + mac_addr = kzalloc(sizeof(struct hl_info_mac_addr), GFP_KERNEL); + if (!mac_addr) + return -ENOMEM; + + rc = hdev->asic_funcs->get_mac_addr(hdev, mac_addr); + if (rc) + goto out; + + rc = copy_to_user(out, mac_addr, + min((size_t) max_size, sizeof(struct hl_info_mac_addr))) ? + -EFAULT : 0; + +out: + kfree(mac_addr); + return rc; +} + static int device_utilization(struct hl_device *hdev, struct hl_info_args *args) { struct hl_info_device_utilization device_util = {0}; @@ -423,6 +450,10 @@ static int _hl_info_ioctl(struct hl_fpriv *hpriv, void *data, rc = hw_idle(hdev, args); break; + case HL_INFO_MAC_ADDR: + rc = mac_addr_info(hdev, args); + break; + case HL_INFO_DEVICE_UTILIZATION: rc = device_utilization(hdev, args); break; diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/misc/habanalabs/gaudi/gaudi.c index d350519a9e31..8ce20e0f8c59 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/misc/habanalabs/gaudi/gaudi.c @@ -7470,6 +7470,7 @@ static const struct hl_asic_funcs gaudi_funcs = { .get_hw_state = gaudi_get_hw_state, .pci_bars_map = gaudi_pci_bars_map, .init_iatu = gaudi_init_iatu, + .get_mac_addr = gaudi_nic_get_mac_addr, .rreg = hl_rreg, .wreg = hl_wreg, .halt_coresight = gaudi_halt_coresight, diff --git a/drivers/misc/habanalabs/gaudi/gaudiP.h b/drivers/misc/habanalabs/gaudi/gaudiP.h index bf3a215e0f8e..17560510a05f 100644 --- a/drivers/misc/habanalabs/gaudi/gaudiP.h +++ b/drivers/misc/habanalabs/gaudi/gaudiP.h @@ -566,6 +566,8 @@ void gaudi_nic_ports_fini(struct hl_device *hdev); int gaudi_nic_hard_reset_prepare(struct hl_device *hdev); void gaudi_nic_stop(struct hl_device *hdev); void gaudi_nic_ports_reopen(struct hl_device *hdev); +int gaudi_nic_get_mac_addr(struct hl_device *hdev, + struct hl_info_mac_addr *mac_addr); void gaudi_nic_ctx_fini(struct hl_ctx *ctx); irqreturn_t gaudi_nic_rx_irq_handler(int irq, void *arg); irqreturn_t gaudi_nic_cq_irq_handler(int irq, void *arg); diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic.c b/drivers/misc/habanalabs/gaudi/gaudi_nic.c index ff08cfc81e69..491f426ab0bb 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_nic.c +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic.c @@ -2743,6 +2743,33 @@ void gaudi_nic_ports_reopen(struct hl_device *hdev) gaudi->hw_cap_initialized |= HW_CAP_NIC_DRV; } +int gaudi_nic_get_mac_addr(struct hl_device *hdev, + struct hl_info_mac_addr *mac_addr) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + struct net_device *ndev; + int i, number_of_ports; + + if (!(gaudi->hw_cap_initialized & HW_CAP_NIC_DRV)) + goto out; + + number_of_ports = min_t(int, NIC_NUMBER_OF_PORTS, + HL_INFO_MAC_ADDR_MAX_NUM); + + for (i = 0 ; i < number_of_ports ; i++) { + if (!(hdev->nic_ports_mask & BIT(i))) + continue; + + ndev = gaudi->nic_devices[i].ndev; + if (!ndev) + continue; + + ether_addr_copy(mac_addr->array[i].addr, ndev->dev_addr); + mac_addr->mask[i / 64] |= BIT_ULL(i % 64); + } +out: + return 0; +} void gaudi_nic_ctx_fini(struct hl_ctx *ctx) { } diff --git a/drivers/misc/habanalabs/goya/goya.c b/drivers/misc/habanalabs/goya/goya.c index 8e15d9f85af8..e49ee24cde50 100644 --- a/drivers/misc/habanalabs/goya/goya.c +++ b/drivers/misc/habanalabs/goya/goya.c @@ -5265,6 +5265,14 @@ static enum hl_device_hw_state goya_get_hw_state(struct hl_device *hdev) return RREG32(mmHW_STATE); } +static int goya_get_mac_addr(struct hl_device *hdev, + struct hl_info_mac_addr *mac_addr) +{ + dev_err_ratelimited(hdev->dev, + "No MAC addresses are assigned to Goya\n"); + return -ENXIO; +} + static int goya_ctx_init(struct hl_ctx *ctx) { return 0; @@ -5384,6 +5392,7 @@ static const struct hl_asic_funcs goya_funcs = { .get_hw_state = goya_get_hw_state, .pci_bars_map = goya_pci_bars_map, .init_iatu = goya_init_iatu, + .get_mac_addr = goya_get_mac_addr, .rreg = hl_rreg, .wreg = hl_wreg, .halt_coresight = goya_halt_coresight, diff --git a/include/uapi/misc/habanalabs.h b/include/uapi/misc/habanalabs.h index e8a5b62b95dd..cd600a52f40a 100644 --- a/include/uapi/misc/habanalabs.h +++ b/include/uapi/misc/habanalabs.h @@ -10,6 +10,7 @@ #include #include +#include /* * Defines that are asic-specific but constitutes as ABI between kernel driver @@ -248,6 +249,8 @@ enum hl_device_status { * internal engine. * HL_INFO_DEVICE_STATUS - Retrieve the device's status. This opcode doesn't * require an open context. + * HL_INFO_MAC_ADDR - Retrieve the list of MAC addresses of the device's + * network ports, if the device has network ports. * HL_INFO_DEVICE_UTILIZATION - Retrieve the total utilization of the device * over the last period specified by the user. * The period can be between 100ms to 1s, in @@ -274,6 +277,7 @@ enum hl_device_status { #define HL_INFO_DRAM_USAGE 2 #define HL_INFO_HW_IDLE 3 #define HL_INFO_DEVICE_STATUS 4 +#define HL_INFO_MAC_ADDR 5 #define HL_INFO_DEVICE_UTILIZATION 6 #define HL_INFO_HW_EVENTS_AGGREGATE 7 #define HL_INFO_CLK_RATE 8 @@ -285,9 +289,11 @@ enum hl_device_status { #define HL_INFO_SYNC_MANAGER 14 #define HL_INFO_TOTAL_ENERGY 15 -#define HL_INFO_VERSION_MAX_LEN 128 +#define HL_INFO_VERSION_MAX_LEN 128 #define HL_INFO_CARD_NAME_MAX_LEN 16 +#define HL_INFO_MAC_ADDR_MAX_NUM 128 + struct hl_info_hw_ip_info { __u64 sram_base_address; __u64 dram_base_address; @@ -334,6 +340,18 @@ struct hl_info_device_status { __u32 pad; }; +struct hl_mac_addr { + __u8 addr[ETH_ALEN]; + __u8 pad[2]; +}; + +struct hl_info_mac_addr { + /* MAC address at index N is of the corresponding PORT ID */ + struct hl_mac_addr array[HL_INFO_MAC_ADDR_MAX_NUM]; + /* Mask of valid entries at the MAC addresses array */ + __u64 mask[2]; +}; + struct hl_info_device_utilization { __u32 utilization; __u32 pad; From patchwork Thu Sep 10 16:11:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DAF9C43461 for ; Thu, 10 Sep 2020 19:02:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 471E7207DE for ; Thu, 10 Sep 2020 19:02:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="COwHUSkw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726683AbgIJStB (ORCPT ); Thu, 10 Sep 2020 14:49:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgIJQMn (ORCPT ); Thu, 10 Sep 2020 12:12:43 -0400 Received: from mail-ej1-x642.google.com (mail-ej1-x642.google.com [IPv6:2a00:1450:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD866C0617A9; Thu, 10 Sep 2020 09:12:15 -0700 (PDT) Received: by mail-ej1-x642.google.com with SMTP id z22so9532776ejl.7; Thu, 10 Sep 2020 09:12:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fTE5okIaZncPcsNAR/BLkNFx9agcEUU8fNdeLliCC+o=; b=COwHUSkwUND8POIhev5vKuglKSVNebMC48IqwQDWG9Wm70rfN5xB/3oZ7ylRboKUe6 gRj1fDZUWbaotI+syrgrRHcgLBV/lKQq+TJaLx7pQuBZNS6VGXUCKAA0JFi8cbRLcp6E +t4MxTnSixYGiALtyCBnuK72Vo8vEhWD6vZDwQYsuAlAdGsup7ThiQXD1byCn895SXYd am45DFn+Uer50fuil/y476kQTDw7MHV77o/J8OuNgrqGm6juxWnX4TUcE3KQrsmctJd+ m3GQ9PFBq5BN/R2oHk8EjM/qz2VdQcR4Trn+o98bgTnK1/2RGQtbFW7qZjP6QkgOPR+a N7yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fTE5okIaZncPcsNAR/BLkNFx9agcEUU8fNdeLliCC+o=; b=ioR00/AAF6MXmKZ7uuS1Xw5bd9a6q93aNznQjvVRYBgrO7ZkSKDGjxl/i7bCynaJZX t9fXU48SKnqPHlmLm36pmhWVx2mtQmP8B5ha39scb8P9WJaNMoFMG+Hnh83EsQHnAVA/ oOHFte1D/Qv+HF0oIljhd3eyhwqUipJr7oiVgXVufCGOXxkoccX7S1xjalDHDg2v5Dd6 jtgtWY/QMTx4w+HmI+hjPTJ7BivdLvTBJ7oJXfChBqf+m6D2XDyXsg596Bufi0qRs25z 1jiGrrWhBwKQ5kJmot8sb/o7Qla0GwpBf7SvPdCzMJjsVlUfPdMNEL1QAyZdXBEeJjZS xWmg== X-Gm-Message-State: AOAM533gVcNIZUyNScS1/hY6sXYFNVPOtXjmLgTAZBCcrEO6ax1kcP8I vL+yeNyjj4cWVaLTD5MWLjiOAoUOWQ4= X-Google-Smtp-Source: ABdhPJwaEE4uFe4Vn7R/VvwhngYIXI6nDxHbKtre9+zCYU1zmA1PikmecS8ay0foRWnc1v0FSv13hQ== X-Received: by 2002:a17:906:a00d:: with SMTP id p13mr9906884ejy.535.1599754333544; Thu, 10 Sep 2020 09:12:13 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:12:12 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 11/15] habanalabs/gaudi: add QP error handling Date: Thu, 10 Sep 2020 19:11:22 +0300 Message-Id: <20200910161126.30948-12-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Add Queue Pair (QP) error notification to the user e.g. security violation, too many retransmissions, invalid QP etc. Whenever a QP caused an error, the firmware will send an event to the driver which will push the error as an error entry to the Completion Queue (if exists). Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/gaudi/gaudi.c | 13 ++++ drivers/misc/habanalabs/gaudi/gaudiP.h | 1 + drivers/misc/habanalabs/gaudi/gaudi_nic.c | 93 +++++++++++++++++++++++ 3 files changed, 107 insertions(+) diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/misc/habanalabs/gaudi/gaudi.c index 34b99bd94ef0..8fc2288fb424 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/misc/habanalabs/gaudi/gaudi.c @@ -6658,6 +6658,19 @@ static void gaudi_handle_eqe(struct hl_device *hdev, hl_fw_unmask_irq(hdev, event_type); break; + case GAUDI_EVENT_NIC0_QP0: + case GAUDI_EVENT_NIC0_QP1: + case GAUDI_EVENT_NIC1_QP0: + case GAUDI_EVENT_NIC1_QP1: + case GAUDI_EVENT_NIC2_QP0: + case GAUDI_EVENT_NIC2_QP1: + case GAUDI_EVENT_NIC3_QP0: + case GAUDI_EVENT_NIC3_QP1: + case GAUDI_EVENT_NIC4_QP0: + case GAUDI_EVENT_NIC4_QP1: + gaudi_nic_handle_qp_err(hdev, event_type); + break; + case GAUDI_EVENT_PSOC_GPIO_U16_0: cause = le64_to_cpu(eq_entry->data[0]) & 0xFF; dev_err(hdev->dev, diff --git a/drivers/misc/habanalabs/gaudi/gaudiP.h b/drivers/misc/habanalabs/gaudi/gaudiP.h index ba3150c073ca..dc1dcff43cd6 100644 --- a/drivers/misc/habanalabs/gaudi/gaudiP.h +++ b/drivers/misc/habanalabs/gaudi/gaudiP.h @@ -578,5 +578,6 @@ netdev_tx_t gaudi_nic_handle_tx_pkt(struct gaudi_nic_device *gaudi_nic, struct sk_buff *skb); int gaudi_nic_sw_init(struct hl_device *hdev); void gaudi_nic_sw_fini(struct hl_device *hdev); +void gaudi_nic_handle_qp_err(struct hl_device *hdev, u16 event_type); #endif /* GAUDIP_H_ */ diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic.c b/drivers/misc/habanalabs/gaudi/gaudi_nic.c index 8f6585c700cf..41789f7ed32e 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_nic.c +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic.c @@ -3958,3 +3958,96 @@ int gaudi_nic_cq_mmap(struct hl_device *hdev, struct vm_area_struct *vma) return rc; } + +static char *get_syndrome_text(u32 syndrome) +{ + char *str; + + switch (syndrome) { + case 0x05: + str = "Rx got invalid QP"; + break; + case 0x06: + str = "Rx transport service mismatch"; + break; + case 0x09: + str = "Rx Rkey check failed"; + break; + case 0x40: + str = "timer retry exceeded"; + break; + case 0x41: + str = "NACK retry exceeded"; + break; + case 0x42: + str = "doorbell on invalid QP"; + break; + case 0x43: + str = "doorbell security check failed"; + break; + case 0x44: + str = "Tx got invalid QP"; + break; + case 0x45: + str = "responder got ACK/NACK on invalid QP"; + break; + case 0x46: + str = "responder try to send ACK/NACK on invalid QP"; + break; + default: + str = "unknown syndrome"; + break; + } + + return str; +} + +void gaudi_nic_handle_qp_err(struct hl_device *hdev, u16 event_type) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + struct gaudi_nic_device *gaudi_nic = + &gaudi->nic_devices[event_type - GAUDI_EVENT_NIC0_QP0]; + struct qp_err *qp_err_arr = gaudi_nic->qp_err_mem_cpu; + struct hl_nic_cqe cqe_sw; + u32 pi, ci; + + mutex_lock(&gaudi->nic_qp_err_lock); + + if (!gaudi->nic_cq_enable) + dev_err_ratelimited(hdev->dev, + "received NIC %d QP error event %d but no CQ to push it\n", + gaudi_nic->port, event_type); + + pi = NIC_RREG32(mmNIC0_QPC0_ERR_FIFO_PRODUCER_INDEX); + ci = gaudi_nic->qp_err_ci; + + cqe_sw.is_err = true; + cqe_sw.port = gaudi_nic->port; + + while (ci < pi) { + cqe_sw.type = QP_ERR_IS_REQ(qp_err_arr[ci]) ? + HL_NIC_CQE_TYPE_REQ : HL_NIC_CQE_TYPE_RES; + cqe_sw.qp_number = QP_ERR_QP_NUM(qp_err_arr[ci]); + cqe_sw.qp_err.syndrome = QP_ERR_ERR_NUM(qp_err_arr[ci]); + + ci = (ci + 1) & (QP_ERR_BUF_LEN - 1); + + dev_err_ratelimited(hdev->dev, + "NIC QP error port: %d, type: %d, qpn: %d, syndrome: %s (0x%x)\n", + cqe_sw.port, cqe_sw.type, cqe_sw.qp_number, + get_syndrome_text(cqe_sw.qp_err.syndrome), + cqe_sw.qp_err.syndrome); + + if (gaudi->nic_cq_enable) + copy_cqe_to_main_queue(hdev, &cqe_sw); + } + + gaudi_nic->qp_err_ci = ci; + NIC_WREG32(mmNIC0_QPC0_ERR_FIFO_CONSUMER_INDEX, ci); + + /* signal the completion queue that there are available CQEs */ + if (gaudi->nic_cq_enable) + complete(&gaudi->nic_cq_comp); + + mutex_unlock(&gaudi->nic_qp_err_lock); +} From patchwork Thu Sep 10 16:11:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2977EC43461 for ; Thu, 10 Sep 2020 19:00:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A384F20731 for ; Thu, 10 Sep 2020 19:00:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VSFNbSRp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727070AbgIJStj (ORCPT ); Thu, 10 Sep 2020 14:49:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726705AbgIJQMn (ORCPT ); Thu, 10 Sep 2020 12:12:43 -0400 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3997BC0617AB; Thu, 10 Sep 2020 09:12:18 -0700 (PDT) Received: by mail-ed1-x542.google.com with SMTP id c10so6889404edk.6; Thu, 10 Sep 2020 09:12:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wptBGDLLzX2MKQextGuJpdWgYcfBlbPlr/5rrFvg6GU=; b=VSFNbSRp5k7mwk/c36aGECKNN6PMnQGCqsO9kuzQxlK7BDqXGdgz19Qgcu0Pzmkw+c 8/TkkXSna2cvvMUUNFaEoHYnPkbzEU/aGHeyUtrMhoaXWjZRM2PRf6grh67oYkEFYcew BRF4ErR7pbcJSJZZZq6ycqh+/ixSpWxasMVZH9D2+fNLrC8Eql75kvqY+ZIo19TStcg9 svjJbaNmdGovDlme3y77eQTFETN5cZ7JWKUQKc6JQA5E279YLNGeZoQl8jAE1u+PUkuz AGQf/diNVIb9QxEGrLpfuAaYBZYr9DCZz6S/K10XSQjHm4J9uOFpzsLFU+EMXGYkk9P5 iLHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wptBGDLLzX2MKQextGuJpdWgYcfBlbPlr/5rrFvg6GU=; b=kMhKmaRBOIkrbYgr0sVROFh5B6Cc+DkaIR0FcvQc7MRNrPn0BoOk/9C2mW8dibmZ6s KX0gQFTI7ItzyCYRUZJeOdLgh7215U/f2CvcljvJQGggJSO9gDpmdWO6hidS6qg7bUUL q+niHQVFkQ/jTxQa5XLl54ZgGDHDKO9QFKPhuUxPYmef9FYCihyXU8AXDawEGIVGkI8r cHaOElj4iI3/5oTWI2xrAhhyrVNmqSMCOqBoo4dBvkPPDDJbjvsoL1Yurgsv12fv70ra KTa9RuElCvENA6hsm62msr6Pj/Oxtm0xog2jJ47T+aJReIXiG7j5HP9UQ75/A4pf0I+J 216g== X-Gm-Message-State: AOAM530kZdDkOaLe0Ey8P9+6OaXASKw1LETqBXhrA66Y2TqbXU1gyG7T WP73gBxuiFjMho1W22RG16edHQ27KDE= X-Google-Smtp-Source: ABdhPJyYyXhGqmNfqGiO6bS6kWE/UNIwLOvhuITAfFuP5VtrOTOZEmUG4+3QUjGHPwDANkVVm28Qtw== X-Received: by 2002:a05:6402:44e:: with SMTP id p14mr10323911edw.1.1599754335923; Thu, 10 Sep 2020 09:12:15 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.12.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:12:14 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 12/15] habanalabs/gaudi: add debugfs entries for the NIC Date: Thu, 10 Sep 2020 19:11:23 +0300 Message-Id: <20200910161126.30948-13-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Add several debugfs entries to help us debug the NIC engines and ports and also the communication layer of the DL training application that use them. There are eight new entries. Detailed description is in the documentation file but here is a summary: - nic_mac_loopback: enable mac loopback mode per port - nic_ports_status: print physical connection status per port - nic_pcs_fail_time_frame: configure windows size for measuring pcs failures - nic_pcs_fail_threshold: configure pcs failures threshold for reconfiguring the link - nic_pam4_tx_taps: configure PAM4 TX taps - nic_polarity: configure polarity for NIC port lanes - nic_check_link: configure whether to check the PCS link periodically - nic_phy_auto_neg_lpbk: enable PHY auto-negotiation loopback Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- .../ABI/testing/debugfs-driver-habanalabs | 69 +++ drivers/misc/habanalabs/gaudi/Makefile | 3 +- drivers/misc/habanalabs/gaudi/gaudi_nic.c | 2 + .../misc/habanalabs/gaudi/gaudi_nic_debugfs.c | 402 ++++++++++++++++++ 4 files changed, 475 insertions(+), 1 deletion(-) create mode 100644 drivers/misc/habanalabs/gaudi/gaudi_nic_debugfs.c diff --git a/Documentation/ABI/testing/debugfs-driver-habanalabs b/Documentation/ABI/testing/debugfs-driver-habanalabs index 2e9ae311e02d..8fca02b92a80 100644 --- a/Documentation/ABI/testing/debugfs-driver-habanalabs +++ b/Documentation/ABI/testing/debugfs-driver-habanalabs @@ -176,3 +176,72 @@ KernelVersion: 5.6 Contact: oded.gabbay@gmail.com Description: Sets the stop-on_error option for the device engines. Value of "0" is for disable, otherwise enable. + +What: /sys/kernel/debug/habanalabs/hl/nic_mac_loopback +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Allows the root user to disable/enable MAC loopback for Gaudi + NIC ports. The ports will function as if a physical loopback + transceiver was connected. A bitmask should be provided where + each bit represents a port, up to 20 bits. + Known issues for this mode: + 1. Odd ports PHY is not stopped so the peer's odd ports will get + PCS link. + 2. Might cause an interrupt storm due to high B/W. + +What: /sys/kernel/debug/habanalabs/hl/nic_ports_status +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Displays a summary the PC link state of all Gaudi NIC ports. + +What: /sys/kernel/debug/habanalabs/hl/nic_pcs_fail_time_frame +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Allows the root user to set the used time frame in seconds for + detecting a loose PCS link of a Gaudi NIC port. We count how + many PCS link failures occurred in a time frame up to a + threshold which will cause PHY reconfiguration for getting a new + PCS link. + +What: /sys/kernel/debug/habanalabs/hl/nic_pcs_fail_threshold +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Allows the root user to set the used threshold for detecting a + loose PCS link of a Gaudi NIC port. We count how many PCS link + failures occurred in a time frame up to the threshold which will + cause PHY reconfiguration for getting a new PCS link. + +What: /sys/kernel/debug/habanalabs/hl/nic_pam4_tx_taps +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Allows the root user to set the PAM4 Tx taps for Gaudi NIC port + lanes. The lanes indices are 0-39. + Acceptable input string form: + . + +What: /sys/kernel/debug/habanalabs/hl/nic_polarity +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Allows the root user to set the polarity for Gaudi NIC port + lanes. The lanes indices are 0-39. + Acceptable input string form: . + +What: /sys/kernel/debug/habanalabs/hl/nic_check_link +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Sets the PCS link periodic check for all Gaudi NIC ports. Value + of "0" is for disable, otherwise enable. + +What: /sys/kernel/debug/habanalabs/hl/nic_phy_auto_neg_lpbk +Date: Nov 2020 +KernelVersion: 5.10 +Contact: oshpigelman@habana.ai +Description: Sets the PHY Autoneg loopback support for all Gaudi NIC ports. + Value of "0" is for disable, otherwise enable. diff --git a/drivers/misc/habanalabs/gaudi/Makefile b/drivers/misc/habanalabs/gaudi/Makefile index c5143cf6f025..437b21e54c95 100644 --- a/drivers/misc/habanalabs/gaudi/Makefile +++ b/drivers/misc/habanalabs/gaudi/Makefile @@ -2,4 +2,5 @@ HL_GAUDI_FILES := gaudi/gaudi.o gaudi/gaudi_hwmgr.o gaudi/gaudi_security.o \ gaudi/gaudi_coresight.o -HL_GAUDI_FILES += gaudi/gaudi_nic.o gaudi/gaudi_phy.o +HL_GAUDI_FILES += gaudi/gaudi_nic.o gaudi/gaudi_phy.o \ + gaudi/gaudi_nic_debugfs.o diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic.c b/drivers/misc/habanalabs/gaudi/gaudi_nic.c index 41789f7ed32e..a73635a4c44b 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi_nic.c +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic.c @@ -3025,6 +3025,8 @@ int gaudi_nic_ports_init(struct hl_device *hdev) } } + gaudi_nic_debugfs_init(hdev); + gaudi->hw_cap_initialized |= HW_CAP_NIC_DRV; return 0; diff --git a/drivers/misc/habanalabs/gaudi/gaudi_nic_debugfs.c b/drivers/misc/habanalabs/gaudi/gaudi_nic_debugfs.c new file mode 100644 index 000000000000..2e99d2683512 --- /dev/null +++ b/drivers/misc/habanalabs/gaudi/gaudi_nic_debugfs.c @@ -0,0 +1,402 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Copyright 2018-2020 HabanaLabs, Ltd. + * All Rights Reserved. + */ + +#include "gaudi_nic.h" +#include +#include + +#ifdef CONFIG_DEBUG_FS + +#define POLARITY_KBUF_SIZE 8 +#define TX_TAPS_KBUF_SIZE 25 + +static ssize_t debugfs_pam4_tx_taps_write(struct file *f, + const char __user *buf, + size_t count, loff_t *ppos) +{ + struct hl_device *hdev = file_inode(f)->i_private; + struct gaudi_device *gaudi = hdev->asic_specific; + char kbuf[TX_TAPS_KBUF_SIZE]; + char *c1, *c2; + ssize_t rc; + u32 lane; + s32 tx_pre2, tx_pre1, tx_main, tx_post1, tx_post2; + s32 *taps; + + if (count > sizeof(kbuf) - 1) + goto err; + if (copy_from_user(kbuf, buf, count)) + goto err; + kbuf[count] = '\0'; + + c1 = kbuf; + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtou32(c1, 10, &lane); + if (rc) + goto err; + + if (lane >= NIC_MAX_NUM_OF_LANES) { + dev_err(hdev->dev, "lane max value is %d\n", + NIC_MAX_NUM_OF_LANES - 1); + return -EINVAL; + } + + /* Turn off speculation due to Spectre vulnerability */ + lane = array_index_nospec(lane, NIC_MAX_NUM_OF_LANES); + + c1 = c2 + 1; + + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtos32(c1, 10, &tx_pre2); + if (rc) + goto err; + + c1 = c2 + 1; + + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtos32(c1, 10, &tx_pre1); + if (rc) + goto err; + + c1 = c2 + 1; + + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtos32(c1, 10, &tx_main); + if (rc) + goto err; + + c1 = c2 + 1; + + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtos32(c1, 10, &tx_post1); + if (rc) + goto err; + + c1 = c2 + 1; + + rc = kstrtos32(c1, 10, &tx_post2); + if (rc) + goto err; + + taps = gaudi->nic_pam4_tx_taps[lane].taps; + taps[0] = tx_pre2; + taps[1] = tx_pre1; + taps[2] = tx_main; + taps[3] = tx_post1; + taps[4] = tx_post2; + + return count; +err: + dev_err(hdev->dev, + "usage: echo > nic_pam4_tx_taps\n"); + + return -EINVAL; +} + +static const struct file_operations debugfs_pam4_tx_taps_fops = { + .owner = THIS_MODULE, + .write = debugfs_pam4_tx_taps_write, +}; + +static ssize_t debugfs_polarity_write(struct file *f, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct hl_device *hdev = file_inode(f)->i_private; + struct gaudi_device *gaudi = hdev->asic_specific; + struct cpucp_nic_info *nic_info = &hdev->asic_prop.cpucp_nic_info; + char kbuf[POLARITY_KBUF_SIZE]; + char *c1, *c2; + ssize_t rc; + u64 val; + u32 lane; + u8 pol_tx, pol_rx; + + if (count > sizeof(kbuf) - 1) + goto err; + if (copy_from_user(kbuf, buf, count)) + goto err; + kbuf[count] = '\0'; + + c1 = kbuf; + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtou32(c1, 10, &lane); + if (rc) + goto err; + + if (lane >= NIC_MAX_NUM_OF_LANES) { + dev_err(hdev->dev, "lane max value is %d\n", + NIC_MAX_NUM_OF_LANES - 1); + return -EINVAL; + } + + c1 = c2 + 1; + + c2 = strchr(c1, ' '); + if (!c2) + goto err; + *c2 = '\0'; + + rc = kstrtou8(c1, 10, &pol_tx); + if (rc) + goto err; + + c1 = c2 + 1; + + rc = kstrtou8(c1, 10, &pol_rx); + if (rc) + goto err; + + if ((pol_tx & ~1) || (pol_rx & ~1)) { + dev_err(hdev->dev, "pol_tx and pol_rx should be 0 or 1\n"); + goto err; + } + + val = le64_to_cpu(nic_info->pol_tx_mask[0]); + val &= ~BIT_ULL(lane); + val |= ((u64) pol_tx) << lane; + nic_info->pol_tx_mask[0] = cpu_to_le64(val); + + val = le64_to_cpu(nic_info->pol_rx_mask[0]); + val &= ~BIT_ULL(lane); + val |= ((u64) pol_rx) << lane; + nic_info->pol_rx_mask[0] = cpu_to_le64(val); + + gaudi->nic_use_fw_polarity = true; + + return count; +err: + dev_err(hdev->dev, + "usage: echo > nic_polarity\n"); + + return -EINVAL; +} + +static const struct file_operations debugfs_polarity_fops = { + .owner = THIS_MODULE, + .write = debugfs_polarity_write, +}; + +static ssize_t debugfs_ports_status_read(struct file *f, char __user *buf, + size_t count, loff_t *ppos) +{ + struct hl_device *hdev = file_inode(f)->i_private; + struct gaudi_device *gaudi = hdev->asic_specific; + char tmp_buf[512] = {0}; + ssize_t rc; + int i, up_cnt = 0, down_cnt = 0; + + if (*ppos) + return 0; + + for (i = 0 ; i < NIC_NUMBER_OF_PORTS ; i++) + if ((hdev->nic_ports_mask & BIT(i))) { + if (gaudi->nic_devices[i].active) + up_cnt++; + else + down_cnt++; + } + + if (up_cnt) { + sprintf(tmp_buf, "%d ports up (", up_cnt); + + for (i = 0 ; i < NIC_NUMBER_OF_PORTS ; i++) + if ((hdev->nic_ports_mask & BIT(i)) && + gaudi->nic_devices[i].active) + sprintf(tmp_buf + strlen(tmp_buf), "%d, ", i); + + sprintf(tmp_buf + strlen(tmp_buf) - 2, ")"); + } + + if (down_cnt) { + if (up_cnt) + sprintf(tmp_buf + strlen(tmp_buf), "\n"); + + sprintf(tmp_buf + strlen(tmp_buf), "%d ports down (", down_cnt); + + for (i = 0 ; i < NIC_NUMBER_OF_PORTS ; i++) + if ((hdev->nic_ports_mask & BIT(i)) && + !gaudi->nic_devices[i].active) + sprintf(tmp_buf + strlen(tmp_buf), "%d, ", i); + + sprintf(tmp_buf + strlen(tmp_buf) - 2, ")"); + } + + sprintf(tmp_buf + strlen(tmp_buf), "\n"); + + rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf, + strlen(tmp_buf) + 1); + + return rc; +} + +static const struct file_operations debugfs_ports_status_fops = { + .owner = THIS_MODULE, + .read = debugfs_ports_status_read, +}; + +#define NIC_DEBUGFS(X, fmt, do_reset) \ +static ssize_t debugfs_##X##_read(struct file *f, \ + char __user *buf, \ + size_t count, \ + loff_t *ppos) \ +{ \ + struct hl_device *hdev = file_inode(f)->i_private; \ + struct gaudi_device *gaudi = hdev->asic_specific; \ + char tmp_buf[32]; \ + ssize_t rc; \ +\ + if (*ppos) \ + return 0; \ +\ + sprintf(tmp_buf, fmt "\n", gaudi->nic_##X); \ + rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf, \ + strlen(tmp_buf) + 1); \ +\ + return rc; \ +} \ +\ +static ssize_t debugfs_##X##_write(struct file *f, \ + const char __user *buf, \ + size_t count, \ + loff_t *ppos) \ +{ \ + struct hl_device *hdev = file_inode(f)->i_private; \ + struct gaudi_device *gaudi = hdev->asic_specific; \ + u64 val, base; \ + ssize_t rc; \ +\ + if (!strcmp(fmt, "%d")) \ + base = 10; \ + else \ + base = 16; \ +\ + rc = kstrtoull_from_user(buf, count, base, &val); \ + if (rc) \ + return rc; \ +\ + if (val == gaudi->nic_##X) \ + return count; \ +\ + if (do_reset && gaudi->nic_debugfs_reset) { \ + gaudi->nic_##X = val; \ + hl_device_reset(hdev, true, false); \ + ssleep(HL_PENDING_RESET_PER_SEC); \ + return count; \ + } \ +\ + dev_info(hdev->dev, "NIC reset for %s started\n", __stringify(X)); \ +\ + rc = gaudi_nic_hard_reset_prepare(hdev); \ + if (rc) \ + return rc; \ +\ + gaudi_nic_stop(hdev); \ +\ + /* must do this so the ports will be reopened */ \ + gaudi->hw_cap_initialized &= ~HW_CAP_NIC_DRV; \ +\ + gaudi->nic_##X = val; \ +\ + gaudi_nic_ports_reopen(hdev); \ +\ + dev_info(hdev->dev, "NIC reset for %s finished\n", __stringify(X)); \ +\ + return count; \ +} \ +\ +static const struct file_operations debugfs_##X##_fops = { \ + .owner = THIS_MODULE, \ + .read = debugfs_##X##_read, \ + .write = debugfs_##X##_write, \ +} + +NIC_DEBUGFS(mac_loopback, "0x%llx", true); +NIC_DEBUGFS(pcs_fail_time_frame, "%d", false); +NIC_DEBUGFS(pcs_fail_threshold, "%d", false); + +void gaudi_nic_debugfs_init(struct hl_device *hdev) +{ + struct gaudi_device *gaudi = hdev->asic_specific; + + debugfs_create_file("nic_mac_loopback", + 0644, + hdev->hl_debugfs.root, + hdev, + &debugfs_mac_loopback_fops); + + debugfs_create_file("nic_ports_status", + 0444, + hdev->hl_debugfs.root, + hdev, + &debugfs_ports_status_fops); + + debugfs_create_file("nic_pcs_fail_time_frame", + 0644, + hdev->hl_debugfs.root, + hdev, + &debugfs_pcs_fail_time_frame_fops); + + debugfs_create_file("nic_pcs_fail_threshold", + 0644, + hdev->hl_debugfs.root, + hdev, + &debugfs_pcs_fail_threshold_fops); + + debugfs_create_file("nic_pam4_tx_taps", + 0444, + hdev->hl_debugfs.root, + hdev, + &debugfs_pam4_tx_taps_fops); + + debugfs_create_file("nic_polarity", + 0444, + hdev->hl_debugfs.root, + hdev, + &debugfs_polarity_fops); + + debugfs_create_u8("nic_check_link", + 0644, + hdev->hl_debugfs.root, + &gaudi->nic_check_link); + + debugfs_create_u8("nic_phy_auto_neg_lpbk", + 0644, + hdev->hl_debugfs.root, + &gaudi->nic_phy_auto_neg_lpbk); +} + +#else + +void gaudi_nic_debugfs_init(struct hl_device *hdev) +{ +} + +#endif /* CONFIG_DEBUG_FS */ From patchwork Thu Sep 10 16:11:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oded Gabbay X-Patchwork-Id: 261136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 893DCC433E2 for ; Thu, 10 Sep 2020 17:59:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1DD9121941 for ; Thu, 10 Sep 2020 17:59:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="s8c71q2T" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726769AbgIJR6p (ORCPT ); Thu, 10 Sep 2020 13:58:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726746AbgIJQNa (ORCPT ); Thu, 10 Sep 2020 12:13:30 -0400 Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4BCB7C0617BA; Thu, 10 Sep 2020 09:12:26 -0700 (PDT) Received: by mail-ej1-x643.google.com with SMTP id lo4so9519173ejb.8; Thu, 10 Sep 2020 09:12:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=f9VqUEtIFQY2Ox8OWjG/8do232WTuGvYl5Af6vWbEzQ=; b=s8c71q2T9OhpwDpvCo8qGfBLZ8ilNRYbIJ8INbnC8UECS58S8Wowy11WLbFV1nhw2V FFv8q+7hwBiEaTvrATBsnzZYRClYEgZll0F4qUTpNfsS8lLdyMktci46t37bGid9fbt4 lN9o6cgHjbnenc6IuZLv1Mls8lqXdkvvpFAbJs/ucWmVUaSIy0BbfSPWsL0ZDMdeLrIx Io2xeNgCZEpeDzJK76kpYTBbGM6u5JaSZKC20dg9E9qyUztQgPz8kSOp4/F1p8PxKQz2 cYn+fTEQS7T0b8A9YHSeD8I3Y5P3zDSbKGRFcs482oJK+v+8rSz3sO27p41vhqTCg4X9 KYBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=f9VqUEtIFQY2Ox8OWjG/8do232WTuGvYl5Af6vWbEzQ=; b=edmOLRq/0Du5Mhvc4BX1PFCkHGRHJytoOPQQKTHJQofDQFEOYh6xurTKrTmM2PsHC6 JmX6rGhXLHKSg5060o+XZMeP2qfZFXOSrz7Fl2uQuS7fVbbDosTHGYr/GZumGaOooffu aFUnBtG1r/qcE8Lm3ZBNAWhK/KPz8z9djZbAB04EC4TQKa0vKzludHFP/yfFVmtmEN4U X3mrArfZsq1c0F/90k+eObslVsPmNDC3oTPgQm56DlIUTJ4STNt7Z5MxSiddk0XcZ5Fk SSY2HtPxMuRFfxvoKlsJ87cuoARiRG0twgCWdzqWWEkCr0tSQcxTUYxp+0xgCT5x8ISy S1TQ== X-Gm-Message-State: AOAM532nrkb4oucEHvErzOE4JteNrlyuMwGWOeiLuRbQGIt7hTsZqvUq MPf63lzwrX5TdtVzdjmpNSYfQmnx9lU= X-Google-Smtp-Source: ABdhPJzXx+TNQfI8k6Qb2+sAzMIMwcozUx/uaUa0o/YOYG9m8Wz1VCl0WqWU7JQPbJvBSt6iXOU34Q== X-Received: by 2002:a17:906:a293:: with SMTP id i19mr9908052ejz.428.1599754344331; Thu, 10 Sep 2020 09:12:24 -0700 (PDT) Received: from ogabbay-VM.habana-labs.com ([213.57.90.10]) by smtp.gmail.com with ESMTPSA id k8sm7282911ejz.60.2020.09.10.09.12.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 09:12:22 -0700 (PDT) From: Oded Gabbay To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org Cc: SW_Drivers@habana.ai, gregkh@linuxfoundation.org, davem@davemloft.net, kuba@kernel.org, Omer Shpigelman Subject: [PATCH 15/15] habanalabs/gaudi: add NIC init/fini calls from common code Date: Thu, 10 Sep 2020 19:11:26 +0300 Message-Id: <20200910161126.30948-16-oded.gabbay@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200910161126.30948-1-oded.gabbay@gmail.com> References: <20200910161126.30948-1-oded.gabbay@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Omer Shpigelman Finally, enable the NIC engines. Initialize the NIC ports mask variable with full mask so all ports will be initialized. Call the NIC init/fini from the common code. Signed-off-by: Omer Shpigelman Reviewed-by: Oded Gabbay Signed-off-by: Oded Gabbay --- drivers/misc/habanalabs/common/device.c | 18 +++++++++++++++ drivers/misc/habanalabs/common/habanalabs.h | 6 +++++ .../misc/habanalabs/common/habanalabs_drv.c | 1 + drivers/misc/habanalabs/common/pci.c | 1 + drivers/misc/habanalabs/gaudi/gaudi.c | 23 +++++++++++++++++++ drivers/misc/habanalabs/goya/goya.c | 12 ++++++++++ 6 files changed, 61 insertions(+) diff --git a/drivers/misc/habanalabs/common/device.c b/drivers/misc/habanalabs/common/device.c index 57f5b945fa41..8f8744e4f2c7 100644 --- a/drivers/misc/habanalabs/common/device.c +++ b/drivers/misc/habanalabs/common/device.c @@ -1077,6 +1077,12 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset, goto out_err; } + rc = hdev->asic_funcs->nic_init(hdev); + if (rc) { + dev_err(hdev->dev, "Failed to init NIC driver\n"); + goto out_err; + } + hl_set_max_power(hdev); } else { rc = hdev->asic_funcs->soft_reset_late_init(hdev); @@ -1312,6 +1318,13 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto out_disabled; } + rc = hdev->asic_funcs->nic_init(hdev); + if (rc) { + dev_err(hdev->dev, "Failed to init NIC driver\n"); + rc = 0; + goto out_disabled; + } + /* * Expose devices and sysfs nodes to user. * From here there is no need to add char devices and create sysfs nodes @@ -1463,6 +1476,11 @@ void hl_device_fini(struct hl_device *hdev) hl_cb_pool_fini(hdev); + /* the NIC uses the kernel context for MMU mappings, therefore must be + * cleaned before it + */ + hdev->asic_funcs->nic_fini(hdev); + /* Release kernel context */ if ((hdev->kernel_ctx) && (hl_ctx_put(hdev->kernel_ctx) != 1)) dev_err(hdev->dev, "kernel ctx is still alive\n"); diff --git a/drivers/misc/habanalabs/common/habanalabs.h b/drivers/misc/habanalabs/common/habanalabs.h index 1f3735a64d88..1c1850114aa4 100644 --- a/drivers/misc/habanalabs/common/habanalabs.h +++ b/drivers/misc/habanalabs/common/habanalabs.h @@ -680,6 +680,10 @@ struct hl_info_mac_addr; * then the timeout is the default timeout for the specific * ASIC * @get_hw_state: retrieve the H/W state + * @nic_init: init the NIC H/W and I/F. This should be called in the final satge + * of the init flow, as we must not have anything that might fail + * during its initialization after the NIC init. + * @nic_fini: perform NIC cleanup. * @nic_control: Perform NIC related operations. * @nic_cq_mmap: map the NIC CQ buffer. * @pci_bars_map: Map PCI BARs. @@ -786,6 +790,8 @@ struct hl_asic_funcs { int (*send_cpu_message)(struct hl_device *hdev, u32 *msg, u16 len, u32 timeout, long *result); enum hl_device_hw_state (*get_hw_state)(struct hl_device *hdev); + int (*nic_init)(struct hl_device *hdev); + void (*nic_fini)(struct hl_device *hdev); int (*nic_control)(struct hl_device *hdev, u32 op, void *input, void *output); int (*nic_cq_mmap)(struct hl_device *hdev, struct vm_area_struct *vma); diff --git a/drivers/misc/habanalabs/common/habanalabs_drv.c b/drivers/misc/habanalabs/common/habanalabs_drv.c index df92afc1b9d5..fcb28975fac5 100644 --- a/drivers/misc/habanalabs/common/habanalabs_drv.c +++ b/drivers/misc/habanalabs/common/habanalabs_drv.c @@ -247,6 +247,7 @@ static void set_driver_behavior_per_device(struct hl_device *hdev) hdev->bmc_enable = 1; hdev->hard_reset_on_fw_events = 1; hdev->card_type = cpucp_card_type_pci; + hdev->nic_ports_mask = 0x3FF; hdev->nic_ports_ext_mask = 0x3FF; hdev->nic_auto_neg_mask = 0x3FF; hdev->nic_load_fw = 0; diff --git a/drivers/misc/habanalabs/common/pci.c b/drivers/misc/habanalabs/common/pci.c index 923b2606e29f..c376ab4695ab 100644 --- a/drivers/misc/habanalabs/common/pci.c +++ b/drivers/misc/habanalabs/common/pci.c @@ -230,6 +230,7 @@ int hl_pci_set_inbound_region(struct hl_device *hdev, u8 region, lower_32_bits(pci_region->addr)); rc |= hl_pci_iatu_write(hdev, offset + 0x18, upper_32_bits(pci_region->addr)); + /* Set bar type as memory */ rc |= hl_pci_iatu_write(hdev, offset + 0x0, 0); /* Enable + bar/address match + match enable + bar number */ diff --git a/drivers/misc/habanalabs/gaudi/gaudi.c b/drivers/misc/habanalabs/gaudi/gaudi.c index eb733a48eb72..758a26b43cf2 100644 --- a/drivers/misc/habanalabs/gaudi/gaudi.c +++ b/drivers/misc/habanalabs/gaudi/gaudi.c @@ -882,6 +882,27 @@ static void gaudi_late_fini(struct hl_device *hdev) hdev->hl_chip_info->info = NULL; } +static int gaudi_nic_init(struct hl_device *hdev) +{ + /* + * In init flow we initialize the NIC ports from scratch. In hard reset + * flow, we get here after the NIC ports were halted, hence we only + * need to reopen them. + */ + if (atomic_read(&hdev->in_reset)) { + gaudi_nic_ports_reopen(hdev); + return 0; + } + + return gaudi_nic_ports_init(hdev); +} + +static void gaudi_nic_fini(struct hl_device *hdev) +{ + /* must be called after MSI was disabled */ + gaudi_nic_ports_fini(hdev); +} + static void gaudi_nic_handle_rx(struct gaudi_nic_device *gaudi_nic) { /* at this point, interrupts were disabled by the H/W */ @@ -7482,6 +7503,8 @@ static const struct hl_asic_funcs gaudi_funcs = { .get_eeprom_data = gaudi_get_eeprom_data, .send_cpu_message = gaudi_send_cpu_message, .get_hw_state = gaudi_get_hw_state, + .nic_init = gaudi_nic_init, + .nic_fini = gaudi_nic_fini, .nic_control = gaudi_nic_control, .nic_cq_mmap = gaudi_nic_cq_mmap, .pci_bars_map = gaudi_pci_bars_map, diff --git a/drivers/misc/habanalabs/goya/goya.c b/drivers/misc/habanalabs/goya/goya.c index 6e98c830f6a2..e753b3a0079f 100644 --- a/drivers/misc/habanalabs/goya/goya.c +++ b/drivers/misc/habanalabs/goya/goya.c @@ -5265,6 +5265,16 @@ static enum hl_device_hw_state goya_get_hw_state(struct hl_device *hdev) return RREG32(mmHW_STATE); } +static int goya_nic_init(struct hl_device *hdev) +{ + return 0; +} + +static void goya_nic_fini(struct hl_device *hdev) +{ + +} + static int goya_nic_control(struct hl_device *hdev, u32 op, void *input, void *output) { @@ -5405,6 +5415,8 @@ static const struct hl_asic_funcs goya_funcs = { .get_eeprom_data = goya_get_eeprom_data, .send_cpu_message = goya_send_cpu_message, .get_hw_state = goya_get_hw_state, + .nic_init = goya_nic_init, + .nic_fini = goya_nic_fini, .nic_control = goya_nic_control, .nic_cq_mmap = goya_nic_mmap, .pci_bars_map = goya_pci_bars_map,