From patchwork Tue Jul 27 23:20:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 488492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E954C432BE for ; Tue, 27 Jul 2021 23:21:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E51A560234 for ; Tue, 27 Jul 2021 23:21:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232750AbhG0XVN (ORCPT ); Tue, 27 Jul 2021 19:21:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:35268 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233414AbhG0XU7 (ORCPT ); Tue, 27 Jul 2021 19:20:59 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5894B60F5E; Tue, 27 Jul 2021 23:20:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1627428057; bh=UiKdMONG33AJPMkQRdMxShMFSxi4u5GzfcIWB35xMvs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T2hGdML5/i2/pK8r51mN+fTlhN9mgHpFSgMWc62ncA0FrbELGWD4Q+gMsqLCEpnbM 2/+Vp52gp96tfBqxPDgZ7u6E01DRLRa5hEroc7gHkxvnzP3Mcn0n7gx5Z4fjdaqae0 W3Zt4IZuO8TJc6nIRaJ9fGbu5EAdAlIlJ0y75HEk1OWMGys/qFmMAvyEnKlirVo8+g RP/3dpzviywzv7nvbrp/5hQA0oCfKQIWzaL5TnwWLPA3bV3UHZAQbsr2nlMkjHBces IDdyUAfE44o87eP+vhI0ETIrxINcRVcXW3w7FX5yweNYV0X/KxGP5og4Fv6i6Aw4O1 nvrHMyBEfUocQ== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski Cc: netdev@vger.kernel.org, Tariq Toukan , Aya Levin , Moshe Shemesh , Saeed Mahameed Subject: [net 10/12] net/mlx5: Unload device upon firmware fatal error Date: Tue, 27 Jul 2021 16:20:48 -0700 Message-Id: <20210727232050.606896-11-saeed@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210727232050.606896-1-saeed@kernel.org> References: <20210727232050.606896-1-saeed@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Aya Levin When fw_fatal reporter reports an error, the firmware in not responding. Unload the device to ensure that the driver closes all its resources, even if recovery is not due (user disabled auto-recovery or reporter is in grace period). On successful recovery the device is loaded back up. Fixes: b3bd076f7501 ("net/mlx5: Report devlink health on FW fatal issues") Signed-off-by: Aya Levin Reviewed-by: Moshe Shemesh Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/health.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index 9ff163c5bcde..9abeb80ffa31 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -626,8 +626,16 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) } fw_reporter_ctx.err_synd = health->synd; fw_reporter_ctx.miss_counter = health->miss_counter; - devlink_health_report(health->fw_fatal_reporter, - "FW fatal error reported", &fw_reporter_ctx); + if (devlink_health_report(health->fw_fatal_reporter, + "FW fatal error reported", &fw_reporter_ctx) == -ECANCELED) { + /* If recovery wasn't performed, due to grace period, + * unload the driver. This ensures that the driver + * closes all its resources and it is not subjected to + * requests from the kernel. + */ + mlx5_core_err(dev, "Driver is in error state. Unloading\n"); + mlx5_unload_one(dev); + } } static const struct devlink_health_reporter_ops mlx5_fw_fatal_reporter_ops = {