From patchwork Tue Jun 4 05:09:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801559 Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F0D977A1E; Tue, 4 Jun 2024 05:10:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477840; cv=none; b=VyvdtlMHdYbmbKG49SicS5Uh9BLTF/B6AoUMDQnwAGNW6UukQQli+AXvEqW+fwCm7klTd5YXTsX7q1qsIuSLymWU6a7GyqP41pxitDxV+7phiO66kYTVEljykmZRzMXb+LbgQnHmyYH6kVaqGVNeg7+O3UKCXSlpzV7gvAmKWII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477840; c=relaxed/simple; bh=VBR98JGVknh0QIXU1LbDkiwLPZeprLJur5lg0RvecfE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dXzi8chAOHKg5RPzMD14fmsjxbYOtYi3aYp9aWvZdC1I6IQ+fUWbfb0utLbYR0NcTZaWgq0JCk/59qXI77860zOSs/s0f6R8LT1Cqo4jIEgGfgFTL6wgYHN54pefALlLCp9QY7DDkkvplgxIwYbBxd4AGbEqxA1mqjb7jef6da0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=d1EaCc/x; arc=none smtp.client-ip=209.85.167.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d1EaCc/x" Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3d1facdf12bso224547b6e.0; Mon, 03 Jun 2024 22:10:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477837; x=1718082637; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=gOw8AgrUJDkpfPA0Lzrm6o/e4yMy7flaTek+BD0V1x4=; b=d1EaCc/xzVsuRc/jIMk93NC6Rd6JYq4x6wAQ/2xKJGnTESMLLInq/rxVQauFaUmav/ 3xZdm+/uIBg8su5/+0O8Z62p8N4MlMnzB/e8SA5xo/kffL0WHwKSt7WWjA/2bftmECUg yb880VnAD4J8fVqa6qK+RmHbtrRBJC/kdtyCP8WO8VdsOXlsIdB+5K/q4WxxbDX/lJY/ 9DkFqjY8NOQ2vMOZZ07DgJCdBdUrLxj2rbU1TdDNWIyG/zRDAwUFk7p2Zq7EO+wUXoLn mgZn7rBGJhm31ouvZWoGXFFJk4uES6TR8/ObRocQVcZ72W8ol+T3GLgDWUqv1OwgaeAS VuMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477837; x=1718082637; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=gOw8AgrUJDkpfPA0Lzrm6o/e4yMy7flaTek+BD0V1x4=; b=SqeXS0MK57/bEj9MsENLYM7Ju+k+/LfEtWSFQqjBziSw376gktb7jPT/PF3y9MuHUr x+jyM5DrkG3A1RaHrCW2Xz+BJSxFKd6la3PSbTFnK10uo0VC3mYFP1ZEDWHg/3fecJ1Q 6kayjbY3CvbipB1xGkDPZXnTL0gK8eRI4r7Xd+dyX9rV/eLPSggVtb5jZ615JIpUywki dZI1HzVbvZs8E8cQ6oXxwcR5dYow1AdEcVCWH4rU7lphUON3ZxOJCudLPm7nh5zqgI3a kezgyrLrSGusOqUTn3ZmAS4eYt2sb2wOtpPt89HwQ5RvB26OzJySY6hTpzbj67IIO1dN H6HA== X-Forwarded-Encrypted: i=1; AJvYcCVcQo7+yDJzz5X0FPrSDUUUSnsP3KB/cN/MCebfzUJysqCYcYKsYEL37EEpB3guQb+xK2UMhGjEoz+Iur7/QshjE4n23ltDbP3R+hxwo7WBIhjbW/jcroqQ9c+4w7opxDaQ+naGKT4p2ClPY2Z1MhdtLF2kQKilugW637nUupQtoo8q81KDG/80we+rt3cenSKtaC+HMmAqIVhaCYuwnKOuTApxF+Z1jkKXVLJYyFKRMUA7PcJWHRFiE7CSYBM= X-Gm-Message-State: AOJu0Yyw4csjZmAHEZvO/SxyOgLbWF38UGqIbPp0WDvkl3SD3z+s9UaC 1THVUQqjIHAfJ/LyhHPij2QjyWlVah1+ljJHukeE5EkqEnod9Evv X-Google-Smtp-Source: AGHT+IEyA+H/NUVYyadV6QS+sJdPE+MGaP8srmU6cCY3RydFGapT11Ew5xGOL60RZNIYPFbiiOl3Sw== X-Received: by 2002:a05:6808:602:b0:3d2:5b:df2d with SMTP id 5614622812f47-3d2005be0acmr41704b6e.35.1717477837405; Mon, 03 Jun 2024 22:10:37 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:37 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 01/12] Drivers: hv: vmbus: Drop unsupported VMBus devices earlier Date: Mon, 3 Jun 2024 22:09:29 -0700 Message-Id: <20240604050940.859909-2-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley Because Hyper-V doesn't know ahead-of-time what operating system will be running in a guest VM, it may offer to Linux guests VMBus devices that are intended for use only in Windows guests. Currently in Linux, VMBus channels are created for these devices, and they are processed by vmbus_device_register(). While nothing further happens because no matching driver is found, the channel continues to exist. To avoid having the spurious channel, drop such devices immediately in vmbus_onoffer(), based on identifying the device in the vmbus_unsupported_devs table. If Hyper-V should issue a rescind request for the device, no matching channel will be found and the rescind will also be ignored. Since unsupported devices are dropped early, the check for unsupported devices in hv_get_dev_type() is redundant. Remove it. Signed-off-by: Michael Kelley --- drivers/hv/channel_mgmt.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c index 3c6011a48dab..a216a0aede73 100644 --- a/drivers/hv/channel_mgmt.c +++ b/drivers/hv/channel_mgmt.c @@ -203,7 +203,7 @@ static u16 hv_get_dev_type(const struct vmbus_channel *channel) const guid_t *guid = &channel->offermsg.offer.if_type; u16 i; - if (is_hvsock_channel(channel) || is_unsupported_vmbus_devs(guid)) + if (is_hvsock_channel(channel)) return HV_UNKNOWN; for (i = HV_IDE; i < HV_UNKNOWN; i++) { @@ -1036,6 +1036,16 @@ static void vmbus_onoffer(struct vmbus_channel_message_header *hdr) trace_vmbus_onoffer(offer); + /* + * Hyper-V may offer pseudo-devices with functionality intended for + * Windows guests. Silently ignore them. There's no value in + * cluttering dmesg with error messages. + */ + if (is_unsupported_vmbus_devs(&offer->offer.if_type)) { + atomic_dec(&vmbus_connection.offer_in_progress); + return; + } + if (!vmbus_is_valid_offer(offer)) { pr_err_ratelimited("Invalid offer %d from the host supporting isolation\n", offer->child_relid); From patchwork Tue Jun 4 05:09:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801558 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F47F13E05B; Tue, 4 Jun 2024 05:10:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477842; cv=none; b=e0ldZiF/PTOsxaz/oiwlDz3fdml5x3YNdzpWlnN4VgO/UEyEVbRekclONmJenAS79ZKJ2zUIm9D0mtXTcwD3spKbwujIwXn0/uiAz8IDn4tsQ0KwRAetCzGYZn2ctOdIm9o2vqGk/n9YPcpl9mOLMG4SRf0TphUJXvXtdRM5T4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477842; c=relaxed/simple; bh=zSe+eD0qV+e71QCOWMNlBwLmbidtc2MRHQzXAFw6Uj0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tG1B+NU8M0rftQfv03G2JtiCImCDAKZi2dBfa7EZPoozF+bWMDiZAHDHfulcey+b5tTHowR1EOmVMdxsjTYMDMql6qcTpoN3A6lGNnA3HoULNyCYRAoOSTTBW5l3RP1uMcDbrfQDO1+9P0E6ft68QGj23/uFp4Qr0zmJdyodtHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KaHnNqYa; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KaHnNqYa" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70276322ad8so649008b3a.1; Mon, 03 Jun 2024 22:10:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477840; x=1718082640; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Tyt2LCARZb9G8yUFB+4R0wf0B3JqyvqSEeHJTztg/6g=; b=KaHnNqYa6zww4Gs0ITsJICh+2jdTpNEqbSaT8vj8aXA1FzDPxx0wyvfFOoJz6vsnAI bIP9NtGE/rhQimPSFyeV+X6eyltXiKTxcZyt7oI7Z2q2Euagsrchtkv6UQV9kV6UWk5w Sslp39Zbar4B23I3PX9j0Ax1uQVFlq/CaB4+LW/xuWWDP9Pzysi3lLi7/CozHikUjXma BgLo6CbQsBvKCzizKJbVb02DZKSn0zd/7Bx4mKKK/SVStHC+9g553hAfvMZAOZ91v1J2 xQsg1cwLXVdwErhza6mEcDJEsk4Iu8J4idlDi3jrOmlmWSTmnKSJWzlvRKeO71PXA6P8 wVrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477840; x=1718082640; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Tyt2LCARZb9G8yUFB+4R0wf0B3JqyvqSEeHJTztg/6g=; b=OOBUnWtI2Vz7+DH5fajOqm/oxadjVX3BLRtvDhSLz57jsSj5DdY+isssM6f+EkhuFg M0Jp5nek+oM+Ikn1RhXZFkENyrZzmTLUDeAAEjVgkutqUXGafQ/Iocj3m6JISwTCCVcR SCX83i2L0hhJ/QCrHDJ5Rl15/ams+FgdjqylLErDS58ns61H3PPJQWtg0NflQJxEMlh1 WkwfmdFTRJxJoP3M7ysKjqoSRugVU1uloH5ccw8npeqKjzYJ5y1dBpm8get/6EvUOpBV 1OdIf0u3K3VPcDZR07g82K8V5lWhXWgV+pdXZOd2vJhwID0JEG03xEY+mBHAGUJVVqf+ 4yzA== X-Forwarded-Encrypted: i=1; AJvYcCV+6Yn5P8tDv/GNAGGh/DhpOewAWjfQmXyWr/2XiFJym8rkXiw7pdPg5f/OtVID5+B4oTl++hKpiAsGBVA/NUpEsPjd7A9MKdEOE3wtYuxgWS1HY2pwiE3OUTPJlVvbWffjW1Zc9LYAeN8MbrImzFXpoYzJ8lgWT8n+JQXszbazVgqsFOd42elSZnbe/+iRyewxL9rVZzew8BwE3zdjJaYmzGIT691VC33L6v+OAfPUZUXfLhL3Hkc5ic30aRs= X-Gm-Message-State: AOJu0YyJpdOyGVO6dMe3B1x8mgDO+K88apQyuBA+g7TsfY3ZFw3G6jip FosOxmFpgacuDJkIQTUu+A7kDXuTp6AFLrXGfws1PoSUasKjAsfj X-Google-Smtp-Source: AGHT+IFhTA5PSRFrRUUXGMk5zEUWP+kSdnmr7igsWz9177BNVWX0y0FgoWXqzbK+Nj7MFqlG3jitow== X-Received: by 2002:a05:6a00:230a:b0:6f3:f970:9f2a with SMTP id d2e1a72fcca58-702477e69femr12966122b3a.10.1717477839971; Mon, 03 Jun 2024 22:10:39 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:39 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 03/12] Drivers: hv: vmbus: Add an IRQ name to VMBus channels Date: Mon, 3 Jun 2024 22:09:31 -0700 Message-Id: <20240604050940.859909-4-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley In preparation for assigning Linux IRQs to VMBus channels, assign a string name to each channel. The name is used when the IRQ is requested, and it will appear in the output of /proc/interrupts to make it easier to identify the device the IRQ is associated with. Many VMBus devices are single-instance, with a single channel. For such devices, a default string name can be determined based on the GUID identifying the device. So add default names to the vmbus_devs table that lists recognized devices. When a channel is created, set the channel name to that default so a reasonable name is displayed without requiring VMBus driver modifications. However, individual VMBus device drivers may be optionally modified to override the default name to provide additional information. In cases where a driver supports multiple instances of a device, or a device has multiple channels, the additional information may be helpful to display in /proc/interrupts. Signed-off-by: Michael Kelley --- drivers/hv/channel_mgmt.c | 45 +++++++++++++++++++++++++++++++-------- include/linux/hyperv.h | 5 +++++ 2 files changed, 41 insertions(+), 9 deletions(-) diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c index a216a0aede73..adbe184e5197 100644 --- a/drivers/hv/channel_mgmt.c +++ b/drivers/hv/channel_mgmt.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -33,6 +34,7 @@ const struct vmbus_device vmbus_devs[] = { HV_IDE_GUID, .perf_device = true, .allowed_in_isolated = false, + .irq_name = "storvsc", }, /* SCSI */ @@ -40,6 +42,7 @@ const struct vmbus_device vmbus_devs[] = { HV_SCSI_GUID, .perf_device = true, .allowed_in_isolated = true, + .irq_name = "storvsc", }, /* Fibre Channel */ @@ -47,6 +50,7 @@ const struct vmbus_device vmbus_devs[] = { HV_SYNTHFC_GUID, .perf_device = true, .allowed_in_isolated = false, + .irq_name = "storvsc", }, /* Synthetic NIC */ @@ -54,6 +58,7 @@ const struct vmbus_device vmbus_devs[] = { HV_NIC_GUID, .perf_device = true, .allowed_in_isolated = true, + .irq_name = "netvsc", }, /* Network Direct */ @@ -61,6 +66,7 @@ const struct vmbus_device vmbus_devs[] = { HV_ND_GUID, .perf_device = true, .allowed_in_isolated = false, + .irq_name = "netdirect", }, /* PCIE */ @@ -68,6 +74,7 @@ const struct vmbus_device vmbus_devs[] = { HV_PCIE_GUID, .perf_device = false, .allowed_in_isolated = true, + .irq_name = "vpci", }, /* Synthetic Frame Buffer */ @@ -75,6 +82,7 @@ const struct vmbus_device vmbus_devs[] = { HV_SYNTHVID_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "framebuffer", }, /* Synthetic Keyboard */ @@ -82,6 +90,7 @@ const struct vmbus_device vmbus_devs[] = { HV_KBD_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "keyboard", }, /* Synthetic MOUSE */ @@ -89,6 +98,7 @@ const struct vmbus_device vmbus_devs[] = { HV_MOUSE_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "mouse", }, /* KVP */ @@ -96,6 +106,7 @@ const struct vmbus_device vmbus_devs[] = { HV_KVP_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "kvp", }, /* Time Synch */ @@ -103,6 +114,7 @@ const struct vmbus_device vmbus_devs[] = { HV_TS_GUID, .perf_device = false, .allowed_in_isolated = true, + .irq_name = "timesync", }, /* Heartbeat */ @@ -110,6 +122,7 @@ const struct vmbus_device vmbus_devs[] = { HV_HEART_BEAT_GUID, .perf_device = false, .allowed_in_isolated = true, + .irq_name = "heartbeat", }, /* Shutdown */ @@ -117,6 +130,7 @@ const struct vmbus_device vmbus_devs[] = { HV_SHUTDOWN_GUID, .perf_device = false, .allowed_in_isolated = true, + .irq_name = "shutdown", }, /* File copy */ @@ -126,6 +140,7 @@ const struct vmbus_device vmbus_devs[] = { HV_FCOPY_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "fcopy", }, /* Backup */ @@ -133,6 +148,7 @@ const struct vmbus_device vmbus_devs[] = { HV_VSS_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "vss", }, /* Dynamic Memory */ @@ -140,6 +156,7 @@ const struct vmbus_device vmbus_devs[] = { HV_DM_GUID, .perf_device = false, .allowed_in_isolated = false, + .irq_name = "balloon", }, /* @@ -198,20 +215,30 @@ static bool is_unsupported_vmbus_devs(const guid_t *guid) return false; } -static u16 hv_get_dev_type(const struct vmbus_channel *channel) +static void hv_set_dev_type_and_irq_name(struct vmbus_channel *channel) { const guid_t *guid = &channel->offermsg.offer.if_type; + char *name = NULL; u16 i; - if (is_hvsock_channel(channel)) - return HV_UNKNOWN; - - for (i = HV_IDE; i < HV_UNKNOWN; i++) { - if (guid_equal(guid, &vmbus_devs[i].guid)) - return i; + if (is_hvsock_channel(channel)) { + i = HV_UNKNOWN; + name = "hv_sock"; + goto found; } + + for (i = HV_IDE; i < HV_UNKNOWN; i++) + if (guid_equal(guid, &vmbus_devs[i].guid)) { + name = vmbus_devs[i].irq_name; + goto found; + } + pr_info("Unknown GUID: %pUl\n", guid); - return i; + +found: + channel->device_id = i; + if (name) + strscpy(channel->irq_name, name, VMBUS_CHAN_IRQ_NAME_MAX); } /** @@ -970,7 +997,7 @@ static void vmbus_setup_channel_state(struct vmbus_channel *channel, sizeof(struct vmbus_channel_offer_channel)); channel->monitor_grp = (u8)offer->monitorid / 32; channel->monitor_bit = (u8)offer->monitorid % 32; - channel->device_id = hv_get_dev_type(channel); + hv_set_dev_type_and_irq_name(channel); } /* diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index d52c916cc492..897d96fa19d4 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -826,6 +826,7 @@ struct vmbus_device { guid_t guid; bool perf_device; bool allowed_in_isolated; + char *irq_name; }; #define VMBUS_DEFAULT_MAX_PKT_SIZE 4096 @@ -837,6 +838,8 @@ struct vmbus_gpadl { bool decrypted; }; +#define VMBUS_CHAN_IRQ_NAME_MAX 32 + struct vmbus_channel { struct list_head listentry; @@ -1068,6 +1071,8 @@ struct vmbus_channel { /* The max size of a packet on this channel */ u32 max_pkt_size; + + char irq_name[VMBUS_CHAN_IRQ_NAME_MAX]; }; #define lock_requestor(channel, flags) \ From patchwork Tue Jun 4 05:09:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801557 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41F8B13FD9D; Tue, 4 Jun 2024 05:10:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477844; cv=none; b=qigPTb1BXbFM4oz4WRQ29ddn5x3480uxvXjZgM8SeP+5RxLUKouRBE06XIqEuM7YzgzJtX3j0AuwT0Oe1IsmypxImsNXSEL1H0teNljGPWu8J6ATthDoQ8B6WmtZvnTYJl92+uPdDK6zrlB1/w9uLclTEqG+m4XMbbbZVvwMvaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477844; c=relaxed/simple; bh=IqYFKETOx4ZEABhYk+CFGp7F4wFOKDydc/vQs12EN78=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cnBQAuzecfmB9VvcgU8aw6p1Ghvcm+s0cersmwRlhCmuFU2cdGNJSosj3RFpROtpOK8Zem2QZ+/BIuIFRA0Pw/Q6Zy+rTr3JcHaiPFIYxh0vDY6ng87ugR9JNSZzdVHfr1dZiUWhlB4gZwz3U6UEQOqeyqHTGQlslVrN9GlerIg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TCadiSsd; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TCadiSsd" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7025b84c0daso1946939b3a.2; Mon, 03 Jun 2024 22:10:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477842; x=1718082642; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=EJqMDDGIN+lFQ1M8fhUyjmsJ5wMGfKMbSehKHoi0sAs=; b=TCadiSsdC2MLNKUp7ZPmtYmHtvvyJ8E2H4WwpErN5fiY6MW+2+TTpAWnl1jMNi5OqR r6qBJYgU+eYQf2Ieg3IklH92yfCxeL1QKwoQkipso6SElEtUjwgorcXf9xN1xnujArRR HrTPQD+vdcYbOrkZr/8Ar68hG7EhdrWe0kzREKXu/MXuWvc1g84oAWlmE/rvaA4wf/8O GN5cL38HY4Z7w0RCDmJ3KacqH4XFXb/kqSyiFF9XSbHn8QXE08+8t3EX0UTgQHb45EY/ w8BJK3Rg75T0eyCUGXz5W1JX8rg0iu+JTJYEBx4OMRanr/nt267Mo33ZrzPLMULOUlRH HRpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477842; x=1718082642; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=EJqMDDGIN+lFQ1M8fhUyjmsJ5wMGfKMbSehKHoi0sAs=; b=uN17L4saVxFptCgphL1jcyF5sdnR770kAMwLVcc/OlGNGBDmde+LeeQcHD0aIkZGJA Z1gnAJ2t7dkud6Knp5fK2gA9vMhPnxZ6aE8Ed3Azc9T1zalnQSqyHKLZ+edddYDMDXk2 bOxl6nrdCn1/uyGHsfOCBAVMn3aMLOQj6zuyw3lCfd10L6dc4N9VrbowTXs/5qECDUgR TwRCrMXB8RM8Dw7JdQFKWiAHlcyOaLA6FF4z71dY9f2d7p8enWozNAAR1LwBK1mx5FlH OhAnKdVzJ9XZOUWjydHzyKxt2PskpvWRTbbpR3fUAN9hvvGf3WUb/a2nNCg7b6dLaNCL 0BcQ== X-Forwarded-Encrypted: i=1; AJvYcCWPrt8PfzC+3uKj8aCDZ9czcetS4tntlUrhk90GQjbH+hcFdphhGRPF5klvOXrk4QY9z+3R63mZoK/p8/V+oqmn4XnkI29BE3ojdcQVGkLjsa40ToX0+8FWofVsahO5Mv22C/z3dP/datF1jkIovecm3rafNZsAqbUa90cl0V0mxgyGh1ejpM1vymVPDk+B40HqkETvfUdF5fHXA67mxhkso64ZG9jtLgtwHuvYwEgxtRaO5B15uHzYl9ENicQ= X-Gm-Message-State: AOJu0Yz43P0ID3eqIXaMORUadL0PVriTtEmPVPRQBPGJVtSRiNJMelge QphaxW1RahZPQh2UFtP14T86GUG+uifAyra/oL4br7L+0x5RWJgs X-Google-Smtp-Source: AGHT+IHtRsaw2+6nASXeqSpTfk7BQZcffCsouxI2iBmgROU4XGXj+c/Mw8gWcFSMTNAGumyngsyiDA== X-Received: by 2002:a05:6a00:2e90:b0:702:5950:178b with SMTP id d2e1a72fcca58-702595017e1mr8916905b3a.30.1717477842482; Mon, 03 Jun 2024 22:10:42 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:42 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 05/12] scsi: storvsc: Annotate the VMBus channel IRQ name Date: Mon, 3 Jun 2024 22:09:33 -0700 Message-Id: <20240604050940.859909-6-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley In preparation for assigning Linux IRQs to VMBus channels, annotate the IRQ name in the VMBus channels used by the virtual IDE, SCSI, and Fibre Channel controllers in a Hyper-V guest. Distinguish the primary channel and sub-channels with "pri" and "sub" prefixes. Identify controllers with a numeric suffix that matches the numeric part of the "host" name displayed by "lsscsi" and SCSI-related entries in sysfs. Signed-off-by: Michael Kelley --- drivers/scsi/storvsc_drv.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c index 7ceb982040a5..11b3fc3b24c9 100644 --- a/drivers/scsi/storvsc_drv.c +++ b/drivers/scsi/storvsc_drv.c @@ -686,6 +686,8 @@ static void handle_sc_creation(struct vmbus_channel *new_sc) new_sc->max_pkt_size = STORVSC_MAX_PKT_SIZE; new_sc->next_request_id_callback = storvsc_next_request_id; + snprintf(new_sc->irq_name, VMBUS_CHAN_IRQ_NAME_MAX, "sub@storvsc%d", + stor_device->host->host_no); ret = vmbus_open(new_sc, aligned_ringbuffer_size, @@ -1322,6 +1324,7 @@ static int storvsc_connect_to_vsp(struct hv_device *device, u32 ring_size, bool is_fc) { struct vmstorage_channel_properties props; + struct storvsc_device *stor_device; int ret; memset(&props, 0, sizeof(struct vmstorage_channel_properties)); @@ -1329,6 +1332,12 @@ static int storvsc_connect_to_vsp(struct hv_device *device, u32 ring_size, device->channel->max_pkt_size = STORVSC_MAX_PKT_SIZE; device->channel->next_request_id_callback = storvsc_next_request_id; + stor_device = get_out_stor_device(device); + if (!stor_device) + return -EINVAL; + snprintf(device->channel->irq_name, VMBUS_CHAN_IRQ_NAME_MAX, "pri@storvsc%d", + stor_device->host->host_no); + ret = vmbus_open(device->channel, ring_size, ring_size, From patchwork Tue Jun 4 05:09:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801556 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06AAB1419B1; Tue, 4 Jun 2024 05:10:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477847; cv=none; b=uxX2k1GjZmD1i37HzEJU1qFC14Zc4wYTYrGkJQRbmsnvofAglfnsN9t9U1gPOgsgmmmVEpwABtyPxknSZSzcD4d1F36IBvRGfeDe5F5hLvaH8nqnLfyLKy/B4ta/rUcYiIw+n2wnXbpUQeVTjw7mZ1haTsrSSF8RZL8EPKFJaco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477847; c=relaxed/simple; bh=vVRI2TfY68wado4CSOgxUZSaUZKTk/D7LMeKecnuSBk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=G5y1bdCkGoNWGM9E/z3nirrKsZ0gpns4xKj7wU8gvj0w+6FFUDQQbtDCgrwspeaw9VDLFh0ga2U2wBO0IUtLReG+ilPGVhCKPq6GMzfETb2tWar5Bs87HvY63CweuJEWSXnN7YI20/3y76o6WbM7N+5I4BM13hNKxB+A3CCmlpw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XYYwQnoN; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XYYwQnoN" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70245b22365so2957212b3a.1; Mon, 03 Jun 2024 22:10:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477845; x=1718082645; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=uI7hPio5EOSCOqhNP+fmg82P5jjcL6y/nOk7OMudZ2w=; b=XYYwQnoN+NFssVXY7qDW7r1aRkkg25tQnfiARNrXfqKp6Zsvk5sobOSsPPFYcE8mpM v8mCplCymvf4hlJt4C01bt2gLCWIABmUsbKXd24drl2NAB6d6kcbb12xcCn8cTpYVtoR zaRRxz9XeT3eINwS9t5gOT2Gw08qOjXD1WmPRKrg+YYQqG624rI4Lp9eLRUukspNMyiL 9QYmI5js6ys1N4D4an/f5lQUqh0K4FzvUcC7C3nM2cKSZcutTZwsfSTk4uKAU/MMqOnm 08hvToLUcHWAvF6LSDcr6XSGEyeeG3OCiBYlno+sFpOvt2p06E6BYexboMKNIKd2Ua/b YaWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477845; x=1718082645; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=uI7hPio5EOSCOqhNP+fmg82P5jjcL6y/nOk7OMudZ2w=; b=Mf156JGjEHOWPPbXaryMgVD3HzkIa8iKf33umrJxo+bzmNJsLIdLqgTKW5l/8OCJ7W LJLwSuSkXHlqmEqCjyTI8L9lrMOE+BKI/q53aw3ptMrOByUGIv/xhAkE2SLnKx3gneNK nb0mGAKnZjmAPWhFjDAiXlqdoCM7vEFNUbJxY64Ox7TvFJu7M17bYBiU52JNVhoHVO4m l9eYFenOqaCejSmE/Q7bWQFMsFxa6Elu6e/YJVUm1Gt1AoQu5aMU7kmxYMWkHzBzQr4e Ecid4sBOTI1E3d/if9MCzxD6Tbup9BxqNws30YXq77B3JF/4Xz0AudPNVobu6ISjI4x8 Eypg== X-Forwarded-Encrypted: i=1; AJvYcCXeDUJyACJ7fPm0hDHWH9B7FN80vY2Bdwd2sIOxDwzFqVtkI7hZdFQW+BeoDtQNJ3OF/f+X5fIGR0UUYeAONT9FM409sRmFh+ZD4vKozPOaN8X1L1G3/u27gyGkbEMRhKiNSTeRpjc75JS1G3eFHYfWRXKIVgzXF9s6WfECnJXp9IWYOhgNZ/ZtV6NMHFAQ1IiZywKFFtNr0o4rwKAecWLF95mKbR6Sm0ZwHmoMCnc1yn0G8PwbuTU6eH2MBiE= X-Gm-Message-State: AOJu0YxK9/EaoG2BQKloxegYdzGlXPzeX6fAqe5osPTsyV4/8fOKRJ8M Fj8ulnRvo8VPiEVTJJJjeOrE3nOQRxSEnT10fPOs5Ir1J5T9+qcd X-Google-Smtp-Source: AGHT+IEmnCRnuRNZeIpORES3MssYPt/+L5A69/3IuhurrqBWUQ1ji+4Umw+ygAq+nk7kFKVPAk9Pog== X-Received: by 2002:a05:6a20:8415:b0:1af:7646:fc14 with SMTP id adf61e73a8af0-1b26ef54e19mr11855148637.0.1717477845049; Mon, 03 Jun 2024 22:10:45 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:44 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 07/12] Drivers: hv: vmbus: Set up irqdomain and irqchip for the VMBus connection Date: Mon, 3 Jun 2024 22:09:35 -0700 Message-Id: <20240604050940.859909-8-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley In preparation for assigning Linux IRQs to VMBus channels, set up an irqdomain and irqchip for the VMBus connection. The irqdomain is linear, with the VMBus relid used as the "hwirq" value. A relid is a unique index assigned by Hyper-V to each VMBus channel, with values ranging from 1 to 2047. Because these hwirqs don't map to anything in the architectural hardware, the domain is not part of the domain hierarchy. VMBus channel interrupts provide minimal management functionality, so provide only a minimal set of irqchip functions. The set_affinity function is a place-holder that is populated in a subsequent patch. Signed-off-by: Michael Kelley --- drivers/hv/connection.c | 24 +++++++++----- drivers/hv/hyperv_vmbus.h | 9 +++++ drivers/hv/vmbus_drv.c | 60 +++++++++++++++++++++++++++++++++- include/asm-generic/mshyperv.h | 6 ++++ 4 files changed, 90 insertions(+), 9 deletions(-) diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index f001ae880e1d..cb01784e5c3b 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -21,21 +21,29 @@ #include #include #include +#include +#include +#include #include #include "hyperv_vmbus.h" struct vmbus_connection vmbus_connection = { - .conn_state = DISCONNECTED, - .unload_event = COMPLETION_INITIALIZER( - vmbus_connection.unload_event), - .next_gpadl_handle = ATOMIC_INIT(0xE1E10), - - .ready_for_suspend_event = COMPLETION_INITIALIZER( - vmbus_connection.ready_for_suspend_event), + .conn_state = DISCONNECTED, + .unload_event = COMPLETION_INITIALIZER( + vmbus_connection.unload_event), + .next_gpadl_handle = ATOMIC_INIT(0xE1E10), + + .vmbus_irq_chip.name = "VMBus", + .vmbus_irq_chip.irq_set_affinity = vmbus_irq_set_affinity, + .vmbus_irq_chip.irq_mask = vmbus_irq_mask, + .vmbus_irq_chip.irq_unmask = vmbus_irq_unmask, + + .ready_for_suspend_event = COMPLETION_INITIALIZER( + vmbus_connection.ready_for_suspend_event), .ready_for_resume_event = COMPLETION_INITIALIZER( - vmbus_connection.ready_for_resume_event), + vmbus_connection.ready_for_resume_event), }; EXPORT_SYMBOL_GPL(vmbus_connection); diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h index 76ac5185a01a..95d4d47d34f7 100644 --- a/drivers/hv/hyperv_vmbus.h +++ b/drivers/hv/hyperv_vmbus.h @@ -18,7 +18,11 @@ #include #include #include +#include #include +#include +#include +#include #include "hv_trace.h" @@ -258,6 +262,11 @@ struct vmbus_connection { /* Array of channels */ struct vmbus_channel **channels; + /* IRQ domain data */ + struct fwnode_handle *vmbus_fwnode; + struct irq_domain *vmbus_irq_domain; + struct irq_chip vmbus_irq_chip; + /* * An offer message is handled first on the work_queue, and then * is further handled on handle_primary_chan_wq or diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 291a8358370b..cbccdfad49a2 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -36,6 +36,9 @@ #include #include #include +#include +#include +#include #include #include #include "hyperv_vmbus.h" @@ -1306,6 +1309,40 @@ static irqreturn_t vmbus_percpu_isr(int irq, void *dev_id) return IRQ_HANDLED; } +int vmbus_irq_set_affinity(struct irq_data *data, + const struct cpumask *dest, bool force) +{ + return 0; +} + +/* + * VMBus channel interrupts do not need to be masked or unmasked, and the + * Hyper-V synic doesn't provide any masking functionality anyway. But in the + * absence of these irqchip functions, the IRQ subsystem keeps the IRQ marked + * as "masked". To prevent any problems associated with staying the "masked" + * state, and so that IRQ status shown in debugfs doesn't indicate "masked", + * provide null implementations. + */ +void vmbus_irq_unmask(struct irq_data *data) +{ +} + +void vmbus_irq_mask(struct irq_data *data) +{ +} + +static int vmbus_irq_map(struct irq_domain *d, unsigned int irq, + irq_hw_number_t hw) +{ + irq_set_chip_and_handler(irq, + &vmbus_connection.vmbus_irq_chip, handle_simple_irq); + return 0; +} + +static const struct irq_domain_ops vmbus_domain_ops = { + .map = vmbus_irq_map, +}; + /* * vmbus_bus_init -Main vmbus driver initialization routine. * @@ -1340,6 +1377,7 @@ static int vmbus_bus_init(void) if (vmbus_irq == -1) { hv_setup_vmbus_handler(vmbus_isr); } else { + irq_set_handler(vmbus_irq, handle_percpu_demux_irq); vmbus_evt = alloc_percpu(long); ret = request_percpu_irq(vmbus_irq, vmbus_percpu_isr, "Hyper-V VMbus", vmbus_evt); @@ -1355,6 +1393,20 @@ static int vmbus_bus_init(void) if (ret) goto err_alloc; + /* Create IRQ domain for VMBus devices */ + vmbus_connection.vmbus_fwnode = irq_domain_alloc_named_fwnode("hv-vmbus"); + if (!vmbus_connection.vmbus_fwnode) { + ret = -ENOMEM; + goto err_alloc; + } + vmbus_connection.vmbus_irq_domain = irq_domain_create_linear( + vmbus_connection.vmbus_fwnode, MAX_CHANNEL_RELIDS, + &vmbus_domain_ops, NULL); + if (!vmbus_connection.vmbus_irq_domain) { + ret = -ENOMEM; + goto err_fwnode; + } + /* * Initialize the per-cpu interrupt state and stimer state. * Then connect to the host. @@ -1362,7 +1414,7 @@ static int vmbus_bus_init(void) ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "hyperv/vmbus:online", hv_synic_init, hv_synic_cleanup); if (ret < 0) - goto err_alloc; + goto err_domain; hyperv_cpuhp_online = ret; ret = vmbus_connect(); @@ -1382,6 +1434,10 @@ static int vmbus_bus_init(void) err_connect: cpuhp_remove_state(hyperv_cpuhp_online); +err_domain: + irq_domain_remove(vmbus_connection.vmbus_irq_domain); +err_fwnode: + irq_domain_free_fwnode(vmbus_connection.vmbus_fwnode); err_alloc: hv_synic_free(); if (vmbus_irq == -1) { @@ -2690,6 +2746,8 @@ static void __exit vmbus_exit(void) hv_debug_rm_all_dir(); vmbus_free_channels(); + irq_domain_remove(vmbus_connection.vmbus_irq_domain); + irq_domain_free_fwnode(vmbus_connection.vmbus_fwnode); kfree(vmbus_connection.channels); /* diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 8fe7aaab2599..0488ff8b511f 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -187,6 +188,11 @@ void hv_remove_kexec_handler(void); void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)); void hv_remove_crash_handler(void); +extern void vmbus_irq_mask(struct irq_data *data); +extern void vmbus_irq_unmask(struct irq_data *data); +extern int vmbus_irq_set_affinity(struct irq_data *data, + const struct cpumask *dest, bool force); + extern int vmbus_interrupt; extern int vmbus_irq; From patchwork Tue Jun 4 05:09:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801555 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B763142629; Tue, 4 Jun 2024 05:10:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477850; cv=none; b=EqXGxC0gXq2SiXvje+nGXKbOdLB7E7eXZUHOFns9qipm55m4hys7vdKWUZC0sJIVSRN27EdklySPWZAtOpnQtuMqD/FdM3E6EaB7zHErmjAUodzejRO34Q6Ux+u2KpJTTpUYFIC+nLlIDKEayTb2THvTRq0ZrdfzkHNNDCmzuYE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477850; c=relaxed/simple; bh=Dtv+8BHewlDtrscRUC9E1bIoWTmplqu0ARBJT2pimYM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iD/qWg58Pmzgjy73tgZ5jHGdcStZ21Iud2TyUwt0iH6kEyWlHP0ehzT3BtHtuOOYyXzJDP5wCkvAU1Pm9qD+Fqt2rD/LCmFt9xKWyw1CiQtNYMPodMxM2kWIhNqAZiXwoFaDRQd6WWuaMjfc1XVQS+1sDOaFAGTV+CVN6yvRuA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cGxsXeiW; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cGxsXeiW" Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-7025cb09553so2191891b3a.3; Mon, 03 Jun 2024 22:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477848; x=1718082648; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=X+SPYtiC+eKOOluq0xEXtbfDYX7+i/A8vs71JSRv2/w=; b=cGxsXeiWR9IUNANgy10u9SpNbdv7smq9YxQpMQlD2ws4Gs+doIyxU2vCqS5WN1qzEn u5v1mzM2Li3Jks+3HC/Uy1pphm9Z5kk6Af0f1a2jxiya8VPPcv1+TL+TwzibdQy0FRiE gM3XvNEOoN3aazBEClnbCwvUFSfVLs2wZ82NeS56SwVbWJd13xh9yd9G9xalGe8uxDKj 8LsZMJ8AIDYU6kKyZOM9PJsF551AhCnQIZdIogsGE4aA6hugr6cM/7U3Lqo93BTEL/K7 ivSQPTi8AD2+3F53vcj8fYb1FdaG5xJxBvxrvnJIGzHG/59UI+aAVHzFFurZM0AIsDxI M1GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477848; x=1718082648; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=X+SPYtiC+eKOOluq0xEXtbfDYX7+i/A8vs71JSRv2/w=; b=iLHFT9Z92Qu0EII8dClG4m3siZMVQOXytfboIWpp5lHVTOjgSt0FTlnLs/H/KUfYgR XlkdXtL2mMl3dW2RFMc6c3oIfCvUbLIf5Op7MejMed6x3xW7xkgUy64TnhuVMVidmyDD IKL9BVVJfGgpXOgdPLfNlmzxoSQ8G5qxsWQsM0J2EHYjwQbJ0ieruH5rlNmTTo8k2kur 9sTb0v4T4Bj5bepjltU8mEryuPXBeYO2gIN2C7lx+ffvT4JKaEcOi0+KL4GNCCslEU+Y SzlYslcVQ1trT/DkENgEH9xHnN13/yQkAarnwQC35FBRnSEoNsIVP5Q6eeW9TrPPFLiw kTZQ== X-Forwarded-Encrypted: i=1; AJvYcCUKexbd54Am8PgtEQf1mzk21yOU/wjwr3QIJYBHmF7Dic4RL9NqL3ifStGhmHqiLV2nPqQE1P4B/aW/sAebInF0bGsD6BibVlStB+Fl7Qn6nXNLPuawO9PMaSmbcWQg7uRnP1BHswrbYo/QrG9LdtB3ZVVpmnFYLlV8GWv2ou7uWw6CP+/y6iACMEVPWsR0hJo8Ws24o5GSSStGX38wm6kBbi1lHz88D/6vYgeZ6dATYZZuQVprKUBDin3kVH8= X-Gm-Message-State: AOJu0YxVu9QtuMNGLq7XqhHmTYwZWDom2M3W/xVJuPWC4QAdmhaC/P8+ oqk/tguDNYaHfTywvsLpSm7zsN9emnCEZzpB/UptfyJ3I02w7JvV X-Google-Smtp-Source: AGHT+IF/QjqqduNmy9R3nfgGphf2kfowTvEQN0egF5f68NiQrvZUbfJu9Gb0OBfN+Uknx2sDslo5gg== X-Received: by 2002:a05:6a00:39a6:b0:6f4:4319:4cea with SMTP id d2e1a72fcca58-702478da697mr15516163b3a.33.1717477847750; Mon, 03 Jun 2024 22:10:47 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:47 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 09/12] Drivers: hv: vmbus: Use Linux IRQs to handle VMBus channel interrupts Date: Mon, 3 Jun 2024 22:09:37 -0700 Message-Id: <20240604050940.859909-10-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley Do the following: 1) Create an interrupt handler for VMBus channel interrupts by pulling out portions of vmbus_chan_sched() into vmbus_chan_handler(). The outer part of vmbus_chan_sched() that loops through the synic event page bitmap remains unchanged. But when a pending VMBus channel interrupt is found, call generic_handle_irq_desc() to invoke handle_simple_irq() and then vmbus_chan_handler() for the channel's IRQ. handle_simple_irq() does the IRQ stats for that channel's IRQ, so that per-channel interrupt counts appear in /proc/interrupts. The overall processing of VMBus channel interrupts is unchanged except for the intervening handle_simple_irq() that does the stats. No acks or EOIs are required for VMBus channel IRQs. 2) Update __vmbus_open() to call request_irq(), specifying the previously setup channel IRQ name and vmbus_chan_handler() as the interrupt handler. Set the IRQ affinity to the target_cpu assigned when the channel was created. 3) Update vmbus_isr() to return "false" if it only handles VMBus interrupts, which were passed to the channel IRQ handler. If vmbus_isr() handles one or more control message interrupts, then return "true". Update the related definitions to specify a boolean return value. 4) The callers of vmbus_isr() increment IRQ stats for the top-level IRQ only if "true" is returned. On x86, the caller is sysvec_hyperv_callback(), which manages the stats directly. On arm64, the caller is vmbus_percpu_isr(), which maps the boolean return value to IRQ_NONE ("false") or IRQ_HANDLED ("true"). Then handle_percpu_demux_irq() conditionally updates the stats based on the return value from vmbus_percpu_isr(). With these changes, interrupts from VMBus channels are now processed as Linux IRQs that are demultiplexed from the main VMBus interrupt. Signed-off-by: Michael Kelley --- arch/x86/kernel/cpu/mshyperv.c | 9 ++-- drivers/hv/channel.c | 25 +++++++++- drivers/hv/hv_common.c | 2 +- drivers/hv/vmbus_drv.c | 84 +++++++++++++++++++--------------- include/asm-generic/mshyperv.h | 3 +- 5 files changed, 79 insertions(+), 44 deletions(-) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index e0fd57a8ba84..18bc282a99db 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -110,7 +110,7 @@ void hv_set_msr(unsigned int reg, u64 value) } EXPORT_SYMBOL_GPL(hv_set_msr); -static void (*vmbus_handler)(void); +static bool (*vmbus_handler)(void); static void (*hv_stimer0_handler)(void); static void (*hv_kexec_handler)(void); static void (*hv_crash_handler)(struct pt_regs *regs); @@ -119,9 +119,8 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback) { struct pt_regs *old_regs = set_irq_regs(regs); - inc_irq_stat(irq_hv_callback_count); - if (vmbus_handler) - vmbus_handler(); + if (vmbus_handler && vmbus_handler()) + inc_irq_stat(irq_hv_callback_count); if (ms_hyperv.hints & HV_DEPRECATING_AEOI_RECOMMENDED) apic_eoi(); @@ -129,7 +128,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_hyperv_callback) set_irq_regs(old_regs); } -void hv_setup_vmbus_handler(void (*handler)(void)) +void hv_setup_vmbus_handler(bool (*handler)(void)) { vmbus_handler = handler; } diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c index fb8cd8469328..1aa020b538f1 100644 --- a/drivers/hv/channel.c +++ b/drivers/hv/channel.c @@ -638,6 +638,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel, struct vmbus_channel_open_channel *open_msg; struct vmbus_channel_msginfo *open_info = NULL; struct page *page = newchannel->ringbuffer_page; + u32 relid = newchannel->offermsg.child_relid; u32 send_pages, recv_pages; unsigned long flags; int err; @@ -685,13 +686,31 @@ static int __vmbus_open(struct vmbus_channel *newchannel, if (err) goto error_free_gpadl; + /* Request the IRQ and assign to target_cpu */ + err = request_irq(newchannel->irq, vmbus_chan_handler, 0, + newchannel->irq_name, newchannel); + if (err) { + pr_err("request_irq failed with %d for relid %d irq %d\n", + err, relid, newchannel->irq); + goto error_free_gpadl; + } + err = irq_set_affinity_and_hint(newchannel->irq, + cpumask_of(newchannel->target_cpu)); + if (err) { + pr_err("irq_set_affinity_and_hint failed with %d for relid %d irq %d\n", + err, relid, newchannel->irq); + free_irq(newchannel->irq, newchannel); + goto error_free_gpadl; + } + newchannel->irq_requested = true; + /* Create and init the channel open message */ open_info = kzalloc(sizeof(*open_info) + sizeof(struct vmbus_channel_open_channel), GFP_KERNEL); if (!open_info) { err = -ENOMEM; - goto error_free_gpadl; + goto error_free_irq; } init_completion(&open_info->waitevent); @@ -759,6 +778,10 @@ static int __vmbus_open(struct vmbus_channel *newchannel, spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags); error_free_info: kfree(open_info); +error_free_irq: + irq_update_affinity_hint(newchannel->irq, NULL); + free_irq(newchannel->irq, newchannel); + newchannel->irq_requested = false; error_free_gpadl: vmbus_teardown_gpadl(newchannel, &newchannel->ringbuffer_gpadlhandle); error_clean_ring: diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 9c452bfbd571..38a23add721c 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -610,7 +610,7 @@ bool __weak hv_isolation_type_tdx(void) } EXPORT_SYMBOL_GPL(hv_isolation_type_tdx); -void __weak hv_setup_vmbus_handler(void (*handler)(void)) +void __weak hv_setup_vmbus_handler(bool (*handler)(void)) { } EXPORT_SYMBOL_GPL(hv_setup_vmbus_handler); diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index 8fd03d41e71a..b73be7c02d37 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -1193,6 +1193,45 @@ static void vmbus_force_channel_rescinded(struct vmbus_channel *channel) } #endif /* CONFIG_PM_SLEEP */ +irqreturn_t vmbus_chan_handler(int irq, void *dev_id) +{ + void (*callback_fn)(void *context); + struct vmbus_channel *channel = dev_id; + + /* + * Make sure that the ring buffer data structure doesn't get + * freed while we dereference the ring buffer pointer. Test + * for the channel's onchannel_callback being NULL within a + * sched_lock critical section. See also the inline comments + * in vmbus_reset_channel_cb(). + */ + spin_lock(&channel->sched_lock); + + callback_fn = channel->onchannel_callback; + if (unlikely(callback_fn == NULL)) + goto spin_unlock; + + trace_vmbus_chan_sched(channel); + + ++channel->interrupts; + + switch (channel->callback_mode) { + case HV_CALL_ISR: + (*callback_fn)(channel->channel_callback_context); + break; + + case HV_CALL_BATCHED: + hv_begin_read(&channel->inbound); + fallthrough; + case HV_CALL_DIRECT: + tasklet_schedule(&channel->callback_event); + } + +spin_unlock: + spin_unlock(&channel->sched_lock); + return IRQ_HANDLED; +} + /* * Schedule all channels with events pending */ @@ -1217,7 +1256,6 @@ static void vmbus_chan_sched(struct hv_per_cpu_context *hv_cpu) return; for_each_set_bit(relid, recv_int_page, maxbits) { - void (*callback_fn)(void *context); struct vmbus_channel *channel; struct irq_desc *desc; @@ -1244,43 +1282,14 @@ static void vmbus_chan_sched(struct hv_per_cpu_context *hv_cpu) if (channel->rescind) goto sched_unlock_rcu; - /* - * Make sure that the ring buffer data structure doesn't get - * freed while we dereference the ring buffer pointer. Test - * for the channel's onchannel_callback being NULL within a - * sched_lock critical section. See also the inline comments - * in vmbus_reset_channel_cb(). - */ - spin_lock(&channel->sched_lock); - - callback_fn = channel->onchannel_callback; - if (unlikely(callback_fn == NULL)) - goto sched_unlock; - - trace_vmbus_chan_sched(channel); - - ++channel->interrupts; - - switch (channel->callback_mode) { - case HV_CALL_ISR: - (*callback_fn)(channel->channel_callback_context); - break; - - case HV_CALL_BATCHED: - hv_begin_read(&channel->inbound); - fallthrough; - case HV_CALL_DIRECT: - tasklet_schedule(&channel->callback_event); - } + generic_handle_irq_desc(desc); -sched_unlock: - spin_unlock(&channel->sched_lock); sched_unlock_rcu: rcu_read_unlock(); } } -static void vmbus_isr(void) +static bool vmbus_isr(void) { struct hv_per_cpu_context *hv_cpu = this_cpu_ptr(hv_context.cpu_context); @@ -1299,15 +1308,18 @@ static void vmbus_isr(void) vmbus_signal_eom(msg, HVMSG_TIMER_EXPIRED); } else tasklet_schedule(&hv_cpu->msg_dpc); - } - add_interrupt_randomness(vmbus_interrupt); + add_interrupt_randomness(vmbus_interrupt); + return true; + } + return false; } static irqreturn_t vmbus_percpu_isr(int irq, void *dev_id) { - vmbus_isr(); - return IRQ_HANDLED; + if (vmbus_isr()) + return IRQ_HANDLED; + return IRQ_NONE; } int vmbus_irq_set_affinity(struct irq_data *data, diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index 0488ff8b511f..0a5559b9d5f7 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -178,7 +178,7 @@ static inline void vmbus_signal_eom(struct hv_message *msg, u32 old_msg_type) int hv_get_hypervisor_version(union hv_hypervisor_version_info *info); -void hv_setup_vmbus_handler(void (*handler)(void)); +void hv_setup_vmbus_handler(bool (*handler)(void)); void hv_remove_vmbus_handler(void); void hv_setup_stimer0_handler(void (*handler)(void)); void hv_remove_stimer0_handler(void); @@ -188,6 +188,7 @@ void hv_remove_kexec_handler(void); void hv_setup_crash_handler(void (*handler)(struct pt_regs *regs)); void hv_remove_crash_handler(void); +extern irqreturn_t vmbus_chan_handler(int irq, void *dev_id); extern void vmbus_irq_mask(struct irq_data *data); extern void vmbus_irq_unmask(struct irq_data *data); extern int vmbus_irq_set_affinity(struct irq_data *data, From patchwork Tue Jun 4 05:09:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mhkelley58@gmail.com X-Patchwork-Id: 801554 Received: from mail-oo1-f50.google.com (mail-oo1-f50.google.com [209.85.161.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A5F714291B; Tue, 4 Jun 2024 05:10:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477853; cv=none; b=ZNcHb10xtJjNs4NB4vOeqqUC4Z2BMRqEqrJUA7hPcDH6Ly3qhizI/J4RDOpWG5p5zD/fOt2UjOfz+jFGWqqo7+s0CUxf9frPePTQYGS0EzzStd5Pm0ALbrcdsTXuT6R2VKj8KgZjkoQ7Qp+2/HA4Z0eyIwnZ0UzXvDG3QkDcE8k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717477853; c=relaxed/simple; bh=1jVA1hbkuJFk2X1Wq2r6QErsU8Q1jsab7e/GaD/Uy9A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u6eU6d/srp+MUTTnikiNGGfoDkv7FXUZ7/H4/IbWG8iWF431PTf7bO0MjF5FHKw2TOmUpAzKys9lUnTmTAE2WC3v6SePBjOf0QH+vK71tlSMdRUsxPD3bMyTSRtbGft2naVv+UjWcvrGYK05INV9t8He6uNdvNJt99xj0QNC7ds= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kGuexhTD; arc=none smtp.client-ip=209.85.161.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kGuexhTD" Received: by mail-oo1-f50.google.com with SMTP id 006d021491bc7-5b970a97e8eso364474eaf.1; Mon, 03 Jun 2024 22:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717477850; x=1718082650; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=pVV6Kv0tGk5nmsV3DBWEtw/NPqCxYSXN7XTZ22q11dg=; b=kGuexhTDqFeWTDafOqg1qOhkm5DYBi+TIpnFpS9UgUyFz3jEt7tgd7RhDKpngD35+7 B5OOqs7BsKkCgj24e4ZcwRLeaQPBbP8zvbJB53/mFirRvbYq5Ah05rNgAb7UlNAVC0h/ cZbagt/2bcZ69K4ezI1b9vtQuwLewtfKoP5BA2JGKq4Bq4SXUbQb62w/ENAos3uUV3Cu Z7uh3egA0tokhelxHjEbgFfi9Vh1vtVlsVTJNkLw5xl6jOv9aF8d+XQ5cJVOXf/1LwEc dNMR3OxsX7TJXlj/GFGFR8b1U/zd+1ZVOYBfV7ZaxQwSjsXnJHzdETuNZIdmXyKNd3cZ Oo2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717477850; x=1718082650; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pVV6Kv0tGk5nmsV3DBWEtw/NPqCxYSXN7XTZ22q11dg=; b=jLd1sBne9UX1vW7mJeE5JWnraJPw2lVexNVC8GwnUiOiSraOkx3taz8qVxaKaxrFvz gVhFm7F7hCehOx1P5oHie4GRkNxRrwA7IfTkU1/FLiqqT3Jvx3oEPdDzHW3EhtnMi4E8 sLaWPLPHlwErjT1Gb1kkF00U4Q5Z46XFeg0yHB0HmfoX1C9BGLstGwBHKJX3vs3eKnqc kZ2pRKIy02EOiwALvX1n6nbzUpFEmGlyPc9/E3B5eO02t+pawD29ys06yc1HThmHrd5w DkscbXQrPG19zSRS7WU6NlwkyQXN8ooRoa6ZZH0WXn/m5XvJwLdLXaNQygNkRiPoFzWr 2ETw== X-Forwarded-Encrypted: i=1; AJvYcCX3Wl+oyj1DD0852BwA32pLa/rMem68QB5xMnGVSRnkch+iu3hnNORjVI0WS0lDNwZXOghB2VDjRWLMepzmIBrZFPAu5c6TB3QdbYBmr0+3PqMMdWflO54LWJnwVWq0ID65xHEh2VPMnfTeyr7nexWP/+qUjRAMjvd5pJqdz+Lx6PYJdHfSAtPawF+r9Lbb0aH2MA6I4/TAREqsHRKab6xqOk63DtA7wyqTp3BGJUOPN2t5XLP1mA4rcNDHolQ= X-Gm-Message-State: AOJu0YwrIUcKciZisCei0bTuYpBxDNJ0IdN/MZd/LTOt8lMd4i6kGq3N tkGGBNX10bqHO6RcjpkV0JsVS2zZ1u4VmAmtwcA0pUI7G8JBcIf2 X-Google-Smtp-Source: AGHT+IFF4pZiPOoszBF2RPe9oAaHaW1lz4eLkIWpJh01f0tjODzp1ogdUGgeuXzlAkHajFkd9k+xLA== X-Received: by 2002:a05:6820:1c91:b0:5b5:1f77:9603 with SMTP id 006d021491bc7-5ba05dc205cmr11327808eaf.8.1717477850379; Mon, 03 Jun 2024 22:10:50 -0700 (PDT) Received: from localhost.localdomain (c-67-161-114-176.hsd1.wa.comcast.net. [67.161.114.176]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70242c270c7sm6298153b3a.220.2024.06.03.22.10.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 22:10:50 -0700 (PDT) From: mhkelley58@gmail.com X-Google-Original-From: mhklinux@outlook.com To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, lpieralisi@kernel.org, kw@linux.com, robh@kernel.org, bhelgaas@google.com, James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com, arnd@arndb.de, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-scsi@vger.kernel.org, linux-arch@vger.kernel.org Cc: maz@kernel.org, den@valinux.co.jp, jgowans@amazon.com, dawei.li@shingroup.cn Subject: [RFC 11/12] Drivers: hv: vmbus: Wait for MODIFYCHANNEL to finish when offlining CPUs Date: Mon, 3 Jun 2024 22:09:39 -0700 Message-Id: <20240604050940.859909-12-mhklinux@outlook.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240604050940.859909-1-mhklinux@outlook.com> References: <20240604050940.859909-1-mhklinux@outlook.com> Reply-To: mhklinux@outlook.com Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Kelley vmbus_irq_set_affinity() may issue a MODIFYCHANNEL request to Hyper-V to target a VMBus channel interrupt to a different CPU. While newer versions of Hyper-V send a response to the guest when the change is complete, vmbus_irq_set_affinity() does not wait for the response because it is running with interrupts disabled. So Hyper-V may continue to direct interrupts to the old CPU for a short window after vmbus_irq_set_affinity() completes. This lag is not a problem during normal operation. But if the old CPU is taken offline during that window, Hyper-V may drop the interrupt because the synic in the target CPU is disabled. Dropping the interrupt may cause the VMBus channel to hang. To prevent this, wait for in-process MODIFYCHANNEL requests when taking a CPU offline. On newer versions of Hyper-V, completion can be confirmed by waiting for the response sent by Hyper-V. But on older versions of Hyper-V that don't send a response, wait a fixed interval of time that empirically should be "long enough", as that's the best that can be done. Signed-off-by: Michael Kelley --- drivers/hv/channel.c | 3 ++ drivers/hv/channel_mgmt.c | 32 ++++-------------- drivers/hv/hv.c | 70 +++++++++++++++++++++++++++++++++++---- drivers/hv/hyperv_vmbus.h | 8 +++++ 4 files changed, 81 insertions(+), 32 deletions(-) diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c index b7920072e243..b7ee95373049 100644 --- a/drivers/hv/channel.c +++ b/drivers/hv/channel.c @@ -246,6 +246,9 @@ int vmbus_send_modifychannel(struct vmbus_channel *channel, u32 target_vp) ret = vmbus_post_msg(&msg, sizeof(msg), false); trace_vmbus_send_modifychannel(&msg, ret); + if (!ret) + vmbus_connection.modchan_sent++; + return ret; } EXPORT_SYMBOL_GPL(vmbus_send_modifychannel); diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c index da42aaae994e..960a2f0367d8 100644 --- a/drivers/hv/channel_mgmt.c +++ b/drivers/hv/channel_mgmt.c @@ -1465,40 +1465,20 @@ static void vmbus_ongpadl_created(struct vmbus_channel_message_header *hdr) * vmbus_onmodifychannel_response - Modify Channel response handler. * * This is invoked when we received a response to our channel modify request. - * Find the matching request, copy the response and signal the requesting thread. + * Increment the count of responses received. No locking is needed because + * responses are always received on the VMBUS_CONNECT_CPU. */ static void vmbus_onmodifychannel_response(struct vmbus_channel_message_header *hdr) { struct vmbus_channel_modifychannel_response *response; - struct vmbus_channel_msginfo *msginfo; - unsigned long flags; response = (struct vmbus_channel_modifychannel_response *)hdr; + if (response->status) + pr_err("Error status %x in MODIFYCHANNEL response for relid %d\n", + response->status, response->child_relid); + vmbus_connection.modchan_completed++; trace_vmbus_onmodifychannel_response(response); - - /* - * Find the modify msg, copy the response and signal/unblock the wait event. - */ - spin_lock_irqsave(&vmbus_connection.channelmsg_lock, flags); - - list_for_each_entry(msginfo, &vmbus_connection.chn_msg_list, msglistentry) { - struct vmbus_channel_message_header *responseheader = - (struct vmbus_channel_message_header *)msginfo->msg; - - if (responseheader->msgtype == CHANNELMSG_MODIFYCHANNEL) { - struct vmbus_channel_modifychannel *modifymsg; - - modifymsg = (struct vmbus_channel_modifychannel *)msginfo->msg; - if (modifymsg->child_relid == response->child_relid) { - memcpy(&msginfo->response.modify_response, response, - sizeof(*response)); - complete(&msginfo->waitevent); - break; - } - } - } - spin_unlock_irqrestore(&vmbus_connection.channelmsg_lock, flags); } /* diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index a8ad728354cb..76658dfc5008 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -401,6 +401,56 @@ void hv_synic_disable_regs(unsigned int cpu) disable_percpu_irq(vmbus_irq); } +static void hv_synic_wait_for_modifychannel(int cpu) +{ + int i = 5; + u64 base; + + /* + * If we're on a VMBus version where MODIFYCHANNEL doesn't send acks, + * just sleep for 20 milliseconds and hope that gives Hyper-V enough + * time to process them. Empirical data on recent server-class CPUs + * (both x86 and arm64) shows that the Hyper-V response is typically + * received and processed in the guest within a few hundred + * microseconds. The 20 millisecond wait is somewhat arbitrary and + * intended to give plenty to time in case there are multiple + * MODIFYCHANNEL requests in progress and the host is busy. It's + * the best we can do. + */ + if (vmbus_proto_version < VERSION_WIN10_V5_3) { + usleep_range(20000, 25000); + return; + } + + /* + * Otherwise compare the current value of modchan_completed against + * modchan_sent. If some MODIFYCHANNEL requests have been sent that + * haven't completed, sleep 5 milliseconds and check again. If the + * requests still haven't completed after 5 attempts, output a + * message and proceed anyway. + * + * Hyper-V guarantees to process MODIFYCHANNEL requests in the order + * they are received from the guest, so simply comparing the counts + * is sufficient. + * + * Note that this check may encompass MODIFYCHANNEL requests that are + * unrelated to the CPU that is going offline. But the only effect is + * to potentially wait a little bit longer than necessary. CPUs going + * offline and affinity changes that result in MODIFYCHANNEL are + * relatively rare and it's not worth the complexity to track them more + * precisely. + */ + base = READ_ONCE(vmbus_connection.modchan_sent); + while (READ_ONCE(vmbus_connection.modchan_completed) < base && i) { + usleep_range(5000, 10000); + i--; + } + + if (i == 0) + pr_err("Timed out waiting for MODIFYCHANNEL. CPU %d sent %lld completed %lld\n", + cpu, base, vmbus_connection.modchan_completed); +} + #define HV_MAX_TRIES 3 /* * Scan the event flags page of 'this' CPU looking for any bit that is set. If we find one @@ -485,13 +535,21 @@ int hv_synic_cleanup(unsigned int cpu) /* * channel_found == false means that any channels that were previously * assigned to the CPU have been reassigned elsewhere with a call of - * vmbus_send_modifychannel(). Scan the event flags page looking for - * bits that are set and waiting with a timeout for vmbus_chan_sched() - * to process such bits. If bits are still set after this operation - * and VMBus is connected, fail the CPU offlining operation. + * vmbus_send_modifychannel(). First wait until any MODIFYCHANNEL + * requests have been completed by Hyper-V, after which we know that + * no new bits in the event flags will be set. Then scan the event flags + * page looking for bits that are set and waiting with a timeout for + * vmbus_chan_sched() to process such bits. If bits are still set + * after this operation, fail the CPU offlining operation. */ - if (vmbus_proto_version >= VERSION_WIN10_V4_1 && hv_synic_event_pending()) - return -EBUSY; + if (vmbus_proto_version >= VERSION_WIN10_V4_1) { + hv_synic_wait_for_modifychannel(cpu); + if (hv_synic_event_pending()) { + pr_err("Events pending when trying to offline CPU %d\n", + cpu); + return -EBUSY; + } + } always_cleanup: hv_stimer_legacy_cleanup(cpu); diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h index bf35bb40c55e..571b2955b38e 100644 --- a/drivers/hv/hyperv_vmbus.h +++ b/drivers/hv/hyperv_vmbus.h @@ -264,6 +264,14 @@ struct vmbus_connection { struct irq_domain *vmbus_irq_domain; struct irq_chip vmbus_irq_chip; + /* + * VM-wide counts of MODIFYCHANNEL messages sent and completed. + * Used when taking a CPU offline to make sure the relevant + * MODIFYCHANNEL messages have been completed. + */ + u64 modchan_sent; + u64 modchan_completed; + /* * An offer message is handled first on the work_queue, and then * is further handled on handle_primary_chan_wq or