From patchwork Thu Apr 23 18:39:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 283992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BE89C54FCB for ; Thu, 23 Apr 2020 18:42:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 112D6205C9 for ; Thu, 23 Apr 2020 18:42:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="YiqtKcBH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 112D6205C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgnd-0003Ps-4k for qemu-devel@archiver.kernel.org; Thu, 23 Apr 2020 14:42:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32868) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgm6-0001FF-RO for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jRgm6-0004KE-FK for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:38 -0400 Received: from forwardcorp1p.mail.yandex.net ([77.88.29.217]:45238) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jRgm2-00048m-TR; Thu, 23 Apr 2020 14:40:36 -0400 Received: from mxbackcorp1g.mail.yandex.net (mxbackcorp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::301]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 65C3C2E150C; Thu, 23 Apr 2020 21:40:32 +0300 (MSK) Received: from vla5-58875c36c028.qloud-c.yandex.net (vla5-58875c36c028.qloud-c.yandex.net [2a02:6b8:c18:340b:0:640:5887:5c36]) by mxbackcorp1g.mail.yandex.net (mxbackcorp/Yandex) with ESMTP id GEhWYPOk17-eUQmj5PB; Thu, 23 Apr 2020 21:40:32 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1587667232; bh=6AUSdMRun+4uomw8GjlwQJKUpFkMPJnEm3m4a1og08I=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=YiqtKcBHfLkbWDYhZ0Vcu+Ouj4kWCtgKHPx5Q2WwYgmVXB6mef08ag3YYDDzdlG63 2Z/PaAXBnSbhxgkqPrtC2hXs3Tv382XV4PAshynUzX1kbqhklJFRFCGqTWns+C6c78 fjaBdB3LtScf2B/NeHdQHtIl7tsKD5mlb6l5gPeA= Authentication-Results: mxbackcorp1g.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:1313::1:e]) by vla5-58875c36c028.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id 7hClKENKMK-eUXeJJgf; Thu, 23 Apr 2020 21:40:30 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [RFC PATCH v1 3/7] char-socket: initialize reconnect timer only if close is emitted Date: Thu, 23 Apr 2020 21:39:34 +0300 Message-Id: <23b36a73ce1150cc501f436684ca558608de3322.1587667007.git.dimastep@yandex-team.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=77.88.29.217; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/23 14:40:32 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 77.88.29.217 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, yc-core@yandex-team.ru, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, arei.gonglei@huawei.com, stefanha@redhat.com, marcandre.lureau@redhat.com, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" During vhost-user reconnect functionality testing the following assert was hit: qemu-system-x86_64: chardev/char-socket.c:125: qemu_chr_socket_restart_timer: Assertion `!s->reconnect_timer' failed. Aborted (core dumped) This is observed only if the connection is closed by the vhost-user-blk daemon during the initialization routine. In this case the tcp_chr_disconnect_locked() routine is called twice. First time it is called in the tcp_chr_write() routine, after getting the SIGPIPE signal. Second time it is called when vhost_user_blk_connect() routine return error. In general it looks correct, because the initialization routine can return error in many cases. The tcp_chr_disconnect_locked() routine could be fixed. The timer will be restarted only if the close event is emitted. Signed-off-by: Dima Stepanov --- chardev/char-socket.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/chardev/char-socket.c b/chardev/char-socket.c index c128cca..83ca4d9 100644 --- a/chardev/char-socket.c +++ b/chardev/char-socket.c @@ -476,7 +476,7 @@ static void update_disconnected_filename(SocketChardev *s) static void tcp_chr_disconnect_locked(Chardev *chr) { SocketChardev *s = SOCKET_CHARDEV(chr); - bool emit_close = s->state == TCP_CHARDEV_STATE_CONNECTED; + bool was_connected = s->state == TCP_CHARDEV_STATE_CONNECTED; tcp_chr_free_connection(chr); @@ -485,11 +485,11 @@ static void tcp_chr_disconnect_locked(Chardev *chr) chr, NULL, chr->gcontext); } update_disconnected_filename(s); - if (emit_close) { + if (was_connected) { qemu_chr_be_event(chr, CHR_EVENT_CLOSED); - } - if (s->reconnect_time) { - qemu_chr_socket_restart_timer(chr); + if (s->reconnect_time) { + qemu_chr_socket_restart_timer(chr); + } } } From patchwork Thu Apr 23 18:39:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 283991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D98BC54FD0 for ; Thu, 23 Apr 2020 18:44:27 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14AEC20704 for ; Thu, 23 Apr 2020 18:44:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="yYbBu7FW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14AEC20704 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgpm-0007HK-7L for qemu-devel@archiver.kernel.org; Thu, 23 Apr 2020 14:44:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32902) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgmB-0001Hq-2O for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jRgm9-0004W6-Vv for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:42 -0400 Received: from forwardcorp1p.mail.yandex.net ([2a02:6b8:0:1472:2741:0:8b6:217]:46980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jRgm6-0004EK-Hx; Thu, 23 Apr 2020 14:40:38 -0400 Received: from mxbackcorp2j.mail.yandex.net (mxbackcorp2j.mail.yandex.net [IPv6:2a02:6b8:0:1619::119]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 46DE32E150B; Thu, 23 Apr 2020 21:40:36 +0300 (MSK) Received: from vla5-58875c36c028.qloud-c.yandex.net (vla5-58875c36c028.qloud-c.yandex.net [2a02:6b8:c18:340b:0:640:5887:5c36]) by mxbackcorp2j.mail.yandex.net (mxbackcorp/Yandex) with ESMTP id WNXL47K6ek-eXHeXm5I; Thu, 23 Apr 2020 21:40:36 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1587667236; bh=/03BgTwU/psEMUgiBdmqNYh7VlIfn2pdgG96rmbifE4=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=yYbBu7FWt+zJ+GVyTZuw4ERXVRJm6Bxpcz84NOQaz5eu9S451HXTVJKPYNxoOez19 Me7mjeDqe9tq7XXBzoiCv3riPhipup/k4JdvHrYer8KXddILJ4/o4sYlb+roPF7xdN xBJ1N+hrWOhdp/U11fbw/Y/e/NN+URRT8EsMVOW0= Authentication-Results: mxbackcorp2j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:1313::1:e]) by vla5-58875c36c028.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id 7hClKENKMK-eXXeVA3o; Thu, 23 Apr 2020 21:40:33 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [RFC PATCH v1 4/7] vhost: introduce wrappers to set guest notifiers for virtio device Date: Thu, 23 Apr 2020 21:39:35 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=2a02:6b8:0:1472:2741:0:8b6:217; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/23 14:40:14 X-ACL-Warn: Detected OS = ??? X-Received-From: 2a02:6b8:0:1472:2741:0:8b6:217 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, yc-core@yandex-team.ru, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, arei.gonglei@huawei.com, stefanha@redhat.com, marcandre.lureau@redhat.com, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Introduce new wrappers to set/reset guest notifiers for the virtio device in the vhost device module: vhost_dev_assign_guest_notifiers ->set_guest_notifiers(..., ..., true); vhost_dev_drop_guest_notifiers ->set_guest_notifiers(..., ..., false); This is a preliminary step to refactor code, so the set_guest_notifiers methods could be called based on the vhost device state. Update all vhost used devices to use these wrappers instead of direct method call. Signed-off-by: Dima Stepanov --- backends/cryptodev-vhost.c | 26 +++++++++++++++----------- backends/vhost-user.c | 16 +++++----------- hw/block/vhost-user-blk.c | 15 +++++---------- hw/net/vhost_net.c | 30 +++++++++++++++++------------- hw/scsi/vhost-scsi-common.c | 15 +++++---------- hw/virtio/vhost-user-fs.c | 17 +++++++---------- hw/virtio/vhost-vsock.c | 18 ++++++++---------- hw/virtio/vhost.c | 38 ++++++++++++++++++++++++++++++++++++++ hw/virtio/virtio.c | 13 +++++++++++++ include/hw/virtio/vhost.h | 4 ++++ include/hw/virtio/virtio.h | 1 + 11 files changed, 118 insertions(+), 75 deletions(-) diff --git a/backends/cryptodev-vhost.c b/backends/cryptodev-vhost.c index 8337c9a..4522195 100644 --- a/backends/cryptodev-vhost.c +++ b/backends/cryptodev-vhost.c @@ -169,16 +169,13 @@ vhost_set_vring_enable(CryptoDevBackendClient *cc, int cryptodev_vhost_start(VirtIODevice *dev, int total_queues) { VirtIOCrypto *vcrypto = VIRTIO_CRYPTO(dev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev))); - VirtioBusState *vbus = VIRTIO_BUS(qbus); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus); int r, e; int i; CryptoDevBackend *b = vcrypto->cryptodev; CryptoDevBackendVhost *vhost_crypto; CryptoDevBackendClient *cc; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(dev)) { error_report("binding does not support guest notifiers"); return -ENOSYS; } @@ -198,9 +195,13 @@ int cryptodev_vhost_start(VirtIODevice *dev, int total_queues) } } - r = k->set_guest_notifiers(qbus->parent, total_queues, true); + /* + * Since all the states are handled by one vhost device, + * use the first one in array. + */ + vhost_crypto = cryptodev_get_vhost(b->conf.peers.ccs[0], b, 0); + r = vhost_dev_assign_guest_notifiers(&vhost_crypto->dev, dev, total_queues); if (r < 0) { - error_report("error binding guest notifier: %d", -r); goto err; } @@ -232,7 +233,8 @@ err_start: vhost_crypto = cryptodev_get_vhost(cc, b, i); cryptodev_vhost_stop_one(vhost_crypto, dev); } - e = k->set_guest_notifiers(qbus->parent, total_queues, false); + vhost_crypto = cryptodev_get_vhost(b->conf.peers.ccs[0], b, 0); + e = vhost_dev_drop_guest_notifiers(&vhost_crypto->dev, dev, total_queues); if (e < 0) { error_report("vhost guest notifier cleanup failed: %d", e); } @@ -242,9 +244,6 @@ err: void cryptodev_vhost_stop(VirtIODevice *dev, int total_queues) { - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev))); - VirtioBusState *vbus = VIRTIO_BUS(qbus); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus); VirtIOCrypto *vcrypto = VIRTIO_CRYPTO(dev); CryptoDevBackend *b = vcrypto->cryptodev; CryptoDevBackendVhost *vhost_crypto; @@ -259,7 +258,12 @@ void cryptodev_vhost_stop(VirtIODevice *dev, int total_queues) cryptodev_vhost_stop_one(vhost_crypto, dev); } - r = k->set_guest_notifiers(qbus->parent, total_queues, false); + /* + * Since all the states are handled by one vhost device, + * use the first one in array. + */ + vhost_crypto = cryptodev_get_vhost(b->conf.peers.ccs[0], b, 0); + r = vhost_dev_drop_guest_notifiers(&vhost_crypto->dev, dev, total_queues); if (r < 0) { error_report("vhost guest notifier cleanup failed: %d", r); } diff --git a/backends/vhost-user.c b/backends/vhost-user.c index 2bf3406..e116bc6 100644 --- a/backends/vhost-user.c +++ b/backends/vhost-user.c @@ -60,15 +60,13 @@ vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev, void vhost_user_backend_start(VhostUserBackend *b) { - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(b->vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret, i ; if (b->started) { return; } - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(b->vdev)) { error_report("binding does not support guest notifiers"); return; } @@ -78,9 +76,8 @@ vhost_user_backend_start(VhostUserBackend *b) return; } - ret = k->set_guest_notifiers(qbus->parent, b->dev.nvqs, true); + ret = vhost_dev_assign_guest_notifiers(&b->dev, b->vdev, b->dev.nvqs); if (ret < 0) { - error_report("Error binding guest notifier"); goto err_host_notifiers; } @@ -104,7 +101,7 @@ vhost_user_backend_start(VhostUserBackend *b) return; err_guest_notifiers: - k->set_guest_notifiers(qbus->parent, b->dev.nvqs, false); + vhost_dev_drop_guest_notifiers(&b->dev, b->vdev, b->dev.nvqs); err_host_notifiers: vhost_dev_disable_notifiers(&b->dev, b->vdev); } @@ -112,8 +109,6 @@ err_host_notifiers: void vhost_user_backend_stop(VhostUserBackend *b) { - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(b->vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret = 0; if (!b->started) { @@ -122,9 +117,8 @@ vhost_user_backend_stop(VhostUserBackend *b) vhost_dev_stop(&b->dev, b->vdev); - if (k->set_guest_notifiers) { - ret = k->set_guest_notifiers(qbus->parent, - b->dev.nvqs, false); + if (virtio_device_guest_notifiers_initialized(b->vdev)) { + ret = vhost_dev_drop_guest_notifiers(&b->dev, b->vdev, b->dev.nvqs); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); } diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 17df533..70d7842 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -109,11 +109,9 @@ const VhostDevConfigOps blk_ops = { static int vhost_user_blk_start(VirtIODevice *vdev) { VHostUserBlk *s = VHOST_USER_BLK(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int i, ret; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { error_report("binding does not support guest notifiers"); return -ENOSYS; } @@ -124,9 +122,8 @@ static int vhost_user_blk_start(VirtIODevice *vdev) return ret; } - ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, true); + ret = vhost_dev_assign_guest_notifiers(&s->dev, vdev, s->dev.nvqs); if (ret < 0) { - error_report("Error binding guest notifier: %d", -ret); goto err_host_notifiers; } @@ -163,7 +160,7 @@ static int vhost_user_blk_start(VirtIODevice *vdev) return ret; err_guest_notifiers: - k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); + vhost_dev_drop_guest_notifiers(&s->dev, vdev, s->dev.nvqs); err_host_notifiers: vhost_dev_disable_notifiers(&s->dev, vdev); return ret; @@ -172,17 +169,15 @@ err_host_notifiers: static void vhost_user_blk_stop(VirtIODevice *vdev) { VHostUserBlk *s = VHOST_USER_BLK(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { return; } vhost_dev_stop(&s->dev, vdev); - ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); + ret = vhost_dev_drop_guest_notifiers(&s->dev, vdev, s->dev.nvqs); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); return; diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 6b82803..c13b444 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -303,19 +303,15 @@ static void vhost_net_stop_one(struct vhost_net *net, int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, int total_queues) { - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev))); - VirtioBusState *vbus = VIRTIO_BUS(qbus); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus); + struct vhost_net *net; int r, e, i; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(dev)) { error_report("binding does not support guest notifiers"); return -ENOSYS; } for (i = 0; i < total_queues; i++) { - struct vhost_net *net; - net = get_vhost_net(ncs[i].peer); vhost_net_set_vq_index(net, i * 2); @@ -328,9 +324,13 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, } } - r = k->set_guest_notifiers(qbus->parent, total_queues * 2, true); + /* + * Since all the states are handled by one vhost_net device, + * use the first one in array. + */ + net = get_vhost_net(ncs[0].peer); + r = vhost_dev_assign_guest_notifiers(&net->dev, dev, total_queues * 2); if (r < 0) { - error_report("Error binding guest notifier: %d", -r); goto err; } @@ -357,7 +357,8 @@ err_start: while (--i >= 0) { vhost_net_stop_one(get_vhost_net(ncs[i].peer), dev); } - e = k->set_guest_notifiers(qbus->parent, total_queues * 2, false); + net = get_vhost_net(ncs[0].peer); + e = vhost_dev_drop_guest_notifiers(&net->dev, dev, total_queues * 2); if (e < 0) { fprintf(stderr, "vhost guest notifier cleanup failed: %d\n", e); fflush(stderr); @@ -369,16 +370,19 @@ err: void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs, int total_queues) { - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(dev))); - VirtioBusState *vbus = VIRTIO_BUS(qbus); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus); + struct vhost_net *net; int i, r; for (i = 0; i < total_queues; i++) { vhost_net_stop_one(get_vhost_net(ncs[i].peer), dev); } - r = k->set_guest_notifiers(qbus->parent, total_queues * 2, false); + /* + * Since all the states are handled by one vhost_net device, + * use the first one in array. + */ + net = get_vhost_net(ncs[0].peer); + r = vhost_dev_drop_guest_notifiers(&net->dev, dev, total_queues * 2); if (r < 0) { fprintf(stderr, "vhost guest notifier cleanup failed: %d\n", r); fflush(stderr); diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index 8ec49d7..8f51ec0 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -29,10 +29,8 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) { int ret, i; VirtIODevice *vdev = VIRTIO_DEVICE(vsc); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { error_report("binding does not support guest notifiers"); return -ENOSYS; } @@ -42,9 +40,8 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; } - ret = k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, true); + ret = vhost_dev_assign_guest_notifiers(&vsc->dev, vdev, vsc->dev.nvqs); if (ret < 0) { - error_report("Error binding guest notifier"); goto err_host_notifiers; } @@ -66,7 +63,7 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; err_guest_notifiers: - k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); + vhost_dev_drop_guest_notifiers(&vsc->dev, vdev, vsc->dev.nvqs); err_host_notifiers: vhost_dev_disable_notifiers(&vsc->dev, vdev); return ret; @@ -75,14 +72,12 @@ err_host_notifiers: void vhost_scsi_common_stop(VHostSCSICommon *vsc) { VirtIODevice *vdev = VIRTIO_DEVICE(vsc); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret = 0; vhost_dev_stop(&vsc->dev, vdev); - if (k->set_guest_notifiers) { - ret = k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); + if (virtio_device_guest_notifiers_initialized(vdev)) { + ret = vhost_dev_drop_guest_notifiers(&vsc->dev, vdev, vsc->dev.nvqs); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); } diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c index 6136768..6b101fc 100644 --- a/hw/virtio/vhost-user-fs.c +++ b/hw/virtio/vhost-user-fs.c @@ -38,12 +38,10 @@ static void vuf_get_config(VirtIODevice *vdev, uint8_t *config) static void vuf_start(VirtIODevice *vdev) { VHostUserFS *fs = VHOST_USER_FS(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; int i; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { error_report("binding does not support guest notifiers"); return; } @@ -54,9 +52,9 @@ static void vuf_start(VirtIODevice *vdev) return; } - ret = k->set_guest_notifiers(qbus->parent, fs->vhost_dev.nvqs, true); + ret = vhost_dev_assign_guest_notifiers(&fs->vhost_dev, vdev, + fs->vhost_dev.nvqs); if (ret < 0) { - error_report("Error binding guest notifier: %d", -ret); goto err_host_notifiers; } @@ -79,7 +77,7 @@ static void vuf_start(VirtIODevice *vdev) return; err_guest_notifiers: - k->set_guest_notifiers(qbus->parent, fs->vhost_dev.nvqs, false); + vhost_dev_drop_guest_notifiers(&fs->vhost_dev, vdev, fs->vhost_dev.nvqs); err_host_notifiers: vhost_dev_disable_notifiers(&fs->vhost_dev, vdev); } @@ -87,17 +85,16 @@ err_host_notifiers: static void vuf_stop(VirtIODevice *vdev) { VHostUserFS *fs = VHOST_USER_FS(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { return; } vhost_dev_stop(&fs->vhost_dev, vdev); - ret = k->set_guest_notifiers(qbus->parent, fs->vhost_dev.nvqs, false); + ret = vhost_dev_drop_guest_notifiers(&fs->vhost_dev, vdev, + fs->vhost_dev.nvqs); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); return; diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c index 09b6b07..52489dd 100644 --- a/hw/virtio/vhost-vsock.c +++ b/hw/virtio/vhost-vsock.c @@ -75,12 +75,10 @@ static int vhost_vsock_set_running(VHostVSock *vsock, int start) static void vhost_vsock_start(VirtIODevice *vdev) { VHostVSock *vsock = VHOST_VSOCK(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; int i; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { error_report("binding does not support guest notifiers"); return; } @@ -91,9 +89,9 @@ static void vhost_vsock_start(VirtIODevice *vdev) return; } - ret = k->set_guest_notifiers(qbus->parent, vsock->vhost_dev.nvqs, true); + ret = vhost_dev_assign_guest_notifiers(&vsock->vhost_dev, + vdev, vsock->vhost_dev.nvqs); if (ret < 0) { - error_report("Error binding guest notifier: %d", -ret); goto err_host_notifiers; } @@ -123,7 +121,8 @@ static void vhost_vsock_start(VirtIODevice *vdev) err_dev_start: vhost_dev_stop(&vsock->vhost_dev, vdev); err_guest_notifiers: - k->set_guest_notifiers(qbus->parent, vsock->vhost_dev.nvqs, false); + vhost_dev_drop_guest_notifiers(&vsock->vhost_dev, + vdev, vsock->vhost_dev.nvqs); err_host_notifiers: vhost_dev_disable_notifiers(&vsock->vhost_dev, vdev); } @@ -131,11 +130,9 @@ err_host_notifiers: static void vhost_vsock_stop(VirtIODevice *vdev) { VHostVSock *vsock = VHOST_VSOCK(vdev); - BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); - VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; - if (!k->set_guest_notifiers) { + if (!virtio_device_guest_notifiers_initialized(vdev)) { return; } @@ -147,7 +144,8 @@ static void vhost_vsock_stop(VirtIODevice *vdev) vhost_dev_stop(&vsock->vhost_dev, vdev); - ret = k->set_guest_notifiers(qbus->parent, vsock->vhost_dev.nvqs, false); + ret = vhost_dev_drop_guest_notifiers(&vsock->vhost_dev, + vdev, vsock->vhost_dev.nvqs); if (ret < 0) { error_report("vhost guest notifier cleanup failed: %d", ret); return; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 01ebe12..fa3da9c 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1419,6 +1419,44 @@ void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev) virtio_device_release_ioeventfd(vdev); } +/* + * Assign guest notifiers. + * Should be called after vhost_dev_enable_notifiers. + */ +int vhost_dev_assign_guest_notifiers(struct vhost_dev *hdev, + VirtIODevice *vdev, int nvqs) +{ + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + ret = k->set_guest_notifiers(qbus->parent, nvqs, true); + if (ret < 0) { + error_report("Error binding guest notifier: %d", -ret); + } + + return ret; +} + +/* + * Drop guest notifiers. + * Should be called before vhost_dev_disable_notifiers. + */ +int vhost_dev_drop_guest_notifiers(struct vhost_dev *hdev, + VirtIODevice *vdev, int nvqs) +{ + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + ret = k->set_guest_notifiers(qbus->parent, nvqs, false); + if (ret < 0) { + error_report("Error reset guest notifier: %d", -ret); + } + + return ret; +} + /* Test and clear event pending status. * Should be called after unmask to avoid losing events. */ diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index b6c8ef5..8a95618 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3812,6 +3812,19 @@ bool virtio_device_ioeventfd_enabled(VirtIODevice *vdev) return virtio_bus_ioeventfd_enabled(vbus); } +/* + * Check if set_guest_notifiers() method is set by the init routine. + * Return true if yes, otherwise return false. + */ +bool virtio_device_guest_notifiers_initialized(VirtIODevice *vdev) +{ + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + + return k->set_guest_notifiers; +} + + static const TypeInfo virtio_device_info = { .name = TYPE_VIRTIO_DEVICE, .parent = TYPE_DEVICE, diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 085450c..4d0d2e2 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -100,6 +100,10 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev); void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev); int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev); void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev); +int vhost_dev_assign_guest_notifiers(struct vhost_dev *hdev, + VirtIODevice *vdev, int nvqs); +int vhost_dev_drop_guest_notifiers(struct vhost_dev *hdev, + VirtIODevice *vdev, int nvqs); /* Test and clear masked event pending status. * Should be called after unmask to avoid losing events. diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index b69d517..d9a3d72 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -323,6 +323,7 @@ void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext *ctx, VirtIOHandleAIOOutput handle_output); VirtQueue *virtio_vector_first_queue(VirtIODevice *vdev, uint16_t vector); VirtQueue *virtio_vector_next_queue(VirtQueue *vq); +bool virtio_device_guest_notifiers_initialized(VirtIODevice *vdev); static inline void virtio_add_feature(uint64_t *features, unsigned int fbit) { From patchwork Thu Apr 23 18:39:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 283990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FC60C54FCB for ; Thu, 23 Apr 2020 18:46:42 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 69B8320704 for ; Thu, 23 Apr 2020 18:46:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="oqR/GfWb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 69B8320704 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36968 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgrx-0002Sk-I8 for qemu-devel@archiver.kernel.org; Thu, 23 Apr 2020 14:46:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32974) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRgmK-0001Y0-Gk for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jRgmJ-0004io-TV for qemu-devel@nongnu.org; Thu, 23 Apr 2020 14:40:52 -0400 Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]:59924) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jRgmH-0004c0-Bt; Thu, 23 Apr 2020 14:40:49 -0400 Received: from mxbackcorp1g.mail.yandex.net (mxbackcorp1g.mail.yandex.net [IPv6:2a02:6b8:0:1402::301]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 8E8A22E15CB; Thu, 23 Apr 2020 21:40:46 +0300 (MSK) Received: from vla5-58875c36c028.qloud-c.yandex.net (vla5-58875c36c028.qloud-c.yandex.net [2a02:6b8:c18:340b:0:640:5887:5c36]) by mxbackcorp1g.mail.yandex.net (mxbackcorp/Yandex) with ESMTP id vIyfhVYIID-eiQCVYfP; Thu, 23 Apr 2020 21:40:46 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1587667246; bh=Fv6pZWWviC9Gnz6pDjZHVEUsXW2GPb2wYw220Nlajio=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=oqR/GfWbX7A3DDl+ut8AlJdXeTWoGoZ6mxLR8tsz5wAo6XhSD3pP6BhAmg/26HjZ/ sSp67J+CeKR/M8QUTiGtxLQWWgkPzud5Cng9m7m/feWaMT4MSkFCQB38DoxUNvUZ6B j2BloZ0FlBSaYTnG/gDTd0QHg86CRYYR88azjZNg= Authentication-Results: mxbackcorp1g.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:1313::1:e]) by vla5-58875c36c028.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id 7hClKENKMK-ehXewq95; Thu, 23 Apr 2020 21:40:43 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [RFC PATCH v1 7/7] vhost: add device started check in migration set log Date: Thu, 23 Apr 2020 21:39:38 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=95.108.205.193; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/23 14:40:28 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 95.108.205.193 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, yc-core@yandex-team.ru, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, arei.gonglei@huawei.com, stefanha@redhat.com, marcandre.lureau@redhat.com, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If vhost-user daemon is used as a backend for the vhost device, then we should consider a possibility of disconnect at any moment. If such disconnect happened in the vhost_migration_log() routine the vhost device structure will be clean up. At the start of the vhost_migration_log() function there is a check: if (!dev->started) { dev->log_enabled = enable; return 0; } To be consistent with this check add the same check after calling the vhost_dev_set_log() routine. This in general help not to break a migration due the assert() message. But it looks like that this code should be revised to handle these errors more carefully. In case of vhost-user device backend the fail paths should consider the state of the device. In this case we should skip some function calls during rollback on the error paths, so not to get the NULL dereference errors. Signed-off-by: Dima Stepanov --- hw/virtio/vhost.c | 39 +++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 3ee50c4..d5ab96d 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -787,6 +787,17 @@ static int vhost_dev_set_features(struct vhost_dev *dev, static int vhost_dev_set_log(struct vhost_dev *dev, bool enable_log) { int r, i, idx; + + if (!dev->started) { + /* + * If vhost-user daemon is used as a backend for the + * device and the connection is broken, then the vhost_dev + * structure will be reset all its values to 0. + * Add additional check for the device state. + */ + return -1; + } + r = vhost_dev_set_features(dev, enable_log); if (r < 0) { goto err_features; @@ -801,12 +812,19 @@ static int vhost_dev_set_log(struct vhost_dev *dev, bool enable_log) } return 0; err_vq: - for (; i >= 0; --i) { + /* + * Disconnect with the vhost-user daemon can lead to the + * vhost_dev_cleanup() call which will clean up vhost_dev + * structure. + */ + for (; dev->started && (i >= 0); --i) { idx = dev->vhost_ops->vhost_get_vq_index(dev, dev->vq_index + i); vhost_virtqueue_set_addr(dev, dev->vqs + i, idx, dev->log_enabled); } - vhost_dev_set_features(dev, dev->log_enabled); + if (dev->started) { + vhost_dev_set_features(dev, dev->log_enabled); + } err_features: return r; } @@ -832,7 +850,15 @@ static int vhost_migration_log(MemoryListener *listener, int enable) } else { vhost_dev_log_resize(dev, vhost_get_log_size(dev)); r = vhost_dev_set_log(dev, true); - if (r < 0) { + /* + * The dev log resize can fail, because of disconnect + * with the vhost-user-blk daemon. Check the device + * state before calling the vhost_dev_set_log() + * function. + * Don't return error if device isn't started to be + * consistent with the check above. + */ + if (dev->started && r < 0) { return r; } } @@ -1739,7 +1765,12 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev) fail_log: vhost_log_put(hdev, false); fail_vq: - while (--i >= 0) { + /* + * Disconnect with the vhost-user daemon can lead to the + * vhost_dev_cleanup() call which will clean up vhost_dev + * structure. + */ + while ((--i >= 0) && (hdev->started)) { vhost_virtqueue_stop(hdev, vdev, hdev->vqs + i,