From patchwork Mon Aug 24 08:39:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F397FC433E3 for ; Mon, 24 Aug 2020 08:41:46 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C30C520FC3 for ; Mon, 24 Aug 2020 08:41:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="CGhoeU/u" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C30C520FC3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38134 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA82z-000344-PB for qemu-devel@archiver.kernel.org; Mon, 24 Aug 2020 04:41:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA81n-0001Ru-Co; Mon, 24 Aug 2020 04:40:31 -0400 Received: from forwardcorp1j.mail.yandex.net ([2a02:6b8:0:1619::183]:34758) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA81k-0004w4-Cn; Mon, 24 Aug 2020 04:40:31 -0400 Received: from myt5-23f0be3aa648.qloud-c.yandex.net (myt5-23f0be3aa648.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3e29:0:640:23f0:be3a]) by forwardcorp1j.mail.yandex.net (Yandex) with ESMTP id D0A3B2E1574; Mon, 24 Aug 2020 11:40:24 +0300 (MSK) Received: from myt4-18a966dbd9be.qloud-c.yandex.net (myt4-18a966dbd9be.qloud-c.yandex.net [2a02:6b8:c00:12ad:0:640:18a9:66db]) by myt5-23f0be3aa648.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id ujBd8zSqIq-eMvWb3VZ; Mon, 24 Aug 2020 11:40:24 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598258424; bh=2OTCE6c30ypdv3IVkGlKAJ1dEI3lFCSus1qYAVg52jw=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=CGhoeU/ug07NnX0KgAtBXUU00TD+ZiQzQOKHkcevFHQw2FwnPs/4Mq6AKI5z8+jD9 5gY8riA+6Ok9M5XsEdpKZW+BEuPs/+wIsIGsVnaVbeBBYc6nxqwV5bveifFUikhdDH NCsHApZqHQyngj7Z4WPDLGbuvgoeWz4waJoiGLvk= Authentication-Results: myt5-23f0be3aa648.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b080:6622::1:7]) by myt4-18a966dbd9be.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id B7dErHoRhW-eLl4QAA9; Mon, 24 Aug 2020 11:40:22 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v2 1/7] vhost: recheck dev state in the vhost_migration_log routine Date: Mon, 24 Aug 2020 11:39:26 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=2a02:6b8:0:1619::183; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1j.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" vhost-user devices can get a disconnect in the middle of the VHOST-USER handshake on the migration start. If disconnect event happened right before sending next VHOST-USER command, then the vhost_dev_set_log() call in the vhost_migration_log() function will return error. This error will lead to the assert() and close the QEMU migration source process. For the vhost-user devices the disconnect event should not break the migration process, because: - the device will be in the stopped state, so it will not be changed during migration - if reconnect will be made the migration log will be reinitialized as part of reconnect/init process: #0 vhost_log_global_start (listener=0x563989cf7be0) at hw/virtio/vhost.c:920 #1 0x000056398603d8bc in listener_add_address_space (listener=0x563989cf7be0, as=0x563986ea4340 ) at softmmu/memory.c:2664 #2 0x000056398603dd30 in memory_listener_register (listener=0x563989cf7be0, as=0x563986ea4340 ) at softmmu/memory.c:2740 #3 0x0000563985fd6956 in vhost_dev_init (hdev=0x563989cf7bd8, opaque=0x563989cf7e30, backend_type=VHOST_BACKEND_TYPE_USER, busyloop_timeout=0) at hw/virtio/vhost.c:1385 #4 0x0000563985f7d0b8 in vhost_user_blk_connect (dev=0x563989cf7990) at hw/block/vhost-user-blk.c:315 #5 0x0000563985f7d3f6 in vhost_user_blk_event (opaque=0x563989cf7990, event=CHR_EVENT_OPENED) at hw/block/vhost-user-blk.c:379 Update the vhost-user-blk device with the internal started field which will be used for initialization and clean up. The disconnect event will set the overall VHOST device to the stopped state, so it can be used by the vhost_migration_log routine. Such approach could be propogated to the other vhost-user devices, but better idea is just to make the same connect/disconnect code for all the vhost-user devices. This migration issue was slightly discussed earlier: - https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg01509.html - https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg05241.html Signed-off-by: Dima Stepanov --- hw/block/vhost-user-blk.c | 19 ++++++++++++++++--- hw/virtio/vhost.c | 27 ++++++++++++++++++++++++--- include/hw/virtio/vhost-user-blk.h | 10 ++++++++++ 3 files changed, 50 insertions(+), 6 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index a00b854..5573e89 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -150,6 +150,7 @@ static int vhost_user_blk_start(VirtIODevice *vdev) error_report("Error starting vhost: %d", -ret); goto err_guest_notifiers; } + s->started = true; /* guest_notifier_mask/pending not used yet, so just unmask * everything here. virtio-pci will do the right thing by @@ -175,6 +176,11 @@ static void vhost_user_blk_stop(VirtIODevice *vdev) VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; + if (!s->started) { + return; + } + s->started = false; + if (!k->set_guest_notifiers) { return; } @@ -341,9 +347,7 @@ static void vhost_user_blk_disconnect(DeviceState *dev) } s->connected = false; - if (s->dev.started) { - vhost_user_blk_stop(vdev); - } + vhost_user_blk_stop(vdev); vhost_dev_cleanup(&s->dev); } @@ -399,6 +403,15 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event) NULL, NULL, false); aio_bh_schedule_oneshot(ctx, vhost_user_blk_chr_closed_bh, opaque); } + + /* + * Move vhost device to the stopped state. The vhost-user device + * will be clean up and disconnected in BH. This can be useful in + * the vhost migration code. If disconnect was caught there is an + * option for the general vhost code to get the dev state without + * knowing its type (in this case vhost-user). + */ + s->dev.started = false; break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 1a1384e..ffef7ab 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -861,21 +861,42 @@ static int vhost_migration_log(MemoryListener *listener, bool enable) dev->log_enabled = enable; return 0; } + + r = 0; if (!enable) { r = vhost_dev_set_log(dev, false); if (r < 0) { - return r; + goto check_dev_state; } vhost_log_put(dev, false); } else { vhost_dev_log_resize(dev, vhost_get_log_size(dev)); r = vhost_dev_set_log(dev, true); if (r < 0) { - return r; + goto check_dev_state; } } + +check_dev_state: dev->log_enabled = enable; - return 0; + /* + * vhost-user-* devices could change their state during log + * initialization due to disconnect. So check dev state after + * vhost communication. + */ + if (!dev->started) { + /* + * Since device is in the stopped state, it is okay for + * migration. Return success. + */ + r = 0; + } + if (r) { + /* An error is occured. */ + dev->log_enabled = false; + } + + return r; } static void vhost_log_global_start(MemoryListener *listener) diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index 34ad6f0..f4c0754 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -38,7 +38,17 @@ typedef struct VHostUserBlk { VhostUserState vhost_user; struct vhost_virtqueue *vhost_vqs; VirtQueue **virtqs; + + /* + * There are at least two steps of initialization of the + * vhost-user device. The first is a "connect" step and + * second is a "start" step. Make a separation between + * those initialization phases by using two fields. + */ + /* vhost_user_blk_connect/vhost_user_blk_disconnect */ bool connected; + /* vhost_user_blk_start/vhost_user_blk_stop */ + bool started; } VHostUserBlk; #endif From patchwork Mon Aug 24 08:39:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9B78C433E3 for ; Mon, 24 Aug 2020 08:44:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A9EBC221E2 for ; Mon, 24 Aug 2020 08:44:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="Cft4n1cd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A9EBC221E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA85d-0000Zj-Vi for qemu-devel@archiver.kernel.org; Mon, 24 Aug 2020 04:44:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58724) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA81w-0001kz-DI; Mon, 24 Aug 2020 04:40:40 -0400 Received: from forwardcorp1j.mail.yandex.net ([2a02:6b8:0:1619::183]:34920) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA81u-0004x2-Hd; Mon, 24 Aug 2020 04:40:40 -0400 Received: from myt5-23f0be3aa648.qloud-c.yandex.net (myt5-23f0be3aa648.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3e29:0:640:23f0:be3a]) by forwardcorp1j.mail.yandex.net (Yandex) with ESMTP id B41D72E1574; Mon, 24 Aug 2020 11:40:36 +0300 (MSK) Received: from myt4-18a966dbd9be.qloud-c.yandex.net (myt4-18a966dbd9be.qloud-c.yandex.net [2a02:6b8:c00:12ad:0:640:18a9:66db]) by myt5-23f0be3aa648.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id dNFJqYsUSN-eZvKLQrm; Mon, 24 Aug 2020 11:40:36 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598258436; bh=UciJc8X9njWrEUHJFothSUSQxAE9xsIR5v3e7XbMiqU=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=Cft4n1cd6vWPuuLHdZf+zrVEgDYzoWkPHuLU9qPv09EQW7G4WIg13oR3+McKqeQZQ /YOCSHDcnkxV0W6NJVymumuILz2B22qwk/PSpqzRu6OKQRtvpfj1JVNPgkilqx7ppb RZkZMW8jEI1yeWjOJmwtbD4R6bcRWVHV1Kn3L3P8= Authentication-Results: myt5-23f0be3aa648.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b080:6622::1:7]) by myt4-18a966dbd9be.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id B7dErHoRhW-eYl4VDah; Mon, 24 Aug 2020 11:40:35 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v2 4/7] tests/qtest/libqos/virtio-blk: add support for vhost-user-blk Date: Mon, 24 Aug 2020 11:39:29 +0300 Message-Id: <45fd6995a5e226eaf179b99d6bf3b1cde5e4c309.1598257838.git.dimastep@yandex-team.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=2a02:6b8:0:1619::183; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1j.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add support for the vhost-user-blk-pci device. This node can be used by the vhost-user-blk tests. Tests for the vhost-user-blk device are added in the following patches. Signed-off-by: Dima Stepanov --- tests/qtest/libqos/virtio-blk.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tests/qtest/libqos/virtio-blk.c b/tests/qtest/libqos/virtio-blk.c index 5da0259..959c5dc 100644 --- a/tests/qtest/libqos/virtio-blk.c +++ b/tests/qtest/libqos/virtio-blk.c @@ -36,6 +36,9 @@ static void *qvirtio_blk_get_driver(QVirtioBlk *v_blk, if (!g_strcmp0(interface, "virtio")) { return v_blk->vdev; } + if (!g_strcmp0(interface, "vhost-user-blk")) { + return v_blk; + } fprintf(stderr, "%s not present in virtio-blk-device\n", interface); g_assert_not_reached(); @@ -120,6 +123,17 @@ static void virtio_blk_register_nodes(void) qos_node_produces("virtio-blk-pci", "virtio-blk"); g_free(arg); + + /* vhost-user-blk-pci */ + arg = g_strdup_printf("id=drv0,chardev=chdev0,addr=%x.%x", + PCI_SLOT, PCI_FN); + opts.extra_device_opts = arg; + add_qpci_address(&opts, &addr); + qos_node_create_driver("vhost-user-blk-pci", virtio_blk_pci_create); + qos_node_consumes("vhost-user-blk-pci", "pci-bus", &opts); + qos_node_produces("vhost-user-blk-pci", "vhost-user-blk"); + + g_free(arg); } libqos_init(virtio_blk_register_nodes); From patchwork Mon Aug 24 08:39:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 284CBC433E4 for ; Mon, 24 Aug 2020 08:42:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E91D62075B for ; Mon, 24 Aug 2020 08:42:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="gAkG9F0m" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E91D62075B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA83H-0003i0-50 for qemu-devel@archiver.kernel.org; Mon, 24 Aug 2020 04:42:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA822-0001yM-EJ; Mon, 24 Aug 2020 04:40:46 -0400 Received: from forwardcorp1j.mail.yandex.net ([5.45.199.163]:47016) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA81z-0004xO-QM; Mon, 24 Aug 2020 04:40:46 -0400 Received: from myt5-23f0be3aa648.qloud-c.yandex.net (myt5-23f0be3aa648.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3e29:0:640:23f0:be3a]) by forwardcorp1j.mail.yandex.net (Yandex) with ESMTP id 7E3282E1570; Mon, 24 Aug 2020 11:40:41 +0300 (MSK) Received: from myt4-18a966dbd9be.qloud-c.yandex.net (myt4-18a966dbd9be.qloud-c.yandex.net [2a02:6b8:c00:12ad:0:640:18a9:66db]) by myt5-23f0be3aa648.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id ib5cfUIOd2-edvKr8VR; Mon, 24 Aug 2020 11:40:41 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598258441; bh=0YiyaXGCteV6eAUx85YKD/nISwnScCEnn+mIKN9OdOg=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=gAkG9F0mMYtx/M65u4erPOnj00hG07Ppskht6qy2OlJ5w+mqHuL5e+1IWvRp8o39P Ar0muTvkpvatFCop80rhCKKpXARfbjnxvz4ZoytDgCASX31M3dajJUEDIEb31Mh1IG IH07Zi5VUWHgkXSxfoU8fnbgJrt/utxrn3dHNF80= Authentication-Results: myt5-23f0be3aa648.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b080:6622::1:7]) by myt4-18a966dbd9be.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id B7dErHoRhW-edl4uxhb; Mon, 24 Aug 2020 11:40:39 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v2 5/7] tests/qtest/vhost-user-test: add support for the vhost-user-blk device Date: Mon, 24 Aug 2020 11:39:30 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=5.45.199.163; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1j.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/24 04:40:41 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add vhost_user_ops structure for the vhost-user-blk device class. Add the test_reconnect and test_migrate tests for this device. Signed-off-by: Dima Stepanov --- tests/qtest/vhost-user-test.c | 140 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 138 insertions(+), 2 deletions(-) diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c index 3df5322..9b6e202 100644 --- a/tests/qtest/vhost-user-test.c +++ b/tests/qtest/vhost-user-test.c @@ -24,6 +24,7 @@ #include "libqos/libqos.h" #include "libqos/pci-pc.h" #include "libqos/virtio-pci.h" +#include "libqos/virtio-blk.h" #include "libqos/malloc-pc.h" #include "hw/virtio/virtio-net.h" @@ -31,6 +32,7 @@ #include "standard-headers/linux/vhost_types.h" #include "standard-headers/linux/virtio_ids.h" #include "standard-headers/linux/virtio_net.h" +#include "standard-headers/linux/virtio_blk.h" #ifdef CONFIG_LINUX #include @@ -43,6 +45,7 @@ " -numa node,memdev=mem" #define QEMU_CMD_CHR " -chardev socket,id=%s,path=%s%s" #define QEMU_CMD_NETDEV " -netdev vhost-user,id=hs0,chardev=%s,vhostforce" +#define QEMU_CMD_BLKCHR " -chardev socket,id=chdev0,path=%s%s" #define HUGETLBFS_MAGIC 0x958458f6 @@ -55,6 +58,7 @@ #define VHOST_USER_PROTOCOL_F_MQ 0 #define VHOST_USER_PROTOCOL_F_LOG_SHMFD 1 #define VHOST_USER_PROTOCOL_F_CROSS_ENDIAN 6 +#define VHOST_USER_PROTOCOL_F_CONFIG 9 #define VHOST_LOG_PAGE 0x1000 @@ -78,6 +82,8 @@ typedef enum VhostUserRequest { VHOST_USER_SET_PROTOCOL_FEATURES = 16, VHOST_USER_GET_QUEUE_NUM = 17, VHOST_USER_SET_VRING_ENABLE = 18, + VHOST_USER_GET_CONFIG = 24, + VHOST_USER_SET_CONFIG = 25, VHOST_USER_MAX } VhostUserRequest; @@ -99,6 +105,14 @@ typedef struct VhostUserLog { uint64_t mmap_offset; } VhostUserLog; +#define VHOST_USER_MAX_CONFIG_SIZE 256 +typedef struct VhostUserConfig { + uint32_t offset; + uint32_t size; + uint32_t flags; + uint8_t region[VHOST_USER_MAX_CONFIG_SIZE]; +} VhostUserConfig; + typedef struct VhostUserMsg { VhostUserRequest request; @@ -114,6 +128,7 @@ typedef struct VhostUserMsg { struct vhost_vring_addr addr; VhostUserMemory memory; VhostUserLog log; + VhostUserConfig config; } payload; } QEMU_PACKED VhostUserMsg; @@ -137,6 +152,7 @@ enum { enum { VHOST_USER_NET, + VHOST_USER_BLK, }; typedef struct TestServer { @@ -166,12 +182,15 @@ struct vhost_user_ops { int type; void (*append_opts)(TestServer *s, GString *cmd_line, const char *chr_opts); + void (*driver_init)(void *obj, QGuestAllocator *alloc); /* VHOST-USER commands. */ void (*set_features)(TestServer *s, CharBackend *chr, VhostUserMsg *msg); void (*get_protocol_features)(TestServer *s, CharBackend *chr, VhostUserMsg *msg); + void (*get_config)(TestServer *s, CharBackend *chr, + VhostUserMsg *msg); }; static const char *init_hugepagefs(void); @@ -194,6 +213,14 @@ static void append_vhost_net_opts(TestServer *s, GString *cmd_line, chr_opts, s->chr_name); } +static void append_vhost_blk_opts(TestServer *s, GString *cmd_line, + const char *chr_opts) +{ + g_string_append_printf(cmd_line, QEMU_CMD_BLKCHR, + s->socket_path, + chr_opts); +} + static void append_mem_opts(TestServer *server, GString *cmd_line, int size, enum test_memfd memfd) { @@ -425,6 +452,12 @@ static void chr_read(void *opaque, const uint8_t *buf, int size) qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size); break; + case VHOST_USER_GET_CONFIG: + if (s->vu_ops->get_config) { + s->vu_ops->get_config(s, chr, &msg); + } + break; + default: break; } @@ -727,6 +760,9 @@ static void test_migrate(void *obj, void *arg, QGuestAllocator *alloc) guint8 *log; guint64 size; + if (s->vu_ops->driver_init) { + s->vu_ops->driver_init(obj, alloc); + } if (!wait_for_fds(s)) { return; } @@ -796,6 +832,24 @@ static void test_migrate(void *obj, void *arg, QGuestAllocator *alloc) g_string_free(dest_cmdline, true); } +static void vu_blk_driver_init(void *obj, QGuestAllocator *alloc) +{ + QVirtioBlk *blk_if; + QVirtioDevice *dev; + QVirtQueue *vq; + uint64_t features; + + blk_if = obj; + dev = blk_if->vdev; + features = qvirtio_get_features(dev); + qvirtio_set_features(dev, features); + + vq = qvirtqueue_setup(dev, alloc, 0); + g_assert(vq); + + qvirtio_set_driver_ok(dev); +} + static void wait_for_rings_started(TestServer *s, size_t count) { gint64 end_time; @@ -857,12 +911,22 @@ static void test_reconnect(void *obj, void *arg, QGuestAllocator *alloc) { TestServer *s = arg; GSource *src; + int nq; + if (s->vu_ops->driver_init) { + s->vu_ops->driver_init(obj, alloc); + } if (!wait_for_fds(s)) { return; } - wait_for_rings_started(s, 2); + if (s->vu_ops->type == VHOST_USER_NET) { + /* tx and rx queues */ + nq = 2; + } else if (s->vu_ops->type == VHOST_USER_BLK) { + nq = 1; + } + wait_for_rings_started(s, nq); /* reconnect */ s->fds_num = 0; @@ -872,7 +936,7 @@ static void test_reconnect(void *obj, void *arg, QGuestAllocator *alloc) g_source_attach(src, s->context); g_source_unref(src); g_assert(wait_for_fds(s)); - wait_for_rings_started(s, 2); + wait_for_rings_started(s, nq); } static void *vhost_user_test_setup_connect_fail(GString *cmd_line, void *arg) @@ -960,6 +1024,56 @@ static void vu_net_get_protocol_features(TestServer *s, CharBackend *chr, qemu_chr_fe_write_all(chr, (uint8_t *)msg, VHOST_USER_HDR_SIZE + msg->size); } +static void vu_blk_set_features(TestServer *s, CharBackend *chr, + VhostUserMsg *msg) +{ + if (s->test_flags == TEST_FLAGS_DISCONNECT) { + qemu_chr_fe_disconnect(chr); + s->test_flags = TEST_FLAGS_BAD; + } +} + +static void vu_blk_get_protocol_features(TestServer *s, + CharBackend *chr, VhostUserMsg *msg) +{ + /* send back features to qemu */ + msg->flags |= VHOST_USER_REPLY_MASK; + msg->size = sizeof(m.payload.u64); + msg->payload.u64 = 1 << VHOST_USER_PROTOCOL_F_LOG_SHMFD; + msg->payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_CONFIG; + if (s->queues > 1) { + msg->payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_MQ; + } + qemu_chr_fe_write_all(chr, (uint8_t *)msg, VHOST_USER_HDR_SIZE + msg->size); +} + +static void vu_blk_get_config(TestServer *s, CharBackend *chr, + VhostUserMsg *msg) +{ + VhostUserConfig *config; + struct virtio_blk_config *blk_config; + + config = &msg->payload.config; + memset(config, 0, sizeof(*config)); + config->size = sizeof(*blk_config); + + blk_config = (struct virtio_blk_config *)&config->region; + /* + * Represent 128Mb test disk, with no real backend, just + * to test vhost-user functionality. + */ + blk_config->capacity = 262144; + blk_config->size_max = 0x20000; + blk_config->seg_max = 0x7e; + blk_config->blk_size = 512; + blk_config->min_io_size = 0x1; + blk_config->num_queues = 0x1; + + msg->size = sizeof(*config) - sizeof(config->region) + config->size; + msg->flags |= VHOST_USER_REPLY_MASK; + qemu_chr_fe_write_all(chr, (uint8_t *)msg, VHOST_USER_HDR_SIZE + msg->size); +} + /* Each VHOST-USER device should have its ops structure defined. */ static struct vhost_user_ops g_vu_net_ops = { .type = VHOST_USER_NET, @@ -970,6 +1084,17 @@ static struct vhost_user_ops g_vu_net_ops = { .get_protocol_features = vu_net_get_protocol_features, }; +static struct vhost_user_ops g_vu_blk_ops = { + .type = VHOST_USER_BLK, + + .append_opts = append_vhost_blk_opts, + .driver_init = vu_blk_driver_init, + + .set_features = vu_blk_set_features, + .get_protocol_features = vu_blk_get_protocol_features, + .get_config = vu_blk_get_config, +}; + static void register_vhost_user_test(void) { QOSGraphTestOptions opts = { @@ -1015,5 +1140,16 @@ static void register_vhost_user_test(void) qos_add_test("vhost-user/multiqueue", "virtio-net", test_multiqueue, &opts); + opts.edge.extra_device_opts = NULL; + + /* vhost-user-blk tests */ + opts.arg = &g_vu_blk_ops; + opts.before = vhost_user_test_setup_reconnect; + qos_add_test("reconnect", "vhost-user-blk", + test_reconnect, &opts); + + opts.before = vhost_user_test_setup_memfd; + qos_add_test("migrate", "vhost-user-blk", + test_migrate, &opts); } libqos_init(register_vhost_user_test); From patchwork Mon Aug 24 08:39:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25FE0C433E1 for ; Mon, 24 Aug 2020 08:43:28 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E60962087D for ; Mon, 24 Aug 2020 08:43:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="ZM5/PiAs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E60962087D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:47136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA84d-0006iL-7t for qemu-devel@archiver.kernel.org; Mon, 24 Aug 2020 04:43:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58794) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA825-00024h-47; Mon, 24 Aug 2020 04:40:49 -0400 Received: from forwardcorp1o.mail.yandex.net ([2a02:6b8:0:1a2d::193]:60064) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA822-0004yU-Tv; Mon, 24 Aug 2020 04:40:48 -0400 Received: from myt5-23f0be3aa648.qloud-c.yandex.net (myt5-23f0be3aa648.qloud-c.yandex.net [IPv6:2a02:6b8:c12:3e29:0:640:23f0:be3a]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 041EE2E15D5; Mon, 24 Aug 2020 11:40:45 +0300 (MSK) Received: from myt4-18a966dbd9be.qloud-c.yandex.net (myt4-18a966dbd9be.qloud-c.yandex.net [2a02:6b8:c00:12ad:0:640:18a9:66db]) by myt5-23f0be3aa648.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id C6NFGpzPhY-egvKwCiq; Mon, 24 Aug 2020 11:40:44 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598258444; bh=q0+U/QfA4vXjDbbbTSlxDOvBDrzv7zTyuZdj9kiei3s=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=ZM5/PiAsRzOcn62ngg0MKB7uIeI/pG7FBSg2r2hBQq/KF/5FN0ZyF78G6/pYGqXF+ jugxSiKeTdSAgZ+IQdjWiwKZ1VTyKZc1nzJxcQyR9BWJGLe/xCaOO3Vaz9IS9Jo0Yz 1tsKRtqzUsbjYbrAZke+BhAOlVa31UbxXmu2PcWo= Authentication-Results: myt5-23f0be3aa648.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b080:6622::1:7]) by myt4-18a966dbd9be.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id B7dErHoRhW-egl46IZD; Mon, 24 Aug 2020 11:40:42 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v2 6/7] tests/qtest/vhost-user-test: add migrate_reconnect test Date: Mon, 24 Aug 2020 11:39:31 +0300 Message-Id: <7f5dfc30dcad34a7e2e7525e833e76ac6bfeef81.1598257839.git.dimastep@yandex-team.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=2a02:6b8:0:1a2d::193; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add new migrate_reconnect test for the vhost-user-blk device. Perform a disconnect after sending response for the VHOST_USER_SET_LOG_BASE command. Signed-off-by: Dima Stepanov --- tests/qtest/vhost-user-test.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c index 9b6e202..6b4c147 100644 --- a/tests/qtest/vhost-user-test.c +++ b/tests/qtest/vhost-user-test.c @@ -146,6 +146,7 @@ static VhostUserMsg m __attribute__ ((unused)); enum { TEST_FLAGS_OK, TEST_FLAGS_DISCONNECT, + TEST_FLAGS_MIGRATE_DISCONNECT, TEST_FLAGS_BAD, TEST_FLAGS_END, }; @@ -436,6 +437,15 @@ static void chr_read(void *opaque, const uint8_t *buf, int size) qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE); g_cond_broadcast(&s->data_cond); + /* + * Perform disconnect after sending a response. In this + * case the next write command on the QEMU side (for now + * it is SET_FEATURES will return -1, because of disconnect. + */ + if (s->test_flags == TEST_FLAGS_MIGRATE_DISCONNECT) { + qemu_chr_fe_disconnect(chr); + s->test_flags = TEST_FLAGS_BAD; + } break; case VHOST_USER_SET_VRING_BASE: @@ -737,6 +747,17 @@ static void *vhost_user_test_setup_memfd(GString *cmd_line, void *arg) return server; } +static void *vhost_user_test_setup_migrate_reconnect(GString *cmd_line, + void *arg) +{ + TestServer *server; + + server = vhost_user_test_setup_memfd(cmd_line, arg); + server->test_flags = TEST_FLAGS_MIGRATE_DISCONNECT; + + return server; +} + static void test_read_guest_mem(void *obj, void *arg, QGuestAllocator *alloc) { TestServer *server = arg; @@ -1151,5 +1172,9 @@ static void register_vhost_user_test(void) opts.before = vhost_user_test_setup_memfd; qos_add_test("migrate", "vhost-user-blk", test_migrate, &opts); + + opts.before = vhost_user_test_setup_migrate_reconnect; + qos_add_test("migrate_reconnect", "vhost-user-blk", + test_migrate, &opts); } libqos_init(register_vhost_user_test);