From patchwork Mon Aug 31 09:27:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3559C433E2 for ; Mon, 31 Aug 2020 09:28:54 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9B2DE208DB for ; Mon, 31 Aug 2020 09:28:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="TnEo8cwD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B2DE208DB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kCg7R-00039n-Qh for qemu-devel@archiver.kernel.org; Mon, 31 Aug 2020 05:28:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55636) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6Z-0001Ue-7P; Mon, 31 Aug 2020 05:27:59 -0400 Received: from forwardcorp1p.mail.yandex.net ([2a02:6b8:0:1472:2741:0:8b6:217]:59602) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6W-0004zG-Pm; Mon, 31 Aug 2020 05:27:58 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 38D132E1590; Mon, 31 Aug 2020 12:27:51 +0300 (MSK) Received: from sas1-58a37b48fb94.qloud-c.yandex.net (sas1-58a37b48fb94.qloud-c.yandex.net [2a02:6b8:c08:1d1b:0:640:58a3:7b48]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id nDlPDWXa4i-Rov4wBGi; Mon, 31 Aug 2020 12:27:51 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598866071; bh=mkJN5lUBUWPh+shtqKT1+IzCrqgHMkCefejVB9m0YBA=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=TnEo8cwDgGNwkKLurN6wngR3xc/Ib5aCmlwgSrBy7fe6enLQhEnt7s+tdonW+lRTj oqUW/IMLYjzSFiSQwCh/bZ8GJdl0G+p0VDbPe+tSxrtP+hrjJ0sxarrKebBL4RgJT5 rQ8kxVKssNN/5H8QcuqxE5pOzc3/f8EQuR0JETck= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:216::1:a]) by sas1-58a37b48fb94.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id yaozkQMlmT-Rom8h0Zl; Mon, 31 Aug 2020 12:27:50 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v3 1/7] vhost: recheck dev state in the vhost_migration_log routine Date: Mon, 31 Aug 2020 12:27:17 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=2a02:6b8:0:1472:2741:0:8b6:217; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/31 04:24:12 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" vhost-user devices can get a disconnect in the middle of the VHOST-USER handshake on the migration start. If disconnect event happened right before sending next VHOST-USER command, then the vhost_dev_set_log() call in the vhost_migration_log() function will return error. This error will lead to the assert() and close the QEMU migration source process. For the vhost-user devices the disconnect event should not break the migration process, because: - the device will be in the stopped state, so it will not be changed during migration - if reconnect will be made the migration log will be reinitialized as part of reconnect/init process: #0 vhost_log_global_start (listener=0x563989cf7be0) at hw/virtio/vhost.c:920 #1 0x000056398603d8bc in listener_add_address_space (listener=0x563989cf7be0, as=0x563986ea4340 ) at softmmu/memory.c:2664 #2 0x000056398603dd30 in memory_listener_register (listener=0x563989cf7be0, as=0x563986ea4340 ) at softmmu/memory.c:2740 #3 0x0000563985fd6956 in vhost_dev_init (hdev=0x563989cf7bd8, opaque=0x563989cf7e30, backend_type=VHOST_BACKEND_TYPE_USER, busyloop_timeout=0) at hw/virtio/vhost.c:1385 #4 0x0000563985f7d0b8 in vhost_user_blk_connect (dev=0x563989cf7990) at hw/block/vhost-user-blk.c:315 #5 0x0000563985f7d3f6 in vhost_user_blk_event (opaque=0x563989cf7990, event=CHR_EVENT_OPENED) at hw/block/vhost-user-blk.c:379 Update the vhost-user-blk device with the internal started_vu field which will be used for initialization (vhost_user_blk_start) and clean up (vhost_user_blk_stop). This additional flag in the VhostUserBlk structure will be used to track whether the device really needs to be stopped and cleaned up on a vhost-user level. The disconnect event will set the overall VHOST device (not vhost-user) to the stopped state, so it can be used by the general vhost_migration_log routine. Such approach could be propogated to the other vhost-user devices, but better idea is just to make the same connect/disconnect code for all the vhost-user devices. This migration issue was slightly discussed earlier: - https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg01509.html - https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg05241.html Signed-off-by: Dima Stepanov Reviewed-by: Raphael Norwitz --- hw/block/vhost-user-blk.c | 19 ++++++++++++++++--- hw/virtio/vhost.c | 27 ++++++++++++++++++++++++--- include/hw/virtio/vhost-user-blk.h | 10 ++++++++++ 3 files changed, 50 insertions(+), 6 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 39aec42..a076b1e 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -150,6 +150,7 @@ static int vhost_user_blk_start(VirtIODevice *vdev) error_report("Error starting vhost: %d", -ret); goto err_guest_notifiers; } + s->started_vu = true; /* guest_notifier_mask/pending not used yet, so just unmask * everything here. virtio-pci will do the right thing by @@ -175,6 +176,11 @@ static void vhost_user_blk_stop(VirtIODevice *vdev) VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); int ret; + if (!s->started_vu) { + return; + } + s->started_vu = false; + if (!k->set_guest_notifiers) { return; } @@ -341,9 +347,7 @@ static void vhost_user_blk_disconnect(DeviceState *dev) } s->connected = false; - if (s->dev.started) { - vhost_user_blk_stop(vdev); - } + vhost_user_blk_stop(vdev); vhost_dev_cleanup(&s->dev); } @@ -399,6 +403,15 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event) NULL, NULL, false); aio_bh_schedule_oneshot(ctx, vhost_user_blk_chr_closed_bh, opaque); } + + /* + * Move vhost device to the stopped state. The vhost-user device + * will be clean up and disconnected in BH. This can be useful in + * the vhost migration code. If disconnect was caught there is an + * option for the general vhost code to get the dev state without + * knowing its type (in this case vhost-user). + */ + s->dev.started = false; break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 1a1384e..ffef7ab 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -861,21 +861,42 @@ static int vhost_migration_log(MemoryListener *listener, bool enable) dev->log_enabled = enable; return 0; } + + r = 0; if (!enable) { r = vhost_dev_set_log(dev, false); if (r < 0) { - return r; + goto check_dev_state; } vhost_log_put(dev, false); } else { vhost_dev_log_resize(dev, vhost_get_log_size(dev)); r = vhost_dev_set_log(dev, true); if (r < 0) { - return r; + goto check_dev_state; } } + +check_dev_state: dev->log_enabled = enable; - return 0; + /* + * vhost-user-* devices could change their state during log + * initialization due to disconnect. So check dev state after + * vhost communication. + */ + if (!dev->started) { + /* + * Since device is in the stopped state, it is okay for + * migration. Return success. + */ + r = 0; + } + if (r) { + /* An error is occured. */ + dev->log_enabled = false; + } + + return r; } static void vhost_log_global_start(MemoryListener *listener) diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h index 292d171..4d6f4c0 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -40,7 +40,17 @@ typedef struct VHostUserBlk { VhostUserState vhost_user; struct vhost_virtqueue *vhost_vqs; VirtQueue **virtqs; + + /* + * There are at least two steps of initialization of the + * vhost-user device. The first is a "connect" step and + * second is a "start" step. Make a separation between + * those initialization phases by using two fields. + */ + /* vhost_user_blk_connect/vhost_user_blk_disconnect */ bool connected; + /* vhost_user_blk_start/vhost_user_blk_stop */ + bool started_vu; } VHostUserBlk; #endif From patchwork Mon Aug 31 09:27:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C48C433E6 for ; Mon, 31 Aug 2020 09:30:40 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 31F65206F0 for ; Mon, 31 Aug 2020 09:30:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="zf3Lzc2t" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31F65206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53664 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kCg99-0005Yh-C9 for qemu-devel@archiver.kernel.org; Mon, 31 Aug 2020 05:30:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55662) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6b-0001YW-NS; Mon, 31 Aug 2020 05:28:01 -0400 Received: from forwardcorp1p.mail.yandex.net ([77.88.29.217]:58212) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6Z-0004zq-Cz; Mon, 31 Aug 2020 05:28:01 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id 1C7C92E1666; Mon, 31 Aug 2020 12:27:57 +0300 (MSK) Received: from sas1-58a37b48fb94.qloud-c.yandex.net (sas1-58a37b48fb94.qloud-c.yandex.net [2a02:6b8:c08:1d1b:0:640:58a3:7b48]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id YIJyRZsaXO-RsvKR2Ht; Mon, 31 Aug 2020 12:27:56 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598866076; bh=gDOPq1RV3CLuGyPljyeYmAP4zg0EO7xHieFeyJevpn0=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=zf3Lzc2t6+6X9RKsKEw3iaTl3BgCuO8FGwaoVmEBF0JgrKdLnSCBdqxilkBi1LHMF v9FjXHBDzxusBply0wYBwjfy22x7JoaKSpAgb67mbgziuHiqFLwAxAc1alDDop1JmE TRXOJjytXQfjt5ZscZ4USXHs2Ad7er2xo1MOAXH4= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:216::1:a]) by sas1-58a37b48fb94.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id yaozkQMlmT-Rsm8S903; Mon, 31 Aug 2020 12:27:54 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v3 3/7] tests/qtest/vhost-user-test: prepare the tests for adding new dev class Date: Mon, 31 Aug 2020 12:27:19 +0300 Message-Id: <769cf817bad80e2534093e5887837b1c5e4675e2.1598865610.git.dimastep@yandex-team.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=77.88.29.217; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/31 04:37:44 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" For now only vhost-user-net device is supported by the test. Other vhost-user devices are not tested. As a first step make source code refactoring so new devices can reuse the same test routines. To make this provide a new vhost_user_ops structure with the methods to initialize device, its command line or make a proper vhost-user responses. Signed-off-by: Dima Stepanov --- tests/qtest/vhost-user-test.c | 105 ++++++++++++++++++++++++++++++------------ 1 file changed, 76 insertions(+), 29 deletions(-) diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c index 9ee0f1e..3df5322 100644 --- a/tests/qtest/vhost-user-test.c +++ b/tests/qtest/vhost-user-test.c @@ -135,6 +135,10 @@ enum { TEST_FLAGS_END, }; +enum { + VHOST_USER_NET, +}; + typedef struct TestServer { gchar *socket_path; gchar *mig_path; @@ -154,10 +158,25 @@ typedef struct TestServer { bool test_fail; int test_flags; int queues; + struct vhost_user_ops *vu_ops; } TestServer; +struct vhost_user_ops { + /* Device types. */ + int type; + void (*append_opts)(TestServer *s, GString *cmd_line, + const char *chr_opts); + + /* VHOST-USER commands. */ + void (*set_features)(TestServer *s, CharBackend *chr, + VhostUserMsg *msg); + void (*get_protocol_features)(TestServer *s, + CharBackend *chr, VhostUserMsg *msg); +}; + static const char *init_hugepagefs(void); -static TestServer *test_server_new(const gchar *name); +static TestServer *test_server_new(const gchar *name, + struct vhost_user_ops *ops); static void test_server_free(TestServer *server); static void test_server_listen(TestServer *server); @@ -167,7 +186,7 @@ enum test_memfd { TEST_MEMFD_NO, }; -static void append_vhost_opts(TestServer *s, GString *cmd_line, +static void append_vhost_net_opts(TestServer *s, GString *cmd_line, const char *chr_opts) { g_string_append_printf(cmd_line, QEMU_CMD_CHR QEMU_CMD_NETDEV, @@ -332,25 +351,15 @@ static void chr_read(void *opaque, const uint8_t *buf, int size) break; case VHOST_USER_SET_FEATURES: - g_assert_cmpint(msg.payload.u64 & (0x1ULL << VHOST_USER_F_PROTOCOL_FEATURES), - !=, 0ULL); - if (s->test_flags == TEST_FLAGS_DISCONNECT) { - qemu_chr_fe_disconnect(chr); - s->test_flags = TEST_FLAGS_BAD; + if (s->vu_ops->set_features) { + s->vu_ops->set_features(s, chr, &msg); } break; case VHOST_USER_GET_PROTOCOL_FEATURES: - /* send back features to qemu */ - msg.flags |= VHOST_USER_REPLY_MASK; - msg.size = sizeof(m.payload.u64); - msg.payload.u64 = 1 << VHOST_USER_PROTOCOL_F_LOG_SHMFD; - msg.payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_CROSS_ENDIAN; - if (s->queues > 1) { - msg.payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_MQ; + if (s->vu_ops->get_protocol_features) { + s->vu_ops->get_protocol_features(s, chr, &msg); } - p = (uint8_t *) &msg; - qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE + msg.size); break; case VHOST_USER_GET_VRING_BASE: @@ -467,7 +476,8 @@ static const char *init_hugepagefs(void) #endif } -static TestServer *test_server_new(const gchar *name) +static TestServer *test_server_new(const gchar *name, + struct vhost_user_ops *ops) { TestServer *server = g_new0(TestServer, 1); char template[] = "/tmp/vhost-test-XXXXXX"; @@ -495,6 +505,7 @@ static TestServer *test_server_new(const gchar *name) server->log_fd = -1; server->queues = 1; + server->vu_ops = ops; return server; } @@ -669,11 +680,11 @@ static void vhost_user_test_cleanup(void *s) static void *vhost_user_test_setup(GString *cmd_line, void *arg) { - TestServer *server = test_server_new("vhost-user-test"); + TestServer *server = test_server_new("vhost-user-test", arg); test_server_listen(server); append_mem_opts(server, cmd_line, 256, TEST_MEMFD_AUTO); - append_vhost_opts(server, cmd_line, ""); + server->vu_ops->append_opts(server, cmd_line, ""); g_test_queue_destroy(vhost_user_test_cleanup, server); @@ -682,11 +693,11 @@ static void *vhost_user_test_setup(GString *cmd_line, void *arg) static void *vhost_user_test_setup_memfd(GString *cmd_line, void *arg) { - TestServer *server = test_server_new("vhost-user-test"); + TestServer *server = test_server_new("vhost-user-test", arg); test_server_listen(server); append_mem_opts(server, cmd_line, 256, TEST_MEMFD_YES); - append_vhost_opts(server, cmd_line, ""); + server->vu_ops->append_opts(server, cmd_line, ""); g_test_queue_destroy(vhost_user_test_cleanup, server); @@ -720,7 +731,7 @@ static void test_migrate(void *obj, void *arg, QGuestAllocator *alloc) return; } - dest = test_server_new("dest"); + dest = test_server_new("dest", s->vu_ops); dest_cmdline = g_string_new(qos_get_current_command_line()); uri = g_strdup_printf("%s%s", "unix:", dest->mig_path); @@ -730,7 +741,7 @@ static void test_migrate(void *obj, void *arg, QGuestAllocator *alloc) test_server_listen(dest); g_string_append_printf(dest_cmdline, " -incoming %s", uri); append_mem_opts(dest, dest_cmdline, 256, TEST_MEMFD_AUTO); - append_vhost_opts(dest, dest_cmdline, ""); + dest->vu_ops->append_opts(dest, dest_cmdline, ""); to = qtest_init(dest_cmdline->str); /* This would be where you call qos_allocate_objects(to, NULL), if you want @@ -831,11 +842,11 @@ connect_thread(gpointer data) static void *vhost_user_test_setup_reconnect(GString *cmd_line, void *arg) { - TestServer *s = test_server_new("reconnect"); + TestServer *s = test_server_new("reconnect", arg); g_thread_new("connect", connect_thread, s); append_mem_opts(s, cmd_line, 256, TEST_MEMFD_AUTO); - append_vhost_opts(s, cmd_line, ",server"); + s->vu_ops->append_opts(s, cmd_line, ",server"); g_test_queue_destroy(vhost_user_test_cleanup, s); @@ -866,13 +877,13 @@ static void test_reconnect(void *obj, void *arg, QGuestAllocator *alloc) static void *vhost_user_test_setup_connect_fail(GString *cmd_line, void *arg) { - TestServer *s = test_server_new("connect-fail"); + TestServer *s = test_server_new("connect-fail", arg); s->test_fail = true; g_thread_new("connect", connect_thread, s); append_mem_opts(s, cmd_line, 256, TEST_MEMFD_AUTO); - append_vhost_opts(s, cmd_line, ",server"); + s->vu_ops->append_opts(s, cmd_line, ",server"); g_test_queue_destroy(vhost_user_test_cleanup, s); @@ -881,13 +892,13 @@ static void *vhost_user_test_setup_connect_fail(GString *cmd_line, void *arg) static void *vhost_user_test_setup_flags_mismatch(GString *cmd_line, void *arg) { - TestServer *s = test_server_new("flags-mismatch"); + TestServer *s = test_server_new("flags-mismatch", arg); s->test_flags = TEST_FLAGS_DISCONNECT; g_thread_new("connect", connect_thread, s); append_mem_opts(s, cmd_line, 256, TEST_MEMFD_AUTO); - append_vhost_opts(s, cmd_line, ",server"); + s->vu_ops->append_opts(s, cmd_line, ",server"); g_test_queue_destroy(vhost_user_test_cleanup, s); @@ -924,11 +935,47 @@ static void test_multiqueue(void *obj, void *arg, QGuestAllocator *alloc) wait_for_rings_started(s, s->queues * 2); } +static void vu_net_set_features(TestServer *s, CharBackend *chr, + VhostUserMsg *msg) +{ + g_assert_cmpint(msg->payload.u64 & + (0x1ULL << VHOST_USER_F_PROTOCOL_FEATURES), !=, 0ULL); + if (s->test_flags == TEST_FLAGS_DISCONNECT) { + qemu_chr_fe_disconnect(chr); + s->test_flags = TEST_FLAGS_BAD; + } +} + +static void vu_net_get_protocol_features(TestServer *s, CharBackend *chr, + VhostUserMsg *msg) +{ + /* send back features to qemu */ + msg->flags |= VHOST_USER_REPLY_MASK; + msg->size = sizeof(m.payload.u64); + msg->payload.u64 = 1 << VHOST_USER_PROTOCOL_F_LOG_SHMFD; + msg->payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_CROSS_ENDIAN; + if (s->queues > 1) { + msg->payload.u64 |= 1 << VHOST_USER_PROTOCOL_F_MQ; + } + qemu_chr_fe_write_all(chr, (uint8_t *)msg, VHOST_USER_HDR_SIZE + msg->size); +} + +/* Each VHOST-USER device should have its ops structure defined. */ +static struct vhost_user_ops g_vu_net_ops = { + .type = VHOST_USER_NET, + + .append_opts = append_vhost_net_opts, + + .set_features = vu_net_set_features, + .get_protocol_features = vu_net_get_protocol_features, +}; + static void register_vhost_user_test(void) { QOSGraphTestOptions opts = { .before = vhost_user_test_setup, .subprocess = true, + .arg = &g_vu_net_ops, }; qemu_add_opts(&qemu_chardev_opts); From patchwork Mon Aug 31 09:27:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E70BC433E6 for ; Mon, 31 Aug 2020 09:32:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 06663206F0 for ; Mon, 31 Aug 2020 09:32:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="SjXXhxVc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06663206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57980 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kCgAc-0007QU-65 for qemu-devel@archiver.kernel.org; Mon, 31 Aug 2020 05:32:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55668) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6c-0001aE-IX; Mon, 31 Aug 2020 05:28:02 -0400 Received: from forwardcorp1p.mail.yandex.net ([77.88.29.217]:58258) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6a-00050E-Py; Mon, 31 Aug 2020 05:28:02 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1p.mail.yandex.net (Yandex) with ESMTP id E92F42E1592; Mon, 31 Aug 2020 12:27:58 +0300 (MSK) Received: from sas1-58a37b48fb94.qloud-c.yandex.net (sas1-58a37b48fb94.qloud-c.yandex.net [2a02:6b8:c08:1d1b:0:640:58a3:7b48]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id EC2gTuV1PB-RwveTQa1; Mon, 31 Aug 2020 12:27:58 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598866078; bh=UciJc8X9njWrEUHJFothSUSQxAE9xsIR5v3e7XbMiqU=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=SjXXhxVcZ5yfjC9ANLoye+PAjU4F4d4Zp34VnndKZJhK1auWBLX8jgJUdf+s293l4 eU1loKmPvMKieHQrqezSW+Rt2WF7d3+QDdZYoiYoCineYRcVyle6P4lgAhpiEjPmOW q4Be38qNYWCMULqKSFuHiX2wHn9NYl1InGxOGE7s= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:216::1:a]) by sas1-58a37b48fb94.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id yaozkQMlmT-Rwm8kf8T; Mon, 31 Aug 2020 12:27:58 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v3 4/7] tests/qtest/libqos/virtio-blk: add support for vhost-user-blk Date: Mon, 31 Aug 2020 12:27:20 +0300 Message-Id: <50d827a7b383de531ac1452e1e0ce3ad961d00af.1598865610.git.dimastep@yandex-team.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=77.88.29.217; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1p.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/31 04:37:44 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add support for the vhost-user-blk-pci device. This node can be used by the vhost-user-blk tests. Tests for the vhost-user-blk device are added in the following patches. Signed-off-by: Dima Stepanov --- tests/qtest/libqos/virtio-blk.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tests/qtest/libqos/virtio-blk.c b/tests/qtest/libqos/virtio-blk.c index 5da0259..959c5dc 100644 --- a/tests/qtest/libqos/virtio-blk.c +++ b/tests/qtest/libqos/virtio-blk.c @@ -36,6 +36,9 @@ static void *qvirtio_blk_get_driver(QVirtioBlk *v_blk, if (!g_strcmp0(interface, "virtio")) { return v_blk->vdev; } + if (!g_strcmp0(interface, "vhost-user-blk")) { + return v_blk; + } fprintf(stderr, "%s not present in virtio-blk-device\n", interface); g_assert_not_reached(); @@ -120,6 +123,17 @@ static void virtio_blk_register_nodes(void) qos_node_produces("virtio-blk-pci", "virtio-blk"); g_free(arg); + + /* vhost-user-blk-pci */ + arg = g_strdup_printf("id=drv0,chardev=chdev0,addr=%x.%x", + PCI_SLOT, PCI_FN); + opts.extra_device_opts = arg; + add_qpci_address(&opts, &addr); + qos_node_create_driver("vhost-user-blk-pci", virtio_blk_pci_create); + qos_node_consumes("vhost-user-blk-pci", "pci-bus", &opts); + qos_node_produces("vhost-user-blk-pci", "vhost-user-blk"); + + g_free(arg); } libqos_init(virtio_blk_register_nodes); From patchwork Mon Aug 31 09:27:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dima Stepanov X-Patchwork-Id: 275134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 262F7C433E2 for ; Mon, 31 Aug 2020 09:34:19 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDB60206F0 for ; Mon, 31 Aug 2020 09:34:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="CLU2rzZ5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDB60206F0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kCgCg-0001Ci-1R for qemu-devel@archiver.kernel.org; Mon, 31 Aug 2020 05:34:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6h-0001nC-DQ; Mon, 31 Aug 2020 05:28:07 -0400 Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]:54564) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kCg6f-00050l-FT; Mon, 31 Aug 2020 05:28:07 -0400 Received: from sas1-ec30c78b6c5b.qloud-c.yandex.net (sas1-ec30c78b6c5b.qloud-c.yandex.net [IPv6:2a02:6b8:c14:2704:0:640:ec30:c78b]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 8A9F92E16F1; Mon, 31 Aug 2020 12:28:02 +0300 (MSK) Received: from sas1-58a37b48fb94.qloud-c.yandex.net (sas1-58a37b48fb94.qloud-c.yandex.net [2a02:6b8:c08:1d1b:0:640:58a3:7b48]) by sas1-ec30c78b6c5b.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id IDywwa8oyI-S1vGRjJ5; Mon, 31 Aug 2020 12:28:02 +0300 Precedence: bulk DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1598866082; bh=+qjcP8GjY4KQVRU34GsQvdwzzL2IhmdJebgwakks6KM=; h=In-Reply-To:In-Reply-To:Message-Id:References:References:Date: Subject:To:From:Cc; b=CLU2rzZ5QAyqNA79dpjQzZiUG4hWU0A1bdLhc3hzavTtjt6IM9G1XN33zyOQx4EkY yECj+ci1dStP3OPxaSgZlZlGhox/ZhfpL2rNvpiQbTmjRQqK4kYSm/G9kCShygEwnk 5DnfbJ2+wk4ifZgucXeTo9fd6J7rykptLRPP/Rtw= Authentication-Results: sas1-ec30c78b6c5b.qloud-c.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:216::1:a]) by sas1-58a37b48fb94.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id yaozkQMlmT-S1m8VtUD; Mon, 31 Aug 2020 12:28:01 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Dima Stepanov To: qemu-devel@nongnu.org Subject: [PATCH v3 6/7] tests/qtest/vhost-user-test: add migrate_reconnect test Date: Mon, 31 Aug 2020 12:27:22 +0300 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: In-Reply-To: References: Received-SPF: pass client-ip=95.108.205.193; envelope-from=dimastep@yandex-team.ru; helo=forwardcorp1o.mail.yandex.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/31 05:28:02 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, lvivier@redhat.com, thuth@redhat.com, qemu-block@nongnu.org, mst@redhat.com, jasowang@redhat.com, dgilbert@redhat.com, mreitz@redhat.com, fengli@smartx.com, yc-core@yandex-team.ru, pbonzini@redhat.com, raphael.norwitz@nutanix.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add new migrate_reconnect test for the vhost-user-blk device. Perform a disconnect after sending response for the VHOST_USER_SET_LOG_BASE command. Signed-off-by: Dima Stepanov --- tests/qtest/vhost-user-test.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c index a8af613..4b715d3 100644 --- a/tests/qtest/vhost-user-test.c +++ b/tests/qtest/vhost-user-test.c @@ -146,6 +146,7 @@ static VhostUserMsg m __attribute__ ((unused)); enum { TEST_FLAGS_OK, TEST_FLAGS_DISCONNECT, + TEST_FLAGS_MIGRATE_DISCONNECT, TEST_FLAGS_BAD, TEST_FLAGS_END, }; @@ -436,6 +437,15 @@ static void chr_read(void *opaque, const uint8_t *buf, int size) qemu_chr_fe_write_all(chr, p, VHOST_USER_HDR_SIZE); g_cond_broadcast(&s->data_cond); + /* + * Perform disconnect after sending a response. In this + * case the next write command on the QEMU side (for now + * it is SET_FEATURES will return -1, because of disconnect. + */ + if (s->test_flags == TEST_FLAGS_MIGRATE_DISCONNECT) { + qemu_chr_fe_disconnect(chr); + s->test_flags = TEST_FLAGS_BAD; + } break; case VHOST_USER_SET_VRING_BASE: @@ -737,6 +747,17 @@ static void *vhost_user_test_setup_memfd(GString *cmd_line, void *arg) return server; } +static void *vhost_user_test_setup_migrate_reconnect(GString *cmd_line, + void *arg) +{ + TestServer *server; + + server = vhost_user_test_setup_memfd(cmd_line, arg); + server->test_flags = TEST_FLAGS_MIGRATE_DISCONNECT; + + return server; +} + static void test_read_guest_mem(void *obj, void *arg, QGuestAllocator *alloc) { TestServer *server = arg; @@ -1150,5 +1171,9 @@ static void register_vhost_user_test(void) opts.before = vhost_user_test_setup_memfd; qos_add_test("migrate", "vhost-user-blk", test_migrate, &opts); + + opts.before = vhost_user_test_setup_migrate_reconnect; + qos_add_test("migrate_reconnect", "vhost-user-blk", + test_migrate, &opts); } libqos_init(register_vhost_user_test);