From patchwork Mon Jun 1 18:10:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 166FDC433E0 for ; Mon, 1 Jun 2020 18:13:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DA5F920776 for ; Mon, 1 Jun 2020 18:13:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="j1yyaMNW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA5F920776 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50718 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfovl-0006s3-0e for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:13:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37664) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouR-0004nI-G2; Mon, 01 Jun 2020 14:11:39 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouQ-00026c-OL; Mon, 01 Jun 2020 14:11:39 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h+t5KfniMETPi1ayaxrRUHTvCn3oeVKGzFXLT9RfYkFM7P22NV4VET2H+D/U+b8zMrU8KD1kJJN7M7C898GGKOuve252P3q/4crkEvGKzoG39/oGlRotijGQJdsBAp6f1oMre/OVOTu2HtXogYSqQ0r1kMRUe+RO3g7dRYp05kzPJDZLx9QTkCGC0GSIku3Q+W/mqVSWuhZMIUYG5kWNI7xc/qf+Rb+tMMI0gXXmLx//vRJr9SV3VACUyUW3av8h66qUqzfywyBJEMjE1hCCM6+c5uSX9fD359vAPhFHsOmE515MrN4d1fwZ0qPct6EnRchn1JVd5Ki/YeygiDT4WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3AiKf4IdqJuJVI4XnnAK3JP1HTdcTOI5m8XjztJ7fQ=; b=fyMaa5xKDhhvJGKrR0+gQSLh7wOP0Dmg6oM+whyRbTQxyGquj9YmLq88xjwvwaVSkNkkav10Xl16KbxZCKQpvIjocUyNb6Ys5X3aqA0MKcd2rrxegOy4Wo3xsGmbKBdV3igvKjzxkSAezVIX/24odFBAHVY4gtZSksDom3RJJ0zGbuurhYTkcXFWVBLufNKIBX5xdEO1k7YgZLoUMr4qdxCPOBwM8Ur9uDic42bfeuYYuntDBbxEqM76NHOa/gU4ZLq1xyTBnbaNmSh+kAUZpG11dB0s0lCRumcMIFMIdmkdV3DQW1aKSN2qbHarujJ80n3ytFeutdSj0/AjwrJnBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k3AiKf4IdqJuJVI4XnnAK3JP1HTdcTOI5m8XjztJ7fQ=; b=j1yyaMNW8gzVEiR6Z+dxy+N+RibYO/hYfhoA5GmrM7NwDeRdL3KwLtCb1QAttZNUqJ2di5w+oD1ins51bewx/Sr3tA2ThMgZUgS9hMg4A5UMLR1GzzCGQag9AJLfbCZfr5gpLSD8Jj/GAW9w0NL9ElLTUS464k0JMOmfWOJBADc= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:32 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:32 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 01/20] block/block-copy: block_copy_dirty_clusters: fix failure check Date: Mon, 1 Jun 2020 21:10:59 +0300 Message-Id: <20200601181118.579-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:31 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0af8347d-9b2f-4c10-588b-08d806573688 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +DehiY1v2FqMcSWtxbFUbISiCkOPQ2tyy8t6MmXH42glOcv/yG6alEBVHuiq9ys/XcLBodmDvgRfKyAGgoOLSimy8TTQGPE4HkXUtjQWsv23fr222E+kpKhXx0H91nZpra5JKqAHVoDAUs6F93ixEO7idbcJcjcgqBV5A+pshBxNFuaKif5i6VyIQ2KcYr4Rp5VGBa22nA8BNSrkFaD3mQHPbeoTS05MhqqtBYTvHUbnNXnPj42GrojQmV3h8ZPttZ/6OTnn5ijsp4tGpOYqHelwYbg6dKjkHBgYl7OVqGzxk7fDLRY/G48TSOmgALfnU15ha20vjEdh+ZKy0/I62DE94kROGMIkbKHf7ddmAV0BoSXmMcdp3N44UvLLQ/Q/0g7JSOfOXeIBMikcQI5nUil3sV9pZRbjIPtvI7ugPD8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(4744005)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007)(14143004); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: cCTlH+pPmBe/1fhuQQLHggSo28CJ15GUCjM42JoqO6wsHk6+Udkqam885LzPwJJ4OlQLaH/B/Ab1we5LFYphvpRLfW6Aj4FSWlKR73Vo4iwHNQjkxJF4ocXTmToNnmYVbCUtSNJx7LOFdMUMRxuTBB7W4utMEyHIkcaDZhqzS9Xjty6HVN8XF6DJAOAjUN+LjnREkQKxX/wIz+4QU8G0yUm0YIlc+Xg7mCKeoktlA91q8t3kVvjgIaV1pcPFPpMaXUhaeM0NIuCZVIB3+o6dz4nls/lOY7cjLEonm/WHpCfDw9sJYSc21bX2e5gc3AmaWDEDBUryxqlfVGgtx8vrhvNcVE4diCkSmKix4h84lV1z1Kr8jXVSpCUY82e7bD5b4Fjd07h5FvrAc4AQv0i5kYyrVSVqcUFbdhDiUPbSqKI86bF4S7VF9d6o1kpwfScLzjoRthUIn5KH4imCSA5x/cgChHelv/mnsuPVKIVIYzQ= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0af8347d-9b2f-4c10-588b-08d806573688 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:32.1743 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WiIPmIVVE/Xx/TVUop2ELuDa6VIu6FzIEQOugNf2914UZVFAtYoUGpo3k+vlIpmEDqKlRVHrbqP/2vrQL5WZdduC4NDKlP74coqx+MDIW3Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" ret may be > 0 on success path at this point. Fix assertion, which may crash currently. Fixes: 4ce5dd3e9b5ee0fac18625860eb3727399ee965e Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-copy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/block-copy.c b/block/block-copy.c index bb8d0569f2..f7428a7c08 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -622,8 +622,10 @@ out: * block_copy_task_run. If it fails, it means some task already failed * for real reason, let's return first failure. * Still, assert that we don't rewrite failure by success. + * + * Note: ret may be positive here because of block-status result. */ - assert(ret == 0 || aio_task_pool_status(aio) < 0); + assert(ret >= 0 || aio_task_pool_status(aio) < 0); ret = aio_task_pool_status(aio); aio_task_pool_free(aio); From patchwork Mon Jun 1 18:11:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E68B6C433DF for ; Mon, 1 Jun 2020 18:15:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ACDE52073B for ; Mon, 1 Jun 2020 18:15:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="co2Ut2up" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ACDE52073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59022 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfoyK-0001tG-SR for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:15:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37686) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouW-0004ws-Bh; Mon, 01 Jun 2020 14:11:44 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouV-00026c-C9; Mon, 01 Jun 2020 14:11:44 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QsEqA4FZmwCbOz9Cq9bcRo1VZ4tlGjWPQ+6k9sRZ3VhUg54Mp6tIe/SWX9ySoaAdKKGo8JsW84OQg2+KnZumqp9RfRGIXjcopNvtIrTEmh+Qv9Vf776a3+vynMdCxMJ7DZkAjxExCanlkfO7j2rm2trYFI8nNbJ9xcHNIhRV40IAM2olT73S2KywePzfS3zzvm0N8hCOPu3eQaZpvPW90hzbuCIGPsxO+c/7WNkZhgEUQYfnt1lw/Fy79RU83LxYQty3PlHkpBXbD3DH4eK/ID+3SPEG/YXRFE/K8yLJvPNBxoyVSXxED91b+W9t6AHdLYnIrOD4+8awZku59nIR8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B7Eoq722JIj/34+zavBcv7aOy9uvQayxsFq2nYvNM6M=; b=moxEfACG2rwHgTNgcIUz073/Qf+mK4XVZZ/J8ujARy3u7c0R8/SHYfw54erj6HB8hPrQSt+Ot8V6ZT2lWk6SL6yqDNGBXlMuFoYW6U1f9GgRJE4AYZuLY5aqNIbjqJacATdcXX+VWKzR63QTfavqH84PTs8RpYvz39vnvNBTuo8qkA5xU06vi4kSmR2wIj+YM6oaMIbynKwhsklEP1fFjpnKSOA9+VToh/erMSlZ/svB29Lss8BL6dLb9II1XirJq1FooHJpB3gSiqnCVDCFnZDQxBg9MBor8k1DWQ9FqoP0rDKyPmZzh4ruR/OnDgiiXVqR8UMMvlCSrvJf3cI2HQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B7Eoq722JIj/34+zavBcv7aOy9uvQayxsFq2nYvNM6M=; b=co2Ut2upqfwnsML7NApViB8b2dMQc0J7O1+Sg4nr47mQNJeOk20wxVSIvbVNWCAto0LjjFtgWT+bB1btG03yPH0T7g9Wzk8m7amUnECrN7f+Gx5hgcsZcL/Zx8rWIvc2b/G4DyoSpmCBghXjyb+hFZy8mK1ZLx1WktJQ0sgZRiA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:35 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:35 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 05/20] block/block-copy: implement block_copy_async Date: Mon, 1 Jun 2020 21:11:03 +0300 Message-Id: <20200601181118.579-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:34 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3b7f00bb-0765-46d2-9a95-08d8065738a2 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:22; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O387vFOASovECo9qBGVW3epKQDBwLzFAIHjNNe7lYjtNwdCrgCgMyf1jvE4RHh2ALG8lii55Z6J5ZarEzMfIdlgNQQGYlPf/yDJuxEm5z1Ced7wtgi69nurD5iL/hUL7sDp5KNowse7WiRszp+SURMswMyqFhOYjpm0uQJ/CTV1dlNmkn1MdVdxjhO/+tkp8STrWO/6rpGrkkfvX7Sv0nUI106JzoSqmUFRlQt0uTSZWludgb1OrRry3fRoECCqKk4Ra9Jo+UFO1K4wNqZon7LzsI/OAX6KHeYElai/yBrnmUl8pCn0BN1NwbO1pNKXOQGEaBgMR3oPtocRJoXQJckySfncHMI+VjyRB4nHd+fIPUCzuhCWWAKdBQNBkSWGU X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: eGEfu+LIfNTyv0tlixfSGZ3I7x090VjfWe0ynaTHmMoo7FNDwU+mkEZjtXnobO3cjLqlQFkCDBk7KuY5xq9Qb1SZeR8QIhSGGe8nEpr9LbAxKmC6Oec4vWWkups3Xzqd4xCers1jBA3UgctC2ula3NYcU8Bp2hwsH4iHpuoH7BV0+Wqpxih3C2uKbM/YqBS7hK6kg03dWSkxlskoM1EH6CeB1I8ag46V7mUPDmqrm+R3uPWbDsX+ikly+3EG6069t7Q1Z871Q6ymvJuX0b+MuhD+e0Z5grHFFYyLYHfQqfX6wbHMy7p4oCwQIKbJgUl7ZuWI4EFnRxFzfAesguKCCrqHWlAo+hN5SCZL7PAEpwkGIT9PjdkJMiMjcnfEm7laNMaZdqAvGNMLROF0cZc9P9IUp+iEa8nuJzh+uiv2jaW8FMZJV1N7NhsRTL4IJ12vZ1O+IY2s8gy8d3ipbPpEwW8KSomkm7cx26sDjn2y21s= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b7f00bb-0765-46d2-9a95-08d8065738a2 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:35.6216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tAlVUYrcQlCRkR4NGHIQg6UjimEfYhaZEACvGRtZSOzF/O7DOsR0Rg62eTo8RIrZdNVRfRDCwsM0wRaN8GxucsNqMLs1ISeznlGJLtFGfpw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We'll need async block-copy invocation to use in backup directly. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 13 +++++++++++++ block/block-copy.c | 40 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 6397505f30..ada0d99566 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -19,7 +19,10 @@ #include "qemu/co-shared-resource.h" typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque); +typedef void (*BlockCopyAsyncCallbackFunc)(int ret, bool error_is_read, + void *opaque); typedef struct BlockCopyState BlockCopyState; +typedef struct BlockCopyCallState BlockCopyCallState; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range, @@ -41,6 +44,16 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes, bool *error_is_read); +/* + * Run block-copy in a coroutine, return state pointer. If finished early + * returns NULL (@cb is called anyway). + */ +BlockCopyCallState *block_copy_async(BlockCopyState *s, + int64_t offset, int64_t bytes, + bool ratelimit, int max_workers, + int64_t max_chunk, + BlockCopyAsyncCallbackFunc cb); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); diff --git a/block/block-copy.c b/block/block-copy.c index 75882a094c..a0477d90f3 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -34,9 +34,11 @@ typedef struct BlockCopyCallState { BlockCopyState *s; int64_t offset; int64_t bytes; + BlockCopyAsyncCallbackFunc cb; /* State */ bool failed; + bool finished; /* OUT parameters */ bool error_is_read; @@ -676,6 +678,13 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) */ } while (ret > 0); + if (call_state->cb) { + call_state->cb(ret, call_state->error_is_read, + call_state->s->progress_opaque); + } + + call_state->finished = true; + return ret; } @@ -697,6 +706,37 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes, return ret; } +static void coroutine_fn block_copy_async_co_entry(void *opaque) +{ + block_copy_common(opaque); +} + +BlockCopyCallState *block_copy_async(BlockCopyState *s, + int64_t offset, int64_t bytes, + bool ratelimit, int max_workers, + int64_t max_chunk, + BlockCopyAsyncCallbackFunc cb) +{ + BlockCopyCallState *call_state = g_new(BlockCopyCallState, 1); + Coroutine *co = qemu_coroutine_create(block_copy_async_co_entry, + call_state); + + *call_state = (BlockCopyCallState) { + .s = s, + .offset = offset, + .bytes = bytes, + .cb = cb, + }; + + qemu_coroutine_enter(co); + + if (call_state->finished) { + g_free(call_state); + return NULL; + } + + return call_state; +} BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) { return s->copy_bitmap; From patchwork Mon Jun 1 18:11:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08E18C433E1 for ; Mon, 1 Jun 2020 18:19:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDE792068D for ; Mon, 1 Jun 2020 18:19:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="oMmjYehz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDE792068D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp2C-0005z6-25 for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:19:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37694) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouY-00052V-7A; Mon, 01 Jun 2020 14:11:46 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouX-00026c-1a; Mon, 01 Jun 2020 14:11:45 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QqzTAByICg9bl7mhpJDK9GdOn3zjN2a7afflnJcnEtg2RelUyz/L7Gc17Rvgw6ZSqbh8CQLMPbbHUg8WDEnmEO6uvrI60mxLnU8DiXZdtei2ikKSL0tR6Dpvzg2pBZES8QiUtaxPVxMyDAbsvHiuO+BytU3KK0X5eNzT3vw80T1WaLf6Z8c5nSeAkhDjcUqP40pfI1Do/n7RGglU89nX/beb63/HMuwOYK5ZYUCDEy7uQXh+yXkfNOJekSuKvMZcWCi10EgVdqIqhgYaUJjyK1f0EILjaq42ZI9qMcJh+h+Em/G82LJrXAPKeZFjSnBvF05NH7Qr/CDgJ+wlI8RXgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=85i5+xBw2B6rOENuXOGZJnaqvBK5TcYKVhlQx0CFcz4=; b=SvYU0gweRoDopflodnzzPdYypsD5KjXUtZ8+YFLlB8d2qdHI5zlSj/xQLqB4wML/q6MILnAqUZ5+C9Z9LVMG0/Y45jY74+yeNgxEHoJMBwlHq+kqaqVYwA1efWCwYIuNsLYQHEf2lFcxV6oVVa9UWs/kemPDURITqqZN84A73BJFWObEN5g/YjuvhOCoPUJ4rrs2LF6FndxKzyIU9aeJDsdLxZCqx2FRDxAKayGfcYJQGAXud4V3yrKkm6MjLX4NHQWd1vOCRyvCrfdBfcLJMWKkocuqi2YubT6KNXatsLP68BGIPEFt0z/r2sPF4/JNW8KiIH0uK8M8w9I/eenv5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=85i5+xBw2B6rOENuXOGZJnaqvBK5TcYKVhlQx0CFcz4=; b=oMmjYehz43lERJrNmEHfFFPrHJe/ipYkQqAe0O3vXZFuFi/3DE50kBweMGMCQU4WAA14eztHlUzkLpQPCB+jUp3/WixQmdP35L4Og+RrZXS4UazZ7baEp9R7IhId7irdEyjitkNG8Y1RGeyrXkFN4HpoMraQVs2JPCtYbya1DJU= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:37 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:37 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 07/20] block/block-copy: add ratelimit to block-copy Date: Mon, 1 Jun 2020 21:11:05 +0300 Message-Id: <20200601181118.579-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:36 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9269cd3c-beb5-476f-d1cf-08d8065739a2 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Kr2+j103aaNjRDv9Nal3anWEwQNqaVxJOLKYqVhto904H5zopXxofFN23QGqlR7+JPshwGnjTW7fAcCOU9EN+hc22Tg2bbuLdvZQ/cXz6yoA7poYdAXC/MGU9n+bl66eiZIs/lAen5D647xT57IqnVjXfkbqL7F9cV/RpYe/XrHurMfucQtYCa5RnuyNUpJFffr4BC0WvIrBrRcT79nMtx0KKhYORr0IVENtM3E8EXoSfahKxYYv/CxWMzeJD0Sg655bk1s0SnHr3717ULtUkRQ19YFTQvE0zQAEMWpJkdVCTt8wuU1BECvXFE9UKgrKs+f5XeDtSJcl3/LahlFYQdd1+/Jq0jflhSE108mbWImQltUnsp2p5dGoYOjOJNk9 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: Y+oCV6PbTcAIUoXkokxj55Zfm6vwbdnhf4PJ7D+MzQcJQFtroc53VBJlcQSUzYPYmiwV7Yi8Fp2xNIWazPbZRNHp3wVc8qP13LGtHQD8UXBy5dbXy6PQsUfpvvhcHahYKR+a+IPr5vsVngi1mUaEjep8BUwdt02GbUb1Yay5RifjSpl74QjuBWggSLxOfVaAeQe6YiCYXF7VuQqUOqEvrFeVLMXiteK98EkKgpjUMDK6vu0Sd0h+UVTIhwXK21C+nqaoJpQBteP8KyoWw4f2NUQYgRXO6qEr+nkorGZpyufVUDN+tmzUOiMBx681dqE9L7OAcMNi34U7Mf8ETY7j+034P4sY85hzEvVaTV9CWX0On80LS8SOTGUyvwn9b0fOrTJ6vqmN3SbTn9iQmpPPi7EdR/+YOsEGnaUKRGYTEFOSQqpz64LkH4VQ5NacX8M4g2kFLiGJ9BVIrSOflBo2JZnQrjZvug/foHVxBet32sw= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9269cd3c-beb5-476f-d1cf-08d8065739a2 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:37.3306 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: N+FPlSG32niV16jp/NfurEpflgExoC0HyZola6BgHroc2TmhWLhUaGMit5twiCHlH6jNNWy43KazQM1ITovhX/asbETOB8A77F8hb/pLL7U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We are going to directly use one async block-copy operation for backup job, so we need rate limitator. We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limitator for block-copy state and boolean flag for block-copy call state for actual limitation. Note, that we can't just calculate each chunk in limitator after successful copying: it will not save us from starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation: 1. If at the moment limit is not exceeded, create the request and account it immediately. 2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep). With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior). Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 8 +++++++ block/block-copy.c | 44 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index 600984c733..d40e691123 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -59,6 +59,14 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, int64_t max_chunk, BlockCopyAsyncCallbackFunc cb); +/* + * Set speed limit for block-copy instance. All block-copy operations related to + * this BlockCopyState will participate in speed calculation, but only + * block_copy_async calls with @ratelimit=true will be actually limited. + */ +void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_state, + uint64_t speed); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); diff --git a/block/block-copy.c b/block/block-copy.c index 4114d1fd25..851d9c8aaf 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -26,6 +26,7 @@ #define BLOCK_COPY_MAX_BUFFER (1 * MiB) #define BLOCK_COPY_MAX_MEM (128 * MiB) #define BLOCK_COPY_MAX_WORKERS 64 +#define BLOCK_COPY_SLICE_TIME 100000000ULL /* ns */ static coroutine_fn int block_copy_task_entry(AioTask *task); @@ -36,11 +37,13 @@ typedef struct BlockCopyCallState { int64_t bytes; int max_workers; int64_t max_chunk; + bool ratelimit; BlockCopyAsyncCallbackFunc cb; /* State */ bool failed; bool finished; + QemuCoSleepState *sleep_state; /* OUT parameters */ bool error_is_read; @@ -103,6 +106,9 @@ typedef struct BlockCopyState { void *progress_opaque; SharedResource *mem; + + uint64_t speed; + RateLimit rate_limit; } BlockCopyState; static BlockCopyTask *find_conflicting_task(BlockCopyState *s, @@ -611,6 +617,21 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state) } task->zeroes = ret & BDRV_BLOCK_ZERO; + if (s->speed) { + if (call_state->ratelimit) { + uint64_t ns = ratelimit_calculate_delay(&s->rate_limit, 0); + if (ns > 0) { + block_copy_task_end(task, -EAGAIN); + g_free(task); + qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, ns, + &call_state->sleep_state); + continue; + } + } + + ratelimit_calculate_delay(&s->rate_limit, task->bytes); + } + trace_block_copy_process(s, task->offset); co_get_from_shres(s->mem, task->bytes); @@ -649,6 +670,13 @@ out: return ret < 0 ? ret : found_dirty; } +static void block_copy_kick(BlockCopyCallState *call_state) +{ + if (call_state->sleep_state) { + qemu_co_sleep_wake(call_state->sleep_state); + } +} + /* * block_copy_common * @@ -729,6 +757,7 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, .s = s, .offset = offset, .bytes = bytes, + .ratelimit = ratelimit, .cb = cb, .max_workers = max_workers ?: BLOCK_COPY_MAX_WORKERS, .max_chunk = max_chunk, @@ -752,3 +781,18 @@ void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip) { s->skip_unallocated = skip; } + +void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_state, + uint64_t speed) +{ + uint64_t old_speed = s->speed; + + s->speed = speed; + if (speed > 0) { + ratelimit_set_speed(&s->rate_limit, speed, BLOCK_COPY_SLICE_TIME); + } + + if (call_state && old_speed && (speed > old_speed || speed == 0)) { + block_copy_kick(call_state); + } +} From patchwork Mon Jun 1 18:11:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D18A3C433E0 for ; Mon, 1 Jun 2020 18:22:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F2312073B for ; Mon, 1 Jun 2020 18:22:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="N/IO0Rxx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F2312073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48678 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp4W-0001GX-O1 for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:22:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouZ-00054n-2r; Mon, 01 Jun 2020 14:11:47 -0400 Received: from mail-eopbgr70117.outbound.protection.outlook.com ([40.107.7.117]:10211 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouY-00027N-1n; Mon, 01 Jun 2020 14:11:46 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EAq0Q7eHysrcDJ3wKjQNPHj2Io8gpsrEN43J1M1w9eOGzJitivucXND/KBECISQaZnSW8Eoe6S3+oaCuVjaDbAw2G9y8SGtXR1QcfMXOZGudqp/QZTLuPw1xmsJ2Ku6D8pkpwSUu42UVsfO23umM7eRiPR0MIub/3zc+6TTZRt6Zc07XY2EO+aIZHHEh3vbswgNtecdFhwsLhMAVvapHbnYS22PVXLCvo98+rdyXWG++GjfprEWrnnFZqLYvANdFBQwQSYC9xe2RqrKBx8q77Q74kNZK+GLOo5hXlpyj18iAKZL7tf1xfx1g+9Rnu7qnlLhX/cZyUsfiKwtWilUR/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xvM1RkOIAPydDLspA2665/HBjdPiJEGHYHpWD/s5Y+8=; b=ACOkJFs3KwWmJ9L/hqTNKhgsEli1m3ARHRavpLe1WwEbiJXdVRVZYMQ9y68oHE4ygyysadJrFJmeMS8eVtU3ZxjoJCHiXD62SoVx/pyH5NlY+xRKfd0b9frwqc7ys7AvEb8yHQcimUpnvrmK/3yawP6Gm5d26YisT/Of0A1gGI45p0zi8pSsWakbNzq7kzAysyGEHEK6Cufqixo43IRKBipxSN8PY8mVAcrjYJbb6ylFcX/9RDmEZGMI3u+uDtNMZG24XG24j2aQ8x/FThRBkUOOfs497GMsmodv8LRK6bRq024DbiyzyHGjmBbvzAYgHvxW3e+Tuj0Ezndly8RIHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xvM1RkOIAPydDLspA2665/HBjdPiJEGHYHpWD/s5Y+8=; b=N/IO0RxxcqRpSoSGvSOy2sg6l/iezPTia4SWekg92OklwLjxV2EwwwGn4mYMkh4CqYJvHBdkrx7xwCJ91hwZrYuIRUru8uVsv22+aT4cmEhltKEPLt4CItxpUeAhNd7EukXR+9hQ1fYVStwRy/xdYkLSz9s7KsNMV5Gwpdj+MDI= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:38 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:38 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 08/20] block/block-copy: add block_copy_cancel Date: Mon, 1 Jun 2020 21:11:06 +0300 Message-Id: <20200601181118.579-9-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:37 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4b6352d2-3c41-4b6d-b69e-08d806573a2a X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:46; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zigWdqXs2cPX9seloCEHrCHT4+IOo5O+3S7EssSADD8rBM4rUDjCE/GM6pDSl2h2vp94B2NYVV+BR1vHqQI+VheE9vDSnaqvG8VzNOqdzhJpISbehvnS/+biKxtrnehhwAsuMxRx+XCekgVw/kY1Uz2rXR5oMHh0FbsUm5WTCS8WP1LUET+npcJmAA1aiC5JYuV/zE+COrH3VXEwjAcjDVrqwVF2w5S6p9o4TPg+ddnqJ+06BcwZ6OMCfNEjSzhO/FZhecfUKIEwFtwDif/yaAY/o3zlzKWnSjIubo7GDT1WSsNMtTfEaBzyXmXyUkNM2rosuAJMTpW6LlosRveMY073A/5brvpIb4LKZIZ8RCF8OXVM8yRYo4QLf1+jN96W X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: eANPC1iiIc0Be+b2wTRpqgsNMvqYK3vvy6TGTOr/nPpt7QKfifBVo9KnkHotCN4TPwpCzB8YNMKxwuXPs4bJwiXxU77jhZ7fbbHs4yvHozc2Q86nWjP2y8SdwwQ3QHi3ULaoV0Ga5RCOzLrlP/1QNBQ5T+M6cCBCq/Cppvtwr7MRDX8nP9T4Ecq0de/WFqM8eLrrZGxvgGI+6JjKB11AtvSFKP1669S3SNADp5gqCd4klkHB9Fcpt0at3ETDgVIuSMWjqD4gFBY8w7frhjKla9vaLqiE5NlNuKae5sDCkHZKtU9I9btH3WuCQ7ZCwMa8cGwmJmh2KhUwUCgxwNv/3z3L0f8JaTGuVjFyEVLabTkFFQOrgQMSNkxzkMJf6Kio0kyjUr/Tu0dYyteRGtFDjtEi0mT0GY6xfRD7zwM9KjFnOdnf7yFvV7CnkV12U9ThSh+CATvXcI0KniAGcafPZHW7lGIJsxItACYOArcQ+xU= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b6352d2-3c41-4b6d-b69e-08d806573a2a X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:38.1993 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tFe0+cxho6ay5q8m014dTsF9UlgK/+8GyYV4ghoYzIqx6VpsLNjvAvH8/6j1eLSLCit5YpcLqJcixHPEE8Mb7H9hvNKqT4Ia4/URGcEbNDk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.117; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:40 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add function to cancel running async block-copy call. It will be used in backup. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block-copy.h | 7 +++++++ block/block-copy.c | 22 +++++++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/include/block/block-copy.h b/include/block/block-copy.h index d40e691123..370a194d3c 100644 --- a/include/block/block-copy.h +++ b/include/block/block-copy.h @@ -67,6 +67,13 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_state, uint64_t speed); +/* + * Cancel running block-copy call. + * Cancel leaves block-copy state valid: dirty bits are correct and you may use + * cancel + to emulate pause/resume. + */ +void block_copy_cancel(BlockCopyCallState *call_state); + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s); void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip); diff --git a/block/block-copy.c b/block/block-copy.c index 851d9c8aaf..b551feb6c2 100644 --- a/block/block-copy.c +++ b/block/block-copy.c @@ -44,6 +44,8 @@ typedef struct BlockCopyCallState { bool failed; bool finished; QemuCoSleepState *sleep_state; + bool cancelled; + Coroutine *canceller; /* OUT parameters */ bool error_is_read; @@ -582,7 +584,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state) assert(QEMU_IS_ALIGNED(offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(bytes, s->cluster_size)); - while (bytes && aio_task_pool_status(aio) == 0) { + while (bytes && aio_task_pool_status(aio) == 0 && !call_state->cancelled) { BlockCopyTask *task; int64_t status_bytes; @@ -693,7 +695,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) do { ret = block_copy_dirty_clusters(call_state); - if (ret == 0) { + if (ret == 0 && !call_state->cancelled) { ret = block_copy_wait_one(call_state->s, call_state->offset, call_state->bytes); } @@ -707,13 +709,18 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) * 2. We have waited for some intersecting block-copy request * It may have failed and produced new dirty bits. */ - } while (ret > 0); + } while (ret > 0 && !call_state->cancelled); if (call_state->cb) { call_state->cb(ret, call_state->error_is_read, call_state->s->progress_opaque); } + if (call_state->canceller) { + aio_co_wake(call_state->canceller); + call_state->canceller = NULL; + } + call_state->finished = true; return ret; @@ -772,6 +779,15 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, return call_state; } + +void block_copy_cancel(BlockCopyCallState *call_state) +{ + call_state->cancelled = true; + call_state->canceller = qemu_coroutine_self(); + block_copy_kick(call_state); + qemu_coroutine_yield(); +} + BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s) { return s->copy_bitmap; From patchwork Mon Jun 1 18:11:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39FEEC433E1 for ; Mon, 1 Jun 2020 18:16:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 071FB2068D for ; Mon, 1 Jun 2020 18:16:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="c0hOJmHq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 071FB2068D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33652 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfozW-00039P-9M for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:16:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37718) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoub-0005Ch-UA; Mon, 01 Jun 2020 14:11:49 -0400 Received: from mail-eopbgr70117.outbound.protection.outlook.com ([40.107.7.117]:10211 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoua-00027N-Vj; Mon, 01 Jun 2020 14:11:49 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HtF02i3DTLkYhaqC4/mlY5xoctweB4sluvZzjP6lZbw6kv2L1OXa2T2mr5DQuiOUEMIgOnJ6DmhtADVUT2zaxjqatJ9grqhvkMsufmm9+0pYiGYulzqZHUFG5i85ikyuRV7xTbzPgteLxXbz0QnpJ7Qoy4d5i9DnIlma1pVB0UmoTMky02QB2moLxE7864cgMZXO/ckx6TQede12Fv/YKcS9bYzZo44by3axml5e7MuR0Ph88itVdVqkcRy/EGqL6knwcJ7eoqNG8KndnwLY2G60aWLNSMsEwMaIucSQ1goLq4TK7eiupGWRIkglefoMtRzfhQQg0GTCD5p+zpNquA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jBfmbhmk/W92+2NFh6AEO3VnEZYVU6jp2wHuwN3JFeQ=; b=EnwnPjlGRoJdtLyLbtOPZFt19nogGYL+kJ2pVX3hJmPkpl786SV7KKKis4DAVIRQUOWS/HzL56BxtoYjfh8nTRiclG7wwidHvGUUeOkg17GneZLQGF5tiWhVd+9zgLKzV5XkLEV8xoN1hkTVbEl5/FiYGcEyZ5UPmL3WiTI6nFllLwHCVRwH860vLagVX9SdgBco/+JE8ifg1LFyNSpst0OnN7YE4JmTxHJczo/htw+ssEph/edQCZ/edlJxVpUWW/ME6+V+0zokeVTJNGeKBcUSyT+6RNHn4Mhs4+eNt07DtgRy92TdgIB+LtfjIYMgrcsQqKwcD4aJd97mSPLB6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jBfmbhmk/W92+2NFh6AEO3VnEZYVU6jp2wHuwN3JFeQ=; b=c0hOJmHqQz5Beek7fPoZrSn82q8knIknPVqexMmOsI/4C6BL5xG6mgy+MrvMucX4WzNmYXUcx8NBxbTzarKfXu4aGGyHw6REIEAgn32LIRntkbLxKZ3JFFPx6fL8r/IP9pkWRQlXK3hn8EJK+QP/pTV8kLBq8ifjZ4knCV8pb0E= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:41 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:41 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 12/20] iotests: 56: prepare for backup over block-copy Date: Mon, 1 Jun 2020 21:11:10 +0300 Message-Id: <20200601181118.579-13-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:41 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d0d08185-a529-4c15-19dc-08d806573c59 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NtbfqwliqLp8+2z5ac5ppInbXmM14Cs5gyPZPLn8TL1q+zg25mVZs4GGwsb7ZuHO5+U0s3fZme6D1R81nAHKLR7aE/TS0BlZFrlzT+exPGq84s12862QaoVuo2bpFEZDEIqWcYgnYOshcCAhcZ+1hBxMOE025qcozcQG0XC82WUNDzsteGvCrOnSi9v1RDFVKBTgDxBu93Xrf5EbVpd8+gtSXl+BRoaMT/TdciCAVdSqewtB9+Manwduj+rI+jbVxHdS9nwUiFDw1kADenbgkq2MRxDX+c8/vh6JO995OJrZ27Ns293JosYbVSjP9i9K1nup3RiHuD5H5KDjzgZevG9/NYaB+fuey3fYmhbwr0Xpake5Is9G7bLyTYmrGzfg X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: /QGWQ4bRFzDgcp/iTLWpmssjk2brAMxMgJgFaw9aynWn9MGfsPJtkuHIACFdimF5Oy43NCaqm0ulaeMboEmENRSTYIBTf/QwyasvPDyHsakWVMk64JELHD1yogdHPf6FFKIyX3C8BnMYzByP1cElh/6O4QiS4u/9Z4OQw2zV8LB98GvWO1smizZN0OXWroaBeg1vuPlYgCXRDsKMsfShyBQYzj5LJ5Tc/mIOuQzVvK5FrIIUCTCsz/tVwN+dV9ZPky8lR7qsZKIf2SAXvyMwhW9Stw4a6g6IR4bvjcSPo1h3RcT+r6d+zE412vpdfCx0tCwzdtwcdN89LfGE/wLqOzEWoJskmgwGULO/JF6A8YR5XuVr+Oy0JQmLp/AaG7EQTKdkOdI7OTgDH+33gyjQTkUySieA/Ajj7xl/Btx2HdgXbNaGDyACxhsc3tV/Fu/uKev6zf3x1SAb6h9AqaduCewQDqCKqpDyQY+bWY01WRo= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: d0d08185-a529-4c15-19dc-08d806573c59 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:41.8384 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Hbze60JTLb5HpZH3G+xcRHhmL5RiHTtiQYF8DRsyMCwj331L3eNTlqdqeiJ9mVDzJ3KgHSCMjTMVql0KcswAMbz8WwfKVPkpdEOhOcT8fe4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.117; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:40 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/056 | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index f73fc74457..2ced356a43 100755 --- a/tests/qemu-iotests/056 +++ b/tests/qemu-iotests/056 @@ -306,8 +306,12 @@ class BackupTest(iotests.QMPTestCase): event = self.vm.event_wait(name="BLOCK_JOB_ERROR", match={'data': {'device': 'drive0'}}) self.assertNotEqual(event, None) - # OK, job should be wedged - res = self.vm.qmp('query-block-jobs') + # OK, job should pause, but it can't do it immediately, as it can't + # cancel other parallel requests (which didn't fail) + while True: + res = self.vm.qmp('query-block-jobs') + if res['return'][0]['status'] == 'paused': + break self.assert_qmp(res, 'return[0]/status', 'paused') res = self.vm.qmp('block-job-dismiss', id='drive0') self.assert_qmp(res, 'error/desc', From patchwork Mon Jun 1 18:11:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD070C433DF for ; Mon, 1 Jun 2020 18:20:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A7CFE2073B for ; Mon, 1 Jun 2020 18:20:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="Ub44yxjR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7CFE2073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42868 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp3E-0007Bo-RK for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:20:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37724) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoud-0005GD-5j; Mon, 01 Jun 2020 14:11:51 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouc-00026c-8N; Mon, 01 Jun 2020 14:11:50 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YaVEGV1Q0G8BQ9Clj0TCg/RFljGm8XjUzAluOrE/31byWEq6WfDQnhKFGvoTIS0xcrrZ+ep42jZOgGTqgMg7SB9ZnkRF0x0rHjLeXvSNhQqRz+EcuxUmn2kS4aneZajjUBR+npkZoJDOyMn0GbqBP1vdVaMEJJTZZJHEbADvwcfkVo89f6di0goN8G4orEM6OJ8nampPGB7+PeZ0kyFOjjB+wV7Cz+M7k7CYu9dNXQUl1Zsuyiton7COut6ULSkrWkGEPalZAaaC76ulM9xTlj6cGcxoitOPqQ74DnEj9921jf8Qow6uNQciSafU+EbvT39+8erLULM4xn6bLUOvug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h82TpyYMbuKPfS8sjUB/b//yqwwG7Jeo3MhfNUlVMK8=; b=Tkii+hwrAkAx5luQyWZ/pYv0PP8mivvFpH/AFsD/G8BGMtR6DnohqRS2UelE8l51r+qLhnf1w87NnINRWjPLEK+Y7/EJPcceaxEko3nru8JAQy05uz16c+cz0uAZ+963e3JHnTBKGUsCwdwZIISpt4h2c+g83WEkQGSv5CeQEqJqDyxFRfKPCufSJaigve7iEzUO0v+Cgnkc/Ypet6ZB0aek/TBmjfIXybulwy8RApOL8EXDjPBSmp1wuI3TDi8uvbb2hWuuN3B3kkggAKSWywYKXdZqmuXsA1QXGPoC/uBqYh0nHtfWsW3hA31UsxDgbzicShrqk2lI0F5vPxjHXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h82TpyYMbuKPfS8sjUB/b//yqwwG7Jeo3MhfNUlVMK8=; b=Ub44yxjRf2P1ehJMehu+bZp86dF9PkJHYS2Qh2bMnN+ZBiSPWAQxTnF8IbXwqzofzOFpzCzavTYWXrsy/ek+pK1SfJ0L2nj+Db5kfTQNqMR55qXTdFJaEqVbTnCT3/JQ/+JPNuWDKZ8lwU0+ndyj9dZb0gb5e9uTft0tVr6JhhU= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:43 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:42 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 13/20] iotests: 129: prepare for backup over block-copy Date: Mon, 1 Jun 2020 21:11:11 +0300 Message-Id: <20200601181118.579-14-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:41 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f870c227-7dc6-4e9e-bdaa-08d806573cdc X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:88; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BvaCRRmYmZXMm6uf97o5JXICcAwRP9OY0dTEK1ZoCGhIjxaIh1vm2NNIMpo/HwtCpFG7ig/ZtikGajTjmMsahynvkPVM/EBOxbFBsqZ4DJS+wS8kOfunRm/JUKkcdZVVu0D2qivHSECFjfEPaJPSFp0krrfqPwX3KcaV4ViI8ZDvxsfoADIXwoElD2ALqixfEA21UABDzs/cz2SXsBc/BRuFZUVb26EEHVZveONeWByqRSxoM3ZH0VL4i9zKeCVYSDsYreW8mqY5aOC/XGg66ASbtpl/HOyQ888orG3cwPuplOIQ/SuieMJ2t8osw80saP4euBYZD6K/REI0EAthiGo+27/L+CI+6aStbdrFAO6DKSrVdytU2kHlGzgFU04S X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(4744005)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: r/mhPF3G0ZxdZwmRikjRbhmCoATTqOoZIvXOWEFW1EvB+UvBJ3RMFnF1hoPLqPujnte2Q2VDDRuEON69wYNxWrzAUibpO5nwEnT6MC8M8B3aZgxFFgaDF1mz0cD+y2I2BXrwFBzNUcCkMQDonl1V13aAvreURd6dVU1AoHvlCSHW0u52H9iw0AckUOm+bnhGKrOmvRvR0r8f/2wfJl/7IaI89pBCvx0xcd1R5qkdUIYKuC17hdi8KOVrDaHNvclpv/fqJAGB30J1jO07O5NhMDRcgV2c0fH0vQL8wzyYbXMEYRO5YxwixC555Jo68x68J1jooVBN4FwOebKWmRceg3nw9+LabK5WzrcXU3ds+UrMjv1OnHmYzZMVoKwr7AMZIDppXGEZYyUI3oXi3w1RC4zJ+tiVVkUlFDRGrK1H6tHCMa+Hv7V0OzRoFpitTatzKRtK2pfZlOzJFdG8tCnsAbIdt45LJfpSy0B6WgazE0g= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: f870c227-7dc6-4e9e-bdaa-08d806573cdc X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:42.7260 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: b4R78oB6xYDLQdmo4RIeUohcK4deBes/iiD0qhYcmlPjL/Aki4Zar1XHXGlDGmlx8jWMXsUcDmR9yutgbcCAHIOYoKfklmytLfdXJHJhGzk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in do_test_stop. Let's require slow backup explicitly by specifying speed parameter. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129 index 4db5eca441..bca56b589d 100755 --- a/tests/qemu-iotests/129 +++ b/tests/qemu-iotests/129 @@ -76,7 +76,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", target=self.target_img, - sync="full") + sync="full", speed=1024) def test_block_commit(self): self.do_test_stop("block-commit", device="drive0") From patchwork Mon Jun 1 18:11:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F83FC433E0 for ; Mon, 1 Jun 2020 18:24:23 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DEB232073B for ; Mon, 1 Jun 2020 18:24:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="H72/atmC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DEB232073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp6k-0004vl-3G for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:24:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37726) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoud-0005HJ-Hf; Mon, 01 Jun 2020 14:11:51 -0400 Received: from mail-eopbgr70117.outbound.protection.outlook.com ([40.107.7.117]:10211 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouc-00027N-He; Mon, 01 Jun 2020 14:11:51 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YUPxsbTaaPkc7uefdq/o9XGJ/J69Yd1chnPMn9b8B5DADsqlqotnX9IaPXoER78lFNVzpb8jFGxee4LYv2lv2kHXTO5CFywJMEKuEZ2HCFgfzs/7u+Cvwwp+ZokpV/3gW7G+5KbfmFN4S8IUojO8tmPyMMgMESz5uAHzmNKz/pkrCBFeoWbka8sD2bF6WAosQLbRB2VOgXZiYn6HsaGt3Ri8WdfSQmGyiiR1QrnMQNuSfjB6SInzjlQKVkRZb2Yvh59aB1jSmXpwYhZRJVH3usppImRs4f5WL7re+EN6pAa7D3NlwouUS3HyMPwHhvhGckp3+fqAR16UETSj7rkK9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A8yqqP4TiNxeZhmnF2MxKR+ffrvUBh0d/iUcko9ddiw=; b=IREJydDqH6/G6EyowtjFwFaEHQItoxHDf/r7km9WRVNyzPHm70qfiIAlqtdkFULKD7y4nCJ/qPHPspK4wX2MhNzo3UJwr/kwxMCocXBQymzCIGXwSQ0g92ap8K3LXkLxEtbC5k1PQJGxjWMbqqKI31VyMuOMa3sD1CNuCP5C/wpY1nXijrmAtMk/50XUUOtmRJEBIsDHfY3FQGHKdtwifljNDzqhSAazBZvJkKZX5/O7kRz2XofbsYJ7dh6Gg+a/fez9hFwD9DMJjRWAvSayn1/1ufWYB/PJiJrI/P9ZQkRbNPJ/sMcAONTQESm/qtqhnFOBPdKXrG3x5SeiFu8CgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A8yqqP4TiNxeZhmnF2MxKR+ffrvUBh0d/iUcko9ddiw=; b=H72/atmCndqXDZyKnuX1VwWxiQMYy7eH40YkVJR6Eug+zoJNZ3DxUsx8nrM59zphCzs2MSLSeNaT7kgvIKtwfp8JCBXFSjnylLGHuqBqeGLyysfpFX2R7ftKQmluIDXuEG73/BssUXqmraYVyLszVbEl/z556Z0U9vVNhKq4II8= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:43 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:43 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 14/20] iotests: 185: prepare for backup over block-copy Date: Mon, 1 Jun 2020 21:11:12 +0300 Message-Id: <20200601181118.579-15-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:42 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e699ce92-1544-4364-376b-08d806573d61 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1fkkVK6PM5UL1GdbnpD1hOMWlXc4A1wGv44ZshYjAgmgO9jVz4zWyGvV0VuABFLzkIvyLoEeS9L7sAlsK7D6K/GdIE14nUVBT6yjPnhbjfsp0WjY+ZcZ7SJIb2IZlnN9a5yZ5w9BuC36ruea6HKo5RgzSLDO0JcTqh40nYjyHw8RqZQsiNBjml4M5W9QEqYLKdJoorpR+wQwhLYs5PLW+OPDqBDWmjw61snRVaSZ6KWXzc12r/kNwWg8PWElhI9cMkmWvjfBtuaX5lM/8S3xymf1bPJ61S4g/9PbfsBH0XEzEDyMj1faQcUKIwJO0BbTG/ynv6EN9FnbmUWY8jPpMIe85TTqzO8FI7Gf/3wwRlIcYFrZiq1iKB1KA0c/CDQ0 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: cciS4roC2dZSbyIYdG0iaSJId0TNts3z/44CXjTLOMWVj2YdvvuO0o3UZb5capplItzmI5oaI1EP8SPytz8VtSULSxx/iv9Se965mTFl/Uaob5nAom0zNJ3T+jhxjOLH/imNLOuX5boCs4aq1SFDmyVEETIwI3cjSyiURU3Mwh0GeQ7Sa7jBZlVDBIkwNjypvdQyL5vZMePbjFlpmvp3XqesJBSaYSBIPLLbYZHimahf2uHgVnqs3f7TJN8DU20gDG9fbcMeqyn13mlvrX9RdbJBroHcw8maLzYcyofZScB202vnIAYLXh8FxgUoFfYMk48gOLIPr6b25Q2NNfFUcheU8obTDC8q3VfSMP/kvOpNJjxFuRAbDnimTJxPQoZQp1RhXGmIBESZTS9okeBfwrm7PK/YnUjLdX9ZO6/mRcf7PXs+6saa/UsJ00KEEEO3zarM3/N2hdT44ywjAjcMAxZlJfuPbbWVa9Jvf4HPy1Y= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e699ce92-1544-4364-376b-08d806573d61 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:43.6001 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XNdiSoPyKSepzmKErvkjpK3LB3WcMxmSHKfsbbVyPdXfg27z4wOrRckGoIjhUu5QjEK7LzQmY6/q/Lk6CJ6Al4KKjpntSwIqIsNDKGUfyTg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.117; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:40 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The further change of moving backup to be a on block-copy call will make copying chunk-size and cluster-size a separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. 185 test however assumes, that with speed limited to 64K, one iteration would result in offset=64K. It will change, as first iteration would result in offset=1M independently of speed. So, let's explicitly specify, what test wants: set max-chunk to 64K, so that one iteration is 64K. Note, that we don't need to limit max-workers, as block-copy rate limitator will handle the situation and wouldn't start new workers when speed limit is obviously reached. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/185 | 3 ++- tests/qemu-iotests/185.out | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 index fd5e6ebe11..6afb3fc82f 100755 --- a/tests/qemu-iotests/185 +++ b/tests/qemu-iotests/185 @@ -182,7 +182,8 @@ _send_qemu_cmd $h \ 'target': '$TEST_IMG.copy', 'format': '$IMGFMT', 'sync': 'full', - 'speed': 65536 } }" \ + 'speed': 65536, + 'x-max-chunk': 65536 } }" \ "return" # If we don't sleep here 'quit' command races with disk I/O diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out index ac5ab16bc8..5232647972 100644 --- a/tests/qemu-iotests/185.out +++ b/tests/qemu-iotests/185.out @@ -61,7 +61,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l { 'execute': 'qmp_capabilities' } {"return": {}} -{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } +{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536, 'x-max-chunk': 65536 } } Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} From patchwork Mon Jun 1 18:11:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4075C433E0 for ; Mon, 1 Jun 2020 18:23:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9032B2073B for ; Mon, 1 Jun 2020 18:23:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="F5IzlBus" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9032B2073B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp5h-0002iz-OP for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:23:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37742) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfouh-0005Qn-8h; Mon, 01 Jun 2020 14:11:55 -0400 Received: from mail-eopbgr70117.outbound.protection.outlook.com ([40.107.7.117]:10211 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfoug-00027N-8D; Mon, 01 Jun 2020 14:11:54 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R53cB136DB3Eg+/dxd9tX+LvQ6bbCTXobvg0Q5wN9YW94fzTmz9WqCg5VbGoik5XDnyorNuqRWC5VFd8wSPt4ILVJ0GvxrxCLjSnDdC+P0teZqxzanKLS+TnHLca2p0DfbY4qZ1cUxB9lvgldai3DWmNr/nlRImL4uhjBSXuHUc6peyJQKJP12RNmVuMf0LeEyDtqpVqX84JSwbzpyiSC2r4OSvSDZ0xW9IGnk3WzNG26GiGJg83gAMbLg0YlxuwBlhuYdtKwtuw1OPbKsrGXjFgkFWHlRLFcWtWm9cdqRVp+EzqHQtbr3ow7TVfqL4Z1jTwuoPjTaf+vOtjhH+iJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cRYGwYFnRiNvUOKySjXGOxsgPnCdKbAgtWwjjH0icW4=; b=T6ojhMi6vKuwR3LBBgjjGfoFB6dxkJo+a6Y8Cy2iWLRnvQ4NVNGtLJTCXHqSytwNy+lbfYsiBR6w/aI2iZSFwXUF6i51MTCN6XaGnN2mhL/UANCWIH9LJAskNDYdevkuZcej+kvh+2R8PuvK0W4MfEBhC+XYigOI6aR98adoYfmqg9Orjc+YIsTw5hKvb6bnS/RHa/MUUS/JGIroMgoPUPoKCk+4W71fry3Drv6/iBru/7+EmqpIgLsG+6dElQ/egM4uZRQrdNtfbQj8rpx+RSmOgscTWffXzHW/tvPvquzQvMlVLG5Pw7phhGxLJVDuTsSA0wFO3Xz/RMcM/DVdew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cRYGwYFnRiNvUOKySjXGOxsgPnCdKbAgtWwjjH0icW4=; b=F5IzlBusOH4M2WKvHzCTkc0Y1mclw3PZLkRbov6aQx2r9BczhIOD356QJpRjTKIvvv3BTV4nItbXS9IsvbXn2+CtnajRclhMfVf1pvN7DX9yKrReioY4uOUHmiQxGreucqqgFDtQVbB6Iq/DGiqAyKlapIABtxPZPF8B/6NrTfk= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:48 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:48 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 19/20] simplebench: bench_block_job: add cmd_options argument Date: Mon, 1 Jun 2020 21:11:17 +0300 Message-Id: <20200601181118.579-20-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:47 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e7597b3b-2e1b-49ff-fa49-08d80657400f X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1751; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yK4ZQGfVRhUTdLZOW2tyye4dmcl3uKXeq51E1PS7XjuvWlcJeiZRTBuz16gGef1GYbFmPhUTYweBWqqZmHdLBnZkcjWTeQf40eCYL2Zv95AYNzbWGuGJuIPU7GoeZJtmPn2nnV67eP5P8Awoouh4kgYAho8wqpW24rFny9cr3SZbYOiIUP9iyFQ+zmMTfR38saOzaZOr/U5exl6UTX7V3XkKCTdSBccUFYd1MlFERAKCIqrp56N60WELQlw4mv9pH/E5qH3keTsbZgTkq9+zVa8Q5J7M1OAUnlZAXAxG9AQI+hPG5WLwMrqA1J3WzOR1hQjNNeirAAfFB4N8zvrMJxWc6vz5btC8R5mH4PEguWEY+2mq1T+08UM/2d+t0z8p X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: FlR9tnOaFcncFPkTkGjyZHu8JKXbT2pDTdHs/hIpsHR/2pX1rsD6qH9+cb11JytCHwJjqmbUAKd17PKzF0KGHkOruLQXy0JpGIHcvGT4CUkIum/TfsUZOhaAqiD5vDIszn9psReVLxprGnqK4n617ymOVVIPHMlVr1XAtWgSQiMl9ss5h5qjgmX21T4X8YWhJWlNuJ2rDiMr16siHy+6z09buJXs+WxoGpCP5tMkFjUSjmIuOe2P3H47ntlQg16kpzGEs4FywEsAqTgPQRUzdJ6kguUQsIZskLx9CueupbASIz4ifvzqpBJzR+1LkXV0gv/mh+kgpZmg3wrZO9JNiSTBVOA9LWm6esSP6WznLfBKs0mzSo/nHBeLpVP9rFmY63KW4mQsAIPqTST2UgHghE85OhAGSawivhATbR6wEtiTbX4D14dYn7c/zUx5lkRC10GwxymYjbrtxtk2wlAffY8jBaMfbAWUtFasCdwng1M= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7597b3b-2e1b-49ff-fa49-08d80657400f X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:48.1104 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uFP/aXrtLP16xBdjrCxgLf3fbmTso1gNClhWeBk3iFsthSrJ21YY/x8wDQgKZRNciuAG1qs5g6RjsfNTiQto2tJPusy7IJ0LrmGYaBri2CI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.117; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:40 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add argument to allow additional block-job options. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/simplebench/bench-example.py | 2 +- scripts/simplebench/bench_block_job.py | 13 ++++++++----- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/scripts/simplebench/bench-example.py b/scripts/simplebench/bench-example.py index c642a5b891..c3d17213e3 100644 --- a/scripts/simplebench/bench-example.py +++ b/scripts/simplebench/bench-example.py @@ -24,7 +24,7 @@ from bench_block_job import bench_block_copy, drv_file, drv_nbd def bench_func(env, case): """ Handle one "cell" of benchmarking table. """ - return bench_block_copy(env['qemu_binary'], env['cmd'], + return bench_block_copy(env['qemu_binary'], env['cmd'], {} case['source'], case['target']) diff --git a/scripts/simplebench/bench_block_job.py b/scripts/simplebench/bench_block_job.py index 9808d696cf..7332845c1c 100755 --- a/scripts/simplebench/bench_block_job.py +++ b/scripts/simplebench/bench_block_job.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # # Benchmark block jobs # @@ -78,16 +78,19 @@ def bench_block_job(cmd, cmd_args, qemu_args): # Bench backup or mirror -def bench_block_copy(qemu_binary, cmd, source, target): +def bench_block_copy(qemu_binary, cmd, cmd_options, source, target): """Helper to run bench_block_job() for mirror or backup""" assert cmd in ('blockdev-backup', 'blockdev-mirror') source['node-name'] = 'source' target['node-name'] = 'target' - return bench_block_job(cmd, - {'job-id': 'job0', 'device': 'source', - 'target': 'target', 'sync': 'full'}, + cmd_options['job-id'] = 'job0' + cmd_options['device'] = 'source' + cmd_options['target'] = 'target' + cmd_options['sync'] = 'full' + + return bench_block_job(cmd, cmd_options, [qemu_binary, '-blockdev', json.dumps(source), '-blockdev', json.dumps(target)]) From patchwork Mon Jun 1 18:11:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 281525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 429E0C433E0 for ; Mon, 1 Jun 2020 18:26:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 023362074B for ; Mon, 1 Jun 2020 18:26:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="T6v30bRv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 023362074B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35166 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jfp8Q-0007ZU-QL for qemu-devel@archiver.kernel.org; Mon, 01 Jun 2020 14:26:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfov2-00068A-JS; Mon, 01 Jun 2020 14:12:16 -0400 Received: from mail-eopbgr70093.outbound.protection.outlook.com ([40.107.7.93]:39047 helo=EUR04-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jfov1-00026c-CZ; Mon, 01 Jun 2020 14:12:16 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XNAY83t2OTCyCQOymN3eGJoNm7Cz9RytKHxSpTqC4vhjS65EPU4+Y6Y1wllG0RI4hDogVKstDddiL6V/FUb2/S8AwROsixoV2rQnn1YARcN+ZmmECeQhBDuM1bxwB1qRcyRS5eFVTAQJZnuq0vO6kp7DstM9bAyfdqM6qfFwCHFZYeJ81pVYgvG6jlPC3C9seTwJtId4lsYwPGhrXqPYZZ5qD6Ilhi9vA/de4VomITXec81bjWz/1GToq5uv+Wdxlf3HtrOou8VDObT5G1/7aFoD+FNBfOtXhCwrj4xycHaktgdHNP8bNc+Mz8mOm7j4LxYSXnMoqPC+MTgGz+xilw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JuoEurP90kYN5NvdWyFGpyZ0Z+G5YuQ4ObNKY8rXjBY=; b=nzSJ1bA5BCS+7KWdeaZ6pRzAS2vvgPyL0ILYdWrDbwRKHxpG2EiI4/s1QZWICNYJK3thOox4T50c4yuFi+J+pfoQX9ofSkYdcciEcPE57RoehF7JzvHK8c37KYMqaFGdFTpYp9CqNqWT8PfeuRQCNFBguZaG03FoavhDdSZJVfR7Z2pBolithQNsNFhzHOwvpAni6u+ZdUqHnO70NS1q8E9E9G+nIV9hXcU4UGx0zD5vd8qH2GR6IBAO3NKCVuZ/oMzhrE1eZS3QT4aLPaLdRS5hzr7FvhHoQRlle8H2qNAxpgtiv4woAQIJuKSVl9+lzn45sNqJbXkEQcXveiWNRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JuoEurP90kYN5NvdWyFGpyZ0Z+G5YuQ4ObNKY8rXjBY=; b=T6v30bRvmX0oETG/uon095dsCtlDdJLxQ1lysYldmw2s+ax0GteRcf8NtSi+F3WcF2YaSyBUyqgbf9GgfJSJwGP7KzA/4Mdo7oB0n/RSp2kREjjTkrhJp5pxPk6Shj9O23ZpSlhZqB93tTyHPSfo8+/Ffq4JW3cuewkhreS8RQk= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none; nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5317.eurprd08.prod.outlook.com (2603:10a6:20b:101::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Mon, 1 Jun 2020 18:11:49 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%4]) with mapi id 15.20.3045.024; Mon, 1 Jun 2020 18:11:49 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH v2 20/20] simplebench: add bench-backup.py Date: Mon, 1 Jun 2020 21:11:18 +0300 Message-Id: <20200601181118.579-21-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200601181118.579-1-vsementsov@virtuozzo.com> References: <20200601181118.579-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.170) by AM0P190CA0028.EURP190.PROD.OUTLOOK.COM (2603:10a6:208:190::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Mon, 1 Jun 2020 18:11:48 +0000 X-Mailer: git-send-email 2.21.0 X-Originating-IP: [185.215.60.170] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 41057229-1195-4c55-0fa3-08d806574095 X-MS-TrafficTypeDiagnostic: AM7PR08MB5317: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 0421BF7135 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /pz5LwcDfwij/C3nZb0eusAdGwAW3WgLkKidsnWmmgtiFQcvm/X9ZAGTBa1uGxRT8wFdQxMTapR16tb8yOO9EzDXE9p5MFlv4wIgKX4YPLZgTmkJkmXkOuD194bYEUYoA+QhDYlFUiGYiwihiEWbMllSgJWQwhvrctTLCd5RGODl/gZ4CwceazDmYtC53i99nQYbVkVMbiJE7I69NV1b/rgP25ZHYEjpEJDgf2Y1orC3sIFesRYvdzlmdVCK2Qd9j+oePs/wx+zFkaxjyA4HVzAHboQ3ElzVuVoUx7hfnmEXmgg5vmigIeQxCULXWU9U42yLUMVYmKno7gL7/piR/bbc4Wc3zBMEdNACMZVueT2F2P8ZMFzb0x/r0qHvrettHzN7LWE8V7uOgGsxVNNGYkbM+LktuVbeZJWxOCG4z+EyrewthNwwrz2LBd/whMMeMlcbq3WoMaJDggQSxE7Ol41NMad3I77GqSyIFMEYix8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(376002)(39840400004)(346002)(366004)(186003)(16526019)(8936002)(2906002)(8676002)(26005)(107886003)(36756003)(316002)(6916009)(52116002)(2616005)(956004)(478600001)(4326008)(86362001)(6486002)(66946007)(66476007)(66556008)(6512007)(6666004)(1076003)(69590400007)(5660300002)(83380400001)(6506007)(2004002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: A/9HkfnLnLVEzTHfj1PVPgaA0aO4OLSBoO7j2udIvK+Hhj2SS0Gv1ClO8T/FWp0AE+JcXUWvB0xKVAzpoTQYxXAzhEB3hSczCOXJzLIacgiw5ZPQAxv6EcpSjgeUUqyna38X5heuV6TWYTFSAo0v6ZoDoqF1cc4vYwofofdP8QzL9BKrQVcqECGL+5i5eViFV7BwaI5Rpq4TTmp4IPz4m9d22bf85x3kWYkXnKj+1eFNZd+1ak7tWPZnTW5HsLgzSiEmpA5uU15huV1FbkPsngOq5ya8yDY7Ae+lBc44vT4C/SPac0Uli6beAE6qIMdc7XdcmAAhKmFLl21rjA+ImiVGk2csOtk/91dS1kQvYsf1RJ04/ndXcViGYxGnw+oMW8QzuEYWl26PtKR2bRy4CIwBhR2CzCFzjKtLJ9WfTamtzP1Zou5GfLcLX2KCPTlyzPLOK0dg3u8yr/WvinlcTfPvtjNfrK/bVv2PeLWKHtU= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 41057229-1195-4c55-0fa3-08d806574095 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Jun 2020 18:11:48.9631 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9G5ONW4AqIQYCvog2Y2wfqNnN4eA4FzdpFxB5GeG+ly5snpf1dj+r72L4ExntPz8DD3WUjf24YxI6IdrH+nylEKl0XIZUv+3kY7MU+pdoIE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5317 Received-SPF: pass client-ip=40.107.7.93; envelope-from=vsementsov@virtuozzo.com; helo=EUR04-HE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/01 14:11:32 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, qemu-devel@nongnu.org, wencongyang2@huawei.com, xiechanglong.d@gmail.com, armbru@redhat.com, mreitz@redhat.com, den@openvz.org, jsnow@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add script to benchmark new backup architecture. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/simplebench/bench-backup.py | 132 ++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100755 scripts/simplebench/bench-backup.py diff --git a/scripts/simplebench/bench-backup.py b/scripts/simplebench/bench-backup.py new file mode 100755 index 0000000000..8930d23887 --- /dev/null +++ b/scripts/simplebench/bench-backup.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python3 +# +# Bench backup block-job +# +# Copyright (c) 2020 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import argparse +import simplebench +from bench_block_job import bench_block_copy, drv_file, drv_nbd + + +def bench_func(env, case): + """ Handle one "cell" of benchmarking table. """ + cmd_options = env['cmd-options'] if 'cmd-options' in env else {} + return bench_block_copy(env['qemu-binary'], env['cmd'], + cmd_options, + case['source'], case['target']) + + +def bench(args): + test_cases = [] + + sources = {} + targets = {} + for d in args.dir: + label, path = d.split(':') + sources[label] = drv_file(path + '/test-source') + targets[label] = drv_file(path + '/test-target') + + if args.nbd: + nbd = args.nbd.split(':') + host = nbd[0] + port = '10809' if len(nbd) == 1 else nbd[1] + drv = drv_nbd(host, port) + sources['nbd'] = drv + targets['nbd'] = drv + + for t in args.test: + src, dst = t.split(':') + + test_cases.append({ + 'id': t, + 'source': sources[src], + 'target': targets[dst] + }) + + binaries = [] + upstream = None + for i, q in enumerate(args.qemu): + name_path = q.split(':') + if len(name_path) == 1: + binaries.append((f'q{i}', name_path[0])) + else: + binaries.append((name_path[0], name_path[1])) + if name_path[0] == 'upstream' or name_path[0] == 'master': + upstream = binaries[-1] + + test_envs = [] + if upstream: + label, path = upstream + test_envs.append({ + 'id': f'mirror({label})', + 'cmd': 'blockdev-mirror', + 'qemu-binary': path + }) + + for label, path in binaries: + test_envs.append({ + 'id': f'backup({label})', + 'cmd': 'blockdev-backup', + 'qemu-binary': path + }) + test_envs.append({ + 'id': f'backup({label}, no-copy-range)', + 'cmd': 'blockdev-backup', + 'cmd-options': {'x-use-copy-range': False}, + 'qemu-binary': path + }) + if label == 'new': + test_envs.append({ + 'id': f'backup({label}, copy-range-1w)', + 'cmd': 'blockdev-backup', + 'cmd-options': {'x-use-copy-range': True, + 'x-max-workers': 1}, + 'qemu-binary': path + }) + + result = simplebench.bench(bench_func, test_envs, test_cases, count=3) + print(simplebench.ascii(result)) + + +class ExtendAction(argparse.Action): + def __call__(self, parser, namespace, values, option_string=None): + items = getattr(namespace, self.dest) or [] + items.extend(values) + setattr(namespace, self.dest, items) + + +if __name__ == '__main__': + p = argparse.ArgumentParser('Backup benchmark') + p.add_argument('--qemu', nargs='+', help='Qemu binaries to compare, just ' + 'file path with label, like label:/path/to/qemu. Qemu ' + 'labeled "new" should support x-max-workers argument for ' + 'backup job, labeled "upstream" will be used also to run ' + 'mirror benchmark for comparison.', + action=ExtendAction) + p.add_argument('--dir', nargs='+', help='Directories, each containing ' + '"test-source" and/or "test-target" files, raw images to ' + 'used in benchmarking. File path with label, like ' + 'label:/path/to/directory', action=ExtendAction) + p.add_argument('--nbd', help='host:port for remote NBD ' + 'image, (or just host, for default port 10809). Use it in ' + 'tests, label is "nbd" (but you cannot create test ' + 'nbd:nbd).') + p.add_argument('--test', nargs='+', help='Tests, in form ' + 'source-dir-label:target-dir-label', action=ExtendAction) + + bench(p.parse_args())