From patchwork Mon May 25 18:08:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ED56C433E0 for ; Mon, 25 May 2020 18:20:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73AD420776 for ; Mon, 25 May 2020 18:20:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="mKwu4DDi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73AD420776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHiV-0004Te-IF for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:20:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXr-0003uG-GR; Mon, 25 May 2020 14:09:51 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50855) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007pB-H0; Mon, 25 May 2020 14:09:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=x44WykpORUb7jKPsPZokuc5MU7JgoaeSdWXwYoNGUz0=; b=mKwu4DDivtGLM/Ppvq76ZQpN1jmDbKNdwnnB99KxA1x8HaDh3r24mBYJN/h1ZgXDpOT7/Sl0BBty7QcZ4s9eRnFutU3ndfgYuYD0pdGssDpgWPVTNY79wXVj3bJK7AawyeomzHpfs0F6Nha44nyuHuqyQAi9Xo62LlPP2pupxHK4W2dL1Hc+pr93OOtf1dGa4hcC4vlu46dyJrsDwzbFreONEvGOiCS73EfTIoaBQZJ4FQIymiRTttCZbrfsiozW1uTAEQra394mtp7l/HTlLwhdy6flmbT9OOY0eMLYGuTA7ToxJj1jzrE5b127y4Abh/0wyHhhQD32CygqpAhykw==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXO-00088C-Cd; Mon, 25 May 2020 20:09:22 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002Ma-Hv; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 04/32] qcow2: Split cluster_needs_cow() out of count_cow_clusters() Date: Mon, 25 May 2020 20:08:29 +0200 Message-Id: <6e6d29aa0e7de2449a7f1951df61802934cfd8ac.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We are going to need it in other places. Signed-off-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 61ad638bdc..80f9787461 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1087,6 +1087,24 @@ static void calculate_l2_meta(BlockDriverState *bs, QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight); } +/* Returns true if writing to a cluster requires COW */ +static bool cluster_needs_cow(BlockDriverState *bs, uint64_t l2_entry) +{ + switch (qcow2_get_cluster_type(bs, l2_entry)) { + case QCOW2_CLUSTER_NORMAL: + if (l2_entry & QCOW_OFLAG_COPIED) { + return false; + } + case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_CLUSTER_ZERO_PLAIN: + case QCOW2_CLUSTER_ZERO_ALLOC: + return true; + default: + abort(); + } +} + /* * Returns the number of contiguous clusters that can be used for an allocating * write, but require COW to be performed (this includes yet unallocated space, @@ -1099,25 +1117,11 @@ static int count_cow_clusters(BlockDriverState *bs, int nb_clusters, for (i = 0; i < nb_clusters; i++) { uint64_t l2_entry = be64_to_cpu(l2_slice[l2_index + i]); - QCow2ClusterType cluster_type = qcow2_get_cluster_type(bs, l2_entry); - - switch(cluster_type) { - case QCOW2_CLUSTER_NORMAL: - if (l2_entry & QCOW_OFLAG_COPIED) { - goto out; - } + if (!cluster_needs_cow(bs, l2_entry)) { break; - case QCOW2_CLUSTER_UNALLOCATED: - case QCOW2_CLUSTER_COMPRESSED: - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: - break; - default: - abort(); } } -out: assert(i <= nb_clusters); return i; } From patchwork Mon May 25 18:08:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32BAEC433DF for ; Mon, 25 May 2020 18:10:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE69A2071C for ; Mon, 25 May 2020 18:10:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="DCc6nreN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE69A2071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45130 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHYm-00055n-1x for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:10:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45066) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXX-00039A-2Z; Mon, 25 May 2020 14:09:31 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50909) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXV-0007pO-DI; Mon, 25 May 2020 14:09:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=nbO3nb5Jr8LDIl5pGkOOovKAiJ7AK3DfITB5Uw23ytY=; b=DCc6nreNGFr/NccsF/CIkPWgZYEezRVQqHhGaMDjn1COwQMd1NSnfv91GBGz9WyVgMW5e28NzwQXAdV+K3B+JA3DrTz99RQIGI0a+lrAZxJj+lK17018/+WZHl3B3L3gHWtleki0LS7yMkpVnABCI4QKeWOUEwOaJiBXTXuPurJ0dppbhnLtfXp9xtKnW+coLO4g5MUGk6D7DrKuaVTmcrMkZs2F7OoNPMnYEspL73bfPjyjPCClJsTxR3RX0EkIlrbNvAsd4ZCdtu/HcA8HZchppVH8aYxWl8repph1jLT8vA0f9+PWZ2DVHVBx1LQEQmQzJNt6dz0towM+kBAOMA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXO-00088F-VC; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002Mg-Lh; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 07/32] qcow2: Document the Extended L2 Entries feature Date: Mon, 25 May 2020 20:08:32 +0200 Message-Id: <4d3335f72753f4f1696a8d4e0fd42b34b901baa1.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- docs/interop/qcow2.txt | 68 ++++++++++++++++++++++++++++++++++++++++-- docs/qcow2-cache.txt | 19 +++++++++++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb723463f2..64e9345fb4 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -42,6 +42,9 @@ The first cluster of a qcow2 image contains the file header: as the maximum cluster size and won't be able to open images with larger cluster sizes. + Note: if the image has Extended L2 Entries then cluster_bits + must be at least 14 (i.e. 16384 byte clusters). + 24 - 31: size Virtual disk size in bytes. @@ -117,7 +120,12 @@ the next fields through header_length. clusters. The compression_type field must be present and not zero. - Bits 4-63: Reserved (set to 0) + Bit 4: Extended L2 Entries. If this bit is set then + L2 table entries use an extended format that + allows subcluster-based allocation. See the + Extended L2 Entries section for more details. + + Bits 5-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -498,7 +506,7 @@ cannot be relaxed without an incompatible layout change). Given an offset into the virtual disk, the offset into the image file can be obtained as follows: - l2_entries = (cluster_size / sizeof(uint64_t)) + l2_entries = (cluster_size / sizeof(uint64_t)) [*] l2_index = (offset / cluster_size) % l2_entries l1_index = (offset / cluster_size) / l2_entries @@ -508,6 +516,8 @@ obtained as follows: return cluster_offset + (offset % cluster_size) + [*] this changes if Extended L2 Entries are enabled, see next section + L1 table entry: Bit 0 - 8: Reserved (set to 0) @@ -548,7 +558,8 @@ Standard Cluster Descriptor: nor is data read from the backing file if the cluster is unallocated. - With version 2, this is always 0. + With version 2 or with extended L2 entries (see the next + section), this is always 0. 1 - 8: Reserved (set to 0) @@ -585,6 +596,57 @@ file (except if bit 0 in the Standard Cluster Descriptor is set). If there is no backing file or the backing file is smaller than the image, they shall read zeros for all parts that are not covered by the backing file. +== Extended L2 Entries == + +An image uses Extended L2 Entries if bit 4 is set on the incompatible_features +field of the header. + +In these images standard data clusters are divided into 32 subclusters of the +same size. They are contiguous and start from the beginning of the cluster. +Subclusters can be allocated independently and the L2 entry contains information +indicating the status of each one of them. Compressed data clusters don't have +subclusters so they are treated the same as in images without this feature. + +The size of an extended L2 entry is 128 bits so the number of entries per table +is calculated using this formula: + + l2_entries = (cluster_size / (2 * sizeof(uint64_t))) + +The first 64 bits have the same format as the standard L2 table entry described +in the previous section, with the exception of bit 0 of the standard cluster +descriptor. + +The last 64 bits contain a subcluster allocation bitmap with this format: + +Subcluster Allocation Bitmap (for standard clusters): + + Bit 0 - 31: Allocation status (one bit per subcluster) + + 1: the subcluster is allocated. In this case the + host cluster offset field must contain a valid + offset. + 0: the subcluster is not allocated. In this case + read requests shall go to the backing file or + return zeros if there is no backing file data. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x). + + 32 - 63 Subcluster reads as zeros (one bit per subcluster) + + 1: the subcluster reads as zeros. In this case the + allocation status bit must be unset. The host + cluster offset field may or may not be set. + 0: no effect. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x - 32). + +Subcluster Allocation Bitmap (for compressed clusters): + + Bit 0 - 63: Reserved (set to 0) + Compressed clusters don't have subclusters, + so this field is not used. == Snapshots == diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index d57f409861..5f763aa6bb 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -1,6 +1,6 @@ qcow2 L2/refcount cache configuration ===================================== -Copyright (C) 2015, 2018 Igalia, S.L. +Copyright (C) 2015, 2018-2020 Igalia, S.L. Author: Alberto Garcia This work is licensed under the terms of the GNU GPL, version 2 or @@ -222,3 +222,20 @@ support this functionality, and is 0 (disabled) on other platforms. This functionality currently relies on the MADV_DONTNEED argument for madvise() to actually free the memory. This is a Linux-specific feature, so cache-clean-interval is not supported on other systems. + + +Extended L2 Entries +------------------- +All numbers shown in this document are valid for qcow2 images with normal +64-bit L2 entries. + +Images with extended L2 entries need twice as much L2 metadata, so the L2 +cache size must be twice as large for the same disk space. + + disk_size = l2_cache_size * cluster_size / 16 + +i.e. + + l2_cache_size = disk_size * 16 / cluster_size + +Refcount blocks are not affected by this. From patchwork Mon May 25 18:08:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1734AC433DF for ; Mon, 25 May 2020 18:14:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DADD42071C for ; Mon, 25 May 2020 18:14:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="VmGla1hG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DADD42071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34244 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHcR-0004ED-1c for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:14:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45200) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0003sF-Nt; Mon, 25 May 2020 14:09:50 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50945) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007pV-3v; Mon, 25 May 2020 14:09:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=uMSNfuA2S8NkneqRVDi3oow07lOfWgFh5Rz5YXTcmCg=; b=VmGla1hG7hCFRm0Mjg9T4eBxOOYky5gJzw7q/+kTpVygouSQg/krGf/cu4/avqhrD3HpruvQ5R5nsQyaBrCbmGkvm7z8VaN0RqlbMgXvH5fgediJlfZTszPLxpEqBK9KgF47aGtIdTjRLaoxMrg93WU8dm5qVnVOGZRO17sBG13GfpiLILUJy9o90vVp3bgMVmr0/sxJ8KHpkA60IvraNnkG6CjbLPuN4e+zPnoqMSiD3eBPa7S2MMDSed0dWhvLWlh9IK4GdQ7v57BgBmPmgRmi9TrgEG5IopsH4P3hug+5yunwFGGuxuPZAaQznW66Mm/Np/zTY3VlCZlIN64U5A==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088G-8K; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002Mi-Mf; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 08/32] qcow2: Add dummy has_subclusters() function Date: Mon, 25 May 2020 20:08:33 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function will be used by the qcow2 code to check if an image has subclusters or not. At the moment this simply returns false. Once all patches needed for subcluster support are ready then QEMU will be able to create and read images with subclusters and this function will return the actual value. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index afee84f41f..f41bfd743f 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -510,6 +510,12 @@ typedef enum QCow2MetadataOverlap { #define INV_OFFSET (-1ULL) +static inline bool has_subclusters(BDRVQcow2State *s) +{ + /* FIXME: Return false until this feature is complete */ + return false; +} + static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx) { From patchwork Mon May 25 18:08:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96AE6C433E0 for ; Mon, 25 May 2020 18:12:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5CC2C2071C for ; Mon, 25 May 2020 18:12:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="MTlAVfWo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CC2C2071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHah-0000EH-FS for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:12:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45068) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXX-00039K-HP; Mon, 25 May 2020 14:09:31 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50907) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXV-0007pM-HX; Mon, 25 May 2020 14:09:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=8zplI1UiwFKJ83AGO9lzNzyl1XqngZYwma9UQPZedjE=; b=MTlAVfWolUnZfVYtdLi8huumMYXXS8V1C3zXLRkvlHbF8UJ3OSb/VHAqfhhssoSCfd3cgaSdxJNLXTcaHGOX1wHl3YLEjK6owmX1Bpcph8UpIkRBZ4FnIrUFTze4Zg7r73Wcn+hGhhJgi3olCXabWoGv7iK2yXjN85u9ilExPH7v2sf18jBic7m6M5s+UwXbvrbEbY88Snfo+Jy8ljZlz/iqlB9x8wPe9EIui/as7L00v/gxedncCIYvLAHtJYmE/SvLQIbp5TvQeEo7SzPqsMr2WLhiVkelaHNvuUuilOtHjL6ZZLP6FDDyOAk59NEYNng7GG0hvHnb9yEGVV1r6g==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXO-00088H-Vi; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002Mk-Ns; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 09/32] qcow2: Add subcluster-related fields to BDRVQcow2State Date: Mon, 25 May 2020 20:08:34 +0200 Message-Id: <1beaf5e17851bde3a91af1f88abc7211351f968e.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This patch adds the following new fields to BDRVQcow2State: - subclusters_per_cluster: Number of subclusters in a cluster - subcluster_size: The size of each subcluster, in bytes - subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size Images without subclusters are treated as if they had exactly one subcluster per cluster (i.e. subcluster_size = cluster_size). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 5 +++++ block/qcow2.c | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index f41bfd743f..4e9c732831 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -78,6 +78,8 @@ /* The cluster reads as all zeros */ #define QCOW_OFLAG_ZERO (1ULL << 0) +#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 + #define MIN_CLUSTER_BITS 9 #define MAX_CLUSTER_BITS 21 @@ -295,6 +297,9 @@ typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; int l2_slice_size; + int subcluster_bits; + int subcluster_size; + int subclusters_per_cluster; int l2_bits; int l2_size; int l1_size; diff --git a/block/qcow2.c b/block/qcow2.c index 6562eb1590..916b6c3f92 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1433,6 +1433,11 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } } + s->subclusters_per_cluster = + has_subclusters(s) ? QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER : 1; + s->subcluster_size = s->cluster_size / s->subclusters_per_cluster; + s->subcluster_bits = ctz32(s->subcluster_size); + /* Check support for various header values */ if (header.refcount_order > 6) { error_setg(errp, "Reference count entry width too large; may not " From patchwork Mon May 25 18:08:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281924 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52580C433DF for ; Mon, 25 May 2020 18:26:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2129C20776 for ; Mon, 25 May 2020 18:26:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="GQKMWxQ/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2129C20776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49834 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHnd-0006kV-B7 for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:26:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45250) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXw-00046u-Ed; Mon, 25 May 2020 14:09:56 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51079) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0007q1-28; Mon, 25 May 2020 14:09:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Zq9XNkVxKpLd5N9/U3MTc61RDU9ATTpNnf0GgQTwBzU=; b=GQKMWxQ/2RAFmd/FLvoWBWXyotDvxAWM3qrC+P1N1GM5ZDy7ByDiJKfsVdPIOsbluZBUl9OJIkbNIoMVjOznJeT/dUJTkawkcCiJTIoQ+HMYU9/E0rdBWBhM5UsisGY4VP4B4v0jYjQWmvWKqMup/Ms1oap9xo4TJOCJtyxbR55MBdXDSgY18c8YsjjTZvrVn8cB1KXkRMZCZyqGhRznOKjsa9Z+M5UravIpOk1kgKaCqCfSIYiSazEXW1AFOcEFc0KNnScgpUOK7y/pbmxNTxqk5DRHhuABZ0yZP8OGVnYVqSUWkbetd+j4YUduA+IbKkOI5eEGeLzqVoYtiAd3+w==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088K-Fj; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002Mv-R8; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 12/32] qcow2: Add l2_entry_size() Date: Mon, 25 May 2020 20:08:37 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2 images with subclusters have 128-bit L2 entries. The first 64 bits contain the same information as traditional images and the last 64 bits form a bitmap with the status of each individual subcluster. Because of that we cannot assume that L2 entries are sizeof(uint64_t) anymore. This function returns the proper value for the image. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2.h | 9 +++++++++ block/qcow2-cluster.c | 12 ++++++------ block/qcow2-refcount.c | 14 ++++++++------ block/qcow2.c | 8 ++++---- 4 files changed, 27 insertions(+), 16 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 79c4f82383..95cc16a089 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -80,6 +80,10 @@ #define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 +/* Size of normal and extended L2 entries */ +#define L2E_SIZE_NORMAL (sizeof(uint64_t)) +#define L2E_SIZE_EXTENDED (sizeof(uint64_t) * 2) + #define MIN_CLUSTER_BITS 9 #define MAX_CLUSTER_BITS 21 @@ -521,6 +525,11 @@ static inline bool has_subclusters(BDRVQcow2State *s) return false; } +static inline size_t l2_entry_size(BDRVQcow2State *s) +{ + return has_subclusters(s) ? L2E_SIZE_EXTENDED : L2E_SIZE_NORMAL; +} + static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx) { diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 76fd0f3cdb..8b2fc550b7 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -208,7 +208,7 @@ static int l2_load(BlockDriverState *bs, uint64_t offset, uint64_t l2_offset, uint64_t **l2_slice) { BDRVQcow2State *s = bs->opaque; - int start_of_slice = sizeof(uint64_t) * + int start_of_slice = l2_entry_size(s) * (offset_to_l2_index(s, offset) - offset_to_l2_slice_index(s, offset)); return qcow2_cache_get(bs, s->l2_table_cache, l2_offset + start_of_slice, @@ -281,7 +281,7 @@ static int l2_allocate(BlockDriverState *bs, int l1_index) /* allocate a new l2 entry */ - l2_offset = qcow2_alloc_clusters(bs, s->l2_size * sizeof(uint64_t)); + l2_offset = qcow2_alloc_clusters(bs, s->l2_size * l2_entry_size(s)); if (l2_offset < 0) { ret = l2_offset; goto fail; @@ -305,7 +305,7 @@ static int l2_allocate(BlockDriverState *bs, int l1_index) /* allocate a new entry in the l2 cache */ - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; trace_qcow2_l2_allocate_get_empty(bs, l1_index); @@ -369,7 +369,7 @@ fail: } s->l1_table[l1_index] = old_l2_offset; if (l2_offset > 0) { - qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t), + qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s), QCOW2_DISCARD_ALWAYS); } return ret; @@ -716,7 +716,7 @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset, /* Then decrease the refcount of the old table */ if (l2_offset) { - qcow2_free_clusters(bs, l2_offset, s->l2_size * sizeof(uint64_t), + qcow2_free_clusters(bs, l2_offset, s->l2_size * l2_entry_size(s), QCOW2_DISCARD_OTHER); } @@ -1913,7 +1913,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, int ret; int i, j; - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; if (!is_active_l1) { diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 04546838e8..770c5dbc83 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1254,7 +1254,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, l2_slice = NULL; l1_table = NULL; l1_size2 = l1_size * sizeof(uint64_t); - slice_size2 = s->l2_slice_size * sizeof(uint64_t); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; s->cache_discards = true; @@ -1605,7 +1605,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, int i, l2_size, nb_csectors, ret; /* Read L2 table from disk */ - l2_size = s->l2_size * sizeof(uint64_t); + l2_size = s->l2_size * l2_entry_size(s); l2_table = g_malloc(l2_size); ret = bdrv_pread(bs->file, l2_offset, l2_table, l2_size); @@ -1680,15 +1680,16 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", offset); if (fix & BDRV_FIX_ERRORS) { + int idx = i * (l2_entry_size(s) / sizeof(uint64_t)); uint64_t l2e_offset = - l2_offset + (uint64_t)i * sizeof(uint64_t); + l2_offset + (uint64_t)i * l2_entry_size(s); int ign = active ? QCOW2_OL_ACTIVE_L2 : QCOW2_OL_INACTIVE_L2; l2_entry = QCOW_OFLAG_ZERO; set_l2_entry(s, l2_table, i, l2_entry); ret = qcow2_pre_write_overlap_check(bs, ign, - l2e_offset, sizeof(uint64_t), false); + l2e_offset, l2_entry_size(s), false); if (ret < 0) { fprintf(stderr, "ERROR: Overlap check failed\n"); res->check_errors++; @@ -1698,7 +1699,8 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, } ret = bdrv_pwrite_sync(bs->file, l2e_offset, - &l2_table[i], sizeof(uint64_t)); + &l2_table[idx], + l2_entry_size(s)); if (ret < 0) { fprintf(stderr, "ERROR: Failed to overwrite L2 " "table entry: %s\n", strerror(-ret)); @@ -1905,7 +1907,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, } ret = bdrv_pread(bs->file, l2_offset, l2_table, - s->l2_size * sizeof(uint64_t)); + s->l2_size * l2_entry_size(s)); if (ret < 0) { fprintf(stderr, "ERROR: Could not read L2 table: %s\n", strerror(-ret)); diff --git a/block/qcow2.c b/block/qcow2.c index 916b6c3f92..d4851e0e18 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -870,7 +870,7 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, uint64_t max_l2_entries = DIV_ROUND_UP(virtual_disk_size, s->cluster_size); /* An L2 table is always one cluster in size so the max cache size * should be a multiple of the cluster size. */ - uint64_t max_l2_cache = ROUND_UP(max_l2_entries * sizeof(uint64_t), + uint64_t max_l2_cache = ROUND_UP(max_l2_entries * l2_entry_size(s), s->cluster_size); combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); @@ -1031,7 +1031,7 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, } } - r->l2_slice_size = l2_cache_entry_size / sizeof(uint64_t); + r->l2_slice_size = l2_cache_entry_size / l2_entry_size(s); r->l2_table_cache = qcow2_cache_create(bs, l2_cache_size, l2_cache_entry_size); r->refcount_block_cache = qcow2_cache_create(bs, refcount_cache_size, @@ -1478,7 +1478,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, bs->encrypted = true; } - s->l2_bits = s->cluster_bits - 3; /* L2 is always one cluster */ + s->l2_bits = s->cluster_bits - ctz32(l2_entry_size(s)); s->l2_size = 1 << s->l2_bits; /* 2^(s->refcount_order - 3) is the refcount width in bytes */ s->refcount_block_bits = s->cluster_bits - (s->refcount_order - 3); @@ -4245,7 +4245,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, * preallocation. All that matters is that we will not have to allocate * new refcount structures for them.) */ nb_new_l2_tables = DIV_ROUND_UP(nb_new_data_clusters, - s->cluster_size / sizeof(uint64_t)); + s->cluster_size / l2_entry_size(s)); /* The cluster range may not be aligned to L2 boundaries, so add one L2 * table for a potential head/tail */ nb_new_l2_tables++; From patchwork Mon May 25 18:08:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AB87C433DF for ; Mon, 25 May 2020 18:17:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5E3542071C for ; Mon, 25 May 2020 18:17:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="eaTTv25z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E3542071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45536 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHfb-0000hL-JJ for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:17:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0003qy-5D; Mon, 25 May 2020 14:09:50 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51004) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXo-0007pp-St; Mon, 25 May 2020 14:09:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=mZe7TNIehJMws3QyTodGPWlYY7BX5/x1bf4eeES1kLs=; b=eaTTv25z/ofaRjbTM5Cisny7GAVHiAv6RTjoP+jlwxxbA6WxnlQbw2TrHE/+ZXel5gtZJ0oRFBJPJfVo3D1lc2S5rZ90JfGb8Cix67pPu837fbFtmm/blgXdG4di/HuOEwnBWuUbIUZ42nZZfDa4TuKpPJEKuIS6WHMuI3TovvaQulq8jdivo7cYXdWEbkZRLHITh//C/MCu3rRd7mgj5H1TxGH67atWW7nYt39iQs/IPsVUWyKRbqQDP7B1fM8W5LU3wntsdf/xoAFEEMKGSP105iz3Mz2qqzFpbieMtr6zvOAsA1JJnSamfdeWQdo/dCNTOYR559EaRDyeqs0alA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088M-DN; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002N0-T5; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 14/32] qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type() Date: Mon, 25 May 2020 20:08:39 +0200 Message-Id: <961e6f3e3af13a25c666859b391b7ed147873d8b.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This patch adds QCow2SubclusterType, which is the subcluster-level version of QCow2ClusterType. All QCOW2_SUBCLUSTER_* values have the the same meaning as their QCOW2_CLUSTER_* equivalents (when they exist). See below for details and caveats. In images without extended L2 entries clusters are treated as having exactly one subcluster so it is possible to replace one data type with the other while keeping the exact same semantics. With extended L2 entries there are new possible values, and every subcluster in the same cluster can obviously have a different QCow2SubclusterType so functions need to be adapted to work on the subcluster level. There are several things that have to be taken into account: a) QCOW2_SUBCLUSTER_COMPRESSED means that the whole cluster is compressed. We do not support compression at the subcluster level. b) There are two different values for unallocated subclusters: QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN which means that the whole cluster is unallocated, and QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC which means that the cluster is allocated but the subcluster is not. The latter can only happen in images with extended L2 entries. c) QCOW2_SUBCLUSTER_INVALID is used to detect the cases where an L2 entry has a value that violates the specification. The caller is responsible for handling these situations. To prevent compatibility problems with images that have invalid values but are currently being read by QEMU without causing side effects, QCOW2_SUBCLUSTER_INVALID is only returned for images with extended L2 entries. qcow2_cluster_to_subcluster_type() is added as a separate function from qcow2_get_subcluster_type(), but this is only temporary and both will be merged in a subsequent patch. Signed-off-by: Alberto Garcia --- block/qcow2.h | 126 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 125 insertions(+), 1 deletion(-) diff --git a/block/qcow2.h b/block/qcow2.h index 5c6bf48c7a..27dbcbc502 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -80,6 +80,21 @@ #define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 +/* The subcluster X [0..31] is allocated */ +#define QCOW_OFLAG_SUB_ALLOC(X) (1ULL << (X)) +/* The subcluster X [0..31] reads as zeroes */ +#define QCOW_OFLAG_SUB_ZERO(X) (QCOW_OFLAG_SUB_ALLOC(X) << 32) +/* Subclusters [X, Y) (0 <= X <= Y <= 32) are allocated */ +#define QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) \ + (QCOW_OFLAG_SUB_ALLOC(Y) - QCOW_OFLAG_SUB_ALLOC(X)) +/* Subclusters [X, Y) (0 <= X <= Y <= 32) read as zeroes */ +#define QCOW_OFLAG_SUB_ZERO_RANGE(X, Y) \ + (QCOW_OFLAG_SUB_ALLOC_RANGE(X, Y) << 32) +/* L2 entry bitmap with all allocation bits set */ +#define QCOW_L2_BITMAP_ALL_ALLOC (QCOW_OFLAG_SUB_ALLOC_RANGE(0, 32)) +/* L2 entry bitmap with all "read as zeroes" bits set */ +#define QCOW_L2_BITMAP_ALL_ZEROES (QCOW_OFLAG_SUB_ZERO_RANGE(0, 32)) + /* Size of normal and extended L2 entries */ #define L2E_SIZE_NORMAL (sizeof(uint64_t)) #define L2E_SIZE_EXTENDED (sizeof(uint64_t) * 2) @@ -462,6 +477,33 @@ typedef struct QCowL2Meta QLIST_ENTRY(QCowL2Meta) next_in_flight; } QCowL2Meta; +/* + * In images with standard L2 entries all clusters are treated as if + * they had one subcluster so QCow2ClusterType and QCow2SubclusterType + * can be mapped to each other and have the exact same meaning + * (QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC cannot happen in these images). + * + * In images with extended L2 entries QCow2ClusterType refers to the + * complete cluster and QCow2SubclusterType to each of the individual + * subclusters, so there are several possible combinations: + * + * |--------------+---------------------------| + * | Cluster type | Possible subcluster types | + * |--------------+---------------------------| + * | UNALLOCATED | UNALLOCATED_PLAIN | + * | | ZERO_PLAIN | + * |--------------+---------------------------| + * | NORMAL | UNALLOCATED_ALLOC | + * | | ZERO_ALLOC | + * | | NORMAL | + * |--------------+---------------------------| + * | COMPRESSED | COMPRESSED | + * |--------------+---------------------------| + * + * QCOW2_SUBCLUSTER_INVALID means that the L2 entry is incorrect and + * the image should be marked corrupt. + */ + typedef enum QCow2ClusterType { QCOW2_CLUSTER_UNALLOCATED, QCOW2_CLUSTER_ZERO_PLAIN, @@ -470,6 +512,16 @@ typedef enum QCow2ClusterType { QCOW2_CLUSTER_COMPRESSED, } QCow2ClusterType; +typedef enum QCow2SubclusterType { + QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN, + QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC, + QCOW2_SUBCLUSTER_ZERO_PLAIN, + QCOW2_SUBCLUSTER_ZERO_ALLOC, + QCOW2_SUBCLUSTER_NORMAL, + QCOW2_SUBCLUSTER_COMPRESSED, + QCOW2_SUBCLUSTER_INVALID, +} QCow2SubclusterType; + typedef enum QCow2MetadataOverlap { QCOW2_OL_MAIN_HEADER_BITNR = 0, QCOW2_OL_ACTIVE_L1_BITNR = 1, @@ -634,9 +686,11 @@ static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s) static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, uint64_t l2_entry) { + BDRVQcow2State *s = bs->opaque; + if (l2_entry & QCOW_OFLAG_COMPRESSED) { return QCOW2_CLUSTER_COMPRESSED; - } else if (l2_entry & QCOW_OFLAG_ZERO) { + } else if ((l2_entry & QCOW_OFLAG_ZERO) && !has_subclusters(s)) { if (l2_entry & L2E_OFFSET_MASK) { return QCOW2_CLUSTER_ZERO_ALLOC; } @@ -656,6 +710,76 @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, } } +/* + * For an image without extended L2 entries, return the + * QCow2SubclusterType equivalent of a given QCow2ClusterType. + */ +static inline +QCow2SubclusterType qcow2_cluster_to_subcluster_type(QCow2ClusterType type) +{ + switch (type) { + case QCOW2_CLUSTER_COMPRESSED: + return QCOW2_SUBCLUSTER_COMPRESSED; + case QCOW2_CLUSTER_ZERO_PLAIN: + return QCOW2_SUBCLUSTER_ZERO_PLAIN; + case QCOW2_CLUSTER_ZERO_ALLOC: + return QCOW2_SUBCLUSTER_ZERO_ALLOC; + case QCOW2_CLUSTER_NORMAL: + return QCOW2_SUBCLUSTER_NORMAL; + case QCOW2_CLUSTER_UNALLOCATED: + return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; + default: + g_assert_not_reached(); + } +} + +/* + * In an image without subsclusters @l2_bitmap is ignored and + * @sc_index must be 0. + * Return QCOW2_SUBCLUSTER_INVALID if an invalid l2 entry is detected + * (this checks the whole entry and bitmap, not only the bits related + * to subcluster @sc_index). + */ +static inline +QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs, + uint64_t l2_entry, + uint64_t l2_bitmap, + unsigned sc_index) +{ + BDRVQcow2State *s = bs->opaque; + QCow2ClusterType type = qcow2_get_cluster_type(bs, l2_entry); + assert(sc_index < s->subclusters_per_cluster); + + if (has_subclusters(s)) { + switch (type) { + case QCOW2_CLUSTER_COMPRESSED: + return QCOW2_SUBCLUSTER_COMPRESSED; + case QCOW2_CLUSTER_NORMAL: + if ((l2_bitmap >> 32) & l2_bitmap) { + return QCOW2_SUBCLUSTER_INVALID; + } else if (l2_bitmap & QCOW_OFLAG_SUB_ZERO(sc_index)) { + return QCOW2_SUBCLUSTER_ZERO_ALLOC; + } else if (l2_bitmap & QCOW_OFLAG_SUB_ALLOC(sc_index)) { + return QCOW2_SUBCLUSTER_NORMAL; + } else { + return QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC; + } + case QCOW2_CLUSTER_UNALLOCATED: + if (l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC) { + return QCOW2_SUBCLUSTER_INVALID; + } else if (l2_bitmap & QCOW_OFLAG_SUB_ZERO(sc_index)) { + return QCOW2_SUBCLUSTER_ZERO_PLAIN; + } else { + return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; + } + default: + g_assert_not_reached(); + } + } else { + return qcow2_cluster_to_subcluster_type(type); + } +} + /* Check whether refcounts are eager or lazy */ static inline bool qcow2_need_accurate_refcounts(BDRVQcow2State *s) { From patchwork Mon May 25 18:08:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4614CC433DF for ; Mon, 25 May 2020 18:14:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 160372071C for ; Mon, 25 May 2020 18:14:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="M24tpS/W" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 160372071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHcn-0004km-7z for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:14:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45074) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXY-00039j-Av; Mon, 25 May 2020 14:09:32 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50994) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXV-0007ph-Co; Mon, 25 May 2020 14:09:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=VM/l5Zr2yE+YJDPCCI/z23hervw4AluXjgu6TqCtgN8=; b=M24tpS/Wxkal3Q/A4EGTCQsmK/TgsAKl2gCX66EPMq3GO6Zwx3Y09DWlafghnVBTzgzZelbreVSCs3b3+S7J8FQaPsrg9C1vtt21gG7IST49HoqnTHg+oMJDIApJRANzVv2nnWvJw/il6wXGh43GmuK3fOXx4hZXF4M0+ZFs2miqmtf89PWbOy6t+jfR6QiYQP8VInHD0BgiSvO/SdmDWGA6PLqnYeDux24iUKKj4RBKPU8Yl0YheT1nbN73sFReAh/Q02yKA/ONStV6bcd1oCVao5gGlPe/CPkIPinHy+UDYq4GeAxmBzwiHAcN8M0w3+rXwe1Jr3EHD1HzlhXe/A==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088N-EX; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHX9-0002N3-Tx; Mon, 25 May 2020 20:09:07 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 15/32] qcow2: Add qcow2_get_subcluster_range_type() Date: Mon, 25 May 2020 20:08:40 +0200 Message-Id: <211d0be592c32aebd29ed20339fa68e7c4c4ac90.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There are situations in which we want to know how many contiguous subclusters of the same type there are in a given cluster. This can be done by simply iterating over the subclusters and repeatedly calling qcow2_get_subcluster_type() for each one of them. However once we determined the type of a subcluster we can check the rest efficiently by counting the number of adjacent ones (or zeroes) in the bitmap. This is what this function does. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 51 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 8b2fc550b7..32dc6e75e3 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -375,6 +375,57 @@ fail: return ret; } +/* + * For a given L2 entry, count the number of contiguous subclusters of + * the same type starting from @sc_from. Compressed clusters are + * treated as if they were divided into subclusters of size + * s->subcluster_size. + * + * Return the number of contiguous subclusters and set @type to the + * subcluster type. + * + * If the L2 entry is invalid return -errno and set @type to + * QCOW2_SUBCLUSTER_INVALID. + */ +G_GNUC_UNUSED +static int qcow2_get_subcluster_range_type(BlockDriverState *bs, + uint64_t l2_entry, + uint64_t l2_bitmap, + unsigned sc_from, + QCow2SubclusterType *type) +{ + BDRVQcow2State *s = bs->opaque; + uint32_t val; + + *type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_from); + + if (*type == QCOW2_SUBCLUSTER_INVALID) { + return -EINVAL; + } else if (!has_subclusters(s) || *type == QCOW2_SUBCLUSTER_COMPRESSED) { + return s->subclusters_per_cluster - sc_from; + } + + switch (*type) { + case QCOW2_SUBCLUSTER_NORMAL: + val = l2_bitmap | QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + val = (l2_bitmap | QCOW_OFLAG_SUB_ZERO_RANGE(0, sc_from)) >> 32; + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + val = ((l2_bitmap >> 32) | l2_bitmap) + & ~QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return ctz32(val) - sc_from; + + default: + g_assert_not_reached(); + } +} + /* * Checks how many clusters in a given L2 slice are contiguous in the image * file. As soon as one of the flags in the bitmask stop_flags changes compared From patchwork Mon May 25 18:08:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 982A0C433E0 for ; Mon, 25 May 2020 18:20:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5270F2071C for ; Mon, 25 May 2020 18:20:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="fqJDgqWC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5270F2071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52008 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHhf-0003Vf-Ek for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:19:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXt-0003yI-5G; Mon, 25 May 2020 14:09:53 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51106) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007qJ-6k; Mon, 25 May 2020 14:09:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=pUDQIBuLiiwjL8nRIb6y+RnaUwjFdwwzqljOft/ShgQ=; b=fqJDgqWCzSYZK7AM3VIxI+nyu3brxz6gjOa+gziGJqJ6T0yuIPHykb0iAGRKj3/8MWsXDunYXwhShcczTJyx47XU55G57nSfO6dwmo0Wym6NWM7NcEcmHwW8q1KFR9eEyN1JDMgWmUvhyVvgmnPbetGldjJhwB8AvpI5fpA2xAXYluG1defRIqTJH62RMayqk889IrcXC7zocwVO0j7sOahv7BinlxbW46EYCCx5j3LiIpcScMauU6KXckAzSI3AlFnzZv934cKAqIGzd1Zd7JUY1zwiWqYx+1i8P7TiuKuVqMS7cUxdghg9L6gUTx3l03+W2ox4JGdYussjW8mCiA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088Q-Gk; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002ND-1a; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 18/32] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_* Date: Mon, 25 May 2020 20:08:43 +0200 Message-Id: <281cf6c52f28d4123ed4ce81f701562dd73a7694.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" In order to support extended L2 entries some functions of the qcow2 driver need to start dealing with subclusters instead of clusters. qcow2_get_host_offset() is modified to return the subcluster type instead of the cluster type, and all callers are updated to replace all values of QCow2ClusterType with their QCow2SubclusterType equivalents. This patch only changes the data types, there are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 2 +- block/qcow2-cluster.c | 10 +++---- block/qcow2.c | 70 ++++++++++++++++++++++--------------------- 3 files changed, 42 insertions(+), 40 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 9cd7cbd895..84c06d9bba 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -894,7 +894,7 @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type); + QCow2SubclusterType *subcluster_type); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 1e5681b0c6..ed7b92dbb2 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -564,15 +564,15 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * offset that we are interested in. * * On exit, *bytes is the number of bytes starting at offset that have the same - * cluster type and (if applicable) are stored contiguously in the image file. - * The cluster type is stored in *cluster_type. - * Compressed clusters are always returned one by one. + * subcluster type and (if applicable) are stored contiguously in the image + * file. The subcluster type is stored in *subcluster_type. + * Compressed clusters are always processed one by one. * * Returns 0 on success, -errno in error cases. */ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type) + QCow2SubclusterType *subcluster_type) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; @@ -713,7 +713,7 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - *cluster_type = type; + *subcluster_type = qcow2_cluster_to_subcluster_type(type); return 0; diff --git a/block/qcow2.c b/block/qcow2.c index 2806642f68..5303b0070b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2033,7 +2033,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; uint64_t host_offset; unsigned int bytes; - QCow2ClusterType type; + QCow2SubclusterType type; int ret, status = 0; qemu_co_mutex_lock(&s->lock); @@ -2053,15 +2053,16 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) && - !s->crypto) { + if ((type == QCOW2_SUBCLUSTER_NORMAL || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) { + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (type != QCOW2_CLUSTER_UNALLOCATED) { + } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2158,7 +2159,7 @@ typedef struct Qcow2AioTask { AioTask task; BlockDriverState *bs; - QCow2ClusterType cluster_type; /* only for read */ + QCow2SubclusterType subcluster_type; /* only for read */ uint64_t host_offset; /* or full descriptor in compressed clusters */ uint64_t offset; uint64_t bytes; @@ -2171,7 +2172,7 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); static coroutine_fn int qcow2_add_task(BlockDriverState *bs, AioTaskPool *pool, AioTaskFunc func, - QCow2ClusterType cluster_type, + QCow2SubclusterType subcluster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, @@ -2185,7 +2186,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, *task = (Qcow2AioTask) { .task.func = func, .bs = bs, - .cluster_type = cluster_type, + .subcluster_type = subcluster_type, .qiov = qiov, .host_offset = host_offset, .offset = offset, @@ -2196,7 +2197,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, func == qcow2_co_preadv_task_entry ? "read" : "write", - cluster_type, host_offset, offset, bytes, + subcluster_type, host_offset, offset, bytes, qiov, qiov_offset); if (!pool) { @@ -2209,7 +2210,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, } static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, - QCow2ClusterType cluster_type, + QCow2SubclusterType subc_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, @@ -2217,24 +2218,24 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; - switch (cluster_type) { - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + switch (subc_type) { + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: /* Both zero types are handled in qcow2_co_preadv_part */ g_assert_not_reached(); - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */ BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); return bdrv_co_preadv_part(bs->backing, offset, bytes, qiov, qiov_offset, 0); - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: return qcow2_co_preadv_compressed(bs, host_offset, offset, bytes, qiov, qiov_offset); - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: if (bs->encrypted) { return qcow2_co_preadv_encrypted(bs, host_offset, offset, bytes, qiov, qiov_offset); @@ -2257,8 +2258,9 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task) assert(!t->l2meta); - return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset, - t->offset, t->bytes, t->qiov, t->qiov_offset); + return qcow2_co_preadv_task(t->bs, t->subcluster_type, + t->host_offset, t->offset, t->bytes, + t->qiov, t->qiov_offset); } static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, @@ -2270,7 +2272,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t host_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2289,9 +2291,9 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, goto out; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || - type == QCOW2_CLUSTER_ZERO_ALLOC || - (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { @@ -2524,7 +2526,7 @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type); + assert(!t->subcluster_type); return qcow2_co_pwritev_task(t->bs, t->host_offset, t->offset, t->bytes, t->qiov, t->qiov_offset, @@ -3844,7 +3846,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, if (head || tail) { uint64_t off; unsigned int nr; - QCow2ClusterType type; + QCow2SubclusterType type; assert(head + bytes <= s->cluster_size); @@ -3862,9 +3864,9 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, nr = s->cluster_size; ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); if (ret < 0 || - (type != QCOW2_CLUSTER_UNALLOCATED && - type != QCOW2_CLUSTER_ZERO_PLAIN && - type != QCOW2_CLUSTER_ZERO_ALLOC)) { + (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); return -ENOTSUP; } @@ -3928,7 +3930,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, while (bytes != 0) { uint64_t copy_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; @@ -3940,7 +3942,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } switch (type) { - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); if (src_offset >= backing_length) { @@ -3955,16 +3957,16 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } break; - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: cur_write_flags |= BDRV_REQ_ZERO_WRITE; break; - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: ret = -ENOTSUP; goto out; - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: child = s->data_file; break; @@ -4482,7 +4484,7 @@ static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type && !t->l2meta); + assert(!t->subcluster_type && !t->l2meta); return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov, t->qiov_offset); From patchwork Mon May 25 18:08:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3E95C433E0 for ; Mon, 25 May 2020 18:27:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9610A20776 for ; Mon, 25 May 2020 18:27:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="CKOgjOT1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9610A20776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:56986 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHpD-0001Az-Pn for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:27:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXx-00048x-AD; Mon, 25 May 2020 14:09:57 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51077) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0007q4-6B; Mon, 25 May 2020 14:09:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=zzfPBeB2ItEp/1XWL1acaEjXDL4NY9IH4UTpJ570sAI=; b=CKOgjOT1NnRdDtkQSuLzYDASwN2CHgurea1KmeaGLTFwEGu4dD60Vf9Ci2U8+Ww83ziy3l4vC9jMbGehehzoemDBMFUDAzP+st/BpWMiwtEe2XciJ5b/TPYnNddf2K0CtIiE6ddaozqNPC6QogRW0o7tCVGrKyLY0fh7Dt+/DoYUWmJrFOOU8DtGSTPuB3G/WRY1GI2/yBZcmmBnBBAxXIPF/U1I64wqu4ZB73MamudLt+9gEbGKnd0DpuMNjM0c0/7FDfGjEYL2PjVbpxl6Z+PJez99AuYn3Zas89gPHqfxM1OYpI+HeznQ1bEZy613TaYPHBAZOBlMdhgmbzptzw==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088T-Iu; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002NI-3g; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 20/32] qcow2: Add subcluster support to calculate_l2_meta() Date: Mon, 25 May 2020 20:08:45 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If an image has subclusters then there are more copy-on-write scenarios that we need to consider. Let's say we have a write request from the middle of subcluster #3 until the end of the cluster: 1) If we are writing to a newly allocated cluster then we need copy-on-write. The previous contents of subclusters #0 to #3 must be copied to the new cluster. We can optimize this process by skipping all leading unallocated or zero subclusters (the status of those skipped subclusters will be reflected in the new L2 bitmap). 2) If we are overwriting an existing cluster: 2.1) If subcluster #3 is unallocated or has the all-zeroes bit set then we need copy-on-write (on subcluster #3 only). 2.2) If subcluster #3 was already allocated then there is no need for any copy-on-write. However we still need to update the L2 bitmap to reflect possible changes in the allocation status of subclusters #4 to #31. Because of this, this function checks if all the overwritten subclusters are already allocated and in this case it returns without creating a new QCowL2Meta structure. After all these changes l2meta_cow_start() and l2meta_cow_end() are not necessarily cluster-aligned anymore. We need to update the calculation of old_start and old_end in handle_dependencies() to guarantee that no two requests try to write on the same cluster. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 163 +++++++++++++++++++++++++++++++++--------- 1 file changed, 131 insertions(+), 32 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index ed7b92dbb2..59dd9bda29 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -387,7 +387,6 @@ fail: * If the L2 entry is invalid return -errno and set @type to * QCOW2_SUBCLUSTER_INVALID. */ -G_GNUC_UNUSED static int qcow2_get_subcluster_range_type(BlockDriverState *bs, uint64_t l2_entry, uint64_t l2_bitmap, @@ -1110,56 +1109,148 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) * If @keep_old is true it means that the clusters were already * allocated and will be overwritten. If false then the clusters are * new and we have to decrease the reference count of the old ones. + * + * Returns 0 on success, -errno on failure. */ -static void calculate_l2_meta(BlockDriverState *bs, - uint64_t host_cluster_offset, - uint64_t guest_offset, unsigned bytes, - uint64_t *l2_slice, QCowL2Meta **m, bool keep_old) +static int calculate_l2_meta(BlockDriverState *bs, uint64_t host_cluster_offset, + uint64_t guest_offset, unsigned bytes, + uint64_t *l2_slice, QCowL2Meta **m, bool keep_old) { BDRVQcow2State *s = bs->opaque; - int l2_index = offset_to_l2_slice_index(s, guest_offset); - uint64_t l2_entry; + int sc_index, l2_index = offset_to_l2_slice_index(s, guest_offset); + uint64_t l2_entry, l2_bitmap; unsigned cow_start_from, cow_end_to; unsigned cow_start_to = offset_into_cluster(s, guest_offset); unsigned cow_end_from = cow_start_to + bytes; unsigned nb_clusters = size_to_clusters(s, cow_end_from); QCowL2Meta *old_m = *m; - QCow2ClusterType type; + QCow2SubclusterType type; + int i; + bool skip_cow = keep_old; assert(nb_clusters <= s->l2_slice_size - l2_index); - /* Return if there's no COW (all clusters are normal and we keep them) */ - if (keep_old) { - int i; - for (i = 0; i < nb_clusters; i++) { - l2_entry = get_l2_entry(s, l2_slice, l2_index + i); - if (qcow2_get_cluster_type(bs, l2_entry) != QCOW2_CLUSTER_NORMAL) { - break; + /* Check the type of all affected subclusters */ + for (i = 0; i < nb_clusters; i++) { + l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + if (skip_cow) { + unsigned write_from = MAX(cow_start_to, i << s->cluster_bits); + unsigned write_to = MIN(cow_end_from, (i + 1) << s->cluster_bits); + int first_sc = offset_to_sc_index(s, write_from); + int last_sc = offset_to_sc_index(s, write_to - 1); + int cnt = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap, + first_sc, &type); + /* Is any of the subclusters of type != QCOW2_SUBCLUSTER_NORMAL ? */ + if (type != QCOW2_SUBCLUSTER_NORMAL || first_sc + cnt <= last_sc) { + skip_cow = false; } + } else { + /* If we can't skip the cow we can still look for invalid entries */ + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, 0); } - if (i == nb_clusters) { - return; + if (type == QCOW2_SUBCLUSTER_INVALID) { + int l1_index = offset_to_l1_index(s, guest_offset); + uint64_t l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK; + qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster " + "entry found (L2 offset: %#" PRIx64 + ", L2 index: %#x)", + l2_offset, l2_index + i); + return -EIO; } } + if (skip_cow) { + return 0; + } + /* Get the L2 entry of the first cluster */ l2_entry = get_l2_entry(s, l2_slice, l2_index); - type = qcow2_get_cluster_type(bs, l2_entry); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); + sc_index = offset_to_sc_index(s, guest_offset); + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); - if (type == QCOW2_CLUSTER_NORMAL && keep_old) { - cow_start_from = cow_start_to; + if (!keep_old) { + switch (type) { + case QCOW2_SUBCLUSTER_COMPRESSED: + cow_start_from = 0; + break; + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + if (has_subclusters(s)) { + /* Skip all leading zero and unallocated subclusters */ + uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC; + cow_start_from = + MIN(sc_index, ctz32(alloc_bitmap)) << s->subcluster_bits; + } else { + cow_start_from = 0; + } + break; + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + cow_start_from = sc_index << s->subcluster_bits; + break; + default: + g_assert_not_reached(); + } } else { - cow_start_from = 0; + switch (type) { + case QCOW2_SUBCLUSTER_NORMAL: + cow_start_from = cow_start_to; + break; + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_start_from = sc_index << s->subcluster_bits; + break; + default: + g_assert_not_reached(); + } } /* Get the L2 entry of the last cluster */ - l2_entry = get_l2_entry(s, l2_slice, l2_index + nb_clusters - 1); - type = qcow2_get_cluster_type(bs, l2_entry); + l2_index += nb_clusters - 1; + l2_entry = get_l2_entry(s, l2_slice, l2_index); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); + sc_index = offset_to_sc_index(s, guest_offset + bytes - 1); + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); - if (type == QCOW2_CLUSTER_NORMAL && keep_old) { - cow_end_to = cow_end_from; + if (!keep_old) { + switch (type) { + case QCOW2_SUBCLUSTER_COMPRESSED: + cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + break; + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + if (has_subclusters(s)) { + /* Skip all trailing zero and unallocated subclusters */ + uint32_t alloc_bitmap = l2_bitmap & QCOW_L2_BITMAP_ALL_ALLOC; + cow_end_to -= + MIN(s->subclusters_per_cluster - sc_index - 1, + clz32(alloc_bitmap)) << s->subcluster_bits; + } + break; + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size); + break; + default: + g_assert_not_reached(); + } } else { - cow_end_to = ROUND_UP(cow_end_from, s->cluster_size); + switch (type) { + case QCOW2_SUBCLUSTER_NORMAL: + cow_end_to = cow_end_from; + break; + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + cow_end_to = ROUND_UP(cow_end_from, s->subcluster_size); + break; + default: + g_assert_not_reached(); + } } *m = g_malloc0(sizeof(**m)); @@ -1184,6 +1275,8 @@ static void calculate_l2_meta(BlockDriverState *bs, qemu_co_queue_init(&(*m)->dependent_requests); QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight); + + return 0; } /* @@ -1272,8 +1365,8 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, uint64_t start = guest_offset; uint64_t end = start + bytes; - uint64_t old_start = l2meta_cow_start(old_alloc); - uint64_t old_end = l2meta_cow_end(old_alloc); + uint64_t old_start = start_of_cluster(s, l2meta_cow_start(old_alloc)); + uint64_t old_end = ROUND_UP(l2meta_cow_end(old_alloc), s->cluster_size); if (end <= old_start || start >= old_end) { /* No intersection */ @@ -1398,8 +1491,11 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, - offset_into_cluster(s, guest_offset)); assert(*bytes != 0); - calculate_l2_meta(bs, cluster_offset, guest_offset, - *bytes, l2_slice, m, true); + ret = calculate_l2_meta(bs, cluster_offset, guest_offset, + *bytes, l2_slice, m, true); + if (ret < 0) { + goto out; + } ret = 1; } else { @@ -1575,8 +1671,11 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, *bytes = MIN(*bytes, nb_bytes - offset_into_cluster(s, guest_offset)); assert(*bytes != 0); - calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, l2_slice, - m, false); + ret = calculate_l2_meta(bs, alloc_cluster_offset, guest_offset, *bytes, + l2_slice, m, false); + if (ret < 0) { + goto out; + } ret = 1; From patchwork Mon May 25 18:08:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 825D3C433E0 for ; Mon, 25 May 2020 18:24:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4655020776 for ; Mon, 25 May 2020 18:24:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="PL8Pn8G6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4655020776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43334 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHm7-0003VF-F8 for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:24:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45248) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXw-00046a-8q; Mon, 25 May 2020 14:09:56 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50867) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0007pC-6i; Mon, 25 May 2020 14:09:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=r9h4dLQSBF3SFB6+9xcKHSJ+KZ4mJEr2CS21yaFblV0=; b=PL8Pn8G6tPQp+CyU10ypK9ZVDQBPnfjaSV/TlkInOgKggdGOJDREeXDs1zMRwidOhPUSmdMfN5WloZpMmi1W3MIKLs8x7BUa0GFi/Ptk7vt1AOhJv3Ib5EwG1iyp+w1Q8NT8G2bHs/v7poIGUyD2nRPUv5UsRDMa9cM9aHNUo9KLcnm6wGYrAiZ+HEdrJn3jnTIyZUiNrRtQU/tHVe99Hjv5XvfIumMMwR51qe0HiCjpxv3W3rYGyOaMlwfAWS1XL1i3NmWl5e0ER+zlEBxFGfO1/ct9q0DpaSlXG8/eOhgoWDQicA1AMMjYeWXGwYkaJULwuypjVFglfa9VRWrSGA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXO-00088U-MM; Mon, 25 May 2020 20:09:22 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002NM-52; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 21/32] qcow2: Add subcluster support to qcow2_get_host_offset() Date: Mon, 25 May 2020 20:08:46 +0200 Message-Id: <46add38625edc8ffec0056018d2cc512efda9a55.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The logic of this function remains pretty much the same, except that it uses count_contiguous_subclusters(), which combines the logic of count_contiguous_clusters() / count_contiguous_clusters_unallocated() and checks individual subclusters. qcow2_cluster_to_subcluster_type() is not necessary as a separate function anymore so it's inlined into its caller. Signed-off-by: Alberto Garcia --- block/qcow2.h | 38 ++++------- block/qcow2-cluster.c | 150 ++++++++++++++++++++++-------------------- 2 files changed, 92 insertions(+), 96 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 84c06d9bba..32c68ead9a 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -710,29 +710,6 @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, } } -/* - * For an image without extended L2 entries, return the - * QCow2SubclusterType equivalent of a given QCow2ClusterType. - */ -static inline -QCow2SubclusterType qcow2_cluster_to_subcluster_type(QCow2ClusterType type) -{ - switch (type) { - case QCOW2_CLUSTER_COMPRESSED: - return QCOW2_SUBCLUSTER_COMPRESSED; - case QCOW2_CLUSTER_ZERO_PLAIN: - return QCOW2_SUBCLUSTER_ZERO_PLAIN; - case QCOW2_CLUSTER_ZERO_ALLOC: - return QCOW2_SUBCLUSTER_ZERO_ALLOC; - case QCOW2_CLUSTER_NORMAL: - return QCOW2_SUBCLUSTER_NORMAL; - case QCOW2_CLUSTER_UNALLOCATED: - return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; - default: - g_assert_not_reached(); - } -} - /* * In an image without subsclusters @l2_bitmap is ignored and * @sc_index must be 0. @@ -776,7 +753,20 @@ QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs, g_assert_not_reached(); } } else { - return qcow2_cluster_to_subcluster_type(type); + switch (type) { + case QCOW2_CLUSTER_COMPRESSED: + return QCOW2_SUBCLUSTER_COMPRESSED; + case QCOW2_CLUSTER_ZERO_PLAIN: + return QCOW2_SUBCLUSTER_ZERO_PLAIN; + case QCOW2_CLUSTER_ZERO_ALLOC: + return QCOW2_SUBCLUSTER_ZERO_ALLOC; + case QCOW2_CLUSTER_NORMAL: + return QCOW2_SUBCLUSTER_NORMAL; + case QCOW2_CLUSTER_UNALLOCATED: + return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; + default: + g_assert_not_reached(); + } } } diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 59dd9bda29..2f3bd3a882 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -426,66 +426,66 @@ static int qcow2_get_subcluster_range_type(BlockDriverState *bs, } /* - * Checks how many clusters in a given L2 slice are contiguous in the image - * file. As soon as one of the flags in the bitmask stop_flags changes compared - * to the first cluster, the search is stopped and the cluster is not counted - * as contiguous. (This allows it, for example, to stop at the first compressed - * cluster which may require a different handling) + * Return the number of contiguous subclusters of the exact same type + * in a given L2 slice, starting from cluster @l2_index, subcluster + * @sc_index. Allocated subclusters are required to be contiguous in + * the image file. + * At most @nb_clusters are checked (note that this means clusters, + * not subclusters). + * Compressed clusters are always processed one by one but for the + * purpose of this count they are treated as if they were divided into + * subclusters of size s->subcluster_size. + * On failure return -errno and update @l2_index to point to the + * invalid entry. */ -static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, - int cluster_size, uint64_t *l2_slice, int l2_index, uint64_t stop_flags) +static int count_contiguous_subclusters(BlockDriverState *bs, int nb_clusters, + unsigned sc_index, uint64_t *l2_slice, + unsigned *l2_index) { BDRVQcow2State *s = bs->opaque; - int i; - QCow2ClusterType first_cluster_type; - uint64_t mask = stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED; - uint64_t first_entry = get_l2_entry(s, l2_slice, l2_index); - uint64_t offset = first_entry & mask; + int i, count = 0; + bool check_offset; + uint64_t expected_offset; + QCow2SubclusterType expected_type, type; - first_cluster_type = qcow2_get_cluster_type(bs, first_entry); - if (first_cluster_type == QCOW2_CLUSTER_UNALLOCATED) { - return 0; - } - - /* must be allocated */ - assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || - first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); + assert(*l2_index + nb_clusters <= s->l2_size); for (i = 0; i < nb_clusters; i++) { - uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index + i) & mask; - if (offset + (uint64_t) i * cluster_size != l2_entry) { + unsigned first_sc = (i == 0) ? sc_index : 0; + uint64_t l2_entry = get_l2_entry(s, l2_slice, *l2_index + i); + uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, *l2_index + i); + int ret = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap, + first_sc, &type); + if (ret < 0) { + *l2_index += i; /* Point to the invalid entry */ + return -EIO; + } + if (i == 0) { + if (type == QCOW2_SUBCLUSTER_COMPRESSED) { + /* Compressed clusters are always processed one by one */ + return ret; + } + expected_type = type; + expected_offset = l2_entry & L2E_OFFSET_MASK; + check_offset = (type == QCOW2_SUBCLUSTER_NORMAL || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC); + } else if (type != expected_type) { break; + } else if (check_offset) { + expected_offset += s->cluster_size; + if (expected_offset != (l2_entry & L2E_OFFSET_MASK)) { + break; + } } - } - - return i; -} - -/* - * Checks how many consecutive unallocated clusters in a given L2 - * slice have the same cluster type. - */ -static int count_contiguous_clusters_unallocated(BlockDriverState *bs, - int nb_clusters, - uint64_t *l2_slice, - int l2_index, - QCow2ClusterType wanted_type) -{ - BDRVQcow2State *s = bs->opaque; - int i; - - assert(wanted_type == QCOW2_CLUSTER_ZERO_PLAIN || - wanted_type == QCOW2_CLUSTER_UNALLOCATED); - for (i = 0; i < nb_clusters; i++) { - uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i); - QCow2ClusterType type = qcow2_get_cluster_type(bs, entry); - - if (type != wanted_type) { + count += ret; + /* Stop if there are type changes before the end of the cluster */ + if (first_sc + ret < s->subclusters_per_cluster) { break; } } - return i; + return count; } static int coroutine_fn do_perform_cow_read(BlockDriverState *bs, @@ -574,12 +574,12 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, QCow2SubclusterType *subcluster_type) { BDRVQcow2State *s = bs->opaque; - unsigned int l2_index; - uint64_t l1_index, l2_offset, *l2_slice, l2_entry; - int c; + unsigned int l2_index, sc_index; + uint64_t l1_index, l2_offset, *l2_slice, l2_entry, l2_bitmap; + int sc; unsigned int offset_in_cluster; uint64_t bytes_available, bytes_needed, nb_clusters; - QCow2ClusterType type; + QCow2SubclusterType type; int ret; offset_in_cluster = offset_into_cluster(s, offset); @@ -600,13 +600,13 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, l1_index = offset_to_l1_index(s, offset); if (l1_index >= s->l1_size) { - type = QCOW2_CLUSTER_UNALLOCATED; + type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; goto out; } l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK; if (!l2_offset) { - type = QCOW2_CLUSTER_UNALLOCATED; + type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; goto out; } @@ -627,7 +627,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, /* find the cluster offset for the given disk offset */ l2_index = offset_to_l2_slice_index(s, offset); + sc_index = offset_to_sc_index(s, offset); l2_entry = get_l2_entry(s, l2_slice, l2_index); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); nb_clusters = size_to_clusters(s, bytes_needed); /* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned @@ -635,9 +637,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, * true */ assert(nb_clusters <= INT_MAX); - type = qcow2_get_cluster_type(bs, l2_entry); - if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN || - type == QCOW2_CLUSTER_ZERO_ALLOC)) { + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); + if (s->qcow_version < 3 && (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found" " in pre-v3 image (L2 offset: %#" PRIx64 ", L2 index: %#x)", l2_offset, l2_index); @@ -645,7 +647,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, goto fail; } switch (type) { - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_INVALID: + break; /* This is handled by count_contiguous_subclusters() below */ + case QCOW2_SUBCLUSTER_COMPRESSED: if (has_data_file(bs)) { qcow2_signal_corruption(bs, true, -1, -1, "Compressed cluster " "entry found in image with external data " @@ -654,24 +658,17 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, ret = -EIO; goto fail; } - /* Compressed clusters can only be processed one by one */ - c = 1; *host_offset = l2_entry & L2E_COMPRESSED_OFFSET_SIZE_MASK; break; - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_UNALLOCATED: - /* how many empty clusters ? */ - c = count_contiguous_clusters_unallocated(bs, nb_clusters, - l2_slice, l2_index, type); + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: *host_offset = 0; break; - case QCOW2_CLUSTER_ZERO_ALLOC: - case QCOW2_CLUSTER_NORMAL: { + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: { uint64_t host_cluster_offset = l2_entry & L2E_OFFSET_MASK; *host_offset = host_cluster_offset + offset_in_cluster; - /* how many allocated clusters ? */ - c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, - l2_slice, l2_index, QCOW_OFLAG_ZERO); if (offset_into_cluster(s, host_cluster_offset)) { qcow2_signal_corruption(bs, true, -1, -1, "Cluster allocation offset %#" @@ -697,9 +694,18 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, abort(); } + sc = count_contiguous_subclusters(bs, nb_clusters, sc_index, + l2_slice, &l2_index); + if (sc < 0) { + qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster entry found " + " (L2 offset: %#" PRIx64 ", L2 index: %#x)", + l2_offset, l2_index); + ret = -EIO; + goto fail; + } qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - bytes_available = (int64_t)c * s->cluster_size; + bytes_available = ((int64_t)sc + sc_index) << s->subcluster_bits; out: if (bytes_available > bytes_needed) { @@ -712,7 +718,7 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - *subcluster_type = qcow2_cluster_to_subcluster_type(type); + *subcluster_type = type; return 0; From patchwork Mon May 25 18:08:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D74C433E1 for ; Mon, 25 May 2020 18:10:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C8DC20723 for ; Mon, 25 May 2020 18:10:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="C2J/cHVZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C8DC20723 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHYv-0005Rh-6k for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:10:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45080) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXZ-0003B7-Hb; Mon, 25 May 2020 14:09:33 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51002) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXV-0007pn-IE; Mon, 25 May 2020 14:09:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=//w0RWG0HTTNwsZSH8Y3F78WOUGFQck1pgrxaNq9wA0=; b=C2J/cHVZZRjyRZwevccNxBcOHUIQwn3x/ag7OHRugaFT0BK/AWPH7BPyISCY7z5PthbItHhtELUqzZsGKFO9kpoVguskXvtYnxXSrj+I/FQ/J1mvPgqQ6yB03Sq9FxrC7C17mrfn8FHBt6bpeofVmYTMt9CBai3/ffBZNfa/9h3Ui6Rrg+QnWkTf9JelCfS9oQUQkR6vQyRvqZ4dST1AyM+6qDppq6Fhe9muyXyuq5bDdtiXaKg2HfbR4WtUSkb2A7XTAMJPyDTnXgrhg5PM7mvcPv5fBex6q/EQqmqALD4eXhYrEznXXUiYaRjKjrdjtCkV56rwDTA/4uTDy1CJsg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088V-Go; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002NP-5u; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 22/32] qcow2: Add subcluster support to zero_in_l2_slice() Date: Mon, 25 May 2020 20:08:47 +0200 Message-Id: <850d2649b9dab20bcfe4a216509666b82444ad26.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The QCOW_OFLAG_ZERO bit that indicates that a cluster reads as zeroes is only used in standard L2 entries. Extended L2 entries use individual 'all zeroes' bits for each subcluster. This must be taken into account when updating the L2 entry and also when deciding that an existing entry does not need to be updated. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 2f3bd3a882..4e59bbd545 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1956,7 +1956,6 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, int l2_index; int ret; int i; - bool unmap = !!(flags & BDRV_REQ_MAY_UNMAP); ret = get_cluster_table(bs, offset, &l2_slice, &l2_index); if (ret < 0) { @@ -1968,28 +1967,31 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset, assert(nb_clusters <= INT_MAX); for (i = 0; i < nb_clusters; i++) { - uint64_t old_offset; - QCow2ClusterType cluster_type; + uint64_t old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + uint64_t old_l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + QCow2ClusterType type = qcow2_get_cluster_type(bs, old_l2_entry); + bool unmap = (type == QCOW2_CLUSTER_COMPRESSED) || + ((flags & BDRV_REQ_MAY_UNMAP) && qcow2_cluster_is_allocated(type)); + uint64_t new_l2_entry = unmap ? 0 : old_l2_entry; + uint64_t new_l2_bitmap = old_l2_bitmap; - old_offset = get_l2_entry(s, l2_slice, l2_index + i); + if (has_subclusters(s)) { + new_l2_bitmap = QCOW_L2_BITMAP_ALL_ZEROES; + } else { + new_l2_entry |= QCOW_OFLAG_ZERO; + } - /* - * Minimize L2 changes if the cluster already reads back as - * zeroes with correct allocation. - */ - cluster_type = qcow2_get_cluster_type(bs, old_offset); - if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN || - (cluster_type == QCOW2_CLUSTER_ZERO_ALLOC && !unmap)) { + if (old_l2_entry == new_l2_entry && old_l2_bitmap == new_l2_bitmap) { continue; } qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); - if (cluster_type == QCOW2_CLUSTER_COMPRESSED || unmap) { - set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); - qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUEST); - } else { - uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i); - set_l2_entry(s, l2_slice, l2_index + i, entry | QCOW_OFLAG_ZERO); + if (unmap) { + qcow2_free_any_clusters(bs, old_l2_entry, 1, QCOW2_DISCARD_REQUEST); + } + set_l2_entry(s, l2_slice, l2_index + i, new_l2_entry); + if (has_subclusters(s)) { + set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap); } } From patchwork Mon May 25 18:08:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABDE3C433E0 for ; Mon, 25 May 2020 18:13:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7C9442071C for ; Mon, 25 May 2020 18:13:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="nQ9WQkTp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C9442071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHay-0000oY-JY for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:13:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45072) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXY-00039W-22; Mon, 25 May 2020 14:09:32 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51000) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXV-0007pk-EX; Mon, 25 May 2020 14:09:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=uAoLov4Hiwn4AXZ4pavGoUMOgtIf8/FsuCkPRLotPUc=; b=nQ9WQkTpDQ3cdHVeWvA9oTLr3wfg+RRnMzwRZSecP0aWkqB0455UTy6i3/yzZSsQtAWmEOydAv5OpciF+AJ1+5UiOkEuCzkzRfwF0dUdJ6y/W75U43UVJbYCRMwnOneUCvL/7TeEUGJuYatiL/Qm+FomXoFRmhdY7+KQsQd9Zu5zy9W508RsBVg0cMggS32atv8/WOl+VuoY3Xoh1XQ30SxYwJahq+TCw3FwCzpG4yF20mtHpIWNrqQjBcZfg9R/FfDSdDujmstDXwevaQLeqqn6rQaNZzgxIUwytL2SBXmd92LB8o8xAAESpPhXq6IqgXBMKtLISqLnIGkKXCVctA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088W-FW; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002NR-74; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 23/32] qcow2: Add subcluster support to discard_in_l2_slice() Date: Mon, 25 May 2020 20:08:48 +0200 Message-Id: <9df951fae42d4ee19104913315835c033b28e14f.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Two things need to be taken into account here: 1) With full_discard == true the L2 entry must be cleared completely. This also includes the L2 bitmap if the image has extended L2 entries. 2) With full_discard == false we have to make the discarded cluster read back as zeroes. With normal L2 entries this is done with the QCOW_OFLAG_ZERO bit, whereas with extended L2 entries this is done with the individual 'all zeroes' bits for each subcluster. Note however that QCOW_OFLAG_ZERO is not supported in v2 qcow2 images so, if there is a backing file, discard cannot guarantee that the image will read back as zeroes. If this is important for the caller it should forbid it as qcow2_co_pdiscard() does (see 80f5c01183 for more details). Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 52 +++++++++++++++++++------------------------ 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 4e59bbd545..edfc8ea91c 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1847,11 +1847,17 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, assert(nb_clusters <= INT_MAX); for (i = 0; i < nb_clusters; i++) { - uint64_t old_l2_entry; - - old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + uint64_t old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + uint64_t old_l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + uint64_t new_l2_entry = old_l2_entry; + uint64_t new_l2_bitmap = old_l2_bitmap; + QCow2ClusterType cluster_type = + qcow2_get_cluster_type(bs, old_l2_entry); /* + * If full_discard is true, the cluster should not read back as zeroes, + * but rather fall through to the backing file. + * * If full_discard is false, make sure that a discarded area reads back * as zeroes for v3 images (we cannot do it for v2 without actually * writing a zero-filled buffer). We can skip the operation if the @@ -1860,40 +1866,28 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, * * TODO We might want to use bdrv_block_status(bs) here, but we're * holding s->lock, so that doesn't work today. - * - * If full_discard is true, the sector should not read back as zeroes, - * but rather fall through to the backing file. */ - switch (qcow2_get_cluster_type(bs, old_l2_entry)) { - case QCOW2_CLUSTER_UNALLOCATED: - if (full_discard || !bs->backing) { - continue; + if (full_discard) { + new_l2_entry = new_l2_bitmap = 0; + } else if (bs->backing || qcow2_cluster_is_allocated(cluster_type)) { + if (has_subclusters(s)) { + new_l2_entry = 0; + new_l2_bitmap = QCOW_L2_BITMAP_ALL_ZEROES; + } else { + new_l2_entry = s->qcow_version >= 3 ? QCOW_OFLAG_ZERO : 0; } - break; + } - case QCOW2_CLUSTER_ZERO_PLAIN: - if (!full_discard) { - continue; - } - break; - - case QCOW2_CLUSTER_ZERO_ALLOC: - case QCOW2_CLUSTER_NORMAL: - case QCOW2_CLUSTER_COMPRESSED: - break; - - default: - abort(); + if (old_l2_entry == new_l2_entry && old_l2_bitmap == new_l2_bitmap) { + continue; } /* First remove L2 entries */ qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); - if (!full_discard && s->qcow_version >= 3) { - set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); - } else { - set_l2_entry(s, l2_slice, l2_index + i, 0); + set_l2_entry(s, l2_slice, l2_index + i, new_l2_entry); + if (has_subclusters(s)) { + set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap); } - /* Then decrease the refcount */ qcow2_free_any_clusters(bs, old_l2_entry, 1, type); } From patchwork Mon May 25 18:08:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281926 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11BB0C433E1 for ; Mon, 25 May 2020 18:22:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DAB6D20776 for ; Mon, 25 May 2020 18:22:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="DUF/FPNl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DAB6D20776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34976 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHkR-00005o-0t for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:22:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45246) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXv-000453-OE; Mon, 25 May 2020 14:09:55 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0007pv-6t; Mon, 25 May 2020 14:09:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=jdA6WaY7hKoW016heKIZFDBwkSoL0B2SxxYO4r5/VA8=; b=DUF/FPNlx6eBU/3dijc5Np5n7wOOOxVftxKaaDqrhhRJr7NnVIwbUVnkN3vEQICQKgjc34NB7UN0AY8bjS50q01PHKiKlaKawMyzO9deWgq8zwaorCVUmTTzdBJNamQYbN6FI3CE6Jt/qMn2o6le3991JBRLJBSZ9GGamSQEojhY+S2Qvfz/8Uk/mudwgQxYiWCE5zVyqQThEQwE6gSjgexsBMwDy9jO3YoilqK0h3eQDgVm+gq33k0vzGggbR4WBOPu05yClk7bY917u+LMnw7vp0G25Xy9uhAZM0YBW49MyQxFDE2w4wSc/tI5WwNeOONpjb3Z1EB/ywWZlLQXDw==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088Z-J9; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002Nb-AM; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 26/32] qcow2: Clear the L2 bitmap when allocating a compressed cluster Date: Mon, 25 May 2020 20:08:51 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Compressed clusters always have the bitmap part of the extended L2 entry set to 0. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 2276cee6d6..deff838fe8 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -861,6 +861,9 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, BLKDBG_EVENT(bs->file, BLKDBG_L2_UPDATE_COMPRESSED); qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); set_l2_entry(s, l2_slice, l2_index, cluster_offset); + if (has_subclusters(s)) { + set_l2_bitmap(s, l2_slice, l2_index, 0); + } qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); *host_offset = cluster_offset & s->cluster_offset_mask; From patchwork Mon May 25 18:08:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F93C433E1 for ; Mon, 25 May 2020 18:22:48 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1DE94207FB for ; Mon, 25 May 2020 18:22:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="M/poncsa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1DE94207FB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHkN-0008NM-8S for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:22:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45224) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXr-0003va-TO; Mon, 25 May 2020 14:09:51 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007qC-UM; Mon, 25 May 2020 14:09:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=kaPNNW3QBgq2po7mlaGmxNCYnAPeNNVUXPsq1YJk0BM=; b=M/poncsataf8Co4IGxwpDYabgb45x0ddR57nG0N84D6ehQYfbgjTj40ugH3Joq9jDT95oD5hBjKezcMyrXe3OifAKga543JacSBm60GKrN2UtioTYo1LHhNy3s18u9v8T6U/+uy3HCyJRd5ZhPuPg7pafO6wEI4qywco/XzLI/5bACOnsr29wogjxXpCiTpoGucgmPVdwVsYJufqq+yb4dTW+ITqKmt0shPZkQdujoj/hstN3BRpEdw2WuHMcv5cgT67Xing9alAzMYVUfgs7SkuZ/lzCGSGYzoQEKEE6eFXkgDaYpc/Fis3mTmnkZwCg7QFE5ffr5BjTgTxChNJdg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088a-Ii; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002Ne-BM; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 27/32] qcow2: Add subcluster support to handle_alloc_space() Date: Mon, 25 May 2020 20:08:52 +0200 Message-Id: <14b60dde8fe7863ae580e3f9d88e5755db28a108.1590429901.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The bdrv_co_pwrite_zeroes() call here fills complete clusters with zeroes, but it can happen that some subclusters are not part of the write request or the copy-on-write. This patch makes sure that only the affected subclusters are overwritten. A potential improvement would be to also fill with zeroes the other subclusters if we can guarantee that we are not overwriting existing data. However this would waste more disk space, so we should first evaluate if it's really worth doing. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 8126ad8fa0..430b4e423a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2411,6 +2411,9 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) for (m = l2meta; m != NULL; m = m->next) { int ret; + uint64_t start_offset = m->alloc_offset + m->cow_start.offset; + unsigned nb_bytes = m->cow_end.offset + m->cow_end.nb_bytes - + m->cow_start.offset; if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { continue; @@ -2425,16 +2428,14 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) * efficiently zero out the whole clusters */ - ret = qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = qcow2_pre_write_overlap_check(bs, 0, start_offset, nb_bytes, true); if (ret < 0) { return ret; } BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE); - ret = bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = bdrv_co_pwrite_zeroes(s->data_file, start_offset, nb_bytes, BDRV_REQ_NO_FALLBACK); if (ret < 0) { if (ret != -ENOTSUP && ret != -EAGAIN) { From patchwork Mon May 25 18:08:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DF44C433E0 for ; Mon, 25 May 2020 18:29:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4A4220776 for ; Mon, 25 May 2020 18:29:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="oh1kLmqY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4A4220776 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33756 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHqk-0003E1-VX for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:29:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45258) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXy-0004C7-Ca; Mon, 25 May 2020 14:09:58 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007q8-P3; Mon, 25 May 2020 14:09:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=9FdIsY7lruAKnCja4lDwDdi6miwzZyfVeRcpgYnkABQ=; b=oh1kLmqYsmVZUL6r/NFlEN3EeW2ox0VT/B7HzNyQ+TEu3ViRXtDHODLcVKpbbnUsDvAqBjPebI4hcV2lJ8kKrVZn+vazLxoomas6W4Q1Zqg9oafYKeAkoiKhs2GFYN/N28poEtVF1m5nJmRFclinFT3V8F0pengE4YaGiQmwS0RogRTq9QgiIJ/mWMwsBRgfu1nVCAYgL8+Uexfu75pPReCaDuo/EV1vZ6SPlNjT5wptuNtnH35QW5MZc2naAdbwTJ6GnFVRjoJ5aySxp268euF0CfMA2pk8VhheANQUX7/okIVgFuYjE7zEQ43FhuUE4QK+s8grH9kiTFprlxjwYg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088j-HW; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002Nm-EZ; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 30/32] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit Date: Mon, 25 May 2020 20:08:55 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia --- qapi/block-core.json | 7 +++ block/qcow2.h | 8 ++- include/block/block_int.h | 1 + block/qcow2.c | 74 ++++++++++++++++++++-- tests/qemu-iotests/031.out | 8 +-- tests/qemu-iotests/036.out | 4 +- tests/qemu-iotests/049.out | 102 +++++++++++++++---------------- tests/qemu-iotests/060.out | 1 + tests/qemu-iotests/061.out | 20 +++--- tests/qemu-iotests/065 | 12 ++-- tests/qemu-iotests/082.out | 48 ++++++++++++--- tests/qemu-iotests/085.out | 38 ++++++------ tests/qemu-iotests/144.out | 4 +- tests/qemu-iotests/182.out | 2 +- tests/qemu-iotests/185.out | 8 +-- tests/qemu-iotests/198.out | 2 + tests/qemu-iotests/206.out | 4 ++ tests/qemu-iotests/242.out | 5 ++ tests/qemu-iotests/255.out | 8 +-- tests/qemu-iotests/274.out | 49 ++++++++------- tests/qemu-iotests/280.out | 2 +- tests/qemu-iotests/common.filter | 1 + 22 files changed, 268 insertions(+), 140 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index 6fbacddab2..0cb80bda9b 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -66,6 +66,9 @@ # standalone (read-only) raw image without looking at qcow2 # metadata (since: 4.0) # +# @extended-l2: true if the image has extended L2 entries; only valid for +# compat >= 1.1 (since 5.1) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for @@ -87,6 +90,7 @@ 'compat': 'str', '*data-file': 'str', '*data-file-raw': 'bool', + '*extended-l2': 'bool', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int', @@ -4312,6 +4316,8 @@ # @data-file-raw: True if the external data file must stay valid as a # standalone (read-only) raw image without looking at qcow2 # metadata (default: false; since: 4.0) +# @extended-l2 True to make the image have extended L2 entries +# (default: false; since 5.1) # @size: Size of the virtual disk in bytes # @version: Compatibility level (default: v3) # @backing-file: File name of the backing file if a backing file @@ -4332,6 +4338,7 @@ 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', '*data-file-raw': 'bool', + '*extended-l2': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str', diff --git a/block/qcow2.h b/block/qcow2.h index ece5f1cb5a..0f26e39d12 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -246,15 +246,18 @@ enum { QCOW2_INCOMPAT_CORRUPT_BITNR = 1, QCOW2_INCOMPAT_DATA_FILE_BITNR = 2, QCOW2_INCOMPAT_COMPRESSION_BITNR = 3, + QCOW2_INCOMPAT_EXTL2_BITNR = 4, QCOW2_INCOMPAT_DIRTY = 1 << QCOW2_INCOMPAT_DIRTY_BITNR, QCOW2_INCOMPAT_CORRUPT = 1 << QCOW2_INCOMPAT_CORRUPT_BITNR, QCOW2_INCOMPAT_DATA_FILE = 1 << QCOW2_INCOMPAT_DATA_FILE_BITNR, QCOW2_INCOMPAT_COMPRESSION = 1 << QCOW2_INCOMPAT_COMPRESSION_BITNR, + QCOW2_INCOMPAT_EXTL2 = 1 << QCOW2_INCOMPAT_EXTL2_BITNR, QCOW2_INCOMPAT_MASK = QCOW2_INCOMPAT_DIRTY | QCOW2_INCOMPAT_CORRUPT | QCOW2_INCOMPAT_DATA_FILE - | QCOW2_INCOMPAT_COMPRESSION, + | QCOW2_INCOMPAT_COMPRESSION + | QCOW2_INCOMPAT_EXTL2, }; /* Compatible feature bits */ @@ -573,8 +576,7 @@ typedef enum QCow2MetadataOverlap { static inline bool has_subclusters(BDRVQcow2State *s) { - /* FIXME: Return false until this feature is complete */ - return false; + return s->incompatible_features & QCOW2_INCOMPAT_EXTL2; } static inline size_t l2_entry_size(BDRVQcow2State *s) diff --git a/include/block/block_int.h b/include/block/block_int.h index 791de6a59c..36e1993788 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -58,6 +58,7 @@ #define BLOCK_OPT_DATA_FILE "data_file" #define BLOCK_OPT_DATA_FILE_RAW "data_file_raw" #define BLOCK_OPT_COMPRESSION_TYPE "compression_type" +#define BLOCK_OPT_EXTL2 "extended_l2" #define BLOCK_PROBE_BUF_SIZE 512 diff --git a/block/qcow2.c b/block/qcow2.c index bc7a4a1729..6690b39510 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1438,6 +1438,12 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->subcluster_size = s->cluster_size / s->subclusters_per_cluster; s->subcluster_bits = ctz32(s->subcluster_size); + if (s->subcluster_size < (1 << MIN_CLUSTER_BITS)) { + error_setg(errp, "Unsupported subcluster size: %d", s->subcluster_size); + ret = -EINVAL; + goto fail; + } + /* Check support for various header values */ if (header.refcount_order > 6) { error_setg(errp, "Reference count entry width too large; may not " @@ -2923,6 +2929,11 @@ int qcow2_update_header(BlockDriverState *bs) .bit = QCOW2_INCOMPAT_COMPRESSION_BITNR, .name = "compression type", }, + { + .type = QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit = QCOW2_INCOMPAT_EXTL2_BITNR, + .name = "extended L2 entries", + }, { .type = QCOW2_FEAT_TYPE_COMPATIBLE, .bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR, @@ -3269,7 +3280,8 @@ static int64_t qcow2_calc_prealloc_size(int64_t total_size, return meta_size + aligned_total_size; } -static bool validate_cluster_size(size_t cluster_size, Error **errp) +static bool validate_cluster_size(size_t cluster_size, bool extended_l2, + Error **errp) { int cluster_bits = ctz32(cluster_size); if (cluster_bits < MIN_CLUSTER_BITS || cluster_bits > MAX_CLUSTER_BITS || @@ -3279,16 +3291,28 @@ static bool validate_cluster_size(size_t cluster_size, Error **errp) "%dk", 1 << MIN_CLUSTER_BITS, 1 << (MAX_CLUSTER_BITS - 10)); return false; } + + if (extended_l2) { + unsigned min_cluster_size = + (1 << MIN_CLUSTER_BITS) * QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER; + if (cluster_size < min_cluster_size) { + error_setg(errp, "Extended L2 entries are only supported with " + "cluster sizes of at least %u bytes", min_cluster_size); + return false; + } + } + return true; } -static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, Error **errp) +static size_t qcow2_opt_get_cluster_size_del(QemuOpts *opts, bool extended_l2, + Error **errp) { size_t cluster_size; cluster_size = qemu_opt_get_size_del(opts, BLOCK_OPT_CLUSTER_SIZE, DEFAULT_CLUSTER_SIZE); - if (!validate_cluster_size(cluster_size, errp)) { + if (!validate_cluster_size(cluster_size, extended_l2, errp)) { return 0; } return cluster_size; @@ -3403,7 +3427,20 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) cluster_size = DEFAULT_CLUSTER_SIZE; } - if (!validate_cluster_size(cluster_size, errp)) { + if (!qcow2_opts->has_extended_l2) { + qcow2_opts->extended_l2 = false; + } + if (qcow2_opts->extended_l2) { + if (version < 3) { + error_setg(errp, "Extended L2 entries are only supported with " + "compatibility level 1.1 and above (use version=v3 or " + "greater)"); + ret = -EINVAL; + goto out; + } + } + + if (!validate_cluster_size(cluster_size, qcow2_opts->extended_l2, errp)) { ret = -EINVAL; goto out; } @@ -3554,6 +3591,11 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) cpu_to_be64(QCOW2_INCOMPAT_COMPRESSION); } + if (qcow2_opts->extended_l2) { + header->incompatible_features |= + cpu_to_be64(QCOW2_INCOMPAT_EXTL2); + } + ret = blk_pwrite(blk, 0, header, cluster_size, 0); g_free(header); if (ret < 0) { @@ -3734,6 +3776,7 @@ static int coroutine_fn qcow2_co_create_opts(BlockDriver *drv, { BLOCK_OPT_BACKING_FMT, "backing-fmt" }, { BLOCK_OPT_CLUSTER_SIZE, "cluster-size" }, { BLOCK_OPT_LAZY_REFCOUNTS, "lazy-refcounts" }, + { BLOCK_OPT_EXTL2, "extended-l2" }, { BLOCK_OPT_REFCOUNT_BITS, "refcount-bits" }, { BLOCK_OPT_ENCRYPT, BLOCK_OPT_ENCRYPT_FORMAT }, { BLOCK_OPT_COMPAT_LEVEL, "version" }, @@ -4837,11 +4880,14 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, PreallocMode prealloc; bool has_backing_file; bool has_luks; - bool extended_l2 = false; /* Set to false until the option is added */ + bool extended_l2; size_t l2e_size; /* Parse image creation options */ - cluster_size = qcow2_opt_get_cluster_size_del(opts, &local_err); + extended_l2 = qemu_opt_get_bool_del(opts, BLOCK_OPT_EXTL2, false); + + cluster_size = qcow2_opt_get_cluster_size_del(opts, extended_l2, + &local_err); if (local_err) { goto err; } @@ -5037,6 +5083,8 @@ static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs, .corrupt = s->incompatible_features & QCOW2_INCOMPAT_CORRUPT, .has_corrupt = true, + .has_extended_l2 = true, + .extended_l2 = has_subclusters(s), .refcount_bits = s->refcount_bits, .has_bitmaps = !!bitmaps, .bitmaps = bitmaps, @@ -5466,6 +5514,14 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, "is not supported"); return -ENOTSUP; } + } else if (!strcmp(desc->name, BLOCK_OPT_EXTL2)) { + bool extended_l2 = qemu_opt_get_bool(opts, BLOCK_OPT_EXTL2, + has_subclusters(s)); + if (extended_l2 != has_subclusters(s)) { + error_setg(errp, "Toggling extended L2 entries " + "is not supported"); + return -EINVAL; + } } else { /* if this point is reached, this probably means a new option was * added without having it covered here */ @@ -5726,6 +5782,12 @@ static QemuOptsList qcow2_create_opts = { .help = "Postpone refcount updates", .def_value_str = "off" }, + { + .name = BLOCK_OPT_EXTL2, + .type = QEMU_OPT_BOOL, + .help = "Extended L2 tables", + .def_value_str = "off" + }, { .name = BLOCK_OPT_REFCOUNT_BITS, .type = QEMU_OPT_NUMBER, diff --git a/tests/qemu-iotests/031.out b/tests/qemu-iotests/031.out index 5a4beda6a2..cdda111c75 100644 --- a/tests/qemu-iotests/031.out +++ b/tests/qemu-iotests/031.out @@ -117,7 +117,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data Header extension: @@ -150,7 +150,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data Header extension: @@ -164,7 +164,7 @@ No errors were found on the image. magic 0x514649fb version 3 -backing_file_offset 0x210 +backing_file_offset 0x240 backing_file_size 0x17 cluster_bits 16 size 67108864 @@ -188,7 +188,7 @@ data 'host_device' Header extension: magic 0x6803f857 -length 336 +length 384 data Header extension: diff --git a/tests/qemu-iotests/036.out b/tests/qemu-iotests/036.out index e409acf60e..0f6cffc2c3 100644 --- a/tests/qemu-iotests/036.out +++ b/tests/qemu-iotests/036.out @@ -26,7 +26,7 @@ compatible_features [] autoclear_features [63] Header extension: magic 0x6803f857 -length 336 +length 384 data @@ -38,7 +38,7 @@ compatible_features [] autoclear_features [] Header extension: magic 0x6803f857 -length 336 +length 384 data *** done diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out index c54ae21b86..c8356be551 100644 --- a/tests/qemu-iotests/049.out +++ b/tests/qemu-iotests/049.out @@ -4,90 +4,90 @@ QA output created by 049 == 1. Traditional size parameter == qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024.0b -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5k -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5K -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5G -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 TEST_DIR/t.qcow2 1.5T -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == 2. Specifying size via -o == qemu-img create -f qcow2 -o size=1024 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1024b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1048576 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1099511627776 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1024.0 TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1024.0b TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1.5k TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1.5K TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1536 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1.5M TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1572864 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1.5G TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1610612736 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o size=1.5T TEST_DIR/t.qcow2 -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1649267441664 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == 3. Invalid sizes == @@ -129,84 +129,84 @@ qemu-img: TEST_DIR/t.qcow2: The image size must be specified only once == Check correct interpretation of suffixes for cluster size == qemu-img create -f qcow2 -o cluster_size=1024 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1024b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1048576 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1024.0 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=1024.0b TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=1024 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=0.5k TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=0.5K TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=512 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o cluster_size=0.5M TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=524288 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == Check compat level option == qemu-img create -f qcow2 -o compat=0.10 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=1.1 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=0.42 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter '0.42' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.42 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=foobar TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter 'foobar' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=foobar cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == Check preallocation option == qemu-img create -f qcow2 -o preallocation=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=off lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o preallocation=metadata TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=metadata lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o preallocation=1234 TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Invalid parameter '1234' -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 cluster_size=65536 preallocation=1234 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == Check encryption option == qemu-img create -f qcow2 -o encryption=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=off cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 --object secret,id=sec0,data=123456 -o encryption=on,encrypt.key-secret=sec0 TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 encryption=on encrypt.key-secret=sec0 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib == Check lazy_refcounts option (only with v3) == qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=1.1 cluster_size=65536 lazy_refcounts=on extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=off TEST_DIR/t.qcow2 64M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib qemu-img create -f qcow2 -o compat=0.10,lazy_refcounts=on TEST_DIR/t.qcow2 64M qemu-img: TEST_DIR/t.qcow2: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater) -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 compat=0.10 cluster_size=65536 lazy_refcounts=on extended_l2=off refcount_bits=16 compression_type=zlib *** done diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index be5f8707a3..b420a89e6c 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -21,6 +21,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: true + extended l2: false qemu-io: can't open device TEST_DIR/t.IMGFMT: IMGFMT: Image is corrupt; cannot be opened read/write no file open, try 'help open' read 512/512 bytes at offset 0 diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index a51ad1b5ba..39812d8cf8 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -26,7 +26,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data magic 0x514649fb @@ -84,7 +84,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data magic 0x514649fb @@ -140,7 +140,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data ERROR cluster 5 refcount=0 reference=1 @@ -195,7 +195,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data magic 0x514649fb @@ -264,7 +264,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data read 65536/65536 bytes at offset 44040192 @@ -326,7 +326,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data ERROR cluster 5 refcount=0 reference=1 @@ -355,7 +355,7 @@ header_length 112 Header extension: magic 0x6803f857 -length 336 +length 384 data read 131072/131072 bytes at offset 0 @@ -525,6 +525,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. === Try changing the external data file === @@ -546,6 +547,7 @@ Format specific information: data file: foo data file raw: false corrupt: false + extended l2: false qemu-img: Could not open 'TEST_DIR/t.IMGFMT': 'data-file' is required for this image image: TEST_DIR/t.IMGFMT @@ -559,6 +561,7 @@ Format specific information: refcount bits: 16 data file raw: false corrupt: false + extended l2: false === Clearing and setting data-file-raw === @@ -575,6 +578,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: true corrupt: false + extended l2: false No errors were found on the image. image: TEST_DIR/t.IMGFMT file format: IMGFMT @@ -588,6 +592,7 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. qemu-img: data-file-raw cannot be set on existing images image: TEST_DIR/t.IMGFMT @@ -602,5 +607,6 @@ Format specific information: data file: TEST_DIR/t.IMGFMT.data data file raw: false corrupt: false + extended l2: false No errors were found on the image. *** done diff --git a/tests/qemu-iotests/065 b/tests/qemu-iotests/065 index 18dc488c7a..29a7f7ad60 100755 --- a/tests/qemu-iotests/065 +++ b/tests/qemu-iotests/065 @@ -98,20 +98,20 @@ class TestQCow3NotLazy(TestQemuImgInfo): img_options = 'compat=1.1,lazy_refcounts=off' json_compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: false', 'refcount bits: 16', - 'corrupt: false' ] + 'corrupt: false', 'extended l2: false' ] class TestQCow3Lazy(TestQemuImgInfo): '''Testing a qcow2 version 3 image with lazy refcounts enabled''' img_options = 'compat=1.1,lazy_refcounts=on' json_compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } human_compare = [ 'compat: 1.1', 'compression type: zlib', 'lazy refcounts: true', 'refcount bits: 16', - 'corrupt: false' ] + 'corrupt: false', 'extended l2: false' ] class TestQCow3NotLazyQMP(TestQMP): '''Testing a qcow2 version 3 image with lazy refcounts disabled, opening @@ -120,7 +120,7 @@ class TestQCow3NotLazyQMP(TestQMP): qemu_options = 'lazy-refcounts=on' compare = { 'compat': '1.1', 'lazy-refcounts': False, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } class TestQCow3LazyQMP(TestQMP): @@ -130,7 +130,7 @@ class TestQCow3LazyQMP(TestQMP): qemu_options = 'lazy-refcounts=off' compare = { 'compat': '1.1', 'lazy-refcounts': True, 'refcount-bits': 16, 'corrupt': False, - 'compression-type': 'zlib' } + 'compression-type': 'zlib', 'extended-l2': False } TestImageInfoSpecific = None TestQemuImgInfo = None diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index 529a1214e1..efc0328e8d 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -3,14 +3,14 @@ QA output created by 082 === create: Options specified more than once === Testing: create -f foo -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) cluster_size: 65536 Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=4096 lazy_refcounts=on extended_l2=off refcount_bits=16 compression_type=zlib image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -21,9 +21,10 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: create -f qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=on extended_l2=off refcount_bits=16 compression_type=zlib image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -34,9 +35,10 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: create -f qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=8192 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 128 MiB (134217728 bytes) @@ -62,6 +64,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -86,6 +89,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -110,6 +114,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -134,6 +139,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -158,6 +164,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -182,6 +189,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -206,6 +214,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -230,6 +239,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -237,10 +247,10 @@ Supported options: size= - Virtual disk size Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,help cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib Testing: create -f qcow2 -u -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2,,? cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib Testing: create -f qcow2 -o backing_file=TEST_DIR/t.qcow2, -o help TEST_DIR/t.qcow2 128M qemu-img: Invalid option list: backing_file=TEST_DIR/t.qcow2, @@ -269,6 +279,7 @@ Supported qcow2 options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -290,7 +301,7 @@ qemu-img: Format driver 'bochs' does not support image creation === convert: Options specified more than once === Testing: create -f qcow2 TEST_DIR/t.qcow2 128M -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib Testing: convert -f foo -f qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -314,6 +325,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: convert -O qcow2 -o cluster_size=4k -o lazy_refcounts=on -o cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -326,6 +338,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: convert -O qcow2 -o cluster_size=4k,cluster_size=8k TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.base image: TEST_DIR/t.IMGFMT.base @@ -353,6 +366,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -377,6 +391,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -401,6 +416,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -425,6 +441,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -449,6 +466,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -473,6 +491,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -497,6 +516,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -521,6 +541,7 @@ Supported options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates nocow= - Turn off copy-on-write (valid only on btrfs) preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) @@ -560,6 +581,7 @@ Supported qcow2 options: encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -605,6 +627,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=130M -o lazy_refcounts=off TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT @@ -617,6 +640,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=8M -o lazy_refcounts=on -o size=132M TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT @@ -629,6 +653,7 @@ Format specific information: lazy refcounts: true refcount bits: 16 corrupt: false + extended l2: false Testing: amend -f qcow2 -o size=4M,size=148M TEST_DIR/t.qcow2 image: TEST_DIR/t.IMGFMT @@ -656,6 +681,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -681,6 +707,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -706,6 +733,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -731,6 +759,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -756,6 +785,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -781,6 +811,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -806,6 +837,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -831,6 +863,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits @@ -873,6 +906,7 @@ Creation options for 'qcow2': encrypt.ivgen-hash-alg= - Name of IV generator hash algorithm encrypt.key-secret= - ID of secret providing qcow AES key or LUKS passphrase encryption= - Encrypt the image with format 'aes'. (Deprecated in favor of encrypt.format=aes) + extended_l2= - Extended L2 tables lazy_refcounts= - Postpone refcount updates preallocation= - Preallocation mode (allowed values: off, metadata, falloc, full) refcount_bits= - Width of a reference count entry in bits diff --git a/tests/qemu-iotests/085.out b/tests/qemu-iotests/085.out index a822ff4ef6..0431743808 100644 --- a/tests/qemu-iotests/085.out +++ b/tests/qemu-iotests/085.out @@ -13,7 +13,7 @@ Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=134217728 === Create a single snapshot on virtio0 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/1-snapshot-v0.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.1 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} === Invalid command - missing device and nodename === @@ -30,40 +30,40 @@ Formatting 'TEST_DIR/1-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file === Create several transactional group snapshots === { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/2-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/2-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/2-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/1-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/2-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/t.qcow2.2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/3-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/3-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/3-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/3-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/2-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/4-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/4-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/4-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/4-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/3-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/5-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/5-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/5-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/5-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/4-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/6-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/6-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/6-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/6-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/5-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/7-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/7-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/7-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/7-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/6-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/8-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/8-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/8-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/8-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/7-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/9-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/9-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/9-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/9-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/8-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'transaction', 'arguments': {'actions': [ { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio0', 'snapshot-file': 'TEST_DIR/10-snapshot-v0.IMGFMT' } }, { 'type': 'blockdev-snapshot-sync', 'data' : { 'device': 'virtio1', 'snapshot-file': 'TEST_DIR/10-snapshot-v1.IMGFMT' } } ] } } -Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/10-snapshot-v0.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v0.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/10-snapshot-v1.qcow2', fmt=qcow2 size=134217728 backing_file=TEST_DIR/9-snapshot-v1.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} === Create a couple of snapshots using blockdev-snapshot === diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out index 885a8874a5..a2627f0cc5 100644 --- a/tests/qemu-iotests/144.out +++ b/tests/qemu-iotests/144.out @@ -9,7 +9,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=536870912 { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} === Performing block-commit on active layer === @@ -31,6 +31,6 @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/ === Performing Live Snapshot 2 === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'virtio0', 'snapshot-file':'TEST_DIR/tmp2.IMGFMT', 'format': 'IMGFMT' } } -Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/tmp2.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/t.qcow2 backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} *** done diff --git a/tests/qemu-iotests/182.out b/tests/qemu-iotests/182.out index ae43654d32..e4290ec028 100644 --- a/tests/qemu-iotests/182.out +++ b/tests/qemu-iotests/182.out @@ -13,7 +13,7 @@ Is another process using the image [TEST_DIR/t.qcow2]? {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node0', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}} {'execute': 'blockdev-snapshot-sync', 'arguments': { 'node-name': 'node0', 'snapshot-file': 'TEST_DIR/t.IMGFMT.overlay', 'snapshot-node-name': 'node1' } } -Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.overlay', fmt=qcow2 size=197120 backing_file=TEST_DIR/t.qcow2 backing_fmt=file cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} {'execute': 'blockdev-add', 'arguments': { 'node-name': 'node1', 'driver': 'file', 'filename': 'TEST_DIR/t.IMGFMT', 'locking': 'on' } } {"return": {}} diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out index ac5ab16bc8..4e0f018bf2 100644 --- a/tests/qemu-iotests/185.out +++ b/tests/qemu-iotests/185.out @@ -9,14 +9,14 @@ Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 === Creating backing chain === { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT.mid', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.mid', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} { 'execute': 'human-monitor-command', 'arguments': { 'command-line': 'qemu-io disk "write 0 4M"' } } wrote 4194304/4194304 bytes at offset 0 4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) {"return": ""} { 'execute': 'blockdev-snapshot-sync', 'arguments': { 'device': 'disk', 'snapshot-file': 'TEST_DIR/t.IMGFMT', 'format': 'IMGFMT', 'mode': 'absolute-paths' } } -Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.qcow2.mid backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"return": {}} === Start commit job and exit qemu === @@ -48,7 +48,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-mirror', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"return": {}} @@ -62,7 +62,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l { 'execute': 'qmp_capabilities' } {"return": {}} { 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } -Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "disk"}} diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out index 6280ae6eed..821a052bb1 100644 --- a/tests/qemu-iotests/198.out +++ b/tests/qemu-iotests/198.out @@ -73,6 +73,7 @@ Format specific information: key offset: 1810432 payload offset: 2068480 master key iters: 1024 + extended l2: false == checking image layer == image: json:{ /* filtered */ } @@ -117,4 +118,5 @@ Format specific information: key offset: 1810432 payload offset: 2068480 master key iters: 1024 + extended l2: false *** done diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out index 1a14255a83..363c5abe35 100644 --- a/tests/qemu-iotests/206.out +++ b/tests/qemu-iotests/206.out @@ -22,6 +22,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false === Successful image creation (inline blockdev-add, explicit defaults) === @@ -45,6 +46,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false === Successful image creation (v3 non-default options) === @@ -68,6 +70,7 @@ Format specific information: lazy refcounts: true refcount bits: 1 corrupt: false + extended l2: false === Successful image creation (v2 non-default options) === @@ -146,6 +149,7 @@ Format specific information: payload offset: 528384 master key iters: XXX corrupt: false + extended l2: false === Invalid BlockdevRef === diff --git a/tests/qemu-iotests/242.out b/tests/qemu-iotests/242.out index 091b9126ce..3759c99284 100644 --- a/tests/qemu-iotests/242.out +++ b/tests/qemu-iotests/242.out @@ -16,6 +16,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false No bitmap in JSON format output @@ -42,6 +43,7 @@ Format specific information: granularity: 32768 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -80,6 +82,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -123,6 +126,7 @@ Format specific information: granularity: 65536 refcount bits: 16 corrupt: false + extended l2: false The same bitmaps in JSON format: [ @@ -167,5 +171,6 @@ Format specific information: granularity: 16384 refcount bits: 16 corrupt: false + extended l2: false Test complete diff --git a/tests/qemu-iotests/255.out b/tests/qemu-iotests/255.out index a3c99fd62e..38ac1a4e95 100644 --- a/tests/qemu-iotests/255.out +++ b/tests/qemu-iotests/255.out @@ -3,9 +3,9 @@ Finishing a commit job with background reads === Create backing chain and start VM === -Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-t.qcow2.mid', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-t.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib === Start background read requests === @@ -23,9 +23,9 @@ Closing the VM while a job is being cancelled === Create images and start VM === -Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-src.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-dst.qcow2', fmt=qcow2 size=134217728 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out index d24ff681af..47edc3c423 100644 --- a/tests/qemu-iotests/274.out +++ b/tests/qemu-iotests/274.out @@ -1,9 +1,9 @@ == Commit tests == -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -56,6 +56,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -64,11 +65,11 @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) === Testing HMP commit (top -> mid) === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -86,6 +87,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -94,11 +96,11 @@ read 1048576/1048576 bytes at offset 1048576 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) === Testing QMP active commit (top -> mid) === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=2097152 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-mid', fmt=qcow2 size=1048576 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=2097152 backing_file=TEST_DIR/PID-mid cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 2097152/2097152 bytes at offset 0 2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -122,6 +124,7 @@ Format specific information: lazy refcounts: false refcount bits: 16 corrupt: false + extended l2: false read 1048576/1048576 bytes at offset 0 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -131,9 +134,9 @@ read 1048576/1048576 bytes at offset 1048576 == Resize tests == === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=6442450944 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=1073741824 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 5368709120 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -150,9 +153,9 @@ read 65536/65536 bytes at offset 5368709120 { "start": 1073741824, "length": 7516192768, "depth": 0, "zero": true, "data": false}] === preallocation=metadata === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=34359738368 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=32212254720 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 33285996544 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -174,9 +177,9 @@ read 65536/65536 bytes at offset 33285996544 { "start": 34896609280, "length": 536870912, "depth": 0, "zero": true, "data": false, "offset": 2685075456}] === preallocation=falloc === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=10485760 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=5242880 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 9437184 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -193,9 +196,9 @@ read 65536/65536 bytes at offset 9437184 { "start": 5242880, "length": 10485760, "depth": 0, "zero": false, "data": true, "offset": 327680}] === preallocation=full === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=16777216 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=8388608 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 11534336 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -212,9 +215,9 @@ read 65536/65536 bytes at offset 11534336 { "start": 8388608, "length": 4194304, "depth": 0, "zero": false, "data": true, "offset": 327680}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=393216 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=393216 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=259072 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=259072 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 259072 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -232,9 +235,9 @@ read 65536/65536 bytes at offset 259072 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=409600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=409600 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 344064 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -251,9 +254,9 @@ read 65536/65536 bytes at offset 344064 { "start": 262144, "length": 262144, "depth": 0, "zero": true, "data": false}] === preallocation=off === -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=524288 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=524288 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib -Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-top', fmt=qcow2 size=262144 backing_file=TEST_DIR/PID-base cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib wrote 65536/65536 bytes at offset 446464 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/qemu-iotests/280.out b/tests/qemu-iotests/280.out index 92e4d14079..372e9f70ad 100644 --- a/tests/qemu-iotests/280.out +++ b/tests/qemu-iotests/280.out @@ -1,4 +1,4 @@ -Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 compression_type=zlib +Formatting 'TEST_DIR/PID-base', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off extended_l2=off refcount_bits=16 compression_type=zlib === Launch VM === Enabling migration QMP events on VM... diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 03e4f71808..2ca66de0f0 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -146,6 +146,7 @@ _filter_img_create() -e "s# adapter_type=[^ ]*##g" \ -e "s# hwversion=[^ ]*##g" \ -e "s# lazy_refcounts=\\(on\\|off\\)##g" \ + -e "s# extended_l2=\\(on\\|off\\)##g" \ -e "s# block_size=[0-9]\\+##g" \ -e "s# block_state_zero=\\(on\\|off\\)##g" \ -e "s# log_size=[0-9]\\+##g" \ From patchwork Mon May 25 18:08:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 006B9C433DF for ; Mon, 25 May 2020 18:17:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BFB2F2071C for ; Mon, 25 May 2020 18:16:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="QY6LIkS+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BFB2F2071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43312 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHek-0007wV-VY for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:16:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXs-0003xb-O3; Mon, 25 May 2020 14:09:52 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:50992) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXq-0007pf-2W; Mon, 25 May 2020 14:09:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=Oe86ilax3C8jctFM722sy6VnDUpDOutRe3DDBEhiVz4=; b=QY6LIkS+kqitLscv5WV1Z8J+dlpTv9FbMlUS2a5Vt+NpBVLmMooAZVyghjmzrN0lhjmw8wJRTneBWPBO34RaG3qihoy97Z6lCp/hhHl/Uu5kW+AUFkZrvoOCtP6UcfWRsqqLTIcbdwDVgZ1AVh5BiwrvsSHBckKkW6K9EuAcIQK4PLDKXxU2mrNAEtvw8niAkH2g41uhy/SzOF4qBpeUcZig/yQomTvHHytLKaxDMkUHR03XjUAjAdtVBhShXTVndkKlglG+vKOncw6uxr++NlPNN7Tby4D+CyG6qkcgCPpC8pmLlD3pYr/XzhEhzbwTDJG1gQFnJVeLD07GfIIvzg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088l-FL; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002Nq-Fo; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 31/32] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters Date: Mon, 25 May 2020 20:08:56 +0200 Message-Id: <46c7f6775d80ad10d791c94cf49e402858466b58.1590429902.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function is only used by qcow2_expand_zero_clusters() to downgrade a qcow2 image to a previous version. It is however not possible to downgrade an image with extended L2 entries because older versions of qcow2 do not have this feature. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 8 +++++++- tests/qemu-iotests/061 | 6 ++++++ tests/qemu-iotests/061.out | 5 +++++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 1641976028..2ceeb22f29 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -2157,6 +2157,9 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, int ret; int i, j; + /* qcow2_downgrade() is not allowed in images with subclusters */ + assert(!has_subclusters(s)); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; @@ -2225,7 +2228,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { if (!bs->backing) { /* not backed; therefore we can simply deallocate the - * cluster */ + * cluster. No need to call set_l2_bitmap(), this + * function doesn't support images with subclusters. */ set_l2_entry(s, l2_slice, j, 0); l2_dirty = true; continue; @@ -2296,6 +2300,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } else { set_l2_entry(s, l2_slice, j, offset); } + /* No need to call set_l2_bitmap() after set_l2_entry() because + * this function doesn't support images with subclusters. */ l2_dirty = true; } diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index 10eb243164..23add2dfe3 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -303,6 +303,12 @@ $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" _img_info --format-specific _check_test_img +echo +echo "=== Testing version downgrade with extended L2 entries ===" +echo +_make_test_img -o "compat=1.1,extended_l2=on" 64M +$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" + echo echo "=== Try changing the external data file ===" echo diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index 39812d8cf8..c1acdbd751 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -528,6 +528,11 @@ Format specific information: extended l2: false No errors were found on the image. +=== Testing version downgrade with extended L2 entries === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Cannot downgrade an image with incompatible features 0x10 set + === Try changing the external data file === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 From patchwork Mon May 25 18:08:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 281921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6035BC433DF for ; Mon, 25 May 2020 18:30:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0610B20671 for ; Mon, 25 May 2020 18:30:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="qWTA1wu+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0610B20671 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38396 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jdHrv-0005Ab-8I for qemu-devel@archiver.kernel.org; Mon, 25 May 2020 14:30:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jdHXz-0004Fu-Oh; Mon, 25 May 2020 14:09:59 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:51101) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jdHXp-0007qB-Jd; Mon, 25 May 2020 14:09:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=2GTZxdPC1KRB/CooM/8Z/rox7iMvqhdJWjaOaGyDg/M=; b=qWTA1wu+nQTqqQ0qh6UTIxqVMG4cHT9eiZUNH94M018aDjXfMlwj20QlWmgxE1/+nL/Us5C2C4UVHty8RRiU7E+E5fD48+5zOAGRMoe7Y3LYXwWI8FXpgE1wkzBlEkfddb2gEzlTqMqeX3PuziT+ad2Khr2ewbrIdE7CgwdD/+Nqjuj96m3H8aepKrhxR9DeHG1nKupgiLp/BEIKf0fTo0wPpnXn2Tl8W/IAyfxUxNETugHi4Ld7HDD2/QpDxbBxXa8mRBOwQ1kIQkRzm2MVe35ekNmyYWFPNPAVGt2cVDbH6Gq+0XmUDcHv94A8Is/BXftraD3jqBcR7BBpd82k4A==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jdHXP-00088n-Ih; Mon, 25 May 2020 20:09:23 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jdHXA-0002Nt-H8; Mon, 25 May 2020 20:09:08 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v7 32/32] iotests: Add tests for qcow2 images with extended L2 entries Date: Mon, 25 May 2020 20:08:57 +0200 Message-Id: <117ba99f5a1fbea62cfed60fe74a460cc628948f.1590429902.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/25 14:09:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Alberto Garcia --- tests/qemu-iotests/271 | 705 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/271.out | 603 +++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 1309 insertions(+) create mode 100755 tests/qemu-iotests/271 create mode 100644 tests/qemu-iotests/271.out diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 new file mode 100755 index 0000000000..4dd11772cf --- /dev/null +++ b/tests/qemu-iotests/271 @@ -0,0 +1,705 @@ +#!/bin/bash +# +# Test qcow2 images with extended L2 entries +# +# Copyright (C) 2019-2020 Igalia, S.L. +# Author: Alberto Garcia +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +# creator +owner=berto@igalia.com + +seq="$(basename $0)" +echo "QA output created by $seq" + +here="$PWD" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_IMG.raw" +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file nfs +_supported_os Linux +_unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file + +l2_offset=262144 # 0x40000 + +_verify_img() +{ + $QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.raw" | grep -v 'Images are identical' + $QEMU_IMG check "$TEST_IMG" | _filter_qemu_img_check | \ + grep -v 'No errors were found on the image' +} + +# Compare the bitmap of an extended L2 entry against an expected value +_verify_l2_bitmap() +{ + entry_no="$1" # L2 entry number, starting from 0 + expected_alloc="$alloc" # Space-separated list of allocated subcluster indexes + expected_zero="$zero" # Space-separated list of zero subcluster indexes + + offset=$(($l2_offset + $entry_no * 16)) + entry=`peek_file_be "$TEST_IMG" $offset 8` + offset=$(($offset + 8)) + bitmap=`peek_file_be "$TEST_IMG" $offset 8` + + expected_bitmap=0 + for bit in $expected_alloc; do + expected_bitmap=$(($expected_bitmap | (1 << $bit))) + done + for bit in $expected_zero; do + expected_bitmap=$(($expected_bitmap | (1 << (32 + $bit)))) + done + expected_bitmap=`printf "%llu" $expected_bitmap` + + printf "L2 entry #%d: 0x%016lx %016lx\n" "$entry_no" "$entry" "$bitmap" + if [ "$bitmap" != "$expected_bitmap" ]; then + printf "ERROR: expecting bitmap 0x%016lx\n" "$expected_bitmap" + fi +} + +_test_write() +{ + cmd="$1" + l2_entry_idx="$2" + [ -n "$l2_entry_idx" ] || l2_entry_idx=0 + raw_cmd=`echo $cmd | sed s/-c//` # Raw images don't support -c + echo "$cmd" + $QEMU_IO -c "$cmd" "$TEST_IMG" | _filter_qemu_io + $QEMU_IO -c "$raw_cmd" -f raw "$TEST_IMG.raw" | _filter_qemu_io + _verify_img + _verify_l2_bitmap "$l2_entry_idx" +} + +_reset_img() +{ + size="$1" + $QEMU_IMG create -f raw "$TEST_IMG.raw" "$size" | _filter_img_create + if [ "$use_backing_file" = "yes" ]; then + $QEMU_IMG create -f raw "$TEST_IMG.base" "$size" | _filter_img_create + $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.base" | _filter_qemu_io + $QEMU_IO -c "write -q -P 0xFF 0 $size" -f raw "$TEST_IMG.raw" | _filter_qemu_io + _make_test_img -o extended_l2=on -b "$TEST_IMG.base" "$size" + else + _make_test_img -o extended_l2=on "$size" + fi +} + +# Test that writing to an image with subclusters produces the expected +# results, in images with and without backing files +for use_backing_file in yes no; do + echo + echo "### Standard write tests (backing file: $use_backing_file) ###" + echo + _reset_img 1M + ### Write subcluster #0 (beginning of subcluster) ### + alloc="0"; zero="" + _test_write 'write -q -P 1 0 1k' + + ### Write subcluster #1 (middle of subcluster) ### + alloc="0 1"; zero="" + _test_write 'write -q -P 2 3k 512' + + ### Write subcluster #2 (end of subcluster) ### + alloc="0 1 2"; zero="" + _test_write 'write -q -P 3 5k 1k' + + ### Write subcluster #3 (full subcluster) ### + alloc="0 1 2 3"; zero="" + _test_write 'write -q -P 4 6k 2k' + + ### Write subclusters #4-6 (full subclusters) ### + alloc="`seq 0 6`"; zero="" + _test_write 'write -q -P 5 8k 6k' + + ### Write subclusters #7-9 (partial subclusters) ### + alloc="`seq 0 9`"; zero="" + _test_write 'write -q -P 6 15k 4k' + + ### Write subcluster #16 (partial subcluster) ### + alloc="`seq 0 9` 16"; zero="" + _test_write 'write -q -P 7 32k 1k' + + ### Write subcluster #31-#33 (cluster overlap) ### + alloc="`seq 0 9` 16 31"; zero="" + _test_write 'write -q -P 8 63k 4k' + alloc="0 1" ; zero="" + _verify_l2_bitmap 1 + + ### Zero subcluster #1 + alloc="0 `seq 2 9` 16 31"; zero="1" + _test_write 'write -q -z 2k 2k' + + ### Zero cluster #0 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z 0 64k' + + ### Fill cluster #0 with data + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -P 9 0 64k' + + ### Zero and unmap half of cluster #0 (this won't unmap it) + alloc="`seq 16 31`"; zero="`seq 0 15`" + _test_write 'write -q -z -u 0 32k' + + ### Zero and unmap cluster #0 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z -u 0 64k' + + ### Write subcluster #1 (middle of subcluster) + alloc="1"; zero="0 `seq 2 31`" + _test_write 'write -q -P 10 3k 512' + + ### Fill cluster #0 with data + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -P 11 0 64k' + + ### Discard cluster #0 + alloc=""; zero="`seq 0 31`" + _test_write 'discard -q 0 64k' + + ### Write compressed data to cluster #0 + alloc=""; zero="" + _test_write 'write -q -c -P 12 0 64k' + + ### Write subcluster #2 (middle of subcluster) + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -P 13 3k 512' +done + +# calculate_l2_meta() checks if none of the clusters affected by a +# write operation need COW or changes to their L2 metadata and simply +# returns when they don't. This is a test for that optimization. +# Here clusters #0-#3 are overwritten but only #1 and #2 need changes. +echo +echo '### Overwriting several clusters without COW ###' +echo +use_backing_file="no" _reset_img 1M +# Write cluster #0, subclusters #12-#31 +alloc="`seq 12 31`"; zero="" +_test_write 'write -q 24k 40k' 0 + +# Write cluster #1, subcluster #13 +alloc="13"; zero="" +_test_write 'write -q 90k 2k' 1 + +# Zeroize cluster #2, subcluster #14 +alloc="14"; zero="" +_test_write 'write -q 156k 2k' 2 +alloc=""; zero="14" +_test_write 'write -q -z 156k 2k' 2 + +# Write cluster #3, subclusters #15-#31 +alloc="`seq 0 16`"; zero="" +_test_write 'write -q 192k 34k' 3 + +# Write from cluster #0, subcluster #12 to cluster #3, subcluster #11 +alloc="`seq 12 31`"; zero="" +_test_write 'write -q 24k 192k' 0 + +alloc="`seq 0 31`"; zero="" +_verify_l2_bitmap 1 +_verify_l2_bitmap 2 + +alloc="`seq 0 16`"; zero="" +_verify_l2_bitmap 3 + +# Test different patterns of writing zeroes +for use_backing_file in yes no; do + echo + echo "### Writing zeroes 1: unallocated clusters (backing file: $use_backing_file) ###" + echo + # Note that the image size is not a multiple of the cluster size + _reset_img 2083k + + # Cluster-aligned request from clusters #0 to #2 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z 0 192k' 0 + _verify_l2_bitmap 1 + _verify_l2_bitmap 2 + + # Subcluster-aligned request from clusters #3 to #5 + alloc=""; zero="`seq 16 31`" + _test_write 'write -q -z 224k 128k' 3 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 4 + alloc=""; zero="`seq 0 15`" + _verify_l2_bitmap 5 + + # Unaligned request from clusters #6 to #8 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="`seq 16 31`" # copy-on-write happening here + else + alloc=""; zero="`seq 15 31`" + fi + _test_write 'write -q -z 415k 128k' 6 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 7 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="`seq 0 14`" # copy-on-write happening here + else + alloc=""; zero="`seq 0 15`" + fi + _verify_l2_bitmap 8 + + echo + echo "### Writing zeroes 2: allocated clusters (backing file: $use_backing_file) ###" + echo + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -P 1 576k 576k' 9 + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + _verify_l2_bitmap 12 + _verify_l2_bitmap 13 + _verify_l2_bitmap 14 + _verify_l2_bitmap 15 + _verify_l2_bitmap 16 + _verify_l2_bitmap 17 + + # Cluster-aligned request from clusters #9 to #11 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z 576k 192k' 9 + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + + # Subcluster-aligned request from clusters #12 to #14 + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z 800k 128k' 12 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 13 + alloc="`seq 16 31`"; zero="`seq 0 15`" + _verify_l2_bitmap 14 + + # Unaligned request from clusters #15 to #17 + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z 991k 128k' 15 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 16 + alloc="`seq 15 31`"; zero="`seq 0 14`" + _verify_l2_bitmap 17 + + echo + echo "### Writing zeroes 3: compressed clusters (backing file: $use_backing_file) ###" + echo + alloc=""; zero="" + for c in `seq 18 28`; do + _test_write "write -q -c -P 2 $(($c*64))k 64k" $c + done + + # Cluster-aligned request from clusters #18 to #20 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z 1152k 192k' 18 + _verify_l2_bitmap 19 + _verify_l2_bitmap 20 + + # Subcluster-aligned request from clusters #21 to #23. + # We cannot partially zero a compressed cluster so the code + # returns -ENOTSUP, which means copy-on-write of the compressed + # data and fill the rest with actual zeroes on disk. + # TODO: cluster #22 should use the 'all zeroes' bits. + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -z 1376k 128k' 21 + _verify_l2_bitmap 22 + _verify_l2_bitmap 23 + + # Unaligned request from clusters #24 to #26 + # In this case QEMU internally sends a 1k request followed by a + # subcluster-aligned 128k request. The first request decompresses + # cluster #24, but that's not enough to perform efficiently the + # second request because it partially writes to cluster #26 (which + # is compressed) so we hit the same problem as before. + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -z 1567k 129k' 24 + _verify_l2_bitmap 25 + _verify_l2_bitmap 26 + + # Unaligned request from clusters #27 to #29 + # Similar to the previous case, but this time the tail of the + # request does not correspond to a compressed cluster, so it can + # be zeroed efficiently. + # Note that the very last subcluster is partially written, so if + # there's a backing file we need to perform cow. + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z 1759k 128k' 27 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 28 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="`seq 0 14`" # copy-on-write happening here + else + alloc=""; zero="`seq 0 15`" + fi + _verify_l2_bitmap 29 + + echo + echo "### Writing zeroes 4: other tests (backing file: $use_backing_file) ###" + echo + # Unaligned request in the middle of cluster #30. + # If there's a backing file we need to allocate and do + # copy-on-write on the partially zeroed subclusters. + # If not we can set the 'all zeroes' bit on them. + if [ "$use_backing_file" = "yes" ]; then + alloc="15 19"; zero="`seq 16 18`" # copy-on-write happening here + else + alloc=""; zero="`seq 15 19`" + fi + _test_write 'write -q -z 1951k 8k' 30 + + # Fill the last cluster with zeroes, up to the end of the image + # (the image size is not a multiple of the cluster or subcluster size). + alloc=""; zero="`seq 0 17`" + _test_write 'write -q -z 2048k 35k' 32 +done + +# Zero + unmap +for use_backing_file in yes no; do + echo + echo "### Zero + unmap 1: allocated clusters (backing file: $use_backing_file) ###" + echo + # Note that the image size is not a multiple of the cluster size + _reset_img 2083k + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -P 1 576k 576k' 9 + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + _verify_l2_bitmap 12 + _verify_l2_bitmap 13 + _verify_l2_bitmap 14 + _verify_l2_bitmap 15 + _verify_l2_bitmap 16 + _verify_l2_bitmap 17 + + # Cluster-aligned request from clusters #9 to #11 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z -u 576k 192k' 9 + _verify_l2_bitmap 10 + _verify_l2_bitmap 11 + + # Subcluster-aligned request from clusters #12 to #14 + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z -u 800k 128k' 12 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 13 + alloc="`seq 16 31`"; zero="`seq 0 15`" + _verify_l2_bitmap 14 + + # Unaligned request from clusters #15 to #17 + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z -u 991k 128k' 15 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 16 + alloc="`seq 15 31`"; zero="`seq 0 14`" + _verify_l2_bitmap 17 + + echo + echo "### Zero + unmap 2: compressed clusters (backing file: $use_backing_file) ###" + echo + alloc=""; zero="" + for c in `seq 18 28`; do + _test_write "write -q -c -P 2 $(($c*64))k 64k" $c + done + + # Cluster-aligned request from clusters #18 to #20 + alloc=""; zero="`seq 0 31`" + _test_write 'write -q -z -u 1152k 192k' 18 + _verify_l2_bitmap 19 + _verify_l2_bitmap 20 + + # Subcluster-aligned request from clusters #21 to #23. + # We cannot partially zero a compressed cluster so the code + # returns -ENOTSUP, which means copy-on-write of the compressed + # data and fill the rest with actual zeroes on disk. + # TODO: cluster #22 should use the 'all zeroes' bits. + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -z -u 1376k 128k' 21 + _verify_l2_bitmap 22 + _verify_l2_bitmap 23 + + # Unaligned request from clusters #24 to #26 + # In this case QEMU internally sends a 1k request followed by a + # subcluster-aligned 128k request. The first request decompresses + # cluster #24, but that's not enough to perform efficiently the + # second request because it partially writes to cluster #26 (which + # is compressed) so we hit the same problem as before. + alloc="`seq 0 31`"; zero="" + _test_write 'write -q -z -u 1567k 129k' 24 + _verify_l2_bitmap 25 + _verify_l2_bitmap 26 + + # Unaligned request from clusters #27 to #29 + # Similar to the previous case, but this time the tail of the + # request does not correspond to a compressed cluster, so it can + # be zeroed efficiently. + # Note that the very last subcluster is partially written, so if + # there's a backing file we need to perform cow. + alloc="`seq 0 15`"; zero="`seq 16 31`" + _test_write 'write -q -z -u 1759k 128k' 27 + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 28 + if [ "$use_backing_file" = "yes" ]; then + alloc="15"; zero="`seq 0 14`" # copy-on-write happening here + else + alloc=""; zero="`seq 0 15`" + fi + _verify_l2_bitmap 29 +done + +# Test qcow2_cluster_discard() with full and normal discards +for use_backing_file in yes no; do + echo + echo "### Discarding clusters with non-zero bitmaps (backing file: $use_backing_file) ###" + echo + if [ "$use_backing_file" = "yes" ]; then + _make_test_img -o extended_l2=on -b "$TEST_IMG.base" 1M + else + _make_test_img -o extended_l2=on 1M + fi + # Write clusters #0-#2 and then discard them + $QEMU_IO -c 'write -q 0 128k' "$TEST_IMG" + $QEMU_IO -c 'discard -q 0 128k' "$TEST_IMG" + # 'qemu-io discard' doesn't do a full discard, it zeroizes the + # cluster, so both clusters have all zero bits set now + alloc=""; zero="`seq 0 31`" + _verify_l2_bitmap 0 + _verify_l2_bitmap 1 + # Now mark the 2nd half of the subclusters from cluster #0 as unallocated + poke_file "$TEST_IMG" $(($l2_offset+8)) "\x00\x00" + # Discard cluster #0 again to see how the zero bits have changed + $QEMU_IO -c 'discard -q 0 64k' "$TEST_IMG" + # And do a full discard of cluster #1 by shrinking and growing the image + $QEMU_IMG resize --shrink "$TEST_IMG" 64k + $QEMU_IMG resize "$TEST_IMG" 1M + # A normal discard sets all 'zero' bits only if the image has a + # backing file, otherwise it won't touch them. + if [ "$use_backing_file" = "yes" ]; then + alloc=""; zero="`seq 0 31`" + else + alloc=""; zero="`seq 0 15`" + fi + _verify_l2_bitmap 0 + # A full discard should clear the L2 entry completely. However + # when growing an image with a backing file the new clusters are + # zeroized to hide the stale data from the backing file + if [ "$use_backing_file" = "yes" ]; then + alloc=""; zero="`seq 0 31`" + else + alloc=""; zero="" + fi + _verify_l2_bitmap 1 +done + +# Test that corrupted L2 entries are detected in both read and write +# operations +for corruption_test_cmd in read write; do + echo + echo "### Corrupted L2 entries - $corruption_test_cmd test (allocated) ###" + echo + echo "# 'cluster is zero' bit set on the standard cluster descriptor" + echo + # We actually don't consider this a corrupted image. + # The 'cluster is zero' bit is unused is unused in extended L2 entries + # so QEMU ignores it. + # TODO: maybe treat the image as corrupted and make qemu-img check fix it? + _make_test_img -o extended_l2=on 1M + $QEMU_IO -c 'write -q -P 0x11 0 2k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01" + alloc="0"; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -q -P 0x11 0 1k" "$TEST_IMG" + if [ "$corruption_test_cmd" = "write" ]; then + alloc="0"; zero="" + fi + _verify_l2_bitmap 0 + + echo + echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set" + echo + _make_test_img -o extended_l2=on 1M + # Write from the middle of cluster #0 to the middle of cluster #2 + $QEMU_IO -c 'write -q 32k 128k' "$TEST_IMG" + # Corrupt the L2 entry from cluster #1 + poke_file "$TEST_IMG" $(($l2_offset+24)) "\x00\x00\x00\x01" + alloc="`seq 0 31`"; zero="0" + _verify_l2_bitmap 1 + $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG" + + echo + echo "### Corrupted L2 entries - $corruption_test_cmd test (unallocated) ###" + echo + echo "# 'cluster is zero' bit set on the standard cluster descriptor" + echo + # We actually don't consider this a corrupted image. + # The 'cluster is zero' bit is unused is unused in extended L2 entries + # so QEMU ignores it. + # TODO: maybe treat the image as corrupted and make qemu-img check fix it? + _make_test_img -o extended_l2=on 1M + # We want to modify the (empty) L2 entry from cluster #0, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+7)) "\x01" + alloc=""; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -q 0 1k" "$TEST_IMG" + if [ "$corruption_test_cmd" = "write" ]; then + alloc="0"; zero="" + fi + _verify_l2_bitmap 0 + + echo + echo "# 'subcluster is allocated' bit set" + echo + _make_test_img -o extended_l2=on 1M + # We want to corrupt the (empty) L2 entry from cluster #0, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + poke_file "$TEST_IMG" $(($l2_offset+15)) "\x01" + alloc="0"; zero="" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd 0 1k" "$TEST_IMG" + + echo + echo "# Both 'subcluster is zero' and 'subcluster is allocated' bits set" + echo + _make_test_img -o extended_l2=on 1M + # We want to corrupt the (empty) L2 entry from cluster #1, + # but we write to #4 in order to initialize the L2 table first + $QEMU_IO -c 'write -q 256k 1k' "$TEST_IMG" + # Corrupt the L2 entry from cluster #1 + poke_file "$TEST_IMG" $(($l2_offset+24)) "\x00\x00\x00\x01\x00\x00\x00\x01" + alloc="0"; zero="0" + _verify_l2_bitmap 1 + $QEMU_IO -c "$corruption_test_cmd 0 192k" "$TEST_IMG" + + echo + echo "### Compressed cluster with subcluster bitmap != 0 - $corruption_test_cmd test ###" + echo + # We actually don't consider this a corrupted image. + # The bitmap in compressed clusters is unused so QEMU should just ignore it. + _make_test_img -o extended_l2=on 1M + $QEMU_IO -c 'write -q -P 11 -c 0 64k' "$TEST_IMG" + # Change the L2 bitmap to allocate subcluster #31 and zeroize subcluster #0 + poke_file "$TEST_IMG" $(($l2_offset+11)) "\x01\x80" + alloc="31"; zero="0" + _verify_l2_bitmap 0 + $QEMU_IO -c "$corruption_test_cmd -P 11 0 64k" "$TEST_IMG" | _filter_qemu_io + # Writing allocates a new uncompressed cluster so we get a new bitmap + if [ "$corruption_test_cmd" = "write" ]; then + alloc="`seq 0 31`"; zero="" + fi + _verify_l2_bitmap 0 +done + +echo +echo "### Image creation options ###" +echo +echo "# cluster_size < 16k" +_make_test_img -o extended_l2=on,cluster_size=8k 1M + +# TODO: allow preallocation with backing files when subclusters are used +echo "# backing file and preallocation=metadata" +_make_test_img -o extended_l2=on,preallocation=metadata -b "$TEST_IMG.base" 1M + +echo "# backing file and preallocation=falloc" +_make_test_img -o extended_l2=on,preallocation=falloc -b "$TEST_IMG.base" 1M + +echo "# backing file and preallocation=full" +_make_test_img -o extended_l2=on,preallocation=full -b "$TEST_IMG.base" 1M + +echo +echo "### qemu-img measure ###" +echo +echo "# 512MB, extended_l2=off" # This needs one L2 table +$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=off +echo "# 512MB, extended_l2=on" # This needs two L2 tables +$QEMU_IMG measure --size 512M -O qcow2 -o extended_l2=on + +echo "# 16K clusters, 64GB, extended_l2=off" # This needs one full L1 table cluster +$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=off +echo "# 16K clusters, 64GB, extended_l2=on" # This needs two full L2 table clusters +$QEMU_IMG measure --size 64G -O qcow2 -o cluster_size=16k,extended_l2=on + +echo "# 8k clusters" # This should fail +$QEMU_IMG measure --size 1M -O qcow2 -o cluster_size=8k,extended_l2=on + +echo "# 1024 TB" # Maximum allowed size with extended_l2=on and 64K clusters +$QEMU_IMG measure --size 1024T -O qcow2 -o extended_l2=on +echo "# 1025 TB" # This should fail +$QEMU_IMG measure --size 1025T -O qcow2 -o extended_l2=on + +echo +echo "### qemu-img amend ###" +echo +_make_test_img -o extended_l2=on 1M +$QEMU_IMG amend -o extended_l2=off "$TEST_IMG" + +_make_test_img -o extended_l2=off 1M +$QEMU_IMG amend -o extended_l2=on "$TEST_IMG" + +echo +echo "### Test copy-on-write on an image with snapshots ###" +echo +_make_test_img -o extended_l2=on 1M + +# For each cluster from #0 to #9 this loop zeroes subcluster #7 +# and allocates subclusters #13 and #18. +alloc="13 18"; zero="7" +for c in `seq 0 9`; do + $QEMU_IO -c "write -q -z $((64*$c+14))k 2k" \ + -c "write -q -P $((0xd0+$c)) $((64*$c+26))k 2k" \ + -c "write -q -P $((0xe0+$c)) $((64*$c+36))k 2k" "$TEST_IMG" + _verify_l2_bitmap "$c" +done + +# Create a snapshot and set l2_offset to the new L2 table +$QEMU_IMG snapshot -c snap1 "$TEST_IMG" +l2_offset=1114112 # 0x110000 + +# Write different patterns to each one of the clusters +# in order to see how copy-on-write behaves in each case. +$QEMU_IO -c "write -q -P 0xf0 $((64*0+30))k 1k" \ + -c "write -q -P 0xf1 $((64*1+20))k 1k" \ + -c "write -q -P 0xf2 $((64*2+40))k 1k" \ + -c "write -q -P 0xf3 $((64*3+26))k 1k" \ + -c "write -q -P 0xf4 $((64*4+14))k 1k" \ + -c "write -q -P 0xf5 $((64*5+1))k 1k" \ + -c "write -q -z $((64*6+30))k 3k" \ + -c "write -q -z $((64*7+26))k 2k" \ + -c "write -q -z $((64*8+26))k 1k" \ + -c "write -q -z $((64*9+12))k 1k" \ + "$TEST_IMG" +alloc="`seq 13 18`"; zero="7" _verify_l2_bitmap 0 +alloc="`seq 10 18`"; zero="7" _verify_l2_bitmap 1 +alloc="`seq 13 20`"; zero="7" _verify_l2_bitmap 2 +alloc="`seq 13 18`"; zero="7" _verify_l2_bitmap 3 +alloc="`seq 7 18`"; zero="" _verify_l2_bitmap 4 +alloc="`seq 0 18`"; zero="" _verify_l2_bitmap 5 +alloc="13 18"; zero="7 15 16" _verify_l2_bitmap 6 +alloc="18"; zero="7 13" _verify_l2_bitmap 7 +alloc="`seq 13 18`"; zero="7" _verify_l2_bitmap 8 +alloc="13 18"; zero="6 7" _verify_l2_bitmap 9 + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/271.out b/tests/qemu-iotests/271.out new file mode 100644 index 0000000000..6ac53d5e10 --- /dev/null +++ b/tests/qemu-iotests/271.out @@ -0,0 +1,603 @@ +QA output created by 271 + +### Standard write tests (backing file: yes) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=1048576 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base +write -q -P 1 0 1k +L2 entry #0: 0x8000000000050000 0000000000000001 +write -q -P 2 3k 512 +L2 entry #0: 0x8000000000050000 0000000000000003 +write -q -P 3 5k 1k +L2 entry #0: 0x8000000000050000 0000000000000007 +write -q -P 4 6k 2k +L2 entry #0: 0x8000000000050000 000000000000000f +write -q -P 5 8k 6k +L2 entry #0: 0x8000000000050000 000000000000007f +write -q -P 6 15k 4k +L2 entry #0: 0x8000000000050000 00000000000003ff +write -q -P 7 32k 1k +L2 entry #0: 0x8000000000050000 00000000000103ff +write -q -P 8 63k 4k +L2 entry #0: 0x8000000000050000 00000000800103ff +L2 entry #1: 0x8000000000060000 0000000000000003 +write -q -z 2k 2k +L2 entry #0: 0x8000000000050000 00000002800103fd +write -q -z 0 64k +L2 entry #0: 0x8000000000050000 ffffffff00000000 +write -q -P 9 0 64k +L2 entry #0: 0x8000000000050000 00000000ffffffff +write -q -z -u 0 32k +L2 entry #0: 0x8000000000050000 0000ffffffff0000 +write -q -z -u 0 64k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +write -q -P 10 3k 512 +L2 entry #0: 0x8000000000050000 fffffffd00000002 +write -q -P 11 0 64k +L2 entry #0: 0x8000000000050000 00000000ffffffff +discard -q 0 64k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +write -q -c -P 12 0 64k +L2 entry #0: 0x4000000000050000 0000000000000000 +write -q -P 13 3k 512 +L2 entry #0: 0x8000000000070000 00000000ffffffff + +### Standard write tests (backing file: no) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +write -q -P 1 0 1k +L2 entry #0: 0x8000000000050000 0000000000000001 +write -q -P 2 3k 512 +L2 entry #0: 0x8000000000050000 0000000000000003 +write -q -P 3 5k 1k +L2 entry #0: 0x8000000000050000 0000000000000007 +write -q -P 4 6k 2k +L2 entry #0: 0x8000000000050000 000000000000000f +write -q -P 5 8k 6k +L2 entry #0: 0x8000000000050000 000000000000007f +write -q -P 6 15k 4k +L2 entry #0: 0x8000000000050000 00000000000003ff +write -q -P 7 32k 1k +L2 entry #0: 0x8000000000050000 00000000000103ff +write -q -P 8 63k 4k +L2 entry #0: 0x8000000000050000 00000000800103ff +L2 entry #1: 0x8000000000060000 0000000000000003 +write -q -z 2k 2k +L2 entry #0: 0x8000000000050000 00000002800103fd +write -q -z 0 64k +L2 entry #0: 0x8000000000050000 ffffffff00000000 +write -q -P 9 0 64k +L2 entry #0: 0x8000000000050000 00000000ffffffff +write -q -z -u 0 32k +L2 entry #0: 0x8000000000050000 0000ffffffff0000 +write -q -z -u 0 64k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +write -q -P 10 3k 512 +L2 entry #0: 0x8000000000050000 fffffffd00000002 +write -q -P 11 0 64k +L2 entry #0: 0x8000000000050000 00000000ffffffff +discard -q 0 64k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +write -q -c -P 12 0 64k +L2 entry #0: 0x4000000000050000 0000000000000000 +write -q -P 13 3k 512 +L2 entry #0: 0x8000000000070000 00000000ffffffff + +### Overwriting several clusters without COW ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +write -q 24k 40k +L2 entry #0: 0x8000000000050000 00000000fffff000 +write -q 90k 2k +L2 entry #1: 0x8000000000060000 0000000000002000 +write -q 156k 2k +L2 entry #2: 0x8000000000070000 0000000000004000 +write -q -z 156k 2k +L2 entry #2: 0x8000000000070000 0000400000000000 +write -q 192k 34k +L2 entry #3: 0x8000000000080000 000000000001ffff +write -q 24k 192k +L2 entry #0: 0x8000000000050000 00000000fffff000 +L2 entry #1: 0x8000000000060000 00000000ffffffff +L2 entry #2: 0x8000000000070000 00000000ffffffff +L2 entry #3: 0x8000000000080000 000000000001ffff + +### Writing zeroes 1: unallocated clusters (backing file: yes) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 backing_file=TEST_DIR/t.IMGFMT.base +write -q -z 0 192k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +L2 entry #1: 0x0000000000000000 ffffffff00000000 +L2 entry #2: 0x0000000000000000 ffffffff00000000 +write -q -z 224k 128k +L2 entry #3: 0x0000000000000000 ffff000000000000 +L2 entry #4: 0x0000000000000000 ffffffff00000000 +L2 entry #5: 0x0000000000000000 0000ffff00000000 +write -q -z 415k 128k +L2 entry #6: 0x8000000000050000 ffff000000008000 +L2 entry #7: 0x0000000000000000 ffffffff00000000 +L2 entry #8: 0x8000000000060000 00007fff00008000 + +### Writing zeroes 2: allocated clusters (backing file: yes) ### + +write -q -P 1 576k 576k +L2 entry #9: 0x8000000000070000 00000000ffffffff +L2 entry #10: 0x8000000000080000 00000000ffffffff +L2 entry #11: 0x8000000000090000 00000000ffffffff +L2 entry #12: 0x80000000000a0000 00000000ffffffff +L2 entry #13: 0x80000000000b0000 00000000ffffffff +L2 entry #14: 0x80000000000c0000 00000000ffffffff +L2 entry #15: 0x80000000000d0000 00000000ffffffff +L2 entry #16: 0x80000000000e0000 00000000ffffffff +L2 entry #17: 0x80000000000f0000 00000000ffffffff +write -q -z 576k 192k +L2 entry #9: 0x8000000000070000 ffffffff00000000 +L2 entry #10: 0x8000000000080000 ffffffff00000000 +L2 entry #11: 0x8000000000090000 ffffffff00000000 +write -q -z 800k 128k +L2 entry #12: 0x80000000000a0000 ffff00000000ffff +L2 entry #13: 0x80000000000b0000 ffffffff00000000 +L2 entry #14: 0x80000000000c0000 0000ffffffff0000 +write -q -z 991k 128k +L2 entry #15: 0x80000000000d0000 ffff00000000ffff +L2 entry #16: 0x80000000000e0000 ffffffff00000000 +L2 entry #17: 0x80000000000f0000 00007fffffff8000 + +### Writing zeroes 3: compressed clusters (backing file: yes) ### + +write -q -c -P 2 1152k 64k +L2 entry #18: 0x4000000000100000 0000000000000000 +write -q -c -P 2 1216k 64k +L2 entry #19: 0x4000000000110000 0000000000000000 +write -q -c -P 2 1280k 64k +L2 entry #20: 0x4000000000120000 0000000000000000 +write -q -c -P 2 1344k 64k +L2 entry #21: 0x4000000000130000 0000000000000000 +write -q -c -P 2 1408k 64k +L2 entry #22: 0x4000000000140000 0000000000000000 +write -q -c -P 2 1472k 64k +L2 entry #23: 0x4000000000150000 0000000000000000 +write -q -c -P 2 1536k 64k +L2 entry #24: 0x4000000000160000 0000000000000000 +write -q -c -P 2 1600k 64k +L2 entry #25: 0x4000000000170000 0000000000000000 +write -q -c -P 2 1664k 64k +L2 entry #26: 0x4000000000180000 0000000000000000 +write -q -c -P 2 1728k 64k +L2 entry #27: 0x4000000000190000 0000000000000000 +write -q -c -P 2 1792k 64k +L2 entry #28: 0x40000000001a0000 0000000000000000 +write -q -z 1152k 192k +L2 entry #18: 0x0000000000000000 ffffffff00000000 +L2 entry #19: 0x0000000000000000 ffffffff00000000 +L2 entry #20: 0x0000000000000000 ffffffff00000000 +write -q -z 1376k 128k +L2 entry #21: 0x8000000000100000 00000000ffffffff +L2 entry #22: 0x8000000000110000 00000000ffffffff +L2 entry #23: 0x8000000000120000 00000000ffffffff +write -q -z 1567k 129k +L2 entry #24: 0x8000000000130000 00000000ffffffff +L2 entry #25: 0x8000000000140000 00000000ffffffff +L2 entry #26: 0x8000000000150000 00000000ffffffff +write -q -z 1759k 128k +L2 entry #27: 0x8000000000160000 ffff00000000ffff +L2 entry #28: 0x0000000000000000 ffffffff00000000 +L2 entry #29: 0x8000000000170000 00007fff00008000 + +### Writing zeroes 4: other tests (backing file: yes) ### + +write -q -z 1951k 8k +L2 entry #30: 0x8000000000180000 0007000000088000 +write -q -z 2048k 35k +L2 entry #32: 0x0000000000000000 0003ffff00000000 + +### Writing zeroes 1: unallocated clusters (backing file: no) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 +write -q -z 0 192k +L2 entry #0: 0x0000000000000000 ffffffff00000000 +L2 entry #1: 0x0000000000000000 ffffffff00000000 +L2 entry #2: 0x0000000000000000 ffffffff00000000 +write -q -z 224k 128k +L2 entry #3: 0x0000000000000000 ffff000000000000 +L2 entry #4: 0x0000000000000000 ffffffff00000000 +L2 entry #5: 0x0000000000000000 0000ffff00000000 +write -q -z 415k 128k +L2 entry #6: 0x0000000000000000 ffff800000000000 +L2 entry #7: 0x0000000000000000 ffffffff00000000 +L2 entry #8: 0x0000000000000000 0000ffff00000000 + +### Writing zeroes 2: allocated clusters (backing file: no) ### + +write -q -P 1 576k 576k +L2 entry #9: 0x8000000000050000 00000000ffffffff +L2 entry #10: 0x8000000000060000 00000000ffffffff +L2 entry #11: 0x8000000000070000 00000000ffffffff +L2 entry #12: 0x8000000000080000 00000000ffffffff +L2 entry #13: 0x8000000000090000 00000000ffffffff +L2 entry #14: 0x80000000000a0000 00000000ffffffff +L2 entry #15: 0x80000000000b0000 00000000ffffffff +L2 entry #16: 0x80000000000c0000 00000000ffffffff +L2 entry #17: 0x80000000000d0000 00000000ffffffff +write -q -z 576k 192k +L2 entry #9: 0x8000000000050000 ffffffff00000000 +L2 entry #10: 0x8000000000060000 ffffffff00000000 +L2 entry #11: 0x8000000000070000 ffffffff00000000 +write -q -z 800k 128k +L2 entry #12: 0x8000000000080000 ffff00000000ffff +L2 entry #13: 0x8000000000090000 ffffffff00000000 +L2 entry #14: 0x80000000000a0000 0000ffffffff0000 +write -q -z 991k 128k +L2 entry #15: 0x80000000000b0000 ffff00000000ffff +L2 entry #16: 0x80000000000c0000 ffffffff00000000 +L2 entry #17: 0x80000000000d0000 00007fffffff8000 + +### Writing zeroes 3: compressed clusters (backing file: no) ### + +write -q -c -P 2 1152k 64k +L2 entry #18: 0x40000000000e0000 0000000000000000 +write -q -c -P 2 1216k 64k +L2 entry #19: 0x40000000000f0000 0000000000000000 +write -q -c -P 2 1280k 64k +L2 entry #20: 0x4000000000100000 0000000000000000 +write -q -c -P 2 1344k 64k +L2 entry #21: 0x4000000000110000 0000000000000000 +write -q -c -P 2 1408k 64k +L2 entry #22: 0x4000000000120000 0000000000000000 +write -q -c -P 2 1472k 64k +L2 entry #23: 0x4000000000130000 0000000000000000 +write -q -c -P 2 1536k 64k +L2 entry #24: 0x4000000000140000 0000000000000000 +write -q -c -P 2 1600k 64k +L2 entry #25: 0x4000000000150000 0000000000000000 +write -q -c -P 2 1664k 64k +L2 entry #26: 0x4000000000160000 0000000000000000 +write -q -c -P 2 1728k 64k +L2 entry #27: 0x4000000000170000 0000000000000000 +write -q -c -P 2 1792k 64k +L2 entry #28: 0x4000000000180000 0000000000000000 +write -q -z 1152k 192k +L2 entry #18: 0x0000000000000000 ffffffff00000000 +L2 entry #19: 0x0000000000000000 ffffffff00000000 +L2 entry #20: 0x0000000000000000 ffffffff00000000 +write -q -z 1376k 128k +L2 entry #21: 0x80000000000e0000 00000000ffffffff +L2 entry #22: 0x80000000000f0000 00000000ffffffff +L2 entry #23: 0x8000000000100000 00000000ffffffff +write -q -z 1567k 129k +L2 entry #24: 0x8000000000110000 00000000ffffffff +L2 entry #25: 0x8000000000120000 00000000ffffffff +L2 entry #26: 0x8000000000130000 00000000ffffffff +write -q -z 1759k 128k +L2 entry #27: 0x8000000000140000 ffff00000000ffff +L2 entry #28: 0x0000000000000000 ffffffff00000000 +L2 entry #29: 0x0000000000000000 0000ffff00000000 + +### Writing zeroes 4: other tests (backing file: no) ### + +write -q -z 1951k 8k +L2 entry #30: 0x0000000000000000 000f800000000000 +write -q -z 2048k 35k +L2 entry #32: 0x0000000000000000 0003ffff00000000 + +### Zero + unmap 1: allocated clusters (backing file: yes) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 backing_file=TEST_DIR/t.IMGFMT.base +write -q -P 1 576k 576k +L2 entry #9: 0x8000000000050000 00000000ffffffff +L2 entry #10: 0x8000000000060000 00000000ffffffff +L2 entry #11: 0x8000000000070000 00000000ffffffff +L2 entry #12: 0x8000000000080000 00000000ffffffff +L2 entry #13: 0x8000000000090000 00000000ffffffff +L2 entry #14: 0x80000000000a0000 00000000ffffffff +L2 entry #15: 0x80000000000b0000 00000000ffffffff +L2 entry #16: 0x80000000000c0000 00000000ffffffff +L2 entry #17: 0x80000000000d0000 00000000ffffffff +write -q -z -u 576k 192k +L2 entry #9: 0x0000000000000000 ffffffff00000000 +L2 entry #10: 0x0000000000000000 ffffffff00000000 +L2 entry #11: 0x0000000000000000 ffffffff00000000 +write -q -z -u 800k 128k +L2 entry #12: 0x8000000000080000 ffff00000000ffff +L2 entry #13: 0x0000000000000000 ffffffff00000000 +L2 entry #14: 0x80000000000a0000 0000ffffffff0000 +write -q -z -u 991k 128k +L2 entry #15: 0x80000000000b0000 ffff00000000ffff +L2 entry #16: 0x0000000000000000 ffffffff00000000 +L2 entry #17: 0x80000000000d0000 00007fffffff8000 + +### Zero + unmap 2: compressed clusters (backing file: yes) ### + +write -q -c -P 2 1152k 64k +L2 entry #18: 0x4000000000050000 0000000000000000 +write -q -c -P 2 1216k 64k +L2 entry #19: 0x4000000000060000 0000000000000000 +write -q -c -P 2 1280k 64k +L2 entry #20: 0x4000000000070000 0000000000000000 +write -q -c -P 2 1344k 64k +L2 entry #21: 0x4000000000090000 0000000000000000 +write -q -c -P 2 1408k 64k +L2 entry #22: 0x40000000000c0000 0000000000000000 +write -q -c -P 2 1472k 64k +L2 entry #23: 0x40000000000e0000 0000000000000000 +write -q -c -P 2 1536k 64k +L2 entry #24: 0x40000000000f0000 0000000000000000 +write -q -c -P 2 1600k 64k +L2 entry #25: 0x4000000000100000 0000000000000000 +write -q -c -P 2 1664k 64k +L2 entry #26: 0x4000000000110000 0000000000000000 +write -q -c -P 2 1728k 64k +L2 entry #27: 0x4000000000120000 0000000000000000 +write -q -c -P 2 1792k 64k +L2 entry #28: 0x4000000000130000 0000000000000000 +write -q -z -u 1152k 192k +L2 entry #18: 0x0000000000000000 ffffffff00000000 +L2 entry #19: 0x0000000000000000 ffffffff00000000 +L2 entry #20: 0x0000000000000000 ffffffff00000000 +write -q -z -u 1376k 128k +L2 entry #21: 0x8000000000050000 00000000ffffffff +L2 entry #22: 0x8000000000060000 00000000ffffffff +L2 entry #23: 0x8000000000070000 00000000ffffffff +write -q -z -u 1567k 129k +L2 entry #24: 0x8000000000090000 00000000ffffffff +L2 entry #25: 0x80000000000e0000 00000000ffffffff +L2 entry #26: 0x80000000000f0000 00000000ffffffff +write -q -z -u 1759k 128k +L2 entry #27: 0x80000000000c0000 ffff00000000ffff +L2 entry #28: 0x0000000000000000 ffffffff00000000 +L2 entry #29: 0x8000000000100000 00007fff00008000 + +### Zero + unmap 1: allocated clusters (backing file: no) ### + +Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=2132992 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2132992 +write -q -P 1 576k 576k +L2 entry #9: 0x8000000000050000 00000000ffffffff +L2 entry #10: 0x8000000000060000 00000000ffffffff +L2 entry #11: 0x8000000000070000 00000000ffffffff +L2 entry #12: 0x8000000000080000 00000000ffffffff +L2 entry #13: 0x8000000000090000 00000000ffffffff +L2 entry #14: 0x80000000000a0000 00000000ffffffff +L2 entry #15: 0x80000000000b0000 00000000ffffffff +L2 entry #16: 0x80000000000c0000 00000000ffffffff +L2 entry #17: 0x80000000000d0000 00000000ffffffff +write -q -z -u 576k 192k +L2 entry #9: 0x0000000000000000 ffffffff00000000 +L2 entry #10: 0x0000000000000000 ffffffff00000000 +L2 entry #11: 0x0000000000000000 ffffffff00000000 +write -q -z -u 800k 128k +L2 entry #12: 0x8000000000080000 ffff00000000ffff +L2 entry #13: 0x0000000000000000 ffffffff00000000 +L2 entry #14: 0x80000000000a0000 0000ffffffff0000 +write -q -z -u 991k 128k +L2 entry #15: 0x80000000000b0000 ffff00000000ffff +L2 entry #16: 0x0000000000000000 ffffffff00000000 +L2 entry #17: 0x80000000000d0000 00007fffffff8000 + +### Zero + unmap 2: compressed clusters (backing file: no) ### + +write -q -c -P 2 1152k 64k +L2 entry #18: 0x4000000000050000 0000000000000000 +write -q -c -P 2 1216k 64k +L2 entry #19: 0x4000000000060000 0000000000000000 +write -q -c -P 2 1280k 64k +L2 entry #20: 0x4000000000070000 0000000000000000 +write -q -c -P 2 1344k 64k +L2 entry #21: 0x4000000000090000 0000000000000000 +write -q -c -P 2 1408k 64k +L2 entry #22: 0x40000000000c0000 0000000000000000 +write -q -c -P 2 1472k 64k +L2 entry #23: 0x40000000000e0000 0000000000000000 +write -q -c -P 2 1536k 64k +L2 entry #24: 0x40000000000f0000 0000000000000000 +write -q -c -P 2 1600k 64k +L2 entry #25: 0x4000000000100000 0000000000000000 +write -q -c -P 2 1664k 64k +L2 entry #26: 0x4000000000110000 0000000000000000 +write -q -c -P 2 1728k 64k +L2 entry #27: 0x4000000000120000 0000000000000000 +write -q -c -P 2 1792k 64k +L2 entry #28: 0x4000000000130000 0000000000000000 +write -q -z -u 1152k 192k +L2 entry #18: 0x0000000000000000 ffffffff00000000 +L2 entry #19: 0x0000000000000000 ffffffff00000000 +L2 entry #20: 0x0000000000000000 ffffffff00000000 +write -q -z -u 1376k 128k +L2 entry #21: 0x8000000000050000 00000000ffffffff +L2 entry #22: 0x8000000000060000 00000000ffffffff +L2 entry #23: 0x8000000000070000 00000000ffffffff +write -q -z -u 1567k 129k +L2 entry #24: 0x8000000000090000 00000000ffffffff +L2 entry #25: 0x80000000000e0000 00000000ffffffff +L2 entry #26: 0x80000000000f0000 00000000ffffffff +write -q -z -u 1759k 128k +L2 entry #27: 0x80000000000c0000 ffff00000000ffff +L2 entry #28: 0x0000000000000000 ffffffff00000000 +L2 entry #29: 0x0000000000000000 0000ffff00000000 + +### Discarding clusters with non-zero bitmaps (backing file: yes) ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base +L2 entry #0: 0x0000000000000000 ffffffff00000000 +L2 entry #1: 0x0000000000000000 ffffffff00000000 +Image resized. +Image resized. +L2 entry #0: 0x0000000000000000 ffffffff00000000 +L2 entry #1: 0x0000000000000000 ffffffff00000000 + +### Discarding clusters with non-zero bitmaps (backing file: no) ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x0000000000000000 ffffffff00000000 +L2 entry #1: 0x0000000000000000 ffffffff00000000 +Image resized. +Image resized. +L2 entry #0: 0x0000000000000000 0000ffff00000000 +L2 entry #1: 0x0000000000000000 0000000000000000 + +### Corrupted L2 entries - read test (allocated) ### + +# 'cluster is zero' bit set on the standard cluster descriptor + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x8000000000050001 0000000000000001 +L2 entry #0: 0x8000000000050001 0000000000000001 + +# Both 'subcluster is zero' and 'subcluster is allocated' bits set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #1: 0x8000000000060000 00000001ffffffff +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed +read failed: Input/output error + +### Corrupted L2 entries - read test (unallocated) ### + +# 'cluster is zero' bit set on the standard cluster descriptor + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x0000000000000001 0000000000000000 +L2 entry #0: 0x0000000000000001 0000000000000000 + +# 'subcluster is allocated' bit set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x0000000000000000 0000000000000001 +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed +read failed: Input/output error + +# Both 'subcluster is zero' and 'subcluster is allocated' bits set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #1: 0x0000000000000000 0000000100000001 +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed +read failed: Input/output error + +### Compressed cluster with subcluster bitmap != 0 - read test ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x4000000000050000 0000000180000000 +read 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +L2 entry #0: 0x4000000000050000 0000000180000000 + +### Corrupted L2 entries - write test (allocated) ### + +# 'cluster is zero' bit set on the standard cluster descriptor + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x8000000000050001 0000000000000001 +L2 entry #0: 0x8000000000050001 0000000000000001 + +# Both 'subcluster is zero' and 'subcluster is allocated' bits set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #1: 0x8000000000060000 00000001ffffffff +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed +write failed: Input/output error + +### Corrupted L2 entries - write test (unallocated) ### + +# 'cluster is zero' bit set on the standard cluster descriptor + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x0000000000000001 0000000000000000 +L2 entry #0: 0x8000000000060000 0000000000000001 + +# 'subcluster is allocated' bit set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x0000000000000000 0000000000000001 +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed +write failed: Input/output error + +# Both 'subcluster is zero' and 'subcluster is allocated' bits set + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #1: 0x0000000000000000 0000000100000001 +qcow2: Marking image as corrupt: Invalid cluster entry found (L2 offset: 0x40000, L2 index: 0x1); further corruption events will be suppressed +write failed: Input/output error + +### Compressed cluster with subcluster bitmap != 0 - write test ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x4000000000050000 0000000180000000 +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +L2 entry #0: 0x8000000000060000 00000000ffffffff + +### Image creation options ### + +# cluster_size < 16k +qemu-img: TEST_DIR/t.IMGFMT: Extended L2 entries are only supported with cluster sizes of at least 16384 bytes +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +# backing file and preallocation=metadata +qemu-img: TEST_DIR/t.IMGFMT: Backing file and preallocation cannot be used at the same time +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base preallocation=metadata +# backing file and preallocation=falloc +qemu-img: TEST_DIR/t.IMGFMT: Backing file and preallocation cannot be used at the same time +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base preallocation=falloc +# backing file and preallocation=full +qemu-img: TEST_DIR/t.IMGFMT: Backing file and preallocation cannot be used at the same time +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.base preallocation=full + +### qemu-img measure ### + +# 512MB, extended_l2=off +required size: 327680 +fully allocated size: 537198592 +# 512MB, extended_l2=on +required size: 393216 +fully allocated size: 537264128 +# 16K clusters, 64GB, extended_l2=off +required size: 42008576 +fully allocated size: 68761485312 +# 16K clusters, 64GB, extended_l2=on +required size: 75579392 +fully allocated size: 68795056128 +# 8k clusters +qemu-img: Extended L2 entries are only supported with cluster sizes of at least 16384 bytes +# 1024 TB +required size: 309285027840 +fully allocated size: 1126209191870464 +# 1025 TB +qemu-img: The image size is too large (try using a larger cluster size) + +### qemu-img amend ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +qemu-img: Toggling extended L2 entries is not supported +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +qemu-img: Toggling extended L2 entries is not supported + +### Test copy-on-write on an image with snapshots ### + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 +L2 entry #0: 0x8000000000050000 0000008000042000 +L2 entry #1: 0x8000000000060000 0000008000042000 +L2 entry #2: 0x8000000000070000 0000008000042000 +L2 entry #3: 0x8000000000080000 0000008000042000 +L2 entry #4: 0x8000000000090000 0000008000042000 +L2 entry #5: 0x80000000000a0000 0000008000042000 +L2 entry #6: 0x80000000000b0000 0000008000042000 +L2 entry #7: 0x80000000000c0000 0000008000042000 +L2 entry #8: 0x80000000000d0000 0000008000042000 +L2 entry #9: 0x80000000000e0000 0000008000042000 +L2 entry #0: 0x8000000000120000 000000800007e000 +L2 entry #1: 0x8000000000130000 000000800007fc00 +L2 entry #2: 0x8000000000140000 00000080001fe000 +L2 entry #3: 0x8000000000150000 000000800007e000 +L2 entry #4: 0x8000000000160000 000000000007ff80 +L2 entry #5: 0x8000000000170000 000000000007ffff +L2 entry #6: 0x00000000000b0000 0001808000042000 +L2 entry #7: 0x00000000000c0000 0000208000040000 +L2 entry #8: 0x8000000000180000 000000800007e000 +L2 entry #9: 0x00000000000e0000 000000c000042000 +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index 445c26f8d2..50c9bbc750 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -284,6 +284,7 @@ 267 rw auto quick snapshot 268 rw auto quick 270 rw backing quick +271 rw auto 272 rw 273 backing quick 274 rw backing