From patchwork Wed Jun 10 15:02:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98EDEC433DF for ; Wed, 10 Jun 2020 15:14:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 52DD7206F4 for ; Wed, 10 Jun 2020 15:14:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="gcgq0Jqz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 52DD7206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55048 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2RO-0001Jc-JD for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:14:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36136) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gu-0002fK-JE; Wed, 10 Jun 2020 11:04:08 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58018) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006vs-5O; Wed, 10 Jun 2020 11:04:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=1k1NszbsVANaMD9ZSg9A5CB/u9n/APSlGoVn08ULCFs=; b=gcgq0JqzdnVPzmTqnFhLzIXg0yf/3uVHkN/FUVotb1lmgMsLaNrVaObSPOCV/BXhce6YU7EetPnG9E3pXu3i3OPmiWv5o1qB6JpBk9HgI+9LforcEsGl4JuEnoF0/c/rJFbdn/DBWJzUe7NVDY0LbLM+R3aRUuc7jKnbwOww1qNZBA4I8TrXuFqFiPq18R08Vid4UGYlC5gqz4meSMT/82AT00Ayri87NjE8EUI0VUxfTOiUM0xijzKbUfwoeIYH96brYhUhZHm276nvj8g0jlFb/YWufahJ3heEiDsNAwYi/DsUeiIwE7ym9Ny4in/Ia29eodPQ7I8zZt4XWQub2w==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GQ-0007gd-DL; Wed, 10 Jun 2020 17:03:38 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GB-0007MW-Mb; Wed, 10 Jun 2020 17:03:23 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 02/34] qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset() Date: Wed, 10 Jun 2020 17:02:40 +0200 Message-Id: <1853ae2f94506504044f2aebd8e042fc615f56fd.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" qcow2_get_cluster_offset() takes an (unaligned) guest offset and returns the (aligned) offset of the corresponding cluster in the qcow2 image. In practice none of the callers need to know where the cluster starts so this patch makes the function calculate and return the final host offset directly. The function is also renamed accordingly. There is a pre-existing exception with compressed clusters: in this case the function returns the complete cluster descriptor (containing the offset and size of the compressed data). This does not change with this patch but it is now documented. Signed-off-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 4 ++-- block/qcow2-cluster.c | 42 +++++++++++++++++++++++------------------- block/qcow2.c | 24 +++++++----------------- 3 files changed, 32 insertions(+), 38 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 7ce2c23bdb..06475e0849 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -694,8 +694,8 @@ int qcow2_write_l1_entry(BlockDriverState *bs, int l1_index); int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, uint8_t *buf, int nb_sectors, bool enc, Error **errp); -int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *cluster_offset); +int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, + unsigned int *bytes, uint64_t *host_offset); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 4b5fc8c4a7..9ab41cb728 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -496,10 +496,15 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, /* - * get_cluster_offset + * get_host_offset * - * For a given offset of the virtual disk, find the cluster type and offset in - * the qcow2 file. The offset is stored in *cluster_offset. + * For a given offset of the virtual disk find the equivalent host + * offset in the qcow2 file and store it in *host_offset. Neither + * offset needs to be aligned to a cluster boundary. + * + * If the cluster is unallocated then *host_offset will be 0. + * If the cluster is compressed then *host_offset will contain the + * complete compressed cluster descriptor. * * On entry, *bytes is the maximum number of contiguous bytes starting at * offset that we are interested in. @@ -511,12 +516,12 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * Returns the cluster type (QCOW2_CLUSTER_*) on success, -errno in error * cases. */ -int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *cluster_offset) +int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, + unsigned int *bytes, uint64_t *host_offset) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; - uint64_t l1_index, l2_offset, *l2_slice; + uint64_t l1_index, l2_offset, *l2_slice, l2_entry; int c; unsigned int offset_in_cluster; uint64_t bytes_available, bytes_needed, nb_clusters; @@ -537,8 +542,6 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, bytes_needed = bytes_available; } - *cluster_offset = 0; - /* seek to the l2 offset in the l1 table */ l1_index = offset_to_l1_index(s, offset); @@ -570,7 +573,7 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, /* find the cluster offset for the given disk offset */ l2_index = offset_to_l2_slice_index(s, offset); - *cluster_offset = be64_to_cpu(l2_slice[l2_index]); + l2_entry = be64_to_cpu(l2_slice[l2_index]); nb_clusters = size_to_clusters(s, bytes_needed); /* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned @@ -578,7 +581,7 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, * true */ assert(nb_clusters <= INT_MAX); - type = qcow2_get_cluster_type(bs, *cluster_offset); + type = qcow2_get_cluster_type(bs, l2_entry); if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC)) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found" @@ -599,42 +602,43 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset, } /* Compressed clusters can only be processed one by one */ c = 1; - *cluster_offset &= L2E_COMPRESSED_OFFSET_SIZE_MASK; + *host_offset = l2_entry & L2E_COMPRESSED_OFFSET_SIZE_MASK; break; case QCOW2_CLUSTER_ZERO_PLAIN: case QCOW2_CLUSTER_UNALLOCATED: /* how many empty clusters ? */ c = count_contiguous_clusters_unallocated(bs, nb_clusters, &l2_slice[l2_index], type); - *cluster_offset = 0; + *host_offset = 0; break; case QCOW2_CLUSTER_ZERO_ALLOC: - case QCOW2_CLUSTER_NORMAL: + case QCOW2_CLUSTER_NORMAL: { + uint64_t host_cluster_offset = l2_entry & L2E_OFFSET_MASK; + *host_offset = host_cluster_offset + offset_in_cluster; /* how many allocated clusters ? */ c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, &l2_slice[l2_index], QCOW_OFLAG_ZERO); - *cluster_offset &= L2E_OFFSET_MASK; - if (offset_into_cluster(s, *cluster_offset)) { + if (offset_into_cluster(s, host_cluster_offset)) { qcow2_signal_corruption(bs, true, -1, -1, "Cluster allocation offset %#" PRIx64 " unaligned (L2 offset: %#" PRIx64 - ", L2 index: %#x)", *cluster_offset, + ", L2 index: %#x)", host_cluster_offset, l2_offset, l2_index); ret = -EIO; goto fail; } - if (has_data_file(bs) && *cluster_offset != offset - offset_in_cluster) - { + if (has_data_file(bs) && *host_offset != offset) { qcow2_signal_corruption(bs, true, -1, -1, "External data file host cluster offset %#" PRIx64 " does not match guest cluster " "offset: %#" PRIx64 - ", L2 index: %#x)", *cluster_offset, + ", L2 index: %#x)", host_cluster_offset, offset - offset_in_cluster, l2_index); ret = -EIO; goto fail; } break; + } default: abort(); } diff --git a/block/qcow2.c b/block/qcow2.c index d792137af6..afb00ada42 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2026,7 +2026,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BlockDriverState **file) { BDRVQcow2State *s = bs->opaque; - uint64_t cluster_offset; + uint64_t host_offset; unsigned int bytes; int ret, status = 0; @@ -2039,7 +2039,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, } bytes = MIN(INT_MAX, count); - ret = qcow2_get_cluster_offset(bs, offset, &bytes, &cluster_offset); + ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { return ret; @@ -2049,7 +2049,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) && !s->crypto) { - *map = cluster_offset | offset_into_cluster(s, offset); + *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } @@ -2263,7 +2263,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ - uint64_t cluster_offset = 0; + uint64_t host_offset = 0; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2275,7 +2275,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, } qemu_co_mutex_lock(&s->lock); - ret = qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_offset); + ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto out; @@ -2287,15 +2287,6 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { - /* - * For compressed clusters the variable cluster_offset - * does not actually store the offset but the full - * descriptor. We need to leave it unchanged because - * that's what qcow2_co_preadv_compressed() expects. - */ - uint64_t host_offset = (ret == QCOW2_CLUSTER_COMPRESSED) ? - cluster_offset : - cluster_offset + offset_into_cluster(s, offset); if (!aio && cur_bytes != bytes) { aio = aio_task_pool_new(QCOW2_MAX_WORKERS); } @@ -3860,7 +3851,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, offset = QEMU_ALIGN_DOWN(offset, s->cluster_size); bytes = s->cluster_size; nr = s->cluster_size; - ret = qcow2_get_cluster_offset(bs, offset, &nr, &off); + ret = qcow2_get_host_offset(bs, offset, &nr, &off); if (ret != QCOW2_CLUSTER_UNALLOCATED && ret != QCOW2_CLUSTER_ZERO_PLAIN && ret != QCOW2_CLUSTER_ZERO_ALLOC) { @@ -3931,7 +3922,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; - ret = qcow2_get_cluster_offset(bs, src_offset, &cur_bytes, ©_offset); + ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, ©_offset); if (ret < 0) { goto out; } @@ -3963,7 +3954,6 @@ qcow2_co_copy_range_from(BlockDriverState *bs, case QCOW2_CLUSTER_NORMAL: child = s->data_file; - copy_offset += offset_into_cluster(s, src_offset); break; default: From patchwork Wed Jun 10 15:02:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D81CC433DF for ; Wed, 10 Jun 2020 15:13:57 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 08716206F4 for ; Wed, 10 Jun 2020 15:13:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="dwgjedS1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 08716206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51170 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2QO-0007lb-4y for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:13:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36134) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gu-0002f0-AL; Wed, 10 Jun 2020 11:04:08 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58037) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006vx-95; Wed, 10 Jun 2020 11:04:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=FDdJ3NUfcSN6UfYGz2xPvNrqQCi8ZzpI2orXDpb0voQ=; b=dwgjedS1zFaXaok6qeivU/cPEzSZAkiU0pc9pr5MoOYfsxq9DGvo4UmELmOuK0fLQl92SqX2ELja8lm58kgUCVR1NweVTMYi4tZc5U11RHVxKISCZ1NVdD4RvJTY4Mq3AUpKrWAIAqCDzKV7YIbffwf77CkVFDz3+9WyRDe6HJSUcOHeRpxl+29YChA4XHxOlXYPpmLovL6JaOkUYOAaaX3xVcd9uDx2zKM5YLEqbx2cPF+6mV0WKjO+/HlNDl+FOz+0ipE6qnm9dOw0aTavBxy1pnYfHYGmrLI09dnCFRoE5w3DGWqHVTOSLvLtXtdSOdZEbAyFEv1neM02zi017g==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gm-87; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GB-0007Mg-Rt; Wed, 10 Jun 2020 17:03:23 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 07/34] qcow2: Document the Extended L2 Entries feature Date: Wed, 10 Jun 2020 17:02:45 +0200 Message-Id: <64e1c0d0debf8f483c952ba9af9b2387667c4147.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- docs/interop/qcow2.txt | 68 ++++++++++++++++++++++++++++++++++++++++-- docs/qcow2-cache.txt | 19 +++++++++++- 2 files changed, 83 insertions(+), 4 deletions(-) diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index cb723463f2..64e9345fb4 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -42,6 +42,9 @@ The first cluster of a qcow2 image contains the file header: as the maximum cluster size and won't be able to open images with larger cluster sizes. + Note: if the image has Extended L2 Entries then cluster_bits + must be at least 14 (i.e. 16384 byte clusters). + 24 - 31: size Virtual disk size in bytes. @@ -117,7 +120,12 @@ the next fields through header_length. clusters. The compression_type field must be present and not zero. - Bits 4-63: Reserved (set to 0) + Bit 4: Extended L2 Entries. If this bit is set then + L2 table entries use an extended format that + allows subcluster-based allocation. See the + Extended L2 Entries section for more details. + + Bits 5-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can @@ -498,7 +506,7 @@ cannot be relaxed without an incompatible layout change). Given an offset into the virtual disk, the offset into the image file can be obtained as follows: - l2_entries = (cluster_size / sizeof(uint64_t)) + l2_entries = (cluster_size / sizeof(uint64_t)) [*] l2_index = (offset / cluster_size) % l2_entries l1_index = (offset / cluster_size) / l2_entries @@ -508,6 +516,8 @@ obtained as follows: return cluster_offset + (offset % cluster_size) + [*] this changes if Extended L2 Entries are enabled, see next section + L1 table entry: Bit 0 - 8: Reserved (set to 0) @@ -548,7 +558,8 @@ Standard Cluster Descriptor: nor is data read from the backing file if the cluster is unallocated. - With version 2, this is always 0. + With version 2 or with extended L2 entries (see the next + section), this is always 0. 1 - 8: Reserved (set to 0) @@ -585,6 +596,57 @@ file (except if bit 0 in the Standard Cluster Descriptor is set). If there is no backing file or the backing file is smaller than the image, they shall read zeros for all parts that are not covered by the backing file. +== Extended L2 Entries == + +An image uses Extended L2 Entries if bit 4 is set on the incompatible_features +field of the header. + +In these images standard data clusters are divided into 32 subclusters of the +same size. They are contiguous and start from the beginning of the cluster. +Subclusters can be allocated independently and the L2 entry contains information +indicating the status of each one of them. Compressed data clusters don't have +subclusters so they are treated the same as in images without this feature. + +The size of an extended L2 entry is 128 bits so the number of entries per table +is calculated using this formula: + + l2_entries = (cluster_size / (2 * sizeof(uint64_t))) + +The first 64 bits have the same format as the standard L2 table entry described +in the previous section, with the exception of bit 0 of the standard cluster +descriptor. + +The last 64 bits contain a subcluster allocation bitmap with this format: + +Subcluster Allocation Bitmap (for standard clusters): + + Bit 0 - 31: Allocation status (one bit per subcluster) + + 1: the subcluster is allocated. In this case the + host cluster offset field must contain a valid + offset. + 0: the subcluster is not allocated. In this case + read requests shall go to the backing file or + return zeros if there is no backing file data. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x). + + 32 - 63 Subcluster reads as zeros (one bit per subcluster) + + 1: the subcluster reads as zeros. In this case the + allocation status bit must be unset. The host + cluster offset field may or may not be set. + 0: no effect. + + Bits are assigned starting from the least significant + one (i.e. bit x is used for subcluster x - 32). + +Subcluster Allocation Bitmap (for compressed clusters): + + Bit 0 - 63: Reserved (set to 0) + Compressed clusters don't have subclusters, + so this field is not used. == Snapshots == diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt index d57f409861..5f763aa6bb 100644 --- a/docs/qcow2-cache.txt +++ b/docs/qcow2-cache.txt @@ -1,6 +1,6 @@ qcow2 L2/refcount cache configuration ===================================== -Copyright (C) 2015, 2018 Igalia, S.L. +Copyright (C) 2015, 2018-2020 Igalia, S.L. Author: Alberto Garcia This work is licensed under the terms of the GNU GPL, version 2 or @@ -222,3 +222,20 @@ support this functionality, and is 0 (disabled) on other platforms. This functionality currently relies on the MADV_DONTNEED argument for madvise() to actually free the memory. This is a Linux-specific feature, so cache-clean-interval is not supported on other systems. + + +Extended L2 Entries +------------------- +All numbers shown in this document are valid for qcow2 images with normal +64-bit L2 entries. + +Images with extended L2 entries need twice as much L2 metadata, so the L2 +cache size must be twice as large for the same disk space. + + disk_size = l2_cache_size * cluster_size / 16 + +i.e. + + l2_cache_size = disk_size * 16 / cluster_size + +Refcount blocks are not affected by this. From patchwork Wed Jun 10 15:02:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280854 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30ECC433E0 for ; Wed, 10 Jun 2020 15:18:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA376206F4 for ; Wed, 10 Jun 2020 15:18:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="aGLP12Qr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA376206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2UO-0005wA-UW for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:18:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gv-0002g0-9q; Wed, 10 Jun 2020 11:04:09 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006vy-AB; Wed, 10 Jun 2020 11:04:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=FLnW2yKPd876mhIBSmNOIZZESH7oUMAItbps7FGNKa4=; b=aGLP12Qr5492cCrhDe3LfjNDycEKGUjWlehspliefxHRaTEwUFdvHW4Fo0NRUcwjaSweMQO81zliNJssnmIhRxOgswfVdN2uZ3Y4NEh2Tuw/Zr+cUGliqiHt9VdnEwJ1ny/YsMuSStnEZ2NyyFGJTZKGqxQjVfLju0qQ9NYawM36IIxCkY5W33lH9lhg91n21ZtFntkZASGKZimjNNQsOuosb84IIUEiT9IRek7S4Yv0bzYKpT4EgUTtMlsY5i7oEWD0v9uTRvnPjHElRxWZtGRObNrhNGWC41dIvnSzBzECM6WQmHEoXx9Nm2e6XgWmpVVQVvAjDHptt3u78/BlPA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gl-Cb; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GB-0007Mi-Ss; Wed, 10 Jun 2020 17:03:23 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 08/34] qcow2: Add dummy has_subclusters() function Date: Wed, 10 Jun 2020 17:02:46 +0200 Message-Id: <773d28d4d2ff8c0cf3d724c53a06269b8d106766.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function will be used by the qcow2 code to check if an image has subclusters or not. At the moment this simply returns false. Once all patches needed for subcluster support are ready then QEMU will be able to create and read images with subclusters and this function will return the actual value. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index eecbadc4cb..2064dd3d85 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -510,6 +510,12 @@ typedef enum QCow2MetadataOverlap { #define INV_OFFSET (-1ULL) +static inline bool has_subclusters(BDRVQcow2State *s) +{ + /* FIXME: Return false until this feature is complete */ + return false; +} + static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx) { From patchwork Wed Jun 10 15:02:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280852 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D0B7C433E0 for ; Wed, 10 Jun 2020 15:21:01 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 31B6F20734 for ; Wed, 10 Jun 2020 15:21:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="HHEFoiDu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31B6F20734 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2XE-00018L-E6 for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:21:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36146) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gw-0002hk-Br; Wed, 10 Jun 2020 11:04:10 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58032) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006vu-6X; Wed, 10 Jun 2020 11:04:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=zbMhjtGKVeY7EryanAE+iwZTdNLcuFqDn9UZxBryOf4=; b=HHEFoiDueQ/iHJeSFStNw/FYSRQtYb8ED3nAvjXleSyW/2JK3B3Scyh1FRXwhC7yBVnuLdV+rzMWvLivTbwbCe0pc9AcRBMimUxW2hIH0zuQC2T1QSANhi6caDQf8uiY9szzMlET/ErWXxv0RH1EgFC3gXjb+ocV2OqbBQD8zIfFoeLbZNOoOSrr3zPpB8G3uIWUYzfWMpsVm83BI/TTtQz/ddpbpKhxtks9fbeCKxWrkVp+tVeMYdZ97B8qiJoGKM+foygzsUQDlUGQQn/3i9jnx04TP3lQQmHFM16B8NEQtDeUdWa2G+UUX/Q6gXvVUHV5c6F8kUm3kukEwQl+wA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gn-3j; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GB-0007Mm-Tv; Wed, 10 Jun 2020 17:03:23 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 09/34] qcow2: Add subcluster-related fields to BDRVQcow2State Date: Wed, 10 Jun 2020 17:02:47 +0200 Message-Id: <17dc6f6519e90b2351565dc2f807f8cddc2aaa90.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This patch adds the following new fields to BDRVQcow2State: - subclusters_per_cluster: Number of subclusters in a cluster - subcluster_size: The size of each subcluster, in bytes - subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size Images without subclusters are treated as if they had exactly one subcluster per cluster (i.e. subcluster_size = cluster_size). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 5 +++++ block/qcow2.c | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 2064dd3d85..eee4c8de9c 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -78,6 +78,8 @@ /* The cluster reads as all zeros */ #define QCOW_OFLAG_ZERO (1ULL << 0) +#define QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER 32 + #define MIN_CLUSTER_BITS 9 #define MAX_CLUSTER_BITS 21 @@ -295,6 +297,9 @@ typedef struct BDRVQcow2State { int cluster_bits; int cluster_size; int l2_slice_size; + int subcluster_bits; + int subcluster_size; + int subclusters_per_cluster; int l2_bits; int l2_size; int l1_size; diff --git a/block/qcow2.c b/block/qcow2.c index afb00ada42..5c175a314c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1433,6 +1433,11 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, } } + s->subclusters_per_cluster = + has_subclusters(s) ? QCOW_EXTL2_SUBCLUSTERS_PER_CLUSTER : 1; + s->subcluster_size = s->cluster_size / s->subclusters_per_cluster; + s->subcluster_bits = ctz32(s->subcluster_size); + /* Check support for various header values */ if (header.refcount_order > 6) { error_setg(errp, "Reference count entry width too large; may not " From patchwork Wed Jun 10 15:02:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2583EC433DF for ; Wed, 10 Jun 2020 15:11:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5233206F4 for ; Wed, 10 Jun 2020 15:11:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="O0zQyyaw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5233206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2Nv-0003tA-3Q for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:11:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36062) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0002dt-U0; Wed, 10 Jun 2020 11:04:07 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58019) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006vr-5Z; Wed, 10 Jun 2020 11:04:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=DWFzz4T+AhoA6h3Vs/ROK38xw63e6XfJbmyzdTE6EKY=; b=O0zQyyawnXvo1D8s5tpWg27zkTDAuctzABLps5Dis7WRMH1Fahq5VMilW9sJbjA435BtBls6v5EY2h5WcpTlCshNhtdrrVeRy18CYu1LtRePG6lLsjaoqy+3rjPYOQA9lDT/BmA84rOgXgJJXBLNoMsQ6ofj/jd2qmyfsRmvt8AwKdnff/pKhN/CAVG0wGXx0oaz6hV8eWiJ421gn5JTgdj7NimAy1QCdPkfTbK6I44Dov8YRKw4zWcwn3+u6HA6SBEObdc5TWWXBrxCf8xIPf2zjAgrRm3jiuFKk9BmPK4sWgx0BxuZE4qF/avCbcC5RAnJTEp1WmKEm1YuWHfzaw==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007go-5G; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GB-0007Mp-Uz; Wed, 10 Jun 2020 17:03:23 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 10/34] qcow2: Add offset_to_sc_index() Date: Wed, 10 Jun 2020 17:02:48 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" For a given offset, return the subcluster number within its cluster (i.e. with 32 subclusters per cluster it returns a number between 0 and 31). Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index eee4c8de9c..2503374677 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -581,6 +581,11 @@ static inline int offset_to_l2_slice_index(BDRVQcow2State *s, int64_t offset) return (offset >> s->cluster_bits) & (s->l2_slice_size - 1); } +static inline int offset_to_sc_index(BDRVQcow2State *s, int64_t offset) +{ + return (offset >> s->subcluster_bits) & (s->subclusters_per_cluster - 1); +} + static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s) { return (int64_t)s->l1_vm_state_index << (s->cluster_bits + s->l2_bits); From patchwork Wed Jun 10 15:02:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8908CC433DF for ; Wed, 10 Jun 2020 15:06:35 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 54347206F4 for ; Wed, 10 Jun 2020 15:06:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="hf0hgFPg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54347206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:52538 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2JG-0004lW-Ag for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:06:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2GZ-0002Gs-3X; Wed, 10 Jun 2020 11:03:47 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58076) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2GW-0006w1-SO; Wed, 10 Jun 2020 11:03:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=3TcxrFGVkCtUtsIcvfPDHVNJ3SGTTueaU2gMyJDgFo0=; b=hf0hgFPg4+cvB5RAm2eQIiOk7f/jreIuLu/HovOBI86dpmPewieRVJ+BNQzp1SSjqJFOMjci+xLXjfenCWhPJkeZ6ESFvKgCYyfTdz4Fbuy9rrASdAOXlsrFBZwkcUFP1adNvTuyio7S/soknyMZ7mN1FN2RbGFBlYvbeUm/UtamPeZRZ2GIW+KXLISjjXnXZEZqUZ/m+GxFIQzM77qrZy4XEF3wGKEjvTAP2xq8yEPGkxGsIolHH6YogQINeLjwmQ4VoGQosq4Wz37EkGAEQjedP6x+wGDqN/m2MDzg/FDE4e2Bq0h2ZK/GkiIhusiMKEXu8wIQ7+YJ2QZwUppA4Q==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gt-CR; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007My-1f; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 13/34] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap() Date: Wed, 10 Jun 2020 17:02:51 +0200 Message-Id: <5373a508b5a8b9a961d1aa79833de988dc370666.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Extended L2 entries are 128-bit wide: 64 bits for the entry itself and 64 bits for the subcluster allocation bitmap. In order to support them correctly get/set_l2_entry() need to be updated so they take the entry width into account in order to calculate the correct offset. This patch also adds the get/set_l2_bitmap() functions that are used to access the bitmaps. For convenience we allow calling get_l2_bitmap() on images without subclusters. In this case the returned value is always 0 and has no meaning. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 46b351229a..82b86f6cec 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -533,15 +533,36 @@ static inline size_t l2_entry_size(BDRVQcow2State *s) static inline uint64_t get_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx) { + idx *= l2_entry_size(s) / sizeof(uint64_t); return be64_to_cpu(l2_slice[idx]); } +static inline uint64_t get_l2_bitmap(BDRVQcow2State *s, uint64_t *l2_slice, + int idx) +{ + if (has_subclusters(s)) { + idx *= l2_entry_size(s) / sizeof(uint64_t); + return be64_to_cpu(l2_slice[idx + 1]); + } else { + return 0; /* For convenience only; this value has no meaning. */ + } +} + static inline void set_l2_entry(BDRVQcow2State *s, uint64_t *l2_slice, int idx, uint64_t entry) { + idx *= l2_entry_size(s) / sizeof(uint64_t); l2_slice[idx] = cpu_to_be64(entry); } +static inline void set_l2_bitmap(BDRVQcow2State *s, uint64_t *l2_slice, + int idx, uint64_t bitmap) +{ + assert(has_subclusters(s)); + idx *= l2_entry_size(s) / sizeof(uint64_t); + l2_slice[idx + 1] = cpu_to_be64(bitmap); +} + static inline bool has_data_file(BlockDriverState *bs) { BDRVQcow2State *s = bs->opaque; From patchwork Wed Jun 10 15:02:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F8B3C433DF for ; Wed, 10 Jun 2020 15:06:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5E650206F4 for ; Wed, 10 Jun 2020 15:06:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="jL2wnb5M" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E650206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53170 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2JQ-00051T-H5 for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:06:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35962) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gb-0002J6-Th; Wed, 10 Jun 2020 11:03:49 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58071) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2GX-0006vz-C7; Wed, 10 Jun 2020 11:03:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=VTJ0AleYo0FtKqUbQGMQJVxhmwOgUpH8CMhdCjrgWgs=; b=jL2wnb5Mdn4V8Lplb5aCBWpZiAQgITHO91qcKVX705DrdvSUAhhKspvDEQRmdLblkj06YYO+nfZtGoAcUc3qRfkCPTShzHQV+OM2gtSeSC+YXtZC4rgUnSqpeXpqxXJeoxz5F1VItEgc3rd30kQjTccn57NpYDEW3nX5pYIoRsNTdKmmYmeWQJHht6fgsKNkdhvJKNdqzlOlngW73QjxQf+5+EDepzqkYVTpNsI0szi/in3UtVQxejg+4EnSIECwD9FaMmPX1DmWzEZ9cUt0MkZ8XAe9i+Slf01I4Ag6NDjFCAU5OoaB+dmebBvaksgk+X5Q7z/XwHDM6csjv8woeA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gv-CG; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007N4-3H; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 15/34] qcow2: Add qcow2_get_subcluster_range_type() Date: Wed, 10 Jun 2020 17:02:53 +0200 Message-Id: <812b66824859bee38019fc4d35bda0985b13789f.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" There are situations in which we want to know how many contiguous subclusters of the same type there are in a given cluster. This can be done by simply iterating over the subclusters and repeatedly calling qcow2_get_subcluster_type() for each one of them. However once we determined the type of a subcluster we can check the rest efficiently by counting the number of adjacent ones (or zeroes) in the bitmap. This is what this function does. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 51 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 8b2fc550b7..32dc6e75e3 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -375,6 +375,57 @@ fail: return ret; } +/* + * For a given L2 entry, count the number of contiguous subclusters of + * the same type starting from @sc_from. Compressed clusters are + * treated as if they were divided into subclusters of size + * s->subcluster_size. + * + * Return the number of contiguous subclusters and set @type to the + * subcluster type. + * + * If the L2 entry is invalid return -errno and set @type to + * QCOW2_SUBCLUSTER_INVALID. + */ +G_GNUC_UNUSED +static int qcow2_get_subcluster_range_type(BlockDriverState *bs, + uint64_t l2_entry, + uint64_t l2_bitmap, + unsigned sc_from, + QCow2SubclusterType *type) +{ + BDRVQcow2State *s = bs->opaque; + uint32_t val; + + *type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_from); + + if (*type == QCOW2_SUBCLUSTER_INVALID) { + return -EINVAL; + } else if (!has_subclusters(s) || *type == QCOW2_SUBCLUSTER_COMPRESSED) { + return s->subclusters_per_cluster - sc_from; + } + + switch (*type) { + case QCOW2_SUBCLUSTER_NORMAL: + val = l2_bitmap | QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + val = (l2_bitmap | QCOW_OFLAG_SUB_ZERO_RANGE(0, sc_from)) >> 32; + return cto32(val) - sc_from; + + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: + val = ((l2_bitmap >> 32) | l2_bitmap) + & ~QCOW_OFLAG_SUB_ALLOC_RANGE(0, sc_from); + return ctz32(val) - sc_from; + + default: + g_assert_not_reached(); + } +} + /* * Checks how many clusters in a given L2 slice are contiguous in the image * file. As soon as one of the flags in the bitmask stop_flags changes compared From patchwork Wed Jun 10 15:02:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54B24C433DF for ; Wed, 10 Jun 2020 15:09:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2034C207C3 for ; Wed, 10 Jun 2020 15:09:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="S+NjU1Y8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2034C207C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:35606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2Lh-0000wy-9r for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:09:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36138) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gu-0002fW-Ph; Wed, 10 Jun 2020 11:04:08 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58074) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006w0-Ep; Wed, 10 Jun 2020 11:04:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=PelaHdUpXyQLyETSNjZxQedA2munXec4UfgCc0HKoeA=; b=S+NjU1Y8CDLgM9AQC08vuWqBZ44iwVtWa19KnJIw2cPrwEaGyBQKLPQtHQkrXwnEyr51WL2eLu1FtRv3XmCuNEtqwgWMEj/MsGvg8W9Wv8GxvEEZOY+k7Tt21JbrznzVDqPNHuV53G3IgROjeuTkuIR38LFDnKa9pB2YHV5xwJuTWiTfkjKA9lR1SNJeWWEZHdI84+3I/iw9MCPzoKlug51CaHtbuiEtgblCkLcvs2fXRoe50yp4cQpY4XXpGWJ5LZPKFdP2uiQZ9H3kClBwnlKHWhQKu00BNJLpzq0oEMCgXKs2OLN3/GC6zWt+S0alvXxgwFAq0gLogjJL20IsOg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gw-DE; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007N7-4D; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 16/34] qcow2: Add qcow2_cluster_is_allocated() Date: Wed, 10 Jun 2020 17:02:54 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This helper function tells us if a cluster is allocated (that is, there is an associated host offset for it). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index 3aec6f452a..ea647c8bb5 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -780,6 +780,12 @@ QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs, } } +static inline bool qcow2_cluster_is_allocated(QCow2ClusterType type) +{ + return (type == QCOW2_CLUSTER_COMPRESSED || type == QCOW2_CLUSTER_NORMAL || + type == QCOW2_CLUSTER_ZERO_ALLOC); +} + /* Check whether refcounts are eager or lazy */ static inline bool qcow2_need_accurate_refcounts(BDRVQcow2State *s) { From patchwork Wed Jun 10 15:02:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0956C433DF for ; Wed, 10 Jun 2020 15:19:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 97319206F4 for ; Wed, 10 Jun 2020 15:19:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="ZxmEfkVZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97319206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2W5-0008Ag-OK for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:19:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36150) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gw-0002im-Og; Wed, 10 Jun 2020 11:04:10 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58247) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006wb-DC; Wed, 10 Jun 2020 11:04:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=ctmJoE6t9F+l9JkkxXkf9Wgs/rm6W2SVqo/5HAl3/Ag=; b=ZxmEfkVZa/9wPm2ufNo1ntgb17jQRxErJp7/ssod4vTAS7m1ybqOddDqvPEI4O1xl2HjEJd8wGRKUbgcuUN8s+g7LlrLB0ovz248D7V5ie6oGiHMKV2etkWjSr2jj3Sk7wyUedkUcaZbmInVXAgC5kQt78Yoei50Hxoj4x9p1TMpVv2gTGPhOltgTwtk+k2QO9MITXscCGCv9K7+JozGSt4Lui09kGrBx/UBJjzdRkH3MAB5EuRISnZLSRMGL/551I7NPXD59fUszRt8DVBi/VGfgitCw0KW/xpvbspfJVrP3O2QHf5TSkQ2wYEQLjgAiJSvzRNHfP42UNz5iyQV7g==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gx-NA; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007NA-5T; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 17/34] qcow2: Add cluster type parameter to qcow2_get_host_offset() Date: Wed, 10 Jun 2020 17:02:55 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function returns an integer that can be either an error code or a cluster type (a value from the QCow2ClusterType enum). We are going to start using subcluster types instead of cluster types in some functions so it's better to use the exact data types instead of integers for clarity and in order to detect errors more easily. This patch makes qcow2_get_host_offset() return 0 on success and puts the returned cluster type in a separate parameter. There are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 ++- block/qcow2-cluster.c | 11 +++++++---- block/qcow2.c | 37 ++++++++++++++++++++++--------------- 3 files changed, 31 insertions(+), 20 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index ea647c8bb5..74f65793bd 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -893,7 +893,8 @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, uint8_t *buf, int nb_sectors, bool enc, Error **errp); int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset); + unsigned int *bytes, uint64_t *host_offset, + QCow2ClusterType *cluster_type); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 32dc6e75e3..1e5681b0c6 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -565,13 +565,14 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * * On exit, *bytes is the number of bytes starting at offset that have the same * cluster type and (if applicable) are stored contiguously in the image file. + * The cluster type is stored in *cluster_type. * Compressed clusters are always returned one by one. * - * Returns the cluster type (QCOW2_CLUSTER_*) on success, -errno in error - * cases. + * Returns 0 on success, -errno in error cases. */ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, - unsigned int *bytes, uint64_t *host_offset) + unsigned int *bytes, uint64_t *host_offset, + QCow2ClusterType *cluster_type) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; @@ -712,7 +713,9 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - return type; + *cluster_type = type; + + return 0; fail: qcow2_cache_put(s->l2_table_cache, (void **)&l2_slice); diff --git a/block/qcow2.c b/block/qcow2.c index 7df55a88a8..89e17bdaba 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2033,6 +2033,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; uint64_t host_offset; unsigned int bytes; + QCow2ClusterType type; int ret, status = 0; qemu_co_mutex_lock(&s->lock); @@ -2044,7 +2045,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, } bytes = MIN(INT_MAX, count); - ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset); + ret = qcow2_get_host_offset(bs, offset, &bytes, &host_offset, &type); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { return ret; @@ -2052,15 +2053,15 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if ((ret == QCOW2_CLUSTER_NORMAL || ret == QCOW2_CLUSTER_ZERO_ALLOC) && + if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } - if (ret == QCOW2_CLUSTER_ZERO_PLAIN || ret == QCOW2_CLUSTER_ZERO_ALLOC) { + if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (ret != QCOW2_CLUSTER_UNALLOCATED) { + } else if (type != QCOW2_CLUSTER_UNALLOCATED) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2269,6 +2270,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t host_offset = 0; + QCow2ClusterType type; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2280,22 +2282,23 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, } qemu_co_mutex_lock(&s->lock); - ret = qcow2_get_host_offset(bs, offset, &cur_bytes, &host_offset); + ret = qcow2_get_host_offset(bs, offset, &cur_bytes, + &host_offset, &type); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { goto out; } - if (ret == QCOW2_CLUSTER_ZERO_PLAIN || - ret == QCOW2_CLUSTER_ZERO_ALLOC || - (ret == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + if (type == QCOW2_CLUSTER_ZERO_PLAIN || + type == QCOW2_CLUSTER_ZERO_ALLOC || + (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { if (!aio && cur_bytes != bytes) { aio = aio_task_pool_new(QCOW2_MAX_WORKERS); } - ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret, + ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, type, host_offset, offset, cur_bytes, qiov, qiov_offset, NULL); if (ret < 0) { @@ -3841,6 +3844,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, if (head || tail) { uint64_t off; unsigned int nr; + QCow2ClusterType type; assert(head + bytes <= s->cluster_size); @@ -3856,10 +3860,11 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, offset = QEMU_ALIGN_DOWN(offset, s->cluster_size); bytes = s->cluster_size; nr = s->cluster_size; - ret = qcow2_get_host_offset(bs, offset, &nr, &off); - if (ret != QCOW2_CLUSTER_UNALLOCATED && - ret != QCOW2_CLUSTER_ZERO_PLAIN && - ret != QCOW2_CLUSTER_ZERO_ALLOC) { + ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); + if (ret < 0 || + (type != QCOW2_CLUSTER_UNALLOCATED && + type != QCOW2_CLUSTER_ZERO_PLAIN && + type != QCOW2_CLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); return -ENOTSUP; } @@ -3923,16 +3928,18 @@ qcow2_co_copy_range_from(BlockDriverState *bs, while (bytes != 0) { uint64_t copy_offset = 0; + QCow2ClusterType type; /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; - ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, ©_offset); + ret = qcow2_get_host_offset(bs, src_offset, &cur_bytes, + ©_offset, &type); if (ret < 0) { goto out; } - switch (ret) { + switch (type) { case QCOW2_CLUSTER_UNALLOCATED: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); From patchwork Wed Jun 10 15:02:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF02FC433E0 for ; Wed, 10 Jun 2020 15:26:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6ECC12063A for ; Wed, 10 Jun 2020 15:26:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="n8+fyNvx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6ECC12063A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33598 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2c6-0000wm-Jd for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:26:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36200) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2H6-0002qA-7A; Wed, 10 Jun 2020 11:04:20 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58245) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006wY-O5; Wed, 10 Jun 2020 11:04:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=eX0gs7cy4Um4G492fOBEncGdlWNFQIufd+lO6E+xWIQ=; b=n8+fyNvxi61ncnXXMB0VUdo3c3Pj2aOnJSljZlER4cJeGARwgMsasx2UAjvCGbMRh/mAx73mGjS2zU06NTLCC15w9owLx27Yuu2Qxx+6JLwKnPRPraJFjo+45tc8HduIB6ejniPUQp9fQexCgnGbPyGs8qogCUqB6VR423TkPSP/OlL7+/IKxrGkhkEM+0Br1Sn5lcEwqRDimvhseGtJJBHV9vEL847TjVt0PlmbBMvkVng8GlpaIjjkb2wBILu7YWsTL7349Wix4ZIEWMbE2y8t+jK9WoJX4p5v1xn4SdDBg6vjraLs/rwP9cZCsbmiUc0S8WP0IHUS+7Zzy2GrlA==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007gy-Na; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007ND-6Y; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 18/34] qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_* Date: Wed, 10 Jun 2020 17:02:56 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" In order to support extended L2 entries some functions of the qcow2 driver need to start dealing with subclusters instead of clusters. qcow2_get_host_offset() is modified to return the subcluster type instead of the cluster type, and all callers are updated to replace all values of QCow2ClusterType with their QCow2SubclusterType equivalents. This patch only changes the data types, there are no semantic changes. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 2 +- block/qcow2-cluster.c | 10 +++---- block/qcow2.c | 70 ++++++++++++++++++++++--------------------- 3 files changed, 42 insertions(+), 40 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 74f65793bd..5df761edc3 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -894,7 +894,7 @@ int qcow2_encrypt_sectors(BDRVQcow2State *s, int64_t sector_num, int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type); + QCow2SubclusterType *subcluster_type); int qcow2_alloc_cluster_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, QCowL2Meta **m); diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 1e5681b0c6..ed7b92dbb2 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -564,15 +564,15 @@ static int coroutine_fn do_perform_cow_write(BlockDriverState *bs, * offset that we are interested in. * * On exit, *bytes is the number of bytes starting at offset that have the same - * cluster type and (if applicable) are stored contiguously in the image file. - * The cluster type is stored in *cluster_type. - * Compressed clusters are always returned one by one. + * subcluster type and (if applicable) are stored contiguously in the image + * file. The subcluster type is stored in *subcluster_type. + * Compressed clusters are always processed one by one. * * Returns 0 on success, -errno in error cases. */ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, unsigned int *bytes, uint64_t *host_offset, - QCow2ClusterType *cluster_type) + QCow2SubclusterType *subcluster_type) { BDRVQcow2State *s = bs->opaque; unsigned int l2_index; @@ -713,7 +713,7 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - *cluster_type = type; + *subcluster_type = qcow2_cluster_to_subcluster_type(type); return 0; diff --git a/block/qcow2.c b/block/qcow2.c index 89e17bdaba..2f7a2a7c7a 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2033,7 +2033,7 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, BDRVQcow2State *s = bs->opaque; uint64_t host_offset; unsigned int bytes; - QCow2ClusterType type; + QCow2SubclusterType type; int ret, status = 0; qemu_co_mutex_lock(&s->lock); @@ -2053,15 +2053,16 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; - if ((type == QCOW2_CLUSTER_NORMAL || type == QCOW2_CLUSTER_ZERO_ALLOC) && - !s->crypto) { + if ((type == QCOW2_SUBCLUSTER_NORMAL || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) && !s->crypto) { *map = host_offset; *file = s->data_file->bs; status |= BDRV_BLOCK_OFFSET_VALID; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || type == QCOW2_CLUSTER_ZERO_ALLOC) { + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC) { status |= BDRV_BLOCK_ZERO; - } else if (type != QCOW2_CLUSTER_UNALLOCATED) { + } else if (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN) { status |= BDRV_BLOCK_DATA; } if (s->metadata_preallocation && (status & BDRV_BLOCK_DATA) && @@ -2158,7 +2159,7 @@ typedef struct Qcow2AioTask { AioTask task; BlockDriverState *bs; - QCow2ClusterType cluster_type; /* only for read */ + QCow2SubclusterType subcluster_type; /* only for read */ uint64_t host_offset; /* or full descriptor in compressed clusters */ uint64_t offset; uint64_t bytes; @@ -2171,7 +2172,7 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); static coroutine_fn int qcow2_add_task(BlockDriverState *bs, AioTaskPool *pool, AioTaskFunc func, - QCow2ClusterType cluster_type, + QCow2SubclusterType subcluster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, @@ -2185,7 +2186,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, *task = (Qcow2AioTask) { .task.func = func, .bs = bs, - .cluster_type = cluster_type, + .subcluster_type = subcluster_type, .qiov = qiov, .host_offset = host_offset, .offset = offset, @@ -2196,7 +2197,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, func == qcow2_co_preadv_task_entry ? "read" : "write", - cluster_type, host_offset, offset, bytes, + subcluster_type, host_offset, offset, bytes, qiov, qiov_offset); if (!pool) { @@ -2209,7 +2210,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs, } static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, - QCow2ClusterType cluster_type, + QCow2SubclusterType subc_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, @@ -2217,24 +2218,24 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, { BDRVQcow2State *s = bs->opaque; - switch (cluster_type) { - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + switch (subc_type) { + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: /* Both zero types are handled in qcow2_co_preadv_part */ g_assert_not_reached(); - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: assert(bs->backing); /* otherwise handled in qcow2_co_preadv_part */ BLKDBG_EVENT(bs->file, BLKDBG_READ_BACKING_AIO); return bdrv_co_preadv_part(bs->backing, offset, bytes, qiov, qiov_offset, 0); - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: return qcow2_co_preadv_compressed(bs, host_offset, offset, bytes, qiov, qiov_offset); - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: if (bs->encrypted) { return qcow2_co_preadv_encrypted(bs, host_offset, offset, bytes, qiov, qiov_offset); @@ -2257,8 +2258,9 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task) assert(!t->l2meta); - return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset, - t->offset, t->bytes, t->qiov, t->qiov_offset); + return qcow2_co_preadv_task(t->bs, t->subcluster_type, + t->host_offset, t->offset, t->bytes, + t->qiov, t->qiov_offset); } static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, @@ -2270,7 +2272,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t host_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; AioTaskPool *aio = NULL; while (bytes != 0 && aio_task_pool_status(aio) == 0) { @@ -2289,9 +2291,9 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, goto out; } - if (type == QCOW2_CLUSTER_ZERO_PLAIN || - type == QCOW2_CLUSTER_ZERO_ALLOC || - (type == QCOW2_CLUSTER_UNALLOCATED && !bs->backing)) + if (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + (type == QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && !bs->backing)) { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { @@ -2524,7 +2526,7 @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type); + assert(!t->subcluster_type); return qcow2_co_pwritev_task(t->bs, t->host_offset, t->offset, t->bytes, t->qiov, t->qiov_offset, @@ -3844,7 +3846,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, if (head || tail) { uint64_t off; unsigned int nr; - QCow2ClusterType type; + QCow2SubclusterType type; assert(head + bytes <= s->cluster_size); @@ -3862,9 +3864,9 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs, nr = s->cluster_size; ret = qcow2_get_host_offset(bs, offset, &nr, &off, &type); if (ret < 0 || - (type != QCOW2_CLUSTER_UNALLOCATED && - type != QCOW2_CLUSTER_ZERO_PLAIN && - type != QCOW2_CLUSTER_ZERO_ALLOC)) { + (type != QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_PLAIN && + type != QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qemu_co_mutex_unlock(&s->lock); return -ENOTSUP; } @@ -3928,7 +3930,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, while (bytes != 0) { uint64_t copy_offset = 0; - QCow2ClusterType type; + QCow2SubclusterType type; /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); cur_write_flags = write_flags; @@ -3940,7 +3942,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } switch (type) { - case QCOW2_CLUSTER_UNALLOCATED: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: if (bs->backing && bs->backing->bs) { int64_t backing_length = bdrv_getlength(bs->backing->bs); if (src_offset >= backing_length) { @@ -3955,16 +3957,16 @@ qcow2_co_copy_range_from(BlockDriverState *bs, } break; - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_ZERO_ALLOC: cur_write_flags |= BDRV_REQ_ZERO_WRITE; break; - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_COMPRESSED: ret = -ENOTSUP; goto out; - case QCOW2_CLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_NORMAL: child = s->data_file; break; @@ -4493,7 +4495,7 @@ static coroutine_fn int qcow2_co_pwritev_compressed_task_entry(AioTask *task) { Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); - assert(!t->cluster_type && !t->l2meta); + assert(!t->subcluster_type && !t->l2meta); return qcow2_co_pwritev_compressed_task(t->bs, t->offset, t->bytes, t->qiov, t->qiov_offset); From patchwork Wed Jun 10 15:02:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0266BC433DF for ; Wed, 10 Jun 2020 15:28:15 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B26DE2063A for ; Wed, 10 Jun 2020 15:28:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="dXV4Stkl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B26DE2063A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2eD-0004Vs-UU for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:28:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2H4-0002ot-6D; Wed, 10 Jun 2020 11:04:18 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58237) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006wV-LU; Wed, 10 Jun 2020 11:04:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=HX/BNY9+fnF3/IdpMYeBixighVMLrzZ1bFMk4+G70IU=; b=dXV4Stkl/KnM30XvDXpUp6kg1kb89n4EaV7fpGcVRzqOgupcTDnLLZfMlZ9QOb6w/Evzl+z34qu8Yb1tKAnyNQDShm5dZaKamA8cBaqtCLAh4oKlPPBtuMdaxUKabj7Zkbvviyflq0NG/znvNJ0vC7rqfx8I77WF8/fYaje2old+2ibiOTBrBuq9/qMy9aeqv2RLledLrMgXGVBcs99SLZkQhXM5jGYrBkPVrnscwuIy1qcoR+pvuHaUlesNfAl5MZnKM4Nbxak65s36BhW1RMVzxY3X1vKXFSZm7pmNSYMbcW0ZiM1st7XBj7H6r3owybR+4+WzZk2kiEJ/L6j4Tg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007h1-MX; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007NN-9Q; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 21/34] qcow2: Add subcluster support to qcow2_get_host_offset() Date: Wed, 10 Jun 2020 17:02:59 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The logic of this function remains pretty much the same, except that it uses count_contiguous_subclusters(), which combines the logic of count_contiguous_clusters() / count_contiguous_clusters_unallocated() and checks individual subclusters. qcow2_cluster_to_subcluster_type() is not necessary as a separate function anymore so it's inlined into its caller. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2.h | 38 ++++------- block/qcow2-cluster.c | 150 ++++++++++++++++++++++-------------------- 2 files changed, 92 insertions(+), 96 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 5df761edc3..4fad40b96b 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -710,29 +710,6 @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, } } -/* - * For an image without extended L2 entries, return the - * QCow2SubclusterType equivalent of a given QCow2ClusterType. - */ -static inline -QCow2SubclusterType qcow2_cluster_to_subcluster_type(QCow2ClusterType type) -{ - switch (type) { - case QCOW2_CLUSTER_COMPRESSED: - return QCOW2_SUBCLUSTER_COMPRESSED; - case QCOW2_CLUSTER_ZERO_PLAIN: - return QCOW2_SUBCLUSTER_ZERO_PLAIN; - case QCOW2_CLUSTER_ZERO_ALLOC: - return QCOW2_SUBCLUSTER_ZERO_ALLOC; - case QCOW2_CLUSTER_NORMAL: - return QCOW2_SUBCLUSTER_NORMAL; - case QCOW2_CLUSTER_UNALLOCATED: - return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; - default: - g_assert_not_reached(); - } -} - /* * In an image without subsclusters @l2_bitmap is ignored and * @sc_index must be 0. @@ -776,7 +753,20 @@ QCow2SubclusterType qcow2_get_subcluster_type(BlockDriverState *bs, g_assert_not_reached(); } } else { - return qcow2_cluster_to_subcluster_type(type); + switch (type) { + case QCOW2_CLUSTER_COMPRESSED: + return QCOW2_SUBCLUSTER_COMPRESSED; + case QCOW2_CLUSTER_ZERO_PLAIN: + return QCOW2_SUBCLUSTER_ZERO_PLAIN; + case QCOW2_CLUSTER_ZERO_ALLOC: + return QCOW2_SUBCLUSTER_ZERO_ALLOC; + case QCOW2_CLUSTER_NORMAL: + return QCOW2_SUBCLUSTER_NORMAL; + case QCOW2_CLUSTER_UNALLOCATED: + return QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; + default: + g_assert_not_reached(); + } } } diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 59dd9bda29..2f3bd3a882 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -426,66 +426,66 @@ static int qcow2_get_subcluster_range_type(BlockDriverState *bs, } /* - * Checks how many clusters in a given L2 slice are contiguous in the image - * file. As soon as one of the flags in the bitmask stop_flags changes compared - * to the first cluster, the search is stopped and the cluster is not counted - * as contiguous. (This allows it, for example, to stop at the first compressed - * cluster which may require a different handling) + * Return the number of contiguous subclusters of the exact same type + * in a given L2 slice, starting from cluster @l2_index, subcluster + * @sc_index. Allocated subclusters are required to be contiguous in + * the image file. + * At most @nb_clusters are checked (note that this means clusters, + * not subclusters). + * Compressed clusters are always processed one by one but for the + * purpose of this count they are treated as if they were divided into + * subclusters of size s->subcluster_size. + * On failure return -errno and update @l2_index to point to the + * invalid entry. */ -static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters, - int cluster_size, uint64_t *l2_slice, int l2_index, uint64_t stop_flags) +static int count_contiguous_subclusters(BlockDriverState *bs, int nb_clusters, + unsigned sc_index, uint64_t *l2_slice, + unsigned *l2_index) { BDRVQcow2State *s = bs->opaque; - int i; - QCow2ClusterType first_cluster_type; - uint64_t mask = stop_flags | L2E_OFFSET_MASK | QCOW_OFLAG_COMPRESSED; - uint64_t first_entry = get_l2_entry(s, l2_slice, l2_index); - uint64_t offset = first_entry & mask; + int i, count = 0; + bool check_offset; + uint64_t expected_offset; + QCow2SubclusterType expected_type, type; - first_cluster_type = qcow2_get_cluster_type(bs, first_entry); - if (first_cluster_type == QCOW2_CLUSTER_UNALLOCATED) { - return 0; - } - - /* must be allocated */ - assert(first_cluster_type == QCOW2_CLUSTER_NORMAL || - first_cluster_type == QCOW2_CLUSTER_ZERO_ALLOC); + assert(*l2_index + nb_clusters <= s->l2_size); for (i = 0; i < nb_clusters; i++) { - uint64_t l2_entry = get_l2_entry(s, l2_slice, l2_index + i) & mask; - if (offset + (uint64_t) i * cluster_size != l2_entry) { + unsigned first_sc = (i == 0) ? sc_index : 0; + uint64_t l2_entry = get_l2_entry(s, l2_slice, *l2_index + i); + uint64_t l2_bitmap = get_l2_bitmap(s, l2_slice, *l2_index + i); + int ret = qcow2_get_subcluster_range_type(bs, l2_entry, l2_bitmap, + first_sc, &type); + if (ret < 0) { + *l2_index += i; /* Point to the invalid entry */ + return -EIO; + } + if (i == 0) { + if (type == QCOW2_SUBCLUSTER_COMPRESSED) { + /* Compressed clusters are always processed one by one */ + return ret; + } + expected_type = type; + expected_offset = l2_entry & L2E_OFFSET_MASK; + check_offset = (type == QCOW2_SUBCLUSTER_NORMAL || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC || + type == QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC); + } else if (type != expected_type) { break; + } else if (check_offset) { + expected_offset += s->cluster_size; + if (expected_offset != (l2_entry & L2E_OFFSET_MASK)) { + break; + } } - } - - return i; -} - -/* - * Checks how many consecutive unallocated clusters in a given L2 - * slice have the same cluster type. - */ -static int count_contiguous_clusters_unallocated(BlockDriverState *bs, - int nb_clusters, - uint64_t *l2_slice, - int l2_index, - QCow2ClusterType wanted_type) -{ - BDRVQcow2State *s = bs->opaque; - int i; - - assert(wanted_type == QCOW2_CLUSTER_ZERO_PLAIN || - wanted_type == QCOW2_CLUSTER_UNALLOCATED); - for (i = 0; i < nb_clusters; i++) { - uint64_t entry = get_l2_entry(s, l2_slice, l2_index + i); - QCow2ClusterType type = qcow2_get_cluster_type(bs, entry); - - if (type != wanted_type) { + count += ret; + /* Stop if there are type changes before the end of the cluster */ + if (first_sc + ret < s->subclusters_per_cluster) { break; } } - return i; + return count; } static int coroutine_fn do_perform_cow_read(BlockDriverState *bs, @@ -574,12 +574,12 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, QCow2SubclusterType *subcluster_type) { BDRVQcow2State *s = bs->opaque; - unsigned int l2_index; - uint64_t l1_index, l2_offset, *l2_slice, l2_entry; - int c; + unsigned int l2_index, sc_index; + uint64_t l1_index, l2_offset, *l2_slice, l2_entry, l2_bitmap; + int sc; unsigned int offset_in_cluster; uint64_t bytes_available, bytes_needed, nb_clusters; - QCow2ClusterType type; + QCow2SubclusterType type; int ret; offset_in_cluster = offset_into_cluster(s, offset); @@ -600,13 +600,13 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, l1_index = offset_to_l1_index(s, offset); if (l1_index >= s->l1_size) { - type = QCOW2_CLUSTER_UNALLOCATED; + type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; goto out; } l2_offset = s->l1_table[l1_index] & L1E_OFFSET_MASK; if (!l2_offset) { - type = QCOW2_CLUSTER_UNALLOCATED; + type = QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN; goto out; } @@ -627,7 +627,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, /* find the cluster offset for the given disk offset */ l2_index = offset_to_l2_slice_index(s, offset); + sc_index = offset_to_sc_index(s, offset); l2_entry = get_l2_entry(s, l2_slice, l2_index); + l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index); nb_clusters = size_to_clusters(s, bytes_needed); /* bytes_needed <= *bytes + offset_in_cluster, both of which are unsigned @@ -635,9 +637,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, * true */ assert(nb_clusters <= INT_MAX); - type = qcow2_get_cluster_type(bs, l2_entry); - if (s->qcow_version < 3 && (type == QCOW2_CLUSTER_ZERO_PLAIN || - type == QCOW2_CLUSTER_ZERO_ALLOC)) { + type = qcow2_get_subcluster_type(bs, l2_entry, l2_bitmap, sc_index); + if (s->qcow_version < 3 && (type == QCOW2_SUBCLUSTER_ZERO_PLAIN || + type == QCOW2_SUBCLUSTER_ZERO_ALLOC)) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found" " in pre-v3 image (L2 offset: %#" PRIx64 ", L2 index: %#x)", l2_offset, l2_index); @@ -645,7 +647,9 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, goto fail; } switch (type) { - case QCOW2_CLUSTER_COMPRESSED: + case QCOW2_SUBCLUSTER_INVALID: + break; /* This is handled by count_contiguous_subclusters() below */ + case QCOW2_SUBCLUSTER_COMPRESSED: if (has_data_file(bs)) { qcow2_signal_corruption(bs, true, -1, -1, "Compressed cluster " "entry found in image with external data " @@ -654,24 +658,17 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, ret = -EIO; goto fail; } - /* Compressed clusters can only be processed one by one */ - c = 1; *host_offset = l2_entry & L2E_COMPRESSED_OFFSET_SIZE_MASK; break; - case QCOW2_CLUSTER_ZERO_PLAIN: - case QCOW2_CLUSTER_UNALLOCATED: - /* how many empty clusters ? */ - c = count_contiguous_clusters_unallocated(bs, nb_clusters, - l2_slice, l2_index, type); + case QCOW2_SUBCLUSTER_ZERO_PLAIN: + case QCOW2_SUBCLUSTER_UNALLOCATED_PLAIN: *host_offset = 0; break; - case QCOW2_CLUSTER_ZERO_ALLOC: - case QCOW2_CLUSTER_NORMAL: { + case QCOW2_SUBCLUSTER_ZERO_ALLOC: + case QCOW2_SUBCLUSTER_NORMAL: + case QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC: { uint64_t host_cluster_offset = l2_entry & L2E_OFFSET_MASK; *host_offset = host_cluster_offset + offset_in_cluster; - /* how many allocated clusters ? */ - c = count_contiguous_clusters(bs, nb_clusters, s->cluster_size, - l2_slice, l2_index, QCOW_OFLAG_ZERO); if (offset_into_cluster(s, host_cluster_offset)) { qcow2_signal_corruption(bs, true, -1, -1, "Cluster allocation offset %#" @@ -697,9 +694,18 @@ int qcow2_get_host_offset(BlockDriverState *bs, uint64_t offset, abort(); } + sc = count_contiguous_subclusters(bs, nb_clusters, sc_index, + l2_slice, &l2_index); + if (sc < 0) { + qcow2_signal_corruption(bs, true, -1, -1, "Invalid cluster entry found " + " (L2 offset: %#" PRIx64 ", L2 index: %#x)", + l2_offset, l2_index); + ret = -EIO; + goto fail; + } qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); - bytes_available = (int64_t)c * s->cluster_size; + bytes_available = ((int64_t)sc + sc_index) << s->subcluster_bits; out: if (bytes_available > bytes_needed) { @@ -712,7 +718,7 @@ out: assert(bytes_available - offset_in_cluster <= UINT_MAX); *bytes = bytes_available - offset_in_cluster; - *subcluster_type = qcow2_cluster_to_subcluster_type(type); + *subcluster_type = type; return 0; From patchwork Wed Jun 10 15:03:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6CECC433E0 for ; Wed, 10 Jun 2020 15:17:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC60B206F4 for ; Wed, 10 Jun 2020 15:17:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Gr5+q/YR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC60B206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:32998 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2TT-0004MA-U5 for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:17:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36142) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gv-0002gd-IN; Wed, 10 Jun 2020 11:04:09 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58130) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006wD-FP; Wed, 10 Jun 2020 11:04:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=wwP+F61UHuBD2MYWJK+ZYjI142FX+Mf725oBfkWGky0=; b=Gr5+q/YR5Qtn1aQNXbzPMc0bF3iA+k7xsybnQlVTWyPRyKfl8SY1hPIq4kKiNGyDvSuS3ux429tlYtOKYjoQjQsCACdiyZ1dBr6PzSeRcRsQ/92Ng8BNvw65YJgK/xP8kP2BQJ69Yvuwsp5WyVmJXsqma4iopB82WBm95fS9xixvm6l2FvbvZ5dVDTyqN7Xh3OYZdrCiZNViWg2bfLIBndauZMi8chFCi7PUQC5RZju4FnYB1a+rv8ohOgLUVm0A72jiyJzP96/f9cqulzwy+LHedRcxgryNgeDJtA2POUQxurESpvUVtEHBgEllbkjl0Sq2iC9YdqN+/X2VuB/uug==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007h7-Jp; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007NT-BL; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 23/34] qcow2: Add subcluster support to discard_in_l2_slice() Date: Wed, 10 Jun 2020 17:03:01 +0200 Message-Id: <402a16641f750c1069e0926c454f1da7b2eca10b.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Two things need to be taken into account here: 1) With full_discard == true the L2 entry must be cleared completely. This also includes the L2 bitmap if the image has extended L2 entries. 2) With full_discard == false we have to make the discarded cluster read back as zeroes. With normal L2 entries this is done with the QCOW_OFLAG_ZERO bit, whereas with extended L2 entries this is done with the individual 'all zeroes' bits for each subcluster. Note however that QCOW_OFLAG_ZERO is not supported in v2 qcow2 images so, if there is a backing file, discard cannot guarantee that the image will read back as zeroes. If this is important for the caller it should forbid it as qcow2_co_pdiscard() does (see 80f5c01183 for more details). Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 52 +++++++++++++++++++------------------------ 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 4e59bbd545..edfc8ea91c 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1847,11 +1847,17 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, assert(nb_clusters <= INT_MAX); for (i = 0; i < nb_clusters; i++) { - uint64_t old_l2_entry; - - old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + uint64_t old_l2_entry = get_l2_entry(s, l2_slice, l2_index + i); + uint64_t old_l2_bitmap = get_l2_bitmap(s, l2_slice, l2_index + i); + uint64_t new_l2_entry = old_l2_entry; + uint64_t new_l2_bitmap = old_l2_bitmap; + QCow2ClusterType cluster_type = + qcow2_get_cluster_type(bs, old_l2_entry); /* + * If full_discard is true, the cluster should not read back as zeroes, + * but rather fall through to the backing file. + * * If full_discard is false, make sure that a discarded area reads back * as zeroes for v3 images (we cannot do it for v2 without actually * writing a zero-filled buffer). We can skip the operation if the @@ -1860,40 +1866,28 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset, * * TODO We might want to use bdrv_block_status(bs) here, but we're * holding s->lock, so that doesn't work today. - * - * If full_discard is true, the sector should not read back as zeroes, - * but rather fall through to the backing file. */ - switch (qcow2_get_cluster_type(bs, old_l2_entry)) { - case QCOW2_CLUSTER_UNALLOCATED: - if (full_discard || !bs->backing) { - continue; + if (full_discard) { + new_l2_entry = new_l2_bitmap = 0; + } else if (bs->backing || qcow2_cluster_is_allocated(cluster_type)) { + if (has_subclusters(s)) { + new_l2_entry = 0; + new_l2_bitmap = QCOW_L2_BITMAP_ALL_ZEROES; + } else { + new_l2_entry = s->qcow_version >= 3 ? QCOW_OFLAG_ZERO : 0; } - break; + } - case QCOW2_CLUSTER_ZERO_PLAIN: - if (!full_discard) { - continue; - } - break; - - case QCOW2_CLUSTER_ZERO_ALLOC: - case QCOW2_CLUSTER_NORMAL: - case QCOW2_CLUSTER_COMPRESSED: - break; - - default: - abort(); + if (old_l2_entry == new_l2_entry && old_l2_bitmap == new_l2_bitmap) { + continue; } /* First remove L2 entries */ qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); - if (!full_discard && s->qcow_version >= 3) { - set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); - } else { - set_l2_entry(s, l2_slice, l2_index + i, 0); + set_l2_entry(s, l2_slice, l2_index + i, new_l2_entry); + if (has_subclusters(s)) { + set_l2_bitmap(s, l2_slice, l2_index + i, new_l2_bitmap); } - /* Then decrease the refcount */ qcow2_free_any_clusters(bs, old_l2_entry, 1, type); } From patchwork Wed Jun 10 15:03:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6875CC433E0 for ; Wed, 10 Jun 2020 15:26:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 380792063A for ; Wed, 10 Jun 2020 15:26:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="Y6dzKxN5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 380792063A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34484 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2cP-0001MO-Dw for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:26:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36190) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2H4-0002pE-NX; Wed, 10 Jun 2020 11:04:19 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58279) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006wr-Oa; Wed, 10 Jun 2020 11:04:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=rYtb2FYfGO+eQGiZ0oQYJSWRUo7OpthnEHRNgAuR++M=; b=Y6dzKxN59T24mSf5AOCApQQw/jWrRhJJhMcGdeoxIUaAoUtttc23v5J3FzWVD9PejyOryH+gcq+Tab4N0/mXd50OoAAXjkp8/Vnrta4822KIpdwUr+l9V5lg/fGAwD3pKR8/YsmntCqnnU5+cXLVjQL77OOgKjcRdlqsrofKgk/PDH71Tkjo/RW5F/eD1h0LeNpksw3XG9r+sQvyOTiNR/+aer48UodZlw49lCF+DkqYJQ04+2QbIy+u04roccN5hEPeCjnZoLBy9SSgkBIfKUVBMio1V6dajj2oHjBJ0aSNEZJt3Di1uJfeDQd+QU/8mOp1ecUafIt7YiGXCBkGCQ==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007hA-OO; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007NW-Ce; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 24/34] qcow2: Add subcluster support to check_refcounts_l2() Date: Wed, 10 Jun 2020 17:03:02 +0200 Message-Id: X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/qcow2-refcount.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 770c5dbc83..696e4dad07 100644 --- a/block/qcow2-refcount.c +++ b/block/qcow2-refcount.c @@ -1686,8 +1686,13 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, int ign = active ? QCOW2_OL_ACTIVE_L2 : QCOW2_OL_INACTIVE_L2; - l2_entry = QCOW_OFLAG_ZERO; - set_l2_entry(s, l2_table, i, l2_entry); + if (has_subclusters(s)) { + set_l2_entry(s, l2_table, i, 0); + set_l2_bitmap(s, l2_table, i, + QCOW_L2_BITMAP_ALL_ZEROES); + } else { + set_l2_entry(s, l2_table, i, QCOW_OFLAG_ZERO); + } ret = qcow2_pre_write_overlap_check(bs, ign, l2e_offset, l2_entry_size(s), false); if (ret < 0) { From patchwork Wed Jun 10 15:03:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E94BC433E0 for ; Wed, 10 Jun 2020 15:23:03 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE34020734 for ; Wed, 10 Jun 2020 15:23:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="eZV4qFck" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE34020734 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2ZC-0004cy-1B for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:23:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36168) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2H0-0002o6-TR; Wed, 10 Jun 2020 11:04:15 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58283) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006ws-OY; Wed, 10 Jun 2020 11:04:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=jdA6WaY7hKoW016heKIZFDBwkSoL0B2SxxYO4r5/VA8=; b=eZV4qFckifDjt4FTfYAVFCQDiJ8ALLbuca/d4A/JvZDfEFVb7A0slXps0tuoDWVey5MA5Qhr/nmeQi96pbLIYH+xSvs1eWcUuwIbxdq1TgTAZpq8mUGkCgNyN5XY4gAJAhXzjjjbBmpUR/5XFkNvFaj/BFLYXfqjpon/nPJVRGFqxB+7Ybma0KB3xJ7lxfmYWFhcbtGWP0CUx6Mc8vWQW71edYe9wgLVwSt9iiNsw9DVlktuf7833n84sAQwiHFeZYU38yiSgpulJxl+fL18Hd/cr+vioY31zO8rSCwGFI8yEXayvQ10qvRfXsCvUK+ZpAZtSTR6ZpPaFfB0Pml3dg==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007hC-OP; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007Nc-Eb; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 26/34] qcow2: Clear the L2 bitmap when allocating a compressed cluster Date: Wed, 10 Jun 2020 17:03:04 +0200 Message-Id: <616a16649a459538624d5197f988d90fc68d1281.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Compressed clusters always have the bitmap part of the extended L2 entry set to 0. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-cluster.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 2276cee6d6..deff838fe8 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -861,6 +861,9 @@ int qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, BLKDBG_EVENT(bs->file, BLKDBG_L2_UPDATE_COMPRESSED); qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); set_l2_entry(s, l2_slice, l2_index, cluster_offset); + if (has_subclusters(s)) { + set_l2_bitmap(s, l2_slice, l2_index, 0); + } qcow2_cache_put(s->l2_table_cache, (void **) &l2_slice); *host_offset = cluster_offset & s->cluster_offset_mask; From patchwork Wed Jun 10 15:03:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0BF1C433E0 for ; Wed, 10 Jun 2020 15:23:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 681A42078C for ; Wed, 10 Jun 2020 15:23:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="SefJlcnh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 681A42078C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:54206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2Zc-0005Ze-Em for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:23:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36154) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gx-0002jk-Fb; Wed, 10 Jun 2020 11:04:11 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58256) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gr-0006wh-OV; Wed, 10 Jun 2020 11:04:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=4RyZy5V804lVKxA8KqkjTU8F4A+oI5KX0VxrhskdhLk=; b=SefJlcnhxfqNWH7mxpS/YYV7ICFQUa8mps1f6+vA2x1R2BE77qhnAmKv4Y3h8Bj/EJRCDfiU6v73fJ1DRcGSJIHJ6BiNzesK8Hp4qFSH/PqLTy0uZUgqhzg6iKZaBkN1LkV4ECLP91p2MbVS7ZEmd8oJ7ETs0TVpT3SXpe8FY0SOfn4HhEVQAyenT9XHlgxziB6X34PxBAFy5+a2ktA63W1X6zZaPLC0dCnoW26N7wMnZbEwiYUIw48Mb7LWmj7ZHC9jjhoAmGttfWynaTO4Xo/j7Sbir1k13kgpHqWb/9powBsJZixNYUzaltb3iBuVugd8NTzXftblvBOKNQx/bw==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007hD-Qg; Wed, 10 Jun 2020 17:03:39 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007Ng-FW; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 27/34] qcow2: Add subcluster support to handle_alloc_space() Date: Wed, 10 Jun 2020 17:03:05 +0200 Message-Id: <42651a32704f79337daa357aba4840ec9219b3f2.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The bdrv_co_pwrite_zeroes() call here fills complete clusters with zeroes, but it can happen that some subclusters are not part of the write request or the copy-on-write. This patch makes sure that only the affected subclusters are overwritten. A potential improvement would be to also fill with zeroes the other subclusters if we can guarantee that we are not overwriting existing data. However this would waste more disk space, so we should first evaluate if it's really worth doing. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index a3481cd85b..86258fbc22 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2411,6 +2411,9 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) for (m = l2meta; m != NULL; m = m->next) { int ret; + uint64_t start_offset = m->alloc_offset + m->cow_start.offset; + unsigned nb_bytes = m->cow_end.offset + m->cow_end.nb_bytes - + m->cow_start.offset; if (!m->cow_start.nb_bytes && !m->cow_end.nb_bytes) { continue; @@ -2425,16 +2428,14 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) * efficiently zero out the whole clusters */ - ret = qcow2_pre_write_overlap_check(bs, 0, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = qcow2_pre_write_overlap_check(bs, 0, start_offset, nb_bytes, true); if (ret < 0) { return ret; } BLKDBG_EVENT(bs->file, BLKDBG_CLUSTER_ALLOC_SPACE); - ret = bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, - m->nb_clusters * s->cluster_size, + ret = bdrv_co_pwrite_zeroes(s->data_file, start_offset, nb_bytes, BDRV_REQ_NO_FALLBACK); if (ret < 0) { if (ret != -ENOTSUP && ret != -EAGAIN) { From patchwork Wed Jun 10 15:03:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alberto Garcia X-Patchwork-Id: 280858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2433DC433E1 for ; Wed, 10 Jun 2020 15:12:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E7DB4206F4 for ; Wed, 10 Jun 2020 15:12:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="OxAYhRR5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7DB4206F4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=igalia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jj2Oc-0004w5-3X for qemu-devel@archiver.kernel.org; Wed, 10 Jun 2020 11:12:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36144) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jj2Gv-0002hJ-Uk; Wed, 10 Jun 2020 11:04:10 -0400 Received: from fanzine.igalia.com ([178.60.130.6]:58259) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jj2Gs-0006wi-6i; Wed, 10 Jun 2020 11:04:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=FNEfjpoQrBQwt7pMv2/+wQIBqEVUgJ9upnhqgM5UNv0=; b=OxAYhRR5FB2uT+XcSB3+EH46eh0ct4ezQpCGwSR5wlY676cP8JIysWd7u5ttn91oAITfzMsLj5CW7kv3dJWaQ/bsKSnbClGin53vIllLsovwg11PfNidaNpZKKJ9TJnA/nYM/4jAUBkmWcKSeGyLPLSn/tRP16hoiEGa/hzTNU45cz8SdVQreEKXttJcBS0jepj9+UJh/wpQ2A3wMMuRgWG0k+HCimJs7c/xXXjv9L2635UrFmLhHNylsnJJVgDcd1no6/7Xa82esKSKvMAjbcRTXVEKMrf6+A2o0EUH0Vk3BGu//v6PvFMm63csrGn0L6cFze4JtrRpqebzA4u68w==; Received: from [81.0.38.199] (helo=perseus.local) by fanzine.igalia.com with esmtpsa (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim) id 1jj2GR-0007hM-SW; Wed, 10 Jun 2020 17:03:40 +0200 Received: from berto by perseus.local with local (Exim 4.92) (envelope-from ) id 1jj2GC-0007Nx-LD; Wed, 10 Jun 2020 17:03:24 +0200 From: Alberto Garcia To: qemu-devel@nongnu.org Subject: [PATCH v8 33/34] qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters Date: Wed, 10 Jun 2020 17:03:11 +0200 Message-Id: <19afb93b267b9698edf5e0253458c5db69ef91a2.1591801197.git.berto@igalia.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=178.60.130.6; envelope-from=berto@igalia.com; helo=fanzine.igalia.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/10 11:03:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Alberto Garcia , qemu-block@nongnu.org, Derek Su , Max Reitz Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This function is only used by qcow2_expand_zero_clusters() to downgrade a qcow2 image to a previous version. It is however not possible to downgrade an image with extended L2 entries because older versions of qcow2 do not have this feature. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 8 +++++++- tests/qemu-iotests/061 | 6 ++++++ tests/qemu-iotests/061.out | 5 +++++ 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index c8217081f2..e8bb1f32f3 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -2157,6 +2157,9 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, int ret; int i, j; + /* qcow2_downgrade() is not allowed in images with subclusters */ + assert(!has_subclusters(s)); + slice_size2 = s->l2_slice_size * l2_entry_size(s); n_slices = s->cluster_size / slice_size2; @@ -2225,7 +2228,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, if (cluster_type == QCOW2_CLUSTER_ZERO_PLAIN) { if (!bs->backing) { /* not backed; therefore we can simply deallocate the - * cluster */ + * cluster. No need to call set_l2_bitmap(), this + * function doesn't support images with subclusters. */ set_l2_entry(s, l2_slice, j, 0); l2_dirty = true; continue; @@ -2296,6 +2300,8 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table, } else { set_l2_entry(s, l2_slice, j, offset); } + /* No need to call set_l2_bitmap() after set_l2_entry() because + * this function doesn't support images with subclusters. */ l2_dirty = true; } diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index 10eb243164..23add2dfe3 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -303,6 +303,12 @@ $QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" _img_info --format-specific _check_test_img +echo +echo "=== Testing version downgrade with extended L2 entries ===" +echo +_make_test_img -o "compat=1.1,extended_l2=on" 64M +$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" + echo echo "=== Try changing the external data file ===" echo diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index 39812d8cf8..c1acdbd751 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -528,6 +528,11 @@ Format specific information: extended l2: false No errors were found on the image. +=== Testing version downgrade with extended L2 entries === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +qemu-img: Cannot downgrade an image with incompatible features 0x10 set + === Try changing the external data file === Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864