From patchwork Tue Sep 22 21:09:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 252949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E25B7C2D0E2 for ; Tue, 22 Sep 2020 21:05:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8D9D023600 for ; Tue, 22 Sep 2020 21:05:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ML0s+z4J" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726615AbgIVVFf (ORCPT ); Tue, 22 Sep 2020 17:05:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726550AbgIVVFe (ORCPT ); Tue, 22 Sep 2020 17:05:34 -0400 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE14AC061755; Tue, 22 Sep 2020 14:05:34 -0700 (PDT) Received: by mail-pf1-x443.google.com with SMTP id z18so13627880pfg.0; Tue, 22 Sep 2020 14:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=981H7COrDglg2y2OsdhN4Ry4swXs13dJrOliSNOnl9g=; b=ML0s+z4JiTGGpbGPOVB62fUQXSz8pBkrRGE1exLoA5L8vRau2DwvSWAXq5wm8EfgfS eO88W0lcJ0tX/ctYDhv/PwogvmF3gpvH7/XNB5AAcGy4qZwD0VnfGjN7iNStIFA4+Fxy wwTL4MzsEQAkh2IEJc/x9MH7j3pe5wliicssOGpMOFguasGvioTiF87/VBdMCrlDhJaz vCxOZJw/Q+TH2sSIr+kVXxR8lOFWnCWswm6nDTZSRPz9GFvZBl8VTRsyEtf/GIaeVGbQ Z6+Ld/5dYMMq+fGLtJ8DOS0JNLuc5IB0IaSeUql+ezqKOXj7lCQVXHfU7W/GpalbKAjE qEjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=981H7COrDglg2y2OsdhN4Ry4swXs13dJrOliSNOnl9g=; b=Uj+7zJoQLKko/RZP7vBPRNEZb7sO3PVQiGkTUy8HfDm2q+Yd/FHju02TXkycH4XHS9 Bcq1FIAa3uqYZKUt3xdkKQ9o6k64mOvDU07sUWYEJMtKoLlKSwDn0aYiuzPLe9fz2ovf GOBal5NTuXrGVpvRmPs4uiHQr44yzTg0iCcSMfWVfHBbwZM8O+TkseIxAnRjzY8WOjnK sWxnvfJqoCLaJGr7X69Bu3hNB+mupP5az14zqXzMY7tegCUjAjzniSTtQrNEPQcWX+Iz yDsM1tHmY1lph2nERLHQWRV2gBQiaWv6Df9vhYzODaRMnC4P0GkZmpPbEWcn2ijh8M+3 sfXg== X-Gm-Message-State: AOAM533Ou77c32jJzmotcgGA7cGsNnWjE/qlA/6UEuCGPcrymWEdp9Mb CaOqbug6DSGISwtapnwanno= X-Google-Smtp-Source: ABdhPJzpIRe/uXi9oko6PATw8guaQq7mv8V7LLBJQB2a9sT+vfvvH75/JxXt+7bx/mgfndJLJK6jQQ== X-Received: by 2002:a17:902:aa02:b029:d0:cbe1:e7b4 with SMTP id be2-20020a170902aa02b02900d0cbe1e7b4mr6181491plb.37.1600808734213; Tue, 22 Sep 2020 14:05:34 -0700 (PDT) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id i15sm16118945pfk.145.2020.09.22.14.05.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 14:05:33 -0700 (PDT) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v2 2/9] lib: zstd: Add decompress_sources.h for decompress_unzstd Date: Tue, 22 Sep 2020 14:09:17 -0700 Message-Id: <20200922210924.1725-3-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200922210924.1725-1-nickrterrell@gmail.com> References: <20200922210924.1725-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Nick Terrell Adds decompress_sources.h which includes every .c file necessary for zstd decompression. This is used in decompress_unzstd.c so the internal structure of the library isn't exposed. This allows us to upgrade the zstd library version without modifying any callers. Instead we just need to update decompress_sources.h. Signed-off-by: Nick Terrell --- lib/decompress_unzstd.c | 6 +----- lib/zstd/decompress_sources.h | 14 ++++++++++++++ 2 files changed, 15 insertions(+), 5 deletions(-) create mode 100644 lib/zstd/decompress_sources.h diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c index dbc290af26b4..a79f705f236d 100644 --- a/lib/decompress_unzstd.c +++ b/lib/decompress_unzstd.c @@ -68,11 +68,7 @@ #ifdef STATIC # define UNZSTD_PREBOOT # include "xxhash.c" -# include "zstd/entropy_common.c" -# include "zstd/fse_decompress.c" -# include "zstd/huf_decompress.c" -# include "zstd/zstd_common.c" -# include "zstd/decompress.c" +# include "zstd/decompress_sources.h" #endif #include diff --git a/lib/zstd/decompress_sources.h b/lib/zstd/decompress_sources.h new file mode 100644 index 000000000000..ccb4960ea0cd --- /dev/null +++ b/lib/zstd/decompress_sources.h @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/* + * This file includes every .c file needed for decompression. + * It is used by lib/decompress_unzstd.c to include the decompression + * source into the translation-unit, so it can be used for kernel + * decompression. + */ + +#include "entropy_common.c" +#include "fse_decompress.c" +#include "huf_decompress.c" +#include "zstd_common.c" +#include "decompress.c" From patchwork Tue Sep 22 21:09:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 252948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA0FC2D0E2 for ; Tue, 22 Sep 2020 21:06:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ABEA323600 for ; Tue, 22 Sep 2020 21:06:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OxNT+QNu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726865AbgIVVGg (ORCPT ); Tue, 22 Sep 2020 17:06:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42392 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726583AbgIVVGg (ORCPT ); Tue, 22 Sep 2020 17:06:36 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 586B7C061755; Tue, 22 Sep 2020 14:06:36 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id jw11so2041668pjb.0; Tue, 22 Sep 2020 14:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2E/IwoJzArQOI0tLeIWcAV63W6gwJ92+ohjyeEw8pg8=; b=OxNT+QNuKonqqEDoLxfT0evE2G9xxahxWfwna/whIy+Phh4fehxfZzuzQiKaY/3E7L P738gfvMJiE3Ef+6MWIXCrtMiimEOApRHUa4Rx0mkRYuKy+Te16Gn0Ort9Cjy4JUA3YS n0WiF+RrFhlWK3g+M1cMPsUXXk6hGfjK7HwyCL15KfhH2feuz3DZD/Mof0shnhpIRLL7 pH4yytxKasaTPWLiNPtcuwWerS/VpC5eWYpXKiebPEAmNfsTmahh+lUsyHtqJ04mMVMR sDef/8wqxfLSzNX0nkp5zeKChnGxYmXTk2HQOQ5ge1b1U38uDEneRtMrrNuHrEf1UT+H hSrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2E/IwoJzArQOI0tLeIWcAV63W6gwJ92+ohjyeEw8pg8=; b=fkG3jwroza4Rf+iYQ4vWep8eRkM5IF0TCsjvKfmy/h9VhHx3woh/w6LieOqsB6mRnQ Ux+yS73ovKyO4gbgkMhxZlhOeNZ4RIYOrbvAEVx4UGNI+xMYBs5wMoNIyWHa5tStNrRr A9OPmcIupyUrg1h/89sA8a2jbqibTUboDRsZCjv+g1qFTAlSsQXaUdqpQn09FbnhPP6r LaqNtBF4pMahbzN6Ju/P3bvh7ZW40R16mlqMpavEKLaReMMISL2CsPsAKEvB10TIui3V u811hDO0puj9hFhwT1I3wITvH193MBt6iuLCuC4V46aJRse87yeQ7mWKrN8jbfSYRcwF 5AtA== X-Gm-Message-State: AOAM53180ECZxTFE5z97TNAe5gZqzyewhVkBhYsklWdG3Hh2oO0nmvSb s2h8JCcZIgUM+Daf9QIcmMA= X-Google-Smtp-Source: ABdhPJytFO5yq1IRr/bPG4viE/+Jdt804NAlJGq8B+rnglr0auId1lXZAT05/cvBuVG/SIVkOxQLUg== X-Received: by 2002:a17:90b:4ac4:: with SMTP id mh4mr5502426pjb.224.1600808795872; Tue, 22 Sep 2020 14:06:35 -0700 (PDT) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id i15sm16118945pfk.145.2020.09.22.14.06.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 14:06:35 -0700 (PDT) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v2 5/9] btrfs: zstd: Switch to the zstd-1.4.6 API Date: Tue, 22 Sep 2020 14:09:20 -0700 Message-Id: <20200922210924.1725-6-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200922210924.1725-1-nickrterrell@gmail.com> References: <20200922210924.1725-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell --- fs/btrfs/zstd.c | 48 ++++++++++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/zstd.c b/fs/btrfs/zstd.c index a7367ff573d4..6b466e090cd7 100644 --- a/fs/btrfs/zstd.c +++ b/fs/btrfs/zstd.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include "misc.h" #include "compression.h" #include "ctree.h" @@ -159,8 +159,8 @@ static void zstd_calc_ws_mem_sizes(void) zstd_get_btrfs_parameters(level, ZSTD_BTRFS_MAX_INPUT); size_t level_size = max_t(size_t, - ZSTD_CStreamWorkspaceBound(params.cParams), - ZSTD_DStreamWorkspaceBound(ZSTD_BTRFS_MAX_INPUT)); + ZSTD_estimateCStreamSize_usingCParams(params.cParams), + ZSTD_estimateDStreamSize(ZSTD_BTRFS_MAX_INPUT)); max_size = max_t(size_t, max_size, level_size); zstd_ws_mem_sizes[level - 1] = max_size; @@ -389,13 +389,23 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, *total_in = 0; /* Initialize the stream */ - stream = ZSTD_initCStream(params, len, workspace->mem, - workspace->size); + stream = ZSTD_initStaticCStream(workspace->mem, workspace->size); if (!stream) { - pr_warn("BTRFS: ZSTD_initCStream failed\n"); + pr_warn("BTRFS: ZSTD_initStaticCStream failed\n"); ret = -EIO; goto out; } + { + size_t ret2; + + ret2 = ZSTD_initCStream_advanced(stream, NULL, 0, params, len); + if (ZSTD_isError(ret2)) { + pr_warn("BTRFS: ZSTD_initCStream_advanced returned %s\n", + ZSTD_getErrorName(ret2)); + ret = -EIO; + goto out; + } + } /* map in the first page of input data */ in_page = find_get_page(mapping, start >> PAGE_SHIFT); @@ -421,8 +431,8 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret2 = ZSTD_compressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_compressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_compressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto out; } @@ -489,8 +499,8 @@ int zstd_compress_pages(struct list_head *ws, struct address_space *mapping, ret2 = ZSTD_endStream(stream, &workspace->out_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_endStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_endStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto out; } @@ -557,10 +567,9 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb) unsigned long buf_start; unsigned long total_out = 0; - stream = ZSTD_initDStream( - ZSTD_BTRFS_MAX_INPUT, workspace->mem, workspace->size); + stream = ZSTD_initStaticDStream(workspace->mem, workspace->size); if (!stream) { - pr_debug("BTRFS: ZSTD_initDStream failed\n"); + pr_debug("BTRFS: ZSTD_initStaticDStream failed\n"); ret = -EIO; goto done; } @@ -579,8 +588,8 @@ int zstd_decompress_bio(struct list_head *ws, struct compressed_bio *cb) ret2 = ZSTD_decompressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_decompressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_decompressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto done; } @@ -633,10 +642,9 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in, unsigned long pg_offset = 0; char *kaddr; - stream = ZSTD_initDStream( - ZSTD_BTRFS_MAX_INPUT, workspace->mem, workspace->size); + stream = ZSTD_initStaticDStream(workspace->mem, workspace->size); if (!stream) { - pr_warn("BTRFS: ZSTD_initDStream failed\n"); + pr_warn("BTRFS: ZSTD_initStaticDStream failed\n"); ret = -EIO; goto finish; } @@ -667,8 +675,8 @@ int zstd_decompress(struct list_head *ws, unsigned char *data_in, ret2 = ZSTD_decompressStream(stream, &workspace->out_buf, &workspace->in_buf); if (ZSTD_isError(ret2)) { - pr_debug("BTRFS: ZSTD_decompressStream returned %d\n", - ZSTD_getErrorCode(ret2)); + pr_debug("BTRFS: ZSTD_decompressStream returned %s\n", + ZSTD_getErrorName(ret2)); ret = -EIO; goto finish; } From patchwork Tue Sep 22 21:09:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 252947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF30BC2D0E2 for ; Tue, 22 Sep 2020 21:06:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF23023600 for ; Tue, 22 Sep 2020 21:06:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BWzf+T/n" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726608AbgIVVGw (ORCPT ); Tue, 22 Sep 2020 17:06:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726902AbgIVVGt (ORCPT ); Tue, 22 Sep 2020 17:06:49 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 955E9C0613D0; Tue, 22 Sep 2020 14:06:49 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id a9so2085555pjg.1; Tue, 22 Sep 2020 14:06:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=krGpYOVpUat0gERMlMR1P1lhK//ROWK8HUHnVTxyKd4=; b=BWzf+T/n2kbFP0tByp9/Jj9Wok9svu79fWXRTG9+przzI26Wwi9XywjndiuHBMJjd5 +LNbN8ae8kf9EFs0aL4J/l0Cuj+MS4Q+HYB3PvpbTpkc7t8X/646ZwKWiRE2aIQ5tKoE XH4yjWacRXjeTTALpzhzI/6K9eept34kFVOYaG/c/DAum0CvTWisUUoWXXQ8UgwwKasb ZrnfFWT4aa+WPtdMe2styka5E1D4AzDdPq8WMjzJcOkdTxlzbH+wPZx7KFevGNK+cxN+ 9AkwOmHx7Yk5hTKAdWR3bcoW5UtqGLtpiqPIYligeiOKjLeGpoXXx+21Y5eKgVRL9TuP XFSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=krGpYOVpUat0gERMlMR1P1lhK//ROWK8HUHnVTxyKd4=; b=T+rLDFcgvjOIJAnly8L0nfTaUA2wjC4JOQ7tsM0n7qIdHL2HPE9DKKWXNTqigATALw kGU4nqv38VvugUor1QgAFeVRJuBhChaIF2FJnqTRLTkwvr+oav4R4ZGWc51k05fWcqj7 /gnqjbA4PQ6mKBaZ3NWS+pVP1rAZD8jWct5PiNR6QUXJph8F0hGACYnuDVCciLCnyK9Y rHRZ3hLYJxXQPP0xriuPAZ/vRXQoqmVqBITnuR/hUfk959HwWNc1oAd4nEZ7oAZTshLX n/EtM4gsBFEr8NWe+oKWVwHuhU8Ay5WKBQ78APcAEt02sTlSNISFlM3HHhnsibcGt/go MXKg== X-Gm-Message-State: AOAM5316CCN0P7qDtCoiLS/6mWbLACB7uUf4z95ZfxDoLMBJte75+ZlF 7FYxlsOqbsZmkCZmW/U85cQ= X-Google-Smtp-Source: ABdhPJw8m2iBA2+aOJR+wkwV3XyuTSZLKB0OYrh4Sz3KjUcIV0mUpuLoIuh/IHHxxuh13lJQBUDCEA== X-Received: by 2002:a17:90a:7487:: with SMTP id p7mr5440688pjk.189.1600808809074; Tue, 22 Sep 2020 14:06:49 -0700 (PDT) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id i15sm16118945pfk.145.2020.09.22.14.06.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 14:06:48 -0700 (PDT) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v2 8/9] lib: unzstd: Switch to the zstd-1.4.6 API Date: Tue, 22 Sep 2020 14:09:23 -0700 Message-Id: <20200922210924.1725-9-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200922210924.1725-1-nickrterrell@gmail.com> References: <20200922210924.1725-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Nick Terrell Move away from the compatibility wrapper to the zstd-1.4.6 API. This code is functionally equivalent. Signed-off-by: Nick Terrell --- lib/decompress_unzstd.c | 40 ++++++++++++++-------------------------- 1 file changed, 14 insertions(+), 26 deletions(-) diff --git a/lib/decompress_unzstd.c b/lib/decompress_unzstd.c index a79f705f236d..d4685df0e120 100644 --- a/lib/decompress_unzstd.c +++ b/lib/decompress_unzstd.c @@ -73,7 +73,8 @@ #include #include -#include +#include +#include /* 128MB is the maximum window size supported by zstd. */ #define ZSTD_WINDOWSIZE_MAX (1 << ZSTD_WINDOWLOG_MAX) @@ -120,9 +121,9 @@ static int INIT decompress_single(const u8 *in_buf, long in_len, u8 *out_buf, long out_len, long *in_pos, void (*error)(char *x)) { - const size_t wksp_size = ZSTD_DCtxWorkspaceBound(); + const size_t wksp_size = ZSTD_estimateDCtxSize(); void *wksp = large_malloc(wksp_size); - ZSTD_DCtx *dctx = ZSTD_initDCtx(wksp, wksp_size); + ZSTD_DCtx *dctx = ZSTD_initStaticDCtx(wksp, wksp_size); int err; size_t ret; @@ -165,7 +166,6 @@ static int INIT __unzstd(unsigned char *in_buf, long in_len, { ZSTD_inBuffer in; ZSTD_outBuffer out; - ZSTD_frameParams params; void *in_allocated = NULL; void *out_allocated = NULL; void *wksp = NULL; @@ -229,36 +229,24 @@ static int INIT __unzstd(unsigned char *in_buf, long in_len, out.size = out_len; /* - * We need to know the window size to allocate the ZSTD_DStream. - * Since we are streaming, we need to allocate a buffer for the sliding - * window. The window size varies from 1 KB to ZSTD_WINDOWSIZE_MAX - * (8 MB), so it is important to use the actual value so as not to - * waste memory when it is smaller. + * Zstd determines the workspace size from the window size written + * into the frame header. This ensures that we use the minimum value + * possible, since the window size varies from 1 KB to ZSTD_WINDOWSIZE_MAX + * (1 GB), so it is very important to use the actual value. */ - ret = ZSTD_getFrameParams(¶ms, in.src, in.size); + wksp_size = ZSTD_estimateDStreamSize_fromFrame(in.src, in.size); err = handle_zstd_error(ret, error); if (err) goto out; - if (ret != 0) { - error("ZSTD-compressed data has an incomplete frame header"); - err = -1; - goto out; - } - if (params.windowSize > ZSTD_WINDOWSIZE_MAX) { - error("ZSTD-compressed data has too large a window size"); + wksp = large_malloc(wksp_size); + if (wksp == NULL) { + error("Out of memory while allocating ZSTD_DStream"); err = -1; goto out; } - - /* - * Allocate the ZSTD_DStream now that we know how much memory is - * required. - */ - wksp_size = ZSTD_DStreamWorkspaceBound(params.windowSize); - wksp = large_malloc(wksp_size); - dstream = ZSTD_initDStream(params.windowSize, wksp, wksp_size); + dstream = ZSTD_initStaticDStream(wksp, wksp_size); if (dstream == NULL) { - error("Out of memory while allocating ZSTD_DStream"); + error("ZSTD_initStaticDStream failed"); err = -1; goto out; } From patchwork Tue Sep 22 21:09:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Terrell X-Patchwork-Id: 252946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A356C4741F for ; Tue, 22 Sep 2020 21:07:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D907E2371F for ; Tue, 22 Sep 2020 21:07:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lRIEw3LI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726641AbgIVVHA (ORCPT ); Tue, 22 Sep 2020 17:07:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726932AbgIVVGy (ORCPT ); Tue, 22 Sep 2020 17:06:54 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B128C061755; Tue, 22 Sep 2020 14:06:54 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id y14so2926136pgf.12; Tue, 22 Sep 2020 14:06:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s2HCAxyO+XuYMLzto3YJY+q/fwyGyhZhVewL5HVF+qc=; b=lRIEw3LIKpK43VEPPd4xH2mox+Cahw0bTKVkceOo4e4tsWnt5YzukXsfzlUNgRv9rX ZCl62JqQcu/+AzptIeEUCJ11pojyLp62XOrlMaTVNn9lCfe/ffIP6+RmmcR3uv3OKRF4 jNnFhLVl1f+5wv+iMlTTIzAe25W9N1BZMJ6loO/LdtXqJmPVN+KCGn/JDdnTP8ejlNqV dDdKSRfBL4fY1GZAZ2hXSXAnSLH9TU4avltFpZHyyXiN4qKziIXMgnY+0hDHktBy3w7V Yw5zbps37F/hmIAR7uIAuNYIQDvvot4GkiFWKxLolgxnriU/+Ii9AzDDmFvr5R1btozD uHNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s2HCAxyO+XuYMLzto3YJY+q/fwyGyhZhVewL5HVF+qc=; b=myb0OtUg6jv8AjmZ7Gnr4raAWrruEu0AatCB1qCaWIrTGTZ32A41lx8MiWptYpDVkz C3sid3Gaeu/tPR2GTQR/F/d3hJhHE13kDZFA5bZxJaelncY9DSBfYpMZ2Jciy0RC2usF sAzrilAeSdWp5OijAhJdcYoLorMLWs9zfDk+PhVoIrjztamN2L62whIr9fjTztzjc26n 5kmQ6mlDjcmF1FA/Z+wJ24I2zgNF1kZ18FXGe9LAvVU+LFdwFbr+X+JYfIdLDLYrEHfQ XIhnSChRvGF5PYjnJ692P8w648is0WBEDEFA7pgbwUMkxNjW+mUjfk27LpcvOtEilorm j9Kg== X-Gm-Message-State: AOAM531cqcPWTMYPHkcAXsjzdOPgsiN6jQD2uLMDqdnhdPXIyb9KTEll OrR3kG5YpJeeoHuAzud79I8= X-Google-Smtp-Source: ABdhPJyTivGi+ISDXozBH1tMjx78d74kOuBv8lXeZEL7dpO1lsp2fJNTibT4pdbAXO+tflxBsJijqw== X-Received: by 2002:aa7:9850:0:b029:142:2501:34f0 with SMTP id n16-20020aa798500000b0290142250134f0mr5610701pfq.73.1600808813520; Tue, 22 Sep 2020 14:06:53 -0700 (PDT) Received: from nickserv.localdomain (c-98-33-101-203.hsd1.ca.comcast.net. [98.33.101.203]) by smtp.gmail.com with ESMTPSA id i15sm16118945pfk.145.2020.09.22.14.06.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2020 14:06:52 -0700 (PDT) From: Nick Terrell To: Herbert Xu Cc: linux-crypto@vger.kernel.org, linux-btrfs@vger.kernel.org, squashfs-devel@lists.sourceforge.net, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, Kernel Team , Nick Terrell , Nick Terrell , Chris Mason , Petr Malat , Johannes Weiner , Niket Agarwal , Yann Collet Subject: [PATCH v2 9/9] lib: zstd: Remove zstd compatibility wrapper Date: Tue, 22 Sep 2020 14:09:24 -0700 Message-Id: <20200922210924.1725-10-nickrterrell@gmail.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200922210924.1725-1-nickrterrell@gmail.com> References: <20200922210924.1725-1-nickrterrell@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: Nick Terrell All callers have been transitioned to the new zstd-1.4.6 API. There are no more callers of the zstd compatibility wrapper, so delete it. Signed-off-by: Nick Terrell --- include/linux/zstd_compat.h | 116 ------------------------------------ 1 file changed, 116 deletions(-) delete mode 100644 include/linux/zstd_compat.h diff --git a/include/linux/zstd_compat.h b/include/linux/zstd_compat.h deleted file mode 100644 index cda9208bf04a..000000000000 --- a/include/linux/zstd_compat.h +++ /dev/null @@ -1,116 +0,0 @@ -/* - * Copyright (c) 2016-present, Facebook, Inc. - * All rights reserved. - * - * This source code is licensed under the BSD-style license found in the - * LICENSE file in the root directory of https://github.com/facebook/zstd. - * An additional grant of patent rights can be found in the PATENTS file in the - * same directory. - * - * This program is free software; you can redistribute it and/or modify it under - * the terms of the GNU General Public License version 2 as published by the - * Free Software Foundation. This program is dual-licensed; you may select - * either version 2 of the GNU General Public License ("GPL") or BSD license - * ("BSD"). - */ - -#ifndef ZSTD_COMPAT_H -#define ZSTD_COMPAT_H - -#include - -#if defined(ZSTD_VERSION_NUMBER) && (ZSTD_VERSION_NUMBER >= 10406) -/* - * This header provides backwards compatibility for the zstd-1.4.6 library - * upgrade. This header allows us to upgrade the zstd library version without - * modifying any callers. Then we will migrate callers from the compatibility - * wrapper one at a time until none remain. At which point we will delete this - * header. - * - * It is temporary and will be deleted once the upgrade is complete. - */ - -#include - -static inline size_t ZSTD_CCtxWorkspaceBound(ZSTD_compressionParameters compression_params) -{ - return ZSTD_estimateCCtxSize_usingCParams(compression_params); -} - -static inline size_t ZSTD_CStreamWorkspaceBound(ZSTD_compressionParameters compression_params) -{ - return ZSTD_estimateCStreamSize_usingCParams(compression_params); -} - -static inline size_t ZSTD_DCtxWorkspaceBound(void) -{ - return ZSTD_estimateDCtxSize(); -} - -static inline size_t ZSTD_DStreamWorkspaceBound(unsigned long long window_size) -{ - return ZSTD_estimateDStreamSize(window_size); -} - -static inline ZSTD_CCtx* ZSTD_initCCtx(void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - return ZSTD_initStaticCCtx(wksp, wksp_size); -} - -static inline ZSTD_CStream* ZSTD_initCStream_compat(ZSTD_parameters params, uint64_t pledged_src_size, void* wksp, size_t wksp_size) -{ - ZSTD_CStream* cstream; - size_t ret; - - if (wksp == NULL) - return NULL; - - cstream = ZSTD_initStaticCStream(wksp, wksp_size); - if (cstream == NULL) - return NULL; - - /* 0 means unknown in old API but means 0 in new API */ - if (pledged_src_size == 0) - pledged_src_size = ZSTD_CONTENTSIZE_UNKNOWN; - - ret = ZSTD_initCStream_advanced(cstream, NULL, 0, params, pledged_src_size); - if (ZSTD_isError(ret)) - return NULL; - - return cstream; -} -#define ZSTD_initCStream ZSTD_initCStream_compat - -static inline ZSTD_DCtx* ZSTD_initDCtx(void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - return ZSTD_initStaticDCtx(wksp, wksp_size); -} - -static inline ZSTD_DStream* ZSTD_initDStream_compat(unsigned long long window_size, void* wksp, size_t wksp_size) -{ - if (wksp == NULL) - return NULL; - (void)window_size; - return ZSTD_initStaticDStream(wksp, wksp_size); -} -#define ZSTD_initDStream ZSTD_initDStream_compat - -typedef ZSTD_frameHeader ZSTD_frameParams; - -static inline size_t ZSTD_getFrameParams(ZSTD_frameParams* frame_params, const void* src, size_t src_size) -{ - return ZSTD_getFrameHeader(frame_params, src, src_size); -} - -static inline size_t ZSTD_compressCCtx_compat(ZSTD_CCtx* cctx, void* dst, size_t dst_capacity, const void* src, size_t src_size, ZSTD_parameters params) -{ - return ZSTD_compress_advanced(cctx, dst, dst_capacity, src, src_size, NULL, 0, params); -} -#define ZSTD_compressCCtx ZSTD_compressCCtx_compat - -#endif /* ZSTD_VERSION_NUMBER >= 10406 */ -#endif /* ZSTD_COMPAT_H */