| Message ID | 1578452513-7045-1-git-send-email-weijie.gao@mediatek.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [01/16] configs: mtmips: add missing board selection for gardena-smart-gateway-mt7688 | expand |
On 1/8/20 4:01 AM, Weijie Gao wrote: > This patch enables LZMA decompression support for SPL build > > Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> > --- > lib/Kconfig | 5 +++++ > lib/Makefile | 1 + > 2 files changed, 6 insertions(+) > > diff --git a/lib/Kconfig b/lib/Kconfig > index 965cf7bc03..2f38e37802 100644 > --- a/lib/Kconfig > +++ b/lib/Kconfig > @@ -418,6 +418,11 @@ config SPL_LZ4 > fast compression and decompression speed. It belongs to the LZ77 > family of byte-oriented compression schemes. > > +config SPL_LZMA > + bool "Enable LZMA decompression support for SPL build" > + help > + This enables support for LZMA compression altorithm for SPL boot. %s/altorithm/algorithm/g Why do we need yet another algorithm for compression? Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression Best regards Heinrich > + > config SPL_LZO > bool "Enable LZO decompression support in SPL" > help > diff --git a/lib/Makefile b/lib/Makefile > index 1fb650cd90..8e51eec01c 100644 > --- a/lib/Makefile > +++ b/lib/Makefile > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > obj-$(CONFIG_LIBAVB) += libavb/ >
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > This patch enables LZMA decompression support for SPL build > > > > Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> > > --- > > lib/Kconfig | 5 +++++ > > lib/Makefile | 1 + > > 2 files changed, 6 insertions(+) > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > index 965cf7bc03..2f38e37802 100644 > > --- a/lib/Kconfig > > +++ b/lib/Kconfig > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > fast compression and decompression speed. It belongs to the LZ77 > > family of byte-oriented compression schemes. > > > > +config SPL_LZMA > > + bool "Enable LZMA decompression support for SPL build" > > + help > > + This enables support for LZMA compression altorithm for SPL boot. > > %s/altorithm/algorithm/g > > Why do we need yet another algorithm for compression? > > Doesn't Zstandard provide a much faster decompression? > Cf. https://sites.google.com/site/powturbo/home/web-compression > > Best regards > > Heinrich > > > + > > config SPL_LZO > > bool "Enable LZO decompression support in SPL" > > help > > diff --git a/lib/Makefile b/lib/Makefile > > index 1fb650cd90..8e51eec01c 100644 > > --- a/lib/Makefile > > +++ b/lib/Makefile > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > Hi Heinrich, I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible. I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time. Best Regards, Weijie
On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > This patch enables LZMA decompression support for SPL build > > > > > > Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> > > > --- > > > lib/Kconfig | 5 +++++ > > > lib/Makefile | 1 + > > > 2 files changed, 6 insertions(+) > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > index 965cf7bc03..2f38e37802 100644 > > > --- a/lib/Kconfig > > > +++ b/lib/Kconfig > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > fast compression and decompression speed. It belongs to the LZ77 > > > family of byte-oriented compression schemes. > > > > > > +config SPL_LZMA > > > + bool "Enable LZMA decompression support for SPL build" > > > + help > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > %s/altorithm/algorithm/g > > > > Why do we need yet another algorithm for compression? > > > > Doesn't Zstandard provide a much faster decompression? > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > Best regards > > > > Heinrich > > > > > + > > > config SPL_LZO > > > bool "Enable LZO decompression support in SPL" > > > help > > > diff --git a/lib/Makefile b/lib/Makefile > > > index 1fb650cd90..8e51eec01c 100644 > > > --- a/lib/Makefile > > > +++ b/lib/Makefile > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > Hi Heinrich, > > I need an algorithm with the maximum compression ratio to reduce the > size of the u-boot payload as much as possible. > > I believe currently lzma and xz are the best choices. > Refer to: > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > Although they will take longer time for compression and decompression, > considering that the u-boot payload is usually small (<= 1MiB), there > will not be significant effect on decompression time. Where does zstd fall in the list however? Can you please try that and report back? Thanks!
On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote: > On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > > This patch enables LZMA decompression support for SPL build > > > > > > > > Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> > > > > --- > > > > lib/Kconfig | 5 +++++ > > > > lib/Makefile | 1 + > > > > 2 files changed, 6 insertions(+) > > > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > > index 965cf7bc03..2f38e37802 100644 > > > > --- a/lib/Kconfig > > > > +++ b/lib/Kconfig > > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > > fast compression and decompression speed. It belongs to the LZ77 > > > > family of byte-oriented compression schemes. > > > > > > > > +config SPL_LZMA > > > > + bool "Enable LZMA decompression support for SPL build" > > > > + help > > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > > > %s/altorithm/algorithm/g > > > > > > Why do we need yet another algorithm for compression? > > > > > > Doesn't Zstandard provide a much faster decompression? > > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > > > Best regards > > > > > > Heinrich > > > > > > > + > > > > config SPL_LZO > > > > bool "Enable LZO decompression support in SPL" > > > > help > > > > diff --git a/lib/Makefile b/lib/Makefile > > > > index 1fb650cd90..8e51eec01c 100644 > > > > --- a/lib/Makefile > > > > +++ b/lib/Makefile > > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > > > > Hi Heinrich, > > > > I need an algorithm with the maximum compression ratio to reduce the > > size of the u-boot payload as much as possible. > > > > I believe currently lzma and xz are the best choices. > > Refer to: > > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > > > Although they will take longer time for compression and decompression, > > considering that the u-boot payload is usually small (<= 1MiB), there > > will not be significant effect on decompression time. > > Where does zstd fall in the list however? Can you please try that and > report back? Thanks! > My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed. So there are two points must be taken into consideration: the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...). Below is my test results, including all compression algorithms supported by uImage, plus zstd: Platform: mips - mt7628 Uncompressed size of u-boot.bin: 302736 (295KiB) Commands for compression: gzip -c -f -k -9 bzip2 -c -f -k -9 lzma -c -z -k -9 lzop -c -f -9 lz4 -f -9 zstd --ultra -f Results: algo. | spl.bin size | compressed size | % of uncomp | total size ------------------------------------------------------------------------ gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB) ------------------------------------------------------------------------ bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB) ------------------------------------------------------------------------ lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB) ------------------------------------------------------------------------ lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB) ------------------------------------------------------------------------ lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB) ------------------------------------------------------------------------ zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB) As you can see lzma has the best compression ratio. The is the reason I chose lzma here. Best Regards
On Fri, Jan 10, 2020 at 04:21:35PM +0800, Weijie Gao wrote: > On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote: > > On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote: > > > On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote: > > > > > > > > On 1/8/20 4:01 AM, Weijie Gao wrote: > > > > > This patch enables LZMA decompression support for SPL build > > > > > > > > > > Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> > > > > > --- > > > > > lib/Kconfig | 5 +++++ > > > > > lib/Makefile | 1 + > > > > > 2 files changed, 6 insertions(+) > > > > > > > > > > diff --git a/lib/Kconfig b/lib/Kconfig > > > > > index 965cf7bc03..2f38e37802 100644 > > > > > --- a/lib/Kconfig > > > > > +++ b/lib/Kconfig > > > > > @@ -418,6 +418,11 @@ config SPL_LZ4 > > > > > fast compression and decompression speed. It belongs to the LZ77 > > > > > family of byte-oriented compression schemes. > > > > > > > > > > +config SPL_LZMA > > > > > + bool "Enable LZMA decompression support for SPL build" > > > > > + help > > > > > + This enables support for LZMA compression altorithm for SPL boot. > > > > > > > > %s/altorithm/algorithm/g > > > > > > > > Why do we need yet another algorithm for compression? > > > > > > > > Doesn't Zstandard provide a much faster decompression? > > > > Cf. https://sites.google.com/site/powturbo/home/web-compression > > > > > > > > Best regards > > > > > > > > Heinrich > > > > > > > > > + > > > > > config SPL_LZO > > > > > bool "Enable LZO decompression support in SPL" > > > > > help > > > > > diff --git a/lib/Makefile b/lib/Makefile > > > > > index 1fb650cd90..8e51eec01c 100644 > > > > > --- a/lib/Makefile > > > > > +++ b/lib/Makefile > > > > > @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ > > > > > obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ > > > > > obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o > > > > > obj-$(CONFIG_$(SPL_)LZO) += lzo/ > > > > > +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ > > > > > obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o > > > > > > > > > > obj-$(CONFIG_LIBAVB) += libavb/ > > > > > > > > > > > Hi Heinrich, > > > > > > I need an algorithm with the maximum compression ratio to reduce the > > > size of the u-boot payload as much as possible. > > > > > > I believe currently lzma and xz are the best choices. > > > Refer to: > > > https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO > > > > > > Although they will take longer time for compression and decompression, > > > considering that the u-boot payload is usually small (<= 1MiB), there > > > will not be significant effect on decompression time. > > > > Where does zstd fall in the list however? Can you please try that and > > report back? Thanks! > > > > My purpose is to reduce the size of u-boot binary, and my plan is to > compress the payload and the SPL decompress it at runtime. > I need smallest size here, not fastest decompression speed. > > So there are two points must be taken into consideration: > the size of the decompressor (i.e. u-boot-spl.bin) and the size of the > compressed payload (u-boot.bin.gz/bz2/...). > > Below is my test results, including all compression algorithms supported > by uImage, plus zstd: > > Platform: mips - mt7628 > > Uncompressed size of u-boot.bin: 302736 (295KiB) > > Commands for compression: > gzip -c -f -k -9 > bzip2 -c -f -k -9 > lzma -c -z -k -9 > lzop -c -f -9 > lz4 -f -9 > zstd --ultra -f > > Results: > algo. | spl.bin size | compressed size | % of uncomp | total size > ------------------------------------------------------------------------ > gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB) > ------------------------------------------------------------------------ > bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB) > ------------------------------------------------------------------------ > lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB) > ------------------------------------------------------------------------ > lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB) > ------------------------------------------------------------------------ > lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB) > ------------------------------------------------------------------------ > zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB) > > As you can see lzma has the best compression ratio. The is the reason I > chose lzma here. Thanks, that's exactly the kind of data I was looking for. Reviewed-by: Tom Rini <trini at konsulko.com>
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes. +config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot. + config SPL_LZO bool "Enable LZO decompression support in SPL" help diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o obj-$(CONFIG_LIBAVB) += libavb/
This patch enables LZMA decompression support for SPL build Signed-off-by: Weijie Gao <weijie.gao at mediatek.com> --- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)