From patchwork Mon Jul 11 15:55:14 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laszlo Ersek X-Patchwork-Id: 71743 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp23877qga; Mon, 11 Jul 2016 08:55:26 -0700 (PDT) X-Received: by 10.66.121.136 with SMTP id lk8mr35873400pab.51.1468252526721; Mon, 11 Jul 2016 08:55:26 -0700 (PDT) Return-Path: Received: from ml01.01.org (ml01.01.org. [198.145.21.10]) by mx.google.com with ESMTPS id wo10si4431892pab.206.2016.07.11.08.55.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 11 Jul 2016 08:55:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 198.145.21.10 as permitted sender) client-ip=198.145.21.10; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 198.145.21.10 as permitted sender) smtp.mailfrom=edk2-devel-bounces@lists.01.org Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 716ED1A1FB1; Mon, 11 Jul 2016 08:56:10 -0700 (PDT) X-Original-To: edk2-devel@ml01.01.org Delivered-To: edk2-devel@ml01.01.org Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8F7CE1A1E85 for ; Mon, 11 Jul 2016 08:56:08 -0700 (PDT) Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E075464070; Mon, 11 Jul 2016 15:55:23 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-116-91.phx2.redhat.com [10.3.116.91]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u6BFtHHv022101; Mon, 11 Jul 2016 11:55:22 -0400 From: Laszlo Ersek To: edk2-devel-01 Date: Mon, 11 Jul 2016 17:55:14 +0200 Message-Id: <1468252514-9533-3-git-send-email-lersek@redhat.com> In-Reply-To: <1468252514-9533-1-git-send-email-lersek@redhat.com> References: <1468252514-9533-1-git-send-email-lersek@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 11 Jul 2016 15:55:23 +0000 (UTC) Subject: [edk2] [PATCH 2/2] BaseTools/GenFds: speed up Region.PadBuffer() X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Liming Gao MIME-Version: 1.0 Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" The current implementation calls both pack() and Buffer.write() Size times. The new implementation calls both of these methods only once; the full data to write are constructed locally [1]. The range() function is replaced by xrange() because the latter is supposed to be faster / lighter weight [2]. On my laptop, I tested the change as follows: I pre-built the series at [3] with build -a X64 -p OvmfPkg/OvmfPkgX64.dsc -t GCC48 -b DEBUG \ -D HTTP_BOOT_ENABLE -D SECURE_BOOT_ENABLE (The series at [3] is relevant because it increases the size of one of the padded regions by 8.5 MB, slowing down the build quite a bit.) With all source code already compiled, repeating the above command takes approximately 45 seconds. With the patch applied, it goes down to 29 seconds. [1] http://stackoverflow.com/questions/27384093/fastest-way-to-write-huge-data-in-file [2] https://docs.python.org/2/library/functions.html?highlight=xrange#xrange [3] http://thread.gmane.org/gmane.comp.bios.edk2.devel/14214 We can also measure the impact with a synthetic test: > import timeit > > test_old = """ > import struct, string, StringIO > Size = (8 * 1024 + 512) * 1024 > Buffer = StringIO.StringIO() > PadData = 0xFF > for i in range(0, Size): > Buffer.write(struct.pack('B', PadData)) > """ > > test_new = """ > import struct, string, StringIO > Size = (8 * 1024 + 512) * 1024 > Buffer = StringIO.StringIO() > PadByte = struct.pack('B', 0xFF) > PadData = string.join(PadByte for i in xrange(0, Size)) > Buffer.write(PadData) > """ > > print(timeit.repeat(stmt=test_old, number=1, repeat=3)) > print(timeit.repeat(stmt=test_new, number=1, repeat=3)) The output is [8.231637001037598, 8.81188416481018, 8.948754072189331] [0.5503702163696289, 0.5461571216583252, 0.578315019607544] Cc: Yonghong Zhu Cc: Liming Gao Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Laszlo Ersek --- BaseTools/Source/Python/GenFds/Region.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) -- 1.8.3.1 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel diff --git a/BaseTools/Source/Python/GenFds/Region.py b/BaseTools/Source/Python/GenFds/Region.py index 6769b39ba7e8..7548a4f014b1 100644 --- a/BaseTools/Source/Python/GenFds/Region.py +++ b/BaseTools/Source/Python/GenFds/Region.py @@ -18,6 +18,7 @@ from struct import * from GenFdsGlobalVariable import GenFdsGlobalVariable import StringIO +import string from CommonDataClass.FdfClass import RegionClassObject import Common.LongFilePathOs as os from stat import * @@ -52,11 +53,11 @@ class Region(RegionClassObject): def PadBuffer(self, Buffer, ErasePolarity, Size): if Size > 0: if (ErasePolarity == '1') : - PadData = 0xFF + PadByte = pack('B', 0xFF) else: - PadData = 0 - for i in range(0, Size): - Buffer.write(pack('B', PadData)) + PadByte = pack('B', 0) + PadData = string.join(PadByte for i in xrange(0, Size)) + Buffer.write(PadData) ## AddToBuffer() #