From patchwork Fri Apr 4 09:15:04 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koen Kooi X-Patchwork-Id: 27797 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pb0-f69.google.com (mail-pb0-f69.google.com [209.85.160.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 15FCD20490 for ; Fri, 4 Apr 2014 09:15:33 +0000 (UTC) Received: by mail-pb0-f69.google.com with SMTP id md12sf11415889pbc.8 for ; Fri, 04 Apr 2014 02:15:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:cc:subject:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:content-type:content-transfer-encoding; bh=YxcRNaTOWAa/KSDcfazkKbTjtRIJOuneyna2scb7yKo=; b=acEvy6ccx1/I70qUWPZrKMPQ+87ryYPi09E39lpBf99poDDdH85aELRvyPPnvjc/Z4 xZw9MIsJwr97kOp7Ijg+M4X9/sZdUkrjTupAs2ripL7H3KsEc5HYnQg2y6IovDtKqKfj zj5Zm58JHxFN2lSNpPYjJq+7hheVbBpFIl8/YhIE6/Hd49LgSIF3PnXJs1z4NvlEXvl2 5U6HrzHEreFtvmnwiyl1uy64ClT+XMwmTnYGBnjCzXFznvRCvjecKMeZGE2+c4MhfvyT v+zubS20BAncHC7KX0sdTSE4lPZLmZpdSbihqQWkuJsTJXxLJWe/Sk5tq2IyWUiqfkoi MbMQ== X-Gm-Message-State: ALoCoQl90dlJzyJOsiv0cWUUt6fxpgq8Qh/GZUgDndcFtHK4B9UXTpV0l2Ip3Wcql0UR3TMJDKdX X-Received: by 10.66.221.137 with SMTP id qe9mr7545326pac.4.1396602933006; Fri, 04 Apr 2014 02:15:33 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.51.137 with SMTP id u9ls953864qga.81.gmail; Fri, 04 Apr 2014 02:15:32 -0700 (PDT) X-Received: by 10.220.105.4 with SMTP id r4mr47563vco.27.1396602932875; Fri, 04 Apr 2014 02:15:32 -0700 (PDT) Received: from mail-ve0-f173.google.com (mail-ve0-f173.google.com [209.85.128.173]) by mx.google.com with ESMTPS id jl10si687958veb.69.2014.04.04.02.15.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 04 Apr 2014 02:15:32 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.173 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.173; Received: by mail-ve0-f173.google.com with SMTP id oy12so1239409veb.32 for ; Fri, 04 Apr 2014 02:15:32 -0700 (PDT) X-Received: by 10.59.9.38 with SMTP id dp6mr169096ved.24.1396602932713; Fri, 04 Apr 2014 02:15:32 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.12.8 with SMTP id v8csp97850vcv; Fri, 4 Apr 2014 02:15:25 -0700 (PDT) X-Received: by 10.66.177.168 with SMTP id cr8mr13863887pac.128.1396602924954; Fri, 04 Apr 2014 02:15:24 -0700 (PDT) Received: from mail.openembedded.org (mail.openembedded.org. [140.211.169.62]) by mx.google.com with ESMTP id bn1si4356017pad.103.2014.04.04.02.15.24 for ; Fri, 04 Apr 2014 02:15:24 -0700 (PDT) Received-SPF: pass (google.com: domain of openembedded-core-bounces@lists.openembedded.org designates 140.211.169.62 as permitted sender) client-ip=140.211.169.62; Received: from mail.openembedded.org (localhost [127.0.0.1]) by mail.openembedded.org (Postfix) with ESMTP id D08EF6E7D3; Fri, 4 Apr 2014 09:15:18 +0000 (UTC) X-Original-To: openembedded-core@lists.openembedded.org Delivered-To: openembedded-core@lists.openembedded.org Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mail.openembedded.org (Postfix) with ESMTP id 3A60C60144 for ; Fri, 4 Apr 2014 09:15:12 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hi2so821263wib.17 for ; Fri, 04 Apr 2014 02:15:13 -0700 (PDT) X-Received: by 10.181.13.112 with SMTP id ex16mr2762135wid.23.1396602913062; Fri, 04 Apr 2014 02:15:13 -0700 (PDT) Received: from localhost ([2001:610:612:0:2ad2:44ff:fe4b:bc17]) by mx.google.com with ESMTPSA id u46sm18216714eel.1.2014.04.04.02.15.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Apr 2014 02:15:12 -0700 (PDT) From: Koen Kooi To: openembedded-core@lists.openembedded.org Date: Fri, 4 Apr 2014 11:15:04 +0200 Message-Id: <1396602904-14596-1-git-send-email-koen.kooi@linaro.org> X-Mailer: git-send-email 1.9.0 Cc: Paul Eggleton , Koen Kooi Subject: [OE-core] [Dora][PATCH] buildhistory-diff: add ability to report version changes X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: openembedded-core-bounces@lists.openembedded.org Errors-To: openembedded-core-bounces@lists.openembedded.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: koen.kooi@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.173 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 From: Paul Eggleton Add a -v/--report-ver option to report changes in PKGE/PKGV/PKGR even if the value is the same as the default from PE/PV/PR. Also add a -a/--report-all option to report all changes instead of just the default significant ones. Addresses [YOCTO #5263]. Signed-off-by: Paul Eggleton Signed-off-by: Saul Wold (From OE-Core master rev: b7de1eaac9eed559b2d68058f5de67de74a6cb58) Dora backport a51d96c44e6feac8322284c54bfc01ef598f8821 broke buildhistory-diff: koen@beast:/build/v2013.12$ buildhistory-diff Traceback (most recent call last): File "/build/v2013.12/sources/openembedded-core/scripts/buildhistory-diff", line 98, in main() File "/build/v2013.12/sources/openembedded-core/scripts/buildhistory-diff", line 82, in main changes = oe.buildhistory_analysis.process_changes(options.buildhistory_dir, fromrev, torev) File "/build/v2013.12/sources/openembedded-core/meta/lib/oe/buildhistory_analysis.py", line 403, in process_changes changes.extend(compare_dict_blobs(path, d.a_blob, d.b_blob, report_all, report_ver)) Cherry-pick the commit that the backport needs. And since it's a cherry-pick: Signed-off-by: Koen Kooi --- meta/lib/oe/buildhistory_analysis.py | 41 +++++++++++++++++++----------------- scripts/buildhistory-diff | 8 ++++++- 2 files changed, 29 insertions(+), 20 deletions(-) diff --git a/meta/lib/oe/buildhistory_analysis.py b/meta/lib/oe/buildhistory_analysis.py index ffe12d0..ed3a1ec 100644 --- a/meta/lib/oe/buildhistory_analysis.py +++ b/meta/lib/oe/buildhistory_analysis.py @@ -19,10 +19,11 @@ import bb.utils # How to display fields list_fields = ['DEPENDS', 'RPROVIDES', 'RDEPENDS', 'RRECOMMENDS', 'RSUGGESTS', 'RREPLACES', 'RCONFLICTS', 'FILES', 'FILELIST', 'USER_CLASSES', 'IMAGE_CLASSES', 'IMAGE_FEATURES', 'IMAGE_LINGUAS', 'IMAGE_INSTALL', 'BAD_RECOMMENDATIONS'] list_order_fields = ['PACKAGES'] -defaultval_fields = ['PKG', 'PKGE', 'PKGV', 'PKGR'] +defaultval_map = {'PKG': 'PKG', 'PKGE': 'PE', 'PKGV': 'PV', 'PKGR': 'PR'} numeric_fields = ['PKGSIZE', 'IMAGESIZE'] # Fields to monitor -monitor_fields = ['RPROVIDES', 'RDEPENDS', 'RRECOMMENDS', 'RREPLACES', 'RCONFLICTS', 'PACKAGES', 'FILELIST', 'PKGSIZE', 'IMAGESIZE', 'PKG', 'PKGE', 'PKGV', 'PKGR'] +monitor_fields = ['RPROVIDES', 'RDEPENDS', 'RRECOMMENDS', 'RREPLACES', 'RCONFLICTS', 'PACKAGES', 'FILELIST', 'PKGSIZE', 'IMAGESIZE', 'PKG'] +ver_monitor_fields = ['PKGE', 'PKGV', 'PKGR'] # Percentage change to alert for numeric fields monitor_numeric_threshold = 10 # Image files to monitor (note that image-info.txt is handled separately) @@ -94,7 +95,7 @@ class ChangeRecord: else: percentchg = 100 out = '%s changed from %s to %s (%s%d%%)' % (self.fieldname, self.oldvalue or "''", self.newvalue or "''", '+' if percentchg > 0 else '', percentchg) - elif self.fieldname in defaultval_fields: + elif self.fieldname in defaultval_map: out = '%s changed from %s to %s' % (self.fieldname, self.oldvalue, self.newvalue) if self.fieldname == 'PKG' and '[default]' in self.newvalue: out += ' - may indicate debian renaming failure' @@ -305,24 +306,32 @@ def compare_pkg_lists(astr, bstr): return (depvera, depverb) -def compare_dict_blobs(path, ablob, bblob, report_all): +def compare_dict_blobs(path, ablob, bblob, report_all, report_ver): adict = blob_to_dict(ablob) bdict = blob_to_dict(bblob) pkgname = os.path.basename(path) + defaultvals = {} defaultvals['PKG'] = pkgname - defaultvals['PKGE'] = adict.get('PE', '0') - defaultvals['PKGV'] = adict.get('PV', '') - defaultvals['PKGR'] = adict.get('PR', '') - for key in defaultvals: - defaultvals[key] = '%s [default]' % defaultvals[key] + defaultvals['PKGE'] = '0' changes = [] - keys = list(set(adict.keys()) | set(bdict.keys())) + keys = list(set(adict.keys()) | set(bdict.keys()) | set(defaultval_map.keys())) for key in keys: astr = adict.get(key, '') bstr = bdict.get(key, '') + if key in ver_monitor_fields: + monitored = report_ver or astr or bstr + else: + monitored = key in monitor_fields + mapped_key = defaultval_map.get(key, '') + if mapped_key: + if not astr: + astr = '%s [default]' % adict.get(mapped_key, defaultvals.get(key, '')) + if not bstr: + bstr = '%s [default]' % bdict.get(mapped_key, defaultvals.get(key, '')) + if astr != bstr: if (not report_all) and key in numeric_fields: aval = int(astr or 0) @@ -350,18 +359,12 @@ def compare_dict_blobs(path, ablob, bblob, report_all): if ' '.join(alist) == ' '.join(blist): continue - if key in defaultval_fields: - if not astr: - astr = defaultvals[key] - elif not bstr: - bstr = defaultvals[key] - - chg = ChangeRecord(path, key, astr, bstr, key in monitor_fields) + chg = ChangeRecord(path, key, astr, bstr, monitored) changes.append(chg) return changes -def process_changes(repopath, revision1, revision2 = 'HEAD', report_all = False): +def process_changes(repopath, revision1, revision2='HEAD', report_all=False, report_ver=False): repo = git.Repo(repopath) assert repo.bare == False commit = repo.commit(revision1) @@ -373,7 +376,7 @@ def process_changes(repopath, revision1, revision2 = 'HEAD', report_all = False) if path.startswith('packages/'): filename = os.path.basename(d.a_blob.path) if filename == 'latest': - changes.extend(compare_dict_blobs(path, d.a_blob, d.b_blob, report_all)) + changes.extend(compare_dict_blobs(path, d.a_blob, d.b_blob, report_all, report_ver)) elif filename.startswith('latest.'): chg = ChangeRecord(path, filename, d.a_blob.data_stream.read(), d.b_blob.data_stream.read(), True) changes.append(chg) diff --git a/scripts/buildhistory-diff b/scripts/buildhistory-diff index b82240d..ad50414 100755 --- a/scripts/buildhistory-diff +++ b/scripts/buildhistory-diff @@ -27,6 +27,12 @@ def main(): parser.add_option("-p", "--buildhistory-dir", help = "Specify path to buildhistory directory (defaults to buildhistory/ under cwd)", action="store", dest="buildhistory_dir", default='buildhistory/') + parser.add_option("-v", "--report-version", + help = "Report changes in PKGE/PKGV/PKGR even when the values are still the default (PE/PV/PR)", + action="store_true", dest="report_ver", default=False) + parser.add_option("-a", "--report-all", + help = "Report all changes, not just the default significant ones", + action="store_true", dest="report_all", default=False) options, args = parser.parse_args(sys.argv) @@ -79,7 +85,7 @@ def main(): import gitdb try: - changes = oe.buildhistory_analysis.process_changes(options.buildhistory_dir, fromrev, torev) + changes = oe.buildhistory_analysis.process_changes(options.buildhistory_dir, fromrev, torev, options.report_all, options.report_ver) except gitdb.exc.BadObject as e: if len(args) == 1: sys.stderr.write("Unable to find previous build revision in buildhistory repository\n\n")