From patchwork Wed Oct 17 16:33:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milo Casagrande X-Patchwork-Id: 12321 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 5749B1D5170 for ; Wed, 17 Oct 2012 16:33:14 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id DAAFFA185F0 for ; Wed, 17 Oct 2012 16:33:13 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id e10so11699640iej.11 for ; Wed, 17 Oct 2012 09:33:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=I9TdK+7oH+B4e1AjMFVErgn4sqDkiyFLD8QSJSiKVuI=; b=gxIUwlAHLCfzvZuDw/EItYrp4HdxjAbaQc01btG53jOQ8DpbPoRLee35IcpPqa5bxM HUYctM0mzjMZT/ScihP/4VXZ/cT0J3gULkidHvW4O3/reRWRYxin101YAX5rjtj5V2qV ZfBfIA6lZUHU12ohTY+0Xs0zqFp+ofw87LtM7vzXeLi90ODQdJ3C71o4QZ6AgarRU6BZ VhJTLA3ARk9xIdKc3jEATAG5kKIXlx3wIG+u2D8RKrs0ONK2D0lvfz1tjMkmbFSXsKCs 9FaorbTD5PnIWq4nc+LbrkDKlLhG2VX7sus+ChLrwtJiJvLcHRZI4moIQ6IsUpz51f3Z PwLg== Received: by 10.50.140.97 with SMTP id rf1mr480562igb.70.1350491593327; Wed, 17 Oct 2012 09:33:13 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp923677igt; Wed, 17 Oct 2012 09:33:12 -0700 (PDT) Received: by 10.180.81.37 with SMTP id w5mr5238609wix.10.1350491591755; Wed, 17 Oct 2012 09:33:11 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id i20si24658319wej.51.2012.10.17.09.33.11 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 17 Oct 2012 09:33:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TOWYR-0003Rb-3i for ; Wed, 17 Oct 2012 16:33:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 0C215E01A6 for ; Wed, 17 Oct 2012 16:33:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: linaro-image-tools X-Launchpad-Branch: ~linaro-image-tools/linaro-image-tools/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 579 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-image-tools/linaro-image-tools/trunk] Rev 579: Code refactoring for build() method. Message-Id: <20121017163311.12558.42636.launchpad@ackee.canonical.com> Date: Wed, 17 Oct 2012 16:33:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16152"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: b9a235ef857591e5b31e7886c289ea89d435258c X-Gm-Message-State: ALoCoQn7ZNY9OHZBi6vMfOehC7u7CeWGcH1jlrRq8nKbS4PrUoR8BmknHGySJrH6jD+kt/giDejm Merge authors: Milo Casagrande (milo) Related merge proposals: https://code.launchpad.net/~milo/linaro-image-tools/method-refactor/+merge/130123 proposed by: Milo Casagrande (milo) review: Approve - James Tunnicliffe (dooferlad) ------------------------------------------------------------ revno: 579 [merge] committer: Milo Casagrande branch nick: trunk timestamp: Wed 2012-10-17 18:32:37 +0200 message: Code refactoring for build() method. modified: linaro_image_tools/hwpack/builder.py --- lp:linaro-image-tools https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk You are subscribed to branch lp:linaro-image-tools. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk/+edit-subscription === modified file 'linaro_image_tools/hwpack/builder.py' --- linaro_image_tools/hwpack/builder.py 2012-10-12 12:31:22 +0000 +++ linaro_image_tools/hwpack/builder.py 2012-10-17 16:04:35 +0000 @@ -270,125 +270,166 @@ self.packages, download_content=self.config.include_debs) - # On a v3 hwpack, all the values we need to check are - # in the bootloaders and boards section, so we loop - # through both of them changing what is necessary. - - if self.config.format.format_as_string == '3.0': + if self.format.format_as_string == '3.0': self.extract_files() else: - bootloader_package = None - if self.config.bootloader_file is not None: - assert(self.config.bootloader_package - is not None) - bootloader_package = self.find_fetched_package( - self.packages, - self.config.bootloader_package) - self.hwpack.metadata.u_boot = \ - self.add_file_to_hwpack( - bootloader_package, - self.config.bootloader_file, - self.hwpack.U_BOOT_DIR) - - spl_package = None - if self.config.spl_file is not None: - assert self.config.spl_package is not None - spl_package = self.find_fetched_package( - self.packages, - self.config.spl_package) - self.hwpack.metadata.spl = \ - self.add_file_to_hwpack( - spl_package, - self.config.spl_file, - self.hwpack.SPL_DIR) - - # bootloader_package and spl_package can be - # identical - if (bootloader_package is not None and - bootloader_package in self.packages): - self.packages.remove(bootloader_package) - if (spl_package is not None and - spl_package in self.packages): - self.packages.remove(spl_package) - - logger.debug("Adding packages to hwpack") - self.hwpack.add_packages(self.packages) - for local_package in local_packages: - if local_package not in self.packages: - logger.warning( - "Local package '%s' not included", - local_package.name) - self.hwpack.add_dependency_package( - self.config.packages) + self._old_format_extract_files() + + self._add_packages_to_hwpack(local_packages) + out_name = self.out_name if not out_name: out_name = self.hwpack.filename() - with open(out_name, 'w') as f: - self.hwpack.to_file(f) - logger.info("Wrote %s" % out_name) + manifest_name = os.path.splitext(out_name)[0] if manifest_name.endswith('.tar'): manifest_name = os.path.splitext(manifest_name)[0] manifest_name += '.manifest.txt' - with open(manifest_name, 'w') as f: - f.write(self.hwpack.manifest_text()) - - logger.debug("Extracting build-info") - build_info_dir = os.path.join(fetcher.cache.tempdir, - 'build-info') - build_info_available = 0 - for deb_pkg in self.packages: - deb_pkg_file_path = deb_pkg.filepath - if os.path.islink(deb_pkg_file_path): - # Skip symlink-ed debian package file - # e.g. fetched package with dummy information - continue - try: - # Extract Build-Info attribute from debian - # control - deb_control = \ - DebFile(deb_pkg_file_path).control.debcontrol() - build_info = deb_control.get('Build-Info') - except ArError: - # Skip invalid debian package file - # e.g. fetched package with dummy information - continue - if build_info is not None: - build_info_available += 1 - # Extract debian packages with build - # information - env = os.environ - env['LC_ALL'] = 'C' - env['NO_PKG_MANGLE'] = '1' - proc = cmd_runner.Popen(['dpkg-deb', '-x', - deb_pkg_file_path, build_info_dir], - env=env, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - (stdoutdata, stderrdata) = proc.communicate() - if proc.returncode: - raise ValueError('dpkg-deb extract failed!' - '\n%s' % stderrdata) - if stderrdata: - raise ValueError('dpkg-deb extract had ' - 'warnings:\n%s' % stderrdata) - - # Concatenate BUILD-INFO.txt files - dst_file = open('BUILD-INFO.txt', 'wb') - if build_info_available > 0: - build_info_path = \ - r'%s/usr/share/doc/*/BUILD-INFO.txt' % \ - build_info_dir - for src_file in iglob(build_info_path): - with open(src_file, 'rb') as f: - dst_file.write('\nFiles-Pattern: %s\n' % \ - out_name) - shutil.copyfileobj(f, dst_file) - dst_file.write('\nFiles-Pattern: %s\n' - 'License-Type: open\n' %\ - manifest_name) - else: - dst_file.write('Format-Version: 0.1\n' - 'Files-Pattern: %s, %s\n' - 'License-Type: open\n' % \ - (out_name, manifest_name)) - dst_file.close() + + self._write_hwpack_and_manifest(out_name, + manifest_name) + + cache_dir = fetcher.cache.tempdir + self._extract_build_info(cache_dir, out_name, + manifest_name) + + def _write_hwpack_and_manifest(self, out_name, manifest_name): + """Write the real hwpack file and its manifest file. + + :param out_name: The name of the file to write. + :type out_name: str + :param manifest_name: The name of the manifest file. + :type manifest_name: str + """ + logger.debug("Writing hwpack file") + with open(out_name, 'w') as f: + self.hwpack.to_file(f) + logger.info("Wrote %s" % out_name) + + logger.debug("Writing manifest file content") + with open(manifest_name, 'w') as f: + f.write(self.hwpack.manifest_text()) + + def _old_format_extract_files(self): + """Extract files for hwpack versions < 3.0.""" + bootloader_package = None + if self.config.bootloader_file is not None: + assert(self.config.bootloader_package is not None) + bootloader_package = self.find_fetched_package( + self.packages, + self.config.bootloader_package) + self.hwpack.metadata.u_boot = self.add_file_to_hwpack( + bootloader_package, + self.config.bootloader_file, + self.hwpack.U_BOOT_DIR) + + spl_package = None + if self.config.spl_file is not None: + assert self.config.spl_package is not None + spl_package = self.find_fetched_package(self.packages, + self.config.spl_package) + self.hwpack.metadata.spl = self.add_file_to_hwpack( + spl_package, + self.config.spl_file, + self.hwpack.SPL_DIR) + + # bootloader_package and spl_package can be identical + if (bootloader_package is not None and + bootloader_package in self.packages): + self.packages.remove(bootloader_package) + if (spl_package is not None and spl_package in self.packages): + self.packages.remove(spl_package) + + def _add_packages_to_hwpack(self, local_packages): + """Adds the packages to the hwpack. + + :param local_packages: The packages to add. + :type local_packages: list + """ + logger.debug("Adding packages to hwpack") + self.hwpack.add_packages(self.packages) + for local_package in local_packages: + if local_package not in self.packages: + logger.warning("Local package '%s' not included", + local_package.name) + self.hwpack.add_dependency_package(self.config.packages) + + def _extract_build_info(self, cache_dir, out_name, manifest_name): + """Extracts build-info from the packages. + + :param cache_dir: The cache directory where build-info should be + located. + :type cache_dir: str + :param out_name: The name of the hwpack file. + :type out_name: str + :param manifest_name: The name of the manifest file. + :type manifest_name: str + """ + logger.debug("Extracting build-info") + build_info_dir = os.path.join(cache_dir, 'build-info') + build_info_available = 0 + for deb_pkg in self.packages: + deb_pkg_file_path = deb_pkg.filepath + if os.path.islink(deb_pkg_file_path): + # Skip symlink-ed debian package file + # e.g. fetched package with dummy information + continue + try: + # Extract Build-Info attribute from debian control + deb_control = DebFile(deb_pkg_file_path).control.debcontrol() + build_info = deb_control.get('Build-Info') + except ArError: + # Skip invalid debian package file + # e.g. fetched package with dummy information + continue + if build_info is not None: + build_info_available += 1 + # Extract debian packages with build information + env = os.environ + env['LC_ALL'] = 'C' + env['NO_PKG_MANGLE'] = '1' + proc = cmd_runner.Popen(['dpkg-deb', '-x', + deb_pkg_file_path, build_info_dir], + env=env, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + + (stdoutdata, stderrdata) = proc.communicate() + if proc.returncode: + raise ValueError('dpkg-deb extract failed!\n%s' % + stderrdata) + if stderrdata: + raise ValueError('dpkg-deb extract had warnings:\n%s' % + stderrdata) + + self._concatenate_build_info(build_info_available, build_info_dir, + out_name, manifest_name) + + def _concatenate_build_info(self, build_info_available, build_info_dir, + out_name, manifest_name): + """Concatenates the build-info text if more than one is available. + + :param build_info_available: The number of available build-info. + :type build_info_available: int + :param build_info_dir: Where build-info files should be. + :type build_info_dir: str + :param out_name: The name of the hwpack file. + :type out_name: str + :param manifest_name: The name of the manifest file. + :type manifest_name: str + """ + logger.debug("Concatenating build-info files") + dst_file = open('BUILD-INFO.txt', 'wb') + if build_info_available > 0: + build_info_path = (r'%s/usr/share/doc/*/BUILD-INFO.txt' % + build_info_dir) + for src_file in iglob(build_info_path): + with open(src_file, 'rb') as f: + dst_file.write('\nFiles-Pattern: %s\n' % out_name) + shutil.copyfileobj(f, dst_file) + dst_file.write('\nFiles-Pattern: %s\nLicense-Type: open\n' % + manifest_name) + else: + dst_file.write('Format-Version: 0.1\n' + 'Files-Pattern: %s, %s\n' + 'License-Type: open\n' % (out_name, manifest_name)) + dst_file.close()