From patchwork Fri Nov 16 17:35:15 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Doan X-Patchwork-Id: 12901 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id E7F9223DFE for ; Fri, 16 Nov 2012 17:35:19 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 64ACAA19D2D for ; Fri, 16 Nov 2012 17:35:19 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id e10so3817467iej.11 for ; Fri, 16 Nov 2012 09:35:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=i3rQpQKCtsdhIkd3xq8mO8q1YEYoHoYSKMh7XLSKPso=; b=noQ5+mnprzHEWYWND3JHw8FlKHLo9c5hW3H+4unmy4meFICbPNRhJPhZ6wBeHGcmH9 gSYmgsD+yjYgPVM+BScsfwFOllBpO6f2tRK7zPKJOk2NBwvohCrMrNEpr8WzX5UAJ0ny Sa+CYTOehnvf+MUJaK6QeKb0pROyubEaGM+xUY3r+qsEUTQj2rUMOzhAtmdxnEz5DCGk 3LqWzXrFYZWydBZP7XgC6O3ylumE0NrYj25Rs3VP8dgJ4MMT3x/jpIUU489rhiz0ace7 0lMPI2NEUXHrYX2O+VgyEJbxckoXKz4PiTk6qKBuJGiyus9U1OLvkDAwbyWwnLSOaXYn fsgw== Received: by 10.50.213.34 with SMTP id np2mr4005447igc.57.1353087318877; Fri, 16 Nov 2012 09:35:18 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp102706igt; Fri, 16 Nov 2012 09:35:17 -0800 (PST) Received: by 10.180.8.234 with SMTP id u10mr6229540wia.17.1353087316849; Fri, 16 Nov 2012 09:35:16 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id x20si1315100weo.96.2012.11.16.09.35.15 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 16 Nov 2012 09:35:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1TZPox-0007ip-PA for ; Fri, 16 Nov 2012 17:35:15 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id AF6D5E0017 for ; Fri, 16 Nov 2012 17:35:15 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 452 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 452: make fastmodel implementation more flexible Message-Id: <20121116173515.26571.77879.launchpad@ackee.canonical.com> Date: Fri, 16 Nov 2012 17:35:15 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16278"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: b10f39c7f75a434bcef2040784b29ef45e140524 X-Gm-Message-State: ALoCoQlYXOPlcn96AeAUg9eoZU6FDAuaHyL4qSZv26JFGwURx0xGL26tQa5hVWwcjCf5uxCyYUX6 Merge authors: Andy Doan (doanac) Related merge proposals: https://code.launchpad.net/~doanac/lava-dispatcher/fastmodel-refactor/+merge/134581 proposed by: Andy Doan (doanac) review: Approve - Michael Hudson-Doyle (mwhudson) ------------------------------------------------------------ revno: 452 [merge] committer: Andy Doan branch nick: lava-dispatcher timestamp: Fri 2012-11-16 11:34:35 -0600 message: make fastmodel implementation more flexible added: lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_v8.conf lava_dispatcher/default-config/lava-dispatcher/device-types/foundationsmodel.conf renamed: lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf => lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_A15x4-A7x4.conf modified: doc/changes.rst lava_dispatcher/config.py lava_dispatcher/device/fastmodel.py lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_A15x4-A7x4.conf --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'doc/changes.rst' --- doc/changes.rst 2012-11-08 03:46:23 +0000 +++ doc/changes.rst 2012-11-16 17:34:35 +0000 @@ -6,6 +6,11 @@ Version 0.22 ============ * Unreleased +* refactor fastmodel implementation to not require code changes for new products +* simplify power_off/sync logic in targets +* boot_options improvements +* extract_tarball API added to target +* change lava-test-shell defintion format to be YAML .. _verison_0_21: === modified file 'lava_dispatcher/config.py' --- lava_dispatcher/config.py 2012-11-14 16:34:06 +0000 +++ lava_dispatcher/config.py 2012-11-16 00:47:20 +0000 @@ -67,9 +67,9 @@ val = schema.StringOption() sdcard_mountpoint_path = schema.StringOption(default="/storage/sdcard0") - simulator_binary = schema.StringOption() - license_server = schema.StringOption() - fastmodel_type = schema.StringOption() + simulator_version_command = schema.StringOption() + simulator_command = schema.StringOption() + simulator_axf_files = schema.ListOption() class OptionDescriptor(object): def __init__(self, name): === renamed file 'lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf' => 'lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_A15x4-A7x4.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf 2012-11-14 16:34:06 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_A15x4-A7x4.conf 2012-11-16 17:34:35 +0000 @@ -1,11 +1,4 @@ client_type=fastmodel -simulator_binary = /opt/arm/RTSM_A15x14-A7x14_VE/bin/RTSM_VE_Cortex-A15x4-A7x4 - -# The type of the fastmodel (ve or foundation) -fastmodel_type = ve - -# The license server must also be specified. eg: -#license_server = 8224@192.168.1.10 # how long the disablesuspend script should take to complete # fm takes longer than other android images do @@ -20,6 +13,14 @@ # we do usermode networking over the loopback default_network_interface = lo +simulator_axf_files = + img.axf + linux-system-ISW.axf + +simulator_version_command = /opt/arm/RTSM_A15-A7x14_VE/Linux64_RTSM_VE_Cortex-A15x4-A7x4/RTSM_VE_Cortex-A15x4-A7x4 --version | grep "Fast Models" | sed 's/Fast Models \[//' | sed 's/\]//' + +simulator_command = sudo -u www-data ARMLMD_LICENSE_FILE="8224@localhost" /opt/arm/RTSM_A15-A7x14_VE/Linux64_RTSM_VE_Cortex-A15x4-A7x4/RTSM_VE_Cortex-A15x4-A7x4 -a coretile.cluster0.*={AXF} -C motherboard.mmc.p_mmc_file={IMG} -C motherboard.hostbridge.userNetPorts='5555=5555' + boot_options = motherboard.smsc_91c111.enabled motherboard.hostbridge.userNetworking === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_v8.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_v8.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel_v8.conf 2012-11-16 00:47:20 +0000 @@ -0,0 +1,23 @@ +client_type=fastmodel + +# The license server must also be specified. eg: +license_server = 8224@localhost + +# how long the disablesuspend script should take to complete +# fm takes longer than other android images do +disablesuspend_timeout = 500 + +# how long ubuntu takes to boot to prompt +boot_linaro_timeout = 500 + +# if you do dhcp on boot, adb will not work (asac) on fastmodels +enable_network_after_boot_android = 0 + +# we do usermode networking over the loopback +default_network_interface = lo + +simulator_axf_files = linux-system.axf + +simulator_version_command = /opt/arm/RTSM_AEMv8/ModelDebugger_7.1/bin/model_shell64 --version | grep "Model Shell" | sed 's/Model Shell //' + +simulator_command = sudo -u www-data ARMLMD_LICENSE_FILE="8224@localhost" /opt/arm/RTSM_AEMv8/ModelDebugger_7.1/bin/model_shell64 -a {AXF} /opt/arm/RTSM_AEMv8/lib/Linux64/RTSM_VE_AEMv8A.so -C motherboard.mmc.p_mmc_file={IMG} === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/foundationsmodel.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/foundationsmodel.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/foundationsmodel.conf 2012-11-16 00:47:20 +0000 @@ -0,0 +1,20 @@ +client_type=fastmodel + +# how long the disablesuspend script should take to complete +# fm takes longer than other android images do +disablesuspend_timeout = 500 + +# how long ubuntu takes to boot to prompt +boot_linaro_timeout = 500 + +# if you do dhcp on boot, adb will not work (asac) on fastmodels +enable_network_after_boot_android = 0 + +# we do usermode networking over the loopback +default_network_interface = lo + +simulator_axf_files = img-foundation.axf + +simulator_version_command = /opt/arm/Foundation_v8pkg/Foundation_v8 --version | grep "ARM V8 Foundation Model" | sed 's/ARM V8 Foundation Model //' + +simulator_command = sudo -u www-data /opt/arm/Foundation_v8pkg/Foundation_v8 --image={AXF} --block-device={IMG} --network=nat === modified file 'lava_dispatcher/device/fastmodel.py' --- lava_dispatcher/device/fastmodel.py 2012-11-16 00:44:40 +0000 +++ lava_dispatcher/device/fastmodel.py 2012-11-16 00:47:20 +0000 @@ -59,30 +59,9 @@ ANDROID_WALLPAPER = 'system/wallpaper_info.xml' SYS_PARTITION = 2 DATA_PARTITION = 5 - FM_VE = 0 - FM_FOUNDATION = 1 - FASTMODELS = {'ve': FM_VE, 'foundation': FM_FOUNDATION} - AXF_IMAGES = {FM_VE: 'img.axf', FM_FOUNDATION: 'img-foundation.axf'} def __init__(self, context, config): super(FastModelTarget, self).__init__(context, config) - self._sim_binary = config.simulator_binary - if not self._sim_binary: - raise RuntimeError("Missing config option for simulator binary") - - try: - self._fastmodel_type = self.FASTMODELS[config.fastmodel_type] - except KeyError: - raise RuntimeError("The fastmodel type for this device is invalid," - " please use 've' or 'foundation'") - - if self._fastmodel_type == self.FM_VE: - lic_server = config.license_server - if not lic_server: - raise RuntimeError("The VE FastModel requires the config " - "option 'license_server'") - - os.putenv('ARMLMD_LICENSE_FILE', lic_server) self._sim_proc = None @@ -102,12 +81,20 @@ self.deployment_data = Target.android_deployment_data - def _copy_axf(self, partno, fname): + def _copy_axf(self, partno, subdir): + self._axf = None with image_partition_mounted(self._sd_image, partno) as mntdir: - src = '%s/%s' % (mntdir, fname) - odir = os.path.dirname(self._sd_image) - self._axf = '%s/%s' % (odir, os.path.split(src)[1]) - shutil.copyfile(src, self._axf) + subdir = os.path.join(mntdir, subdir) + for fname in self.config.simulator_axf_files: + src = os.path.join(subdir, fname) + if os.path.exists(src): + odir = os.path.dirname(self._sd_image) + self._axf = '%s/%s' % (odir, os.path.split(src)[1]) + shutil.copyfile(src, self._axf) + break + + if not self._axf: + raise RuntimeError('No AXF found, %r' % os.listdir(subdir)) def deploy_android(self, boot, system, data): logging.info("Deploying Android on %s" % self.config.hostname) @@ -122,7 +109,7 @@ 'vexpress-a9', self._boot, self._data, self._system, self._sd_image ) - self._copy_axf(self.config.boot_part, 'linux-system-ISW.axf') + self._copy_axf(self.config.boot_part, '') self._customize_android() @@ -133,14 +120,20 @@ generate_fastmodel_image(hwpack, rootfs, odir) self._sd_image = '%s/sd.img' % odir - self._axf = '%s/%s' % (odir, self.AXF_IMAGES[self._fastmodel_type]) + self._axf = None + for f in self.config.simulator_axf_files: + fname = os.path.join(odir, f) + if os.path.exists(fname): + self._axf = fname + break + if not self._axf: + raise RuntimeError('No AXF found, %r' % os.listdir(odir)) self._customize_linux(self._sd_image) def deploy_linaro_prebuilt(self, image): self._sd_image = download_image(image, self.context) - self._copy_axf(self.config.root_part, - 'boot/%s' % self.AXF_IMAGES[self._fastmodel_type]) + self._copy_axf(self.config.root_part, 'boot') self._customize_linux(self._sd_image) @@ -174,17 +167,6 @@ os.chown(self._axf, st.st_uid, st.st_gid) os.chown(self._sd_image, st.st_uid, st.st_gid) - def _get_sim_cmd(self): - options = boot_options.as_string(self, join_pattern=' -C %s=%s') - if self._fastmodel_type == self.FM_VE: - return ("%s -a coretile.cluster0.*=%s " - "-C motherboard.mmc.p_mmc_file=%s " - "-C motherboard.hostbridge.userNetPorts='5555=5555' %s") % ( - self._sim_binary, self._axf, self._sd_image, options) - elif self._fastmodel_type == self.FM_FOUNDATION: - return ("%s --image=%s --block-device=%s --network=nat %s") % ( - self._sim_binary, self._axf, self._sd_image, options) - def power_off(self, proc): super(FastModelTarget, self).power_off(proc) if self._sim_proc is not None: @@ -206,7 +188,11 @@ def power_on(self): self._fix_perms() - sim_cmd = self._get_sim_cmd() + + options = boot_options.as_string(self, join_pattern=' -C %s=%s') + sim_cmd = self.config.simulator_command.format( + AXF=self._axf, IMG=self._sd_image) + sim_cmd = '%s %s' % (sim_cmd, options) # the simulator proc only has stdout/stderr about the simulator # we hook up into a telnet port which emulates a serial console @@ -242,20 +228,12 @@ return [] def get_device_version(self): - cmd = '%s --version' % self._sim_binary + cmd = self.config.simulator_version_command try: - banner = subprocess.check_output(cmd, shell=True) - return self._parse_fastmodel_version(banner) + return subprocess.check_output(cmd, shell=True).strip() except subprocess.CalledProcessError: return "unknown" - def _parse_fastmodel_version(self, banner): - match = re.search('Fast Models \[([0-9.]+)', banner) - if match: - return match.group(1) - else: - return "unknown" - class _pexpect_drain(threading.Thread): ''' The simulator process can dump a lot of information to its console. If