From patchwork Fri Sep 9 05:11:13 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 3999 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id B5ED423EF5 for ; Fri, 9 Sep 2011 05:11:15 +0000 (UTC) Received: from mail-fx0-f52.google.com (mail-fx0-f52.google.com [209.85.161.52]) by fiordland.canonical.com (Postfix) with ESMTP id 96315A18A08 for ; Fri, 9 Sep 2011 05:11:15 +0000 (UTC) Received: by fxd18 with SMTP id 18so3253509fxd.11 for ; Thu, 08 Sep 2011 22:11:15 -0700 (PDT) Received: by 10.223.47.67 with SMTP id m3mr1118316faf.27.1315545075097; Thu, 08 Sep 2011 22:11:15 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.152.11.8 with SMTP id m8cs2953lab; Thu, 8 Sep 2011 22:11:14 -0700 (PDT) Received: by 10.227.20.6 with SMTP id d6mr1581550wbb.49.1315545073947; Thu, 08 Sep 2011 22:11:13 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com [91.189.90.7]) by mx.google.com with ESMTPS id fo18si5404109wbb.123.2011.09.08.22.11.13 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 08 Sep 2011 22:11:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1R1tMv-0007t5-B0 for ; Fri, 09 Sep 2011 05:11:13 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 481C3E033F for ; Fri, 9 Sep 2011 05:11:13 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 105 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 105: merge the config branch of doom. Message-Id: <20110909051113.7902.39243.launchpad@ackee.canonical.com> Date: Fri, 09 Sep 2011 05:11:13 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="13900"; Instance="initZopeless config overlay" X-Launchpad-Hash: a36f584ebb1a93c7a2392d05a4d9d095284f5b91 Merge authors: David Schwarz (david-schwarz) Michael Hudson-Doyle (mwhudson) Paul Larson (pwlars) Paul Larson Spring Zhang (qzhang) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-dispatcher/config-clients-merge/+merge/74712 proposed by: Michael Hudson-Doyle (mwhudson) review: Approve - Paul Larson (pwlars) review: Needs Fixing - Spring Zhang (qzhang) https://code.launchpad.net/~pwlars/lava-dispatcher/config-clients-merge/+merge/71414 proposed by: Paul Larson (pwlars) review: Needs Fixing - Spring Zhang (qzhang) https://code.launchpad.net/~david-schwarz/lava-dispatcher/ssh_qemu_clients_v3/+merge/70892 proposed by: David Schwarz (david-schwarz) ------------------------------------------------------------ revno: 105 [merge] committer: Michael-Doyle Hudson branch nick: trunk timestamp: Fri 2011-09-09 17:08:38 +1200 message: merge the config branch of doom. all configuration is now done through config files which can be overridden in site wide and user local locations. removed: lava_dispatcher/android_config.py added: MANIFEST.in lava_dispatcher/default-config/ lava_dispatcher/default-config/lava-dispatcher/ lava_dispatcher/default-config/lava-dispatcher/README lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf lava_dispatcher/default-config/lava-dispatcher/device-types/ lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf lava_dispatcher/default-config/lava-dispatcher/devices/ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf modified: .bzrignore README doc/QUICKSTART lava_dispatcher/__init__.py lava_dispatcher/actions/android_basic.py lava_dispatcher/actions/android_deploy.py lava_dispatcher/actions/deploy.py lava_dispatcher/actions/launch_control.py* lava_dispatcher/actions/lava-test.py* lava_dispatcher/android_client.py lava_dispatcher/android_util.py lava_dispatcher/client.py lava_dispatcher/config.py lava_dispatcher/tests/test_config.py lava_dispatcher/utils.py setup.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file '.bzrignore' --- .bzrignore 2011-04-07 01:21:51 +0000 +++ .bzrignore 2011-09-09 00:57:07 +0000 @@ -3,3 +3,6 @@ ./.virtualenv ./bin ./lava.egg-info +./dist +./lava_dispatcher.egg-info +./build === added file 'MANIFEST.in' --- MANIFEST.in 1970-01-01 00:00:00 +0000 +++ MANIFEST.in 2011-09-09 00:57:07 +0000 @@ -0,0 +1,2 @@ +include README +recursive-include lava_dispatcher/default-config *.conf \ No newline at end of file === modified file 'README' --- README 2011-06-27 04:55:08 +0000 +++ README 2011-09-09 04:53:56 +0000 @@ -1,33 +1,12 @@ -LAVA - Automated Validation for Linaro -====================================== - -LAVA is a bundle of software that allows automated testing of Linaro images. - -It contains code both for running the tests (the "dispatcher" in local jargon) -and requesting/organizing the running of the tests (the "scheduler"). - -Code-wise, it consists of three major parts: - - - the dispatcher - - the scheduler interface - - the scheduler daemon +LAVA dispatcher - automated testing of Linaro images +==================================================== + +The LAVA dispatcher allows automated testing of Linaro images. Quickstart ---------- -You need to have the python-virtualenv, python-pip, xxx, yyy and zzz -packages installed. - -To set up the virtual environment with all the Python dependencies, -run "make". You can load some sample data to bootstrap interactive playing -around with "make loadsampledata". - -Then, to run the interface, run "./bin/manage runserver" (after -running "make"). - -Running the dispatcher requires a bit of setting up. See below. - -To run the daemon ... +See doc/QUICKSTART. Android Debug Bridge (adb) installation @@ -44,36 +23,3 @@ You might want to add the location of the SDK's tools/ and platform-tools to your PATH environment variable, to provide easy access to the tools. - - -The Dispatcher --------------- - -The code for the dispatcher lives in lava_dispatcher. - -XXX write this -- reference ./doc/QUICKSTART or copy it here. - -The Scheduler Interface ------------------------ - -The code for the scheduler interface lives in -lava.scheduler.interface. - -It is a Django application that provides both a human (i.e. HTML) and -machine (i.e. XML-RPC) interface to the scheduling of jobs. It is a -fairly simple interface to a database in the traditional Django-Python -webapp style. - -It will delegate to Launchpad for auth and the API side will probably -use OAuth for authentication. - -To be able to use the admin interface, log in via Launchpad OpenID and -then run "bin/manage makesuperuser $your_lp_id". - - -The Scheduler Daemon --------------------- - -The scheduler daemon usage: - - * ./lava-schedulerd start|stop|restart === modified file 'doc/QUICKSTART' --- doc/QUICKSTART 2011-06-10 04:46:04 +0000 +++ doc/QUICKSTART 2011-09-09 04:53:56 +0000 @@ -1,14 +1,19 @@ +Dispatcher Quick Start +====================== + Dependencies ------------ -LAVA needs python >= 2.6. - -For the LAVA client, you need the pexpect module. +LAVA needs python >= 2.6. For the dispatcher, you need the pexpect +module. Configuring for local development --------------------------------- +Configuring conmux +++++++++++++++++++ + You will need to have a card containing a 'master image' for your board. The process of creating a master image is outlined on https://wiki.linaro.org/Platform/Validation/Specs/MasterBootImage. @@ -17,7 +22,8 @@ you should be able to make use of most features, even without the use of special equipment such as a console server. -First install conmux and cu: +First install conmux and cu:: + sudo add-apt-repository ppa:linaro-maintainers/tools sudo apt-get update sudo apt-get install conmux cu @@ -26,92 +32,110 @@ You may have permission problem with cu running as root under conmux. Create a configuration file for your board under /etc/conmux which -should look something like this: +should look something like this:: + listener panda01 application console 'panda01 console' 'cu -l /dev/ttyUSB0 -s 115200' + Make sure to give the file a '.cf' extension (e.g. panda01.cf). If you see this permission problem when running cu, you can try adjusting your .cf file to call cu using sg, and the group name owning -the device. For example: +the device. For example:: + sg dialout "cu -l ttyUSB0 -s 115200" -Finally restart conmux: +Finally restart conmux:: + sudo stop conmux sudo start conmux -You can test the connection using: +You can test the connection using:: + conmux-console panda01 (use ~$quit to exit) You should be able to type commands and interact with the shell inside conmux-console. If you cannot, run "sudo stop conmux" and try running 'sg dialout "cu -l ttyUSB0 -s 115200"'. If that doesn't work, you -probably need to add some files to /etc/uucp. Add - -port ttyUSB0 -type direct -device /dev/ttyUSB0 -hardflow false -speed 115200 - -to /etc/uucp/port and append - -system panda01 -port ttyUSB0 -time any +probably need to add some files to /etc/uucp. Add :: + + port ttyUSB0 + type direct + device /dev/ttyUSB0 + hardflow false + speed 115200 + +to /etc/uucp/port and append :: + + system panda01 + port ttyUSB0 + time any to /etc/uucp/sys. If this doesn't let you interact with the shell in conmux-console, complain in #linaro on freenode and hopefully someone will help you out :) -Make a directory which is defined as LAVA_IMAGE_TMPDIR in -lava/dispatcher/config.py: -mkdir -p /linaro/images/tmp - -Define LAVA_SERVER_IP to your local LAVA server if you want to use local one in -config.py. - -Setup a web server(like Apache), and make root directory as /linaro. - -Setup a dashboard server launch-control: -add-apt-repository ppa:zkrynicki/lava -apt-get update -apt-get purge launch-control (if there is an old one) -apt-get install launch-control-pqsql -apt-get install launch-control -sudo -u www-data python /usr/lib/launch-control/manage.py createsuperuser + +Configuring the dispatcher +++++++++++++++++++++++++++ + +The dispatcher looks for config values in ~/.config/lava-dispatcher/, +then /etc/xdg/lava-dispatcher, then its own source tree. + +To get started, you ned to make a directory for storing artefacts +before they are downloaded to the device being tested +(LAVA_IMAGE_TMPDIR in the config and /linaro/images/tmp by default):: + + # mkdir -p /linaro/images/tmp + +You will need to set LAVA_SERVER_IP to the address of the machine +running the dispatcher in ~/.config/lava-dispatcher/lava-dispatcher.conf:: + + $ cat ~/.config/lava-dispatcher/lava-dispatcher.conf + LAVA_SERVER_IP = 192.168.88.77 + +You will need to add a configuration file for your device. It can be +extremely simple, just identifying the type of the device:: + + $ cat ~/.config/lava-dispatcher/devices/panda01.conf + device_type = panda + +You also need a webserver such as Apache set up, and serve +LAVA_IMAGE_TMPDIR/images as /images. This snippet does this for +Apache:: + + Alias /images/ "/linaro/images/" + + Options Indexes MultiViews FollowSymLinks + + +If you want to upload the results to a local dashboard instance, you +need to set one of those up. XXX link to doc on this! Modify the server in job file doc/lava-ltp-job.json to local launch-control url, like: -http://127.0.0.1/launch-control + { "command": "submit_results", "parameters": { - "server": "http://dashboard.linaro.org", + "server": "http://staging.linaro.dev/lava-server", "stream": "panda01-ltp" } } -Logging: -Conmux keeps logs in /var/log/conmux. If it cannot attach to the -device, it may continually send output to the log every few seconds. -For non-production purposes, you may simply want to run 'sudo stop -conmux' when you are not using the console, to avoid needlessly growing -large logfiles. +A note on logging: Conmux keeps logs in /var/log/conmux. If it cannot +attach to the device, it may continually send output to the log every +few seconds. For non-production purposes, you may simply want to run +'sudo stop conmux' when you are not using the console, to avoid +needlessly growing large logfiles. Dispatching a job ----------------- -You will want to edit lava/dispatcher/config.py to change the value of -LAVA_SERVER_IP to, at least, a machine that exists on your network. - -There are other things that you need to edit in here XXX explain some -of this stuff. - From the toplevel, run (as root): ./lava-dispatch doc/lava-ltp-job.json === modified file 'lava_dispatcher/__init__.py' --- lava_dispatcher/__init__.py 2011-09-08 22:12:27 +0000 +++ lava_dispatcher/__init__.py 2011-09-09 04:34:34 +0000 @@ -27,16 +27,19 @@ import pexpect from lava_dispatcher.actions import get_all_cmds +from lava_dispatcher.config import get_config, get_device_config from lava_dispatcher.client import LavaClient, CriticalError, GeneralError from lava_dispatcher.android_client import LavaAndroidClient -__version__ = "0.2.2" +__version__ = "0.3a1" class LavaTestJob(object): def __init__(self, job_json, oob_file): self.job_status = 'pass' self.load_job_data(job_json) - self.context = LavaContext(self.target, self.image_type, oob_file) + dispatcher_config = get_config("lava-dispatcher") + self.context = LavaContext( + self.target, self.image_type, dispatcher_config, oob_file) def load_job_data(self, job_json): self.job_data = json.loads(job_json) @@ -76,6 +79,7 @@ else: status = 'pass' finally: + err_msg = "" if status == 'fail': err_msg = "Lava failed at action %s with error: %s\n" %\ (cmd['command'], err) @@ -102,11 +106,17 @@ class LavaContext(object): - def __init__(self, target, image_type, oob_file): - if image_type != "android": - self._client = LavaClient(target) + def __init__(self, target, image_type, dispatcher_config, oob_file): + self.config = dispatcher_config + device_config = get_device_config(target) + if device_config.get('client_type') != 'conmux': + raise RuntimeError( + "this version of lava-dispatcher only supports conmux " + "clients, not %r" % device_config.get('client_type')) + if image_type == "android": + self._client = LavaAndroidClient(self, device_config) else: - self._client = LavaAndroidClient(target) + self._client = LavaClient(self, device_config) self.test_data = LavaTestData() self.oob_file = oob_file @@ -114,6 +124,26 @@ def client(self): return self._client + @property + def lava_server_ip(self): + return self.config.get("LAVA_SERVER_IP") + + @property + def lava_image_tmpdir(self): + return self.config.get("LAVA_IMAGE_TMPDIR") + + @property + def lava_image_url(self): + return self.config.get("LAVA_IMAGE_URL") + + @property + def lava_result_dir(self): + return self.config.get("LAVA_RESULT_DIR") + + @property + def lava_cachedir(self): + return self.config.get("LAVA_CACHEDIR") + class LavaTestData(object): def __init__(self, test_id='lava'): === modified file 'lava_dispatcher/actions/android_basic.py' --- lava_dispatcher/actions/android_basic.py 2011-09-02 03:00:59 +0000 +++ lava_dispatcher/actions/android_basic.py 2011-09-05 22:39:47 +0000 @@ -60,7 +60,7 @@ test_case_result['result'] = "fail" results['test_results'].append(test_case_result) - savebundlefile("monkey", results, timestring) + savebundlefile("monkey", results, timestring, self.context.lava_result_dir) self.client.proc.sendline("") @@ -146,5 +146,5 @@ test_case_result['result'] = "fail" results['test_results'].append(test_case_result) - savebundlefile("basic", results, timestring) + savebundlefile("basic", results, timestring, self.context.lava_result_dir) self.client.proc.sendline("") === modified file 'lava_dispatcher/actions/android_deploy.py' --- lava_dispatcher/actions/android_deploy.py 2011-09-05 21:52:35 +0000 +++ lava_dispatcher/actions/android_deploy.py 2011-09-05 22:39:47 +0000 @@ -20,7 +20,6 @@ # along with this program; if not, see . from lava_dispatcher.actions import BaseAction -from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL import os import sys import shutil @@ -31,6 +30,8 @@ class cmd_deploy_linaro_android_image(BaseAction): def run(self, boot, system, data, use_cache=True): + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir + LAVA_IMAGE_URL = self.context.lava_image_url client = self.client print "deploying Android on %s" % client.hostname print " boot: %s" % boot @@ -85,14 +86,16 @@ :param data_url: url of the Linaro Android data tarball to download :param use_cache: whether or not to use the cached copy (if it exists) """ + lava_cachedir = self.context.lava_cachedir + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) tarball_dir = self.tarball_dir os.chmod(tarball_dir, 0755) if use_cache: - boot_path = download_with_cache(boot_url, tarball_dir) - system_path = download_with_cache(system_url, tarball_dir) - data_path = download_with_cache(data_url, tarball_dir) + boot_path = download_with_cache(boot_url, tarball_dir, lava_cachedir) + system_path = download_with_cache(system_url, tarball_dir, lava_cachedir) + data_path = download_with_cache(data_url, tarball_dir, lava_cachedir) else: boot_path = download(boot_url, tarball_dir) system_path = download(system_url, tarball_dir) === modified file 'lava_dispatcher/actions/deploy.py' --- lava_dispatcher/actions/deploy.py 2011-09-08 04:23:59 +0000 +++ lava_dispatcher/actions/deploy.py 2011-09-08 23:35:05 +0000 @@ -26,13 +26,14 @@ from tempfile import mkdtemp from lava_dispatcher.actions import BaseAction -from lava_dispatcher.config import LAVA_IMAGE_TMPDIR, LAVA_IMAGE_URL from lava_dispatcher.utils import download, download_with_cache from lava_dispatcher.client import CriticalError class cmd_deploy_linaro_image(BaseAction): def run(self, hwpack, rootfs, use_cache=True): + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir + LAVA_IMAGE_URL = self.context.lava_image_url client = self.client print "deploying on %s" % client.hostname print " hwpack: %s" % hwpack @@ -112,30 +113,31 @@ :param hwpack_url: url of the Linaro hwpack to download :param rootfs_url: url of the Linaro image to download """ + lava_cachedir = self.context.lava_cachedir + LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir client = self.client self.tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) tarball_dir = self.tarball_dir os.chmod(tarball_dir, 0755) if use_cache: - hwpack_path = download_with_cache(hwpack_url, tarball_dir) - rootfs_path = download_with_cache(rootfs_url, tarball_dir) + hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir) + rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir) else: hwpack_path = download(hwpack_url, tarball_dir) rootfs_path = download(rootfs_url, tarball_dir) image_file = os.path.join(tarball_dir, "lava.img") - board = client.board cmd = ("sudo linaro-media-create --hwpack-force-yes --dev %s " "--image_file %s --binary %s --hwpack %s --image_size 3G" % - (board.type, image_file, rootfs_path, hwpack_path)) + (client.device_type, image_file, rootfs_path, hwpack_path)) rc, output = getstatusoutput(cmd) if rc: shutil.rmtree(tarball_dir) tb = traceback.format_exc() client.sio.write(tb) raise RuntimeError("linaro-media-create failed: %s" % output) - boot_offset = self._get_partition_offset(image_file, board.boot_part) - root_offset = self._get_partition_offset(image_file, board.root_part) + boot_offset = self._get_partition_offset(image_file, client.boot_part) + root_offset = self._get_partition_offset(image_file, client.root_part) boot_tgz = os.path.join(tarball_dir, "boot.tgz") root_tgz = os.path.join(tarball_dir, "root.tgz") try: @@ -159,7 +161,7 @@ client.run_cmd_master('mount /dev/disk/by-label/testrootfs /mnt/root') client.run_cmd_master( 'wget -qO- %s |tar --numeric-owner -C /mnt/root -xzf -' % rootfs, - timeout = 3600) + timeout=3600) client.run_cmd_master('echo linaro > /mnt/root/etc/hostname') #DO NOT REMOVE - diverting flash-kernel and linking it to /bin/true #prevents a serious problem where packages getting installed that === modified file 'lava_dispatcher/actions/launch_control.py' (properties changed: +x to -x) --- lava_dispatcher/actions/launch_control.py 2011-09-06 09:35:13 +0000 +++ lava_dispatcher/actions/launch_control.py 2011-09-08 04:28:39 +0000 @@ -25,8 +25,6 @@ import shutil import tarfile from lava_dispatcher.actions import BaseAction -from lava_dispatcher.config import LAVA_RESULT_DIR -from lava_dispatcher.config import LAVA_IMAGE_TMPDIR from lava_dispatcher.client import NetworkError from lava_dispatcher.utils import download from tempfile import mkdtemp @@ -40,9 +38,9 @@ allow_none=True, use_datetime=True) #Upload bundle files to dashboard - bundle_list = os.listdir("/tmp/%s" % LAVA_RESULT_DIR) + bundle_list = os.listdir("/tmp/%s" % self.context.lava_result_dir) for bundle_name in bundle_list: - bundle = "/tmp/%s/%s" % (LAVA_RESULT_DIR, bundle_name) + bundle = "/tmp/%s/%s" % (self.context.lava_result_dir, bundle_name) f = open(bundle) content = f.read() f.close() @@ -53,6 +51,7 @@ print "xmlrpclib.Fault occurred" print "Fault code: %d" % err.faultCode print "Fault string: %s" % err.faultString + # After uploading, remove the bundle file at the host side os.remove(bundle) @@ -79,49 +78,50 @@ client.run_cmd_master('mkdir -p /mnt/root') client.run_cmd_master( 'mount /dev/disk/by-label/%s /mnt/root' % result_disk) - client.run_cmd_master('mkdir -p /tmp/%s' % LAVA_RESULT_DIR) + client.run_cmd_master('mkdir -p /tmp/%s' % self.context.lava_result_dir) client.run_cmd_master( - 'cp /mnt/root/%s/*.bundle /tmp/%s' % (LAVA_RESULT_DIR, - LAVA_RESULT_DIR)) + 'cp /mnt/root/%s/*.bundle /tmp/%s' % (self.context.lava_result_dir, + self.context.lava_result_dir)) client.run_cmd_master('umount /mnt/root') #Create tarball of all results client.run_cmd_master('cd /tmp') client.run_cmd_master( - 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % LAVA_RESULT_DIR) + 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir) master_ip = client.get_master_ip() - if master_ip != None: - # Set 80 as server port - client.run_cmd_master('python -m SimpleHTTPServer 80 &> /dev/null &') - time.sleep(3) - - result_tarball = "http://%s/lava_results.tgz" % master_ip - tarball_dir = mkdtemp(dir=LAVA_IMAGE_TMPDIR) - os.chmod(tarball_dir, 0755) - - # download test result with a retry mechanism - # set retry timeout to 2mins - now = time.time() - timeout = 120 - while time.time() < now+timeout: - try: - result_path = download(result_tarball, tarball_dir) - except: - if time.time() >= now+timeout: - raise - - client.run_cmd_master('kill %1') - - tar = tarfile.open(result_path) - for tarinfo in tar: - if os.path.splitext(tarinfo.name)[1] == ".bundle": - f = tar.extractfile(tarinfo) - content = f.read() - f.close() - self.all_bundles.append(json.loads(content)) - tar.close() - shutil.rmtree(tarball_dir) + if master_ip == None: + raise NetworkError("Getting master image IP address failed") + # Set 80 as server port + client.run_cmd_master('python -m SimpleHTTPServer 80 &> /dev/null &') + time.sleep(3) + + result_tarball = "http://%s/lava_results.tgz" % master_ip + tarball_dir = mkdtemp(dir=self.context.lava_image_tmpdir) + os.chmod(tarball_dir, 0755) + + # download test result with a retry mechanism + # set retry timeout to 2mins + now = time.time() + timeout = 120 + while time.time() < now+timeout: + try: + result_path = download(result_tarball, tarball_dir) + except: + if time.time() >= now+timeout: + raise + + client.run_cmd_master('kill %1') + + tar = tarfile.open(result_path) + for tarinfo in tar: + if os.path.splitext(tarinfo.name)[1] == ".bundle": + f = tar.extractfile(tarinfo) + content = f.read() + f.close() + self.all_bundles.append(json.loads(content)) + tar.close() + shutil.rmtree(tarball_dir) #flush the serial log client.run_shell_command("") === modified file 'lava_dispatcher/actions/lava-test.py' (properties changed: +x to -x) --- lava_dispatcher/actions/lava-test.py 2011-09-08 04:23:59 +0000 +++ lava_dispatcher/actions/lava-test.py 2011-09-08 04:28:39 +0000 @@ -24,7 +24,7 @@ import traceback from lava_dispatcher.actions import BaseAction from lava_dispatcher.client import OperationFailed -from lava_dispatcher.config import LAVA_RESULT_DIR, MASTER_STR + def _setup_testrootfs(client): @@ -50,10 +50,9 @@ 'cp -f /mnt/root/etc/resolv.conf.bak /mnt/root/etc/resolv.conf') cmd = ('cat /proc/mounts | awk \'{print $2}\' | grep "^/mnt/root/dev"' '| sort -r | xargs umount') - client.run_cmd_master( - cmd) - client.run_cmd_master( - 'umount /mnt/root') + client.run_cmd_master(cmd) + client.run_cmd_master('umount /mnt/root') + def _install_lava_test(client): @@ -73,11 +72,10 @@ client.run_shell_command( 'chroot /mnt/root lava-test help', response="list-test", timeout=10) - client.proc.expect(MASTER_STR, timeout=10) + client.proc.expect(client.master_str, timeout=10) except: tb = traceback.format_exc() client.sio.write(tb) - _teardown_testrootfs(client) raise OperationFailed("lava-test deployment failed") @@ -86,12 +84,12 @@ #Make sure in test image now client = self.client client.in_test_shell() - client.run_cmd_tester('mkdir -p %s' % LAVA_RESULT_DIR) + client.run_cmd_tester('mkdir -p %s' % self.context.lava_result_dir) client.export_display() bundle_name = test_name + "-" + datetime.now().strftime("%H%M%S") client.run_cmd_tester( 'lava-test run %s -o %s/%s.bundle' % ( - test_name, LAVA_RESULT_DIR, bundle_name), + test_name, self.context.lava_result_dir, bundle_name), timeout=timeout) === modified file 'lava_dispatcher/android_client.py' --- lava_dispatcher/android_client.py 2011-09-08 04:23:59 +0000 +++ lava_dispatcher/android_client.py 2011-09-08 04:28:39 +0000 @@ -20,12 +20,9 @@ import pexpect import sys from lava_dispatcher.client import LavaClient, OperationFailed -from lava_dispatcher.android_config import BOARDS, TESTER_STR +from utils import string_to_list class LavaAndroidClient(LavaClient): - def __init__(self, hostname): - super(LavaAndroidClient, self).__init__(hostname) - self.board = BOARDS[hostname] def run_adb_shell_command(self, dev_id, cmd, response, timeout=-1): adb_cmd = "adb -s %s shell %s" % (dev_id, cmd) @@ -42,7 +39,7 @@ """ Check that we are in a shell on the test image """ self.proc.sendline("") - id = self.proc.expect([TESTER_STR , pexpect.TIMEOUT]) + id = self.proc.expect([self.tester_str , pexpect.TIMEOUT]) if id == 1: raise OperationFailed @@ -55,11 +52,11 @@ except: self.hard_reboot() self.enter_uboot() - uboot_cmds = self.board.uboot_cmds - self.proc.sendline(uboot_cmds[0]) - for line in range(1, len(uboot_cmds)): + boot_cmds = string_to_list(self.config.get('boot_cmds_android')) + self.proc.sendline(boot_cmds[0]) + for line in range(1, len(boot_cmds)): self.proc.expect("#") - self.proc.sendline(uboot_cmds[line]) + self.proc.sendline(boot_cmds[line]) self.in_test_shell() self.proc.sendline("export PS1=\"root@linaro: \"") @@ -106,7 +103,7 @@ def check_adb_status(self): # XXX: IP could be assigned in other way in the validation farm - network_interface = self.board.default_network_interface + network_interface = self.default_network_interface try: self.run_cmd_tester( 'netcfg %s dhcp' % network_interface, timeout=60) === removed file 'lava_dispatcher/android_config.py' --- lava_dispatcher/android_config.py 2011-08-16 00:15:08 +0000 +++ lava_dispatcher/android_config.py 1970-01-01 00:00:00 +0000 @@ -1,65 +0,0 @@ -# Copyright (C) 2011 Linaro Limited -# -# Author: Linaro Validation Team -# -# This file is part of LAVA Dispatcher. -# -# LAVA Dispatcher is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# LAVA Dispatcher is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, see . - -from lava_dispatcher.config import Board - -class BeagleBoard(Board): - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage;" - "fatload mmc 0:3 0x81600000 uInitrd;" - "bootm 0x80000000 0x81600000'", - "setenv bootargs 'console=tty0 console=ttyO2,115200n8 " - "rootwait rw earlyprintk fixrtc nocompcache " - "vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 " - "init=/init androidboot.console=ttyO2'", - "boot"] - type = "beagle" - - -class PandaBoard(Board): - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:3 0x80200000 uImage;" - "fatload mmc 0:3 0x81600000 uInitrd;" - "bootm 0x80200000 0x81600000'", - "setenv bootargs 'console=tty0 console=ttyO2,115200n8 " - "rootwait rw earlyprintk fixrtc nocompcache vram=32M " - "omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000 " - "omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 " - "init=/init androidboot.console=ttyO2'", - "boot"] - type = "panda" - - -#Here, it still needs to maintain a map from boardid to board, for there is only -#boardid in jobfile.json -BOARDS = { - "beagle01": BeagleBoard, - "beagle02": BeagleBoard, - "beagle03": BeagleBoard, - "beagle04": BeagleBoard, - "panda01": PandaBoard, - "panda02": PandaBoard, - "panda03": PandaBoard, - "panda04": PandaBoard, - } - -#Test image recognization string -TESTER_STR = "root@linaro:" === modified file 'lava_dispatcher/android_util.py' --- lava_dispatcher/android_util.py 2011-06-27 04:55:08 +0000 +++ lava_dispatcher/android_util.py 2011-08-10 18:01:25 +0000 @@ -22,11 +22,10 @@ from datetime import datetime import json import subprocess -from lava_dispatcher.config import LAVA_RESULT_DIR import time # TODO: Result saving could be replaced by linaro_dashboard_bundle probably. -def savebundlefile(testname, results, starttime): +def savebundlefile(testname, results, starttime, lava_result_dir): """ Save results as .bundle file under /tmp/LAVA_RESULT_DIR/ """ @@ -42,9 +41,9 @@ testdata['test_runs'] = test_runs testdata['test_runs'][0].update(results) bundle = testdata - subprocess.call(["mkdir", "-p", "/tmp/%s" % LAVA_RESULT_DIR]) + subprocess.call(["mkdir", "-p", "/tmp/%s" % lava_result_dir]) # The file name should be unique to be distinguishable from others - filename = "/tmp/%s/" % LAVA_RESULT_DIR + testname + \ + filename = "/tmp/%s/" % lava_result_dir + testname + \ str(time.mktime(datetime.utcnow().timetuple())) + ".bundle" with open(filename, "wt") as stream: json.dump(bundle, stream) === modified file 'lava_dispatcher/client.py' --- lava_dispatcher/client.py 2011-09-08 04:23:59 +0000 +++ lava_dispatcher/client.py 2011-09-08 23:35:05 +0000 @@ -22,35 +22,56 @@ import sys import time from cStringIO import StringIO - -from lava_dispatcher.config import ( - BOARDS, - LAVA_SERVER_IP, - MASTER_STR, - TESTER_STR, - ) - +from utils import string_to_list class LavaClient(object): - def __init__(self, hostname): - self._master_str = MASTER_STR - self._tester_str = TESTER_STR - cmd = "conmux-console %s" % hostname + def __init__(self, context, config): + self.context = context + self.config = config + cmd = "conmux-console %s" % self.hostname self.sio = SerialIO(sys.stdout) self.proc = pexpect.spawn(cmd, timeout=3600, logfile=self.sio) #serial can be slow, races do funny things if you don't increase delay self.proc.delaybeforesend=1 - self.hostname = hostname - # will eventually come from the database - self.board = BOARDS[hostname] + + def device_option(self, option_name): + return self.config.get(option_name) + + def device_option_int(self, option_name): + return self.config.getint(option_name) + + @property + def hostname(self): + return self.device_option("hostname") + + @property + def tester_str(self): + return self.device_option("TESTER_STR") @property def master_str(self): - return self._master_str - - @property - def tester_str(self): - return self._tester_str + return self.device_option("MASTER_STR") + + @property + def boot_cmds(self): + uboot_str = self.device_option("boot_cmds") + return string_to_list(uboot_str) + + @property + def device_type(self): + return self.device_option("device_type") + + @property + def boot_part(self): + return self.device_option_int("boot_part") + + @property + def root_part(self): + return self.device_option_int("root_part") + + @property + def default_network_interface(self): + return self.device_option("default_network_interface") def in_master_shell(self): """ Check that we are in a shell on the master image @@ -91,16 +112,16 @@ except: self.hard_reboot() self.enter_uboot() - uboot_cmds = self.board.uboot_cmds - self.proc.sendline(uboot_cmds[0]) - for line in range(1, len(uboot_cmds)): - if self.board.type in ["mx51evk", "mx53loco"]: + boot_cmds = self.boot_cmds + self.proc.sendline(boot_cmds[0]) + for line in range(1, len(boot_cmds)): + if self.device_type in ["mx51evk", "mx53loco"]: self.proc.expect(">", timeout=300) - elif self.board.type == "snowball_sd": + elif self.device_type == "snowball_sd": self.proc.expect("\$", timeout=300) else: self.proc.expect("#", timeout=300) - self.proc.sendline(uboot_cmds[line]) + self.proc.sendline(boot_cmds[line]) self.in_test_shell() def enter_uboot(self): @@ -131,7 +152,8 @@ self.run_shell_command(cmd, self.tester_str, timeout) def check_network_up(self): - self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % LAVA_SERVER_IP) + lava_server_ip = self.context.lava_server_ip + self.proc.sendline("LC_ALL=C ping -W4 -c1 %s" % lava_server_ip) id = self.proc.expect(["1 received", "0 received", "Network is unreachable"], timeout=5) self.proc.expect(self.master_str) @@ -154,7 +176,7 @@ #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" pattern1 = "(\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?\.\d?\d?\d?)" cmd = ("ifconfig %s | grep 'inet addr' | awk -F: '{print $2}' |" - "awk '{print $1}'" % self.board.default_network_interface) + "awk '{print $1}'" % self.default_network_interface) self.proc.sendline(cmd) #if running from ipython, it needs another Enter, don't know why: #self.proc.sendline("") === modified file 'lava_dispatcher/config.py' --- lava_dispatcher/config.py 2011-08-16 00:15:08 +0000 +++ lava_dispatcher/config.py 2011-09-09 00:52:17 +0000 @@ -18,113 +18,72 @@ # along # with this program; if not, see . -""" -This is an ugly hack, the uboot commands for a given board type and the board -type of a test machine need to come from the device registry. This is an -easy way to look it up for now though, just to show the rest of the code -around it -""" - -class Board: - uboot_cmds = None - type = None - # boot partition number, counting from 1 - boot_part = 1 - # root partition number, counting from 1 - root_part = 2 - default_network_interface = "eth0" - -class BeagleBoard(Board): - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage; fatload mmc " - "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'", - "setenv bootargs ' console=tty0 console=ttyO2,115200n8 " - "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache " - "vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60'", - "boot"] - type = "beagle" - -class PandaBoard(Board): - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:3 0x80200000 uImage; fatload mmc " - "0:3 0x81600000 uInitrd; bootm 0x80200000 0x81600000'", - "setenv bootargs ' console=tty0 console=ttyO2,115200n8 " - "root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache " - "vram=48M omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000'", - "boot"] - type = "panda" - -class Snowball(Board): - uboot_cmds = ["mmc init", - "mmc rescan 1", - "setenv bootcmd 'fat load mmc 1:3 0x00100000 /uImage;" - "bootm 0x00100000'", - "setenv bootargs 'console=tty0 console=ttyAMA2,115200n8 " - "root=LABEL=testrootfs rootwait ro earlyprintk rootdelay=1 " - "fixrtc nocompcache mem=96M@0 mem_modem=32M@96M mem=44M@128M " - "pmem=22M@172M mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M " - "hwmem=48M@302M mem=152M@360M'", - "boot"] - type = "snowball_sd" - -class Mx51evkBoard(Board): - boot_part = 2 - root_part = 3 - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:5 0x90000000 uImage; fatload mmc 0:5 " - "0x92000000 uInitrd; fatload mmc 0:5 0x91ff0000 board.dtb; bootm " - "0x90000000 0x92000000 0x91ff0000'", - "setenv bootargs ' console=tty0 console=ttymxc0,115200n8 " - "root=LABEL=testrootfs rootwait ro'", - "boot"] - type = "mx51evk" - -class Mx53locoBoard(Board): - boot_part = 2 - root_part = 3 - uboot_cmds = ["mmc init", - "mmc part 0", - "setenv bootcmd 'fatload mmc 0:5 0x70800000 uImage; fatload mmc " - "0:5 0x71800000 uInitrd; bootm 0x70800000 0x71800000'", - "setenv bootargs ' console=tty0 console=ttymxc0,115200n8 " - "root=LABEL=testrootfs rootwait ro'", - "boot"] - type = "mx53loco" - -#Here, it still needs to maintain a map from boardid to board, for there is -#only boardid in jobfile.json -BOARDS = { - "panda01": PandaBoard, - "panda02": PandaBoard, - "panda03": PandaBoard, - "panda04": PandaBoard, - "beaglexm01": BeagleBoard, - "beaglexm02": BeagleBoard, - "beaglexm03": BeagleBoard, - "beaglexm04": BeagleBoard, - "mx51evk01": Mx51evkBoard, - "mx53loco01": Mx53locoBoard, - "snowball01": Snowball, - "snowball02": Snowball, - "snowball03": Snowball, - "snowball04": Snowball, - } - -#Main LAVA server IP in the boards farm -LAVA_SERVER_IP = "192.168.1.10" -#Location for hosting rootfs/boot tarballs extracted from images -LAVA_IMAGE_TMPDIR = "/linaro/images/tmp" -#URL where LAVA_IMAGE_TMPDIR can be accessed remotely -LAVA_IMAGE_URL = "http://%s/images/tmp" % LAVA_SERVER_IP -#Default test result storage path -LAVA_RESULT_DIR = "/lava/results" -#Location for caching downloaded artifacts such as hwpacks and images -LAVA_CACHEDIR = "/linaro/images/cache" - -#Master image recognization string -MASTER_STR = "root@master:" -#Test image recognization string -TESTER_STR = "root@linaro:" +from ConfigParser import ConfigParser +import os +import StringIO + + +default_config_path = os.path.join( + os.path.dirname(__file__), 'default-config') + + +def load_config_paths(name): + for directory in [os.path.expanduser("~/.config"), + "/etc/xdg", default_config_path]: + path = os.path.join(directory, name) + if os.path.isdir(path): + yield path + + +def _read_into(path, cp): + s = StringIO.StringIO() + s.write('[DEFAULT]\n') + s.write(open(path).read()) + s.seek(0) + cp.readfp(s) + + +def _get_config(name, cp=None): + """Read a config file named name + '.conf'. + + This checks and loads files from the source tree, site wide location and + home directory -- in that order, so home dir settings override site + settings which override source settings. + """ + config_files = [] + for directory in load_config_paths('lava-dispatcher'): + path = os.path.join(directory, '%s.conf' % name) + if os.path.exists(path): + config_files.append(path) + if not config_files: + raise Exception("no config files named %r found" % (name + ".conf")) + config_files.reverse() + if cp is None: + cp = ConfigParser() + print "About to read %s" % str(config_files) + for path in config_files: + _read_into(path, cp) + return cp + + +class ConfigWrapper(object): + def __init__(self, cp): + self.cp = cp + def get(self, key): + return self.cp.get("DEFAULT", key) + def getint(self, key): + return self.cp.getint("DEFAULT", key) + + +def get_config(name): + return ConfigWrapper(_get_config(name)) + + +def get_device_config(name): + device_config = _get_config("devices/%s" % name) + cp = _get_config("device-defaults") + _get_config( + "device-types/%s" % device_config.get('DEFAULT', 'device_type'), cp) + _get_config("devices/%s" % name, cp) + cp.set("DEFAULT", "hostname", name) + return ConfigWrapper(cp) === added directory 'lava_dispatcher/default-config' === added directory 'lava_dispatcher/default-config/lava-dispatcher' === added file 'lava_dispatcher/default-config/lava-dispatcher/README' --- lava_dispatcher/default-config/lava-dispatcher/README 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/README 2011-09-08 23:35:05 +0000 @@ -0,0 +1,41 @@ +Configuration files for lava-dispatcher +======================================= + +lava-dispatcher looks for files in: + + * Alongside the installation/source tree for the default values + (i.e. this directory). + + * /etc/xdg/lava-dispatcher for system-wide settings. + + * ~/.config/lava-dispatcher for user settings. + +Each config directory can contain two files and two directories: + + * lava-dispatcher.conf + + This file defines global settings of the dispatcher. You will + almost certainly need to customize LAVA_SERVER_IP for your install. + + * device-defaults.conf + + This file defines default values for all devices. You probably + won't need to customize it. + + * device-types/ + + This directory contains a config file for each device type. You + probably won't need to customize the settings for device types that + are already supported by lava-dispatcher, but if you are working on + supporting a new class of device, you will need to add a file here. + + Note that the device-type name must match the --dev argument to + linaro-media-create. + + * devices/ + + This directory contains a file per device that can be targeted by + lava-dispatcher. For the most part this file just needs to contain + a line "device_type = ", although other settings can + be included here. You will definitely need to tell lava-dispatcher + about the devices you have! === added file 'lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-defaults.conf 2011-09-09 04:34:34 +0000 @@ -0,0 +1,50 @@ +# The default device settings. + +# All device settings default to these values unless they are +# overwritten by the specific device type file +# (device-types/${TYPE}.conf) or the specific device file +# (devices/${DEVICE}.conf). + +# The client_type. Only 'conmux' (meaning we communicate with the +# device over a serial line via conmux) is supported today but 'qemu' +# and 'ssh' are coming. +client_type = conmux + +# The bootloader commands to boot the device into the test image (we +# assume that the device boots into the master image without bootloader +# intervention). +# +# XXX should be called # boot_test_image_commands ? +boot_cmds = + +# The bootloader commands to boot the device into an android-based test +# image. +# +# XXX should be called # boot_android_test_image_commands ? +boot_cmds_android = + +# The device type. Settings in device-types/${TYPE}.conf override +# settings in this file, but are overridden by the +# devices/${DEVICE}.conf file. +type = + +# The network interface that comes up by default +default_network_interface = eth0 + +# boot partition number, counting from 1 +# +# This is used to divide up the image produced by linaro-media-create +# into sections to write onto the device. +boot_part = 1 + +# root partition number, counting from 1 +# +# This is used to divide up the image produced by linaro-media-create +# into sections to write onto the device. +root_part = 2 + +# Master image recognization string +MASTER_STR = root@master: + +# Test image recognization string +TESTER_STR = root@linaro: === added directory 'lava_dispatcher/default-config/lava-dispatcher/device-types' === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/beagle-xm.conf 2011-09-07 03:46:15 +0000 @@ -0,0 +1,20 @@ +boot_cmds = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:3 0x80000000 uImage; + fatload mmc 0:3 0x81600000 uInitrd; + bootm 0x80000000 0x81600000'", + setenv bootargs "' console=tty0 console=ttyO2,115200n8 + root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache + vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60'", + boot + +boot_cmds_android = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:3 0x80000000 uImage; + fatload mmc 0:3 0x81600000 uInitrd; + bootm 0x80000000 0x81600000'", + setenv bootargs "'console=tty0 console=ttyO2,115200n8 + rootwait rw earlyprintk fixrtc nocompcache + vram=12M omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 + init=/init androidboot.console=ttyO2'", + boot === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/mx51evk.conf 2011-09-07 03:46:15 +0000 @@ -0,0 +1,10 @@ +boot_part = 2 +root_part = 3 +boot_cmds = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:5 0x90000000 uImage; fatload mmc 0:5 + 0x92000000 uInitrd; fatload mmc 0:5 0x91ff0000 board.dtb; bootm + 0x90000000 0x92000000 0x91ff0000'", + setenv bootargs "' console=tty0 console=ttymxc0,115200n8 + root=LABEL=testrootfs rootwait ro'", + boot === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/mx53loco.conf 2011-09-07 03:46:15 +0000 @@ -0,0 +1,9 @@ +boot_part = 2 +root_part = 3 +boot_cmds = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:5 0x70800000 uImage; fatload mmc + 0:5 0x71800000 uInitrd; bootm 0x70800000 0x71800000'", + setenv bootargs "' console=tty0 console=ttymxc0,115200n8 + root=LABEL=testrootfs rootwait ro'", + boot === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/panda.conf 2011-09-07 03:46:15 +0000 @@ -0,0 +1,19 @@ +boot_cmds = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:3 0x80200000 uImage; fatload mmc + 0:3 0x81600000 uInitrd; bootm 0x80200000 0x81600000'", + setenv bootargs "' console=tty0 console=ttyO2,115200n8 + root=LABEL=testrootfs rootwait ro earlyprintk fixrtc nocompcache + vram=48M omapfb.vram=0:24M mem=456M@0x80000000 mem=512M@0xA0000000'", + boot +boot_cmds_android = mmc init, + mmc part 0, + setenv bootcmd "'fatload mmc 0:3 0x80200000 uImage; + fatload mmc 0:3 0x81600000 uInitrd; + bootm 0x80200000 0x81600000'", + setenv bootargs "'console=tty0 console=ttyO2,115200n8 + rootwait rw earlyprintk fixrtc nocompcache vram=32M + omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000 + omapfb.debug=y omapfb.mode=dvi:1280x720MR-16@60 + init=/init androidboot.console=ttyO2'", + boot === added file 'lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/snowball.conf 2011-09-07 03:46:15 +0000 @@ -0,0 +1,12 @@ +boot_cmds = mmc init, + mmc rescan 1, + setenv bootcmd "'fat load mmc 1:3 0x00100000 /uImage; + bootm 0x00100000'", + setenv bootargs "'console=tty0 console=ttyAMA2,115200n8 + root=LABEL=testrootfs rootwait ro earlyprintk rootdelay=1 + fixrtc nocompcache mem=96M@0 mem_modem=32M@96M mem=44M@128M + pmem=22M@172M mem=30M@194M mem_mali=32M@224M pmem_hwb=54M@256M + hwmem=48M@302M mem=152M@360M'", + boot + +#boot_cmds_android = TBD \ No newline at end of file === added directory 'lava_dispatcher/default-config/lava-dispatcher/devices' === added file 'lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf' --- lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 1970-01-01 00:00:00 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 2011-09-08 23:35:05 +0000 @@ -0,0 +1,19 @@ +# General lava-dispatcher settings. + +# Main LAVA server IP in the lab. +# +# This is the IP the device downloads the image parts from. +LAVA_SERVER_IP = 192.168.1.10 + +# Location for rootfs/boot tarballs extracted from images +LAVA_IMAGE_TMPDIR = /linaro/images/tmp + +# URL where LAVA_IMAGE_TMPDIR can be accessed remotely +LAVA_IMAGE_URL = http://%(LAVA_SERVER_IP)s/images/tmp + +# Location on the device for storing test results. +LAVA_RESULT_DIR = /lava/results + +# Location for caching downloaded artifacts such as hwpacks and images +LAVA_CACHEDIR = /linaro/images/cache + === modified file 'lava_dispatcher/tests/test_config.py' --- lava_dispatcher/tests/test_config.py 2011-06-27 04:55:08 +0000 +++ lava_dispatcher/tests/test_config.py 2011-09-08 23:35:05 +0000 @@ -19,12 +19,15 @@ from unittest import TestCase -from lava_dispatcher.config import BOARDS, LAVA_SERVER_IP +from lava_dispatcher.config import get_config, get_device_config +from lava_dispatcher.utils import string_to_list class TestConfigData(TestCase): def test_beagle01_uboot_cmds(self): + beagle01_config = get_device_config("beaglexm01") expected = [ "mmc init", + "mmc part 0", "setenv bootcmd 'fatload mmc 0:3 0x80000000 uImage; fatload mmc " "0:3 0x81600000 uInitrd; bootm 0x80000000 0x81600000'", "setenv bootargs ' console=tty0 console=ttyO2,115200n8 " @@ -32,11 +35,12 @@ "nocompcache vram=12M omapfb.debug=y " "omapfb.mode=dvi:1280x720MR-16@60'", "boot"] - brd = BOARDS["beagle01"] - uboot_cmds = brd.uboot_cmds - self.assertEquals(expected, uboot_cmds) + uboot_cmds = beagle01_config.get("boot_cmds") + self.assertEquals(expected, string_to_list(uboot_cmds)) def test_server_ip(self): + server_config = get_config("lava-dispatcher") expected = "192.168.1.10" - self.assertEqual(expected, LAVA_SERVER_IP) + lava_server_ip = server_config.get("LAVA_SERVER_IP") + self.assertEqual(expected, lava_server_ip) === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2011-08-18 20:38:38 +0000 +++ lava_dispatcher/utils.py 2011-09-01 03:03:09 +0000 @@ -22,8 +22,7 @@ import shutil import urllib2 import urlparse - -from lava_dispatcher.config import LAVA_CACHEDIR +from shlex import shlex def download(url, path=""): urlpath = urlparse.urlsplit(url).path @@ -41,8 +40,8 @@ raise RuntimeError("Could not retrieve %s" % url) return filename -def download_with_cache(url, path=""): - cache_loc = url_to_cache(url) +def download_with_cache(url, path="", cachedir=""): + cache_loc = url_to_cache(url, cachedir) if os.path.exists(cache_loc): filename = os.path.basename(cache_loc) file_location = os.path.join(path, filename) @@ -61,8 +60,16 @@ #so ignore return file_location -def url_to_cache(url): +def url_to_cache(url, cachedir): url_parts = urlparse.urlsplit(url) - path = os.path.join(LAVA_CACHEDIR, url_parts.netloc, + path = os.path.join(cachedir, url_parts.netloc, url_parts.path.lstrip(os.sep)) return path + +def string_to_list(string): + splitter = shlex(string, posix=True) + splitter.whitespace = "," + splitter.whitespace_split = True + newlines_to_spaces = lambda x: x.replace('\n', ' ') + strip_newlines = lambda x: newlines_to_spaces(x).strip(' ') + return map(strip_newlines, list(splitter)) === modified file 'setup.py' --- setup.py 2011-07-21 16:55:47 +0000 +++ setup.py 2011-09-09 00:52:17 +0000 @@ -12,6 +12,15 @@ author='Linaro Validation Team', author_email='linaro-dev@lists.linaro.org', packages=find_packages(), + package_data= { + 'lava_dispatcher': [ + 'default-config/lava-dispatcher/lava-dispatcher.conf', + 'default-config/lava-dispatcher/lava-dispatcher.conf', + 'default-config/lava-dispatcher/device-defaults.conf', + 'default-config/lava-dispatcher/device-types/*.conf', + 'default-config/lava-dispatcher/devices/*.conf', + ], + }, scripts = [ 'lava-dispatch' ],