From patchwork Fri Aug 24 14:22:13 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Doan X-Patchwork-Id: 10947 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id BBC8123E53 for ; Fri, 24 Aug 2012 14:22:18 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id BF3ACA18949 for ; Fri, 24 Aug 2012 14:22:00 +0000 (UTC) Received: by iafj25 with SMTP id j25so1198998iaf.11 for ; Fri, 24 Aug 2012 07:22:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=Z0Zt4IogBDFy2Q4YviYQhu8VqY0Ga/mpETth1up1h14=; b=bSt64bEalBQizJvMummL/hvA0n8Ckggl3hsI0C9pAeWxNSsiEpWAGMHLp3k52eDG7y 2mv8p6A4pEEhKKkykcJrJ99zpzKRZFmmvPRxDaiU8d86AhFqF6k2ks1ibTGiPZWDckUa 4Iy5cGQz0E4quVSSzF8oLooGta9M+dG63tOvDGCRBnbzg4cDj7dQkctv5SeJlgAiZ0dX k4FVAi0X6W+UY7ZK7XUiyHlhUM+R2N76PAqsyxRwo+mJ1RgwI7UHZWNSJtfB6tfAWzML xtphGLA6UEGdsPz6Md5mdM/Xt+SCTh31xH6EqRrZ5nTvje9wrVscl69Tyw7EaAhMTWk+ MXTA== Received: by 10.42.84.69 with SMTP id k5mr4627295icl.5.1345818137762; Fri, 24 Aug 2012 07:22:17 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.184.232 with SMTP id ex8csp322534igc; Fri, 24 Aug 2012 07:22:15 -0700 (PDT) Received: by 10.217.3.7 with SMTP id q7mr3076525wes.47.1345818134381; Fri, 24 Aug 2012 07:22:14 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id o7si6102253wiz.31.2012.08.24.07.22.13 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 24 Aug 2012 07:22:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1T4um5-0008Cy-7Z for ; Fri, 24 Aug 2012 14:22:13 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 2D9CAE039B for ; Fri, 24 Aug 2012 14:22:13 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 373 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 373: fixes to dispatcher for ubuntu fastmodel support Message-Id: <20120824142213.15181.80752.launchpad@ackee.canonical.com> Date: Fri, 24 Aug 2012 14:22:13 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15843"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 3e9ad96a31ef2c573f6e73f9b5ce7d734b7c99a9 X-Gm-Message-State: ALoCoQljgjlRLkmbE9kf0plrsc6V6noBLIwVl/4FUBJUsJkxwoF0w+VH72UsIOY9rub7GkdJtNDd Merge authors: Andy Doan (doanac) ------------------------------------------------------------ revno: 373 [merge] committer: Andy Doan branch nick: lava-dispatcher timestamp: Fri 2012-08-24 09:14:27 -0500 message: fixes to dispatcher for ubuntu fastmodel support modified: lava_dispatcher/client/base.py lava_dispatcher/client/fastmodel.py lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf lava_dispatcher/utils.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/client/base.py' --- lava_dispatcher/client/base.py 2012-08-15 14:34:54 +0000 +++ lava_dispatcher/client/base.py 2012-08-23 15:33:00 +0000 @@ -415,7 +415,8 @@ logging.info("Boot the test image") self._boot_linaro_image() - self.in_test_shell(300) + timeout = self.config.getint("boot_linaro_timeout", 300) + self.in_test_shell(timeout) # set PS1 to include return value of last command # Details: system PS1 is set in /etc/bash.bashrc and user PS1 is set in # /root/.bashrc, it is === modified file 'lava_dispatcher/client/fastmodel.py' --- lava_dispatcher/client/fastmodel.py 2012-08-10 15:25:36 +0000 +++ lava_dispatcher/client/fastmodel.py 2012-08-23 15:00:18 +0000 @@ -31,7 +31,7 @@ import time from lava_dispatcher.client.base import ( - CommandRunner, + TesterCommandRunner, LavaClient, ) from lava_dispatcher.client.lmc_utils import ( @@ -170,12 +170,6 @@ self._customize_ubuntu() - def _close_sim_proc(self): - self._sim_proc.close(True) - - def _close_serial_proc(self): - self.proc.close(True) - def _fix_perms(self): ''' The directory created for the image download/creation gets created with tempfile.mkdtemp which grants permission only to the creator of @@ -216,19 +210,28 @@ def _stop(self): if self.proc is not None: + logging.info("performing sync on target filesystem") + r = TesterCommandRunner(self) + r.run("sync", timeout=10, failok=True) self.proc.close() + self.proc = None if self._sim_proc is not None: self._sim_proc.close() + self._sim_proc = None + + def _create_rtsm_ostream(self, ofile): + '''the RTSM binary uses the windows code page(cp1252), but the + dashboard and celery needs data with a utf-8 encoding''' + return codecs.EncodedFile(ofile, 'cp1252', 'utf-8') + def _drain_sim_proc(self): '''pexpect will continue to get data for the simproc process. We need to keep this pipe drained so that it won't get full and then stop block the process from continuing to execute''' - # NOTE: the RTSM binary uses the windows code page(cp1252), but the - # dashboard needs this with a utf-8 encoding f = cStringIO.StringIO() - self._sim_proc.logfile = codecs.EncodedFile(f, 'cp1252', 'utf-8') + self._sim_proc.logfile = self._create_rtsm_ostream(f) _pexpect_drain(self._sim_proc).start() def _boot_linaro_image(self): @@ -237,11 +240,6 @@ self._fix_perms() sim_cmd = self._get_sim_cmd() - logging.info('ensuring ADB port is ready') - while logging_system("sh -c 'netstat -an | grep 5555.*TIME_WAIT'") == 0: - logging.info ("waiting for TIME_WAIT 5555 socket to finish") - time.sleep(3) - # the simulator proc only has stdout/stderr about the simulator # we hook up into a telnet port which emulates a serial console logging.info('launching fastmodel with command %r' % sim_cmd) @@ -249,7 +247,7 @@ sim_cmd, logfile=self.sio, timeout=1200) - atexit.register(self._close_sim_proc) + atexit.register(self._stop) self._sim_proc.expect(self.PORT_PATTERN, timeout=300) self._serial_port = self._sim_proc.match.groups()[0] logging.info('serial console port on: %s' % self._serial_port) @@ -264,12 +262,18 @@ logging.info('simulator is started connecting to serial port') self.proc = logging_spawn( 'telnet localhost %s' % self._serial_port, - logfile=self.sio, + logfile=self._create_rtsm_ostream(self.sio), timeout=90) - atexit.register(self._close_serial_proc) + atexit.register(self._stop) def _boot_linaro_android_image(self): - ''' booting android or ubuntu style images don't differ for FastModel''' + ''' booting android or ubuntu style images don't differ much''' + + logging.info('ensuring ADB port is ready') + while logging_system("sh -c 'netstat -an | grep 5555.*TIME_WAIT'") == 0: + logging.info ("waiting for TIME_WAIT 5555 socket to finish") + time.sleep(3) + self._boot_linaro_image() def reliable_session(self): === modified file 'lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf' --- lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf 2012-08-14 09:30:52 +0000 +++ lava_dispatcher/default-config/lava-dispatcher/device-types/fastmodel.conf 2012-08-23 15:33:00 +0000 @@ -8,5 +8,8 @@ # fm takes longer than other android images do disablesuspend_timeout = 500 +# how long ubuntu takes to boot to prompt +boot_linaro_timeout = 500 + # if you do dhcp on boot, adb will not work (asac) on fastmodels enable_network_after_boot_android = 0 === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2012-07-22 01:42:10 +0000 +++ lava_dispatcher/utils.py 2012-08-24 14:14:27 +0000 @@ -22,6 +22,7 @@ import errno import logging import os +import select import sys import shutil import urlparse @@ -106,7 +107,12 @@ try: self._spawn__interact_copy(escape_character=chr(29)) except: - logging.warn(sys.exc_info()) + einfo = sys.exc_info() + # since we blindly read this from a thread, it will always wind up + # dying with a select error. we should still make note of other + # exceptions that might happen + if not isinstance(einfo[1], select.error): + logging.warn("error while draining pexpect buffers: %r", einfo) pass # XXX Duplication: we should reuse lava-test TestArtifacts