From patchwork Mon Nov 26 15:37:11 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongqin Liu X-Patchwork-Id: 13214 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id DCE4E23FC0 for ; Mon, 26 Nov 2012 15:37:14 +0000 (UTC) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by fiordland.canonical.com (Postfix) with ESMTP id 3A23BA196E3 for ; Mon, 26 Nov 2012 15:37:14 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id c10so4784083ieb.11 for ; Mon, 26 Nov 2012 07:37:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=MR1i2fLtJavnANXB6RD3rTW8ZDxDJzfZrfs2NUZE0jc=; b=RDYG1pFjvEaybraCL2JIc3jMVK5InJI3CrxF+Sg/qNGAJGQtYbdscZ+bBHnd6xPdeJ LPHnYRswJYiMw5O+wDFppu39sjGfBo8N96ebG3VjE6IbmzRkrw284rtHPEhHVolj/2bE /ptHAToNfnhOkI8rMhLfbHUtMr6NNCCKrqFrpOdWC2bnLP2HLHCukwbZPl8GY4ThegfO qJt81MQ/NMvdhWZYsipc88cEtNIyFEJOFnTb0MB8THEILBQCd9PL99cTs4/N12rmgyQk TFZHTIkfyn09e5vkJPkgT0AIkucyzh/TJqNGplcNBATxe8GXuN/R4NRIQFT2vm/bg19t pRvg== Received: by 10.50.186.199 with SMTP id fm7mr11561674igc.62.1353944233594; Mon, 26 Nov 2012 07:37:13 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.50.67.148 with SMTP id n20csp283748igt; Mon, 26 Nov 2012 07:37:12 -0800 (PST) Received: by 10.216.143.105 with SMTP id k83mr3741182wej.15.1353944232106; Mon, 26 Nov 2012 07:37:12 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id a6si8016562wie.36.2012.11.26.07.37.11 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 26 Nov 2012 07:37:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Td0kB-0007Lp-BQ for ; Mon, 26 Nov 2012 15:37:11 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 49DBEE02DE for ; Mon, 26 Nov 2012 15:37:11 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 466 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 466: Clean the console ouput of telnet command, so that the telnet process will not consume the CPU us... Message-Id: <20121126153711.15207.21329.launchpad@ackee.canonical.com> Date: Mon, 26 Nov 2012 15:37:11 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="16293"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: a72e80306001cbf25692b484bf2c5624d35634bd X-Gm-Message-State: ALoCoQkVL4khoa0TGwDbdXA8JBpOL+lkSiD/3zddpjohg9i8TGlISltDagETpkZcf72UJ9d0OhJq Merge authors: Yongqin Liu (liuyq0307) Related merge proposals: https://code.launchpad.net/~liuyq0307/lava-dispatcher/clean-console-when-test/+merge/135084 proposed by: Yongqin Liu (liuyq0307) ------------------------------------------------------------ revno: 466 [merge] committer: Yongqin Liu branch nick: lava-dispatcher timestamp: Mon 2012-11-26 23:18:32 +0800 message: Clean the console ouput of telnet command, so that the telnet process will not consume the CPU usage when run CTS test modified: lava_dispatcher/actions/lava_android_test.py lava_dispatcher/device/fastmodel.py lava_dispatcher/utils.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/actions/lava_android_test.py' --- lava_dispatcher/actions/lava_android_test.py 2012-11-20 21:22:17 +0000 +++ lava_dispatcher/actions/lava_android_test.py 2012-11-22 02:14:51 +0000 @@ -24,7 +24,7 @@ import logging from lava_dispatcher.actions import BaseAction from lava_dispatcher.errors import OperationFailed, TimeoutError -from lava_dispatcher.utils import generate_bundle_file_name +from lava_dispatcher.utils import generate_bundle_file_name, DrainConsoleOutput class AndroidTestAction(BaseAction): @@ -66,8 +66,11 @@ cmds.insert(0, 'timeout') cmds.insert(1, '%ss' % timeout) + t = DrainConsoleOutput(proc=session._connection, timeout=timeout) + t.start() logging.info("Execute command on host: %s" % (' '.join(cmds))) rc = subprocess.call(cmds) + t.join() if rc == 124: raise TimeoutError( "The test case(%s) on device(%s) times out" % ( === modified file 'lava_dispatcher/device/fastmodel.py' --- lava_dispatcher/device/fastmodel.py 2012-11-20 13:34:19 +0000 +++ lava_dispatcher/device/fastmodel.py 2012-11-22 02:14:51 +0000 @@ -50,6 +50,7 @@ extract_targz, logging_spawn, logging_system, + DrainConsoleOutput, ) @@ -184,7 +185,7 @@ f = cStringIO.StringIO() self._sim_proc.logfile = self._create_rtsm_ostream(f) - _pexpect_drain(self._sim_proc).start() + DrainConsoleOutput(proc=self._sim_proc).start() def power_on(self): self._fix_perms() @@ -235,18 +236,4 @@ return "unknown" -class _pexpect_drain(threading.Thread): - ''' The simulator process can dump a lot of information to its console. If - don't actively read from it, the pipe will get full and the process will - be blocked. This allows us to keep the pipe empty so the process can run - ''' - def __init__(self, proc): - threading.Thread.__init__(self) - self.proc = proc - - self.daemon = True # allow thread to die when main main proc exits - - def run(self): - self.proc.drain() - target_class = FastModelTarget === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2012-11-20 21:22:17 +0000 +++ lava_dispatcher/utils.py 2012-11-22 03:03:40 +0000 @@ -23,10 +23,10 @@ import errno import logging import os -import select -import sys import shutil import tempfile +import threading +import time import urlparse from shlex import shlex @@ -136,6 +136,31 @@ return os.system(cmd) +class DrainConsoleOutput(threading.Thread): + + def __init__(self, proc=None, timeout=None): + threading.Thread.__init__(self) + self.proc = proc + self.timeout = timeout + self._stopevent = threading.Event() + self.daemon = True # allow thread to die when main main proc exits + + def run(self): + expect_end = None + if self.timeout and (self.timeout > -1): + expect_end = time.time() + self.timeout + while not self._stopevent.isSet(): + if expect_end and (expect_end <= time.time()): + logging.info("DrainConsoleOutput times out:%s" % self.timeout) + break + self.proc.empty_buffer() + time.sleep(5) + + def join(self, timeout=None): + self._stopevent.set() + threading.Thread.join(self, timeout) + + class logging_spawn(pexpect.spawn): def sendline(self, *args, **kw): @@ -169,23 +194,8 @@ index = 0 while index == 0: index = self.expect( - ['.+', pexpect.EOF, pexpect.TIMEOUT], timeout=1, lava_no_logging=1) - - def drain(self): - """this is a one-off of the pexect __interact that ignores STDIN and - handles an error that happens when we call read just after the process - exits - """ - try: - self._spawn__interact_copy(escape_character=chr(29)) - except: - einfo = sys.exc_info() - # since we blindly read this from a thread, it will always wind up - # dying with a select error. we should still make note of other - # exceptions that might happen - if not isinstance(einfo[1], select.error): - logging.warn("error while draining pexpect buffers: %r", einfo) - pass + ['.+', pexpect.EOF, pexpect.TIMEOUT], + timeout=1, lava_no_logging=1) # XXX Duplication: we should reuse lava-test TestArtifacts