From patchwork Fri Jan 27 03:25:16 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Larson X-Patchwork-Id: 6418 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id DA89723E16 for ; Fri, 27 Jan 2012 03:25:17 +0000 (UTC) Received: from mail-bk0-f52.google.com (mail-bk0-f52.google.com [209.85.214.52]) by fiordland.canonical.com (Postfix) with ESMTP id BF21EA181F4 for ; Fri, 27 Jan 2012 03:25:17 +0000 (UTC) Received: by bkar19 with SMTP id r19so1401177bka.11 for ; Thu, 26 Jan 2012 19:25:17 -0800 (PST) Received: by 10.205.130.12 with SMTP id hk12mr2214212bkc.56.1327634717456; Thu, 26 Jan 2012 19:25:17 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.204.130.220 with SMTP id u28cs59447bks; Thu, 26 Jan 2012 19:25:17 -0800 (PST) Received: by 10.180.101.35 with SMTP id fd3mr7919190wib.22.1327634716743; Thu, 26 Jan 2012 19:25:16 -0800 (PST) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id h2si5255915wed.83.2012.01.26.19.25.16 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 26 Jan 2012 19:25:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1RqcRA-0004UP-FJ for ; Fri, 27 Jan 2012 03:25:16 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 63F3EE04EB for ; Fri, 27 Jan 2012 03:25:16 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-dispatcher X-Launchpad-Branch: ~linaro-validation/lava-dispatcher/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 206 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-dispatcher/trunk] Rev 206: retry until timeout for getting results Message-Id: <20120127032516.12733.64574.launchpad@ackee.canonical.com> Date: Fri, 27 Jan 2012 03:25:16 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14727"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 28e447faff6e9469e3ba4b4b2add999ae48b5ce5 Merge authors: Le Chi Thu le.chi.thu@linaro.org Related merge proposals: https://code.launchpad.net/~le-chi-thu/lava-dispatcher/fix-download-result-fail/+merge/90248 proposed by: Le Chi Thu (le-chi-thu) review: Approve - Paul Larson (pwlars) ------------------------------------------------------------ revno: 206 [merge] committer: Paul Larson branch nick: chithu-logging-and-results-timeout timestamp: Thu 2012-01-26 21:23:09 -0600 message: retry until timeout for getting results modified: lava_dispatcher/client/master.py lava_dispatcher/utils.py --- lp:lava-dispatcher https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk You are subscribed to branch lp:lava-dispatcher. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-dispatcher/trunk/+edit-subscription === modified file 'lava_dispatcher/client/master.py' --- lava_dispatcher/client/master.py 2012-01-26 10:14:07 +0000 +++ lava_dispatcher/client/master.py 2012-01-27 03:23:09 +0000 @@ -485,29 +485,28 @@ os.chmod(tarball_dir, 0755) # download test result with a retry mechanism - # set retry timeout to 2mins + # set retry timeout to 5 mins logging.info("About to download the result tarball to host") now = time.time() - timeout = 120 + timeout = 300 tries = 0 - try: - while time.time() < now + timeout: - try: - result_path = download( - result_tarball, tarball_dir, - verbose_failure=tries==0) - except RuntimeError: - tries += 1 - if time.time() >= now + timeout: - logging.exception("download failed") - raise - except: - logging.warning(traceback.format_exc()) - err_msg = err_msg + " Can't retrieve test case results." - logging.warning(err_msg) - return 'fail', err_msg, None - return 'pass', None, result_path + while True: + try: + result_path = download( + result_tarball, tarball_dir,False) + return 'pass', None, result_path + except RuntimeError: + tries += 1 + if time.time() >= now + timeout: + logging.error( + "download '%s' failed. Nr tries = %s" % ( + result_tarball, tries)) + return 'fail', err_msg, None + else: + logging.info( + "Sleep one minute and retry (%d)" % tries) + time.sleep(60) finally: session.run('kill %1') session.run('') === modified file 'lava_dispatcher/utils.py' --- lava_dispatcher/utils.py 2012-01-23 21:32:05 +0000 +++ lava_dispatcher/utils.py 2012-01-26 10:11:27 +0000 @@ -40,7 +40,7 @@ response.close() except: if verbose_failure: - logging.exception("download failed") + logging.exception("download '%s' failed" % url) raise RuntimeError("Could not retrieve %s" % url) return filename