From patchwork Fri Apr 20 03:33:12 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael-Doyle Hudson X-Patchwork-Id: 7970 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id CD2CF23E20 for ; Fri, 20 Apr 2012 03:33:15 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 71C65A18184 for ; Fri, 20 Apr 2012 03:33:15 +0000 (UTC) Received: by iage36 with SMTP id e36so17544184iag.11 for ; Thu, 19 Apr 2012 20:33:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf :content-type:mime-version:x-launchpad-project:x-launchpad-branch :x-launchpad-message-rationale:x-launchpad-branch-revision-number :x-launchpad-notification-type:to:from:subject:message-id:date :reply-to:sender:errors-to:precedence:x-generated-by :x-launchpad-hash:x-gm-message-state; bh=oIPafEzfncxChUJMNtfHkzrWgc2mPOf/5qbQjcn7HYI=; b=UyhG0TYV8H8rqJyZHs11Cqdw9gITtoCPgt1AY/Flp8OfZoQEfl96e9OrbFEilaAsem 6Kfjm1ISwH373TNL4OqzdI2x+cFtrI10tUh5KUEt+OlAgvlMcim5XoHWB+EiFmpmsMp5 Dr21y32Cr3Iol3eNnSg/UPLY+TfyGYimjBnhJCYYn55BZeUzBDjMpvUR3HWRv+yoWygg 1PdjDwEonww/mk6aD66Sda4h2Zoz/DMO7QYgYpn33CnD0t6NbLPhmyAHne99pfH+1NZL 03vrpnPRL/C5K7D31brXtJFRD+HjP9+CGDjB7pGBGMbRv297m7EyDwDqv6UtT9C2Mh/v Vt9A== Received: by 10.50.135.36 with SMTP id pp4mr4442375igb.19.1334892794895; Thu, 19 Apr 2012 20:33:14 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.137.198 with SMTP id x6csp6804ibt; Thu, 19 Apr 2012 20:33:14 -0700 (PDT) Received: by 10.216.138.223 with SMTP id a73mr2770912wej.86.1334892793550; Thu, 19 Apr 2012 20:33:13 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id l34si4619066weq.71.2012.04.19.20.33.12 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 19 Apr 2012 20:33:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1SL4au-0003fj-CP for ; Fri, 20 Apr 2012 03:33:12 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 5221FE03F1 for ; Fri, 20 Apr 2012 03:33:12 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-scheduler X-Launchpad-Branch: ~linaro-validation/lava-scheduler/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 163 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-scheduler/trunk] Rev 163: Allow job files to specify addresses to email on completion Message-Id: <20120420033312.7777.89309.launchpad@ackee.canonical.com> Date: Fri, 20 Apr 2012 03:33:12 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="15120"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: fc78a1f8a7949cfbd2104298134592b973f9fb3b X-Gm-Message-State: ALoCoQl9Z0tZKsi7JMpwouaxVyIaEwXIGMG8GqH19IZL5334w9DIhsBLQ8ESxrrArJYEj61mVFu1 Merge authors: Michael Hudson-Doyle (mwhudson) Related merge proposals: https://code.launchpad.net/~mwhudson/lava-scheduler/email-on-health-check-failure/+merge/100890 proposed by: Michael Hudson-Doyle (mwhudson) review: Approve - Zygmunt Krynicki (zkrynicki) ------------------------------------------------------------ revno: 163 [merge] committer: Michael Hudson-Doyle branch nick: trunk timestamp: Fri 2012-04-20 15:31:21 +1200 message: Allow job files to specify addresses to email on completion (possibly only unsuccessful completion). added: lava_scheduler_app/templates/lava_scheduler_app/job_summary_mail.txt modified: doc/changes.rst lava_scheduler_app/models.py lava_scheduler_daemon/dbjobsource.py --- lp:lava-scheduler https://code.launchpad.net/~linaro-validation/lava-scheduler/trunk You are subscribed to branch lp:lava-scheduler. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-scheduler/trunk/+edit-subscription === modified file 'doc/changes.rst' --- doc/changes.rst 2012-04-02 00:06:13 +0000 +++ doc/changes.rst 2012-04-20 03:30:58 +0000 @@ -2,6 +2,12 @@ *************** +Version 0.13 +============ + +* Allow job files to specify addresses to email on completion + (possibly only unsuccessful completion). + .. _version_0_12_1: Version 0.12.1 === modified file 'lava_scheduler_app/models.py' --- lava_scheduler_app/models.py 2012-03-15 20:44:39 +0000 +++ lava_scheduler_app/models.py 2012-04-20 03:22:44 +0000 @@ -1,13 +1,19 @@ +import logging import simplejson +from django.conf import settings from django.contrib.auth.models import User -from django.core.exceptions import ValidationError +from django.contrib.sites.models import Site +from django.core.exceptions import ImproperlyConfigured, ValidationError +from django.core.mail import send_mail +from django.core.validators import validate_email from django.db import models +from django.template.loader import render_to_string from django.utils.translation import ugettext as _ from django_restricted_resource.models import RestrictedResource -from dashboard_app.models import BundleStream +from dashboard_app.models import Bundle, BundleStream from lava_dispatcher.job import validate_job_data @@ -237,20 +243,41 @@ blank = True, editable = False ) + + @property + def duration(self): + if self.end_time is None: + return None + return self.end_time - self.start_time + status = models.IntegerField( choices = STATUS_CHOICES, default = SUBMITTED, verbose_name = _(u"Status"), ) + definition = models.TextField( editable = False, ) + log_file = models.FileField( upload_to='lava-logs', default=None, null=True, blank=True) results_link = models.CharField( max_length=400, default=None, null=True, blank=True) + @property + def results_bundle(self): + # XXX So this is clearly appalling (it depends on the format of bundle + # links, for example). We should just have a fkey to Bundle. + if not self.results_link: + return None + sha1 = self.results_link.strip('/').split('/')[-1] + try: + return Bundle.objects.get(content_sha1=sha1) + except Bundle.DoesNotExist: + return None + def __unicode__(self): r = "%s test job" % self.get_status_display() if self.requested_device: @@ -274,6 +301,23 @@ else: raise JSONDataError( "Neither 'target' nor 'device_type' found in job data.") + + for email_field in 'notify', 'notify_on_incomplete': + if email_field in job_data: + value = job_data[email_field] + msg = ("%r must be a list of email addresses if present" + % email_field) + if not isinstance(value, list): + raise ValueError(msg) + for address in value: + if not isinstance(address, basestring): + raise ValueError(msg) + try: + validate_email(address) + except ValidationError: + raise ValueError( + "%r is not a valid email address." % address) + job_name = job_data.get('job_name', '') is_check = job_data.get('health_check', False) @@ -324,6 +368,40 @@ self.status = TestJob.CANCELED self.save() + def _generate_summary_mail(self): + domain = '???' + try: + site = Site.objects.get_current() + except (Site.DoesNotExist, ImproperlyConfigured): + pass + else: + domain = site.domain + url_prefix = 'http://%s' % domain + return render_to_string( + 'lava_scheduler_app/job_summary_mail.txt', + {'job': self, 'url_prefix': url_prefix}) + + def _get_notification_recipients(self): + job_data = simplejson.loads(self.definition) + recipients = job_data.get('notify', []) + if self.status != self.COMPLETE: + recipients.extend(job_data.get('notify_on_incomplete', [])) + return recipients + + def send_summary_mails(self): + recipients = self._get_notification_recipients() + if not recipients: + return + mail = self._generate_summary_mail() + description = self.description.splitlines()[0] + if len(description) > 200: + description = description[197:] + '...' + logger = logging.getLogger(self.__class__.__name__ + '.' + str(self.pk)) + logger.info("sending mail to %s", recipients) + send_mail( + "LAVA job notification: " + description, mail, + settings.SERVER_EMAIL, recipients) + class DeviceStateTransition(models.Model): created_on = models.DateTimeField(auto_now_add=True) === added file 'lava_scheduler_app/templates/lava_scheduler_app/job_summary_mail.txt' --- lava_scheduler_app/templates/lava_scheduler_app/job_summary_mail.txt 1970-01-01 00:00:00 +0000 +++ lava_scheduler_app/templates/lava_scheduler_app/job_summary_mail.txt 2012-04-05 04:34:13 +0000 @@ -0,0 +1,27 @@ +Hi, + +The job with id {{ job.id }} has finished. It took {{ job.start_time|timesince:job.end_time }}. + +The final status was {{ job.get_status_display }}. + +You can see more details at: + + {{ url_prefix }}{{ job.get_absolute_url }} +{% if job.results_bundle %} +The results can be summarized as: + + +----------------------+--------+--------+ + | Test run | Passes | Total | + +----------------------+--------+--------+ +{% for run in job.results_bundle.test_runs.all %}{% with results=run.get_summary_results %} | {{ run.test.test_id|ljust:20 }} | {{ results.pass|default:0|rjust:6 }} | {{ results.total|default:0|rjust:6 }} | +{% endwith %}{% endfor %} +----------------------+--------+--------+ + +For more details, please see: + + {{ url_prefix }}{{ job.results_bundle.get_absolute_url }} + +{% else %} +No results were reported to the dashboard for this run. + +{% endif %}LAVA +Linaro Automated Validation === modified file 'lava_scheduler_daemon/dbjobsource.py' --- lava_scheduler_daemon/dbjobsource.py 2012-03-09 01:27:51 +0000 +++ lava_scheduler_daemon/dbjobsource.py 2012-04-05 04:34:13 +0000 @@ -33,7 +33,8 @@ implements(IJobSource) - logger = logging.getLogger(__name__ + '.DatabaseJobSource') + def __init__(self): + self.logger = logging.getLogger(__name__ + '.DatabaseJobSource') deferToThread = staticmethod(deferToThread) @@ -235,7 +236,7 @@ created_by=None, device=device, old_state=old_device_status, new_state=device.status, message=None, job=job).save() - if job.health_check is True: + if job.health_check: device.last_health_report_job = job if job.status == TestJob.INCOMPLETE: device.health_status = Device.HEALTH_FAIL @@ -249,6 +250,13 @@ device.save() job.save() token.delete() + try: + job.send_summary_mails() + except: + # Better to catch all exceptions here and log it than have this + # method fail. + self.logger.exception( + 'sending job summary mails for job %r failed', job.pk) transaction.commit() def jobCompleted(self, board_name, exit_code):