From patchwork Fri Oct 14 20:31:12 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Larson X-Patchwork-Id: 4691 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id D939223DEC for ; Fri, 14 Oct 2011 20:31:14 +0000 (UTC) Received: from mail-ey0-f180.google.com (mail-ey0-f180.google.com [209.85.215.180]) by fiordland.canonical.com (Postfix) with ESMTP id C0B25A18031 for ; Fri, 14 Oct 2011 20:31:14 +0000 (UTC) Received: by eyg5 with SMTP id 5so2044518eyg.11 for ; Fri, 14 Oct 2011 13:31:14 -0700 (PDT) Received: by 10.223.85.134 with SMTP id o6mr6007832fal.8.1318624274356; Fri, 14 Oct 2011 13:31:14 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.152.24.41 with SMTP id r9cs29477laf; Fri, 14 Oct 2011 13:31:14 -0700 (PDT) Received: by 10.216.132.136 with SMTP id o8mr419585wei.84.1318624273402; Fri, 14 Oct 2011 13:31:13 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id c1si4174874wed.24.2011.10.14.13.31.13 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 14 Oct 2011 13:31:13 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) smtp.mail=bounces@canonical.com Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1REoPQ-0001yy-QK for ; Fri, 14 Oct 2011 20:31:12 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id B9DECE01FB for ; Fri, 14 Oct 2011 20:31:12 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: lava-test X-Launchpad-Branch: ~linaro-validation/lava-test/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 101 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-validation/lava-test/trunk] Rev 101: Merge Insanity multimedia tests from Collabora Message-Id: <20111014203112.13895.19751.launchpad@ackee.canonical.com> Date: Fri, 14 Oct 2011 20:31:12 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="14124"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 8241963c53c22daebd79e0f22771518e88392127 Merge authors: Le Chi Thu le.chi.thu@linaro.org Related merge proposals: https://code.launchpad.net/~le-chi-thu/lava-test/insanity_2/+merge/79249 proposed by: Le Chi Thu (le-chi-thu) review: Approve - Paul Larson (pwlars) ------------------------------------------------------------ revno: 101 [merge] committer: Paul Larson branch nick: lava-test timestamp: Fri 2011-10-14 15:29:26 -0500 message: Merge Insanity multimedia tests from Collabora added: lava_test/test_definitions/insanity.py modified: lava_test/core/providers.py --- lp:lava-test https://code.launchpad.net/~linaro-validation/lava-test/trunk You are subscribed to branch lp:lava-test. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-validation/lava-test/trunk/+edit-subscription === modified file 'lava_test/core/providers.py' --- lava_test/core/providers.py 2011-09-27 20:27:39 +0000 +++ lava_test/core/providers.py 2011-10-13 10:20:52 +0000 @@ -26,6 +26,7 @@ _builtin_tests = [ 'bootchart', + 'insanity', 'firefox', 'glmemperf', 'gmpbench', === added file 'lava_test/test_definitions/insanity.py' --- lava_test/test_definitions/insanity.py 1970-01-01 00:00:00 +0000 +++ lava_test/test_definitions/insanity.py 2011-10-13 10:20:52 +0000 @@ -0,0 +1,141 @@ +# Copyright (c) 2010 Linaro +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import re +import subprocess +import simplejson +import gzip +import base64 + +from lava_test.core.installers import TestInstaller +from lava_test.core.parsers import TestParser +from lava_test.core.runners import TestRunner +from lava_test.core.tests import Test + + +MAX_TEST_CASE_ID_LEN = 100 +MAX_ATTR_KEY_LEN = 32 +MAX_ATTR_VAL_LEN = 512 + +MAX_ATTACHMENT_SIZE = 10 * 1024 * 1024 + +SETTINGS = "/usr/share/insanity/web/settings.py" + +# Re-running failing tests is expensive, as it produces very large log files. +# Only remove --no-reruns if you have a small number of failing tests. +RUNSTEPS = ["rm -f testrun.db", + "gst-media-test --no-reruns -t playbin-test --settings %s" % SETTINGS, + "gst-media-test --no-reruns -t full-gnlfilesource-scenario " \ + "--settings %s" % SETTINGS, + "gst-media-test --no-reruns -t simple-encoder-scenario", + "echo ////////////////////", + "insanity-dumpresults-json testrun.db --all", + ] + +class InsanityParser(TestParser): + def parse(self, artifacts): + filename = "testoutput.log" + with open(filename, 'r') as stream: + while not stream.readline().startswith("//////////"): + pass + results = simplejson.load(stream) + + self._results = results + self.fixresults({"pass": "pass", "fail": "fail", + "skip": "skip", "expected-failure": "pass"}) + self.fixlengths() + self.attach_logfiles() + + + def fixlengths(self): + for t in self.results['test_results']: + if t.has_key("test_case_id"): + if len(t["test_case_id"]) > MAX_TEST_CASE_ID_LEN: + t["test_case_id"] = \ + t["test_case_id"][-MAX_TEST_CASE_ID_LEN:] + if t.has_key("attributes"): + attributes = t["attributes"] + for k, v in attributes.items(): + if len(k) > MAX_ATTR_KEY_LEN: + attributes.pop(k) + # start includes namespace info + k = k[:MAX_ATTR_KEY_LEN] + attributes[k] = v + if len(v) > MAX_ATTR_VAL_LEN: + # end tends to be more useful than the start. + attributes[k] = v[-MAX_ATTR_VAL_LEN:] + + + def attach_logfiles(self): + attachments = [] + mime_type = "text/plain" + total_attachment_size = 0 + + for test in self.results["test_results"]: + pathname = test.get("log_filename", "") + if not pathname: + continue + if not os.path.exists(pathname): + print "%r not found: skipping." % pathname + continue + + if pathname.endswith(".gz"): + stream = gzip.open(pathname, 'rb') + else: + stream = open(pathname) + + output_text = stream.read() + stream.close() + + total_attachment_size += len(output_text) + if total_attachment_size > MAX_ATTACHMENT_SIZE: + break + + attachments.append({ + "pathname": pathname, + "mime_type": mime_type, + "content": base64.standard_b64encode(output_text) + }) + + self.results.setdefault("attachments", []).extend(attachments) + + def fixresults(self, fixupdict): + """Convert results to a known, standard format + + pass it a dict of keys/values to replace + For instance: + {"TPASS":"pass", "TFAIL":"fail"} + This is really only used for qualitative tests + """ + for t in self.results['test_results']: + if t.has_key("result"): + t['result'] = fixupdict[t['result']] + +inst = TestInstaller(deps=["insanity-tools", + "samplemedia-minimal", + "gstreamer0.10-plugins-base", # videotestsrc et al + "gstreamer0.10-plugins-good", # matroskademux et al + "gstreamer0.10-plugins-bad", # + "gstreamer0.10-plugins-ugly", # asfdemux et al + "gstreamer0.10-ffmpeg", # ffdec_h264 et al + "gstreamer0.10-gnonlin", # gnlfilesource + "gdb", # debugging backtraces + ]) +run = TestRunner(RUNSTEPS) +parse = InsanityParser("") + +testobj = Test(test_id="insanity", installer=inst, runner=run, parser=parse)