From patchwork Wed Jul 6 14:18:23 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Tunnicliffe X-Patchwork-Id: 2481 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 9B3A62413B for ; Wed, 6 Jul 2011 14:18:28 +0000 (UTC) Received: from mail-qy0-f180.google.com (mail-qy0-f180.google.com [209.85.216.180]) by fiordland.canonical.com (Postfix) with ESMTP id DD941A18032 for ; Wed, 6 Jul 2011 14:18:27 +0000 (UTC) Received: by mail-qy0-f180.google.com with SMTP id 30so4864001qyk.11 for ; Wed, 06 Jul 2011 07:18:27 -0700 (PDT) Received: by 10.229.1.140 with SMTP id 12mr6512571qcf.118.1309961907599; Wed, 06 Jul 2011 07:18:27 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.229.48.135 with SMTP id r7cs80341qcf; Wed, 6 Jul 2011 07:18:26 -0700 (PDT) Received: by 10.227.176.72 with SMTP id bd8mr7425287wbb.72.1309961904557; Wed, 06 Jul 2011 07:18:24 -0700 (PDT) Received: from adelie.canonical.com (adelie.canonical.com [91.189.90.139]) by mx.google.com with ESMTP id gb12si12149078wbb.139.2011.07.06.07.18.23; Wed, 06 Jul 2011 07:18:24 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.139 as permitted sender) client-ip=91.189.90.139; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.139 as permitted sender) smtp.mail=bounces@canonical.com Received: from loganberry.canonical.com ([91.189.90.37]) by adelie.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1QeSvn-00059S-G3 for ; Wed, 06 Jul 2011 14:18:23 +0000 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id 711332E890E for ; Wed, 6 Jul 2011 14:18:23 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: linaro-image-tools X-Launchpad-Branch: ~linaro-image-tools/linaro-image-tools/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 360 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-image-tools/linaro-image-tools/trunk] Rev 360: Merge from trunk. Message-Id: <20110706141823.7642.9298.launchpad@loganberry.canonical.com> Date: Wed, 06 Jul 2011 14:18:23 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="13376"; Instance="initZopeless config overlay" X-Launchpad-Hash: 5fe7cc1feac79d52a866eedfbaddb05f2d0ef0dd Merge authors: Angus Ainslie (angus-akkea) James Westby (james-w) Markus Andersson (markus-m-andersson) Mattias Backman (mabac) Ola Borgelin (olab) Related merge proposals: https://code.launchpad.net/~mabac/linaro-image-tools/hwpacks-v2-minsizes/+merge/66339 proposed by: Mattias Backman (mabac) review: Approve - James Westby (james-w) https://code.launchpad.net/~mabac/linaro-image-tools/ste-snowball-android/+merge/66297 proposed by: Ola Borgelin (olab) review: Approve - Mattias Backman (mabac) review: Approve - James Westby (james-w) https://code.launchpad.net/~mabac/linaro-image-tools/use-absolute-imports/+merge/66167 proposed by: Mattias Backman (mabac) https://code.launchpad.net/~mabac/linaro-image-tools/hwpacks-v2-lmc-samsung/+merge/66105 proposed by: Mattias Backman (mabac) review: Approve - James Westby (james-w) https://code.launchpad.net/~mabac/linaro-image-tools/hwpacks-v2-lmc/+merge/65669 proposed by: Mattias Backman (mabac) review: Approve - James Westby (james-w) https://code.launchpad.net/~linaro-landing-team-samsung/linaro-image-tools/v310-spl-name-fix/+merge/65599 proposed by: Angus Ainslie (angus-akkea) https://code.launchpad.net/~mabac/linaro-image-tools/bug-800686/+merge/65504 proposed by: Mattias Backman (mabac) review: Approve - Zach Pfeffer (pfefferz) https://code.launchpad.net/~mabac/linaro-image-tools/chdir-for-hashverify/+merge/65221 proposed by: Mattias Backman (mabac) review: Approve - James Westby (james-w) https://code.launchpad.net/~mabac/linaro-image-tools/hwpacks-v2-hwpack-create/+merge/60177 proposed by: Mattias Backman (mabac) review: Approve - James Westby (james-w) ------------------------------------------------------------ revno: 360 [merge] committer: James Tunnicliffe branch nick: linaro-image-tools timestamp: Wed 2011-07-06 15:16:10 +0100 message: Merge from trunk. Added fetch_image_ui.py added: fetch_image_ui.py linaro_image_tools/hwpack/hardwarepack_format.py modified: linaro-android-media-create linaro-hwpack-install linaro-media-create linaro_image_tools/FetchImage.py linaro_image_tools/cmd_runner.py linaro_image_tools/hwpack/builder.py linaro_image_tools/hwpack/config.py linaro_image_tools/hwpack/hardwarepack.py linaro_image_tools/hwpack/packages.py linaro_image_tools/hwpack/tests/test_builder.py linaro_image_tools/hwpack/tests/test_config.py linaro_image_tools/hwpack/tests/test_hardwarepack.py linaro_image_tools/index_server.py linaro_image_tools/media_create/android_boards.py linaro_image_tools/media_create/boards.py linaro_image_tools/media_create/partitions.py linaro_image_tools/media_create/tests/test_media_create.py linaro_image_tools/tests/fixtures.py linaro_image_tools/tests/test_pyflakes.py linaro_image_tools/tests/test_utils.py linaro_image_tools/utils.py setup.py --- lp:linaro-image-tools https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk You are subscribed to branch lp:linaro-image-tools. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk/+edit-subscription === added file 'fetch_image_ui.py' --- fetch_image_ui.py 1970-01-01 00:00:00 +0000 +++ fetch_image_ui.py 2011-07-06 14:16:10 +0000 @@ -0,0 +1,1661 @@ +#!/usr/bin/env python +# Copyright (C) 2010, 2011 Linaro +# +# Author: James Tunnicliffe +# +# This file is part of Linaro Image Tools. +# +# Linaro Image Tools is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# Linaro Image Tools is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Linaro Image Tools; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, +# USA. + +import wx +import wx.wizard +import wx.wizard as wiz +import sys +import re +import os +import linaro_image_tools.FetchImage as FetchImage +import string +import unittest +import operator +import Queue + + +def add_button(bind_to, + sizer, + label, + style, + select_event, + hover_event, + unhover_event): + + """Create a radio button with event bindings.""" + if(style != None): + radio_button = wx.RadioButton(bind_to, label=label, style=style) + else: + radio_button = wx.RadioButton(bind_to, label=label) + + sizer.Add(radio_button, 0, wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, 5) + bind_to.Bind(wx.EVT_RADIOBUTTON, select_event, radio_button) + wx.EVT_ENTER_WINDOW(radio_button, hover_event) + wx.EVT_LEAVE_WINDOW(radio_button, unhover_event) + + return radio_button + + +class ReleaseOrSnapshotPage(wiz.PyWizardPage): + """Ask the user if they want to use a release or a snapshot""" + + def __init__(self, parent, config): + wiz.PyWizardPage.__init__(self, parent) + self.config = config + self.settings = self.config.settings + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.next = None + self.prev = None + + self.sizer.Add(wx.StaticText(self, -1, +"""This Wizard will write an operating system of your choosing to +either a disk image or to an MMC card. First we need to know if +your priority is stability or the latest and greatest features.""")) + + self.box1 = wx.BoxSizer(wx.VERTICAL) + + self.button_text = {'release': "I would like to run stable, " + "tested software.", + 'snapshot': "I would like to run untested, but " + "more up-to-date software."} + + add_button(self, self.box1, self.button_text['release'], + wx.RB_GROUP, self.event_radio_button_select, None, None) + + # Save the setting for the default selected value + self.settings['release_or_snapshot'] = "release" + + add_button(self, self.box1, self.button_text['snapshot'], None, + self.event_radio_button_select, None, None) + + self.sizer.Add(self.box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.SetSizerAndFit(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def event_radio_button_select(self, event): + self.radio_selected = event.GetEventObject().GetLabel() + # The radio button can be release, snapshot or "latest snapshot" + if(self.radio_selected == self.button_text['release']): + self.settings['release_or_snapshot'] = "release" + else: + self.settings['release_or_snapshot'] = "snapshot" + + def SetNext(self, next): + self.next = next + + def GetNext(self): + return self.next + + +class AboutMyHardwarePage(wiz.WizardPageSimple): + """Ask the user about their hardware. This only asks about the board, not + any specific hardware packs because there can be multiple names for the + same hardware pack or sometimes a hardware pack is only available in the + releases or snapshots repository. We whittle down the choice as we go + and the user can chose a hardare pack (if they don't like the default) + under advanced options in the Linaro Media Create options + page""" + + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.db = db + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.box1 = wx.BoxSizer(wx.VERTICAL) + self.box2 = wx.BoxSizer(wx.VERTICAL) + + header = wx.StaticText(self, + label = "Please select the hardware that you " + "would like to build an image for from " + "the following list") + + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + #--- Hardware Combo Box --- + # Make sure that the displayed release is the one set in settings if + # no selection is made + if "panda" in self.settings['choice']['hardware'].keys(): + default_hardware = "panda" + else: + default_hardware = self.settings['choice']['hardware'].keys()[-1] + + self.settings['hardware'] = default_hardware + self.settings['compatable_hwpacks'] = ( + self.settings['choice']['hwpack'][self.settings['hardware']]) + + self.cb_hardware = wx.ComboBox(self, + value = + self.settings['choice']['hardware'][default_hardware], + style = wx.CB_DROPDOWN | wx.CB_READONLY) + + self.Bind(wx.EVT_COMBOBOX, + self.event_combo_box_hardware, + self.cb_hardware) + self.box1.Add(self.cb_hardware, 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, 5) + + self.sizer.Add(header) + self.sizer.Add(self.box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + self.sizer.Add(self.box2, 0, wx.ALIGN_LEFT | wx.ALL, 5) + self.SetSizerAndFit(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def on_page_changing(self): + self.update_hardware_box() + + def update_hardware_box(self): + self.cb_hardware.Clear() + + sorted_hardware_names = sorted(self.settings['choice']['hardware'] + .iteritems(), + key=operator.itemgetter(1)) + + table = self.settings['release_or_snapshot'] + "_hwpacks" + + for device_name, human_readable_name in sorted_hardware_names: + for hwpack in self.settings['choice']['hwpack'][device_name]: + if self.db.hardware_is_available_in_table(table, hwpack): + self.cb_hardware.Append(human_readable_name, device_name) + break + + #--- Event(s) --- + def event_combo_box_hardware(self, event): + self.settings['hardware'] = (event + .GetEventObject() + .GetClientData(event.GetSelection()) + .encode('ascii')) + + self.settings['compatable_hwpacks'] = ( + self.settings['choice']['hwpack'][self.settings['hardware']]) + #--- END event(s) --- + + +class SelectStableRelease(wiz.WizardPageSimple): + """Ask the user which Linaro release they would like to run.""" + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.db = db + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.wizard = parent + + header = wx.StaticText(self, label = "Please select the stable Linaro " + "release you would like to use") + + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + self.sizer.Add(header) + self.box1 = wx.BoxSizer(wx.VERTICAL) + + platforms = [] + for key, value in self.settings['choice']['platform'].items(): + platforms.append(key) + + default_release = self.settings['UI']['translate'][platforms[-1]] + self.cb_release = wx.ComboBox(self, + value = default_release, + style = wx.CB_DROPDOWN | wx.CB_READONLY) + self.Bind(wx.EVT_COMBOBOX, + self.event_combo_box_release, + self.cb_release) + + if(default_release in self.settings['UI']['translate']): + default_release = self.settings['UI']['translate'][default_release] + self.settings['platform'] = ( + self.settings['UI']['reverse-translate'][default_release]) + + for item in platforms: + if(item in self.settings['UI']['translate']): + new_item = self.settings['UI']['translate'][item] + item = new_item + + self.cb_release.Append(item, item.upper()) + + self.cb_build = wx.ComboBox(self, + style = wx.CB_DROPDOWN | wx.CB_READONLY) + self.Bind(wx.EVT_COMBOBOX, self.event_combo_box_build, self.cb_build) + + self.box1.Add(self.cb_release, 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, 5) + self.box1.Add(self.cb_build, 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, 5) + self.sizer.Add(self.box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + self.SetSizerAndFit(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def update_build_box(self): + """Depending on what hardware has been chosen, the OS list may be + restricted. Filter out anything that is unavailable.""" + self.cb_build.Clear() + + builds = self.db.get_builds(self.settings['platform']) + self.cb_build.SetValue("No build available") + + for build in builds: + if( self.db.hardware_is_available_for_platform_build( + self.settings['compatable_hwpacks'], + self.settings['platform'], + build) + and self.db.build_is_available_for_platform_image( + "release_binaries", + self.settings['platform'], + self.settings['image'], + build)): + + self.cb_build.Append(build) + self.cb_build.SetValue(build) + self.settings['release_build'] = build + + available_hwpacks = ( + self.db.get_available_hwpacks_for_hardware_build_plaform( + self.settings['compatable_hwpacks'], + self.settings['platform'], + self.settings['release_build'])) + + if len(available_hwpacks): + self.settings['hwpack'] = available_hwpacks[0] + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + else: + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + + def update_release_and_build_boxes(self): + """Depending on what hardware has been chosen, some builds may be + unavailable...""" + self.cb_release.Clear() + + default_release = None + for platform, value in self.settings['choice']['platform'].items(): + if(self.db.hardware_is_available_for_platform( + self.settings['compatable_hwpacks'], + platform) + and len(self.db.execute_return_list( + 'select * from release_binaries ' + 'where platform == ? and image == ?', + (platform, self.settings['image'])))): + + if(platform in self.settings['UI']['translate']): + platform = self.settings['UI']['translate'][platform] + + self.cb_release.Append(platform, platform.upper()) + if not default_release or default_release < platform: + default_release = platform + + self.settings['platform'] = ( + self.settings['UI']['reverse-translate'][default_release]) + self.cb_release.SetValue(default_release) + self.update_build_box() + + #--- Event(s) --- + def event_combo_box_release(self, evt): + str = evt.GetString().encode('ascii').lower() + if(str in self.settings['UI']['reverse-translate']): + str = self.settings['UI']['reverse-translate'][str] + self.settings['platform'] = str + + self.update_build_box() + + def event_combo_box_build(self, evt): + self.settings['release_build'] = evt.GetString().encode('ascii') + #--- END event(s) --- + + +class SelectSnapshot(wiz.WizardPageSimple): + """Present the user with a calendar widget and a list of builds available + on the selected date so they can chose a snapshot. Filter out days when + their chosen hardware does not have an available build.""" + + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.db = db + self.wizard = parent + self.width = width + self.sizer = wx.BoxSizer(wx.VERTICAL) + + header = wx.StaticText(self, + label = "Builds are created most days. First " + "please select the day on which the " + "build you would like to use was built," + " then, if there was more than one " + "build that day you will be able to " + "select the build number.") + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + box1 = wx.BoxSizer(wx.VERTICAL) + self.sizer.Add(header) + + # Set today as the default build date in settings + # (matches the date picker) + self.today = wx.DateTime() + self.today.SetToCurrent() + self.settings['build_date'] = self.today.FormatISODate().encode('ascii') + + dpc = wx.DatePickerCtrl(self, size = (120, -1), + style = wx.DP_DEFAULT) + self.Bind(wx.EVT_DATE_CHANGED, self.on_date_changed, dpc) + + #--- Build number Combo Box --- + # Make sure that the displayed build is the one set in settings if no + # selection is made + self.settings['build_number'] = 0 + self.update_build() + self.cb_build = wx.ComboBox(self, + style = wx.CB_DROPDOWN | wx.CB_READONLY) + self.Bind(wx.EVT_COMBOBOX, self.event_combo_box_build, self.cb_build) + + #--- Layout --- + # -- Combo boxes for hardware and image selection -- + + grid2 = wx.FlexGridSizer(0, 2, 0, 0) + grid2.Add(dpc, 0, wx.ALIGN_LEFT | wx.ALL, 5) + grid2.Add(self.cb_build, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + box1.Add(grid2, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.sizer.Add(box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.help_text = wx.StaticText(self) + self.sizer.Add(self.help_text, 1, wx.EXPAND, 5) + + self.SetSizer(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def update_platform(self): + build_and_date = self.settings['snapshot_build'].split(":") + + if len(build_and_date) == 2: + self.settings['platform'] = ( + self.db.execute_return_list( + "select platform from snapshot_binaries " + "where date == ? and build == ?", + (build_and_date[0], build_and_date[1]))) + + if len(self.settings['platform']) > 0: + self.settings['platform'] = self.settings['platform'][0][0] + + def update_build(self): + small_date = re.sub('-', '', self.settings['build_date']) + self.settings['snapshot_build'] = (small_date + + ":" + + str(self.settings['build_number'])) + + def fill_build_combo_box_for_date(self, date): + """Every time a date is chosen, this function should be called. It will + check to see if a compatible build is available. If there isn't, it + will search for one and provide some help text to tell the user when + compatable builds were built.""" + # Re-populate the build combo box + + self.cb_build.Clear() + + builds = self.db.get_binary_builds_on_day_from_db( + self.settings['image'], + date, + self.settings['compatable_hwpacks']) + + if len(builds): + max = 0 + for item in builds: + #Always get a tuple, only interested in the first entry + item = item[0] + self.cb_build.Append(item, item.upper()) + + if item > max: + max = item + + self.cb_build.SetValue(max) + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + self.help_text.SetLabel("") + + else: + self.cb_build.SetValue("No builds available") + future_date, past_date = self.db.get_next_prev_day_with_builds( + self.settings['image'], + date, + self.settings['compatable_hwpacks']) + + help_text = None + + if future_date and past_date: + help_text = ("There are no builds that match your " + "specifications available on the selected date. " + "The previous build was on " + past_date + + " and the next build was on " + future_date + ".") + elif future_date: + help_text = ("There are no builds that match your " + "specifications available on the selected date. " + "The next build was on " + future_date + + " and I couldn't find a past build (looked one " + "year back from the selected date).") + elif past_date: + help_text = ("There are no builds that match your " + "specifications available on the selected date. " + "The previous build was on " + past_date) + if date != self.today.FormatISODate().encode('ascii'): + help_text += (" and I couldn't find a future build (I " + "looked up to one year forward from the " + "selected date).") + else: + help_text = ("I could not find any builds that match your " + "specifications close to the selected date (I " + "looked forward and back one year from the " + "selected date).") + + self.help_text.SetLabel(help_text) + self.help_text.Wrap(self.width - 10) + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + + #--- Event(s) --- + def on_date_changed(self, evt): + self.settings['build_date'] = evt.GetDate().FormatISODate().encode('ascii') + self.fill_build_combo_box_for_date(self.settings['build_date']) + self.update_build() + + def event_combo_box_build(self, evt): + self.settings['build_number'] = evt.GetString().encode('ascii').lower() + self.update_build() + #--- END event(s) --- + + +class SelectOS(wiz.WizardPageSimple): + """Ask the user which OS they would like to run. Filter out any choices + that are unavailable due to previous choices.""" + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.wizard = parent + self.db = db + self.width = width + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.settings['image'] = None + + header = wx.StaticText(self, label = "Please select the operating " + "system you would like to run on " + "your hardware.") + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + self.box1 = wx.BoxSizer(wx.VERTICAL) + self.sizer.Add(header) + + self.cb_image = wx.ComboBox(self, + style = wx.CB_DROPDOWN | wx.CB_READONLY) + self.Bind(wx.EVT_COMBOBOX, self.event_combo_box_os, self.cb_image) + + #--- Layout --- + # -- Combo boxes for hardware and image selection -- + self.box1.Add(self.cb_image, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.sizer.Add(self.box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.help_text = wx.StaticText(self) + self.sizer.Add(self.help_text, 1, wx.EXPAND, 5) + + self.SetSizer(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def get_human_os_name(self, item): + """Given an OS name from the database, return a human name (either + translated from the YAML settings, or just prettified) and if it is a + LEB OS or not""" + + item = re.sub("linaro-", "", item) # Remove any linaro- decoration + + human_name = item + + if item in self.settings['UI']['descriptions']: + human_name = self.settings['UI']['descriptions'][item] + else: + # Make human_name look nicer... + human_name = string.capwords(item) + + leb_search = re.search("^LEB:\s*(.*)$", human_name) + + if leb_search: + return leb_search.group(1), True + + return human_name, False + + def fill_os_list(self): + """Filter the list of OS's from the config file based on the users + preferences so all choices in the list are valid (i.e. their hardware + is supported for the build they have chosen).""" + + # select unique image from snapshot_binaries/release_binaries to + # generate list + os_list = None + if self.settings['release_or_snapshot'] == "release": + os_list = self.db.get_os_list_from('release_binaries') + else: + os_list = self.db.get_os_list_from('snapshot_binaries') + + self.cb_image.Clear() + + printed_tag = None + last_name = None + current_image_setting_valid = False + + for state in ["LEB", "other"]: + for item in os_list: + if item == "old": + # Old is a directory that sometimes hangs around, + # but isn't one we want to display + continue + + # Save the original, untouched image name for use later. + # We give it a more human name for display + original = item + item = re.sub("linaro-", "", item) + + os_hardware_combo_available = ( + self.db.image_hardware_combo_available( + self.settings['release_or_snapshot'], + original, + self.settings['compatable_hwpacks'])) + + if os_hardware_combo_available: + human_name, is_LEB = self.get_human_os_name(item) + + if item == self.settings['image']: + current_image_setting_valid = True + + if state == "LEB" and is_LEB: + + if printed_tag != state: + self.cb_image.Append( + "- Linaro Supported Releases -") + printed_tag = state + + self.cb_image.Append(human_name, original) + + if self.settings['image'] == None: + self.settings['image'] = original + + elif state != "LEB" and not is_LEB: + if printed_tag != state: + self.cb_image.Append( + "- Community Supported Releases -") + printed_tag = state + + self.cb_image.Append(human_name, original) + + last_name = original + + if( self.settings['image'] != None + and current_image_setting_valid == False): + # If we have an image setting, but it doesn't match the OS list, we + # have switched OS list. It may be that adding/removing "linaro-" + # from the name will get a match. + + if re.search("linaro-", self.settings['image']): + test_name = re.sub("linaro-", "", self.settings['image']) + else: + test_name = "linaro-" + self.settings['image'] + + if test_name in os_list: + # Success! We have translated the name and can retain the + # "old setting" + self.settings['image'] = test_name + current_image_setting_valid = True + + if( self.settings['image'] == None + or current_image_setting_valid == False): + # This should only get hit if there are no LEBs available + self.settings['image'] = last_name + + assert self.settings['image'] + + # Make sure the visible selected value matches the saved setting + self.cb_image.SetValue( + self.get_human_os_name(self.settings['image'])[0]) + + #--- Event(s) --- + def event_combo_box_os(self, evt): + self.settings['image'] = self.cb_image.GetClientData( + evt.GetSelection()) + + if self.settings['image']: # Is None for items that aren't an OS + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + image = re.sub("linaro-", "", self.settings['image']) + + if image + "::long" in self.settings['UI']['descriptions']: + self.help_text.SetLabel(self.settings['UI'] + ['descriptions'] + [image + "::long"]) + else: + self.help_text.SetLabel("") + + else: # Have selected help text + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + self.help_text.SetLabel("Please select an operating system to run " + "on your chosen hardware.") + + self.help_text.Wrap(self.width - 10) + #--- END event(s) --- + + +class LMC_settings(wiz.WizardPageSimple): + """Present the user with, intially, the choice of writing the file system + they are going to have created to a file, or directly to a device. Ask + which file/device to write to. + + If writing to a device, the user is asked to tick a box saying that they + understand that the device they have chosen will be erased. + + If the user ticks the advanced box, more options are shown.""" + + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.wizard = parent + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.yes_use_mmc = False + self.db = db + + self.settings['path_selected'] = "" + + header = wx.StaticText(self, + label = "Media Creation Settings\n\n" + "Please select if you would like to write the " + "file system I am about to create to a memory " + "card, or to a file on the local file system.") + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + #--- Build some widgets --- + #-- Target file system -- + file_systems = ["ext3", "ext4", "btrfs", "ext2"] + default_target = file_systems[0] + self.settings['rootfs'] = default_target + cb_rootfs = wx.ComboBox(self, + value = default_target, + style = wx.CB_DROPDOWN | wx.CB_READONLY) + + for item in file_systems: + cb_rootfs.Append(item, item.upper()) + + self.Bind(wx.EVT_COMBOBOX, self.event_combo_box_rootfs, cb_rootfs) + + #-- Image size spinner + self.image_size_spinner = wx.SpinCtrl(self, -1, "") + self.Bind(wx.EVT_SPINCTRL, + self.event_image_size, + self.image_size_spinner) + + #-- Swap size spinner + self.swap_size_spinner = wx.SpinCtrl(self, -1, "") + self.Bind(wx.EVT_SPINCTRL, + self.event_swap_size, + self.swap_size_spinner) + + #--- Layout --- + self.sizer.Add(header, 0, wx.ALIGN_LEFT | wx.ALL, 5) + box1 = wx.BoxSizer(wx.VERTICAL) + file_dev_grid = wx.FlexGridSizer(0, 2, 0, 0) + box1.Add(file_dev_grid, 0, wx.EXPAND) + grid1 = wx.FlexGridSizer(0, 2, 0, 0) + + # self.settings['write_to_file_or_device'] should match the first + # button below... + self.settings['write_to_file_or_device'] = "file" + add_button(self, + file_dev_grid, + "Write to file", + wx.RB_GROUP, + self.event_radio_button_select, + None, None) + + add_button(self, + file_dev_grid, + "Write to device", + None, + self.event_radio_button_select, + None, None) + + self.help_text_values = {"device": "Please select a device to write " + "the file system to:", + "file": "Please select a file to write the " + "file system to:"} + + self.help_text = wx.StaticText( + self, + label = + self.help_text_values[ + self.settings['write_to_file_or_device']]) + self.help_text.Wrap(width - 10) + + #-- File/dev picker -- + file_browse_button = wx.Button(self, -1, "Browse") + file_browse_grid = wx.FlexGridSizer(0, 2, 0, 0) + self.file_path_and_name = wx.TextCtrl(self, -1, "", size=(300, -1)) + + file_browse_grid.Add(self.file_path_and_name, 0, wx.EXPAND) + file_browse_grid.Add(file_browse_button, 0, wx.EXPAND) + + self.Bind(wx.EVT_BUTTON, + self.event_open_file_control, + file_browse_button) + + self.Bind(wx.EVT_TEXT, + self.event_file_path_and_name, + self.file_path_and_name) + + box1.Add(self.help_text, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + box1.Add(file_browse_grid, 0, wx.EXPAND) + + cb1 = wx.CheckBox(self, -1, "Show advanced options") + self.Bind(wx.EVT_CHECKBOX, self.event_show_advanced_options, cb1) + box1.Add(cb1) + + #-- Combo boxes for hardware and image selection -- + optional_settings_box_title = wx.StaticBox( + self, + label = " Optional Settings ") + + self.optional_settings_box = wx.StaticBoxSizer( + optional_settings_box_title, + wx.VERTICAL) + + self.box2 = wx.BoxSizer(wx.VERTICAL) + + self.box2.AddWindow(self.optional_settings_box, + 0, + border=2, + flag=wx.ALL | wx.EXPAND) + + grid1.Add(cb_rootfs, 0, wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, 5) + + grid1.Add(wx.StaticText(self, + label = "The root file system of the image"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + # We want to sub-devide the cell, to add another grid sizer... + file_size_grid = wx.FlexGridSizer(0, 2, 0, 0) + + grid1.Add(file_size_grid, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP) + + # Add a spinner that allows us to type/click a numerical value (defined above) + file_size_grid.Add(self.image_size_spinner, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + # Add a choice of MB or GB for size input + units = ["GB", "MB"] + self.size_unit = units[0] # Set the default unit + unit_choice = wx.Choice(self, -1, (100, 50), choices = units) + self.Bind(wx.EVT_CHOICE, self.event_chose_unit, unit_choice) + file_size_grid.Add(unit_choice, 0, wx.ALIGN_RIGHT | wx.TOP, 5) + + # Back out of the extra grid, add some help text + grid1.Add(wx.StaticText( + self, + label = "Writing to file only: Image file size"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + # The swap size (MB only) + grid1.Add(self.swap_size_spinner, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + grid1.Add(wx.StaticText(self, label = "Swap file size in MB"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.cb_hwpacks = wx.ComboBox( + self, + value = self.settings['compatable_hwpacks'][0], + style = wx.CB_DROPDOWN | wx.CB_READONLY) + + self.Bind(wx.EVT_COMBOBOX, + self.event_combo_box_hwpack, + self.cb_hwpacks) + + grid1.Add(self.cb_hwpacks, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + grid1.Add(wx.StaticText(self, label = "Compatible hardware packs"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.optional_settings_box.Add(grid1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + confirm_mmc_usage_title = wx.StaticBox(self, label = " Are you sure? ") + + self.confirm_mmc_usage_box = wx.StaticBoxSizer(confirm_mmc_usage_title, + wx.VERTICAL) + cb2 = wx.CheckBox( + self, + -1, + "Yes, erase and use the device I have selected above.") + + self.Bind(wx.EVT_CHECKBOX, self.event_use_mmc_tickbox, cb2) + self.confirm_mmc_usage_box.Add(cb2) + + self.box3 = wx.BoxSizer(wx.VERTICAL) + self.box3.AddWindow(self.confirm_mmc_usage_box, + 0, + border=2, + flag=wx.ALL | wx.EXPAND) + + self.sizer.Add(box1, 0, wx.ALIGN_LEFT | wx.ALL, 0) + self.sizer.Add(self.box2, 0, wx.ALIGN_LEFT | wx.ALL, 0) + self.sizer.Add(self.box3, 0, wx.ALIGN_LEFT | wx.ALL, 0) + self.SetSizer(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def on_activate(self): + self.update_forward_active_and_mmc_confirm_box_visible() + self.set_hwpacks_for_hardware() + + def set_hwpacks_for_hardware(self): + self.cb_hwpacks.Clear() + + if self.settings['release_or_snapshot'] == "snapshot": + self.settings['build'] = self.settings['snapshot_build'] + + date_and_build = self.settings['build'].split(":") + + compatable_hwpacks = ( + self.db.get_available_hwpacks_for_hardware_snapshot_build( + self.settings['compatable_hwpacks'], + self.settings['platform'], + date_and_build[0], + date_and_build[1])) + else: + self.settings['build'] = self.settings['release_build'] + compatable_hwpacks = ( + self.db.get_available_hwpacks_for_hardware_build_plaform( + self.settings['compatable_hwpacks'], + self.settings['platform'], + self.settings['build'])) + + for hwpack in compatable_hwpacks: + self.cb_hwpacks.Append(hwpack) + + self.cb_hwpacks.SetStringSelection(compatable_hwpacks[0]) + self.settings['hwpack'] = compatable_hwpacks[0] + + def update_forward_active_and_mmc_confirm_box_visible(self): + if( self.settings['path_selected'] + and self.settings['path_selected'] != ""): + + if ( self.settings['write_to_file_or_device'] == "file" + or self.settings['write_to_file_or_device'] == "device" + and self.yes_use_mmc): + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + else: + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + else: + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + + if self.settings['write_to_file_or_device'] == "device": + self.box3.Show(self.confirm_mmc_usage_box, True) + else: + self.box3.Hide(self.confirm_mmc_usage_box, True) + + # --- Event Handlers --- + def event_open_file_control(self, event): + if self.settings['write_to_file_or_device'] == "file": + + dlg = wx.FileDialog(self, + message="Save file as ...", + defaultDir=os.getcwd(), + defaultFile="", + style=wx.SAVE) + + elif self.settings['write_to_file_or_device'] == "device": + dlg = wx.FileDialog(self, + message="Choose a device", + defaultDir=os.getcwd(), + defaultFile="", + style=wx.OPEN | wx.CHANGE_DIR) + + if dlg.ShowModal() == wx.ID_OK: + self.settings['path_selected'] = dlg.GetPaths()[0] + self.file_path_and_name.SetValue(self.settings['path_selected']) + + dlg.Destroy() + self.update_forward_active_and_mmc_confirm_box_visible() + + def event_file_path_and_name(self, event): + self.settings['path_selected'] = event.GetString() + self.update_forward_active_and_mmc_confirm_box_visible() + + def event_combo_box_hwpack(self, event): + self.settings['hwpack'] = event.GetString().encode('ascii') + + def event_combo_box_rootfs(self, evt): + self.settings['rootfs'] = evt.GetString().encode('ascii').lower() + + def event_radio_button_select(self, event): + """Search the label of the button that has been selected to work out + what we are writing to.""" + setting_search = re.search( + "write to (\w+)", + event + .GetEventObject() + .GetLabel() + .encode('ascii') + .lower()) + + assert setting_search + + self.settings['write_to_file_or_device'] = setting_search.group(1) + + self.help_text.SetLabel( + self.help_text_values[self.settings['write_to_file_or_device']]) + + self.update_forward_active_and_mmc_confirm_box_visible() + + def event_show_advanced_options(self, event): + if event.IsChecked(): + self.box2.Show(self.optional_settings_box, True) + else: + self.box2.Hide(self.optional_settings_box, True) + + def event_pick_file_path(self, evt): + self.settings['path_selected'] = os.path.abspath(evt.GetPath()) + self.update_forward_active_and_mmc_confirm_box_visible() + + def update_image_size_setting(self): + if(self.image_size_spinner.GetValue() > 0): + self.settings['image_size'] = (str(self.image_size_spinner + .GetValue()) + + self.size_unit[0]) + else: + self.settings['image_size'] = None + + def event_image_size(self, event): + self.update_image_size_setting() + + def event_chose_unit(self, event): + self.size_unit = event.GetString() + self.update_image_size_setting() + + def event_swap_size(self, event): + self.settings['swap_file'] = str(self.image_size_spinner.GetValue()) + + def event_use_mmc_tickbox(self, event): + self.yes_use_mmc = event.IsChecked() + self.update_forward_active_and_mmc_confirm_box_visible() + + +class RunLMC(wiz.WizardPageSimple): + """Present the user with some information about their choices and a button + to start linaro-media-create. The linaro-media-create process is started in + a new thread and important events are communicated back to the UI through a + queue.""" + + def __init__(self, parent, config, db, width): + wiz.WizardPageSimple.__init__(self, parent) + self.settings = config.settings + self.sizer = wx.BoxSizer(wx.VERTICAL) + self.db = db + self.width = width + self.wizard = parent + + header = wx.StaticText(self, label = """Installing...""") + header.Wrap(width - 10) # -10 because boarder below is 5 pixels wide + + self.sizer.Add(header) + self.box1 = wx.BoxSizer(wx.VERTICAL) + + # We expect to print 4 lines of information, reserve space using blank + # lines. + self.settings_summary_text = wx.StaticText(self, label = "\n\n\n\n") + self.settings_summary_text.Wrap(width - 10) + + self.box1.Add(self.settings_summary_text, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.start_button = wx.Button(self, 10, "Start", (20, 20)) + self.Bind(wx.EVT_BUTTON, self.start_lmc, self.start_button) + + self.start_button.SetToolTipString("Start creating an image, using the" + "above settings.") + + self.start_button.SetSize(self.start_button.GetBestSize()) + self.box1.Add(self.start_button, 0, wx.ALIGN_LEFT | wx.ALL, 5) + + self.status_grid = wx.FlexGridSizer(0, 2, 0, 0) + + self.status_grid.Add(wx.StaticText(self, label="Downloading files"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.downloading_files_status = wx.StaticText(self, label="") + self.status_grid.Add(self.downloading_files_status, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.status_grid.Add(wx.StaticText(self, label="Unpacking downloads"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.unpacking_files_status = wx.StaticText(self, label="") + + self.status_grid.Add(self.unpacking_files_status, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.status_grid.Add(wx.StaticText(self, label="Installing packages"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.installing_packages_status = wx.StaticText(self, label="") + + self.status_grid.Add(self.installing_packages_status, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.status_grid.Add(wx.StaticText(self, label="Create file system"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.create_file_system_status = wx.StaticText(self, label="") + + self.status_grid.Add(self.create_file_system_status, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.status_grid.Add(wx.StaticText(self, label="Populate file system"), + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.populate_file_system_status = wx.StaticText(self, label="") + + self.status_grid.Add(self.populate_file_system_status, + 0, + wx.ALIGN_LEFT | wx.LEFT | wx.RIGHT | wx.TOP, + 5) + + self.sizer.Add(self.box1, 0, wx.ALIGN_LEFT | wx.ALL, 5) + self.sizer.Add(self.status_grid, 0, wx.ALIGN_LEFT | wx.ALL, 5) + self.SetSizerAndFit(self.sizer) + self.sizer.Fit(self) + self.Move((50, 50)) + + def on_activate(self): + """Called just before the page is displayed to update the text based on + the users preferences.""" + + # The build is stored in different forms depending on if we are using a + # release or snapshot but from here on in it is a common value + if self.settings['release_or_snapshot'] == "snapshot": + self.settings['build'] = self.settings['snapshot_build'] + else: + self.settings['build'] = self.settings['release_build'] + + settings_summary = ("Press start to create an image with the " + "following settings:\n") + settings_summary += "Operating System: " + self.settings['image'] + "\n" + settings_summary += "Hardware: " + self.settings['hardware'] + "\n" + + # Assumption is that a file may be in a long path, we don't know how + # big the font is and we don't want to allow the path to run off the + # end of the line, so if a file is chosen, just show the file name. + # Devices are (probably) /dev/some_short_name and the user really needs + # to check them, so we show the whole thing. + path = self.settings['path_selected'] + if self.settings['write_to_file_or_device'] == "file": + path = self.settings['path_selected'].split(os.sep)[-1] + + settings_summary += ( "Writing image to " + + self.settings['write_to_file_or_device'] + + " " + + path) + + self.settings_summary_text.SetLabel(settings_summary) + self.settings_summary_text.Wrap(self.width - 10) + + def start_lmc(self, event): + """Start a thread that runs linaro-media-create and a timer, which + checks for UI updates every 100ms""" + + if self.settings['write_to_file_or_device'] == "file": + self.settings['image_file'] = self.settings['path_selected'] + elif self.settings['write_to_file_or_device'] == "device": + self.settings['mmc'] = self.settings['path_selected'] + else: + assert False, ("self.config.settings['write_to_file_or_device'] " + "was an unexpected value" + + self.settings['write_to_file_or_device']) + + image_url, hwpack_url = self.db.get_image_and_hwpack_urls(self.settings) + + # Currently the UI is blocked when LMC is running, so grey out the + # buttons to indicate to the user that they won't work! + self.wizard.FindWindowById(wx.ID_BACKWARD).Disable() + self.wizard.FindWindowById(wx.ID_CANCEL).Disable() + + if(image_url and hwpack_url): + + print image_url + print hwpack_url + + self.file_handler = FetchImage.FileHandler() + + tools_dir = os.path.dirname(__file__) + if tools_dir == '': + tools_dir = None + + self.file_handler.create_media(image_url, + hwpack_url, + self.settings, + tools_dir, + True, + self) + + self.timer = wx.Timer(self) + self.Bind(wx.EVT_TIMER, self.timer_ping, self.timer) + self.timer.Start(milliseconds=100, oneShot=True) + + self.start_button.Disable() + self.event_queue = Queue.Queue() + self.file_handler.start_lmc_gui_thread(self.event_queue) + else: + print >> sys.stderr, ("Unable to find files that match the" + "parameters specified") + + def timer_ping(self, event): + """During start_lmc a timer is started to poll for events from + linaro-media-create every 100ms. This is the function which is called + to do that polling.""" + + if self.event_queue.empty() == False: + event = self.event_queue.get() + + if event[0] == "start": + self.event_start(event[1]) + self.timer.Start(milliseconds=100, oneShot=True) + + elif event[0] == "end": + self.event_end(event[1]) + self.timer.Start(milliseconds=100, oneShot=True) + + elif event == "terminate": + # Process complete. Enable next button. + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + self.populate_file_system_status.SetLabel("Done") + + else: + print >> sys.stderr, "timer_ping: Unhandled event", event + + else: + self.timer.Start(milliseconds=100, oneShot=True) + + def unsigned_packages_query(self, package_list): + message = ('In order to continue, I need to install some unsigned' + 'packages into the image. Is this OK? The packages are:' + '\n\n' + package_list) + + dlg = wx.MessageDialog(self, + message, + 'Install Unsigned Packages Into Image?', + wx.YES_NO | wx.NO_DEFAULT) + + choice = dlg.ShowModal() + dlg.Destroy() + + return choice == wx.ID_YES + + #--- Event(s) --- + def event_start(self, event): + if event == "download OS": + self.downloading_files_status.SetLabel("Downloading OS") + elif event == "download hwpack": + self.downloading_files_status.SetLabel("Downloading Hardware Pack") + elif event == "unpack": + self.unpacking_files_status.SetLabel("Running") + elif event == "installing packages": + self.installing_packages_status.SetLabel("Running") + + elif re.search('^unverified_packages:', event): + # Get rid of event ID and whitespace invariance + packages = " ".join(event.split()[1:]) + install_unsigned_packages = self.unsigned_packages_query(packages) + + if install_unsigned_packages == False: + self.file_handler.kill_create_media() + sys.exit(1) + else: + self.file_handler.send_to_create_process("y") + + elif event == "create file system": + self.create_file_system_status.SetLabel("Running") + elif event == "populate file system": + self.populate_file_system_status.SetLabel("Running") + else: + print "Unhandled start event:", event + + def event_end(self, event): + if event == "download OS": + self.downloading_files_status.SetLabel("Done (1/2)") + elif event == "download hwpack": + self.downloading_files_status.SetLabel("Done") + elif event == "unpack": + self.unpacking_files_status.SetLabel("Done") + elif event == "installing packages": + self.installing_packages_status.SetLabel("Done") + elif event == "create file system": + self.create_file_system_status.SetLabel("Done") + elif event == "populate file system": + self.populate_file_system_status.SetLabel("Done") + else: + print "Unhhandled end event:", event + + def event_combo_box_release(self, evt): + pass + + def event_combo_box_build(self, evt): + pass + #--- END event(s) --- + + +class TestDriveWizard(wx.wizard.Wizard): + def __init__(self, title): + wx.wizard.Wizard.__init__(self, None, -1, title, wx.NullBitmap) + self.Bind(wx.wizard.EVT_WIZARD_PAGE_CHANGING, self.on_page_changing) + self.done_startup = False + + def on_page_changing(self, evt): + 'Executed before the page changes.' + + if self.done_startup == False: + self.pages['lmc_settings'].box2.Hide( + self.pages['lmc_settings'].optional_settings_box, + True) + + self.pages['lmc_settings'].box3.Hide( + self.pages['lmc_settings'].confirm_mmc_usage_box, + True) + + self.done_startup = True + + page = evt.GetPage() + + if evt.GetDirection(): # If going forwards... + # Always enable back button if going forwards + self.wizard.FindWindowById(wx.ID_BACKWARD).Enable() + + # If going from a select snapshot or select release page, record + # which we were on so the back button of the next page works + if(self.config.settings['release_or_snapshot'] == "release"): + self.pages['select_os'].SetNext(self.pages['select_release']) + self.pages['select_release'].SetPrev(self.pages['select_os']) + + self.pages['select_release'].SetNext(self.pages['lmc_settings']) + self.pages['lmc_settings'].SetPrev(self.pages['select_release']) + else: + self.pages['select_os'].SetNext(self.pages['select_snapshot']) + self.pages['select_snapshot'].SetPrev(self.pages['select_os']) + + if(page == self.pages['select_os']): + self.pages['select_snapshot'].fill_build_combo_box_for_date( + self.config.settings['build_date']) + + self.pages['select_snapshot'].SetNext(self.pages['lmc_settings']) + self.pages['lmc_settings'].SetPrev(self.pages['select_snapshot']) + + if page == self.pages['hardware_details']: + self.pages['select_os'].fill_os_list() + + if page == self.pages['release_or_snapshot']: + self.pages['hardware_details'].on_page_changing() + + # If about to move into the release selection, make sure the list + # is populated only with releases that are valid with our current + # selection + if( page == self.pages['select_os'] + and self.config.settings['release_or_snapshot'] == "release"): + self.pages['select_release'].update_release_and_build_boxes() + + if page == self.pages['select_snapshot']: + # Execute when exiting page + self.pages['select_snapshot'].update_platform() + + if( page == self.pages['select_snapshot'] + or page == self.pages['select_release']): + self.pages['lmc_settings'].on_activate() + + if page == self.pages['lmc_settings']: + # Forward stays disabled until LMC has finished running + self.wizard.FindWindowById(wx.ID_FORWARD).Disable() + self.pages['run_lmc'].on_activate() + + else: # Always enable the forward button if reversing into a page + self.wizard.FindWindowById(wx.ID_FORWARD).Enable() + + def go(self, first_page): + file_handler = FetchImage.FileHandler() + self.config = FetchImage.FetchImageConfig() + self.config.settings["force_download"] = False + self.config.settings['compatable_hwpacks'] = ['foo'] + + # If the settings file and server index need updating, grab them + file_handler.update_files_from_server(show_wx_progress = True) + + # Load settings YAML, which defines the parameters we ask for and + # acceptable responses from the user + self.config.read_config(file_handler.settings_file) + + # Using the config we have, look up URLs to download data from in + # the server index + db = FetchImage.DB(file_handler.index_file) + + # Create the wizard and the pages + self.wizard = wiz.Wizard(self, -1, "Linaro Media Builder") + + self.pages = {} + self.pages['release_or_snapshot'] = ReleaseOrSnapshotPage(self.wizard, + self.config) + self.wizard.FitToPage(self.pages['release_or_snapshot']) + (width, height) = self.wizard.GetSize() + + self.pages['hardware_details'] = AboutMyHardwarePage(self.wizard, + self.config, + db, + width) + + self.pages['select_release'] = SelectStableRelease(self.wizard, + self.config, + db, + width) + + self.pages['select_snapshot'] = SelectSnapshot(self.wizard, + self.config, + db, + width) + + self.pages['select_os'] = SelectOS(self.wizard, + self.config, + db, + width) + + self.pages['lmc_settings'] = LMC_settings(self.wizard, + self.config, + db, + width) + + self.pages['run_lmc'] = RunLMC(self.wizard, + self.config, + db, + width) + + self.pages['release_or_snapshot'].SetNext( + self.pages['hardware_details']) + + self.pages['hardware_details'].SetPrev( + self.pages['release_or_snapshot']) + + self.pages['hardware_details'].SetNext(self.pages['select_os']) + self.pages['select_os'].SetPrev(self.pages['hardware_details']) + # Select OS goes to select build, which is customised for + # releases or snapshots + self.pages['lmc_settings'].SetNext(self.pages['run_lmc']) + self.pages['run_lmc'].SetPrev(self.pages['lmc_settings']) + + for (name, page) in self.pages.items(): + self.wizard.GetPageAreaSizer().Add(page) + + self.wizard.RunWizard(self.pages['release_or_snapshot']) + + +def run(start_page = None): + """Wrapper around the full wizard. Is encapsulated in its own function to + allow a restart to be performed, as described in __main___, easily""" + app = wx.PySimpleApp() # Start the application + #logging.basicConfig(level=logging.INFO) + w = TestDriveWizard('Simple Wizard') + return w.go(start_page) + + +class TestURLLookupFunctions(unittest.TestCase): + + def setUp(self): + self.file_handler = FetchImage.FileHandler() + self.file_handler.update_files_from_server(show_wx_progress = True) + self.config = FetchImage.FetchImageConfig() + self.config.settings["force_download"] = False + + # Load settings YAML, which defines the parameters we ask for and + # acceptable responses from the user + self.config.read_config(self.file_handler.settings_file) + + # Using the config we have, look up URLs to download data from in the + # server index + self.db = FetchImage.DB(self.file_handler.index_file) + + def test_000_url_lookup_snapshot_builds(self): + self.settings = self.config.settings + self.settings['release_or_snapshot'] = "snapshot" + + #--- Test finding builds near a particular day --- + # This functionality is required for further tests, hence forcing the + # run order by putting numbers in the function name. + today = wx.DateTime() + today.SetToCurrent() + + # -- Don't iterate through platforms for snapshot -- + + # -- Select hardware -- + for self.settings['hardware'] in ( + self.settings['choice']['hardware'].keys()): + + compatable_hwpacks = self.settings['choice']['hwpack'][ + self.settings['hardware']] + + future_date, past_date = self.db.get_next_prev_day_with_builds( + "linaro-alip", + today.FormatISODate().encode('ascii'), + compatable_hwpacks) + + if past_date == None: + # Some hardware packs are not available in the snapshot repo, + # so just skip if they aren't + continue + + builds = self.db.get_binary_builds_on_day_from_db( + "linaro-alip", + past_date, + compatable_hwpacks) + + self.assertTrue(len(builds)) + # If the above assert fails, either the DB is empty, or + # db.get_binary_builds_on_day_from_db failed + + def test_100_url_lookup_snapshots(self): + self.settings = self.config.settings + self.settings['release_or_snapshot'] = "snapshot" + + #--- Test snapshot build lookup --- + # -- Fix a build date -- + # We only need to look up a single snapshot date. Start with today and + # go with the day in the DB, build 0 + today = wx.DateTime() + today.SetToCurrent() + + # -- Don't iterate through platforms for snapshot -- + + # -- Select hardware -- + for self.settings['hardware'] in ( + self.settings['choice']['hardware'].keys()): + + compatable_hwpacks = self.settings['choice']['hwpack'][ + self.settings['hardware']] + + future_date, past_date = self.db.get_next_prev_day_with_builds( + "linaro-alip", + today.FormatISODate().encode('ascii'), + compatable_hwpacks) + + if past_date == None: + # Some hardware packs are not available in the snapshot repo, + # so just skip if they aren't + continue + + builds = self.db.get_binary_builds_on_day_from_db( + "linaro-alip", + past_date, + compatable_hwpacks) + + self.assertTrue(len(builds)) + # The above code is tested in test_000_url_lookup_snapshot_build. + # If the above assert fails, either the DB is empty, or + # db.get_binary_builds_on_day_from_db failed. + + small_date = re.sub('-', '', past_date) + self.settings['build'] = small_date + ":" + "0" + + # -- Iterate through hardware packs -- + for self.settings['hwpack'] in compatable_hwpacks: + + # If hardware pack is available... + if(self.settings['hwpack'] + in self.db.get_hwpacks('snapshot_hwpacks')): + + # -- Iterate through images + os_list = self.db.get_os_list_from('snapshot_binaries') + + for self.settings['image'] in os_list: + if re.search('old', self.settings['image']): + # Directories with old in the name are of no + # interest to us + continue + + # -- Check build which matches these parameters + # (builds that don't match are excluded in UI) -- + if( len(self.db.execute_return_list( + 'select * from snapshot_hwpacks ' + 'where hardware == ? ' + 'and date == ? ' + 'and build == ?', + (self.settings['hwpack'], + small_date, + "0"))) + and len(self.db.execute_return_list( + 'select * from snapshot_binaries ' + 'where image == ? ' + 'and date == ? ' + 'and build == ?', + (self.settings['image'], + small_date, + "0")))): + + # - Run the function under test! - + image_url, hwpack_url = ( + self.db.get_image_and_hwpack_urls(self.settings)) + + self.assertTrue(image_url) + self.assertTrue(hwpack_url) + + def test_101_url_lookup_releases(self): + #--- Test release build lookup --- + self.settings = self.config.settings + self.settings['release_or_snapshot'] = "release" + # -- Select hardware -- + for self.settings['hardware'] in ( + self.settings['choice']['hardware'].keys()): + compatable_hwpacks = ( + self.settings['choice']['hwpack'][self.settings['hardware']]) + + # -- Iterate through hardware packs -- + for self.settings['hwpack'] in compatable_hwpacks: + + # If hardware pack is available... + if(self.settings['hwpack'] + in self.db.get_hwpacks('release_hwpacks')): + + # -- Iterate through images + os_list = self.db.get_os_list_from('release_binaries') + + for self.settings['image'] in os_list: + if re.search('old', self.settings['image']): + # Directories with old in the name are of no + # interest to us + continue + + for platform, ignore in ( + self.settings['choice']['platform'].items()): + self.settings['platform'] = platform + + # -- Iterate through available builds -- + builds = self.db.get_builds( + self.settings['platform'], + self.settings['image']) + + for build in builds: + self.settings['build'] = build + + # -- Check build which matches these parameters + #(builds that don't match are excluded in UI)-- + if( len(self.db.execute_return_list( + 'select * from release_hwpacks ' + 'where platform == ? ' + 'and hardware == ? ' + 'and build == ?', + (self.settings['platform'], + self.settings['hwpack'], + self.settings['build']))) + and len(self.db.execute_return_list( + 'select * from release_binaries ' + 'where platform == ? ' + 'and image == ? ' + 'and build == ?', + (self.settings['platform'], + self.settings['image'], + self.settings['build'])))): + + # - Run the function under test! - + image_url, hwpack_url = ( + self.db.get_image_and_hwpack_urls(self.settings)) + self.assertTrue(image_url) + self.assertTrue(hwpack_url) + +if __name__ == '__main__': + run() === modified file 'linaro-android-media-create' --- linaro-android-media-create 2011-05-26 09:16:35 +0000 +++ linaro-android-media-create 2011-06-30 12:09:46 +0000 @@ -133,12 +133,13 @@ unpack_android_binary_tarball(args.system, SYSTEM_DIR) unpack_android_binary_tarball(args.userdata, DATA_DIR) - # Create partitions + # Create partitions boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition = setup_android_partitions( \ board_config, media, args.image_size, args.boot_label, args.should_create_partitions, args.should_align_boot_part) + board_config.populate_raw_partition(args.device, BOOT_DIR) populate_partition(BOOT_DIR + "/boot", BOOT_DISK, boot_partition) board_config.populate_boot_script(boot_partition, BOOT_DISK, args.consoles) populate_partition(SYSTEM_DIR + "/system", SYSTEM_DISK, system_partition) === modified file 'linaro-hwpack-install' --- linaro-hwpack-install 2011-01-29 16:35:06 +0000 +++ linaro-hwpack-install 2011-06-23 12:52:43 +0000 @@ -36,7 +36,7 @@ FORCE_YES="no" SOURCES_LIST_FILE="${TEMP_DIR}/sources.list" APT_GET_OPTIONS="Dir::Etc::SourceList=${SOURCES_LIST_FILE}" -SUPPORTED_FORMATS="1.0" # A space-separated list of hwpack formats. +SUPPORTED_FORMATS="1.0 2.0" # A space-separated list of hwpack formats. sudo="sudo" if [ $(id -u) -eq 0 ]; then === modified file 'linaro-media-create' --- linaro-media-create 2011-06-17 12:47:16 +0000 +++ linaro-media-create 2011-06-28 13:35:12 +0000 @@ -22,7 +22,6 @@ import os import sys import tempfile -import subprocess from linaro_image_tools import cmd_runner @@ -43,7 +42,11 @@ unpack_binary_tarball, ) from linaro_image_tools.media_create import get_args_parser -from linaro_image_tools.utils import ensure_command, is_arm_host +from linaro_image_tools.utils import ( + ensure_command, + is_arm_host, + verify_file_integrity, + ) # Just define the global variables TMP_DIR = None @@ -101,19 +104,14 @@ ROOT_DISK = os.path.join(TMP_DIR, 'root-disc') board_config = board_configs[args.board] + board_config.set_metadata(args.hwpacks) ensure_required_commands(args) sig_file_list = args.hwpacksigs[:] - verified_files = [] if args.binarysig is not None: sig_file_list.append(args.binarysig) - for sig_file in sig_file_list: - hash_file = sig_file[0:-len('.asc')] - cmd_runner.run(['gpg', '--verify', sig_file]).wait() - sha1sums_out, _ = cmd_runner.run(['sha1sum', '-c', hash_file], - stdout=subprocess.PIPE).communicate() - verified_files.extend(sha1sums_out.replace(': OK', '').splitlines()) + verified_files = verify_file_integrity(sig_file_list) for verified_file in verified_files: print 'Hash verification of file %s OK.' % verified_file === modified file 'linaro_image_tools/FetchImage.py' --- linaro_image_tools/FetchImage.py 2011-06-17 18:04:47 +0000 +++ linaro_image_tools/FetchImage.py 2011-07-06 14:16:10 +0000 @@ -420,8 +420,8 @@ def update_files_from_server(self, force_download=False, show_wx_progress=False): - settings_url = "http://z.nanosheep.org/fetch_image_settings.yaml" - server_index_url = "http://z.nanosheep.org/server_index.bz2" + settings_url = "http://releases.linaro.org/fetch_image/fetch_image_settings.yaml" + server_index_url = "http://releases.linaro.org/fetch_image/server_index.bz2" self.settings_file = self.download_if_old(settings_url, force_download, === modified file 'linaro_image_tools/cmd_runner.py' --- linaro_image_tools/cmd_runner.py 2011-04-04 10:38:07 +0000 +++ linaro_image_tools/cmd_runner.py 2011-06-21 07:41:30 +0000 @@ -36,7 +36,7 @@ def run(args, as_root=False, chroot=None, stdin=None, stdout=None, - stderr=None): + stderr=None, cwd=None): """Run the given command as a sub process. Return a Popen instance. @@ -60,7 +60,7 @@ as_root = True if as_root and os.getuid() != 0: args = SUDO_ARGS + args - return Popen(args, stdin=stdin, stdout=stdout, stderr=stderr) + return Popen(args, stdin=stdin, stdout=stdout, stderr=stderr, cwd=cwd) class Popen(subprocess.Popen): === modified file 'linaro_image_tools/hwpack/builder.py' --- linaro_image_tools/hwpack/builder.py 2011-03-23 22:25:10 +0000 +++ linaro_image_tools/hwpack/builder.py 2011-06-20 13:33:01 +0000 @@ -21,6 +21,12 @@ import logging import errno +import subprocess +import tempfile +import os +import shutil + +from linaro_image_tools import cmd_runner from linaro_image_tools.hwpack.config import Config from linaro_image_tools.hwpack.hardwarepack import HardwarePack, Metadata @@ -45,6 +51,33 @@ "No such config file: '%s'" % self.filename) +class PackageUnpacker(object): + def __enter__(self): + self.tempdir = tempfile.mkdtemp() + return self + + def __exit__(self, type, value, traceback): + if self.tempdir is not None and os.path.exists(self.tempdir): + shutil.rmtree(self.tempdir) + + def unpack_package(self, package_file_name): + # We could extract only a single file, but since dpkg will pipe + # the entire package through tar anyway we might as well extract all. + p = cmd_runner.run(["tar", "-C", self.tempdir, "-xf", "-"], + stdin=subprocess.PIPE) + cmd_runner.run(["dpkg", "--fsys-tarfile", package_file_name], + stdout=p.stdin).communicate() + p.communicate() + + def get_file(self, package, file): + self.unpack_package(package) + logger.debug("Unpacked package %s." % package) + temp_file = os.path.join(self.tempdir, file) + assert os.path.exists(temp_file), "The file '%s' was " \ + "not found in the package '%s'." % (file, package) + return temp_file + + class HardwarePackBuilder(object): def __init__(self, config_path, version, local_debs): @@ -56,9 +89,27 @@ raise ConfigFileMissing(config_path) raise self.config.validate() + self.format = self.config.format self.version = version self.local_debs = local_debs + def find_fetched_package(self, packages, wanted_package_name): + wanted_package = None + for package in packages: + if package.name == wanted_package_name: + wanted_package = package + break + else: + raise AssertionError("Package '%s' was not fetched." % \ + wanted_package_name) + packages.remove(wanted_package) + return wanted_package + + def add_file_to_hwpack(self, package, wanted_file, package_unpacker, hwpack, target_path): + tempfile_name = package_unpacker.get_file( + package.filepath, wanted_file) + return hwpack.add_file(target_path, tempfile_name) + def build(self): for architecture in self.config.architectures: logger.info("Building for %s" % architecture) @@ -70,6 +121,8 @@ hwpack.add_apt_sources(sources) sources = sources.values() packages = self.config.packages[:] + if self.config.u_boot_package is not None: + packages.append(self.config.u_boot_package) local_packages = [ FetchedPackage.from_deb(deb) for deb in self.local_debs] @@ -81,10 +134,18 @@ fetcher = PackageFetcher( sources, architecture=architecture, prefer_label=LOCAL_ARCHIVE_LABEL) - with fetcher: + with fetcher, PackageUnpacker() as package_unpacker: fetcher.ignore_packages(self.config.assume_installed) packages = fetcher.fetch_packages( packages, download_content=self.config.include_debs) + + if self.config.u_boot_package is not None: + u_boot_package = self.find_fetched_package( + packages, self.config.u_boot_package) + hwpack.metadata.u_boot = self.add_file_to_hwpack( + u_boot_package, self.config.u_boot_file, + package_unpacker, hwpack, hwpack.U_BOOT_DIR) + logger.debug("Adding packages to hwpack") hwpack.add_packages(packages) for local_package in local_packages: === modified file 'linaro_image_tools/hwpack/config.py' --- linaro_image_tools/hwpack/config.py 2011-01-28 19:50:48 +0000 +++ linaro_image_tools/hwpack/config.py 2011-07-01 14:55:14 +0000 @@ -22,6 +22,10 @@ import ConfigParser import re +from linaro_image_tools.hwpack.hardwarepack_format import ( + HardwarePackFormatV1, + HardwarePackFormatV2, + ) class HwpackConfigError(Exception): pass @@ -38,10 +42,32 @@ SOURCES_ENTRY_KEY = "sources-entry" PACKAGES_KEY = "packages" PACKAGE_REGEX = NAME_REGEX + PATH_REGEX = r"[a-z0-9][a-z0-9+\-./_]+$" ORIGIN_KEY = "origin" MAINTAINER_KEY = "maintainer" ARCHITECTURES_KEY = "architectures" ASSUME_INSTALLED_KEY = "assume-installed" + U_BOOT_PACKAGE_KEY = "u-boot-package" + U_BOOT_FILE_KEY = "u-boot-file" + SERIAL_TTY_KEY = "serial_tty" + KERNEL_ADDR_KEY = "kernel_addr" + INITRD_ADDR_KEY = "initrd_addr" + LOAD_ADDR_KEY = "load_addr" + WIRED_INTERFACES_KEY = "wired_interfaces" + WIRELESS_INTERFACES_KEY = "wireless_interfaces" + PARTITION_LAYOUT_KEY = "partition_layout" + MMC_ID_KEY = "mmc_id" + FORMAT_KEY = "format" + BOOT_MIN_SIZE_KEY = "boot_min_size" + ROOT_MIN_SIZE_KEY = "root_min_size" + LOADER_MIN_SIZE_KEY = "loader_min_size" + + DEFINED_PARTITION_LAYOUTS = [ + 'bootfs16_rootfs', + 'bootfs_rootfs', + #'reserved_bootfs_rootfs', + ] + def __init__(self, fp): """Create a Config. @@ -58,15 +84,52 @@ """ if not self.parser.has_section(self.MAIN_SECTION): raise HwpackConfigError("No [%s] section" % self.MAIN_SECTION) + self._validate_format() self._validate_name() self._validate_include_debs() self._validate_support() self._validate_packages() self._validate_architectures() self._validate_assume_installed() + + if self.format.has_v2_fields: + self._validate_u_boot_package() + self._validate_u_boot_file() + self._validate_serial_tty() + self._validate_kernel_addr() + self._validate_initrd_addr() + self._validate_load_addr() + self._validate_wired_interfaces() + self._validate_wireless_interfaces() + self._validate_partition_layout() + self._validate_mmc_id() + self._validate_boot_min_size() + self._validate_root_min_size() + self._validate_loader_min_size() + self._validate_sections() @property + def format(self): + """The format of the hardware pack. A subclass of HardwarePackFormat. + """ + try: + format_string = self.parser.get(self.MAIN_SECTION, self.FORMAT_KEY) + except ConfigParser.NoOptionError: + # Default to 1.0 to aviod breaking existing hwpack files. + # When this code no longer supports 1.0, it effectively makes + # explicitly specifying format in hwpack files mandatory. + format_string = "1.0" + + if format_string == '1.0': + return HardwarePackFormatV1() + elif format_string == '2.0': + return HardwarePackFormatV2() + else: + raise HwpackConfigError("Format version '%s' is not supported." % \ + format_string) + + @property def name(self): """The name of the hardware pack. A str.""" return self.parser.get(self.MAIN_SECTION, self.NAME_KEY) @@ -101,6 +164,96 @@ return None @property + def serial_tty(self): + """/dev device name of the serial console for this kernel + + A str. + """ + return self._get_option_from_main_section(self.SERIAL_TTY_KEY) + + @property + def kernel_addr(self): + """address where u-boot should load the kernel + + An int. + """ + return self._get_option_from_main_section(self.KERNEL_ADDR_KEY) + + @property + def initrd_addr(self): + """address where u-boot should load the kernel + + An int. + """ + return self._get_option_from_main_section(self.INITRD_ADDR_KEY) + + @property + def load_addr(self): + """address for uImage generation + + An int. + """ + return self._get_option_from_main_section(self.LOAD_ADDR_KEY) + + @property + def wired_interfaces(self): + """The interfaces for wired networks + + A list of str. + """ + return self._get_list_from_main_section(self.WIRED_INTERFACES_KEY) + + @property + def wireless_interfaces(self): + """The interfaces for wireless networks + + A list of str. + """ + return self._get_list_from_main_section(self.WIRELESS_INTERFACES_KEY) + + @property + def partition_layout(self): + """bootfs16_rootfs, bootfs_rootfs and reserved_bootfs_rootfs; + controls what kind of SD card partition layout we should use when + writing images + + A str. + """ + return self._get_option_from_main_section(self.PARTITION_LAYOUT_KEY) + + @property + def mmc_id(self): + """which MMC drive contains the boot filesystem + + An int. + """ + return self._get_option_from_main_section(self.MMC_ID_KEY) + + @property + def root_min_size(self): + """Minimum size of the root partition, in MiB. + + An int. + """ + return self._get_option_from_main_section(self.ROOT_MIN_SIZE_KEY) + + @property + def boot_min_size(self): + """Minimum size of the boot partition, in MiB. + + An int. + """ + return self._get_option_from_main_section(self.BOOT_MIN_SIZE_KEY) + + @property + def loader_min_size(self): + """Minimum size of the optional loader partition, in MiB. + + An int. + """ + return self._get_option_from_main_section(self.LOADER_MIN_SIZE_KEY) + + @property def origin(self): """The origin that should be recorded in the hwpack. @@ -144,6 +297,22 @@ return self._get_list_from_main_section(self.PACKAGES_KEY) @property + def u_boot_package(self): + """The u-boot package that contains the u-boot bin. + + A str. + """ + return self._get_option_from_main_section(self.U_BOOT_PACKAGE_KEY) + + @property + def u_boot_file(self): + """The u-boot bin file that will be unpacked from the u-boot package. + + A str. + """ + return self._get_option_from_main_section(self.U_BOOT_FILE_KEY) + + @property def architectures(self): """The architectures to build the hwpack for. @@ -174,17 +343,121 @@ section_name, self.SOURCES_ENTRY_KEY) return sources + def _validate_format(self): + format = self.format + if not format: + raise HwpackConfigError("Empty value for format") + if not format.is_supported: + raise HwpackConfigError("Format version '%s' is not supported." % \ + format) + + def _assert_matches_pattern(self, regex, config_item, error_message): + if re.match(regex, config_item) is None: + raise HwpackConfigError(error_message) + def _validate_name(self): try: name = self.name if not name: raise HwpackConfigError("Empty value for name") - if re.match(self.NAME_REGEX, name) is None: - raise HwpackConfigError("Invalid name: %s" % name) + self._assert_matches_pattern( + self.NAME_REGEX, name, "Invalid name: %s" % name) except ConfigParser.NoOptionError: raise HwpackConfigError( "No name in the [%s] section" % self.MAIN_SECTION) + def _validate_u_boot_file(self): + u_boot_file = self.u_boot_file + if not u_boot_file: + raise HwpackConfigError("No u_boot_file in the [%s] section" % \ + self.MAIN_SECTION) + self._assert_matches_pattern( + self.PATH_REGEX, u_boot_file, "Invalid path: %s" % u_boot_file) + + def _validate_serial_tty(self): + serial_tty = self.serial_tty + if serial_tty is None: + return + if len(serial_tty) < 4 or serial_tty[:3] != 'tty': + raise HwpackConfigError("Invalid serial tty: %s" % serial_tty) + + def _validate_addr(self, addr): + return re.match(r"^0x[a-fA-F0-9]{8}$", addr) + + def _validate_kernel_addr(self): + addr = self.kernel_addr + if addr is None: + return + if not self._validate_addr(addr): + raise HwpackConfigError("Invalid kernel address: %s" % addr) + + def _validate_initrd_addr(self): + addr = self.initrd_addr + if addr is None: + return + if not self._validate_addr(addr): + raise HwpackConfigError("Invalid initrd address: %s" % addr) + + def _validate_load_addr(self): + addr = self.load_addr + if addr is None: + return + if not self._validate_addr(addr): + raise HwpackConfigError("Invalid load address: %s" % addr) + + def _validate_wired_interfaces(self): + pass + + def _validate_wireless_interfaces(self): + pass + + def _validate_partition_layout(self): + if self.partition_layout not in self.DEFINED_PARTITION_LAYOUTS: + raise HwpackConfigError( + "Undefined partition layout %s in the [%s] section. " + "Valid partition layouts are %s." + % (self.partition_layout, self.MAIN_SECTION, + ", ".join(self.DEFINED_PARTITION_LAYOUTS))) + + def _validate_mmc_id(self): + mmc_id = self.mmc_id + if mmc_id is None: + return + try: + int(mmc_id) + except: + raise HwpackConfigError("Invalid mmc id %s" % (mmc_id)) + + def _validate_root_min_size(self): + root_min_size = self.root_min_size + if root_min_size is None: + return + try: + assert int(root_min_size) > 0 + except: + raise HwpackConfigError( + "Invalid root min size %s" % (root_min_size)) + + def _validate_boot_min_size(self): + boot_min_size = self.boot_min_size + if boot_min_size is None: + return + try: + assert int(boot_min_size) > 0 + except: + raise HwpackConfigError( + "Invalid boot min size %s" % (boot_min_size)) + + def _validate_loader_min_size(self): + loader_min_size = self.loader_min_size + if loader_min_size is None: + return + try: + assert int(loader_min_size) > 0 + except: + raise HwpackConfigError( + "Invalid loader min size %s" % (loader_min_size)) + def _validate_include_debs(self): try: self.include_debs @@ -206,10 +479,21 @@ "No %s in the [%s] section" % (self.PACKAGES_KEY, self.MAIN_SECTION)) for package in packages: - if re.match(self.PACKAGE_REGEX, package) is None: - raise HwpackConfigError( - "Invalid value in %s in the [%s] section: %s" - % (self.PACKAGES_KEY, self.MAIN_SECTION, package)) + self._assert_matches_pattern( + self.PACKAGE_REGEX, package, "Invalid value in %s in the " \ + "[%s] section: %s" % (self.PACKAGES_KEY, self.MAIN_SECTION, + package)) + + def _validate_u_boot_package(self): + u_boot_package = self.u_boot_package + if not u_boot_package: + raise HwpackConfigError( + "No %s in the [%s] section" + % (self.U_BOOT_PACKAGE_KEY, self.MAIN_SECTION)) + self._assert_matches_pattern( + self.PACKAGE_REGEX, u_boot_package, "Invalid value in %s in the " \ + "[%s] section: %s" % (self.U_BOOT_PACKAGE_KEY, + self.MAIN_SECTION, u_boot_package)) def _validate_architectures(self): architectures = self.architectures @@ -221,11 +505,10 @@ def _validate_assume_installed(self): assume_installed = self.assume_installed for package in assume_installed: - if re.match(self.PACKAGE_REGEX, package) is None: - raise HwpackConfigError( - "Invalid value in %s in the [%s] section: %s" - % (self.ASSUME_INSTALLED_KEY, self.MAIN_SECTION, - package)) + self._assert_matches_pattern( + self.PACKAGE_REGEX, package, "Invalid value in %s in the " \ + "[%s] section: %s" % (self.ASSUME_INSTALLED_KEY, + self.MAIN_SECTION, package)) def _validate_section_sources_entry(self, section_name): try: === modified file 'linaro_image_tools/hwpack/hardwarepack.py' --- linaro_image_tools/hwpack/hardwarepack.py 2011-03-23 22:25:10 +0000 +++ linaro_image_tools/hwpack/hardwarepack.py 2011-06-29 14:25:49 +0000 @@ -20,6 +20,7 @@ # USA. import time +import os from linaro_image_tools.hwpack.better_tarfile import writeable_tarfile from linaro_image_tools.hwpack.packages import ( @@ -27,6 +28,9 @@ get_packages_file, PackageMaker, ) +from linaro_image_tools.hwpack.hardwarepack_format import ( + HardwarePackFormatV1, +) class Metadata(object): @@ -55,11 +59,12 @@ """ def __init__(self, name, version, architecture, origin=None, - maintainer=None, support=None): + maintainer=None, support=None, format=HardwarePackFormatV1()): """Create the Metadata for a hardware pack. See the instance variables for a description of the arguments. """ + self.format = format self.name = name if ' ' in version: raise AssertionError( @@ -72,6 +77,29 @@ self.architecture = architecture @classmethod + def add_v2_config(self, serial_tty=None, kernel_addr=None, initrd_addr=None, + load_addr=None, fdt=None, wired_interfaces=[], + wireless_interfaces=[], partition_layout=None, + mmc_id=None, boot_min_size=None, root_min_size=None, + loader_min_size=None): + """Add fields that are specific to the new format. + + These fields are not present in earlier config files. + """ + self.u_boot = None + self.serial_tty = serial_tty + self.kernel_addr = kernel_addr + self.initrd_addr = initrd_addr + self.load_addr = load_addr + self.wired_interfaces = wired_interfaces + self.wireless_interfaces = wireless_interfaces + self.partition_layout = partition_layout + self.mmc_id = mmc_id + self.boot_min_size = boot_min_size + self.root_min_size = root_min_size + self.loader_min_size = loader_min_size + + @classmethod def from_config(cls, config, version, architecture): """Create a Metadata from a Config object. @@ -89,9 +117,24 @@ targetting. :type architecture: str """ - return cls( + metadata = cls( config.name, version, architecture, origin=config.origin, - maintainer=config.maintainer, support=config.support) + maintainer=config.maintainer, support=config.support, + format=config.format) + + if config.format.has_v2_fields: + metadata.add_v2_config(serial_tty=config.serial_tty, + kernel_addr=config.kernel_addr, + initrd_addr=config.initrd_addr, + load_addr=config.load_addr, + wired_interfaces=config.wired_interfaces, + wireless_interfaces=config.wireless_interfaces, + partition_layout=config.partition_layout, + mmc_id=config.mmc_id, + boot_min_size=config.boot_min_size, + root_min_size=config.root_min_size, + loader_min_size=config.loader_min_size) + return metadata def __str__(self): """Get the contents of the metadata file.""" @@ -104,6 +147,36 @@ metadata += "MAINTAINER=%s\n" % self.maintainer if self.support is not None: metadata += "SUPPORT=%s\n" % self.support + + if not self.format.has_v2_fields: + return metadata + + if self.u_boot is not None: + metadata += "U_BOOT=%s\n" % self.u_boot + if self.serial_tty is not None: + metadata += "SERIAL_TTY=%s\n" % self.serial_tty + if self.kernel_addr is not None: + metadata += "KERNEL_ADDR=%s\n" % self.kernel_addr + if self.initrd_addr is not None: + metadata += "INITRD_ADDR=%s\n" % self.initrd_addr + if self.load_addr is not None: + metadata += "LOAD_ADDR=%s\n" % self.load_addr + if self.wired_interfaces != []: + metadata += "WIRED_INTERFACES=%s\n" % " ".join(self.wired_interfaces) + if self.wireless_interfaces != []: + metadata += "WIRELESS_INTERFACES=%s\n" % " ".join( + self.wireless_interfaces) + if self.partition_layout is not None: + metadata += "PARTITION_LAYOUT=%s\n" % self.partition_layout + if self.mmc_id is not None: + metadata += "MMC_ID=%s\n" % self.mmc_id + if self.boot_min_size is not None: + metadata += "BOOT_MIN_SIZE=%s\n" % self.boot_min_size + if self.root_min_size is not None: + metadata += "ROOT_MIN_SIZE=%s\n" % self.root_min_size + if self.loader_min_size is not None: + metadata += "LOADER_MIN_SIZE=%s\n" % self.loader_min_size + return metadata @@ -116,8 +189,6 @@ :type FORMAT: str """ - # The format version cannot contain white spaces. - FORMAT = "1.0" FORMAT_FILENAME = "FORMAT" METADATA_FILENAME = "metadata" MANIFEST_FILENAME = "manifest" @@ -125,6 +196,7 @@ PACKAGES_FILENAME = "%s/Packages" % PACKAGES_DIRNAME SOURCES_LIST_DIRNAME = "sources.list.d" SOURCES_LIST_GPG_DIRNAME = "sources.list.d.gpg" + U_BOOT_DIR = "u-boot" def __init__(self, metadata): """Create a HardwarePack. @@ -135,6 +207,8 @@ self.metadata = metadata self.sources = {} self.packages = [] + self.format = metadata.format + self.files = [] def filename(self, extension=".tar.gz"): """The filename that this hardware pack should have. @@ -200,6 +274,11 @@ relationships, self.metadata.architecture) self.packages.append(FetchedPackage.from_deb(deb_file_path)) + def add_file(self, dir, file): + target_file = os.path.join(dir, os.path.basename(file)) + self.files.append((file, target_file)) + return target_file + def manifest_text(self): manifest_content = "" for package in self.packages: @@ -225,9 +304,11 @@ kwargs["default_mtime"] = time.time() with writeable_tarfile(fileobj, mode="w:gz", **kwargs) as tf: tf.create_file_from_string( - self.FORMAT_FILENAME, self.FORMAT + "\n") + self.FORMAT_FILENAME, "%s\n" % self.format) tf.create_file_from_string( self.METADATA_FILENAME, str(self.metadata)) + for fs_file_name, arc_file_name in self.files: + tf.add(fs_file_name, arcname=arc_file_name) tf.create_dir(self.PACKAGES_DIRNAME) for package in self.packages: if package.content is not None: === added file 'linaro_image_tools/hwpack/hardwarepack_format.py' --- linaro_image_tools/hwpack/hardwarepack_format.py 1970-01-01 00:00:00 +0000 +++ linaro_image_tools/hwpack/hardwarepack_format.py 2011-06-17 12:28:43 +0000 @@ -0,0 +1,59 @@ +# Copyright (C) 2010, 2011 Linaro +# +# Author: James Westby +# +# This file is part of Linaro Image Tools. +# +# Linaro Image Tools is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# Linaro Image Tools is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Linaro Image Tools; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, +# USA. + +import logging + + +logger = logging.getLogger(__name__) + + +class HardwarePackFormat(object): + def __init__(self): + self.format_as_string = None + self.is_deprecated = False + self.is_supported = False + self.has_v2_fields = False + + def __str__(self): + if self.format_as_string is None: + raise NotImplementedError() + if self.is_deprecated: + logger.warning("The format '%s' is deprecated, please update " \ + "your hardware pack configuration." % \ + self.format_as_string) + return self.format_as_string + + +class HardwarePackFormatV1(HardwarePackFormat): + def __init__(self): + super(HardwarePackFormatV1, self).__init__() + self.format_as_string = "1.0" + self.is_supported = True + self.is_deprecated = False + + +class HardwarePackFormatV2(HardwarePackFormat): + def __init__(self): + super(HardwarePackFormatV2, self).__init__() + self.format_as_string = "2.0" + self.is_supported = True + self.is_deprecated = False + self.has_v2_fields = True === modified file 'linaro_image_tools/hwpack/packages.py' --- linaro_image_tools/hwpack/packages.py 2011-05-25 20:59:19 +0000 +++ linaro_image_tools/hwpack/packages.py 2011-06-20 13:33:01 +0000 @@ -724,4 +724,5 @@ "The item %r could not be fetched: %s" % (acqfile.destfile, acqfile.error_text)) result_package.content = open(destfile) + result_package._file_path = destfile return fetched.values() === modified file 'linaro_image_tools/hwpack/tests/test_builder.py' --- linaro_image_tools/hwpack/tests/test_builder.py 2011-03-24 11:14:44 +0000 +++ linaro_image_tools/hwpack/tests/test_builder.py 2011-06-20 13:46:16 +0000 @@ -27,6 +27,7 @@ from linaro_image_tools.hwpack.builder import ( ConfigFileMissing, + PackageUnpacker, HardwarePackBuilder, logger as builder_logger, ) @@ -50,6 +51,10 @@ Not, ) from linaro_image_tools.testing import TestCaseWithFixtures +from linaro_image_tools.tests.fixtures import ( + MockSomethingFixture, + MockCmdRunnerPopenFixture, + ) class ConfigFileMissingTests(TestCase): @@ -59,6 +64,52 @@ self.assertEqual("No such config file: 'path'", str(exc)) +class PackageUnpackerTests(TestCaseWithFixtures): + + def test_creates_tempdir(self): + with PackageUnpacker() as package_unpacker: + self.assertTrue(os.path.exists(package_unpacker.tempdir)) + + def test_tempfiles_are_removed(self): + tempdir = None + with PackageUnpacker() as package_unpacker: + tempdir = package_unpacker.tempdir + self.assertFalse(os.path.exists(tempdir)) + + def test_unpack_package(self): + fixture = MockCmdRunnerPopenFixture(assert_child_finished=False) + self.useFixture(fixture) + package_file_name = "package-to-unpack" + with PackageUnpacker() as package_unpacker: + package_unpacker.unpack_package(package_file_name) + package_dir = package_unpacker.tempdir + self.assertEquals( + ["tar -C %s -xf -" % package_dir, + "dpkg --fsys-tarfile %s" % package_file_name], + fixture.mock.commands_executed) + + def test_get_file_returns_tempfile(self): + package = 'package' + file = 'dummyfile' + with PackageUnpacker() as package_unpacker: + self.useFixture(MockSomethingFixture( + package_unpacker, 'unpack_package', lambda package: None)) + self.useFixture(MockSomethingFixture( + os.path, 'exists', lambda file: True)) + tempfile = package_unpacker.get_file(package, file) + self.assertEquals(tempfile, + os.path.join(package_unpacker.tempdir, file)) + + def test_get_file_raises(self): + package = 'package' + file = 'dummyfile' + with PackageUnpacker() as package_unpacker: + self.useFixture(MockSomethingFixture( + package_unpacker, 'unpack_package', lambda package: None)) + self.assertRaises(AssertionError, package_unpacker.get_file, + package, file) + + class HardwarePackBuilderTests(TestCaseWithFixtures): def setUp(self): @@ -95,6 +146,58 @@ config = self.useFixture(ConfigFileFixture(config_text)) return Metadata(hwpack_name, hwpack_version, architecture), config + def test_find_fetched_package_finds(self): + package_name = "dummy-package" + wanted_package_name = "wanted-package" + available_package = DummyFetchedPackage(package_name, "1.1") + wanted_package = DummyFetchedPackage(wanted_package_name, "1.1") + + sources_dict = self.sourcesDictForPackages([available_package, + wanted_package]) + _, config = self.makeMetaDataAndConfigFixture( + [package_name, wanted_package_name], sources_dict, + extra_config={'format': '2.0', 'u-boot-package': 'wanted-package', + 'u-boot-file': 'wanted-file', + 'partition_layout': 'bootfs_rootfs'}) + builder = HardwarePackBuilder(config.filename, "1.0", []) + found_package = builder.find_fetched_package( + [available_package, wanted_package], wanted_package_name) + self.assertEquals(wanted_package, found_package) + + def test_find_fetched_package_removes(self): + package_name = "dummy-package" + wanted_package_name = "wanted-package" + available_package = DummyFetchedPackage(package_name, "1.1") + wanted_package = DummyFetchedPackage(wanted_package_name, "1.1") + + sources_dict = self.sourcesDictForPackages([available_package, + wanted_package]) + _, config = self.makeMetaDataAndConfigFixture( + [package_name, wanted_package_name], sources_dict, + extra_config={'format': '2.0', 'u-boot-package': 'wanted-package', + 'u-boot-file': 'wanted-file', + 'partition_layout': 'bootfs_rootfs'}) + builder = HardwarePackBuilder(config.filename, "1.0", []) + packages = [available_package, wanted_package] + builder.find_fetched_package(packages, wanted_package_name) + self.assertEquals(packages, [available_package]) + + def test_find_fetched_package_raises(self): + package_name = "dummy-package" + wanted_package_name = "wanted-package" + available_package = DummyFetchedPackage(package_name, "1.1") + + sources_dict = self.sourcesDictForPackages([available_package]) + _, config = self.makeMetaDataAndConfigFixture( + [package_name], sources_dict, + extra_config={'format': '2.0', 'u-boot-package': 'wanted-package', + 'u-boot-file': 'wanted-file', + 'partition_layout': 'bootfs_rootfs'}) + builder = HardwarePackBuilder(config.filename, "1.0", []) + packages = [available_package] + self.assertRaises(AssertionError, builder.find_fetched_package, + packages, wanted_package_name) + def test_creates_external_manifest(self): available_package = DummyFetchedPackage("foo", "1.1") sources_dict = self.sourcesDictForPackages([available_package]) === modified file 'linaro_image_tools/hwpack/tests/test_config.py' --- linaro_image_tools/hwpack/tests/test_config.py 2011-05-10 15:09:20 +0000 +++ linaro_image_tools/hwpack/tests/test_config.py 2011-06-29 14:25:49 +0000 @@ -30,6 +30,13 @@ valid_start = ( "[hwpack]\nname = ahwpack\npackages = foo\narchitectures = armel\n") + valid_start_v2 = valid_start + "format = 2.0\n" + valid_complete_v2 = (valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n" \ + "u-boot-file = usr/lib/u-boot/smdkv310/" \ + "u-boot.bin\nserial_tty=ttySAC1\n" \ + "partition_layout = bootfs_rootfs\n") + valid_end = "[ubuntu]\nsources-entry = foo bar\n" def test_create(self): config = Config(StringIO()) @@ -170,6 +177,231 @@ "[ubuntu]\nsources-entry = foo bar\n") self.assertEqual(None, config.validate()) + def test_validate_supported_format(self): + config = self.get_config( + self.valid_start + + "\nformat = 0.9\n") + self.assertValidationError( + "Format version '0.9' is not supported.", config) + + def test_validate_invalid_u_boot_package_name(self): + config = self.get_config( + self.valid_start_v2 + "u-boot-package = ~~\n") + self.assertValidationError( + "Invalid value in u-boot-package in the [hwpack] section: ~~", + config) + + def test_validate_empty_u_boot_package(self): + config = self.get_config( + self.valid_start_v2 + "u-boot-package = \n") + self.assertValidationError( + "No u-boot-package in the [hwpack] section", config) + + def test_validate_no_u_boot_file(self): + config = self.get_config(self.valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n") + self.assertValidationError("No u_boot_file in the [hwpack] section", + config) + + def test_validate_empty_u_boot_file(self): + config = self.get_config(self.valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n" \ + "u-boot-file = \n") + self.assertValidationError("No u_boot_file in the [hwpack] section", config) + + def test_validate_invalid_u_boot_file(self): + config = self.get_config(self.valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n" \ + "u-boot-file = ~~\n") + self.assertValidationError("Invalid path: ~~", config) + + def test_validate_partition_layout(self): + partition_layout = 'apafs_bananfs' + config = self.get_config(self.valid_start_v2 + "u-boot-package = " \ + "u-boot-linaro-s5pv310\nu-boot-file = " \ + "u-boot.bin\npartition_layout = %s\n" % \ + partition_layout) + self.assertValidationError( + "Undefined partition layout %s in the [%s] section. " + "Valid partition layouts are %s." + % (partition_layout, 'hwpack', + ", ".join(config.DEFINED_PARTITION_LAYOUTS)), config) + + def test_validate_wired_interfaces(self): + self.assertTrue("XXX What is an invalid interface name?") + + def test_validate_wireless_interfaces(self): + self.assertTrue("XXX What is an invalid interface name?") + + def test_validate_serial_tty(self): + config = self.get_config(self.valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n" \ + "u-boot-file = u-boot.bin\nserial_tty=tty\n") + self.assertValidationError("Invalid serial tty: tty", config) + config = self.get_config(self.valid_start_v2 + + "u-boot-package = u-boot-linaro-s5pv310\n" \ + "u-boot-file = u-boot.bin\n" \ + "serial_tty=ttxSAC1\n") + self.assertValidationError("Invalid serial tty: ttxSAC1", config) + + def test_validate_mmc_id(self): + config = self.get_config(self.valid_complete_v2 + + "mmc_id = x\n") + self.assertValidationError("Invalid mmc id x", config) + + def test_validate_boot_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "boot_min_size = x\n") + self.assertValidationError("Invalid boot min size x", config) + + def test_validate_root_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "root_min_size = x\n") + self.assertValidationError("Invalid root min size x", config) + + def test_validate_loader_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "loader_min_size = x\n") + self.assertValidationError("Invalid loader min size x", config) + + def test_validate_kernel_addr(self): + config = self.get_config(self.valid_complete_v2 + + "kernel_addr = 0x8000000\n") + self.assertValidationError("Invalid kernel address: 0x8000000", config) + config = self.get_config(self.valid_complete_v2 + + "kernel_addr = 0x8000000x\n") + self.assertValidationError("Invalid kernel address: 0x8000000x", config) + config = self.get_config(self.valid_complete_v2 + + "kernel_addr = 80000000\n") + self.assertValidationError("Invalid kernel address: 80000000", config) + + def test_validate_initrd_addr(self): + config = self.get_config(self.valid_complete_v2 + + "initrd_addr = 0x8000000\n") + self.assertValidationError("Invalid initrd address: 0x8000000", config) + config = self.get_config(self.valid_complete_v2 + + "initrd_addr = 0x8000000x\n") + self.assertValidationError("Invalid initrd address: 0x8000000x", config) + config = self.get_config(self.valid_complete_v2 + + "initrd_addr = 80000000\n") + self.assertValidationError("Invalid initrd address: 80000000", config) + + def test_validate_load_addr(self): + config = self.get_config(self.valid_complete_v2 + + "load_addr = 0x8000000\n") + self.assertValidationError("Invalid load address: 0x8000000", config) + config = self.get_config(self.valid_complete_v2 + + "load_addr = 0x8000000x\n") + self.assertValidationError("Invalid load address: 0x8000000x", config) + config = self.get_config(self.valid_complete_v2 + + "load_addr = 80000000\n") + self.assertValidationError("Invalid load address: 80000000", config) + + def test_wired_interfaces(self): + config = self.get_config(self.valid_complete_v2 + + "wired_interfaces = eth0\n" + + self.valid_end) + config.validate() + self.assertEqual(["eth0"], config.wired_interfaces) + config = self.get_config(self.valid_complete_v2 + + "wired_interfaces = eth0 eth1 usb2\n" + + self.valid_end) + config.validate() + self.assertEqual(["eth0", "eth1", "usb2"], config.wired_interfaces) + + def test_wireless_interfaces(self): + config = self.get_config(self.valid_complete_v2 + + "wireless_interfaces = wlan0\n" + + self.valid_end) + config.validate() + self.assertEqual(["wlan0"], config.wireless_interfaces) + config = self.get_config(self.valid_complete_v2 + + "wireless_interfaces = wlan0 wl1 usb2\n" + + self.valid_end) + config.validate() + self.assertEqual(["wlan0", "wl1", "usb2"], config.wireless_interfaces) + + def test_partition_layout(self): + config = self.get_config(self.valid_complete_v2 + self.valid_end) + config.validate() + self.assertEqual("bootfs_rootfs", + config.partition_layout) + + def test_u_boot_file(self): + config = self.get_config(self.valid_complete_v2 + self.valid_end) + config.validate() + self.assertEqual("usr/lib/u-boot/smdkv310/u-boot.bin", + config.u_boot_file) + + def test_serial_tty(self): + config = self.get_config(self.valid_complete_v2 + self.valid_end) + config.validate() + self.assertEqual("ttySAC1", config.serial_tty) + + def test_mmc_id(self): + config = self.get_config(self.valid_complete_v2 + + "mmc_id = 1\n" + + self.valid_end) + config.validate() + self.assertEqual("1", config.mmc_id) + + def test_boot_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "boot_min_size = 50\n" + + self.valid_end) + config.validate() + self.assertEqual("50", config.boot_min_size) + + def test_root_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "root_min_size = 50\n" + + self.valid_end) + config.validate() + self.assertEqual("50", config.root_min_size) + + def test_loader_min_size(self): + config = self.get_config(self.valid_complete_v2 + + "loader_min_size = 2\n" + + self.valid_end) + config.validate() + self.assertEqual("2", config.loader_min_size) + + def test_kernel_addr(self): + config = self.get_config(self.valid_complete_v2 + + "kernel_addr = 0x80000000\n" + + self.valid_end) + config.validate() + self.assertEqual("0x80000000", config.kernel_addr) + config = self.get_config(self.valid_complete_v2 + + "kernel_addr = 0x8aBcdEFf\n" + + self.valid_end) + config.validate() + self.assertEqual("0x8aBcdEFf", config.kernel_addr) + + def test_initrd_addr(self): + config = self.get_config(self.valid_complete_v2 + + "initrd_addr = 0x80000000\n" + + self.valid_end) + config.validate() + self.assertEqual("0x80000000", config.initrd_addr) + config = self.get_config(self.valid_complete_v2 + + "initrd_addr = 0x8aBcdEFf\n" + + self.valid_end) + config.validate() + self.assertEqual("0x8aBcdEFf", config.initrd_addr) + + def test_load_addr(self): + config = self.get_config(self.valid_complete_v2 + + "load_addr = 0x80000000\n" + + self.valid_end) + config.validate() + self.assertEqual("0x80000000", config.load_addr) + config = self.get_config(self.valid_complete_v2 + + "load_addr = 0x8aBcdEFf\n" + + self.valid_end) + config.validate() + self.assertEqual("0x8aBcdEFf", config.load_addr) + def test_name(self): config = self.get_config( "[hwpack]\nname = ahwpack\npackages = foo\n" === modified file 'linaro_image_tools/hwpack/tests/test_hardwarepack.py' --- linaro_image_tools/hwpack/tests/test_hardwarepack.py 2011-03-23 22:25:10 +0000 +++ linaro_image_tools/hwpack/tests/test_hardwarepack.py 2011-06-29 14:30:25 +0000 @@ -37,29 +37,32 @@ MatchesStructure, Not, ) +from linaro_image_tools.hwpack.hardwarepack_format import ( + HardwarePackFormatV1, + HardwarePackFormatV2, + ) class MetadataTests(TestCase): + def setUp(self): + super(MetadataTests, self).setUp() + self.metadata = Metadata("ahwpack", "3", "armel") def test_name(self): - metadata = Metadata("ahwpack", "3", "armel") - self.assertEqual("ahwpack", metadata.name) + self.assertEqual("ahwpack", self.metadata.name) def test_version(self): - metadata = Metadata("ahwpack", "3", "armel") - self.assertEqual("3", metadata.version) + self.assertEqual("3", self.metadata.version) def test_version_with_whitespace(self): self.assertRaises( AssertionError, Metadata, "ahwpack", "3 (with extras)", "armel") def test_architecture(self): - metadata = Metadata("ahwpack", "3", "armel") - self.assertEqual("armel", metadata.architecture) + self.assertEqual("armel", self.metadata.architecture) def test_default_origin_is_None(self): - metadata = Metadata("ahwpack", "4", "armel") - self.assertEqual(None, metadata.origin) + self.assertEqual(None, self.metadata.origin) def test_origin(self): metadata = Metadata("ahwpack", "4", "armel", origin="linaro") @@ -108,12 +111,112 @@ "SUPPORT=unsupported\n", str(metadata)) + def test_str_with_serial_tty(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(serial_tty='ttyO2') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "SERIAL_TTY=ttyO2\n", + str(metadata)) + + def test_str_with_kernel_addr(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(kernel_addr='0x80000000') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "KERNEL_ADDR=0x80000000\n", + str(metadata)) + + def test_str_with_initrd_addr(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(initrd_addr='0x80000000') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "INITRD_ADDR=0x80000000\n", + str(metadata)) + + def test_str_with_load_addr(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(load_addr='0x80000000') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "LOAD_ADDR=0x80000000\n", + str(metadata)) + + def test_str_with_wired_interfaces(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(wired_interfaces=['eth0', 'usb0']) + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "WIRED_INTERFACES=eth0 usb0\n", + str(metadata)) + + def test_str_with_wireless_interfaces(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(wireless_interfaces=['wlan0', 'wl0']) + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "WIRELESS_INTERFACES=wlan0 wl0\n", + str(metadata)) + + def test_str_with_partition_layout(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(partition_layout='bootfs_rootfs') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "PARTITION_LAYOUT=bootfs_rootfs\n", + str(metadata)) + + def test_str_with_mmc_id(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(mmc_id='1') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "MMC_ID=1\n", + str(metadata)) + + def test_str_with_boot_min_size(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(boot_min_size='50') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "BOOT_MIN_SIZE=50\n", + str(metadata)) + + def test_str_with_root_min_size(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(root_min_size='100') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "ROOT_MIN_SIZE=100\n", + str(metadata)) + + def test_str_with_loader_min_size(self): + metadata = Metadata("ahwpack", "4", "armel", + format=HardwarePackFormatV2()) + metadata.add_v2_config(loader_min_size='1') + self.assertEqual( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\n" + "LOADER_MIN_SIZE=1\n", + str(metadata)) + def test_from_config(self): class Config: name = "foo" origin = "linaro" maintainer = "someone" support = "supported" + format = HardwarePackFormatV1() config = Config() metadata = Metadata.from_config(config, "2.0", "i386") self.assertEqual(config.name, metadata.name) @@ -130,14 +233,15 @@ super(HardwarePackTests, self).setUp() self.metadata = Metadata("ahwpack", "4", "armel") - def test_format_is_1_0(self): + def test_format_is_correct(self): + format = '1.0' hwpack = HardwarePack(self.metadata) - self.assertEqual("1.0", hwpack.FORMAT) + self.assertEqual(format, hwpack.format.__str__()) def test_format_has_no_spaces(self): hwpack = HardwarePack(self.metadata) - self.assertIs(None, re.search('\s', hwpack.FORMAT), - "hwpack.FORMAT contains spaces.") + self.assertIs(None, re.search('\s', hwpack.format.__str__()), + "hwpack.format contains spaces.") def test_filename(self): hwpack = HardwarePack(self.metadata) @@ -167,7 +271,7 @@ tf = self.get_tarfile(hwpack) self.assertThat( tf, - HardwarePackHasFile("FORMAT", content=hwpack.FORMAT+"\n")) + HardwarePackHasFile("FORMAT", content=hwpack.format.__str__()+"\n")) def test_creates_metadata_file(self): metadata = Metadata( === modified file 'linaro_image_tools/index_server.py' --- linaro_image_tools/index_server.py 2011-06-17 18:04:47 +0000 +++ linaro_image_tools/index_server.py 2011-06-29 06:29:34 +0000 @@ -22,10 +22,10 @@ import os import re -import FetchImage import urlparse import logging import bz2 +import linaro_image_tools.FetchImage RELEASES_WWW_DOCUMENT_ROOT = "/srv/releases.linaro.org/www/platform/" RELEASE_URL = "http://releases.linaro.org/platform/" @@ -41,7 +41,7 @@ def __init__(self): self.reset() self.db_file_name = "server_index" - self.db = FetchImage.DB(self.db_file_name) + self.db = linaro_image_tools.FetchImage.DB(self.db_file_name) def crawl(self): self.db.set_url_parse_info(self.url_parse) === modified file 'linaro_image_tools/media_create/android_boards.py' --- linaro_image_tools/media_create/android_boards.py 2011-04-22 15:09:08 +0000 +++ linaro_image_tools/media_create/android_boards.py 2011-07-01 09:16:24 +0000 @@ -28,6 +28,8 @@ from linaro_image_tools.media_create.boards import PART_ALIGN_S from linaro_image_tools.media_create.boards import BeagleConfig from linaro_image_tools.media_create.boards import PandaConfig +from linaro_image_tools.media_create.boards import SnowballSdConfig +from linaro_image_tools.media_create.boards import SnowballEmmcConfig from linaro_image_tools.media_create.boards import ( align_up, align_partition, @@ -37,6 +39,7 @@ from linaro_image_tools import cmd_runner import os + class AndroidBoardConfig(object): @classmethod def _get_bootargs(cls, consoles): @@ -78,7 +81,7 @@ as_root=True).wait() boot_env = cls._get_boot_env(consoles) - cmdline_filepath = os.path.join(boot_disk, "cmdline") + cmdline_filepath = os.path.join(boot_disk, "cmdline") cmdline_file = open(cmdline_filepath, 'r') android_kernel_cmdline = cmdline_file.read() boot_env['bootargs'] = boot_env['bootargs'] + ' ' + \ @@ -96,7 +99,8 @@ pass @classmethod - def get_sfdisk_cmd(cls, should_align_boot_part=False): + def get_sfdisk_cmd(cls, should_align_boot_part=False, + start_addr=0, extra_part=False): if cls.fat_size == 32: partition_type = '0x0C' else: @@ -116,7 +120,7 @@ # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( - 1, BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) + start_addr + 1, BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room @@ -131,12 +135,28 @@ _cache_end + 1, USERDATA_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) sdcard_start, _sdcard_end, _sdcard_len = align_partition( _userdata_end + 1, SDCARD_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) - + + # Snowball board needs a raw partition added to the beginning of image. + # If extra_part is True an extra primary partition will be added. + # Due to a maximum of 4 primary partitions cache data will be placed in + # a extended partition + if extra_part == True: + assert start_addr > 0, ("Not possible to add extra partition" \ + "when boot partition starts at '0'") + return '%s,%s,%s,*\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,%s,L\n%s,,,-' % ( + boot_start, boot_len, partition_type, system_start, _system_len, + cache_start, cache_start, _cache_len, userdata_start, + _userdata_len, sdcard_start) + return '%s,%s,%s,*\n%s,%s,L\n%s,%s,L\n%s,-,E\n%s,%s,L\n%s,,,-' % ( boot_start, boot_len, partition_type, system_start, _system_len, cache_start, _cache_len, userdata_start, userdata_start, _userdata_len, sdcard_start) + @classmethod + def populate_raw_partition(cls, media, boot_dir): + super(AndroidBoardConfig, cls).populate_raw_partition(boot_dir, media) + class AndroidOmapConfig(AndroidBoardConfig): pass @@ -149,10 +169,50 @@ class AndroidPandaConfig(AndroidOmapConfig, PandaConfig): _extra_serial_opts = 'console=tty0 console=ttyO2,115200n8' + extra_boot_args_options = ( + 'earlyprintk fixrtc nocompcache vram=48M ' + 'omapfb.vram=0:24M,1:24M mem=456M@0x80000000 mem=512M@0xA0000000') android_specific_args = 'init=/init androidboot.console=ttyO2' +class AndroidSnowballSdConfig(AndroidBoardConfig, SnowballSdConfig): + extra_boot_args_options = ( + 'earlyprintk rootdelay=1 fixrtc nocompcache ' + 'mem=128M@0 mali.mali_mem=64M@128M mem=24M@192M hwmem=167M@216M ' + 'mem_issw=1M@383M mem=640M@384M vmalloc=256M') + _extra_serial_opts = 'console=tty0 console=ttyO2,115200n8' + android_specific_args = 'init=/init androidboot.console=ttyAMA2' + + +class AndroidSnowballEmmcConfig(AndroidBoardConfig, SnowballEmmcConfig): + extra_boot_args_options = ( + 'earlyprintk rootdelay=1 fixrtc nocompcache ' + 'mem=128M@0 mali.mali_mem=64M@128M mem=24M@192M hwmem=167M@216M ' + 'mem_issw=1M@383M mem=640M@384M vmalloc=256M') + _extra_serial_opts = 'console=tty0 console=ttyAMA2,115200n8' + android_specific_args = 'init=/init androidboot.console=ttyAMA2' + + @classmethod + def get_sfdisk_cmd(cls, should_align_boot_part=False): + + LOADER_MIN_SIZE_S = align_up( + 1 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE + + loader_start, loader_end, loader_len = align_partition( + SnowballEmmcConfig.SNOWBALL_LOADER_START_S, + LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) + + command = super(AndroidSnowballEmmcConfig, cls).get_sfdisk_cmd( + should_align_boot_part=True, start_addr=loader_end, + extra_part=True) + + return '%s,%s,0xDA\n%s' % ( + loader_start, loader_len, command) + + android_board_configs = { 'beagle': AndroidBeagleConfig, 'panda': AndroidPandaConfig, + 'snowball_sd': AndroidSnowballSdConfig, + 'snowball_emmc': AndroidSnowballEmmcConfig, } === modified file 'linaro_image_tools/media_create/boards.py' --- linaro_image_tools/media_create/boards.py 2011-05-26 20:19:23 +0000 +++ linaro_image_tools/media_create/boards.py 2011-07-01 14:55:14 +0000 @@ -31,11 +31,15 @@ import tempfile import struct from binascii import crc32 +import tarfile +import ConfigParser +import shutil from linaro_image_tools import cmd_runner from linaro_image_tools.media_create.partitions import SECTOR_SIZE + KERNEL_GLOB = 'vmlinuz-*-%(kernel_flavor)s' INITRD_GLOB = 'initrd.img-*-%(kernel_flavor)s' DTB_GLOB = 'dt-*-%(kernel_flavor)s/%(dtb_name)s' @@ -58,21 +62,11 @@ # align on 4 MiB PART_ALIGN_S = 4 * 1024 * 1024 / SECTOR_SIZE + def align_up(value, align): """Round value to the next multiple of align.""" return (value + align - 1) / align * align -# optional bootloader partition; at least 1 MiB; in theory, an i.MX5x -# bootloader partition could hold RedBoot, FIS table, RedBoot config, kernel, -# and initrd, but we typically use U-Boot which is about 167 KiB as of -# 2011/02/11 and currently doesn't even store its environment there, so this -# should be enough -LOADER_MIN_SIZE_S = align_up(1 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE -# boot partition; at least 50 MiB; XXX this shouldn't be hardcoded -BOOT_MIN_SIZE_S = align_up(50 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE -# root partition; at least 50 MiB; XXX this shouldn't be hardcoded -ROOT_MIN_SIZE_S = align_up(50 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE - # Samsung v310 implementation notes and terminology # # * BL0, BL1 etc. are the various bootloaders in order of execution @@ -104,6 +98,7 @@ assert SAMSUNG_V310_BL2_LEN * SECTOR_SIZE == 512 * 1024, ( "BL1 expects BL2 (u-boot) to be 512 KiB") + def align_partition(min_start, min_length, start_alignment, end_alignment): """Compute partition start and end offsets based on specified constraints. @@ -125,10 +120,103 @@ """A descriptor that provides @property behavior on class methods.""" def __init__(self, getter): self.getter = getter + def __get__(self, instance, cls): return self.getter(cls) +class HardwarepackHandler(object): + FORMAT_1 = '1.0' + FORMAT_2 = '2.0' + FORMAT_MIXED = '1.0and2.0' + metadata_filename = 'metadata' + format_filename = 'FORMAT' + main_section = 'main' + hwpack_tarfiles = [] + tempdir = None + + def __init__(self, hwpacks): + self.hwpacks = hwpacks + self.hwpack_tarfiles = [] + + class FakeSecHead(object): + """ Add a fake section header to the metadata file. + + This is done so we can use ConfigParser to parse the file. + """ + def __init__(self, fp): + self.fp = fp + self.sechead = '[%s]\n' % HardwarepackHandler.main_section + + def readline(self): + if self.sechead: + try: + return self.sechead + finally: + self.sechead = None + else: + return self.fp.readline() + + def __enter__(self): + self.tempdir = tempfile.mkdtemp() + for hwpack in self.hwpacks: + hwpack_tarfile = tarfile.open(hwpack, mode='r:gz') + self.hwpack_tarfiles.append(hwpack_tarfile) + return self + + def __exit__(self, type, value, traceback): + for hwpack_tarfile in self.hwpack_tarfiles: + if hwpack_tarfile is not None: + hwpack_tarfile.close() + self.hwpack_tarfiles = [] + if self.tempdir is not None and os.path.exists(self.tempdir): + shutil.rmtree(self.tempdir) + + def get_field(self, section, field): + data = None + hwpack_with_data = None + for hwpack_tarfile in self.hwpack_tarfiles: + metadata = hwpack_tarfile.extractfile(self.metadata_filename) + # Use RawConfigParser which does not support the magical interpolation + # behavior of ConfigParser so we don't mess up metadata accidentally. + parser = ConfigParser.RawConfigParser() + parser.readfp(self.FakeSecHead(metadata)) + try: + new_data = parser.get(section, field) + if new_data is not None: + assert data is None, "The metadata field '%s' is set to " \ + "'%s' and new value '%s' is found" % (field, data, new_data) + data = new_data + hwpack_with_data = hwpack_tarfile + except ConfigParser.NoOptionError: + continue + return data, hwpack_with_data + + def get_format(self): + format = None + supported_formats = [self.FORMAT_1, self.FORMAT_2] + for hwpack_tarfile in self.hwpack_tarfiles: + format_file = hwpack_tarfile.extractfile(self.format_filename) + format_string = format_file.read().strip() + if not format_string in supported_formats: + raise AssertionError( + "Format version '%s' is not supported." % \ + format_string) + if format is None: + format = format_string + elif format != format_string: + return self.FORMAT_MIXED + return format + + def get_file(self, file_alias): + file_name, hwpack_tarfile = self.get_field(self.main_section, + file_alias) + if file_name is not None: + hwpack_tarfile.extract(file_name, self.tempdir) + file_name = os.path.join(self.tempdir, file_name) + return file_name + + class BoardConfig(object): """The configuration used when building an image for a board.""" # These attributes may not need to be redefined on some subclasses. @@ -138,12 +226,17 @@ mmc_option = '0:1' mmc_part_offset = 0 fat_size = 32 - extra_serial_opts = '' - live_serial_opts = '' + _extra_serial_opts = '' + _live_serial_opts = '' extra_boot_args_options = None supports_writing_to_mmc = True + LOADER_MIN_SIZE_S = align_up(1 * 1024**2, SECTOR_SIZE) / SECTOR_SIZE + BOOT_MIN_SIZE_S = align_up(50 * 1024**2, SECTOR_SIZE) / SECTOR_SIZE + ROOT_MIN_SIZE_S = align_up(50 * 1024**2, SECTOR_SIZE) / SECTOR_SIZE - # These attributes must be defined on all subclasses. + # These attributes must be defined on all subclasses for backwards + # compatibility with hwpacks v1 format. Hwpacks v2 format allows these to + # be specified in the hwpack metadata. kernel_addr = None initrd_addr = None load_addr = None @@ -152,6 +245,88 @@ kernel_flavors = None boot_script = None serial_tty = None + wired_interfaces = None + wireless_interfaces = None + mmc_id = None + + hardwarepack_handler = None + + @classmethod + def get_metadata_field(cls, target, field_name): + """ Return the metadata value for field_name if it can be found. + """ + data, _ = cls.hardwarepack_handler.get_field( + cls.hardwarepack_handler.main_section, field_name) + return data + + @classmethod + def set_metadata(cls, hwpacks): + cls.hardwarepack_handler = HardwarepackHandler(hwpacks) + with cls.hardwarepack_handler: + if (cls.hardwarepack_handler.get_format() == + cls.hardwarepack_handler.FORMAT_1): + return + + if (cls.hardwarepack_handler.get_format() == + cls.hardwarepack_handler.FORMAT_2): + # Clear V1 defaults. + cls.kernel_addr = None + cls.initrd_addr = None + cls.load_addr = None + cls.serial_tty = None + cls.fat_size = None + cls.BOOT_MIN_SIZE_S = None + cls.ROOT_MIN_SIZE_S = None + cls.LOADER_MIN_SIZE_S = None + + # Set new values from metadata. + cls.kernel_addr = cls.get_metadata_field( + cls.kernel_addr, 'kernel_addr') + cls.initrd_addr = cls.get_metadata_field( + cls.initrd_addr, 'initrd_addr') + cls.load_addr = cls.get_metadata_field( + cls.load_addr, 'load_addr') + cls.serial_tty = cls.get_metadata_field( + cls.serial_tty, 'serial_tty') + cls.wired_interfaces = cls.get_metadata_field( + cls.wired_interfaces, 'wired_interfaces') + cls.wireless_interfaces = cls.get_metadata_field( + cls.wireless_interfaces, 'wireless_interfaces') + cls.mmc_id = cls.get_metadata_field( + cls.mmc_id, 'mmc_id') + + partition_layout = cls.get_metadata_field(cls.fat_size, 'partition_layout') + if partition_layout == 'bootfs_rootfs' or partition_layout is None: + cls.fat_size = 32 + elif partition_layout == 'bootfs16_rootfs': + cls.fat_size = 16 + else: + raise AssertionError("Unknown partition layout '%s'." % partition_layout) + + boot_min_size = cls.get_metadata_field( + cls.BOOT_MIN_SIZE_S, 'boot_min_size') + if boot_min_size is not None: + cls.BOOT_MIN_SIZE_S = align_up(int(boot_min_size) * 1024**2, + SECTOR_SIZE) / SECTOR_SIZE + root_min_size = cls.get_metadata_field( + cls.ROOT_MIN_SIZE_S, 'root_min_size') + if root_min_size is not None: + cls.ROOT_MIN_SIZE_S = align_up(int(root_min_size) * 1024**2, + SECTOR_SIZE) / SECTOR_SIZE + loader_min_size = cls.get_metadata_field( + cls.LOADER_MIN_SIZE_S, 'loader_min_size') + if loader_min_size is not None: + cls.LOADER_MIN_SIZE_S = align_up(int(loader_min_size) * 1024**2, + SECTOR_SIZE) / SECTOR_SIZE + + + @classmethod + def get_file(cls, file_alias, default=None): + file_in_hwpack = cls.hardwarepack_handler.get_file(file_alias) + if file_in_hwpack is not None: + return file_in_hwpack + else: + return default @classmethod def get_sfdisk_cmd(cls, should_align_boot_part=False): @@ -167,9 +342,6 @@ else: partition_type = '0x0E' - BOOT_MIN_SIZE_S = align_up(50 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE - ROOT_MIN_SIZE_S = align_up(50 * 1024 * 1024, SECTOR_SIZE) / SECTOR_SIZE - # align on sector 63 for compatibility with broken versions of x-loader # unless align_boot_part is set boot_align = 63 @@ -178,7 +350,7 @@ # can only start on sector 1 (sector 0 is MBR / partition table) boot_start, boot_end, boot_len = align_partition( - 1, BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) + 1, cls.BOOT_MIN_SIZE_S, boot_align, PART_ALIGN_S) # apparently OMAP3 ROMs require the vfat length to be an even number # of sectors (multiple of 1 KiB); decrease the length if it's odd, # there should still be enough room @@ -189,7 +361,7 @@ # instruct the use of all remaining space; XXX if we had some root size # config, we could do something more sensible root_start, _root_end, _root_len = align_partition( - boot_end + 1, ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) + boot_end + 1, cls.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return '%s,%s,%s,*\n%s,,,-' % ( boot_start, boot_len, partition_type, root_start) @@ -299,10 +471,12 @@ if cls.uboot_in_boot_part: assert cls.uboot_flavor is not None, ( "uboot_in_boot_part is set but not uboot_flavor") - uboot_bin = os.path.join(chroot_dir, 'usr', 'lib', 'u-boot', - cls.uboot_flavor, 'u-boot.bin') - cmd_runner.run( - ['cp', '-v', uboot_bin, boot_disk], as_root=True).wait() + with cls.hardwarepack_handler: + uboot_bin = cls.get_file('u_boot', default=os.path.join( + chroot_dir, 'usr', 'lib', 'u-boot', cls.uboot_flavor, + 'u-boot.bin')) + cmd_runner.run( + ['cp', '-v', uboot_bin, boot_disk], as_root=True).wait() cls.make_boot_files( uboot_parts_dir, is_live, is_lowmem, consoles, chroot_dir, @@ -337,9 +511,14 @@ "No kernel found matching %s for flavors %s" % ( KERNEL_GLOB, " ".join(cls.kernel_flavors))) + @classmethod + def populate_raw_partition(cls, media, boot_dir): + # Override in subclass if needed + pass + class OmapConfig(BoardConfig): - kernel_flavors = ['linaro-omap4', 'linaro-omap', 'omap4'] + kernel_flavors = ['linaro-omap4', 'linaro-lt-omap', 'linaro-omap', 'omap4'] uboot_in_boot_part = True # XXX: Here we define these things as dynamic properties because our @@ -471,8 +650,8 @@ class Ux500Config(BoardConfig): serial_tty = 'ttyAMA2' - extra_serial_opts = 'console=tty0 console=%s,115200n8' % serial_tty - live_serial_opts = 'serialtty=%s' % serial_tty + _extra_serial_opts = 'console=tty0 console=%s,115200n8' + _live_serial_opts = 'serialtty=%s' kernel_addr = '0x00100000' initrd_addr = '0x08000000' load_addr = '0x00008000' @@ -485,6 +664,14 @@ 'hwmem=48M@302M mem=152M@360M') mmc_option = '1:1' + @classproperty + def live_serial_opts(cls): + return cls._live_serial_opts % cls.serial_tty + + @classproperty + def extra_serial_opts(cls): + return cls._extra_serial_opts % cls.serial_tty + @classmethod def _make_boot_files(cls, boot_env, chroot_dir, boot_dir, boot_device_or_file, k_img_data, i_img_data, @@ -517,7 +704,7 @@ and u-boot.''' # Boot ROM looks for a boot table of contents (TOC) at 0x20000 # Actually, it first looks at address 0, but that's where l-m-c - # puts the MBR, so the boot loader skips that address. + # puts the MBR, so the boot loader skips that address. supports_writing_to_mmc = False SNOWBALL_LOADER_START_S = (128 * 1024) / SECTOR_SIZE SNOWBALL_STARTUP_FILES_CONFIG = 'startfiles.cfg' @@ -537,20 +724,20 @@ This is done since the boot rom always boots off the internal memory; there simply is no point to having a loader partition on SD card. """ - # boot ROM expects bootloader at 0x20000, which is sector 0x100 + # boot ROM expects bootloader at 0x20000, which is sector 0x100 # with the usual SECTOR_SIZE of 0x200. # (sector 0 is MBR / partition table) loader_start, loader_end, loader_len = align_partition( - SnowballEmmcConfig.SNOWBALL_LOADER_START_S, - LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) + SnowballEmmcConfig.SNOWBALL_LOADER_START_S, + cls.LOADER_MIN_SIZE_S, 1, PART_ALIGN_S) boot_start, boot_end, boot_len = align_partition( - loader_end + 1, BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) + loader_end + 1, cls.BOOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) # we ignore _root_end / _root_len and return an sfdisk command to # instruct the use of all remaining space; XXX if we had some root size # config, we could do something more sensible root_start, _root_end, _root_len = align_partition( - boot_end + 1, ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) + boot_end + 1, cls.ROOT_MIN_SIZE_S, PART_ALIGN_S, PART_ALIGN_S) return '%s,%s,0xDA\n%s,%s,0x0C,*\n%s,,,-' % ( loader_start, loader_len, boot_start, boot_len, root_start) @@ -562,15 +749,22 @@ make_uImage(cls.load_addr, k_img_data, boot_dir) boot_script_path = os.path.join(boot_dir, cls.boot_script) make_boot_script(boot_env, boot_script_path) + cls.populate_raw_partition(chroot_dir, boot_device_or_file) + + @classmethod + def populate_raw_partition(cls, chroot_dir, boot_device_or_file): + # Populate created raw partition with TOC and startup files. + config_files_path = os.path.join(chroot_dir, 'boot') _, toc_filename = tempfile.mkstemp() - atexit.register(os.unlink, toc_filename) - config_files_path = os.path.join(chroot_dir, 'boot') new_files = cls.get_file_info(config_files_path) with open(toc_filename, 'wb') as toc: cls.create_toc(toc, new_files) cls.install_snowball_boot_loader(toc_filename, new_files, boot_device_or_file, cls.SNOWBALL_LOADER_START_S) + cls.delete_file(toc_filename) + cls.delete_file(os.path.join(config_files_path, + cls.SNOWBALL_STARTUP_FILES_CONFIG)) @classmethod def install_snowball_boot_loader(cls, toc_file_name, files, @@ -584,13 +778,21 @@ for file in files: # XXX We need checks that these files do not overwrite each # other. This code assumes that offset and file sizes are ok. + filename = file['filename'] if (file['offset'] % SECTOR_SIZE) != 0: seek_bytes = start_sector * SECTOR_SIZE + file['offset'] - _dd(file['filename'], boot_device_or_file, block_size=1, + _dd(filename, boot_device_or_file, block_size=1, seek=seek_bytes) else: - seek_sectors = start_sector + file['offset']/SECTOR_SIZE - _dd(file['filename'], boot_device_or_file, seek=seek_sectors) + seek_sectors = start_sector + file['offset'] / SECTOR_SIZE + _dd(filename, boot_device_or_file, seek=seek_sectors) + cls.delete_file(filename) + + @classmethod + def delete_file(cls, file_path): + cmd = ["rm", "%s" % file_path] + proc = cmd_runner.run(cmd, as_root=True) + proc.wait() @classmethod def create_toc(cls, f, files): @@ -605,6 +807,8 @@ # i; int; load_address, # 12s; string of char; name # http://igloocommunity.org/support/index.php/ConfigPartitionOverview + assert len(file['section_name']) < 12, ( + "Section name %s too large" % file['section_name']) flags = 0 load_adress = file['align'] data = struct.pack(' # # This file is part of Linaro Image Tools. -# +# # Linaro Image Tools is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or @@ -28,6 +28,7 @@ Device, Disk, PARTITION_NORMAL, + PARTITION_EXTENDED, ) from linaro_image_tools import cmd_runner @@ -71,9 +72,8 @@ bootfs = partitions[0] system = partitions[1] cache = partitions[2] - data = partitions[4] - sdcard = partitions[5] - + data = partitions[3] + sdcard = partitions[4] print "\nFormating boot partition\n" proc = cmd_runner.run( @@ -98,6 +98,7 @@ return bootfs, system, cache, data, sdcard + # I wonder if it'd make sense to convert this into a small shim which calls # the appropriate function for the given type of device? I think it's still # small enough that there's not much benefit in doing that, but if it grows we @@ -296,18 +297,28 @@ # Here we can use parted.Device to read the partitions because we're # reading from a regular file rather than a block device. If it was a # block device we'd need root rights. + vfat_partition = None disk = Disk(Device(image_file)) partition_info = [] for partition in disk.partitions: - geometry = partition.geometry - partition_info.append((geometry.start * SECTOR_SIZE, - geometry.length * SECTOR_SIZE)) + # Will ignore any partitions before boot and of type EXTENDED + if 'boot' in partition.getFlagsAsString(): + vfat_partition = partition + geometry = partition.geometry + partition_info.append((geometry.start * SECTOR_SIZE, + geometry.length * SECTOR_SIZE)) + elif (vfat_partition is not None and + partition.type != PARTITION_EXTENDED): + geometry = partition.geometry + partition_info.append((geometry.start * SECTOR_SIZE, + geometry.length * SECTOR_SIZE)) # NB: don't use vfat_partition.nextPartition() as that might return # a partition of type PARTITION_FREESPACE; it's much easier to # iterate disk.partitions which only returns # parted.PARTITION_NORMAL partitions - - assert len(partition_info) == 6 + assert vfat_partition is not None, ( + "Couldn't find boot partition on %s" % image_file) + assert len(partition_info) == 5 return partition_info @@ -347,6 +358,7 @@ return boot_partition, system_partition, cache_partition, \ data_partition, sdcard_partition + def get_boot_and_root_partitions_for_media(media, board_config): """Return the device files for the boot and root partitions of media. === modified file 'linaro_image_tools/media_create/tests/test_media_create.py' --- linaro_image_tools/media_create/tests/test_media_create.py 2011-06-14 09:45:13 +0000 +++ linaro_image_tools/media_create/tests/test_media_create.py 2011-07-01 14:55:14 +0000 @@ -28,7 +28,10 @@ import textwrap import time import types +import struct +import tarfile +from StringIO import StringIO from testtools import TestCase from linaro_image_tools import cmd_runner @@ -38,9 +41,11 @@ boards, partitions, rootfs, + android_boards, ) from linaro_image_tools.media_create.boards import ( - LOADER_MIN_SIZE_S, + SAMSUNG_V310_BL1_START, + SAMSUNG_V310_BL2_START, SECTOR_SIZE, align_up, align_partition, @@ -56,6 +61,11 @@ _get_file_matching, _get_mlo_file, _run_mkimage, + HardwarepackHandler, + BoardConfig, + ) +from linaro_image_tools.media_create.android_boards import ( + android_board_configs, ) from linaro_image_tools.media_create.chroot_utils import ( copy_file, @@ -113,6 +123,359 @@ sudo_args = " ".join(cmd_runner.SUDO_ARGS) +class TestHardwarepackHandler(TestCaseWithFixtures): + def setUp(self): + super(TestHardwarepackHandler, self).setUp() + self.tar_dir_fixture = CreateTempDirFixture() + self.useFixture(self.tar_dir_fixture) + + self.tarball_fixture = CreateTarballFixture( + self.tar_dir_fixture.get_temp_dir()) + self.useFixture(self.tarball_fixture) + + self.metadata = ( + "NAME=ahwpack\nVERSION=4\nARCHITECTURE=armel\nORIGIN=linaro\n") + + def add_to_tarball(self, files, tarball=None): + if tarball is None: + tarball = self.tarball_fixture.get_tarball() + tar_file = tarfile.open(tarball, mode='w:gz') + for filename, data in files: + tarinfo = tarfile.TarInfo(filename) + tarinfo.size = len(data) + tar_file.addfile(tarinfo, StringIO(data)) + tar_file.close() + return tarball + + def test_get_format_1(self): + data = '1.0' + format = "%s\n" % data + tarball = self.add_to_tarball( + [('FORMAT', format), ('metadata', self.metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + self.assertEquals(hp.get_format(), data) + + def test_get_format_2(self): + data = '2.0' + format = "%s\n" % data + tarball = self.add_to_tarball( + [('FORMAT', format), ('metadata', self.metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + self.assertEquals(hp.get_format(), data) + + def test_get_unknown_format_raises(self): + data = '9.9' + format = "%s\n" % data + tarball = self.add_to_tarball( + [('FORMAT', format), ('metadata', self.metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + self.assertRaises(AssertionError, hp.get_format) + + def test_mixed_formats(self): + format1 = "%s\n" % '1.0' + format2 = "%s\n" % '2.0' + tarball1 = self.add_to_tarball( + [('FORMAT', format1), ('metadata', self.metadata)], + tarball=self.tarball_fixture.get_tarball()) + tarball_fixture2 = CreateTarballFixture( + self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', + filename='secondtarball.tar.gz') + self.useFixture(tarball_fixture2) + tarball2 = self.add_to_tarball( + [('FORMAT', format2), ('metadata', self.metadata)], + tarball=tarball_fixture2.get_tarball()) + hp = HardwarepackHandler([tarball2, tarball1]) + with hp: + self.assertEquals(hp.get_format(), '1.0and2.0') + + def test_identical_formats_ok(self): + format1 = "%s\n" % '2.0' + format2 = "%s\n" % '2.0' + tarball1 = self.add_to_tarball( + [('FORMAT', format1), ('metadata', self.metadata)], + tarball=self.tarball_fixture.get_tarball()) + tarball_fixture2 = CreateTarballFixture( + self.tar_dir_fixture.get_temp_dir(), reldir='tarfile2', + filename='secondtarball.tar.gz') + self.useFixture(tarball_fixture2) + tarball2 = self.add_to_tarball( + [('FORMAT', format2), ('metadata', self.metadata)], + tarball=tarball_fixture2.get_tarball()) + hp = HardwarepackHandler([tarball1, tarball2]) + with hp: + self.assertEquals(hp.get_format(), '2.0') + + def test_get_metadata(self): + data = 'data to test' + metadata = self.metadata + "TEST=%s\n" % data + tarball = self.add_to_tarball( + [('metadata', metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + test_data, _ = hp.get_field(hp.main_section, 'test') + self.assertEqual(test_data, data) + + def test_preserves_formatters(self): + data = '%s%d' + metadata = self.metadata + "TEST=%s\n" % data + tarball = self.add_to_tarball( + [('metadata', metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + test_data, _ = hp.get_field(hp.main_section, 'test') + self.assertEqual(test_data, data) + + def test_creates_tempdir(self): + tarball = self.add_to_tarball( + [('metadata', self.metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + self.assertTrue(os.path.exists(hp.tempdir)) + + def test_tempfiles_are_removed(self): + tempdir = None + tarball = self.add_to_tarball( + [('metadata', self.metadata)]) + hp = HardwarepackHandler([tarball]) + with hp: + tempdir = hp.tempdir + self.assertFalse(os.path.exists(tempdir)) + + def test_get_file(self): + data = 'test file contents\n' + metadata_file = 'TESTFILE' + file_in_archive = 'testfile' + metadata = self.metadata + "%s=%s\n" % (metadata_file, file_in_archive) + tarball = self.add_to_tarball( + [('metadata', metadata), + (file_in_archive, data)]) + hp = HardwarepackHandler([tarball]) + with hp: + test_file = hp.get_file(metadata_file) + self.assertEquals(data, open(test_file, 'r').read()) + + +class TestSetMetadata(TestCaseWithFixtures): + + class MockHardwarepackHandler(HardwarepackHandler): + metadata_dict = {} + + def __enter__(self): + return self + + def get_field(self, section, field): + try: + return self.metadata_dict[field], None + except: + return None, None + + def get_format(self): + return '2.0' + + def get_file(self, file_alias): + return None + + def test_does_not_set_if_old_format(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(None, config.kernel_addr) + + def test_sets_kernel_addr(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'kernel_addr' + data_to_set = '0x8123ABCD' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.kernel_addr) + + def test_sets_initrd_addr(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'initrd_addr' + data_to_set = '0x8123ABCD' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.initrd_addr) + + def test_sets_load_addr(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'load_addr' + data_to_set = '0x8123ABCD' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.load_addr) + + def test_sets_serial_tty(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'serial_tty' + data_to_set = 'ttyAA' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.serial_tty) + + def test_sets_wired_interfaces(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'wired_interfaces' + data_to_set = 'eth0 eth1' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.wired_interfaces) + + def test_sets_wireless_interfaces(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'wireless_interfaces' + data_to_set = 'wlan0 wl1' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.wireless_interfaces) + + def test_sets_mmc_id(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'mmc_id' + data_to_set = '1' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(data_to_set, config.mmc_id) + + def test_sets_boot_min_size(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'boot_min_size' + data_to_set = '100' + expected = align_up(int(data_to_set) * 1024 * 1024, + SECTOR_SIZE) / SECTOR_SIZE + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(expected, config.BOOT_MIN_SIZE_S) + + def test_sets_root_min_size(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'root_min_size' + data_to_set = '3' + expected = align_up(int(data_to_set) * 1024 * 1024, + SECTOR_SIZE) / SECTOR_SIZE + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(expected, config.ROOT_MIN_SIZE_S) + + def test_sets_loader_min_size(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'loader_min_size' + data_to_set = '2' + expected = align_up(int(data_to_set) * 1024 * 1024, + SECTOR_SIZE) / SECTOR_SIZE + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(expected, config.LOADER_MIN_SIZE_S) + + def test_sets_partition_layout_32(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'partition_layout' + data_to_set = 'bootfs_rootfs' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(32, config.fat_size) + + def test_sets_partition_layout_16(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'partition_layout' + data_to_set = 'bootfs16_rootfs' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + config.set_metadata('ahwpack.tar.gz') + self.assertEquals(16, config.fat_size) + + def test_sets_partition_layout_raises(self): + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards, 'HardwarepackHandler', + self.MockHardwarepackHandler)) + field_to_test = 'partition_layout' + data_to_set = 'bootfs_bogus_rootfs' + self.MockHardwarepackHandler.metadata_dict = { + field_to_test: data_to_set, + } + class config(BoardConfig): + pass + self.assertRaises(AssertionError, config.set_metadata, 'ahwpack.tar.gz') + + class TestGetMLOFile(TestCaseWithFixtures): def test_mlo_from_new_xloader(self): @@ -150,6 +513,309 @@ AssertionError, _get_mlo_file, tempdir) +class TestGetSMDKSPL(TestCaseWithFixtures): + + def test_no_file_present(self): + tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + uboot_flavor = "some_uboot_flavour" + self.assertRaises( + AssertionError, boards.SMDKV310Config._get_smdk_spl, tempdir, + uboot_flavor) + + def test_old_file_present(self): + tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + uboot_flavor = "some_uboot_flavour" + path = os.path.join(tempdir, 'usr', 'lib', 'u-boot', uboot_flavor) + os.makedirs(path) + spl_path = os.path.join(path, 'v310_mmc_spl.bin') + open(spl_path, 'w').close() + self.assertEquals(spl_path, boards.SMDKV310Config._get_smdk_spl( + tempdir, uboot_flavor)) + + def test_new_file_present(self): + tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + uboot_flavor = "some_uboot_flavour" + path = os.path.join(tempdir, 'usr', 'lib', 'u-boot', uboot_flavor) + os.makedirs(path) + spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') + open(spl_path, 'w').close() + self.assertEquals(spl_path, boards.SMDKV310Config._get_smdk_spl( + tempdir, uboot_flavor)) + + def test_prefers_old_path(self): + tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + uboot_flavor = "some_uboot_flavour" + path = os.path.join(tempdir, 'usr', 'lib', 'u-boot', uboot_flavor) + os.makedirs(path) + old_spl_path = os.path.join(path, 'v310_mmc_spl.bin') + new_spl_path = os.path.join(path, 'u-boot-mmc-spl.bin') + open(old_spl_path, 'w').close() + open(new_spl_path, 'w').close() + self.assertEquals(old_spl_path, boards.SMDKV310Config._get_smdk_spl( + tempdir, uboot_flavor)) + + +class TestGetSMDKUboot(TestCaseWithFixtures): + + def test_uses_uboot_flavour(self): + chroot_dir = "chroot" + uboot_flavor = "some_uboot_flavour" + uboot_file = os.path.join(chroot_dir, 'usr', 'lib', 'u-boot', uboot_flavor, + 'u-boot.bin') + self.assertEquals(uboot_file, boards.SMDKV310Config._get_smdk_uboot( + chroot_dir, uboot_flavor)) + + +class TestCreateToc(TestCaseWithFixtures): + ''' Tests boards.SnowballEmmcConfig.create_toc()''' + + def setUp(self): + ''' Create a temporary directory to work in''' + super(TestCreateToc, self).setUp() + self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + #Create the test's input data structures + zero = '\x00\x00\x00\x00' + line1 = zero + zero + zero + zero + zero + 'b' + zero + zero + \ + '\x00\x00\x00' + maxint = '\xFF\xFF\xFF\x7F' + minint = '\xFF\xFF\xFF\xFF' + line2 = maxint + maxint + zero + minint + minint + \ + 'hello' + zero + '\x00\x00\x00' + line3 = '\x01\x00\x00\x00' '\x64\x00\x00\x00' + zero + \ + '\x05\x00\x00\x00' '\x05\x00\x00\x00' \ + 'hello' + zero + '\x00\x00\x00' + self.expected = line1 + line2 + line3 + + def create_files_structure(self, src_data): + ''' Creates the data structure that the tested function + needs as input''' + files = [] + for line in src_data: + files.append({'section_name': line[5], + 'filename': 'N/A', + 'align': line[3], + 'offset': line[0], + 'size': line[1], + 'load_adress': 'N/A'}) + return files + + def test_create_toc_normal_case(self): + ''' Creates a toc file, and then reads the created + file and compares it to precomputed data''' + correct_data = [(0, 0, 0, 0, 0, 'b'), + (0x7FFFFFFF, 0x7FFFFFFF, 0x7FFFFFFF, -1, -1, 'hello'), + (1, 100, 1000, 5, 10, 'hello')] + files = self.create_files_structure(correct_data) + filename = os.path.join(self.tempdir, 'toc') + with open(filename, 'w') as f: + boards.SnowballEmmcConfig.create_toc(f, files) + with open(filename, 'r') as f: + actual = f.read() + self.assertEquals(96, len(actual)) + for i in range(len(actual)): + self.assertEquals(self.expected[i], actual[i], 'Mismatch at ix' \ + ' %d, ref=%c, actual=%c' % (i, self.expected[i], actual[i])) + + def test_create_toc_error_too_large_section_name(self): + '''Verify that trying to write past the end of the + section name field raises an exception''' + illegal_name_data = [(0, 0, 0, 0, 0, 'Too_longName')] + files = self.create_files_structure(illegal_name_data) + with open(os.path.join(self.tempdir, 'toc'), 'w') as f: + self.assertRaises(AssertionError, + boards.SnowballEmmcConfig.create_toc, + f, files) + + def test_create_toc_error_negative_unsigned(self): + '''Verify that trying to write a negative number to an unsigned + field raises an exception''' + illegal_unsigned_data = [(-3, 0, 0, 0, 0, 'xxx')] + files = self.create_files_structure(illegal_unsigned_data) + with open(os.path.join(self.tempdir, 'toc'), 'w') as f: + self.assertRaises(struct.error, + boards.SnowballEmmcConfig.create_toc, + f, files) + + +class TestSnowballBootFiles(TestCaseWithFixtures): + ''' Tests boards.SnowballEmmcConfig.install_snowball_boot_loader()''' + ''' Tests boards.SnowballEmmcConfig._make_boot_files()''' + ''' Tests boards.SnowballEmmcConfig.get_file_info()''' + + def setUp(self): + ''' Create temporary directory to work in''' + super(TestSnowballBootFiles, self).setUp() + self.tempdir = self.useFixture(CreateTempDirFixture()).get_temp_dir() + self.temp_bootdir_path = os.path.join(self.tempdir, 'boot') + if not os.path.exists(self.temp_bootdir_path): + os.makedirs(self.temp_bootdir_path) + + def setupFiles(self): + ''' Adds some files in the temp dir that the tested function + can use as input: + * A config file, which the tested function reads to + discover which binary files should be written to + the loader partition. + * Test versions of the binary files themselves, + containing dummy data. + Returns the expected value that the tested function should + return, given these input files. ''' + src_data = [('ISSW', 'boot_image_issw.bin', -1, 0, '5'), + ('X-LOADER', 'boot_image_x-loader.bin', -1, 0, '6'), + ('MEM_INIT', 'mem_init.bin', 0, 0x160000, '7'), + ('PWR_MGT', 'power_management.bin', 0, 0x170000, '8'), + ('NORMAL', 'u-boot.bin', 0, 0xBA0000, '9'), + ('UBOOT_ENV', 'u-boot-env.bin', 0, 0x00C1F000, '10')] + # Create a config file + cfg_file = os.path.join(self.temp_bootdir_path, + boards.SnowballEmmcConfig.SNOWBALL_STARTUP_FILES_CONFIG) + with open(cfg_file, 'w') as f: + for line in src_data: + # Write comments, so we test that the parser can read them + f.write('#Yet another comment\n') + f.write('%s %s %i %#x %s\n' % line) + expected = [] + # Define dummy binary files, containing nothing but their own + # section names. + for line in src_data: + with open(os.path.join(self.temp_bootdir_path, line[1]), 'w') as f: + f.write(line[0]) + #define the expected values read from the config file + expected = [] + ofs = [boards.SnowballEmmcConfig.TOC_SIZE, + boards.SnowballEmmcConfig.TOC_SIZE + len('ISSW'), 0x160000, + 0x170000, 0xBA0000, 0xC1F000] + size = [len('ISSW'), len('X-LOADER'), len('MEM_INIT'), \ + len('PWR_MGT'), len('NORMAL'), len('UBOOT_ENV')] + i = 0 + for line in src_data: + filename = os.path.join(self.temp_bootdir_path, line[1]) + expected.append({'section_name': line[0], + 'filename': filename, + 'align': int(line[2]), + 'offset': ofs[i], + 'size': long(size[i]), + 'load_adress': line[4]}) + i += 1 + return expected + + def test_file_name_size(self): + ''' Test using a to large toc file ''' + _, toc_filename = tempfile.mkstemp() + atexit.register(os.unlink, toc_filename) + filedata = 'X' + bytes = boards.SnowballEmmcConfig.TOC_SIZE + 1 + tmpfile = open(toc_filename, 'wb') + for n in xrange(bytes): + tmpfile.write(filedata) + tmpfile.close() + files = self.setupFiles() + self.assertRaises(AssertionError, + boards.SnowballEmmcConfig.install_snowball_boot_loader, + toc_filename, files, "boot_device_or_file", + boards.SnowballEmmcConfig.SNOWBALL_LOADER_START_S) + + def test_install_snowball_boot_loader_toc(self): + fixture = self.useFixture(MockCmdRunnerPopenFixture()) + toc_filename = self.createTempFileAsFixture() + files = self.setupFiles() + boards.SnowballEmmcConfig.install_snowball_boot_loader(toc_filename, + files, "boot_device_or_file", + boards.SnowballEmmcConfig.SNOWBALL_LOADER_START_S) + expected = [ + '%s dd if=%s of=boot_device_or_file bs=512 conv=notrunc' \ + ' seek=%s' % (sudo_args, toc_filename, + boards.SnowballEmmcConfig.SNOWBALL_LOADER_START_S), + '%s dd if=%s/boot_image_issw.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=257' % (sudo_args, self.temp_bootdir_path), + '%s rm %s/boot_image_issw.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/boot_image_x-loader.bin of=boot_device_or_file' \ + ' bs=1 conv=notrunc seek=131588' + % (sudo_args, self.temp_bootdir_path), + '%s rm %s/boot_image_x-loader.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/mem_init.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=3072' % (sudo_args, self.temp_bootdir_path), + '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), + '%s dd if=%s/power_management.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=3200' % (sudo_args, self.temp_bootdir_path), + '%s rm %s/power_management.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/u-boot.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=24064' % (sudo_args, self.temp_bootdir_path), + '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), + '%s dd if=%s/u-boot-env.bin of=boot_device_or_file bs=512' + ' conv=notrunc seek=25080' % (sudo_args, self.temp_bootdir_path), + '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path)] + + self.assertEqual(expected, fixture.mock.commands_executed) + + def test_snowball_make_boot_files(self): + fixture = self.useFixture(MockCmdRunnerPopenFixture()) + self.useFixture(MockSomethingFixture(tempfile, 'mkstemp', + lambda: (-1, '/tmp/temp_snowball_make_boot_files'))) + self.setupFiles() + k_img_file = os.path.join(self.tempdir, 'vmlinuz-1-ux500') + i_img_file = os.path.join(self.tempdir, 'initrd.img-1-ux500') + + boot_env = board_configs['snowball_emmc']._get_boot_env( + is_live=False, is_lowmem=False, consoles=[], + rootfs_uuid="test_boot_env_uuid", d_img_data=None) + boards.SnowballEmmcConfig._make_boot_files(boot_env, self.tempdir, + self.temp_bootdir_path, 'boot_device_or_file', k_img_file, + i_img_file, None) + expected = [ + '%s mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e' \ + ' 0x00008000 -n Linux -d %s %s/boot/uImage' % (sudo_args, + k_img_file, self.tempdir), + '%s cp /tmp/temp_snowball_make_boot_files %s/boot/boot.txt' + % (sudo_args, self.tempdir), + '%s mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n boot' \ + ' script -d %s/boot/boot.txt %s/boot/flash.scr' + % (sudo_args, self.tempdir, self.tempdir), + '%s dd if=/tmp/temp_snowball_make_boot_files' \ + ' of=boot_device_or_file bs=512 conv=notrunc seek=256' + % (sudo_args), + '%s dd if=%s/boot/boot_image_issw.bin of=boot_device_or_file' \ + ' bs=512 conv=notrunc seek=257' % (sudo_args, self.tempdir), + '%s rm %s/boot_image_issw.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/boot/boot_image_x-loader.bin of=boot_device_or_file' \ + ' bs=1 conv=notrunc seek=131588' % (sudo_args, self.tempdir), + '%s rm %s/boot_image_x-loader.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/boot/mem_init.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=3072' % (sudo_args, self.tempdir), + '%s rm %s/mem_init.bin' % (sudo_args, self.temp_bootdir_path), + '%s dd if=%s/boot/power_management.bin of=boot_device_or_file' \ + ' bs=512 conv=notrunc seek=3200' % (sudo_args, self.tempdir), + '%s rm %s/power_management.bin' % (sudo_args, + self.temp_bootdir_path), + '%s dd if=%s/boot/u-boot.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=24064' % (sudo_args, self.tempdir), + '%s rm %s/u-boot.bin' % (sudo_args, self.temp_bootdir_path), + '%s dd if=%s/boot/u-boot-env.bin of=boot_device_or_file bs=512' \ + ' conv=notrunc seek=25080' % (sudo_args, self.tempdir), + '%s rm %s/u-boot-env.bin' % (sudo_args, self.temp_bootdir_path), + '%s rm /tmp/temp_snowball_make_boot_files' % (sudo_args), + '%s rm %s/startfiles.cfg' % (sudo_args, self.temp_bootdir_path)] + + self.assertEqual(expected, fixture.mock.commands_executed) + + def test_missing_files(self): + '''When the files cannot be read, an IOError should be raised''' + self.assertRaises(IOError, + boards.SnowballEmmcConfig.get_file_info, + self.tempdir) + + def test_normal_case(self): + expected = self.setupFiles() + actual = boards.SnowballEmmcConfig.get_file_info( + self.temp_bootdir_path) + self.assertEquals(expected, actual) + + class TestBootSteps(TestCaseWithFixtures): def setUp(self): @@ -199,6 +865,10 @@ def test_mx5_steps(self): class SomeMx5Config(boards.Mx5Config): uboot_flavor = 'uboot_flavor' + SomeMx5Config.hardwarepack_handler = ( + TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) + SomeMx5Config.hardwarepack_handler.get_format = ( + lambda: '1.0') self.make_boot_files(SomeMx5Config) expected = [ 'install_mx5_boot_loader', 'make_uImage', 'make_uInitrd', @@ -206,9 +876,21 @@ self.assertEqual(expected, self.funcs_calls) def test_smdkv310_steps(self): + def mock_func_creator(name): + return classmethod( + lambda *args, **kwargs: self.funcs_calls.append(name)) + + self.useFixture(MockSomethingFixture( + linaro_image_tools.media_create.boards.SMDKV310Config, + 'install_smdk_boot_loader', + mock_func_creator('install_smdk_boot_loader'))) + boards.SMDKV310Config.hardwarepack_handler = ( + TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) + boards.SMDKV310Config.hardwarepack_handler.get_format = ( + lambda: '1.0') self.make_boot_files(boards.SMDKV310Config) expected = [ - '_dd', '_dd', 'make_flashable_env', '_dd', 'make_uImage', + 'install_smdk_boot_loader', 'make_flashable_env', '_dd', 'make_uImage', 'make_uInitrd', 'make_boot_script'] self.assertEqual(expected, self.funcs_calls) @@ -354,6 +1036,18 @@ '1,8191,0xDA\n8192,106496,0x0C,*\n114688,,,-', board_configs['smdkv310'].get_sfdisk_cmd()) + def test_panda_android(self): + self.assertEqual( + '63,270272,0x0C,*\n270336,524288,L\n794624,524288,L\n' \ + '1318912,-,E\n1318912,1048576,L\n2367488,,,-', + android_boards.AndroidPandaConfig.get_sfdisk_cmd()) + + def test_snowball_emmc_android(self): + self.assertEqual( + '256,7936,0xDA\n8192,262144,0x0C,*\n270336,524288,L\n' \ + '794624,-,E\n794624,524288,L\n1318912,1048576,L\n2367488,,,-', + android_boards.AndroidSnowballEmmcConfig.get_sfdisk_cmd()) + class TestGetBootCmd(TestCase): @@ -509,6 +1203,41 @@ self.assertEqual(expected, boot_commands) +class TestGetBootCmdAndroid(TestCase): + def test_panda(self): + # XXX: To fix bug 697824 we have to change class attributes of our + # OMAP board configs, and some tests do that so to make sure they + # don't interfere with us we'll reset that before doing anything. + config = android_board_configs['panda'] + config.serial_tty = config._serial_tty + boot_commands = config._get_boot_env(consoles=[]) + expected = { + 'bootargs': 'console=tty0 console=ttyO2,115200n8 ' + 'rootwait ro earlyprintk fixrtc ' + 'nocompcache vram=48M omapfb.vram=0:24M,1:24M ' + 'mem=456M@0x80000000 mem=512M@0xA0000000 ' + 'init=/init androidboot.console=ttyO2', + 'bootcmd': 'fatload mmc 0:1 0x80200000 uImage; ' + 'fatload mmc 0:1 0x81600000 uInitrd; ' + 'bootm 0x80200000 0x81600000'} + self.assertEqual(expected, boot_commands) + + def test_android_snowball_emmc(self): + boot_commands = (android_boards.AndroidSnowballEmmcConfig. + _get_boot_env(consoles=[])) + expected = { + 'bootargs': 'console=tty0 console=ttyAMA2,115200n8 ' + 'rootwait ro earlyprintk ' + 'rootdelay=1 fixrtc nocompcache ' + 'mem=128M@0 mali.mali_mem=64M@128M mem=24M@192M ' + 'hwmem=167M@216M mem_issw=1M@383M mem=640M@384M ' + 'vmalloc=256M init=/init androidboot.console=ttyAMA2', + 'bootcmd': 'fatload mmc 1:1 0x00100000 uImage; ' + 'fatload mmc 1:1 0x08000000 uInitrd; ' + 'bootm 0x00100000 0x08000000'} + self.assertEqual(expected, boot_commands) + + class TestUnpackBinaryTarball(TestCaseWithFixtures): def setUp(self): @@ -607,7 +1336,8 @@ def test_install_mx5_boot_loader(self): fixture = self._mock_Popen() imx_file = self.createTempFileAsFixture() - install_mx5_boot_loader(imx_file, "boot_device_or_file") + install_mx5_boot_loader(imx_file, "boot_device_or_file", + BoardConfig.LOADER_MIN_SIZE_S) expected = [ '%s dd if=%s of=boot_device_or_file bs=512 ' 'conv=notrunc seek=2' % (sudo_args, imx_file)] @@ -616,9 +1346,10 @@ def test_install_mx5_boot_loader_too_large(self): self.useFixture(MockSomethingFixture( os.path, "getsize", - lambda s: (LOADER_MIN_SIZE_S - 1) * SECTOR_SIZE + 1)) + lambda s: (BoardConfig.LOADER_MIN_SIZE_S - 1) * SECTOR_SIZE + 1)) self.assertRaises(AssertionError, - install_mx5_boot_loader, "imx_file", "boot_device_or_file") + install_mx5_boot_loader, "imx_file", "boot_device_or_file", + BoardConfig.LOADER_MIN_SIZE_S) def test_install_omap_boot_loader(self): fixture = self._mock_Popen() @@ -630,6 +1361,32 @@ '%s cp -v chroot_dir/MLO boot_disk' % sudo_args, 'sync'] self.assertEqual(expected, fixture.mock.commands_executed) + def test_install_smdk_u_boot(self): + fixture = self._mock_Popen() + uboot_flavor = "some_u_boot_flavour" + self.useFixture(MockSomethingFixture( + boards.SMDKV310Config, '_get_smdk_spl', + classmethod(lambda cls, chroot_dir, uboot_flavor: "%s/%s/SPL" % ( + chroot_dir, uboot_flavor)))) + self.useFixture(MockSomethingFixture( + boards.SMDKV310Config, '_get_smdk_uboot', + classmethod(lambda cls, chroot_dir, uboot_flavor: "%s/%s/uboot" % ( + chroot_dir, uboot_flavor)))) + boards.SMDKV310Config.hardwarepack_handler = ( + TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz')) + boards.SMDKV310Config.hardwarepack_handler.get_format = ( + lambda: '1.0') + self.useFixture(MockSomethingFixture(os.path, 'getsize', + lambda file: 1)) + boards.SMDKV310Config.install_smdk_boot_loader( + "chroot_dir", "boot_disk", uboot_flavor) + expected = [ + '%s dd if=chroot_dir/%s/SPL of=boot_disk bs=512 conv=notrunc ' + 'seek=%d' % (sudo_args, uboot_flavor, SAMSUNG_V310_BL1_START), + '%s dd if=chroot_dir/%s/uboot of=boot_disk bs=512 conv=notrunc ' + 'seek=%d' % (sudo_args, uboot_flavor, SAMSUNG_V310_BL2_START)] + self.assertEqual(expected, fixture.mock.commands_executed) + def test_get_plain_boot_script_contents(self): boot_env = {'bootargs': 'mybootargs', 'bootcmd': 'mybootcmd'} boot_script_data = get_plain_boot_script_contents(boot_env) @@ -893,13 +1650,21 @@ (63 * SECTOR_SIZE, 32768 * SECTOR_SIZE), (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), (98367 * SECTOR_SIZE, 65536 * SECTOR_SIZE), - (294975 * SECTOR_SIZE, (self.android_image_size - - 294975 * SECTOR_SIZE)), ((294975 + ext_part_size) * SECTOR_SIZE, (131072 - ext_part_size) * SECTOR_SIZE), ((426047 + ext_part_size) * SECTOR_SIZE, self.android_image_size - (426047 + ext_part_size) * SECTOR_SIZE) ] + + self.android_snowball_offsets_and_sizes = [ + (8192 * SECTOR_SIZE, 24639 * SECTOR_SIZE), + (32831 * SECTOR_SIZE, 65536 * SECTOR_SIZE), + ((98367 + ext_part_size)* SECTOR_SIZE, + (65536 - ext_part_size) * SECTOR_SIZE), + (294975 * SECTOR_SIZE, 131072 * SECTOR_SIZE), + ((426047 + ext_part_size) * SECTOR_SIZE, + self.android_image_size - (426047 + ext_part_size) * SECTOR_SIZE) + ] def tearDown(self): super(TestPartitionSetup, self).tearDown() @@ -916,6 +1681,13 @@ '63,32768,0x0C,*\n32831,65536,L\n98367,65536,L\n294975,-,E\n' \ '294975,131072,L\n426047,,,-', '%s' % self.android_image_size) + def _create_snowball_android_tmpfile(self): + # raw, boot, system, cache, (extended), userdata and sdcard partitions + return self._create_qemu_img_with_partitions( + '256,7936,0xDA\n8192,24639,0x0C,*\n32831,65536,L\n' \ + '98367,-,E\n98367,65536,L\n294975,131072,L\n' \ + '426047,,,-', '%s' % self.android_image_size) + def test_convert_size_no_suffix(self): self.assertEqual(524288, convert_size_to_bytes('524288')) @@ -945,6 +1717,15 @@ self.android_offsets_and_sizes): self.assertEqual(device_pair, expected_pair) + def test_calculate_snowball_android_partition_size_and_offset(self): + tmpfile = self._create_snowball_android_tmpfile() + device_info = calculate_android_partition_size_and_offset(tmpfile) + # We use map(None, ...) since it would catch if the lists are not of + # equal length and zip() would not in all cases. + for device_pair, expected_pair in map(None, device_info, + self.android_snowball_offsets_and_sizes): + self.assertEqual(device_pair, expected_pair) + def test_partition_numbering(self): # another Linux partition at +24 MiB after the boot/root parts tmpfile = self._create_qemu_img_with_partitions( @@ -1054,7 +1835,7 @@ # get_boot_and_root_loopback_devices will also setup two exit handlers # to de-register the loopback devices set up above. - self.assertEqual(6, len(atexit_fixture.mock.funcs)) + self.assertEqual(5, len(atexit_fixture.mock.funcs)) popen_fixture.mock.calls = [] atexit_fixture.mock.run_funcs() # We did not really run losetup above (as it requires root) so here we @@ -1065,7 +1846,6 @@ '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args, - '%s losetup -d ' % sudo_args, '%s losetup -d ' % sudo_args], popen_fixture.mock.commands_executed) @@ -1159,6 +1939,9 @@ self.config = c self.config.boot_script = 'boot_script' + self.config.hardwarepack_handler = \ + TestSetMetadata.MockHardwarepackHandler('ahwpack.tar.gz') + self.config.hardwarepack_handler.get_format = lambda: '1.0' self.popen_fixture = self.useFixture(MockCmdRunnerPopenFixture()) self.useFixture(MockSomethingFixture( self.config, 'make_boot_files', self.save_args)) === modified file 'linaro_image_tools/tests/fixtures.py' --- linaro_image_tools/tests/fixtures.py 2011-04-04 11:29:01 +0000 +++ linaro_image_tools/tests/fixtures.py 2011-06-20 09:10:10 +0000 @@ -20,6 +20,7 @@ import os import shutil import tempfile +from StringIO import StringIO from linaro_image_tools import cmd_runner @@ -68,8 +69,11 @@ # used in tests to make sure all callsites wait for their child. child_finished = True + def __init__(self, assert_child_finished=True): + self.assert_child_finished = assert_child_finished + def __call__(self, cmd, *args, **kwargs): - if not self.child_finished: + if self.assert_child_finished and not self.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") @@ -97,6 +101,9 @@ def commands_executed(self): return [' '.join(args) for args in self.calls] + @property + def stdin(self): + return StringIO() class MockCmdRunnerPopenFixture(MockSomethingFixture): """A test fixture which mocks cmd_runner.do_run with the given mock. @@ -104,13 +111,13 @@ If no mock is given, a MockCmdRunnerPopen instance is used. """ - def __init__(self): + def __init__(self, assert_child_finished=True): super(MockCmdRunnerPopenFixture, self).__init__( - cmd_runner, 'Popen', MockCmdRunnerPopen()) + cmd_runner, 'Popen', MockCmdRunnerPopen(assert_child_finished)) def tearDown(self): super(MockCmdRunnerPopenFixture, self).tearDown() - if not self.mock.child_finished: + if self.mock.assert_child_finished and not self.mock.child_finished: raise AssertionError( "You should call wait() or communicate() to ensure " "the subprocess is finished before proceeding.") === modified file 'linaro_image_tools/tests/test_pyflakes.py' --- linaro_image_tools/tests/test_pyflakes.py 2011-05-10 15:43:32 +0000 +++ linaro_image_tools/tests/test_pyflakes.py 2011-06-21 09:18:32 +0000 @@ -29,8 +29,8 @@ (stdout, stderr) = proc.communicate() stdout = stdout.splitlines() stdout.sort() - expected = ["./linaro_image_tools/utils.py:26: redefinition of " - "unused 'CommandNotFound' from line 24" ] + expected = ["./linaro_image_tools/utils.py:27: redefinition of " + "unused 'CommandNotFound' from line 25" ] self.assertEquals(expected, stdout) self.assertEquals('', stderr) === modified file 'linaro_image_tools/tests/test_utils.py' --- linaro_image_tools/tests/test_utils.py 2011-03-24 18:41:59 +0000 +++ linaro_image_tools/tests/test_utils.py 2011-06-21 09:16:28 +0000 @@ -35,12 +35,49 @@ install_package_providing, preferred_tools_dir, UnableToFindPackageProvidingCommand, + verify_file_integrity, ) sudo_args = " ".join(cmd_runner.SUDO_ARGS) +class TestVerifyFileIntegrity(TestCaseWithFixtures): + + filenames_in_shafile = ['verified-file1', 'verified-file2'] + + class MockCmdRunnerPopen(object): + def __call__(self, cmd, *args, **kwargs): + self.returncode = 0 + return self + + def communicate(self, input=None): + self.wait() + return ': OK\n'.join( + TestVerifyFileIntegrity.filenames_in_shafile) + ': OK\n', '' + + def wait(self): + return self.returncode + + def test_verify_files(self): + fixture = self.useFixture(MockCmdRunnerPopenFixture()) + hash_filename = "dummy-file.txt" + signature_filename = hash_filename + ".asc" + verify_file_integrity([signature_filename]) + self.assertEqual( + ['gpg --verify %s' % signature_filename, + 'sha1sum -c %s' % hash_filename], + fixture.mock.commands_executed) + + def test_verify_files_returns_files(self): + self.useFixture(MockSomethingFixture(cmd_runner, 'Popen', + self.MockCmdRunnerPopen())) + hash_filename = "dummy-file.txt" + signature_filename = hash_filename + ".asc" + verified_files = verify_file_integrity([signature_filename]) + self.assertEqual(self.filenames_in_shafile, verified_files) + + class TestEnsureCommand(TestCaseWithFixtures): install_pkg_providing_called = False === modified file 'linaro_image_tools/utils.py' --- linaro_image_tools/utils.py 2011-05-10 15:42:59 +0000 +++ linaro_image_tools/utils.py 2011-06-21 09:16:28 +0000 @@ -19,6 +19,7 @@ import os import platform +import subprocess try: from CommandNotFound import CommandNotFound @@ -28,6 +29,31 @@ from linaro_image_tools import cmd_runner +def verify_file_integrity(sig_file_list): + """Verify a list of signature files. + + The parameter is a list of filenames of gpg signature files which will be + verified using gpg. For each of the files it is assumed that there is an + sha1 hash file with the same file name minus the '.asc' extension. + + Each of the sha1 files will be checked using sha1sums. All files listed in + the sha1 hash file must be found in the same directory as the hash file. + """ + verified_files = [] + for sig_file in sig_file_list: + hash_file = sig_file[0:-len('.asc')] + cmd_runner.run(['gpg', '--verify', sig_file]).wait() + if os.path.dirname(hash_file) == '': + sha_cwd = None + else: + sha_cwd = os.path.dirname(hash_file) + sha1sums_out, _ = cmd_runner.run(['sha1sum', '-c', hash_file], + stdout=subprocess.PIPE, cwd=sha_cwd + ).communicate() + verified_files.extend(sha1sums_out.replace(': OK', '').splitlines()) + return verified_files + + def install_package_providing(command): """Install a package which provides the given command. === modified file 'setup.py' --- setup.py 2011-05-26 20:32:40 +0000 +++ setup.py 2011-06-29 19:44:15 +0000 @@ -5,7 +5,7 @@ DistUtilsExtra.auto.setup( name="linaro-image-tools", - version="0.4.8.1", + version="2011.06-1.1", description="Tools to create and write Linaro images", url="https://launchpad.net/linaro-image-tools", license="GPL v3 or later",