From patchwork Thu Oct 3 04:09:32 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trevor Woerner X-Patchwork-Id: 20765 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 772D325E05 for ; Thu, 3 Oct 2013 04:10:00 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id j7sf3854123qaq.3 for ; Wed, 02 Oct 2013 21:10:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=mime-version:x-gm-message-state:delivered-to:from:to:cc:subject :date:message-id:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:list-post:list-help:list-archive :list-unsubscribe; bh=JSPVnfHI8olzHw67ML3klFSSjvl31eCpU09hl9P+RdU=; b=aATFq+5T6Ciq764/qZOnXehdEEI+vqe9CUjnZUbI0M6Wuz7RDzQI3Za2GSn6BQnurT RMHuLBa/xWqMpdADIiXm5CPleeL2O53Eby+MWj5oiJpOpEsX5ticFwoHm8TzE4z2fs8F JwQJpTNEHr7sYyyH7J9dCLU0+CnfRsG2BAvC2JkKpcZCM7OWJWqMVmC2VmmELwB/i2nk 12tbYZ081M52jxIjTsole0Wm7qmrgwlHPp9NXzZiizOOqFrsFtQ0n0H1WciI5/xVBY1+ skrFXtxi2IeTeveXRFl6qJg/W32ZpyedlJRtjMDo2fyXZzYyhN8PKcVgSQnC9cLOmkYB Yq+A== X-Received: by 10.236.216.17 with SMTP id f17mr5387743yhp.18.1380773399850; Wed, 02 Oct 2013 21:09:59 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.13.9 with SMTP id d9ls25153qec.26.gmail; Wed, 02 Oct 2013 21:09:59 -0700 (PDT) X-Received: by 10.52.33.44 with SMTP id o12mr4512795vdi.7.1380773399676; Wed, 02 Oct 2013 21:09:59 -0700 (PDT) Received: from mail-vb0-f51.google.com (mail-vb0-f51.google.com [209.85.212.51]) by mx.google.com with ESMTPS id o3si1237234ves.124.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 21:09:59 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.212.51 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.212.51; Received: by mail-vb0-f51.google.com with SMTP id x16so1211188vbf.38 for ; Wed, 02 Oct 2013 21:09:59 -0700 (PDT) X-Gm-Message-State: ALoCoQnMxD22b5g+pJIUUiUO3Y2TeZdN7cT0Og5Nu1e1pX0+qAtYWk+jjgrOPnXKJzHEb5mVblbk X-Received: by 10.220.169.203 with SMTP id a11mr65605vcz.26.1380773399472; Wed, 02 Oct 2013 21:09:59 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp146761vcz; Wed, 2 Oct 2013 21:09:58 -0700 (PDT) X-Received: by 10.50.67.105 with SMTP id m9mr562348igt.34.1380773398282; Wed, 02 Oct 2013 21:09:58 -0700 (PDT) Received: from mail-ie0-f177.google.com (mail-ie0-f177.google.com [209.85.223.177]) by mx.google.com with ESMTPS id jv4si2618282icc.54.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 21:09:58 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.223.177 is neither permitted nor denied by best guess record for domain of trevor.woerner@linaro.org) client-ip=209.85.223.177; Received: by mail-ie0-f177.google.com with SMTP id qd12so4203848ieb.36 for ; Wed, 02 Oct 2013 21:09:57 -0700 (PDT) X-Received: by 10.50.178.234 with SMTP id db10mr493070igc.35.1380773397731; Wed, 02 Oct 2013 21:09:57 -0700 (PDT) Received: from suse64-i7.site (dsl-67-55-9-50.acanac.net. [67.55.9.50]) by mx.google.com with ESMTPSA id x6sm14893852igb.3.1969.12.31.16.00.00 (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Oct 2013 21:09:56 -0700 (PDT) From: Trevor Woerner To: yocto@yoctoproject.org Cc: patches@linaro.org, Trevor Woerner Subject: [yocto-docs][PATCH] dev-manual: re-write runtime package management section Date: Thu, 3 Oct 2013 00:09:32 -0400 Message-Id: <1380773372-7951-1-git-send-email-trevor.woerner@linaro.org> X-Mailer: git-send-email 1.8.4.rc3.1.gc1ebd90 X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: trevor.woerner@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.212.51 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Signed-off-by: Trevor Woerner --- .../dev-manual/dev-manual-common-tasks.xml | 471 ++++++++++++++------- 1 file changed, 321 insertions(+), 150 deletions(-) diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index fe34e8e..7b4d638 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -3605,61 +3605,179 @@ -
- Setting Up Runtime Package Management - - - For supported package formats, it is possible to set - up a repository that is a host-based package feed from which - you can install packages on the target system during runtime. - Doing so is optional and depends on the following: +
+ Runtime Package Management + + Regardless of anything else, during a build bitbake will + transform a recipe into one or more packages. For example, + the bash recipe currently produces the + following packages: bash-dbg bash-staticdev bash-dev + bash-doc bash-locale bash. Not all generated + packages will be included in an image. + + In several situations you might want to have the ability to + update, add, remove, query, etc the packages on a target + device at runtime (i.e. without having to generate a new + image). Examples of such situations include: - You take specific steps to set up the feed. - + You want to provide in-the-field updates to deployed + devices (e.g. for security updates). + - When you build your image, you select to use the - appropriate package manager by setting the - PACKAGE_CLASSES - variable. - + You want to have a fast turn-around development cycle + for one or more applications which run on your device. + - You have a web server, such as Apache 2, - installed and configured on the development host. - + You want to temporarily install the "debug" packages + of various applications on your device so that + debugging can be greatly improved (access to symbols, + source debugging, etc). + - You enable package management on the target by - listing "package-management" in the - IMAGE_FEATURES - variable. - + You want to deploy a more minimal package selection of + your device but allow in-the-field updates to add a + larger selection for customization. + - + + In all these situations you have something similar to a more + traditional Linux distribution in that in-field devices + are able to grab pre-compiled packages from a server for + installation/update. This is what is termed "runtime package + management". In order to use runtime package management you + need a host/server machine which serves up the pre-compiled + packages plus the required meta data, and you need package + manipulation tools on the target. Note that the build machine + is a likely candidate to act as the server, but the build + machine doesn't necessarily have to be the package server; + the build machine could push its artifacts to another (e.g. + Internet-facing) machine which acts as the server. + + A simple build which targets just one device will produce + more than one package database. In other words, the packages + produced by a build will be separated out into a couple of + different package groupings based on criteria such as the + target's CPU architecture, the target board, or the C library + used on the target. For example, a build targetting the + qemuarm device will produce the following + 3 package databases: all, + armv5te, and + qemuarm. If I wanted my + qemuarm device to be aware of all the + packages which were available to it, I would need to point it + to each of these databases individually. In a similar way, a + traditional Linux distribution usually is configured to be + aware of a number of software repositories from which it + will retrieve packages. + + Using runtime package management is completely optional and + not required for a successful build or deployment in any way. + But if you want to make use of runtime package management + you'll need to do a couple things above and beyond the basics. + + +
+ Build Considerations + + In order to provide support for runtime package management + there are some build considerations of which to be aware. + + When bitbake generates packages it needs to know in + which format(s) you want the packages to be generated. + In your configuration this is handled by the + PACKAGE_CLASSES + variable. Note that you can choose to have more than one, + but at least one is required. + + If you would like your image to start off with a basic + package database of the packages in your current build + as well as having the relevant tools available on the + target for runtime package management, you can include + "package-management" in the + IMAGE_FEATURES + variable. Including "package-management" in this + configuration variable ensures that when the image + is assembled for your target it will include + the currently-known package databases as well as + the target-specific tools required for runtime + package management to be performed on the target. + Note, however, this isn't strictly necessary. + You could start your image off without any databases + but only include the required on-target package + tool(s) (for example you would include "opkg" in your + IMAGE_INSTALL + variable if you are using the IPK package format). You can + then initialize your target's package database(s) later, + once your image is up and running. + + Whenever you perform any sort of build step which can + potentially generate a package or modify an existing + package, it is always a good idea to re-generate the + package index with: + + $ bitbake package-index + + Note that it is not sufficient to simply do: + + $ bitbake <some-package> package-index + + since bitbake won't properly schedule the + package-index target fully after any + other target has completed. Therefore, be sure to run the + package update step separately. + + As described below in the + Using IPK + section, if you are using IPK as your package format, you + can make use of the + distro-feed-configs recipe provided + by meta-oe in order to configure your + target to use your IPK databases. + + When your build is complete your packages will show up in + the + ${TMPDIR}/deploy/<package-format> + directory. For example, if ${TMPDIR} + is tmp and your selected package type + is IPK, then your IPK packages will be available in + tmp/deploy/ipk. + +
- - The following list provides steps for setting up the optional - repository regardless of the package format. - Once you work through these generic steps, see the - "Using RPM" - section or the - "Using IPK" - section for remaining steps specific to the package type. - - The example assumes you are using the Apache 2 server: - - - - Add the directory to your Apache configuration, which - you can find at - /etc/httpd/conf/httpd.conf. - Use commands similar to these on the development system. - These example commands assume a top-level - Source Directory - named poky in your home directory. - The example also assumes an RPM package type. - If you are using a different package type, such as - IPK, use "ipk" in the pathnames: - +
+ Host or Server Machine Setup + + Typically packages are served from a server via HTTP, but + other protocols are possible. If we assume you want to + use HTTP, then you would need to setup and configure a + web server, such as Apache 2 or lighttpd, on the machine + serving the packages. As mentioned above, the build + machine can act as the package server; in the following + server machine setups it is assumed the build machine is + also the server. + + +
+ Serving Packages via Apache 2 + + This example assumes you are using the Apache 2 + server: + + + Add the directory to your Apache + configuration, which you can find at + /etc/httpd/conf/httpd.conf. + Use commands similar to these on the + development system. These example + commands assume a top-level + Source Directory + named poky in your home + directory. The example also assumes an RPM + package type. If you are using a different + package type, such as IPK, use "ipk" in the + pathnames: + <VirtualHost *:80> .... Alias /rpm ~/poky/build/tmp/deploy/rpm @@ -3667,119 +3785,172 @@ Options +Indexes </Directory> </VirtualHost> + + + + Reload the Apache configuration as follows. + For all commands, be sure you have root + privileges. + + If your development system is using Fedora or + CentOS, use the following: + + # service httpd reload + + For Ubuntu and Debian, use the following: + + # /etc/init.d/apache2 reload + + For OpenSUSE, use the following: + + # /etc/init.d/apache2 reload + + + + If you are using Security-Enhanced Linux + (SELinux), you need to label the files as + being accessible through Apache. Use the + following command from the development host + (this example assumes RPM package types): + + # chcon -R -h -t httpd_sys_content_t tmp/deploy/rpm + + + + +
+ +
+ Serving Packages via lighttpd + + If you are using lighttpd all you need + to do is to provide a link from your + ${TMPDIR}/deploy/<package-format> directory to + lighttpd's document-root. You can determine the + specifics of your lighttpd installation by looking + through its configuration file which is usually found + at: /etc/lighttpd/lighttpd.conf. + + For example, if you are using IPK, if + lighttpd's document-root is set to + /var/www/lighttpd, and if you had + packages for a target named "BOARD" + then you might create a link from your build location + to lighttpd's document-root as follows: + + # ln -s $(PWD)/tmp/deploy/ipk /var/www/lighttpd/BOARD-dir - - - Reload the Apache configuration as follows. - For all commands, be sure you have root privileges. - - - If your development system is using Fedora or - CentOS, use the following: + + At this point you need to start the lighttpd server. + The way in which you start the server will vary by + distribution, but one basic way to start it by hand + would be: - service httpd reload + # lighttpd -f /etc/lighttpd/lighttpd.conf - For Ubuntu and Debian, use the following: + +
+
+ +
+ Target Setup + +
+ Using RPM + + The application for performing runtime package + management of RPM packages on the target is called + smart. + + On the target machine, you need to inform + smart of every package database + you wish to use. As an example, suppose your target + device can use the following 3 package databases from + a server named server.name: + all, i586, + and qemux86. Given this example, + issue the following commands on the target: - /etc/init.d/apache2 reload + # smart channel --add all type=rpm-md baseurl=http://server.name/rpm/all + # smart channel --add i585 type=rpm-md baseurl=http://server.name/rpm/i586 + # smart channel --add qemux86 type=rpm-md baseurl=http://server.name/rpm/qemux86 - For OpenSUSE, use the following: + Also from the target machine, fetch the repository + information using this command: - /etc/init.d/apache2 reload + # smart update - - - Re-generate the package index: + You can now use the smart query + and smart install commands to + find and install packages from the repositories. + +
+ +
+ Using IPK + + The application for performing runtime package + management of IPK packages on the target is called + opkg. + + In order to inform opkg of the + package databases you wish to use, simply create one + or more *.conf files in the + /etc/opkg directory on the target + and opkg will use them to find + its available package databases. As an example if you + configured your HTTP server on your machine named + www.mysite.com to serve files + from a BOARD-dir directory under + its document-root you might create a configuration + file on the target called + /etc/opkg/base-feeds.conf which + contains: - bitbake package-index + src/gz all http://www.mysite.com/BOARD-dir/all + src/gz armv7a http://www.mysite.com/BOARD-dir/armv7a + src/gz beagleboard http://www.mysite.com/BOARD-dir/beagleboard - - - If you are using Security-Enhanced Linux (SELinux), - you need to label the files as being accessible - through Apache. - Use the following command from the development host. - Again, the example assumes RPM package types: + + + As a way of making it easier to generate and make + these IPK configuration files available on your + target, the meta-oe layer + provides a recipe called + distro-feed-configs (which + provides a package by the same name). When you + include this package into your image, it will + automatically generate and include a set of + *.conf files in the image's + /etc/opkg directory which will + provide your target's opkg tool with any and all + package databases your build will generate. The only + catch is that this recipe can't possibly imagine your + server's DNS name/IP address, so somewhere in your + configuration you need to set a variable called + DISTRO_FEED_URI which will point + to your server and the location within the + document-root which contains the databases. For + example: if you are serving your packages over HTTP, + your server's IP address is 192.168.7.1, and your + databases are located in a directory called + BOARD-dir underneath your HTTP + server's document-root then set + DISTRO_FEED_URI to + http://192.168.7.1/BOARD-dir. + + + On the target machine, fetch (or refresh) the + repository information using this command: - chcon -R -h -t httpd_sys_content_t tmp/deploy/rpm + # opkg update - - - - -
- Using RPM - - - Following are RPM-specific steps needed for setting up the - optional repository. - Perform these steps after working through the common steps - at the start of this section: - - - On the target machine, add the repository to Smart - for every package architecture. - To see the list of package architectures, list - the contents of the - setting-up-runtime-package-management tmp/deploy/rpm directory - on the host. - - As an example, suppose you list the contents of the - directory and discover three architectures: - all, i586, - and qemux86. - Given this example, use the following commands: - - smart channel ‐‐add all type=rpm-md baseurl=http://server.name/rpm/all - smart channel ‐‐add i585 type=rpm-md baseurl=http://server.name/rpm/i586 - smart channel ‐‐add qemux86 type=rpm-md baseurl=http://server.name/rpm/qemux86 - - - - Also from the target machine, fetch the repository - information using this command: - - smart update - - - - You can now use the smart query - and smart install commands to find - and install packages from the repositories. - -
- -
- Using IPK - - - Following are IPK-specific steps needed for setting up the - optional repository. - Perform these steps after working through the common steps - at the start of this section: - - Install packages onto an - existing running system by first sharing the - tmp/deploy/ipk/ directory - through a web server and then by changing - /etc/opkg/base-feeds.conf - to point at the shared server. - Following is an example: - - src/gz all http://www.mysite.com/somedir/deploy/ipk/all - src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a - src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard - - From the target machine, fetch the - repository information using this command: - - opkg update - - - You can now use the opkg list and - opkg install commands to find and - install packages from the repositories. - + You can now use the opkg list and + opkg install commands to find and + install packages from the repositories. + + +