From patchwork Thu Apr 9 21:08:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 237599 List-Id: U-Boot discussion From: sjg at chromium.org (Simon Glass) Date: Thu, 9 Apr 2020 15:08:48 -0600 Subject: [PATCH v2 21/28] buildman: Update the 'theory of operation' a little In-Reply-To: <20200409210856.160952-1-sjg@chromium.org> References: <20200409210856.160952-1-sjg@chromium.org> Message-ID: <20200409150840.v2.21.Ib73f5c254b06a9d8c7019d86197aa685f6bcb90f@changeid> Make a few updates to this important section of the documentation, to make things clearer. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: None tools/buildman/README | 69 ++++++++++++++++++++++++------------------- 1 file changed, 39 insertions(+), 30 deletions(-) diff --git a/tools/buildman/README b/tools/buildman/README index 4cf01141574..86f5dfe501f 100644 --- a/tools/buildman/README +++ b/tools/buildman/README @@ -51,23 +51,25 @@ Theory of Operation Buildman is a builder. It is not make, although it runs make. It does not produce any useful output on the terminal while building, except for -progress information (except with -v, see below). All the output (errors, -warnings and binaries if you ask for them) is stored in output -directories, which you can look at while the build is progressing, or when -it is finished. +progress information (but see -v below). All the output (errors, warnings and +binaries if you ask for them) is stored in output directories, which you can +look at from a separate 'buildman -s' instance while the build is progressing, +or when it is finished. Buildman is designed to build entire git branches, i.e. muliple commits. It -can be run repeatedly on the same branch. In this case it will automatically -rebuild commits which have changed (and remove its old results for that -commit). It is possible to build a branch for one board, then later build it -for another board. If you want buildman to re-build a commit it has already -built (e.g. because of a toolchain update), use the -f flag. +can be run repeatedly on the same branch after making changes to commits on +that branch. In this case it will automatically rebuild commits which have +changed (and remove its old results for that commit). It is possible to build +a branch for one board, then later build it for another board. This adds to +the output, so now you have results for two boards. If you want buildman to +re-build a commit it has already built (e.g. because of a toolchain update), +use the -f flag. Buildman produces a concise summary of which boards succeeded and failed. It shows which commit introduced which board failure using a simple -red/green colour coding. Full error information can be requested, in which -case it is de-duped and displayed against the commit that introduced the -error. An example workflow is below. +red/green colour coding (with yellow/cyan for warnings). Full error +information can be requested, in which case it is de-duped and displayed +against the commit that introduced the error. An example workflow is below. Buildman stores image size information and can report changes in image size from commit to commit. An example of this is below. @@ -75,16 +77,20 @@ from commit to commit. An example of this is below. Buildman starts multiple threads, and each thread builds for one board at a time. A thread starts at the first commit, configures the source for your board and builds it. Then it checks out the next commit and does an -incremental build. Eventually the thread reaches the last commit and stops. -If errors or warnings are found along the way, the thread will reconfigure -after every commit, and your build will be very slow. This is because a -file that produces just a warning would not normally be rebuilt in an -incremental build. +incremental build (i.e. not using 'make xxx_defconfig' unless you use -C). +Eventually the thread reaches the last commit and stops. If a commit causes +an error or warning, buildman will try it again after reconfiguring (but see +-Q). Thus some commits may be built twice, with the first result silently +discarded. Lots of errors and warnings will causes lots of reconfigures and your +build will be very slow. This is because a file that produces just a warning +would not normally be rebuilt in an incremental build. Once a thread finishes +building all the commits for a board, it starts on the commits for another +board. Buildman works in an entirely separate place from your U-Boot repository. It creates a separate working directory for each thread, and puts the output files in the working directory, organised by commit name and board -name, in a two-level hierarchy. +name, in a two-level hierarchy (but see -P). Buildman is invoked in your U-Boot directory, the one with the .git directory. It clones this repository into a copy for each thread, and the @@ -92,20 +98,23 @@ threads do not affect the state of your git repository. Any checkouts done by the thread affect only the working directory for that thread. Buildman automatically selects the correct tool chain for each board. You -must supply suitable tool chains, but buildman takes care of selecting the -right one. +must supply suitable tool chains (see --fetch-arch), but buildman takes care +of selecting the right one. Buildman generally builds a branch (with the -b flag), and in this case -builds the upstream commit as well, for comparison. It cannot build -individual commits at present, unless (maybe) you point it at an empty -branch. Put all your commits in a branch, set the branch's upstream to a -valid value, and all will be well. Otherwise buildman will perform random -actions. Use -n to check what the random actions might be. - -If you just want to build the current source tree, leave off the -b flag -and add -e. This will display results and errors as they happen. You can -still look at them later using -se. Note that buildman will assume that the -source has changed, and will build all specified boards in this case. +builds the upstream commit as well, for comparison. So even if you have one +commit in your branch, two commits will be built. Put all your commits in a +branch, set the branch's upstream to a valid value, and all will be well. +Otherwise buildman will perform random actions. Use -n to check what the +random actions might be. + +Buildman effectively has two modes: without -s it builds, with -s it +summarises the results of previous (or active) builds. + +If you just want to build the current source tree, leave off the -b flag. +This will display results and errors as they happen. You can still look at +them later using -se. Note that buildman will assume that the source has +changed, and will build all specified boards in this case. Buildman is optimised for building many commits at once, for many boards. On multi-core machines, Buildman is fast because it uses most of the