diff mbox series

[v5,06/12] Dockerfile: capsule: Setup the files needed for capsule update testing

Message ID 20230725085725.350917-7-sughosh.ganu@linaro.org
State New
Headers show
Series Integrate EFI capsule tasks into u-boot's build flow | expand

Commit Message

Sughosh Ganu July 25, 2023, 8:57 a.m. UTC
Support has being added through earlier commits to build capsules
and embed the public key needed for capsule authentication as part of
u-boot build.

From the testing point-of-view, this means the input files needed for
generating the above have to be setup before invoking the build. Set
this up in the CI configuration docker file for testing the capsule
update feature.

Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
---
Changes since V4:
* New patch which moves the setting up of the files needed for testing
  the EFI capsule update feature to the Dockerfile.

Note: Earlier, this setup was being done in the azure and gitlab yaml
files. Now that this has been moved to the Dockerfile, this will
require generating a new container image and referencing that image in
the yaml files for the CI to work when these patches get applied.

 tools/docker/Dockerfile | 12 ++++++++++++
 1 file changed, 12 insertions(+)

Comments

Simon Glass July 25, 2023, 10:52 p.m. UTC | #1
On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
>
> Support has being added through earlier commits to build capsules
> and embed the public key needed for capsule authentication as part of
> u-boot build.
>
> From the testing point-of-view, this means the input files needed for
> generating the above have to be setup before invoking the build. Set
> this up in the CI configuration docker file for testing the capsule
> update feature.
>
> Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> ---
> Changes since V4:
> * New patch which moves the setting up of the files needed for testing
>   the EFI capsule update feature to the Dockerfile.
>
> Note: Earlier, this setup was being done in the azure and gitlab yaml
> files. Now that this has been moved to the Dockerfile, this will
> require generating a new container image and referencing that image in
> the yaml files for the CI to work when these patches get applied.
>
>  tools/docker/Dockerfile | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
>
> diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> index 3d2b64a355..294a0b0a53 100644
> --- a/tools/docker/Dockerfile
> +++ b/tools/docker/Dockerfile
> @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
>         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
>         rm -rf /tmp/qemu-linaro
>
> +# Set up capsule files for UEFI capsule update testing
> +RUN mkdir -p /tmp/capsules && \
> +    cd /tmp/capsules/ && \

You can just use ${UBOOT_TRAVIS_BUILD_DIR} here

> +    echo -n "u-boot:Old" > u-boot.bin.old && \
> +    echo -n "u-boot:New" > u-boot.bin.new && \
> +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> +    echo -n "u-boot-env:New" > u-boot.env.new && \

We don't want these files, just the certs, since they are the things
that take a long time:

> +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> +    chmod -R uog+rw /tmp/capsules/
> +
>  # Build genimage (required by some targets to generate disk images)
>  RUN wget -O - https://github.com/pengutronix/genimage/releases/download/v14/genimage-14.tar.xz | tar -C /tmp -xJ && \
>         cd /tmp/genimage-14 && \
> --
> 2.34.1
>

Regards,
Simon
Tom Rini July 25, 2023, 10:56 p.m. UTC | #2
On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> >
> > Support has being added through earlier commits to build capsules
> > and embed the public key needed for capsule authentication as part of
> > u-boot build.
> >
> > From the testing point-of-view, this means the input files needed for
> > generating the above have to be setup before invoking the build. Set
> > this up in the CI configuration docker file for testing the capsule
> > update feature.
> >
> > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > ---
> > Changes since V4:
> > * New patch which moves the setting up of the files needed for testing
> >   the EFI capsule update feature to the Dockerfile.
> >
> > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > files. Now that this has been moved to the Dockerfile, this will
> > require generating a new container image and referencing that image in
> > the yaml files for the CI to work when these patches get applied.
> >
> >  tools/docker/Dockerfile | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > index 3d2b64a355..294a0b0a53 100644
> > --- a/tools/docker/Dockerfile
> > +++ b/tools/docker/Dockerfile
> > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> >         rm -rf /tmp/qemu-linaro
> >
> > +# Set up capsule files for UEFI capsule update testing
> > +RUN mkdir -p /tmp/capsules && \
> > +    cd /tmp/capsules/ && \
> 
> You can just use ${UBOOT_TRAVIS_BUILD_DIR} here

That's not present in Dockerfiles, only at runtime within jobs (because
we set it).

> > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> 
> We don't want these files, just the certs, since they are the things
> that take a long time:
> 
> > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > +    chmod -R uog+rw /tmp/capsules/

How long does it even take to make these certs? I'm not sure it's great
to make these and stage them in /tmp and expect them to be around at
test time.
Sughosh Ganu July 26, 2023, 9:46 a.m. UTC | #3
On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > >
> > > Support has being added through earlier commits to build capsules
> > > and embed the public key needed for capsule authentication as part of
> > > u-boot build.
> > >
> > > From the testing point-of-view, this means the input files needed for
> > > generating the above have to be setup before invoking the build. Set
> > > this up in the CI configuration docker file for testing the capsule
> > > update feature.
> > >
> > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > ---
> > > Changes since V4:
> > > * New patch which moves the setting up of the files needed for testing
> > >   the EFI capsule update feature to the Dockerfile.
> > >
> > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > files. Now that this has been moved to the Dockerfile, this will
> > > require generating a new container image and referencing that image in
> > > the yaml files for the CI to work when these patches get applied.
> > >
> > >  tools/docker/Dockerfile | 12 ++++++++++++
> > >  1 file changed, 12 insertions(+)
> > >
> > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > index 3d2b64a355..294a0b0a53 100644
> > > --- a/tools/docker/Dockerfile
> > > +++ b/tools/docker/Dockerfile
> > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > >         rm -rf /tmp/qemu-linaro
> > >
> > > +# Set up capsule files for UEFI capsule update testing
> > > +RUN mkdir -p /tmp/capsules && \
> > > +    cd /tmp/capsules/ && \
> >
> > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
>
> That's not present in Dockerfiles, only at runtime within jobs (because
> we set it).

I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
similar to what is being done for the grub*.efi files.

>
> > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> >
> > We don't want these files, just the certs, since they are the things
> > that take a long time:
> >
> > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > +    chmod -R uog+rw /tmp/capsules/
>
> How long does it even take to make these certs? I'm not sure it's great
> to make these and stage them in /tmp and expect them to be around at
> test time.

Should I mimic what is being done for the various grub.efi files? I
believe that these are in the /opt/grub/ directory of the docker
image, and get copied to the build dir at runtime.

-sughosh
Tom Rini July 26, 2023, 1:23 p.m. UTC | #4
On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > >
> > > > Support has being added through earlier commits to build capsules
> > > > and embed the public key needed for capsule authentication as part of
> > > > u-boot build.
> > > >
> > > > From the testing point-of-view, this means the input files needed for
> > > > generating the above have to be setup before invoking the build. Set
> > > > this up in the CI configuration docker file for testing the capsule
> > > > update feature.
> > > >
> > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > ---
> > > > Changes since V4:
> > > > * New patch which moves the setting up of the files needed for testing
> > > >   the EFI capsule update feature to the Dockerfile.
> > > >
> > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > files. Now that this has been moved to the Dockerfile, this will
> > > > require generating a new container image and referencing that image in
> > > > the yaml files for the CI to work when these patches get applied.
> > > >
> > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > >  1 file changed, 12 insertions(+)
> > > >
> > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > index 3d2b64a355..294a0b0a53 100644
> > > > --- a/tools/docker/Dockerfile
> > > > +++ b/tools/docker/Dockerfile
> > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > >         rm -rf /tmp/qemu-linaro
> > > >
> > > > +# Set up capsule files for UEFI capsule update testing
> > > > +RUN mkdir -p /tmp/capsules && \
> > > > +    cd /tmp/capsules/ && \
> > >
> > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> >
> > That's not present in Dockerfiles, only at runtime within jobs (because
> > we set it).
> 
> I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> similar to what is being done for the grub*.efi files.

Yes, copying the files rather than relying on them being in /tmp is
better, but..

> > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > >
> > > We don't want these files, just the certs, since they are the things
> > > that take a long time:
> > >
> > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > +    chmod -R uog+rw /tmp/capsules/
> >
> > How long does it even take to make these certs? I'm not sure it's great
> > to make these and stage them in /tmp and expect them to be around at
> > test time.
> 
> Should I mimic what is being done for the various grub.efi files? I
> believe that these are in the /opt/grub/ directory of the docker
> image, and get copied to the build dir at runtime.

It takes 10 minutes or so to build grub, and we use it in multiple
tests.  Running openssl takes not even a second. Why are we doing this
in the Dockerfile? Is this needed in more than one test? If so, does it
matter if we have the same certs in each test?
Sughosh Ganu July 26, 2023, 2:09 p.m. UTC | #5
On Wed, 26 Jul 2023 at 18:53, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > >
> > > > > Support has being added through earlier commits to build capsules
> > > > > and embed the public key needed for capsule authentication as part of
> > > > > u-boot build.
> > > > >
> > > > > From the testing point-of-view, this means the input files needed for
> > > > > generating the above have to be setup before invoking the build. Set
> > > > > this up in the CI configuration docker file for testing the capsule
> > > > > update feature.
> > > > >
> > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > ---
> > > > > Changes since V4:
> > > > > * New patch which moves the setting up of the files needed for testing
> > > > >   the EFI capsule update feature to the Dockerfile.
> > > > >
> > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > require generating a new container image and referencing that image in
> > > > > the yaml files for the CI to work when these patches get applied.
> > > > >
> > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > >  1 file changed, 12 insertions(+)
> > > > >
> > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > --- a/tools/docker/Dockerfile
> > > > > +++ b/tools/docker/Dockerfile
> > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > >         rm -rf /tmp/qemu-linaro
> > > > >
> > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > +    cd /tmp/capsules/ && \
> > > >
> > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > >
> > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > we set it).
> >
> > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > similar to what is being done for the grub*.efi files.
>
> Yes, copying the files rather than relying on them being in /tmp is
> better, but..
>
> > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > >
> > > > We don't want these files, just the certs, since they are the things
> > > > that take a long time:
> > > >
> > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > +    chmod -R uog+rw /tmp/capsules/
> > >
> > > How long does it even take to make these certs? I'm not sure it's great
> > > to make these and stage them in /tmp and expect them to be around at
> > > test time.
> >
> > Should I mimic what is being done for the various grub.efi files? I
> > believe that these are in the /opt/grub/ directory of the docker
> > image, and get copied to the build dir at runtime.
>
> It takes 10 minutes or so to build grub, and we use it in multiple
> tests.  Running openssl takes not even a second. Why are we doing this
> in the Dockerfile? Is this needed in more than one test? If so, does it
> matter if we have the same certs in each test?

The reason why these files are needed is that they need to be present
before we start the build, for the sandbox variants. I was creating
these in the yml files in the earlier versions. Simon suggested
putting them in the docker file since these commands don't have to be
run on every invocation. These files are needed for the build and
pytest stages of the CI.

-sughosh
Simon Glass July 26, 2023, 2:11 p.m. UTC | #6
Hi Tom,

On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > >
> > > > > Support has being added through earlier commits to build capsules
> > > > > and embed the public key needed for capsule authentication as part of
> > > > > u-boot build.
> > > > >
> > > > > From the testing point-of-view, this means the input files needed for
> > > > > generating the above have to be setup before invoking the build. Set
> > > > > this up in the CI configuration docker file for testing the capsule
> > > > > update feature.
> > > > >
> > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > ---
> > > > > Changes since V4:
> > > > > * New patch which moves the setting up of the files needed for testing
> > > > >   the EFI capsule update feature to the Dockerfile.
> > > > >
> > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > require generating a new container image and referencing that image in
> > > > > the yaml files for the CI to work when these patches get applied.
> > > > >
> > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > >  1 file changed, 12 insertions(+)
> > > > >
> > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > --- a/tools/docker/Dockerfile
> > > > > +++ b/tools/docker/Dockerfile
> > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > >         rm -rf /tmp/qemu-linaro
> > > > >
> > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > +    cd /tmp/capsules/ && \
> > > >
> > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > >
> > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > we set it).
> >
> > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > similar to what is being done for the grub*.efi files.
>
> Yes, copying the files rather than relying on them being in /tmp is
> better, but..
>
> > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > >
> > > > We don't want these files, just the certs, since they are the things
> > > > that take a long time:
> > > >
> > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > +    chmod -R uog+rw /tmp/capsules/
> > >
> > > How long does it even take to make these certs? I'm not sure it's great
> > > to make these and stage them in /tmp and expect them to be around at
> > > test time.
> >
> > Should I mimic what is being done for the various grub.efi files? I
> > believe that these are in the /opt/grub/ directory of the docker
> > image, and get copied to the build dir at runtime.
>
> It takes 10 minutes or so to build grub, and we use it in multiple
> tests.  Running openssl takes not even a second. Why are we doing this
> in the Dockerfile? Is this needed in more than one test? If so, does it
> matter if we have the same certs in each test?

Yes it is actually much faster that I expected, so I suppose we can go
back to having it in the test itself, e.g. in a pytest fixture.

Regards,
Simon
Sughosh Ganu July 26, 2023, 2:41 p.m. UTC | #7
hi Simon,

On Wed, 26 Jul 2023 at 19:41, Simon Glass <sjg@chromium.org> wrote:
>
> Hi Tom,
>
> On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > > >
> > > > > > Support has being added through earlier commits to build capsules
> > > > > > and embed the public key needed for capsule authentication as part of
> > > > > > u-boot build.
> > > > > >
> > > > > > From the testing point-of-view, this means the input files needed for
> > > > > > generating the above have to be setup before invoking the build. Set
> > > > > > this up in the CI configuration docker file for testing the capsule
> > > > > > update feature.
> > > > > >
> > > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > > ---
> > > > > > Changes since V4:
> > > > > > * New patch which moves the setting up of the files needed for testing
> > > > > >   the EFI capsule update feature to the Dockerfile.
> > > > > >
> > > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > > require generating a new container image and referencing that image in
> > > > > > the yaml files for the CI to work when these patches get applied.
> > > > > >
> > > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > > >  1 file changed, 12 insertions(+)
> > > > > >
> > > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > > --- a/tools/docker/Dockerfile
> > > > > > +++ b/tools/docker/Dockerfile
> > > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > > >         rm -rf /tmp/qemu-linaro
> > > > > >
> > > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > > +    cd /tmp/capsules/ && \
> > > > >
> > > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > > >
> > > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > > we set it).
> > >
> > > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > > similar to what is being done for the grub*.efi files.
> >
> > Yes, copying the files rather than relying on them being in /tmp is
> > better, but..
> >
> > > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > > >
> > > > > We don't want these files, just the certs, since they are the things
> > > > > that take a long time:
> > > > >
> > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > > +    chmod -R uog+rw /tmp/capsules/
> > > >
> > > > How long does it even take to make these certs? I'm not sure it's great
> > > > to make these and stage them in /tmp and expect them to be around at
> > > > test time.
> > >
> > > Should I mimic what is being done for the various grub.efi files? I
> > > believe that these are in the /opt/grub/ directory of the docker
> > > image, and get copied to the build dir at runtime.
> >
> > It takes 10 minutes or so to build grub, and we use it in multiple
> > tests.  Running openssl takes not even a second. Why are we doing this
> > in the Dockerfile? Is this needed in more than one test? If so, does it
> > matter if we have the same certs in each test?
>
> Yes it is actually much faster that I expected, so I suppose we can go
> back to having it in the test itself, e.g. in a pytest fixture.

If not part of the docker image, these commands will still have to run
as part of the azure and gitlab yml files. They are needed before the
build is invoked, as they serve as input files needed to generate the
capsules as part of the build. The pytest fixtures would be run a)
only for the pytest stages and b) after the build has completed.

-sughosh
Simon Glass July 26, 2023, 3:40 p.m. UTC | #8
Hi Sughosh,

On Wed, 26 Jul 2023 at 08:41, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
>
> hi Simon,
>
> On Wed, 26 Jul 2023 at 19:41, Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > > > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > > > >
> > > > > > > Support has being added through earlier commits to build capsules
> > > > > > > and embed the public key needed for capsule authentication as part of
> > > > > > > u-boot build.
> > > > > > >
> > > > > > > From the testing point-of-view, this means the input files needed for
> > > > > > > generating the above have to be setup before invoking the build. Set
> > > > > > > this up in the CI configuration docker file for testing the capsule
> > > > > > > update feature.
> > > > > > >
> > > > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > > > ---
> > > > > > > Changes since V4:
> > > > > > > * New patch which moves the setting up of the files needed for testing
> > > > > > >   the EFI capsule update feature to the Dockerfile.
> > > > > > >
> > > > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > > > require generating a new container image and referencing that image in
> > > > > > > the yaml files for the CI to work when these patches get applied.
> > > > > > >
> > > > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > > > >  1 file changed, 12 insertions(+)
> > > > > > >
> > > > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > > > --- a/tools/docker/Dockerfile
> > > > > > > +++ b/tools/docker/Dockerfile
> > > > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > > > >         rm -rf /tmp/qemu-linaro
> > > > > > >
> > > > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > > > +    cd /tmp/capsules/ && \
> > > > > >
> > > > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > > > >
> > > > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > > > we set it).
> > > >
> > > > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > > > similar to what is being done for the grub*.efi files.
> > >
> > > Yes, copying the files rather than relying on them being in /tmp is
> > > better, but..
> > >
> > > > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > > > >
> > > > > > We don't want these files, just the certs, since they are the things
> > > > > > that take a long time:
> > > > > >
> > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > > > +    chmod -R uog+rw /tmp/capsules/
> > > > >
> > > > > How long does it even take to make these certs? I'm not sure it's great
> > > > > to make these and stage them in /tmp and expect them to be around at
> > > > > test time.
> > > >
> > > > Should I mimic what is being done for the various grub.efi files? I
> > > > believe that these are in the /opt/grub/ directory of the docker
> > > > image, and get copied to the build dir at runtime.
> > >
> > > It takes 10 minutes or so to build grub, and we use it in multiple
> > > tests.  Running openssl takes not even a second. Why are we doing this
> > > in the Dockerfile? Is this needed in more than one test? If so, does it
> > > matter if we have the same certs in each test?
> >
> > Yes it is actually much faster that I expected, so I suppose we can go
> > back to having it in the test itself, e.g. in a pytest fixture.
>
> If not part of the docker image, these commands will still have to run
> as part of the azure and gitlab yml files. They are needed before the
> build is invoked, as they serve as input files needed to generate the
> capsules as part of the build. The pytest fixtures would be run a)
> only for the pytest stages and b) after the build has completed.

OK, so let's just create the files before they are needed?

One thought I had that might accelerate this long-running series, is
just to do the binman part first (with tests), then add it into the
build and CI once that is merged?

Regards,
Simon
Tom Rini July 26, 2023, 4:39 p.m. UTC | #9
On Wed, Jul 26, 2023 at 08:11:44PM +0530, Sughosh Ganu wrote:
> hi Simon,
> 
> On Wed, 26 Jul 2023 at 19:41, Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > > > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > > > >
> > > > > > > Support has being added through earlier commits to build capsules
> > > > > > > and embed the public key needed for capsule authentication as part of
> > > > > > > u-boot build.
> > > > > > >
> > > > > > > From the testing point-of-view, this means the input files needed for
> > > > > > > generating the above have to be setup before invoking the build. Set
> > > > > > > this up in the CI configuration docker file for testing the capsule
> > > > > > > update feature.
> > > > > > >
> > > > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > > > ---
> > > > > > > Changes since V4:
> > > > > > > * New patch which moves the setting up of the files needed for testing
> > > > > > >   the EFI capsule update feature to the Dockerfile.
> > > > > > >
> > > > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > > > require generating a new container image and referencing that image in
> > > > > > > the yaml files for the CI to work when these patches get applied.
> > > > > > >
> > > > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > > > >  1 file changed, 12 insertions(+)
> > > > > > >
> > > > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > > > --- a/tools/docker/Dockerfile
> > > > > > > +++ b/tools/docker/Dockerfile
> > > > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > > > >         rm -rf /tmp/qemu-linaro
> > > > > > >
> > > > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > > > +    cd /tmp/capsules/ && \
> > > > > >
> > > > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > > > >
> > > > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > > > we set it).
> > > >
> > > > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > > > similar to what is being done for the grub*.efi files.
> > >
> > > Yes, copying the files rather than relying on them being in /tmp is
> > > better, but..
> > >
> > > > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > > > >
> > > > > > We don't want these files, just the certs, since they are the things
> > > > > > that take a long time:
> > > > > >
> > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > > > +    chmod -R uog+rw /tmp/capsules/
> > > > >
> > > > > How long does it even take to make these certs? I'm not sure it's great
> > > > > to make these and stage them in /tmp and expect them to be around at
> > > > > test time.
> > > >
> > > > Should I mimic what is being done for the various grub.efi files? I
> > > > believe that these are in the /opt/grub/ directory of the docker
> > > > image, and get copied to the build dir at runtime.
> > >
> > > It takes 10 minutes or so to build grub, and we use it in multiple
> > > tests.  Running openssl takes not even a second. Why are we doing this
> > > in the Dockerfile? Is this needed in more than one test? If so, does it
> > > matter if we have the same certs in each test?
> >
> > Yes it is actually much faster that I expected, so I suppose we can go
> > back to having it in the test itself, e.g. in a pytest fixture.
> 
> If not part of the docker image, these commands will still have to run
> as part of the azure and gitlab yml files. They are needed before the
> build is invoked, as they serve as input files needed to generate the
> capsules as part of the build. The pytest fixtures would be run a)
> only for the pytest stages and b) after the build has completed.

So lets just make use of the artifacts feature, generate these once per
CI run and pass them to the jobs that need them?  The documentation
should reflect that you need to make the certs for production, and for
testing purposes see the following sections / stanzas in the GitLab and
Azure CI files.  Then the pytest part doesn't need to handle the case of
running this on your own, since if you are, you need to have read the
docs and followed instructions.
Sughosh Ganu July 26, 2023, 8 p.m. UTC | #10
On Wed, 26 Jul 2023 at 22:09, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Jul 26, 2023 at 08:11:44PM +0530, Sughosh Ganu wrote:
> > hi Simon,
> >
> > On Wed, 26 Jul 2023 at 19:41, Simon Glass <sjg@chromium.org> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > > > > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > > > > >
> > > > > > > > Support has being added through earlier commits to build capsules
> > > > > > > > and embed the public key needed for capsule authentication as part of
> > > > > > > > u-boot build.
> > > > > > > >
> > > > > > > > From the testing point-of-view, this means the input files needed for
> > > > > > > > generating the above have to be setup before invoking the build. Set
> > > > > > > > this up in the CI configuration docker file for testing the capsule
> > > > > > > > update feature.
> > > > > > > >
> > > > > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > > > > ---
> > > > > > > > Changes since V4:
> > > > > > > > * New patch which moves the setting up of the files needed for testing
> > > > > > > >   the EFI capsule update feature to the Dockerfile.
> > > > > > > >
> > > > > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > > > > require generating a new container image and referencing that image in
> > > > > > > > the yaml files for the CI to work when these patches get applied.
> > > > > > > >
> > > > > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > > > > >  1 file changed, 12 insertions(+)
> > > > > > > >
> > > > > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > > > > --- a/tools/docker/Dockerfile
> > > > > > > > +++ b/tools/docker/Dockerfile
> > > > > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > > > > >         rm -rf /tmp/qemu-linaro
> > > > > > > >
> > > > > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > > > > +    cd /tmp/capsules/ && \
> > > > > > >
> > > > > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > > > > >
> > > > > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > > > > we set it).
> > > > >
> > > > > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > > > > similar to what is being done for the grub*.efi files.
> > > >
> > > > Yes, copying the files rather than relying on them being in /tmp is
> > > > better, but..
> > > >
> > > > > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > > > > >
> > > > > > > We don't want these files, just the certs, since they are the things
> > > > > > > that take a long time:
> > > > > > >
> > > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > > > > +    chmod -R uog+rw /tmp/capsules/
> > > > > >
> > > > > > How long does it even take to make these certs? I'm not sure it's great
> > > > > > to make these and stage them in /tmp and expect them to be around at
> > > > > > test time.
> > > > >
> > > > > Should I mimic what is being done for the various grub.efi files? I
> > > > > believe that these are in the /opt/grub/ directory of the docker
> > > > > image, and get copied to the build dir at runtime.
> > > >
> > > > It takes 10 minutes or so to build grub, and we use it in multiple
> > > > tests.  Running openssl takes not even a second. Why are we doing this
> > > > in the Dockerfile? Is this needed in more than one test? If so, does it
> > > > matter if we have the same certs in each test?
> > >
> > > Yes it is actually much faster that I expected, so I suppose we can go
> > > back to having it in the test itself, e.g. in a pytest fixture.
> >
> > If not part of the docker image, these commands will still have to run
> > as part of the azure and gitlab yml files. They are needed before the
> > build is invoked, as they serve as input files needed to generate the
> > capsules as part of the build. The pytest fixtures would be run a)
> > only for the pytest stages and b) after the build has completed.
>
> So lets just make use of the artifacts feature, generate these once per
> CI run and pass them to the jobs that need them?  The documentation
> should reflect that you need to make the certs for production, and for
> testing purposes see the following sections / stanzas in the GitLab and
> Azure CI files.  Then the pytest part doesn't need to handle the case of
> running this on your own, since if you are, you need to have read the
> docs and followed instructions.

So if I understand you right, what you are suggesting is that we add
the generation of the files and certs using the artifacts feature for
the CI runs. And also dropping patch 9 of this series [1].

-sughosh

[1] - https://lists.denx.de/pipermail/u-boot/2023-July/524498.html
Tom Rini July 26, 2023, 8:06 p.m. UTC | #11
On Thu, Jul 27, 2023 at 01:30:04AM +0530, Sughosh Ganu wrote:
> On Wed, 26 Jul 2023 at 22:09, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Jul 26, 2023 at 08:11:44PM +0530, Sughosh Ganu wrote:
> > > hi Simon,
> > >
> > > On Wed, 26 Jul 2023 at 19:41, Simon Glass <sjg@chromium.org> wrote:
> > > >
> > > > Hi Tom,
> > > >
> > > > On Wed, 26 Jul 2023 at 07:23, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Jul 26, 2023 at 03:16:38PM +0530, Sughosh Ganu wrote:
> > > > > > On Wed, 26 Jul 2023 at 04:26, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Tue, Jul 25, 2023 at 04:52:38PM -0600, Simon Glass wrote:
> > > > > > > > On Tue, 25 Jul 2023 at 02:58, Sughosh Ganu <sughosh.ganu@linaro.org> wrote:
> > > > > > > > >
> > > > > > > > > Support has being added through earlier commits to build capsules
> > > > > > > > > and embed the public key needed for capsule authentication as part of
> > > > > > > > > u-boot build.
> > > > > > > > >
> > > > > > > > > From the testing point-of-view, this means the input files needed for
> > > > > > > > > generating the above have to be setup before invoking the build. Set
> > > > > > > > > this up in the CI configuration docker file for testing the capsule
> > > > > > > > > update feature.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Sughosh Ganu <sughosh.ganu@linaro.org>
> > > > > > > > > ---
> > > > > > > > > Changes since V4:
> > > > > > > > > * New patch which moves the setting up of the files needed for testing
> > > > > > > > >   the EFI capsule update feature to the Dockerfile.
> > > > > > > > >
> > > > > > > > > Note: Earlier, this setup was being done in the azure and gitlab yaml
> > > > > > > > > files. Now that this has been moved to the Dockerfile, this will
> > > > > > > > > require generating a new container image and referencing that image in
> > > > > > > > > the yaml files for the CI to work when these patches get applied.
> > > > > > > > >
> > > > > > > > >  tools/docker/Dockerfile | 12 ++++++++++++
> > > > > > > > >  1 file changed, 12 insertions(+)
> > > > > > > > >
> > > > > > > > > diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
> > > > > > > > > index 3d2b64a355..294a0b0a53 100644
> > > > > > > > > --- a/tools/docker/Dockerfile
> > > > > > > > > +++ b/tools/docker/Dockerfile
> > > > > > > > > @@ -206,6 +206,18 @@ RUN mkdir -p /opt/nokia && \
> > > > > > > > >         cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
> > > > > > > > >         rm -rf /tmp/qemu-linaro
> > > > > > > > >
> > > > > > > > > +# Set up capsule files for UEFI capsule update testing
> > > > > > > > > +RUN mkdir -p /tmp/capsules && \
> > > > > > > > > +    cd /tmp/capsules/ && \
> > > > > > > >
> > > > > > > > You can just use ${UBOOT_TRAVIS_BUILD_DIR} here
> > > > > > >
> > > > > > > That's not present in Dockerfiles, only at runtime within jobs (because
> > > > > > > we set it).
> > > > > >
> > > > > > I can copy the files into UBOOT_TRAVIS_BUILD_DIR as part of the job,
> > > > > > similar to what is being done for the grub*.efi files.
> > > > >
> > > > > Yes, copying the files rather than relying on them being in /tmp is
> > > > > better, but..
> > > > >
> > > > > > > > > +    echo -n "u-boot:Old" > u-boot.bin.old && \
> > > > > > > > > +    echo -n "u-boot:New" > u-boot.bin.new && \
> > > > > > > > > +    echo -n "u-boot-env:Old" > u-boot.env.old && \
> > > > > > > > > +    echo -n "u-boot-env:New" > u-boot.env.new && \
> > > > > > > >
> > > > > > > > We don't want these files, just the certs, since they are the things
> > > > > > > > that take a long time:
> > > > > > > >
> > > > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
> > > > > > > > > +    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
> > > > > > > > > +    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
> > > > > > > > > +    chmod -R uog+rw /tmp/capsules/
> > > > > > >
> > > > > > > How long does it even take to make these certs? I'm not sure it's great
> > > > > > > to make these and stage them in /tmp and expect them to be around at
> > > > > > > test time.
> > > > > >
> > > > > > Should I mimic what is being done for the various grub.efi files? I
> > > > > > believe that these are in the /opt/grub/ directory of the docker
> > > > > > image, and get copied to the build dir at runtime.
> > > > >
> > > > > It takes 10 minutes or so to build grub, and we use it in multiple
> > > > > tests.  Running openssl takes not even a second. Why are we doing this
> > > > > in the Dockerfile? Is this needed in more than one test? If so, does it
> > > > > matter if we have the same certs in each test?
> > > >
> > > > Yes it is actually much faster that I expected, so I suppose we can go
> > > > back to having it in the test itself, e.g. in a pytest fixture.
> > >
> > > If not part of the docker image, these commands will still have to run
> > > as part of the azure and gitlab yml files. They are needed before the
> > > build is invoked, as they serve as input files needed to generate the
> > > capsules as part of the build. The pytest fixtures would be run a)
> > > only for the pytest stages and b) after the build has completed.
> >
> > So lets just make use of the artifacts feature, generate these once per
> > CI run and pass them to the jobs that need them?  The documentation
> > should reflect that you need to make the certs for production, and for
> > testing purposes see the following sections / stanzas in the GitLab and
> > Azure CI files.  Then the pytest part doesn't need to handle the case of
> > running this on your own, since if you are, you need to have read the
> > docs and followed instructions.
> 
> So if I understand you right, what you are suggesting is that we add
> the generation of the files and certs using the artifacts feature for
> the CI runs. And also dropping patch 9 of this series [1].

Yes, we should be able to drop that patch in favor of a combination of
documentation and generating and passing it around as needed in CI.
diff mbox series

Patch

diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile
index 3d2b64a355..294a0b0a53 100644
--- a/tools/docker/Dockerfile
+++ b/tools/docker/Dockerfile
@@ -206,6 +206,18 @@  RUN mkdir -p /opt/nokia && \
 	cp /tmp/qemu-linaro/arm-softmmu/qemu-system-arm /opt/nokia && \
 	rm -rf /tmp/qemu-linaro
 
+# Set up capsule files for UEFI capsule update testing
+RUN mkdir -p /tmp/capsules && \
+    cd /tmp/capsules/ && \
+    echo -n "u-boot:Old" > u-boot.bin.old && \
+    echo -n "u-boot:New" > u-boot.bin.new && \
+    echo -n "u-boot-env:Old" > u-boot.env.old && \
+    echo -n "u-boot-env:New" > u-boot.env.new && \
+    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER.key -out SIGNER.crt -nodes -days 365 && \
+    openssl req -x509 -sha256 -newkey rsa:2048 -subj /CN=TEST_SIGNER/ -keyout SIGNER2.key -out SIGNER2.crt -nodes -days 365 && \
+    cert-to-efi-sig-list SIGNER.crt SIGNER.esl && \
+    chmod -R uog+rw /tmp/capsules/
+
 # Build genimage (required by some targets to generate disk images)
 RUN wget -O - https://github.com/pengutronix/genimage/releases/download/v14/genimage-14.tar.xz | tar -C /tmp -xJ && \
 	cd /tmp/genimage-14 && \