[v4,5/6] riscv: Copy the reserved-memory nodes to final DT

The DT used by U-Boot may be different from the DT being passed to
the OS if the DT is loaded from external media such as network or
mmc. In that case, the reserved-memory node needs to be copied to
the DT passed to the OS.

Signed-off-by: Atish Patra <atish.patra at wdc.com>
Reviewed-by: Bin Meng <bmeng.cn at gmail.com>
---
 arch/riscv/lib/bootm.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

On 3/24/20 5:16 AM, Atish Patra wrote:
> The DT used by U-Boot may be different from the DT being passed to
> the OS if the DT is loaded from external media such as network or
> mmc. In that case, the reserved-memory node needs to be copied to
> the DT passed to the OS.

The bootefi command works in the following sequence:

* copy the passed fdt of if none is passed the internal fdt
* call image_setup_libfdt() for the copy
* create memory reservations in the UEFI memory map

Cf.
fef907b2e440 ("efi_loader: create reservations after ft_board_setup")
7be64b885a36 ("cmd: bootefi: Parse reserved-memory node from DT")

image_setup_libfdt() executes among others:

* fdt_chosen()
* arch_fixup_fdt()

Please, explain why you need to copy memory reservations, if
arch_fixup_fdt() is executed inside the bootefi command:

What would be the source of memory reservation that you would otherwise
miss in the final device tree? Do you expect that the Linux device tree
would lack reservation that are generated on the fly before U-Boot?

Best regards

Heinrich

>
> Signed-off-by: Atish Patra <atish.patra at wdc.com>
> Reviewed-by: Bin Meng <bmeng.cn at gmail.com>
> ---
>   arch/riscv/lib/bootm.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c
> index 87cadad5016d..8ff8db6bf533 100644
> --- a/arch/riscv/lib/bootm.c
> +++ b/arch/riscv/lib/bootm.c
> @@ -28,8 +28,8 @@ __weak void board_quiesce_devices(void)
>
>   int arch_fixup_fdt(void *blob)
>   {
> -#ifdef CONFIG_EFI_LOADER
>   	int err;
> +#ifdef CONFIG_EFI_LOADER
>   	u32 size;
>   	int chosen_offset;
>
> @@ -50,6 +50,12 @@ int arch_fixup_fdt(void *blob)
>   	/* Overwrite the boot-hartid as U-Boot is the last stage BL */
>   	fdt_setprop_u32(blob, chosen_offset, "boot-hartid", gd->arch.boot_hart);
>   #endif
> +
> +	/* Copy the reserved-memory node to the DT used by OS */
> +	err = riscv_fdt_copy_resv_mem_node(gd->fdt_blob, blob);
> +	if (err < 0)
> +		return err;
> +
>   	return 0;
>   }
>
>

On Mon, Mar 23, 2020 at 11:23 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>
> On 3/24/20 5:16 AM, Atish Patra wrote:
> > The DT used by U-Boot may be different from the DT being passed to
> > the OS if the DT is loaded from external media such as network or
> > mmc. In that case, the reserved-memory node needs to be copied to
> > the DT passed to the OS.
>
> The bootefi command works in the following sequence:
>
> * copy the passed fdt of if none is passed the internal fdt
> * call image_setup_libfdt() for the copy
> * create memory reservations in the UEFI memory map
>
> Cf.
> fef907b2e440 ("efi_loader: create reservations after ft_board_setup")
> 7be64b885a36 ("cmd: bootefi: Parse reserved-memory node from DT")
>
> image_setup_libfdt() executes among others:
>
> * fdt_chosen()
> * arch_fixup_fdt()
>
> Please, explain why you need to copy memory reservations, if
> arch_fixup_fdt() is executed inside the bootefi command:
>

As per my understanding, there can be two different DTs in U-Boot.
1. Internal device tree or DT passed from previous stage (gd->fdt_blob). This
    device tree is being used to boot OS as well.
2. User can load the DT from MMC/network to any valid address and use
that address in bootefi/booti command line to boot OS.

efi_install_fdt copies DT from the given user address (via cmdline) and copies
to the efi memory.

There is no need to 2nd copy for case 1 as the DT already contains the
reserved-memory node. In case 2, the DT loaded from external media may not
have the reserved memory regions set by the firmware. That's why we need to
copy the reserved-memory nodes. Now, arch_fixup_fdt() is the last place to edit
the DT passed to OS. The copy operation has to be done in arch_fixup_fdt.
For case 1, fdtdec_add_reserved_memory will simply return without
modifying the DT.

> What would be the source of memory reservation that you would otherwise
> miss in the final device tree? Do you expect that the Linux device tree
> would lack reservation that are generated on the fly before U-Boot?
>
> Best regards
>
> Heinrich
>
> >
> > Signed-off-by: Atish Patra <atish.patra at wdc.com>
> > Reviewed-by: Bin Meng <bmeng.cn at gmail.com>
> > ---
> >   arch/riscv/lib/bootm.c | 8 +++++++-
> >   1 file changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c
> > index 87cadad5016d..8ff8db6bf533 100644
> > --- a/arch/riscv/lib/bootm.c
> > +++ b/arch/riscv/lib/bootm.c
> > @@ -28,8 +28,8 @@ __weak void board_quiesce_devices(void)
> >
> >   int arch_fixup_fdt(void *blob)
> >   {
> > -#ifdef CONFIG_EFI_LOADER
> >       int err;
> > +#ifdef CONFIG_EFI_LOADER
> >       u32 size;
> >       int chosen_offset;
> >
> > @@ -50,6 +50,12 @@ int arch_fixup_fdt(void *blob)
> >       /* Overwrite the boot-hartid as U-Boot is the last stage BL */
> >       fdt_setprop_u32(blob, chosen_offset, "boot-hartid", gd->arch.boot_hart);
> >   #endif
> > +
> > +     /* Copy the reserved-memory node to the DT used by OS */
> > +     err = riscv_fdt_copy_resv_mem_node(gd->fdt_blob, blob);
> > +     if (err < 0)
> > +             return err;
> > +
> >       return 0;
> >   }
> >
> >
>

On 3/24/20 8:15 AM, Atish Patra wrote:
> On Mon, Mar 23, 2020 at 11:23 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>>
>> On 3/24/20 5:16 AM, Atish Patra wrote:
>>> The DT used by U-Boot may be different from the DT being passed to
>>> the OS if the DT is loaded from external media such as network or
>>> mmc. In that case, the reserved-memory node needs to be copied to
>>> the DT passed to the OS.
>>
>> The bootefi command works in the following sequence:
>>
>> * copy the passed fdt of if none is passed the internal fdt
>> * call image_setup_libfdt() for the copy
>> * create memory reservations in the UEFI memory map
>>
>> Cf.
>> fef907b2e440 ("efi_loader: create reservations after ft_board_setup")
>> 7be64b885a36 ("cmd: bootefi: Parse reserved-memory node from DT")
>>
>> image_setup_libfdt() executes among others:
>>
>> * fdt_chosen()
>> * arch_fixup_fdt()
>>
>> Please, explain why you need to copy memory reservations, if
>> arch_fixup_fdt() is executed inside the bootefi command:
>>
>
> As per my understanding, there can be two different DTs in U-Boot.
> 1. Internal device tree or DT passed from previous stage (gd->fdt_blob). This
>      device tree is being used to boot OS as well.
> 2. User can load the DT from MMC/network to any valid address and use
> that address in bootefi/booti command line to boot OS.
>
> efi_install_fdt copies DT from the given user address (via cmdline) and copies
> to the efi memory.
>
> There is no need to 2nd copy for case 1 as the DT already contains the
> reserved-memory node. In case 2, the DT loaded from external media may not
> have the reserved memory regions set by the firmware. That's why we need to
> copy the reserved-memory nodes. Now, arch_fixup_fdt() is the last place to edit
> the DT passed to OS. The copy operation has to be done in arch_fixup_fdt.
> For case 1, fdtdec_add_reserved_memory will simply return without
> modifying the DT.

There are three cases of device trees:

* U-Boot's internal device trees
* device trees provided by a earlier stage firmware to U-Boot
   (e.g. on QEMU)
* device trees supplied by the user

We cannot use U-Boot's internal device tree directly because a program
called via bootefi may manipulate the device tree and return afterwards.

As arch_fixup_fdt is called by all boot commands there is no need to
insert reserved memory nodes at an earlier stage.

The copy operation is only needed for bootefi as the other boot commands
cannot return. So it does not make to me to move the device tree copying
to arch_fixup_fdt().

Best regards

Heinrich

>
>> What would be the source of memory reservation that you would otherwise
>> miss in the final device tree? Do you expect that the Linux device tree
>> would lack reservation that are generated on the fly before U-Boot?
>>
>> Best regards
>>
>> Heinrich
>>
>>>
>>> Signed-off-by: Atish Patra <atish.patra at wdc.com>
>>> Reviewed-by: Bin Meng <bmeng.cn at gmail.com>
>>> ---
>>>    arch/riscv/lib/bootm.c | 8 +++++++-
>>>    1 file changed, 7 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c
>>> index 87cadad5016d..8ff8db6bf533 100644
>>> --- a/arch/riscv/lib/bootm.c
>>> +++ b/arch/riscv/lib/bootm.c
>>> @@ -28,8 +28,8 @@ __weak void board_quiesce_devices(void)
>>>
>>>    int arch_fixup_fdt(void *blob)
>>>    {
>>> -#ifdef CONFIG_EFI_LOADER
>>>        int err;
>>> +#ifdef CONFIG_EFI_LOADER
>>>        u32 size;
>>>        int chosen_offset;
>>>
>>> @@ -50,6 +50,12 @@ int arch_fixup_fdt(void *blob)
>>>        /* Overwrite the boot-hartid as U-Boot is the last stage BL */
>>>        fdt_setprop_u32(blob, chosen_offset, "boot-hartid", gd->arch.boot_hart);
>>>    #endif
>>> +
>>> +     /* Copy the reserved-memory node to the DT used by OS */
>>> +     err = riscv_fdt_copy_resv_mem_node(gd->fdt_blob, blob);
>>> +     if (err < 0)
>>> +             return err;
>>> +
>>>        return 0;
>>>    }
>>>
>>>
>>
>
>

On Tue, Mar 24, 2020 at 3:11 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>
> On 3/24/20 8:15 AM, Atish Patra wrote:
> > On Mon, Mar 23, 2020 at 11:23 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> >>
> >> On 3/24/20 5:16 AM, Atish Patra wrote:
> >>> The DT used by U-Boot may be different from the DT being passed to
> >>> the OS if the DT is loaded from external media such as network or
> >>> mmc. In that case, the reserved-memory node needs to be copied to
> >>> the DT passed to the OS.
> >>
> >> The bootefi command works in the following sequence:
> >>
> >> * copy the passed fdt of if none is passed the internal fdt
> >> * call image_setup_libfdt() for the copy
> >> * create memory reservations in the UEFI memory map
> >>
> >> Cf.
> >> fef907b2e440 ("efi_loader: create reservations after ft_board_setup")
> >> 7be64b885a36 ("cmd: bootefi: Parse reserved-memory node from DT")
> >>
> >> image_setup_libfdt() executes among others:
> >>
> >> * fdt_chosen()
> >> * arch_fixup_fdt()
> >>
> >> Please, explain why you need to copy memory reservations, if
> >> arch_fixup_fdt() is executed inside the bootefi command:
> >>
> >
> > As per my understanding, there can be two different DTs in U-Boot.
> > 1. Internal device tree or DT passed from previous stage (gd->fdt_blob). This
> >      device tree is being used to boot OS as well.
> > 2. User can load the DT from MMC/network to any valid address and use
> > that address in bootefi/booti command line to boot OS.
> >
> > efi_install_fdt copies DT from the given user address (via cmdline) and copies
> > to the efi memory.
> >
> > There is no need to 2nd copy for case 1 as the DT already contains the
> > reserved-memory node. In case 2, the DT loaded from external media may not
> > have the reserved memory regions set by the firmware. That's why we need to
> > copy the reserved-memory nodes. Now, arch_fixup_fdt() is the last place to edit
> > the DT passed to OS. The copy operation has to be done in arch_fixup_fdt.
> > For case 1, fdtdec_add_reserved_memory will simply return without
> > modifying the DT.
>
> There are three cases of device trees:
>
> * U-Boot's internal device trees
> * device trees provided by a earlier stage firmware to U-Boot
>    (e.g. on QEMU)
> * device trees supplied by the user
>
Yes. I combined 1st & 2nd case into one.

> We cannot use U-Boot's internal device tree directly because a program
> called via bootefi may manipulate the device tree and return afterwards.
>
> As arch_fixup_fdt is called by all boot commands there is no need to
> insert reserved memory nodes at an earlier stage.
>
> The copy operation is only needed for bootefi as the other boot commands
> cannot return. So it does not make to me to move the device tree copying
> to arch_fixup_fdt().
>

This series is aimed at providing reserved memory support all boot
commands in RISC-V.
All other boot commands booti/bootm also load the device tree supplied
by the user.
Hence, we need to copy the reserved memory nodes so that proper kernel
sees those DT
nodes.

The code path for booti/bootm is
image_setup_linux->image_setup_libfdt->arch_fixup_fdt

This is executed while executing booti/bootm and fixes the DT that
passed to the OS.

> Best regards
>
> Heinrich
>
> >
> >> What would be the source of memory reservation that you would otherwise
> >> miss in the final device tree? Do you expect that the Linux device tree
> >> would lack reservation that are generated on the fly before U-Boot?
> >>
> >> Best regards
> >>
> >> Heinrich
> >>
> >>>
> >>> Signed-off-by: Atish Patra <atish.patra at wdc.com>
> >>> Reviewed-by: Bin Meng <bmeng.cn at gmail.com>
> >>> ---
> >>>    arch/riscv/lib/bootm.c | 8 +++++++-
> >>>    1 file changed, 7 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c
> >>> index 87cadad5016d..8ff8db6bf533 100644
> >>> --- a/arch/riscv/lib/bootm.c
> >>> +++ b/arch/riscv/lib/bootm.c
> >>> @@ -28,8 +28,8 @@ __weak void board_quiesce_devices(void)
> >>>
> >>>    int arch_fixup_fdt(void *blob)
> >>>    {
> >>> -#ifdef CONFIG_EFI_LOADER
> >>>        int err;
> >>> +#ifdef CONFIG_EFI_LOADER
> >>>        u32 size;
> >>>        int chosen_offset;
> >>>
> >>> @@ -50,6 +50,12 @@ int arch_fixup_fdt(void *blob)
> >>>        /* Overwrite the boot-hartid as U-Boot is the last stage BL */
> >>>        fdt_setprop_u32(blob, chosen_offset, "boot-hartid", gd->arch.boot_hart);
> >>>    #endif
> >>> +
> >>> +     /* Copy the reserved-memory node to the DT used by OS */
> >>> +     err = riscv_fdt_copy_resv_mem_node(gd->fdt_blob, blob);
> >>> +     if (err < 0)
> >>> +             return err;
> >>> +
> >>>        return 0;
> >>>    }
> >>>
> >>>
> >>
> >
> >
>