| Message ID | 20231228115804.3626579-6-sumit.garg@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | An effort to bring DT bindings compliance within U-Boot | expand |
On Thu, Dec 28, 2023 at 05:28:01PM +0530, Sumit Garg wrote: > Encourage SoC/board maintainers to migrate to using devicetree-rebasing > subtree and maintain a regular sync with Linux kernel devicetree files > and bindings. > > Along with that add documentation regarding how to run DT bindings > schema checks. > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> Thanks for the related cleanups in there. Reviewed-by: Tom Rini <trini@konsulko.com>
Hi Sumit, On Thu, Dec 28, 2023 at 11:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > Encourage SoC/board maintainers to migrate to using devicetree-rebasing > subtree and maintain a regular sync with Linux kernel devicetree files > and bindings. > > Along with that add documentation regarding how to run DT bindings > schema checks. > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > --- > > Changes in v3: > -------------- > - Replace CONFIG_* with Kconfig options > > Changes in v2: > -------------- > - s/U-boot/U-Boot/ > > doc/develop/devicetree/control.rst | 131 +++++++++++++++++++++-------- > 1 file changed, 96 insertions(+), 35 deletions(-) > > diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst > index cbb65c9b177..65180d33e8d 100644 > --- a/doc/develop/devicetree/control.rst > +++ b/doc/develop/devicetree/control.rst > @@ -1,5 +1,6 @@ > .. SPDX-License-Identifier: GPL-2.0+ > .. sectionauthor:: Copyright 2011 The Chromium OS Authors > +.. Copyright 2023 Linaro Ltd. > > Devicetree Control in U-Boot > ============================ > @@ -22,14 +23,13 @@ for three reasons: > hierarchical format > - It is fairly efficient to read incrementally > > -The arch/<arch>/dts directories contains a Makefile for building the devicetree > -blob and embedding it in the U-Boot image. This is useful since it allows > -U-Boot to configure itself according to what it finds there. If you have > -a number of similar boards with different peripherals, you can describe > -the features of each board in the devicetree file, and have a single > -generic source base. > +The U-Boot Makefile infrastructure allows for building the devicetree blob > +and embedding it in the U-Boot image. This is useful since it allows U-Boot > +to configure itself according to what it finds there. If you have a number > +of similar boards with different peripherals, you can describe the features > +of each board in the devicetree file, and have a single generic source base. > > -To enable this feature, add CONFIG_OF_CONTROL to your board config file. > +To enable this feature, select `OF_CONTROL` via Kconfig. These changes to remove CONFIG_ are good, but I think they should be in a separate, prior patch. > > > What is a Flattened Devicetree? > @@ -68,8 +68,21 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. > Where do I get a devicetree file for my board? > ---------------------------------------------- > > -You may find that the Linux kernel has a suitable file. Look in the > -kernel source in arch/<arch>/boot/dts. > +Linux kernel Git repository has been the place where devicetree files along > +with devicetree bindings are stored and maintained. There is devicetee-rebasing > +(dtrepo_) which maintains a forked copy of devicetree files along with bindings > +at every Linux kernel major release or intermideate release candidates. > + > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > +kept updated with every new kernel major release via subtree pull as follows:: > + > + git subtree pull --prefix devicetree-rebasing \ > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > + <release-tag> --squash > + This is an internal detail and I believe it will confuse people. Can you put this in a section like 'Resyncing with devicetree-rebasing' and explain when this might be needed?...almost enver, right? > +You may find that the `devicetee-rebasing/` sub-directory has a suitable > +devicetree file for your board. Look in `devicetree-rebasing/src/<arch>/`. > > If not you might find other boards with suitable files that you can > modify to your needs. Look in the board directories for files with a > @@ -81,34 +94,34 @@ Failing that, you could write one from scratch yourself! > Configuration > ------------- > > -Use:: > +Traditionally, U-Boot placed copies of devicetree source files from Linux > +kernel into `arch/<arch>/dts/<name>.dts` which can be selected via setting > +"<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. > > - #define CONFIG_DEFAULT_DEVICE_TREE "<name>" > +However, it has become combersome over time for each SoC/board maintainer to spelling > +keep devicetree files in sync with Linux kernel. Thereby, SoC/board maintainers s/Thereby/Therefore/ > +are encouraged to migrate to use mirrored copies from `devicetree-rebasing/` > +into `dts/arch/<arch>/<vendor>`. To do that enable `OF_UPSTREAM` via Kconfig for the SoC being used > and > +set up "<vendor>/<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. > > -to set the filename of the devicetree source. Then put your devicetree > -file into:: > +This should include your CPU or SOC's devicetree file. On top of that any U-Boot > +specific tweaks (see: dttweaks_) can be made for your board. > > - arch/<arch>/dts/<name>.dts > - > -This should include your CPU or SOC's devicetree file, placed in > -`arch/<arch>/dts`, and then make any adjustments required using a u-boot-dtsi > -file for your board. > - > -If CONFIG_OF_EMBED is defined, then it will be picked up and built into > +If `OF_EMBED` is selected by Kconfig, then it will be picked up and built into > the U-Boot image (including u-boot.bin). This is suitable for debugging > and development only and is not recommended for production devices. > > -If CONFIG_OF_SEPARATE is defined, then it will be built and placed in > +If `OF_SEPARATE` is selected by Kconfig, then it will be built and placed in > a u-boot.dtb file alongside u-boot-nodtb.bin with the combined result placed > -in u-boot.bin so you can still just flash u-boot.bin onto your board. If you are > -using CONFIG_SPL_FRAMEWORK, then u-boot.img will be built to include the device > -tree binary. > +in u-boot.bin so you can still just flash u-boot.bin onto your board. If Kconfig > +option `SPL_FRAMEWORK` is enabled, then u-boot.img will be built to include the > +device tree binary. > > -If CONFIG_OF_BOARD is defined, a board-specific routine will provide the > +If `OF_BOARD` is selected by Kconfig, a board-specific routine will provide the > devicetree at runtime, for example if an earlier bootloader stage creates > it and passes it to U-Boot. > > -If CONFIG_SANDBOX is defined, then it will be read from a file on > +If `SANDBOX` is selected by Kconfig, then it will be read from a file on > startup. Use the -d flag to U-Boot to specify the file to read, -D for the > default and -T for the test devicetree, used to run sandbox unit tests. As above, please move the CONFIG changes into a prior patch. > > @@ -142,7 +155,7 @@ Build: > After the board configuration is done, fdt supported u-boot can be built in two > ways: > > -# build the default dts which is defined from CONFIG_DEFAULT_DEVICE_TREE:: > +# build the default dts which is selected by DEFAULT_DEVICE_TREE Kconfig:: > > $ make > > @@ -156,8 +169,9 @@ ways: > Adding tweaks for U-Boot > ------------------------ > > -It is strongly recommended that devicetree files in U-Boot are an exact copy of > -those in Linux, so that it is easy to sync them up from time to time. > +With devicetee-rebasing Git subtree, it is ensured that devicetree files in > +U-Boot are an exact copy of those in Linux kernel via mirroring into > +`dts/arch/<arch>/<vendor>`. > > U-Boot is of course a very different project from Linux, e.g. it operates under > much more restrictive memory and code-size constraints. Where Linux may use a > @@ -170,8 +184,8 @@ constraints are even more extreme and the devicetree is shrunk to remove > unwanted nodes, or even turned into C code to avoid access overhead. > > U-Boot automatically looks for and includes a file with updates to the standard > -devicetree for your board, searching for them in the same directory as the > -main file, in this order:: > +devicetree for your board, searching for them in `arch/<arch>/dts/` in this > +order:: > > <orig_filename>-u-boot.dtsi > <CONFIG_SYS_SOC>-u-boot.dtsi > @@ -195,11 +209,57 @@ As mentioned above, the U-Boot build system automatically includes a > `*-u-boot.dtsi` file, if found, containing U-Boot specific > quirks. However, some data, such as the mentioned public keys, are not > appropriate for upstream U-Boot but are better kept and maintained > -outside the U-Boot repository. You can use CONFIG_DEVICE_TREE_INCLUDES > -to specify a list of .dtsi files that will also be included when > +outside the U-Boot repository. You can use `DEVICE_TREE_INCLUDES` Kconfig > +option to specify a list of .dtsi files that will also be included when > building .dtb files. > > > +Devicetree bindings schema checks > +--------------------------------- > + > +With devicetee-rebasing Git subtree, the devicetree bindings are also regularly > +synced with Linux kernel as `devicetree-rebasing/Bindings/` sub-directory. This > +allows U-Boot to run devicetree bindings schema checks which will bring > +compliance to U-Boot core/drivers regarding usage of devicetree. > + > +Dependencies > +~~~~~~~~~~~~ > + > +The DT schema project must be installed in order to validate the DT schema > +binding documents and validate DTS files using the DT schema. The DT schema > +project can be installed with pip:: > + > + pip3 install dtschema > + > +Note that 'dtschema' installation requires 'swig' and Python development files > +installed first. On Debian/Ubuntu systems:: > + > + apt install swig python3-dev These two are already in doc/build/gcc.rst so please reference that here, rather than repeating it. > + > +Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be > +installed. Ensure they are in your PATH (~/.local/bin by default). > + > +Recommended is also to install yamllint (used by dtschema when present). How? > + > +Running checks > +~~~~~~~~~~~~~~ > + > +In order to perform validation of DTB files, use the ``dtbs_check`` target:: > + > + make dtbs_check > + > +It is also possible to run checks with a subset of matching schema files by > +setting the ``DT_SCHEMA_FILES`` variable to 1 or more specific schema files or > +patterns (partial match of a fixed string). Each file or pattern should be > +separated by ':'. > + > +:: > + > + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml:rtc.yaml > + make dtbs_check DT_SCHEMA_FILES=/gpio/ > + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml > + > + > Relocation, SPL and TPL > ----------------------- > > @@ -261,8 +321,9 @@ used it before Linux (e.g. snow). The two projects developed in parallel > and there are still some differences in the bindings for certain boards. > While there has been discussion of having a separate repository for devicetree > files, in practice the Linux kernel Git repository has become the place where > -these are stored, with U-Boot taking copies and adding tweaks with u-boot.dtsi > -files. > +these are stored, with U-Boot taking copies via devicetree-rebasing repo > +(see: dtrepo_) and adding tweaks with u-boot.dtsi files. > > .. _dtspec: https://www.devicetree.org/specifications/ > .. _dtlist: https://www.spinics.net/lists/devicetree-compiler/ > +.. _dtrepo: https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git > -- > 2.34.1 > Regards, Simon
On Thu, Dec 28, 2023 at 01:37:09PM +0000, Simon Glass wrote: > Hi Sumit, > > On Thu, Dec 28, 2023 at 11:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: [snip] > > @@ -68,8 +68,21 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. > > Where do I get a devicetree file for my board? > > ---------------------------------------------- > > > > -You may find that the Linux kernel has a suitable file. Look in the > > -kernel source in arch/<arch>/boot/dts. > > +Linux kernel Git repository has been the place where devicetree files along > > +with devicetree bindings are stored and maintained. There is devicetee-rebasing > > +(dtrepo_) which maintains a forked copy of devicetree files along with bindings > > +at every Linux kernel major release or intermideate release candidates. > > + > > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > +kept updated with every new kernel major release via subtree pull as follows:: > > + > > + git subtree pull --prefix devicetree-rebasing \ > > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > + <release-tag> --squash > > + > > This is an internal detail and I believe it will confuse people. Can > you put this in a section like 'Resyncing with devicetree-rebasing' > and explain when this might be needed?...almost enver, right? I think this means the wording and sections need to be clearer, in the document. It's unlikely most users would need this (but I can see downstream cases), but I need to do this at least once a U-Boot release cycle, and I want to make sure it's documented on how to do it.
On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > Encourage SoC/board maintainers to migrate to using devicetree-rebasing > subtree and maintain a regular sync with Linux kernel devicetree files > and bindings. > > Along with that add documentation regarding how to run DT bindings > schema checks. > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > --- > > Changes in v3: > -------------- > - Replace CONFIG_* with Kconfig options > > Changes in v2: > -------------- > - s/U-boot/U-Boot/ > > doc/develop/devicetree/control.rst | 131 +++++++++++++++++++++-------- > 1 file changed, 96 insertions(+), 35 deletions(-) > > diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst > index cbb65c9b177..65180d33e8d 100644 > --- a/doc/develop/devicetree/control.rst > +++ b/doc/develop/devicetree/control.rst > @@ -1,5 +1,6 @@ > .. SPDX-License-Identifier: GPL-2.0+ > .. sectionauthor:: Copyright 2011 The Chromium OS Authors > +.. Copyright 2023 Linaro Ltd. > > Devicetree Control in U-Boot > ============================ > @@ -22,14 +23,13 @@ for three reasons: > hierarchical format > - It is fairly efficient to read incrementally > > -The arch/<arch>/dts directories contains a Makefile for building the devicetree > -blob and embedding it in the U-Boot image. This is useful since it allows > -U-Boot to configure itself according to what it finds there. If you have > -a number of similar boards with different peripherals, you can describe > -the features of each board in the devicetree file, and have a single > -generic source base. > +The U-Boot Makefile infrastructure allows for building the devicetree blob > +and embedding it in the U-Boot image. This is useful since it allows U-Boot > +to configure itself according to what it finds there. If you have a number > +of similar boards with different peripherals, you can describe the features > +of each board in the devicetree file, and have a single generic source base. > > -To enable this feature, add CONFIG_OF_CONTROL to your board config file. > +To enable this feature, select `OF_CONTROL` via Kconfig. > > > What is a Flattened Devicetree? > @@ -68,8 +68,21 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. > Where do I get a devicetree file for my board? > ---------------------------------------------- > > -You may find that the Linux kernel has a suitable file. Look in the > -kernel source in arch/<arch>/boot/dts. > +Linux kernel Git repository has been the place where devicetree files along > +with devicetree bindings are stored and maintained. There is devicetee-rebasing > +(dtrepo_) which maintains a forked copy of devicetree files along with bindings > +at every Linux kernel major release or intermideate release candidates. Typo. > + > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > +kept updated with every new kernel major release via subtree pull as follows:: I would suggest dropping "-rebasing" in the u-boot tree. (I wish we had in the original repo). I don't think it's relevant. We're not likely to regenerate the tree, but any clue what 'git subtree pull' would do in this case? It could happen if we switched to git-filter-repo. > + > + git subtree pull --prefix devicetree-rebasing \ > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > + <release-tag> --squash I'd put this in a script to run. Documentation tends to be not quite correct. A script could also be smarter and figure out <release-tag>. > + > +You may find that the `devicetee-rebasing/` sub-directory has a suitable > +devicetree file for your board. Look in `devicetree-rebasing/src/<arch>/`.
On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: [snip] > > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > +kept updated with every new kernel major release via subtree pull as follows:: > > I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > had in the original repo). I don't think it's relevant. > > We're not likely to regenerate the tree, but any clue what 'git > subtree pull' would do in this case? It could happen if we switched to > git-filter-repo. > > > + > > + git subtree pull --prefix devicetree-rebasing \ > > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > + <release-tag> --squash > > I'd put this in a script to run. Documentation tends to be not quite > correct. A script could also be smarter and figure out <release-tag>. Since you had mentioned elsewhere moving the kernel scripts/dtc/ to something using subtree, I do hope to learn some lessons from what you do there. The first thing is that given the size/nature of the commits, I had figured I'd be the one doing this as a subtree pull+squash then push, rather than posting to the ML since it'll be huge and un-reviewable, but with a note sent off to the ML that people should be aware the next sync has been done and retest as needed. But Sumit, were you planning to do some of this instead? The second thing is that if we move the subtree part in to dts/ instead (where we will still have the Makefile/Kconfig we have today and then our Makefiles within the directories, this might get more complex and so really require a script to keep it from getting error prone?
On Wed, 3 Jan 2024 at 22:02, Tom Rini <trini@konsulko.com> wrote: > > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > > On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > [snip] > > > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > > +kept updated with every new kernel major release via subtree pull as follows:: > > > > I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > > had in the original repo). I don't think it's relevant. Sure, as Tom pointed out below that we would shift to put subtree as dts/upstream for v4. Does that make sense to you? > > > > We're not likely to regenerate the tree, but any clue what 'git > > subtree pull' would do in this case? It could happen if we switched to > > git-filter-repo. Yeah we just need to modify the prefix as: git subtree pull --prefix dts/upstream \ git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ <release-tag> --squash > > > > > + > > > + git subtree pull --prefix devicetree-rebasing \ > > > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > > + <release-tag> --squash > > > > I'd put this in a script to run. Documentation tends to be not quite > > correct. A script could also be smarter and figure out <release-tag>. Sure, I will do that for v4. > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > something using subtree, I do hope to learn some lessons from what you > do there. The first thing is that given the size/nature of the commits, > I had figured I'd be the one doing this as a subtree pull+squash then > push, rather than posting to the ML since it'll be huge and > un-reviewable, but with a note sent off to the ML that people should be > aware the next sync has been done and retest as needed. But Sumit, were > you planning to do some of this instead? I am happy for you to do that. I am happy to assist in case any conflicts occur. > The second thing is that if we > move the subtree part in to dts/ instead (where we will still have the > Makefile/Kconfig we have today and then our Makefiles within the > directories, this might get more complex and so really require a script > to keep it from getting error prone? IMO, we should give subtree a try and later if it becomes complex to manage then we can switch to a custom script as well. It is essentially a sub-directory which we start initially to manage via a subtree approach but later we can switch to a custom script to sync that subdirectory if there is a need. -Sumit > > -- > Tom
On 1/3/24 17:32, Tom Rini wrote: > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: >> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > [snip] >>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for >>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly >>> +kept updated with every new kernel major release via subtree pull as follows:: >> >> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we >> had in the original repo). I don't think it's relevant. >> >> We're not likely to regenerate the tree, but any clue what 'git >> subtree pull' would do in this case? It could happen if we switched to >> git-filter-repo. >> >>> + >>> + git subtree pull --prefix devicetree-rebasing \ >>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ >>> + <release-tag> --squash >> >> I'd put this in a script to run. Documentation tends to be not quite >> correct. A script could also be smarter and figure out <release-tag>. > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > something using subtree, I do hope to learn some lessons from what you > do there. The first thing is that given the size/nature of the commits, > I had figured I'd be the one doing this as a subtree pull+squash then > push, rather than posting to the ML since it'll be huge and > un-reviewable, but with a note sent off to the ML that people should be > aware the next sync has been done and retest as needed. But Sumit, were > you planning to do some of this instead? The second thing is that if we > move the subtree part in to dts/ instead (where we will still have the > Makefile/Kconfig we have today and then our Makefiles within the > directories, this might get more complex and so really require a script > to keep it from getting error prone? I played with this series and didn't really have time to dig into subtree mechanics but one thing I see is that when squash and merge happens you can't do simple rebase anymore which is time to time very useful. I see some different rebase strategies and --rebase-merges option but if this is adopted it should be properly described how to rebase u-boot tree which contains some subtrees. Thanks, Michal
Hi Michal, On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: > > > > On 1/3/24 17:32, Tom Rini wrote: > > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > >> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > [snip] > >>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > >>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > >>> +kept updated with every new kernel major release via subtree pull as follows:: > >> > >> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > >> had in the original repo). I don't think it's relevant. > >> > >> We're not likely to regenerate the tree, but any clue what 'git > >> subtree pull' would do in this case? It could happen if we switched to > >> git-filter-repo. > >> > >>> + > >>> + git subtree pull --prefix devicetree-rebasing \ > >>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > >>> + <release-tag> --squash > >> > >> I'd put this in a script to run. Documentation tends to be not quite > >> correct. A script could also be smarter and figure out <release-tag>. > > > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > > something using subtree, I do hope to learn some lessons from what you > > do there. The first thing is that given the size/nature of the commits, > > I had figured I'd be the one doing this as a subtree pull+squash then > > push, rather than posting to the ML since it'll be huge and > > un-reviewable, but with a note sent off to the ML that people should be > > aware the next sync has been done and retest as needed. But Sumit, were > > you planning to do some of this instead? The second thing is that if we > > move the subtree part in to dts/ instead (where we will still have the > > Makefile/Kconfig we have today and then our Makefiles within the > > directories, this might get more complex and so really require a script > > to keep it from getting error prone? > > I played with this series and didn't really have time to dig into subtree > mechanics but one thing I see is that when squash and merge happens you can't do > simple rebase anymore which is time to time very useful. > I see some different rebase strategies and --rebase-merges option but if this is > adopted it should be properly described how to rebase u-boot tree which contains > some subtrees. Did you observe any specific difference with respect to subtree merge commits (Tom will likely do those in the beginning of next release cycle) when compared with normal merge commits when Tom pulls in code from different custodians like one example below? The same rebasing rules should apply to both types of merge commits. commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) Merge: 2b28c3b871c e266d273114 Author: Tom Rini <trini@konsulko.com> Date: Mon Jan 1 12:38:15 2024 -0500 Merge tag 'dm-next-1124' of https://source.denx.de/u-boot/custodians/u-boot-dm into next support propagating supernode properties with bootph schema align bloblist with v0.9 of Firmware Handoff spec -Sumit > > Thanks, > Michal
On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: > Hi Michal, > > On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: > > > > > > > > On 1/3/24 17:32, Tom Rini wrote: > > > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > > >> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > > [snip] > > >>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > >>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > >>> +kept updated with every new kernel major release via subtree pull as follows:: > > >> > > >> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > > >> had in the original repo). I don't think it's relevant. > > >> > > >> We're not likely to regenerate the tree, but any clue what 'git > > >> subtree pull' would do in this case? It could happen if we switched to > > >> git-filter-repo. > > >> > > >>> + > > >>> + git subtree pull --prefix devicetree-rebasing \ > > >>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > >>> + <release-tag> --squash > > >> > > >> I'd put this in a script to run. Documentation tends to be not quite > > >> correct. A script could also be smarter and figure out <release-tag>. > > > > > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > > > something using subtree, I do hope to learn some lessons from what you > > > do there. The first thing is that given the size/nature of the commits, > > > I had figured I'd be the one doing this as a subtree pull+squash then > > > push, rather than posting to the ML since it'll be huge and > > > un-reviewable, but with a note sent off to the ML that people should be > > > aware the next sync has been done and retest as needed. But Sumit, were > > > you planning to do some of this instead? The second thing is that if we > > > move the subtree part in to dts/ instead (where we will still have the > > > Makefile/Kconfig we have today and then our Makefiles within the > > > directories, this might get more complex and so really require a script > > > to keep it from getting error prone? > > > > I played with this series and didn't really have time to dig into subtree > > mechanics but one thing I see is that when squash and merge happens you can't do > > simple rebase anymore which is time to time very useful. > > I see some different rebase strategies and --rebase-merges option but if this is > > adopted it should be properly described how to rebase u-boot tree which contains > > some subtrees. > > Did you observe any specific difference with respect to subtree merge > commits (Tom will likely do those in the beginning of next release > cycle) when compared with normal merge commits when Tom pulls in code > from different custodians like one example below? The same rebasing > rules should apply to both types of merge commits. > > commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) > Merge: 2b28c3b871c e266d273114 > Author: Tom Rini <trini@konsulko.com> > Date: Mon Jan 1 12:38:15 2024 -0500 > > Merge tag 'dm-next-1124' of > https://source.denx.de/u-boot/custodians/u-boot-dm into next > > support propagating supernode properties with bootph schema > align bloblist with v0.9 of Firmware Handoff spec There's two cases. The first and uncommon case is that if you want to do something like: - Branch next - Add this subtree, do a few subtree updates on it (like my experiment of starting with v6.1-dts and merging up to v6.6-dts) - Wait a while, have changes happen in next - Rebase on to current next It gets painful and odd, but I don't know that this will ever really happen. The second, common case is: - Assume "next" already has this subtree + some merge commits in it. - Make a new local branch. - Make change in your new local branch. - This hypothetical "next" branch moves forward with changes - Rebase your local branch on to this "next" branch In this case things work as expected. Even if "next" in that case gets a new subtree merge that you need to rebase in. A third case that I'm not sure if really will happen or is a case where the custodian should rework their tree instead (to drop the subtree merge) is: - Assume "next" already has this subtree + some merge commits in it. - Make a new local branch. - Make a subtree merge+squash in your local branch. - This hypothetical "next" branch moves forward with changes - Rebase your local branch on to this "next" branch. This will then make you try and resolve some conflicts from that subtree merge. But is this a valid use case? Yes, I can see wanting to get a start on testing a dts merge before it happens, but is this a tree which needs to be rebased, or just re-created as needed, if the parent branch moves forward? Can you explain a bit more your case Michal since it sounded to me like you had a problem that happened rather than a worry about what might happen?
Hi Tom, On 1/4/24 14:50, Tom Rini wrote: > On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: >> Hi Michal, >> >> On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: >>> >>> >>> >>> On 1/3/24 17:32, Tom Rini wrote: >>>> On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: >>>>> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: >>>> [snip] >>>>>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for >>>>>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly >>>>>> +kept updated with every new kernel major release via subtree pull as follows:: >>>>> >>>>> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we >>>>> had in the original repo). I don't think it's relevant. >>>>> >>>>> We're not likely to regenerate the tree, but any clue what 'git >>>>> subtree pull' would do in this case? It could happen if we switched to >>>>> git-filter-repo. >>>>> >>>>>> + >>>>>> + git subtree pull --prefix devicetree-rebasing \ >>>>>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ >>>>>> + <release-tag> --squash >>>>> >>>>> I'd put this in a script to run. Documentation tends to be not quite >>>>> correct. A script could also be smarter and figure out <release-tag>. >>>> >>>> Since you had mentioned elsewhere moving the kernel scripts/dtc/ to >>>> something using subtree, I do hope to learn some lessons from what you >>>> do there. The first thing is that given the size/nature of the commits, >>>> I had figured I'd be the one doing this as a subtree pull+squash then >>>> push, rather than posting to the ML since it'll be huge and >>>> un-reviewable, but with a note sent off to the ML that people should be >>>> aware the next sync has been done and retest as needed. But Sumit, were >>>> you planning to do some of this instead? The second thing is that if we >>>> move the subtree part in to dts/ instead (where we will still have the >>>> Makefile/Kconfig we have today and then our Makefiles within the >>>> directories, this might get more complex and so really require a script >>>> to keep it from getting error prone? >>> >>> I played with this series and didn't really have time to dig into subtree >>> mechanics but one thing I see is that when squash and merge happens you can't do >>> simple rebase anymore which is time to time very useful. >>> I see some different rebase strategies and --rebase-merges option but if this is >>> adopted it should be properly described how to rebase u-boot tree which contains >>> some subtrees. >> >> Did you observe any specific difference with respect to subtree merge >> commits (Tom will likely do those in the beginning of next release >> cycle) when compared with normal merge commits when Tom pulls in code >> from different custodians like one example below? The same rebasing >> rules should apply to both types of merge commits. >> >> commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) >> Merge: 2b28c3b871c e266d273114 >> Author: Tom Rini <trini@konsulko.com> >> Date: Mon Jan 1 12:38:15 2024 -0500 >> >> Merge tag 'dm-next-1124' of >> https://source.denx.de/u-boot/custodians/u-boot-dm into next >> >> support propagating supernode properties with bootph schema >> align bloblist with v0.9 of Firmware Handoff spec > > There's two cases. The first and uncommon case is that if you want to do > something like: > - Branch next > - Add this subtree, do a few subtree updates on it (like my experiment > of starting with v6.1-dts and merging up to v6.6-dts) > - Wait a while, have changes happen in next > - Rebase on to current next > > It gets painful and odd, but I don't know that this will ever really > happen. > > The second, common case is: > - Assume "next" already has this subtree + some merge commits in it. > - Make a new local branch. > - Make change in your new local branch. > - This hypothetical "next" branch moves forward with changes > - Rebase your local branch on to this "next" branch > > In this case things work as expected. Even if "next" in that case gets > a new subtree merge that you need to rebase in. > > A third case that I'm not sure if really will happen or is a case where > the custodian should rework their tree instead (to drop the subtree > merge) is: > - Assume "next" already has this subtree + some merge commits in it. > - Make a new local branch. > - Make a subtree merge+squash in your local branch. > - This hypothetical "next" branch moves forward with changes > - Rebase your local branch on to this "next" branch. > > This will then make you try and resolve some conflicts from that subtree > merge. But is this a valid use case? Yes, I can see wanting to get a > start on testing a dts merge before it happens, but is this a tree which > needs to be rebased, or just re-created as needed, if the parent branch > moves forward? > > Can you explain a bit more your case Michal since it sounded to me like > you had a problem that happened rather than a worry about what might > happen? I remember couple of bisects which I was doing and it works fine on the tree till the point you have 2 issues to deal with. It means pretty much you have config change/fix which has to be applied to see second issue. The normal way what I have done was simply find major version apply that fix and then take the latest tree and rebase latest on the top of it. There were normally not so many conflicts to resolved (or easy to resolve/ignore). Then because the first fix is present all the time bisect is able to help me to find second issue. If there is subtree update the whole rebase will horribly fail. I see that Rob and you in DTC case just simply do c&p and create regular commit with all changes inside. From my perspective this is more straightforward to deal with which will handle all cases and it is also proven over that years. Thanks, Michal
On Thu, Jan 04, 2024 at 03:45:10PM +0100, Michal Simek wrote: > Hi Tom, > > On 1/4/24 14:50, Tom Rini wrote: > > On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: > > > Hi Michal, > > > > > > On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: > > > > > > > > > > > > > > > > On 1/3/24 17:32, Tom Rini wrote: > > > > > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > > > > > > On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > > > > [snip] > > > > > > > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > > > > > > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > > > > > > +kept updated with every new kernel major release via subtree pull as follows:: > > > > > > > > > > > > I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > > > > > > had in the original repo). I don't think it's relevant. > > > > > > > > > > > > We're not likely to regenerate the tree, but any clue what 'git > > > > > > subtree pull' would do in this case? It could happen if we switched to > > > > > > git-filter-repo. > > > > > > > > > > > > > + > > > > > > > + git subtree pull --prefix devicetree-rebasing \ > > > > > > > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > > > > > > + <release-tag> --squash > > > > > > > > > > > > I'd put this in a script to run. Documentation tends to be not quite > > > > > > correct. A script could also be smarter and figure out <release-tag>. > > > > > > > > > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > > > > > something using subtree, I do hope to learn some lessons from what you > > > > > do there. The first thing is that given the size/nature of the commits, > > > > > I had figured I'd be the one doing this as a subtree pull+squash then > > > > > push, rather than posting to the ML since it'll be huge and > > > > > un-reviewable, but with a note sent off to the ML that people should be > > > > > aware the next sync has been done and retest as needed. But Sumit, were > > > > > you planning to do some of this instead? The second thing is that if we > > > > > move the subtree part in to dts/ instead (where we will still have the > > > > > Makefile/Kconfig we have today and then our Makefiles within the > > > > > directories, this might get more complex and so really require a script > > > > > to keep it from getting error prone? > > > > > > > > I played with this series and didn't really have time to dig into subtree > > > > mechanics but one thing I see is that when squash and merge happens you can't do > > > > simple rebase anymore which is time to time very useful. > > > > I see some different rebase strategies and --rebase-merges option but if this is > > > > adopted it should be properly described how to rebase u-boot tree which contains > > > > some subtrees. > > > > > > Did you observe any specific difference with respect to subtree merge > > > commits (Tom will likely do those in the beginning of next release > > > cycle) when compared with normal merge commits when Tom pulls in code > > > from different custodians like one example below? The same rebasing > > > rules should apply to both types of merge commits. > > > > > > commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) > > > Merge: 2b28c3b871c e266d273114 > > > Author: Tom Rini <trini@konsulko.com> > > > Date: Mon Jan 1 12:38:15 2024 -0500 > > > > > > Merge tag 'dm-next-1124' of > > > https://source.denx.de/u-boot/custodians/u-boot-dm into next > > > > > > support propagating supernode properties with bootph schema > > > align bloblist with v0.9 of Firmware Handoff spec > > > > There's two cases. The first and uncommon case is that if you want to do > > something like: > > - Branch next > > - Add this subtree, do a few subtree updates on it (like my experiment > > of starting with v6.1-dts and merging up to v6.6-dts) > > - Wait a while, have changes happen in next > > - Rebase on to current next > > > > It gets painful and odd, but I don't know that this will ever really > > happen. > > > > The second, common case is: > > - Assume "next" already has this subtree + some merge commits in it. > > - Make a new local branch. > > - Make change in your new local branch. > > - This hypothetical "next" branch moves forward with changes > > - Rebase your local branch on to this "next" branch > > > > In this case things work as expected. Even if "next" in that case gets > > a new subtree merge that you need to rebase in. > > > > A third case that I'm not sure if really will happen or is a case where > > the custodian should rework their tree instead (to drop the subtree > > merge) is: > > - Assume "next" already has this subtree + some merge commits in it. > > - Make a new local branch. > > - Make a subtree merge+squash in your local branch. > > - This hypothetical "next" branch moves forward with changes > > - Rebase your local branch on to this "next" branch. > > > > This will then make you try and resolve some conflicts from that subtree > > merge. But is this a valid use case? Yes, I can see wanting to get a > > start on testing a dts merge before it happens, but is this a tree which > > needs to be rebased, or just re-created as needed, if the parent branch > > moves forward? > > > > Can you explain a bit more your case Michal since it sounded to me like > > you had a problem that happened rather than a worry about what might > > happen? > > I remember couple of bisects which I was doing and it works fine on the tree > till the point you have 2 issues to deal with. It means pretty much you have > config change/fix which has to be applied to see second issue. > > The normal way what I have done was simply find major version apply that fix > and then take the latest tree and rebase latest on the top of it. There were > normally not so many conflicts to resolved (or easy to resolve/ignore). Then > because the first fix is present all the time bisect is able to help me to > find second issue. > > If there is subtree update the whole rebase will horribly fail. > > I see that Rob and you in DTC case just simply do c&p and create regular > commit with all changes inside. From my perspective this is more > straightforward to deal with which will handle all cases and it is also > proven over that years. Ah, bisecting. Maybe the (annoying) answer is to turn that merge commit in to a regular commit in the bisect branch? This is a good point, and I need to go and construct a bigger example tree and check around at what happens.
On 1/4/24 16:30, Tom Rini wrote: > On Thu, Jan 04, 2024 at 03:45:10PM +0100, Michal Simek wrote: >> Hi Tom, >> >> On 1/4/24 14:50, Tom Rini wrote: >>> On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: >>>> Hi Michal, >>>> >>>> On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: >>>>> >>>>> >>>>> >>>>> On 1/3/24 17:32, Tom Rini wrote: >>>>>> On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: >>>>>>> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: >>>>>> [snip] >>>>>>>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for >>>>>>>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly >>>>>>>> +kept updated with every new kernel major release via subtree pull as follows:: >>>>>>> >>>>>>> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we >>>>>>> had in the original repo). I don't think it's relevant. >>>>>>> >>>>>>> We're not likely to regenerate the tree, but any clue what 'git >>>>>>> subtree pull' would do in this case? It could happen if we switched to >>>>>>> git-filter-repo. >>>>>>> >>>>>>>> + >>>>>>>> + git subtree pull --prefix devicetree-rebasing \ >>>>>>>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ >>>>>>>> + <release-tag> --squash >>>>>>> >>>>>>> I'd put this in a script to run. Documentation tends to be not quite >>>>>>> correct. A script could also be smarter and figure out <release-tag>. >>>>>> >>>>>> Since you had mentioned elsewhere moving the kernel scripts/dtc/ to >>>>>> something using subtree, I do hope to learn some lessons from what you >>>>>> do there. The first thing is that given the size/nature of the commits, >>>>>> I had figured I'd be the one doing this as a subtree pull+squash then >>>>>> push, rather than posting to the ML since it'll be huge and >>>>>> un-reviewable, but with a note sent off to the ML that people should be >>>>>> aware the next sync has been done and retest as needed. But Sumit, were >>>>>> you planning to do some of this instead? The second thing is that if we >>>>>> move the subtree part in to dts/ instead (where we will still have the >>>>>> Makefile/Kconfig we have today and then our Makefiles within the >>>>>> directories, this might get more complex and so really require a script >>>>>> to keep it from getting error prone? >>>>> >>>>> I played with this series and didn't really have time to dig into subtree >>>>> mechanics but one thing I see is that when squash and merge happens you can't do >>>>> simple rebase anymore which is time to time very useful. >>>>> I see some different rebase strategies and --rebase-merges option but if this is >>>>> adopted it should be properly described how to rebase u-boot tree which contains >>>>> some subtrees. >>>> >>>> Did you observe any specific difference with respect to subtree merge >>>> commits (Tom will likely do those in the beginning of next release >>>> cycle) when compared with normal merge commits when Tom pulls in code >>>> from different custodians like one example below? The same rebasing >>>> rules should apply to both types of merge commits. >>>> >>>> commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) >>>> Merge: 2b28c3b871c e266d273114 >>>> Author: Tom Rini <trini@konsulko.com> >>>> Date: Mon Jan 1 12:38:15 2024 -0500 >>>> >>>> Merge tag 'dm-next-1124' of >>>> https://source.denx.de/u-boot/custodians/u-boot-dm into next >>>> >>>> support propagating supernode properties with bootph schema >>>> align bloblist with v0.9 of Firmware Handoff spec >>> >>> There's two cases. The first and uncommon case is that if you want to do >>> something like: >>> - Branch next >>> - Add this subtree, do a few subtree updates on it (like my experiment >>> of starting with v6.1-dts and merging up to v6.6-dts) >>> - Wait a while, have changes happen in next >>> - Rebase on to current next >>> >>> It gets painful and odd, but I don't know that this will ever really >>> happen. >>> >>> The second, common case is: >>> - Assume "next" already has this subtree + some merge commits in it. >>> - Make a new local branch. >>> - Make change in your new local branch. >>> - This hypothetical "next" branch moves forward with changes >>> - Rebase your local branch on to this "next" branch >>> >>> In this case things work as expected. Even if "next" in that case gets >>> a new subtree merge that you need to rebase in. >>> >>> A third case that I'm not sure if really will happen or is a case where >>> the custodian should rework their tree instead (to drop the subtree >>> merge) is: >>> - Assume "next" already has this subtree + some merge commits in it. >>> - Make a new local branch. >>> - Make a subtree merge+squash in your local branch. >>> - This hypothetical "next" branch moves forward with changes >>> - Rebase your local branch on to this "next" branch. >>> >>> This will then make you try and resolve some conflicts from that subtree >>> merge. But is this a valid use case? Yes, I can see wanting to get a >>> start on testing a dts merge before it happens, but is this a tree which >>> needs to be rebased, or just re-created as needed, if the parent branch >>> moves forward? >>> >>> Can you explain a bit more your case Michal since it sounded to me like >>> you had a problem that happened rather than a worry about what might >>> happen? >> >> I remember couple of bisects which I was doing and it works fine on the tree >> till the point you have 2 issues to deal with. It means pretty much you have >> config change/fix which has to be applied to see second issue. >> >> The normal way what I have done was simply find major version apply that fix >> and then take the latest tree and rebase latest on the top of it. There were >> normally not so many conflicts to resolved (or easy to resolve/ignore). Then >> because the first fix is present all the time bisect is able to help me to >> find second issue. >> >> If there is subtree update the whole rebase will horribly fail. >> >> I see that Rob and you in DTC case just simply do c&p and create regular >> commit with all changes inside. From my perspective this is more >> straightforward to deal with which will handle all cases and it is also >> proven over that years. > > Ah, bisecting. Maybe the (annoying) answer is to turn that merge commit > in to a regular commit in the bisect branch? This is a good point, and > I need to go and construct a bigger example tree and check around at > what happens. Sure you can do whatever you like. But I expect that at the end if this started to be used you will have subtree for devicetree, dtc, lwip and maybe something else. It means you have to start to keep track where that subtree are and what version is used at what time. Because dt binding is extending often likely you will need to merge it every Linux release. It means you would have to do it multiple times depends on how many releases you are going over. I know that Qemu is using gitmodules that's definitely another option but not sure if better or worse. I can also imagine that companies with SR IR certified firmware would keep downstream version where they want to just take code fixes, DT fixes and devicetree-rebasing tree to make sure that they are still aligned with DT schema. Can you simply just cherry pick that merged commit without any side effect? Back to my point. At the end what we need to is to document it properly how to deal with it. Thanks, Michal
On Thu, Jan 04, 2024 at 04:50:14PM +0100, Michal Simek wrote: > > > On 1/4/24 16:30, Tom Rini wrote: > > On Thu, Jan 04, 2024 at 03:45:10PM +0100, Michal Simek wrote: > > > Hi Tom, > > > > > > On 1/4/24 14:50, Tom Rini wrote: > > > > On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: > > > > > Hi Michal, > > > > > > > > > > On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On 1/3/24 17:32, Tom Rini wrote: > > > > > > > On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > > > > > > > > On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > > > > > > > [snip] > > > > > > > > > +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > > > > > > > > > +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > > > > > > > > > +kept updated with every new kernel major release via subtree pull as follows:: > > > > > > > > > > > > > > > > I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > > > > > > > > had in the original repo). I don't think it's relevant. > > > > > > > > > > > > > > > > We're not likely to regenerate the tree, but any clue what 'git > > > > > > > > subtree pull' would do in this case? It could happen if we switched to > > > > > > > > git-filter-repo. > > > > > > > > > > > > > > > > > + > > > > > > > > > + git subtree pull --prefix devicetree-rebasing \ > > > > > > > > > + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > > > > > > > > > + <release-tag> --squash > > > > > > > > > > > > > > > > I'd put this in a script to run. Documentation tends to be not quite > > > > > > > > correct. A script could also be smarter and figure out <release-tag>. > > > > > > > > > > > > > > Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > > > > > > > something using subtree, I do hope to learn some lessons from what you > > > > > > > do there. The first thing is that given the size/nature of the commits, > > > > > > > I had figured I'd be the one doing this as a subtree pull+squash then > > > > > > > push, rather than posting to the ML since it'll be huge and > > > > > > > un-reviewable, but with a note sent off to the ML that people should be > > > > > > > aware the next sync has been done and retest as needed. But Sumit, were > > > > > > > you planning to do some of this instead? The second thing is that if we > > > > > > > move the subtree part in to dts/ instead (where we will still have the > > > > > > > Makefile/Kconfig we have today and then our Makefiles within the > > > > > > > directories, this might get more complex and so really require a script > > > > > > > to keep it from getting error prone? > > > > > > > > > > > > I played with this series and didn't really have time to dig into subtree > > > > > > mechanics but one thing I see is that when squash and merge happens you can't do > > > > > > simple rebase anymore which is time to time very useful. > > > > > > I see some different rebase strategies and --rebase-merges option but if this is > > > > > > adopted it should be properly described how to rebase u-boot tree which contains > > > > > > some subtrees. > > > > > > > > > > Did you observe any specific difference with respect to subtree merge > > > > > commits (Tom will likely do those in the beginning of next release > > > > > cycle) when compared with normal merge commits when Tom pulls in code > > > > > from different custodians like one example below? The same rebasing > > > > > rules should apply to both types of merge commits. > > > > > > > > > > commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) > > > > > Merge: 2b28c3b871c e266d273114 > > > > > Author: Tom Rini <trini@konsulko.com> > > > > > Date: Mon Jan 1 12:38:15 2024 -0500 > > > > > > > > > > Merge tag 'dm-next-1124' of > > > > > https://source.denx.de/u-boot/custodians/u-boot-dm into next > > > > > > > > > > support propagating supernode properties with bootph schema > > > > > align bloblist with v0.9 of Firmware Handoff spec > > > > > > > > There's two cases. The first and uncommon case is that if you want to do > > > > something like: > > > > - Branch next > > > > - Add this subtree, do a few subtree updates on it (like my experiment > > > > of starting with v6.1-dts and merging up to v6.6-dts) > > > > - Wait a while, have changes happen in next > > > > - Rebase on to current next > > > > > > > > It gets painful and odd, but I don't know that this will ever really > > > > happen. > > > > > > > > The second, common case is: > > > > - Assume "next" already has this subtree + some merge commits in it. > > > > - Make a new local branch. > > > > - Make change in your new local branch. > > > > - This hypothetical "next" branch moves forward with changes > > > > - Rebase your local branch on to this "next" branch > > > > > > > > In this case things work as expected. Even if "next" in that case gets > > > > a new subtree merge that you need to rebase in. > > > > > > > > A third case that I'm not sure if really will happen or is a case where > > > > the custodian should rework their tree instead (to drop the subtree > > > > merge) is: > > > > - Assume "next" already has this subtree + some merge commits in it. > > > > - Make a new local branch. > > > > - Make a subtree merge+squash in your local branch. > > > > - This hypothetical "next" branch moves forward with changes > > > > - Rebase your local branch on to this "next" branch. > > > > > > > > This will then make you try and resolve some conflicts from that subtree > > > > merge. But is this a valid use case? Yes, I can see wanting to get a > > > > start on testing a dts merge before it happens, but is this a tree which > > > > needs to be rebased, or just re-created as needed, if the parent branch > > > > moves forward? > > > > > > > > Can you explain a bit more your case Michal since it sounded to me like > > > > you had a problem that happened rather than a worry about what might > > > > happen? > > > > > > I remember couple of bisects which I was doing and it works fine on the tree > > > till the point you have 2 issues to deal with. It means pretty much you have > > > config change/fix which has to be applied to see second issue. > > > > > > The normal way what I have done was simply find major version apply that fix > > > and then take the latest tree and rebase latest on the top of it. There were > > > normally not so many conflicts to resolved (or easy to resolve/ignore). Then > > > because the first fix is present all the time bisect is able to help me to > > > find second issue. > > > > > > If there is subtree update the whole rebase will horribly fail. > > > > > > I see that Rob and you in DTC case just simply do c&p and create regular > > > commit with all changes inside. From my perspective this is more > > > straightforward to deal with which will handle all cases and it is also > > > proven over that years. > > > > Ah, bisecting. Maybe the (annoying) answer is to turn that merge commit > > in to a regular commit in the bisect branch? This is a good point, and > > I need to go and construct a bigger example tree and check around at > > what happens. > > Sure you can do whatever you like. But I expect that at the end if this > started to be used you will have subtree for devicetree, dtc, lwip and maybe > something else. It means you have to start to keep track where that subtree > are and what version is used at what time. Yes, lwip would be a subtree too, dtc is trickier but I see and agree with your point. > Because dt binding is extending often likely you will need to merge it every > Linux release. It means you would have to do it multiple times depends on > how many releases you are going over. Yes, the plan is to re-sync every time our next opens with the kernel release at the time (which maybe needs to be clearer in the docs). > I know that Qemu is using gitmodules that's definitely another option but > not sure if better or worse. Yeah, submodules is harder and we're not going that path. > I can also imagine that companies with SR IR certified firmware would keep > downstream version where they want to just take code fixes, DT fixes and > devicetree-rebasing tree to make sure that they are still aligned with DT > schema. Can you simply just cherry pick that merged commit without any side > effect? Life downstream is always fun. Personally, I would re-run the "git subtree" command rather than cherry-pick from U-Boot itself in that case, but it should be no different than any other commit. > > Back to my point. At the end what we need to is to document it properly how > to deal with it. So, here's the test I did. I went back to v2023.10, then added v6.1-dts as a subtree. I rebased v2024.01-rc1 on top of that, added a fake bad commit, synced up subtrees to v6.6, and then rebased the tags along the way to v2024.01-rc6. I then went for a bisect back to the fake bad commit. Everything went fine, git didn't have troubles rebasing things as it went. But to re-iterate, yes, the policy around when dts and bindings will be resynced needs to be clear and documented. Does that cover it?
On 1/4/24 19:03, Tom Rini wrote: > On Thu, Jan 04, 2024 at 04:50:14PM +0100, Michal Simek wrote: >> >> >> On 1/4/24 16:30, Tom Rini wrote: >>> On Thu, Jan 04, 2024 at 03:45:10PM +0100, Michal Simek wrote: >>>> Hi Tom, >>>> >>>> On 1/4/24 14:50, Tom Rini wrote: >>>>> On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: >>>>>> Hi Michal, >>>>>> >>>>>> On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 1/3/24 17:32, Tom Rini wrote: >>>>>>>> On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: >>>>>>>>> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: >>>>>>>> [snip] >>>>>>>>>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for >>>>>>>>>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly >>>>>>>>>> +kept updated with every new kernel major release via subtree pull as follows:: >>>>>>>>> >>>>>>>>> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we >>>>>>>>> had in the original repo). I don't think it's relevant. >>>>>>>>> >>>>>>>>> We're not likely to regenerate the tree, but any clue what 'git >>>>>>>>> subtree pull' would do in this case? It could happen if we switched to >>>>>>>>> git-filter-repo. >>>>>>>>> >>>>>>>>>> + >>>>>>>>>> + git subtree pull --prefix devicetree-rebasing \ >>>>>>>>>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ >>>>>>>>>> + <release-tag> --squash >>>>>>>>> >>>>>>>>> I'd put this in a script to run. Documentation tends to be not quite >>>>>>>>> correct. A script could also be smarter and figure out <release-tag>. >>>>>>>> >>>>>>>> Since you had mentioned elsewhere moving the kernel scripts/dtc/ to >>>>>>>> something using subtree, I do hope to learn some lessons from what you >>>>>>>> do there. The first thing is that given the size/nature of the commits, >>>>>>>> I had figured I'd be the one doing this as a subtree pull+squash then >>>>>>>> push, rather than posting to the ML since it'll be huge and >>>>>>>> un-reviewable, but with a note sent off to the ML that people should be >>>>>>>> aware the next sync has been done and retest as needed. But Sumit, were >>>>>>>> you planning to do some of this instead? The second thing is that if we >>>>>>>> move the subtree part in to dts/ instead (where we will still have the >>>>>>>> Makefile/Kconfig we have today and then our Makefiles within the >>>>>>>> directories, this might get more complex and so really require a script >>>>>>>> to keep it from getting error prone? >>>>>>> >>>>>>> I played with this series and didn't really have time to dig into subtree >>>>>>> mechanics but one thing I see is that when squash and merge happens you can't do >>>>>>> simple rebase anymore which is time to time very useful. >>>>>>> I see some different rebase strategies and --rebase-merges option but if this is >>>>>>> adopted it should be properly described how to rebase u-boot tree which contains >>>>>>> some subtrees. >>>>>> >>>>>> Did you observe any specific difference with respect to subtree merge >>>>>> commits (Tom will likely do those in the beginning of next release >>>>>> cycle) when compared with normal merge commits when Tom pulls in code >>>>>> from different custodians like one example below? The same rebasing >>>>>> rules should apply to both types of merge commits. >>>>>> >>>>>> commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) >>>>>> Merge: 2b28c3b871c e266d273114 >>>>>> Author: Tom Rini <trini@konsulko.com> >>>>>> Date: Mon Jan 1 12:38:15 2024 -0500 >>>>>> >>>>>> Merge tag 'dm-next-1124' of >>>>>> https://source.denx.de/u-boot/custodians/u-boot-dm into next >>>>>> >>>>>> support propagating supernode properties with bootph schema >>>>>> align bloblist with v0.9 of Firmware Handoff spec >>>>> >>>>> There's two cases. The first and uncommon case is that if you want to do >>>>> something like: >>>>> - Branch next >>>>> - Add this subtree, do a few subtree updates on it (like my experiment >>>>> of starting with v6.1-dts and merging up to v6.6-dts) >>>>> - Wait a while, have changes happen in next >>>>> - Rebase on to current next >>>>> >>>>> It gets painful and odd, but I don't know that this will ever really >>>>> happen. >>>>> >>>>> The second, common case is: >>>>> - Assume "next" already has this subtree + some merge commits in it. >>>>> - Make a new local branch. >>>>> - Make change in your new local branch. >>>>> - This hypothetical "next" branch moves forward with changes >>>>> - Rebase your local branch on to this "next" branch >>>>> >>>>> In this case things work as expected. Even if "next" in that case gets >>>>> a new subtree merge that you need to rebase in. >>>>> >>>>> A third case that I'm not sure if really will happen or is a case where >>>>> the custodian should rework their tree instead (to drop the subtree >>>>> merge) is: >>>>> - Assume "next" already has this subtree + some merge commits in it. >>>>> - Make a new local branch. >>>>> - Make a subtree merge+squash in your local branch. >>>>> - This hypothetical "next" branch moves forward with changes >>>>> - Rebase your local branch on to this "next" branch. >>>>> >>>>> This will then make you try and resolve some conflicts from that subtree >>>>> merge. But is this a valid use case? Yes, I can see wanting to get a >>>>> start on testing a dts merge before it happens, but is this a tree which >>>>> needs to be rebased, or just re-created as needed, if the parent branch >>>>> moves forward? >>>>> >>>>> Can you explain a bit more your case Michal since it sounded to me like >>>>> you had a problem that happened rather than a worry about what might >>>>> happen? >>>> >>>> I remember couple of bisects which I was doing and it works fine on the tree >>>> till the point you have 2 issues to deal with. It means pretty much you have >>>> config change/fix which has to be applied to see second issue. >>>> >>>> The normal way what I have done was simply find major version apply that fix >>>> and then take the latest tree and rebase latest on the top of it. There were >>>> normally not so many conflicts to resolved (or easy to resolve/ignore). Then >>>> because the first fix is present all the time bisect is able to help me to >>>> find second issue. >>>> >>>> If there is subtree update the whole rebase will horribly fail. >>>> >>>> I see that Rob and you in DTC case just simply do c&p and create regular >>>> commit with all changes inside. From my perspective this is more >>>> straightforward to deal with which will handle all cases and it is also >>>> proven over that years. >>> >>> Ah, bisecting. Maybe the (annoying) answer is to turn that merge commit >>> in to a regular commit in the bisect branch? This is a good point, and >>> I need to go and construct a bigger example tree and check around at >>> what happens. >> >> Sure you can do whatever you like. But I expect that at the end if this >> started to be used you will have subtree for devicetree, dtc, lwip and maybe >> something else. It means you have to start to keep track where that subtree >> are and what version is used at what time. > > Yes, lwip would be a subtree too, dtc is trickier but I see and agree > with your point. > >> Because dt binding is extending often likely you will need to merge it every >> Linux release. It means you would have to do it multiple times depends on >> how many releases you are going over. > > Yes, the plan is to re-sync every time our next opens with the kernel > release at the time (which maybe needs to be clearer in the docs). > >> I know that Qemu is using gitmodules that's definitely another option but >> not sure if better or worse. > > Yeah, submodules is harder and we're not going that path. > >> I can also imagine that companies with SR IR certified firmware would keep >> downstream version where they want to just take code fixes, DT fixes and >> devicetree-rebasing tree to make sure that they are still aligned with DT >> schema. Can you simply just cherry pick that merged commit without any side >> effect? > > Life downstream is always fun. Personally, I would re-run the "git > subtree" command rather than cherry-pick from U-Boot itself in that > case, but it should be no different than any other commit. > >> >> Back to my point. At the end what we need to is to document it properly how >> to deal with it. > > So, here's the test I did. I went back to v2023.10, then added v6.1-dts > as a subtree. I rebased v2024.01-rc1 on top of that, added a fake bad > commit, synced up subtrees to v6.6, and then rebased the tags along the > way to v2024.01-rc6. I then went for a bisect back to the fake bad > commit. Everything went fine, git didn't have troubles rebasing things > as it went. > > But to re-iterate, yes, the policy around when dts and bindings will be > resynced needs to be clear and documented. Does that cover it? I am fine with it when we have documentation around. Thanks, Michal
On Fri, 5 Jan 2024 at 13:46, Michal Simek <michal.simek@amd.com> wrote: > > > > On 1/4/24 19:03, Tom Rini wrote: > > On Thu, Jan 04, 2024 at 04:50:14PM +0100, Michal Simek wrote: > >> > >> > >> On 1/4/24 16:30, Tom Rini wrote: > >>> On Thu, Jan 04, 2024 at 03:45:10PM +0100, Michal Simek wrote: > >>>> Hi Tom, > >>>> > >>>> On 1/4/24 14:50, Tom Rini wrote: > >>>>> On Thu, Jan 04, 2024 at 06:52:32PM +0530, Sumit Garg wrote: > >>>>>> Hi Michal, > >>>>>> > >>>>>> On Thu, 4 Jan 2024 at 15:39, Michal Simek <michal.simek@amd.com> wrote: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On 1/3/24 17:32, Tom Rini wrote: > >>>>>>>> On Wed, Jan 03, 2024 at 09:19:40AM -0700, Rob Herring wrote: > >>>>>>>>> On Thu, Dec 28, 2023 at 4:59 AM Sumit Garg <sumit.garg@linaro.org> wrote: > >>>>>>>> [snip] > >>>>>>>>>> +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for > >>>>>>>>>> +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly > >>>>>>>>>> +kept updated with every new kernel major release via subtree pull as follows:: > >>>>>>>>> > >>>>>>>>> I would suggest dropping "-rebasing" in the u-boot tree. (I wish we > >>>>>>>>> had in the original repo). I don't think it's relevant. > >>>>>>>>> > >>>>>>>>> We're not likely to regenerate the tree, but any clue what 'git > >>>>>>>>> subtree pull' would do in this case? It could happen if we switched to > >>>>>>>>> git-filter-repo. > >>>>>>>>> > >>>>>>>>>> + > >>>>>>>>>> + git subtree pull --prefix devicetree-rebasing \ > >>>>>>>>>> + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ > >>>>>>>>>> + <release-tag> --squash > >>>>>>>>> > >>>>>>>>> I'd put this in a script to run. Documentation tends to be not quite > >>>>>>>>> correct. A script could also be smarter and figure out <release-tag>. > >>>>>>>> > >>>>>>>> Since you had mentioned elsewhere moving the kernel scripts/dtc/ to > >>>>>>>> something using subtree, I do hope to learn some lessons from what you > >>>>>>>> do there. The first thing is that given the size/nature of the commits, > >>>>>>>> I had figured I'd be the one doing this as a subtree pull+squash then > >>>>>>>> push, rather than posting to the ML since it'll be huge and > >>>>>>>> un-reviewable, but with a note sent off to the ML that people should be > >>>>>>>> aware the next sync has been done and retest as needed. But Sumit, were > >>>>>>>> you planning to do some of this instead? The second thing is that if we > >>>>>>>> move the subtree part in to dts/ instead (where we will still have the > >>>>>>>> Makefile/Kconfig we have today and then our Makefiles within the > >>>>>>>> directories, this might get more complex and so really require a script > >>>>>>>> to keep it from getting error prone? > >>>>>>> > >>>>>>> I played with this series and didn't really have time to dig into subtree > >>>>>>> mechanics but one thing I see is that when squash and merge happens you can't do > >>>>>>> simple rebase anymore which is time to time very useful. > >>>>>>> I see some different rebase strategies and --rebase-merges option but if this is > >>>>>>> adopted it should be properly described how to rebase u-boot tree which contains > >>>>>>> some subtrees. > >>>>>> > >>>>>> Did you observe any specific difference with respect to subtree merge > >>>>>> commits (Tom will likely do those in the beginning of next release > >>>>>> cycle) when compared with normal merge commits when Tom pulls in code > >>>>>> from different custodians like one example below? The same rebasing > >>>>>> rules should apply to both types of merge commits. > >>>>>> > >>>>>> commit dffa6d0210f57793f1e4e1e209d91ca5642e4d05 (origin/next) > >>>>>> Merge: 2b28c3b871c e266d273114 > >>>>>> Author: Tom Rini <trini@konsulko.com> > >>>>>> Date: Mon Jan 1 12:38:15 2024 -0500 > >>>>>> > >>>>>> Merge tag 'dm-next-1124' of > >>>>>> https://source.denx.de/u-boot/custodians/u-boot-dm into next > >>>>>> > >>>>>> support propagating supernode properties with bootph schema > >>>>>> align bloblist with v0.9 of Firmware Handoff spec > >>>>> > >>>>> There's two cases. The first and uncommon case is that if you want to do > >>>>> something like: > >>>>> - Branch next > >>>>> - Add this subtree, do a few subtree updates on it (like my experiment > >>>>> of starting with v6.1-dts and merging up to v6.6-dts) > >>>>> - Wait a while, have changes happen in next > >>>>> - Rebase on to current next > >>>>> > >>>>> It gets painful and odd, but I don't know that this will ever really > >>>>> happen. > >>>>> > >>>>> The second, common case is: > >>>>> - Assume "next" already has this subtree + some merge commits in it. > >>>>> - Make a new local branch. > >>>>> - Make change in your new local branch. > >>>>> - This hypothetical "next" branch moves forward with changes > >>>>> - Rebase your local branch on to this "next" branch > >>>>> > >>>>> In this case things work as expected. Even if "next" in that case gets > >>>>> a new subtree merge that you need to rebase in. > >>>>> > >>>>> A third case that I'm not sure if really will happen or is a case where > >>>>> the custodian should rework their tree instead (to drop the subtree > >>>>> merge) is: > >>>>> - Assume "next" already has this subtree + some merge commits in it. > >>>>> - Make a new local branch. > >>>>> - Make a subtree merge+squash in your local branch. > >>>>> - This hypothetical "next" branch moves forward with changes > >>>>> - Rebase your local branch on to this "next" branch. > >>>>> > >>>>> This will then make you try and resolve some conflicts from that subtree > >>>>> merge. But is this a valid use case? Yes, I can see wanting to get a > >>>>> start on testing a dts merge before it happens, but is this a tree which > >>>>> needs to be rebased, or just re-created as needed, if the parent branch > >>>>> moves forward? > >>>>> > >>>>> Can you explain a bit more your case Michal since it sounded to me like > >>>>> you had a problem that happened rather than a worry about what might > >>>>> happen? > >>>> > >>>> I remember couple of bisects which I was doing and it works fine on the tree > >>>> till the point you have 2 issues to deal with. It means pretty much you have > >>>> config change/fix which has to be applied to see second issue. > >>>> > >>>> The normal way what I have done was simply find major version apply that fix > >>>> and then take the latest tree and rebase latest on the top of it. There were > >>>> normally not so many conflicts to resolved (or easy to resolve/ignore). Then > >>>> because the first fix is present all the time bisect is able to help me to > >>>> find second issue. > >>>> > >>>> If there is subtree update the whole rebase will horribly fail. > >>>> > >>>> I see that Rob and you in DTC case just simply do c&p and create regular > >>>> commit with all changes inside. From my perspective this is more > >>>> straightforward to deal with which will handle all cases and it is also > >>>> proven over that years. > >>> > >>> Ah, bisecting. Maybe the (annoying) answer is to turn that merge commit > >>> in to a regular commit in the bisect branch? This is a good point, and > >>> I need to go and construct a bigger example tree and check around at > >>> what happens. > >> > >> Sure you can do whatever you like. But I expect that at the end if this > >> started to be used you will have subtree for devicetree, dtc, lwip and maybe > >> something else. It means you have to start to keep track where that subtree > >> are and what version is used at what time. > > > > Yes, lwip would be a subtree too, dtc is trickier but I see and agree > > with your point. > > > >> Because dt binding is extending often likely you will need to merge it every > >> Linux release. It means you would have to do it multiple times depends on > >> how many releases you are going over. > > > > Yes, the plan is to re-sync every time our next opens with the kernel > > release at the time (which maybe needs to be clearer in the docs). I will clarify this further in the docs. -Sumit > > > >> I know that Qemu is using gitmodules that's definitely another option but > >> not sure if better or worse. > > > > Yeah, submodules is harder and we're not going that path. > > > >> I can also imagine that companies with SR IR certified firmware would keep > >> downstream version where they want to just take code fixes, DT fixes and > >> devicetree-rebasing tree to make sure that they are still aligned with DT > >> schema. Can you simply just cherry pick that merged commit without any side > >> effect? > > > > Life downstream is always fun. Personally, I would re-run the "git > > subtree" command rather than cherry-pick from U-Boot itself in that > > case, but it should be no different than any other commit. > > > >> > >> Back to my point. At the end what we need to is to document it properly how > >> to deal with it. > > > > So, here's the test I did. I went back to v2023.10, then added v6.1-dts > > as a subtree. I rebased v2024.01-rc1 on top of that, added a fake bad > > commit, synced up subtrees to v6.6, and then rebased the tags along the > > way to v2024.01-rc6. I then went for a bisect back to the fake bad > > commit. Everything went fine, git didn't have troubles rebasing things > > as it went. > > > > But to re-iterate, yes, the policy around when dts and bindings will be > > resynced needs to be clear and documented. Does that cover it? > > I am fine with it when we have documentation around. > > Thanks, > Michal
diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst index cbb65c9b177..65180d33e8d 100644 --- a/doc/develop/devicetree/control.rst +++ b/doc/develop/devicetree/control.rst @@ -1,5 +1,6 @@ .. SPDX-License-Identifier: GPL-2.0+ .. sectionauthor:: Copyright 2011 The Chromium OS Authors +.. Copyright 2023 Linaro Ltd. Devicetree Control in U-Boot ============================ @@ -22,14 +23,13 @@ for three reasons: hierarchical format - It is fairly efficient to read incrementally -The arch/<arch>/dts directories contains a Makefile for building the devicetree -blob and embedding it in the U-Boot image. This is useful since it allows -U-Boot to configure itself according to what it finds there. If you have -a number of similar boards with different peripherals, you can describe -the features of each board in the devicetree file, and have a single -generic source base. +The U-Boot Makefile infrastructure allows for building the devicetree blob +and embedding it in the U-Boot image. This is useful since it allows U-Boot +to configure itself according to what it finds there. If you have a number +of similar boards with different peripherals, you can describe the features +of each board in the devicetree file, and have a single generic source base. -To enable this feature, add CONFIG_OF_CONTROL to your board config file. +To enable this feature, select `OF_CONTROL` via Kconfig. What is a Flattened Devicetree? @@ -68,8 +68,21 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. Where do I get a devicetree file for my board? ---------------------------------------------- -You may find that the Linux kernel has a suitable file. Look in the -kernel source in arch/<arch>/boot/dts. +Linux kernel Git repository has been the place where devicetree files along +with devicetree bindings are stored and maintained. There is devicetee-rebasing +(dtrepo_) which maintains a forked copy of devicetree files along with bindings +at every Linux kernel major release or intermideate release candidates. + +In order to maintain devicetree files sync, U-Boot maintains a Git subtree for +devicetee-rebasing repo as `devicetee-rebasing/` sub-directory. It is regularly +kept updated with every new kernel major release via subtree pull as follows:: + + git subtree pull --prefix devicetree-rebasing \ + git://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git \ + <release-tag> --squash + +You may find that the `devicetee-rebasing/` sub-directory has a suitable +devicetree file for your board. Look in `devicetree-rebasing/src/<arch>/`. If not you might find other boards with suitable files that you can modify to your needs. Look in the board directories for files with a @@ -81,34 +94,34 @@ Failing that, you could write one from scratch yourself! Configuration ------------- -Use:: +Traditionally, U-Boot placed copies of devicetree source files from Linux +kernel into `arch/<arch>/dts/<name>.dts` which can be selected via setting +"<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. - #define CONFIG_DEFAULT_DEVICE_TREE "<name>" +However, it has become combersome over time for each SoC/board maintainer to +keep devicetree files in sync with Linux kernel. Thereby, SoC/board maintainers +are encouraged to migrate to use mirrored copies from `devicetree-rebasing/` +into `dts/arch/<arch>/<vendor>`. To do that enable `OF_UPSTREAM` via Kconfig and +set up "<vendor>/<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. -to set the filename of the devicetree source. Then put your devicetree -file into:: +This should include your CPU or SOC's devicetree file. On top of that any U-Boot +specific tweaks (see: dttweaks_) can be made for your board. - arch/<arch>/dts/<name>.dts - -This should include your CPU or SOC's devicetree file, placed in -`arch/<arch>/dts`, and then make any adjustments required using a u-boot-dtsi -file for your board. - -If CONFIG_OF_EMBED is defined, then it will be picked up and built into +If `OF_EMBED` is selected by Kconfig, then it will be picked up and built into the U-Boot image (including u-boot.bin). This is suitable for debugging and development only and is not recommended for production devices. -If CONFIG_OF_SEPARATE is defined, then it will be built and placed in +If `OF_SEPARATE` is selected by Kconfig, then it will be built and placed in a u-boot.dtb file alongside u-boot-nodtb.bin with the combined result placed -in u-boot.bin so you can still just flash u-boot.bin onto your board. If you are -using CONFIG_SPL_FRAMEWORK, then u-boot.img will be built to include the device -tree binary. +in u-boot.bin so you can still just flash u-boot.bin onto your board. If Kconfig +option `SPL_FRAMEWORK` is enabled, then u-boot.img will be built to include the +device tree binary. -If CONFIG_OF_BOARD is defined, a board-specific routine will provide the +If `OF_BOARD` is selected by Kconfig, a board-specific routine will provide the devicetree at runtime, for example if an earlier bootloader stage creates it and passes it to U-Boot. -If CONFIG_SANDBOX is defined, then it will be read from a file on +If `SANDBOX` is selected by Kconfig, then it will be read from a file on startup. Use the -d flag to U-Boot to specify the file to read, -D for the default and -T for the test devicetree, used to run sandbox unit tests. @@ -142,7 +155,7 @@ Build: After the board configuration is done, fdt supported u-boot can be built in two ways: -# build the default dts which is defined from CONFIG_DEFAULT_DEVICE_TREE:: +# build the default dts which is selected by DEFAULT_DEVICE_TREE Kconfig:: $ make @@ -156,8 +169,9 @@ ways: Adding tweaks for U-Boot ------------------------ -It is strongly recommended that devicetree files in U-Boot are an exact copy of -those in Linux, so that it is easy to sync them up from time to time. +With devicetee-rebasing Git subtree, it is ensured that devicetree files in +U-Boot are an exact copy of those in Linux kernel via mirroring into +`dts/arch/<arch>/<vendor>`. U-Boot is of course a very different project from Linux, e.g. it operates under much more restrictive memory and code-size constraints. Where Linux may use a @@ -170,8 +184,8 @@ constraints are even more extreme and the devicetree is shrunk to remove unwanted nodes, or even turned into C code to avoid access overhead. U-Boot automatically looks for and includes a file with updates to the standard -devicetree for your board, searching for them in the same directory as the -main file, in this order:: +devicetree for your board, searching for them in `arch/<arch>/dts/` in this +order:: <orig_filename>-u-boot.dtsi <CONFIG_SYS_SOC>-u-boot.dtsi @@ -195,11 +209,57 @@ As mentioned above, the U-Boot build system automatically includes a `*-u-boot.dtsi` file, if found, containing U-Boot specific quirks. However, some data, such as the mentioned public keys, are not appropriate for upstream U-Boot but are better kept and maintained -outside the U-Boot repository. You can use CONFIG_DEVICE_TREE_INCLUDES -to specify a list of .dtsi files that will also be included when +outside the U-Boot repository. You can use `DEVICE_TREE_INCLUDES` Kconfig +option to specify a list of .dtsi files that will also be included when building .dtb files. +Devicetree bindings schema checks +--------------------------------- + +With devicetee-rebasing Git subtree, the devicetree bindings are also regularly +synced with Linux kernel as `devicetree-rebasing/Bindings/` sub-directory. This +allows U-Boot to run devicetree bindings schema checks which will bring +compliance to U-Boot core/drivers regarding usage of devicetree. + +Dependencies +~~~~~~~~~~~~ + +The DT schema project must be installed in order to validate the DT schema +binding documents and validate DTS files using the DT schema. The DT schema +project can be installed with pip:: + + pip3 install dtschema + +Note that 'dtschema' installation requires 'swig' and Python development files +installed first. On Debian/Ubuntu systems:: + + apt install swig python3-dev + +Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be +installed. Ensure they are in your PATH (~/.local/bin by default). + +Recommended is also to install yamllint (used by dtschema when present). + +Running checks +~~~~~~~~~~~~~~ + +In order to perform validation of DTB files, use the ``dtbs_check`` target:: + + make dtbs_check + +It is also possible to run checks with a subset of matching schema files by +setting the ``DT_SCHEMA_FILES`` variable to 1 or more specific schema files or +patterns (partial match of a fixed string). Each file or pattern should be +separated by ':'. + +:: + + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml:rtc.yaml + make dtbs_check DT_SCHEMA_FILES=/gpio/ + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml + + Relocation, SPL and TPL ----------------------- @@ -261,8 +321,9 @@ used it before Linux (e.g. snow). The two projects developed in parallel and there are still some differences in the bindings for certain boards. While there has been discussion of having a separate repository for devicetree files, in practice the Linux kernel Git repository has become the place where -these are stored, with U-Boot taking copies and adding tweaks with u-boot.dtsi -files. +these are stored, with U-Boot taking copies via devicetree-rebasing repo +(see: dtrepo_) and adding tweaks with u-boot.dtsi files. .. _dtspec: https://www.devicetree.org/specifications/ .. _dtlist: https://www.spinics.net/lists/devicetree-compiler/ +.. _dtrepo: https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git
Encourage SoC/board maintainers to migrate to using devicetree-rebasing subtree and maintain a regular sync with Linux kernel devicetree files and bindings. Along with that add documentation regarding how to run DT bindings schema checks. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> --- Changes in v3: -------------- - Replace CONFIG_* with Kconfig options Changes in v2: -------------- - s/U-boot/U-Boot/ doc/develop/devicetree/control.rst | 131 +++++++++++++++++++++-------- 1 file changed, 96 insertions(+), 35 deletions(-)