| Message ID | 20240202130534.1051438-9-sumit.garg@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | An effort to bring DT bindings compliance within U-Boot | expand |
On 02/02/2024 13:05, Sumit Garg wrote: > Encourage SoC/board maintainers to migrate to using devicetree-rebasing > subtree and maintain a regular sync with Linux kernel devicetree files > and bindings. > > Along with that add documentation regarding how to run DT bindings > schema checks. > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > --- > > Changes in v5: > - Document how to cherry-pick fixes from devicetree-rebasing tree. > > Changes in v4: > - Switched subtree to be imported as dts/upstream sub-directory rather > than devicetree-rebasing sub-directory to better suite U-Boot > directory structure. > - Since we now have v6.7-dts tag available now, so switch subtree to > that from its beginning. > - Clarify subtree uprev schedule as a separate documentation section. > Also, fixed documentation typos. > > Changes in v3: > - Replace CONFIG_* with Kconfig options > > Changes in v2: > - s/U-boot/U-Boot/ > > doc/develop/devicetree/control.rst | 117 ++++++++++++++++++++++++----- > 1 file changed, 97 insertions(+), 20 deletions(-) > > diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst > index 9a0cb90336df..4440d4b82c6a 100644 > --- a/doc/develop/devicetree/control.rst > +++ b/doc/develop/devicetree/control.rst > @@ -1,5 +1,6 @@ > .. SPDX-License-Identifier: GPL-2.0+ > .. sectionauthor:: Copyright 2011 The Chromium OS Authors > +.. Copyright 2023-2024 Linaro Ltd. > > Devicetree Control in U-Boot > ============================ > @@ -22,12 +23,11 @@ for three reasons: > hierarchical format > - It is fairly efficient to read incrementally > > -The arch/<arch>/dts directories contains a Makefile for building the devicetree > -blob and embedding it in the U-Boot image. This is useful since it allows > -U-Boot to configure itself according to what it finds there. If you have > -a number of similar boards with different peripherals, you can describe > -the features of each board in the devicetree file, and have a single > -generic source base. > +The U-Boot Makefile infrastructure allows for building the devicetree blob > +and embedding it in the U-Boot image. This is useful since it allows U-Boot > +to configure itself according to what it finds there. If you have a number > +of similar boards with different peripherals, you can describe the features > +of each board in the devicetree file, and have a single generic source base. > > To enable this feature, select `OF_CONTROL` via Kconfig. > > @@ -68,8 +68,14 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. > Where do I get a devicetree file for my board? > ---------------------------------------------- > > -You may find that the Linux kernel has a suitable file. Look in the > -kernel source in arch/<arch>/boot/dts. > +Linux kernel Git repository has been the place where devicetree files along > +with devicetree bindings are stored and maintained. There is devicetee-rebasing > +(dtrepo_) which maintains a forked copy of devicetree files along with bindings > +at every Linux kernel major release or intermediate release candidates. This can be written more clearly, we can steal what you wrote later and expand it to say what was wrong with the previous situation: "The devicetree files and devicetree bindings are maintained as part of the Linux kernel git repository. Traditionally, U-Boot placed copies of devicetree source files from the Linux kernel into `arch/<arch>/dts/<name>.dts`. However, this required each board maintainer to manually keep their device tree in sync with Linux and often led to divergence between these copies." We can then introduce the `dts/upstream` directory and tell developers why it is a better solution. I think the docs should talk about the `dts/upstream` directory first, then the devicetree-rebasing repository afterwards. The directory in the u-boot source tree is what most developers will see and interact with, use of the devicetree-rebasing repository is an implementation detail of how that subtree is sync'd with Linux. I don't think we need to mention the devicetree-rebasing repository until the "Resyncing with devicetree-rebasing" section below. Also, is devicetree-rebasing a "forked copy"? A fork would imply some difference from upstream. I would guess this is more like a mirror. > + > +U-Boot maintains a Git subtree for devicetee-rebasing repo as `dts/upstream/` > +sub-directory. You may find that the `dts/upstream/` sub-directory has a > +suitable devicetree file for your board. Look in `dts/upstream/src/<arch>/`. > > If not you might find other boards with suitable files that you can > modify to your needs. Look in the board directories for files with a > @@ -78,17 +84,38 @@ modify to your needs. Look in the board directories for files with a > Failing that, you could write one from scratch yourself! > > > +Resyncing with devicetree-rebasing > +---------------------------------- > + > +U-Boot regularly sync `dts/upstream/` subtree whenever the next window opens "The U-Boot maintainers regularly sync the..." What's the "next window" here? Is this the merge window? Or the period after the merge window when the next branch is open? There should be a link here to relevant section of the U-Boot Development Cycle page in case the reader isn't familiar with the development process. > +with the next available kernel major release. `dts/update-dts-subtree.sh` script I think this should be "with the latest mainline kernel release." > +provides a wrapper around git subtree pull command, usage from the top level> +U-Boot source tree, run:: The use of `git subtree pull` is an implementation detail. I think the docs need to focus on what the script does, not how it does it. Maybe replace this sentence with something like: "To sync the `dts/upstream/` subtree, run the following command::" > + > + ./dts/update-dts-subtree.sh pull <devicetree-rebasing-release-tag> > + > +If required it is also possible to cherry-pick fixes from devicetree-rebasing "the devicetree-rebasing repository" here and each time it is referenced. > +tree prior to next sync, usage:: > + > + ./dts/update-dts-subtree.sh pick <devicetree-rebasing-commit-id> > + > + > Configuration > ------------- > > -Set up "<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. Then put > -your devicetree file into:: > +Traditionally, U-Boot placed copies of devicetree source files from Linux "the Linux kernel", here and each time "Linux kernel" is used. > +kernel into `arch/<arch>/dts/<name>.dts` which can be selected via setting > +"<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. > > - arch/<arch>/dts/<name>.dts > +However, it has become cumbersome over time for each SoC/board maintainer to > +keep devicetree files in sync with Linux kernel. Therefore, SoC/board > +maintainers are encouraged to migrate to use synced copies from > +`dts/upstream/src/<arch>/<vendor>`. To do that enable `OF_UPSTREAM` for the > +SoC being used via Kconfig and set up "<vendor>/<name>" when prompted for > +`DEFAULT_DEVICE_TREE` by Kconfig. I'd prefer "set `OF_UPSTREAM=y`" and "set `DEFAULT_DEVICE_TREE=<vendor>/<name>`". > > -This should include your CPU or SOC's devicetree file, placed in > -`arch/<arch>/dts`, and then make any adjustments required using a u-boot-dtsi > -file for your board. > +This should include your CPU or SOC's devicetree file. On top of that any U-Boot Change "SOC" to "SoC" while this is being modified. > +specific tweaks (see: dttweaks_) can be made for your board. > > If `OF_EMBED` is selected by Kconfig, then it will be picked up and built into > the U-Boot image (including u-boot.bin). This is suitable for debugging > @@ -155,8 +182,9 @@ ways: > Adding tweaks for U-Boot > ------------------------ > > -It is strongly recommended that devicetree files in U-Boot are an exact copy of > -those in Linux, so that it is easy to sync them up from time to time. > +With `dts/upstream` Git subtree, it is ensured that devicetree files in U-Boot > +are an exact copy of those in Linux kernel available under > +`dts/upstream/src/<arch>/<vendor>`. > > U-Boot is of course a very different project from Linux, e.g. it operates under > much more restrictive memory and code-size constraints. Where Linux may use a > @@ -169,8 +197,8 @@ constraints are even more extreme and the devicetree is shrunk to remove > unwanted nodes, or even turned into C code to avoid access overhead. > > U-Boot automatically looks for and includes a file with updates to the standard > -devicetree for your board, searching for them in the same directory as the > -main file, in this order:: > +devicetree for your board, searching for them in `arch/<arch>/dts/` in this > +order:: > > <orig_filename>-u-boot.dtsi > <CONFIG_SYS_SOC>-u-boot.dtsi > @@ -199,6 +227,54 @@ option to specify a list of .dtsi files that will also be included when > building .dtb files. > > > +Devicetree bindings schema checks > +--------------------------------- > + > +With devicetee-rebasing Git subtree, the devicetree bindings are also regularly > +synced with Linux kernel as `dts/upstream/Bindings/` sub-directory. This > +allows U-Boot to run devicetree bindings schema checks which will bring > +compliance to U-Boot core/drivers regarding usage of devicetree. > + > +Dependencies > +~~~~~~~~~~~~ > + > +The DT schema project must be installed in order to validate the DT schema > +binding documents and validate DTS files using the DT schema. The DT schema > +project can be installed with pip:: > + > + pip3 install dtschema Unfortunately this won't work on recent distro versions, e.g. on Debian 12 I get: error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.11/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. I don't have a good solution to recommend here - there's no consensus on how to install Python tools for use in development. You could use `pipx`, you could create a virtualenv, and I'm sure there are other options as well. Perhaps we just need to leave it up to the reader to find out how to install dtschema on their system. > + > +Note that 'dtschema' installation requires 'swig' and Python development files > +installed first. Please, refer to the GCC build documentation for installation > +instructions :doc:`../../build/gcc`. > + > +Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be > +installed. Ensure they are in your PATH (~/.local/bin by default). > + > +Recommended is also to install yamllint (used by dtschema when present). On "You should also install yamllint" > +Debian/Ubuntu systems:: > + > + apt install yamllint > + > +Running checks > +~~~~~~~~~~~~~~ > + > +In order to perform validation of DTB files, use the ``dtbs_check`` target:: > + > + make dtbs_check > + > +It is also possible to run checks with a subset of matching schema files by > +setting the ``DT_SCHEMA_FILES`` variable to 1 or more specific schema files or > +patterns (partial match of a fixed string). Each file or pattern should be > +separated by ':'. > + > +:: > + > + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml:rtc.yaml > + make dtbs_check DT_SCHEMA_FILES=/gpio/ > + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml > + > + > Relocation, SPL and TPL > ----------------------- > > @@ -260,8 +336,9 @@ used it before Linux (e.g. snow). The two projects developed in parallel > and there are still some differences in the bindings for certain boards. > While there has been discussion of having a separate repository for devicetree > files, in practice the Linux kernel Git repository has become the place where > -these are stored, with U-Boot taking copies and adding tweaks with u-boot.dtsi > -files. > +these are stored, with U-Boot taking copies via devicetree-rebasing repo > +(see: dtrepo_) and adding tweaks with u-boot.dtsi files. > > .. _dtspec: https://www.devicetree.org/specifications/ > .. _dtlist: https://www.spinics.net/lists/devicetree-compiler/ > +.. _dtrepo: https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git Thanks,
+ Ian Campbell (Maintainer for devicetree-rebasing tree) Hi Paul, Thanks for your nice documentation review. On Wed, 14 Feb 2024 at 03:01, Paul Barker <paul.barker.ct@bp.renesas.com> wrote: > > On 02/02/2024 13:05, Sumit Garg wrote: > > Encourage SoC/board maintainers to migrate to using devicetree-rebasing > > subtree and maintain a regular sync with Linux kernel devicetree files > > and bindings. > > > > Along with that add documentation regarding how to run DT bindings > > schema checks. > > > > Signed-off-by: Sumit Garg <sumit.garg@linaro.org> > > --- > > > > Changes in v5: > > - Document how to cherry-pick fixes from devicetree-rebasing tree. > > > > Changes in v4: > > - Switched subtree to be imported as dts/upstream sub-directory rather > > than devicetree-rebasing sub-directory to better suite U-Boot > > directory structure. > > - Since we now have v6.7-dts tag available now, so switch subtree to > > that from its beginning. > > - Clarify subtree uprev schedule as a separate documentation section. > > Also, fixed documentation typos. > > > > Changes in v3: > > - Replace CONFIG_* with Kconfig options > > > > Changes in v2: > > - s/U-boot/U-Boot/ > > > > doc/develop/devicetree/control.rst | 117 ++++++++++++++++++++++++----- > > 1 file changed, 97 insertions(+), 20 deletions(-) > > > > diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst > > index 9a0cb90336df..4440d4b82c6a 100644 > > --- a/doc/develop/devicetree/control.rst > > +++ b/doc/develop/devicetree/control.rst > > @@ -1,5 +1,6 @@ > > .. SPDX-License-Identifier: GPL-2.0+ > > .. sectionauthor:: Copyright 2011 The Chromium OS Authors > > +.. Copyright 2023-2024 Linaro Ltd. > > > > Devicetree Control in U-Boot > > ============================ > > @@ -22,12 +23,11 @@ for three reasons: > > hierarchical format > > - It is fairly efficient to read incrementally > > > > -The arch/<arch>/dts directories contains a Makefile for building the devicetree > > -blob and embedding it in the U-Boot image. This is useful since it allows > > -U-Boot to configure itself according to what it finds there. If you have > > -a number of similar boards with different peripherals, you can describe > > -the features of each board in the devicetree file, and have a single > > -generic source base. > > +The U-Boot Makefile infrastructure allows for building the devicetree blob > > +and embedding it in the U-Boot image. This is useful since it allows U-Boot > > +to configure itself according to what it finds there. If you have a number > > +of similar boards with different peripherals, you can describe the features > > +of each board in the devicetree file, and have a single generic source base. > > > > To enable this feature, select `OF_CONTROL` via Kconfig. > > > > @@ -68,8 +68,14 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. > > Where do I get a devicetree file for my board? > > ---------------------------------------------- > > > > -You may find that the Linux kernel has a suitable file. Look in the > > -kernel source in arch/<arch>/boot/dts. > > +Linux kernel Git repository has been the place where devicetree files along > > +with devicetree bindings are stored and maintained. There is devicetee-rebasing > > +(dtrepo_) which maintains a forked copy of devicetree files along with bindings > > +at every Linux kernel major release or intermediate release candidates. > > This can be written more clearly, we can steal what you wrote later and > expand it to say what was wrong with the previous situation: > > "The devicetree files and devicetree bindings are maintained as part of > the Linux kernel git repository. Traditionally, U-Boot placed copies of > devicetree source files from the Linux kernel into > `arch/<arch>/dts/<name>.dts`. However, this required each board > maintainer to manually keep their device tree in sync with Linux and > often led to divergence between these copies." > > We can then introduce the `dts/upstream` directory and tell developers > why it is a better solution. Ack, I will try to reorganize the contents as per your suggestion. > > I think the docs should talk about the `dts/upstream` directory first, > then the devicetree-rebasing repository afterwards. The directory in the > u-boot source tree is what most developers will see and interact with, > use of the devicetree-rebasing repository is an implementation detail of > how that subtree is sync'd with Linux. I don't think we need to mention > the devicetree-rebasing repository until the "Resyncing with > devicetree-rebasing" section below. Ack. > > Also, is devicetree-rebasing a "forked copy"? A fork would imply some > difference from upstream. I would guess this is more like a mirror. AFAIK, it's a mirror from upstream with a different directory (Makefile) structure. Ian may clarify further as to how this mirror is kept updated. > > > + > > +U-Boot maintains a Git subtree for devicetee-rebasing repo as `dts/upstream/` > > +sub-directory. You may find that the `dts/upstream/` sub-directory has a > > +suitable devicetree file for your board. Look in `dts/upstream/src/<arch>/`. > > > > If not you might find other boards with suitable files that you can > > modify to your needs. Look in the board directories for files with a > > @@ -78,17 +84,38 @@ modify to your needs. Look in the board directories for files with a > > Failing that, you could write one from scratch yourself! > > > > > > +Resyncing with devicetree-rebasing > > +---------------------------------- > > + > > +U-Boot regularly sync `dts/upstream/` subtree whenever the next window opens > > "The U-Boot maintainers regularly sync the..." > > What's the "next window" here? Is this the merge window? Or the period > after the merge window when the next branch is open? It's when the next branch is open. I will correct it. > There should be a > link here to relevant section of the U-Boot Development Cycle page in > case the reader isn't familiar with the development process. Ack, I will add that. > > > +with the next available kernel major release. `dts/update-dts-subtree.sh` script > > I think this should be "with the latest mainline kernel release." Ack. > > > +provides a wrapper around git subtree pull command, usage from the top level> +U-Boot source tree, run:: > > The use of `git subtree pull` is an implementation detail. I think the > docs need to focus on what the script does, not how it does it. > > Maybe replace this sentence with something like: > > "To sync the `dts/upstream/` subtree, run the following command::" > Ack. > > + > > + ./dts/update-dts-subtree.sh pull <devicetree-rebasing-release-tag> > > + > > +If required it is also possible to cherry-pick fixes from devicetree-rebasing > > "the devicetree-rebasing repository" here and each time it is referenced. Ack. > > > +tree prior to next sync, usage:: > > + > > + ./dts/update-dts-subtree.sh pick <devicetree-rebasing-commit-id> > > + > > + > > Configuration > > ------------- > > > > -Set up "<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. Then put > > -your devicetree file into:: > > +Traditionally, U-Boot placed copies of devicetree source files from Linux > > "the Linux kernel", here and each time "Linux kernel" is used. Ack. > > > +kernel into `arch/<arch>/dts/<name>.dts` which can be selected via setting > > +"<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. > > > > - arch/<arch>/dts/<name>.dts > > +However, it has become cumbersome over time for each SoC/board maintainer to > > +keep devicetree files in sync with Linux kernel. Therefore, SoC/board > > +maintainers are encouraged to migrate to use synced copies from > > +`dts/upstream/src/<arch>/<vendor>`. To do that enable `OF_UPSTREAM` for the > > +SoC being used via Kconfig and set up "<vendor>/<name>" when prompted for > > +`DEFAULT_DEVICE_TREE` by Kconfig. > > I'd prefer "set `OF_UPSTREAM=y`" As discussed earlier OF_UPSTREAM is a SoC specific config option, so this can be more explicit via: "... add `imply OF_UPSTREAM` for the SoC ...". > and "set `DEFAULT_DEVICE_TREE=<vendor>/<name>`". Ack. > > > > > -This should include your CPU or SOC's devicetree file, placed in > > -`arch/<arch>/dts`, and then make any adjustments required using a u-boot-dtsi > > -file for your board. > > +This should include your CPU or SOC's devicetree file. On top of that any U-Boot > > Change "SOC" to "SoC" while this is being modified. Ack. > > > +specific tweaks (see: dttweaks_) can be made for your board. > > > > If `OF_EMBED` is selected by Kconfig, then it will be picked up and built into > > the U-Boot image (including u-boot.bin). This is suitable for debugging > > @@ -155,8 +182,9 @@ ways: > > Adding tweaks for U-Boot > > ------------------------ > > > > -It is strongly recommended that devicetree files in U-Boot are an exact copy of > > -those in Linux, so that it is easy to sync them up from time to time. > > +With `dts/upstream` Git subtree, it is ensured that devicetree files in U-Boot > > +are an exact copy of those in Linux kernel available under > > +`dts/upstream/src/<arch>/<vendor>`. > > > > U-Boot is of course a very different project from Linux, e.g. it operates under > > much more restrictive memory and code-size constraints. Where Linux may use a > > @@ -169,8 +197,8 @@ constraints are even more extreme and the devicetree is shrunk to remove > > unwanted nodes, or even turned into C code to avoid access overhead. > > > > U-Boot automatically looks for and includes a file with updates to the standard > > -devicetree for your board, searching for them in the same directory as the > > -main file, in this order:: > > +devicetree for your board, searching for them in `arch/<arch>/dts/` in this > > +order:: > > > > <orig_filename>-u-boot.dtsi > > <CONFIG_SYS_SOC>-u-boot.dtsi > > @@ -199,6 +227,54 @@ option to specify a list of .dtsi files that will also be included when > > building .dtb files. > > > > > > +Devicetree bindings schema checks > > +--------------------------------- > > + > > +With devicetee-rebasing Git subtree, the devicetree bindings are also regularly > > +synced with Linux kernel as `dts/upstream/Bindings/` sub-directory. This > > +allows U-Boot to run devicetree bindings schema checks which will bring > > +compliance to U-Boot core/drivers regarding usage of devicetree. > > + > > +Dependencies > > +~~~~~~~~~~~~ > > + > > +The DT schema project must be installed in order to validate the DT schema > > +binding documents and validate DTS files using the DT schema. The DT schema > > +project can be installed with pip:: > > + > > + pip3 install dtschema > > Unfortunately this won't work on recent distro versions, e.g. on Debian > 12 I get: > > error: externally-managed-environment > > × This environment is externally managed > ╰─> To install Python packages system-wide, try apt install > python3-xyz, where xyz is the package you are trying to > install. > > If you wish to install a non-Debian-packaged Python package, > create a virtual environment using python3 -m venv path/to/venv. > Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make > sure you have python3-full installed. > > If you wish to install a non-Debian packaged Python application, > it may be easiest to use pipx install xyz, which will manage a > virtual environment for you. Make sure you have pipx installed. > > See /usr/share/doc/python3.11/README.venv for more information. > > note: If you believe this is a mistake, please contact your Python > installation or OS distribution provider. You can override this, at the > risk of breaking your Python installation or OS, by passing > --break-system-packages. > hint: See PEP 668 for the detailed specification. > > I don't have a good solution to recommend here - there's no consensus on > how to install Python tools for use in development. You could use > `pipx`, you could create a virtualenv, and I'm sure there are other > options as well. Perhaps we just need to leave it up to the reader to > find out how to install dtschema on their system. I suppose you haven't installed python3-pip package [1] on your Debian 12. BTW, `pip3` is being used as the common way to install dtschema [2]. Also, every user may not be aware about python virtual environments. So it's better to provide the commonly used pip3 option for installation. [1] https://packages.debian.org/bookworm/python3-pip [2] https://github.com/devicetree-org/dt-schema/tree/main?tab=readme-ov-file#installing > > > + > > +Note that 'dtschema' installation requires 'swig' and Python development files > > +installed first. Please, refer to the GCC build documentation for installation > > +instructions :doc:`../../build/gcc`. > > + > > +Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be > > +installed. Ensure they are in your PATH (~/.local/bin by default). > > + > > +Recommended is also to install yamllint (used by dtschema when present). On > > "You should also install yamllint" Ack. -Sumit
On Wed, 2024-02-14 at 19:02 +0530, Sumit Garg wrote: > > > > > Also, is devicetree-rebasing a "forked copy"? A fork would imply some > > difference from upstream. I would guess this is more like a mirror. > > AFAIK, it's a mirror from upstream with a different directory > (Makefile) structure. Ian may clarify further as to how this mirror is > kept updated. It's not really a mirror nor a fork, I guess it is something in between. It's a rewritten tree which extracts the DTS bits only from the kernel source into a standalone source repo. It's produced using "git filter- branch", all the scripting to do so is at: https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git/tree/scripts The conversion state is part of https://github.com/ijc/devicetree-conversion-state-v3/, that repo contains all of upstream Linux commits too, since the branches there track the most recently converted inputs to support incremental updates. There's also the filter-map branch which maintains the state needed to not have to redo the conversion every time (which from scratch took days last time...) Ian
On 14/02/2024 13:32, Sumit Garg wrote: > On Wed, 14 Feb 2024 at 03:01, Paul Barker <paul.barker.ct@bp.renesas.com> wrote: >> On 02/02/2024 13:05, Sumit Garg wrote: >>> +Dependencies >>> +~~~~~~~~~~~~ >>> + >>> +The DT schema project must be installed in order to validate the DT schema >>> +binding documents and validate DTS files using the DT schema. The DT schema >>> +project can be installed with pip:: >>> + >>> + pip3 install dtschema >> >> Unfortunately this won't work on recent distro versions, e.g. on Debian >> 12 I get: >> >> error: externally-managed-environment >> >> × This environment is externally managed >> ╰─> To install Python packages system-wide, try apt install >> python3-xyz, where xyz is the package you are trying to >> install. >> >> If you wish to install a non-Debian-packaged Python package, >> create a virtual environment using python3 -m venv path/to/venv. >> Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make >> sure you have python3-full installed. >> >> If you wish to install a non-Debian packaged Python application, >> it may be easiest to use pipx install xyz, which will manage a >> virtual environment for you. Make sure you have pipx installed. >> >> See /usr/share/doc/python3.11/README.venv for more information. >> >> note: If you believe this is a mistake, please contact your Python >> installation or OS distribution provider. You can override this, at the >> risk of breaking your Python installation or OS, by passing >> --break-system-packages. >> hint: See PEP 668 for the detailed specification. >> >> I don't have a good solution to recommend here - there's no consensus on >> how to install Python tools for use in development. You could use >> `pipx`, you could create a virtualenv, and I'm sure there are other >> options as well. Perhaps we just need to leave it up to the reader to >> find out how to install dtschema on their system. > > I suppose you haven't installed python3-pip package [1] on your Debian > 12. BTW, `pip3` is being used as the common way to install dtschema > [2]. Also, every user may not be aware about python virtual > environments. So it's better to provide the commonly used pip3 option > for installation. The python3-pip package is installed, it's the pip3 program that is printing the error message I saw. This is the new behaviour in Debian 12, also seen in rolling distributions like Arch Linux and OpenSuSE Tumbleweed since the middle of last year. There is a good guide to this from Python=>Speed [1] and more detail in the Python packaging spec [2]. Sadly it's something we now have to live with in recent Linux distro releases, and I expect all distros will enable this in upcoming releases. [1]: https://pythonspeed.com/articles/externally-managed-environment-pep-668/ [2]: https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments Thanks,
On Fri, 16 Feb 2024 at 16:17, Paul Barker <paul.barker.ct@bp.renesas.com> wrote: > > On 14/02/2024 13:32, Sumit Garg wrote: > > On Wed, 14 Feb 2024 at 03:01, Paul Barker <paul.barker.ct@bp.renesas.com> wrote: > >> On 02/02/2024 13:05, Sumit Garg wrote: > >>> +Dependencies > >>> +~~~~~~~~~~~~ > >>> + > >>> +The DT schema project must be installed in order to validate the DT schema > >>> +binding documents and validate DTS files using the DT schema. The DT schema > >>> +project can be installed with pip:: > >>> + > >>> + pip3 install dtschema > >> > >> Unfortunately this won't work on recent distro versions, e.g. on Debian > >> 12 I get: > >> > >> error: externally-managed-environment > >> > >> × This environment is externally managed > >> ╰─> To install Python packages system-wide, try apt install > >> python3-xyz, where xyz is the package you are trying to > >> install. > >> > >> If you wish to install a non-Debian-packaged Python package, > >> create a virtual environment using python3 -m venv path/to/venv. > >> Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make > >> sure you have python3-full installed. > >> > >> If you wish to install a non-Debian packaged Python application, > >> it may be easiest to use pipx install xyz, which will manage a > >> virtual environment for you. Make sure you have pipx installed. > >> > >> See /usr/share/doc/python3.11/README.venv for more information. > >> > >> note: If you believe this is a mistake, please contact your Python > >> installation or OS distribution provider. You can override this, at the > >> risk of breaking your Python installation or OS, by passing > >> --break-system-packages. > >> hint: See PEP 668 for the detailed specification. > >> > >> I don't have a good solution to recommend here - there's no consensus on > >> how to install Python tools for use in development. You could use > >> `pipx`, you could create a virtualenv, and I'm sure there are other > >> options as well. Perhaps we just need to leave it up to the reader to > >> find out how to install dtschema on their system. > > > > I suppose you haven't installed python3-pip package [1] on your Debian > > 12. BTW, `pip3` is being used as the common way to install dtschema > > [2]. Also, every user may not be aware about python virtual > > environments. So it's better to provide the commonly used pip3 option > > for installation. > > The python3-pip package is installed, it's the pip3 program that is > printing the error message I saw. This is the new behaviour in Debian > 12, also seen in rolling distributions like Arch Linux and OpenSuSE > Tumbleweed since the middle of last year. There is a good guide to this > from Python=>Speed [1] and more detail in the Python packaging spec [2]. > Sadly it's something we now have to live with in recent Linux distro > releases, and I expect all distros will enable this in upcoming > releases. Okay I see, thanks for bringing it up. I hope Rob will take note of this and update dtschema installation instructions [1] accordingly. As part of v6, I will rather guide users towards the dtschema github page in order to get installation instructions. [1] https://github.com/devicetree-org/dt-schema/tree/main?tab=readme-ov-file#installing -Sumit > > [1]: https://pythonspeed.com/articles/externally-managed-environment-pep-668/ > [2]: https://packaging.python.org/en/latest/specifications/externally-managed-environments/#externally-managed-environments > > Thanks, > > -- > Paul Barker
diff --git a/doc/develop/devicetree/control.rst b/doc/develop/devicetree/control.rst index 9a0cb90336df..4440d4b82c6a 100644 --- a/doc/develop/devicetree/control.rst +++ b/doc/develop/devicetree/control.rst @@ -1,5 +1,6 @@ .. SPDX-License-Identifier: GPL-2.0+ .. sectionauthor:: Copyright 2011 The Chromium OS Authors +.. Copyright 2023-2024 Linaro Ltd. Devicetree Control in U-Boot ============================ @@ -22,12 +23,11 @@ for three reasons: hierarchical format - It is fairly efficient to read incrementally -The arch/<arch>/dts directories contains a Makefile for building the devicetree -blob and embedding it in the U-Boot image. This is useful since it allows -U-Boot to configure itself according to what it finds there. If you have -a number of similar boards with different peripherals, you can describe -the features of each board in the devicetree file, and have a single -generic source base. +The U-Boot Makefile infrastructure allows for building the devicetree blob +and embedding it in the U-Boot image. This is useful since it allows U-Boot +to configure itself according to what it finds there. If you have a number +of similar boards with different peripherals, you can describe the features +of each board in the devicetree file, and have a single generic source base. To enable this feature, select `OF_CONTROL` via Kconfig. @@ -68,8 +68,14 @@ a binary file. U-Boot adds its own `fdtgrep` for creating subsets of the file. Where do I get a devicetree file for my board? ---------------------------------------------- -You may find that the Linux kernel has a suitable file. Look in the -kernel source in arch/<arch>/boot/dts. +Linux kernel Git repository has been the place where devicetree files along +with devicetree bindings are stored and maintained. There is devicetee-rebasing +(dtrepo_) which maintains a forked copy of devicetree files along with bindings +at every Linux kernel major release or intermediate release candidates. + +U-Boot maintains a Git subtree for devicetee-rebasing repo as `dts/upstream/` +sub-directory. You may find that the `dts/upstream/` sub-directory has a +suitable devicetree file for your board. Look in `dts/upstream/src/<arch>/`. If not you might find other boards with suitable files that you can modify to your needs. Look in the board directories for files with a @@ -78,17 +84,38 @@ modify to your needs. Look in the board directories for files with a Failing that, you could write one from scratch yourself! +Resyncing with devicetree-rebasing +---------------------------------- + +U-Boot regularly sync `dts/upstream/` subtree whenever the next window opens +with the next available kernel major release. `dts/update-dts-subtree.sh` script +provides a wrapper around git subtree pull command, usage from the top level +U-Boot source tree, run:: + + ./dts/update-dts-subtree.sh pull <devicetree-rebasing-release-tag> + +If required it is also possible to cherry-pick fixes from devicetree-rebasing +tree prior to next sync, usage:: + + ./dts/update-dts-subtree.sh pick <devicetree-rebasing-commit-id> + + Configuration ------------- -Set up "<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. Then put -your devicetree file into:: +Traditionally, U-Boot placed copies of devicetree source files from Linux +kernel into `arch/<arch>/dts/<name>.dts` which can be selected via setting +"<name>" when prompted for `DEFAULT_DEVICE_TREE` by Kconfig. - arch/<arch>/dts/<name>.dts +However, it has become cumbersome over time for each SoC/board maintainer to +keep devicetree files in sync with Linux kernel. Therefore, SoC/board +maintainers are encouraged to migrate to use synced copies from +`dts/upstream/src/<arch>/<vendor>`. To do that enable `OF_UPSTREAM` for the +SoC being used via Kconfig and set up "<vendor>/<name>" when prompted for +`DEFAULT_DEVICE_TREE` by Kconfig. -This should include your CPU or SOC's devicetree file, placed in -`arch/<arch>/dts`, and then make any adjustments required using a u-boot-dtsi -file for your board. +This should include your CPU or SOC's devicetree file. On top of that any U-Boot +specific tweaks (see: dttweaks_) can be made for your board. If `OF_EMBED` is selected by Kconfig, then it will be picked up and built into the U-Boot image (including u-boot.bin). This is suitable for debugging @@ -155,8 +182,9 @@ ways: Adding tweaks for U-Boot ------------------------ -It is strongly recommended that devicetree files in U-Boot are an exact copy of -those in Linux, so that it is easy to sync them up from time to time. +With `dts/upstream` Git subtree, it is ensured that devicetree files in U-Boot +are an exact copy of those in Linux kernel available under +`dts/upstream/src/<arch>/<vendor>`. U-Boot is of course a very different project from Linux, e.g. it operates under much more restrictive memory and code-size constraints. Where Linux may use a @@ -169,8 +197,8 @@ constraints are even more extreme and the devicetree is shrunk to remove unwanted nodes, or even turned into C code to avoid access overhead. U-Boot automatically looks for and includes a file with updates to the standard -devicetree for your board, searching for them in the same directory as the -main file, in this order:: +devicetree for your board, searching for them in `arch/<arch>/dts/` in this +order:: <orig_filename>-u-boot.dtsi <CONFIG_SYS_SOC>-u-boot.dtsi @@ -199,6 +227,54 @@ option to specify a list of .dtsi files that will also be included when building .dtb files. +Devicetree bindings schema checks +--------------------------------- + +With devicetee-rebasing Git subtree, the devicetree bindings are also regularly +synced with Linux kernel as `dts/upstream/Bindings/` sub-directory. This +allows U-Boot to run devicetree bindings schema checks which will bring +compliance to U-Boot core/drivers regarding usage of devicetree. + +Dependencies +~~~~~~~~~~~~ + +The DT schema project must be installed in order to validate the DT schema +binding documents and validate DTS files using the DT schema. The DT schema +project can be installed with pip:: + + pip3 install dtschema + +Note that 'dtschema' installation requires 'swig' and Python development files +installed first. Please, refer to the GCC build documentation for installation +instructions :doc:`../../build/gcc`. + +Several executables (dt-doc-validate, dt-mk-schema, dt-validate) will be +installed. Ensure they are in your PATH (~/.local/bin by default). + +Recommended is also to install yamllint (used by dtschema when present). On +Debian/Ubuntu systems:: + + apt install yamllint + +Running checks +~~~~~~~~~~~~~~ + +In order to perform validation of DTB files, use the ``dtbs_check`` target:: + + make dtbs_check + +It is also possible to run checks with a subset of matching schema files by +setting the ``DT_SCHEMA_FILES`` variable to 1 or more specific schema files or +patterns (partial match of a fixed string). Each file or pattern should be +separated by ':'. + +:: + + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml:rtc.yaml + make dtbs_check DT_SCHEMA_FILES=/gpio/ + make dtbs_check DT_SCHEMA_FILES=trivial-devices.yaml + + Relocation, SPL and TPL ----------------------- @@ -260,8 +336,9 @@ used it before Linux (e.g. snow). The two projects developed in parallel and there are still some differences in the bindings for certain boards. While there has been discussion of having a separate repository for devicetree files, in practice the Linux kernel Git repository has become the place where -these are stored, with U-Boot taking copies and adding tweaks with u-boot.dtsi -files. +these are stored, with U-Boot taking copies via devicetree-rebasing repo +(see: dtrepo_) and adding tweaks with u-boot.dtsi files. .. _dtspec: https://www.devicetree.org/specifications/ .. _dtlist: https://www.spinics.net/lists/devicetree-compiler/ +.. _dtrepo: https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git
Encourage SoC/board maintainers to migrate to using devicetree-rebasing subtree and maintain a regular sync with Linux kernel devicetree files and bindings. Along with that add documentation regarding how to run DT bindings schema checks. Signed-off-by: Sumit Garg <sumit.garg@linaro.org> --- Changes in v5: - Document how to cherry-pick fixes from devicetree-rebasing tree. Changes in v4: - Switched subtree to be imported as dts/upstream sub-directory rather than devicetree-rebasing sub-directory to better suite U-Boot directory structure. - Since we now have v6.7-dts tag available now, so switch subtree to that from its beginning. - Clarify subtree uprev schedule as a separate documentation section. Also, fixed documentation typos. Changes in v3: - Replace CONFIG_* with Kconfig options Changes in v2: - s/U-boot/U-Boot/ doc/develop/devicetree/control.rst | 117 ++++++++++++++++++++++++----- 1 file changed, 97 insertions(+), 20 deletions(-)