| Message ID | 20230908135320.7066-1-maxim.uvarov@linaro.org |
|---|---|
| Headers | show |
| Series | net/lwip: add lwip library for the network stack | expand |
On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: > Before apply these patches it is needed to create lwIP merge into U-Boot: > git subtree add --prefix net/lwip/lwip-external https://git.savannah.nongnu.org/git/lwip.git master --squash > or > create git submodule, depends how it's more easy to maintain external > library. So, I think we're going to go with subtree. Please work out how to integrate the above in to the build process automatically (and such that we can maintain it via upgrades moving forward).
On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini@konsulko.com> wrote: > On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: > > > Before apply these patches it is needed to create lwIP merge into > U-Boot: > > git subtree add --prefix net/lwip/lwip-external > https://git.savannah.nongnu.org/git/lwip.git master --squash > > or > > create git submodule, depends how it's more easy to maintain external > > library. > > So, I think we're going to go with subtree. Please work out how to > integrate the above in to the build process automatically (and such that > we can maintain it via upgrades moving forward). > > -- > Tom > I did not find a good way to friend git format-patch, git am and subtree. And now with using subtree I can provide my thoughts, in general I do not see any big advantages with maintaining subtree code. Problem is that: 1. subtree does some root reset. So rebase looks like: label onto # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2 reset [new root] pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit 84fde1ebbf label acbc0469a49de7055141cc730aa9c728e61b6de2-2 reset onto merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as 'net/lwip/lwip-external' pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst pick f0ecab85e0 net/lwip: integrate lwIP library 2. if --rebase-merges option was not provided to rebase, then rebase will omit subtree directory and try to apply rebase patches to root directory. I.e. in current case squashed commit for lwip, will be applied to uboot root directory instead of ./net/lwip/lwip-external. 3. due to broken rebases without --rebase-merges more likely things like git bisect also will not work. 4. change in subtree code ./net/lwip/lwip-external/../something.c will create a git commit which looks like a standard U-Boot commit. I.e. path starts with ./net/lwip/lwip-external/ 5. lwip maintains code with a mailing list. So we don't need to push subtree git somewhere to create a pull request. 6. I rechecked the latest edk2 and they use submodules now. (openssl, libfdt, berkeley-softfloat-3 and others). 7. dynamic download also looks horrible for me. I.e. create subtree in Makefile on compilation process. I think maintaining that will give you a bunch of problems. I think we should not touch git structure after cloning. So what I can here suggest: 1. lwip source code is 9.4M. If we compare all code it will be 564M in total. So just having 1 commit witn copy of lwip library will work here. or 2. use git submodules. Size of the project will be lower. Submodule will not allow you to use local changes. I.e. change needs to be merged into the upstream project and then you can update git HEAD for the submodule. or 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. Then use git submoule to connect this branch as a folder to the main U-Boot code. BR, Maxim.
Hi Maxim, On Tue, 12 Sept 2023 at 05:42, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini@konsulko.com> wrote: > > > On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: > > > > > Before apply these patches it is needed to create lwIP merge into > > U-Boot: > > > git subtree add --prefix net/lwip/lwip-external > > https://git.savannah.nongnu.org/git/lwip.git master --squash > > > or > > > create git submodule, depends how it's more easy to maintain external > > > library. > > > > So, I think we're going to go with subtree. Please work out how to > > integrate the above in to the build process automatically (and such that > > we can maintain it via upgrades moving forward). > > > > -- > > Tom > > > > I did not find a good way to friend git format-patch, git am and subtree. > And now with using subtree I can provide my thoughts, in general I do not > see any big advantages > with maintaining subtree code. > > Problem is that: > > 1. subtree does some root reset. So rebase looks like: > label onto > > > > # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2 > > reset [new root] > > pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit > 84fde1ebbf > label acbc0469a49de7055141cc730aa9c728e61b6de2-2 > > > > reset onto > > merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge > commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as > 'net/lwip/lwip-external' > pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst > > pick f0ecab85e0 net/lwip: integrate lwIP library > > 2. if --rebase-merges option was not provided to rebase, then rebase will > omit subtree directory and try to apply rebase patches to root directory. > I.e. in current case squashed commit for lwip, will be applied to uboot > root directory instead of ./net/lwip/lwip-external. > > 3. due to broken rebases without --rebase-merges more likely things like > git bisect also will not work. > > 4. change in subtree code ./net/lwip/lwip-external/../something.c will > create a git commit which looks like a standard U-Boot commit. I.e. path > starts with ./net/lwip/lwip-external/ I don't really understand most of the above, but I take it that subtree has some problems...I did find an article about subtree [1] > > 5. lwip maintains code with a mailing list. So we don't need to push > subtree git somewhere to create a pull request. > > 6. I rechecked the latest edk2 and they use submodules now. (openssl, > libfdt, berkeley-softfloat-3 and others). > > 7. dynamic download also looks horrible for me. I.e. create subtree in > Makefile on compilation process. I think maintaining that will give you a > bunch of problems. I think we should not touch git structure after cloning. > > So what I can here suggest: > 1. lwip source code is 9.4M. If we compare all code it will be 564M in > total. So just having 1 commit witn copy of lwip library will work here. So we add a 9.4MB patch for the code we need? I suppose that is OK, although it is much larger than net/ today (0.5MB). What is the churn on lwip? E.g. would it be easy to add a commit every few months to bring in upstream changes? > > or > > 2. use git submodules. Size of the project will be lower. Submodule will > not allow you to use local changes. I.e. change needs to be merged into the > upstream project and then you can update git HEAD for the submodule. I really don't want to work with a submodule project. I've just had too many problems. > > or > > 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. Then > use git submoule to connect this branch as a folder to the main U-Boot code. It really needs to be properly part of U-Boot. > > BR, > Maxim. Regards, Simon [1] https://www.atlassian.com/git/tutorials/git-subtree
On Wed, 13 Sept 2023 at 01:27, Simon Glass <sjg@google.com> wrote: > Hi Maxim, > > On Tue, 12 Sept 2023 at 05:42, Maxim Uvarov <maxim.uvarov@linaro.org> > wrote: > > > > On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini@konsulko.com> wrote: > > > > > On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: > > > > > > > Before apply these patches it is needed to create lwIP merge into > > > U-Boot: > > > > git subtree add --prefix net/lwip/lwip-external > > > https://git.savannah.nongnu.org/git/lwip.git master --squash > > > > or > > > > create git submodule, depends how it's more easy to maintain external > > > > library. > > > > > > So, I think we're going to go with subtree. Please work out how to > > > integrate the above in to the build process automatically (and such > that > > > we can maintain it via upgrades moving forward). > > > > > > -- > > > Tom > > > > > > > I did not find a good way to friend git format-patch, git am and subtree. > > And now with using subtree I can provide my thoughts, in general I do not > > see any big advantages > > with maintaining subtree code. > > > > Problem is that: > > > > 1. subtree does some root reset. So rebase looks like: > > label onto > > > > > > > > # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2 > > > > reset [new root] > > > > pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit > > 84fde1ebbf > > label acbc0469a49de7055141cc730aa9c728e61b6de2-2 > > > > > > > > reset onto > > > > merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge > > commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as > > 'net/lwip/lwip-external' > > pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst > > > > pick f0ecab85e0 net/lwip: integrate lwIP library > > > > 2. if --rebase-merges option was not provided to rebase, then rebase > will > > omit subtree directory and try to apply rebase patches to root directory. > > I.e. in current case squashed commit for lwip, will be applied to uboot > > root directory instead of ./net/lwip/lwip-external. > > > > 3. due to broken rebases without --rebase-merges more likely things like > > git bisect also will not work. > > > > 4. change in subtree code ./net/lwip/lwip-external/../something.c will > > create a git commit which looks like a standard U-Boot commit. I.e. path > > starts with ./net/lwip/lwip-external/ > > I don't really understand most of the above, but I take it that > subtree has some problems...I did find an article about subtree [1] > > > > > 5. lwip maintains code with a mailing list. So we don't need to push > > subtree git somewhere to create a pull request. > > > > 6. I rechecked the latest edk2 and they use submodules now. (openssl, > > libfdt, berkeley-softfloat-3 and others). > > > > 7. dynamic download also looks horrible for me. I.e. create subtree in > > Makefile on compilation process. I think maintaining that will give you a > > bunch of problems. I think we should not touch git structure after > cloning. > > > > So what I can here suggest: > > 1. lwip source code is 9.4M. If we compare all code it will be 564M in > > total. So just having 1 commit witn copy of lwip library will work here. > > So we add a 9.4MB patch for the code we need? I suppose that is OK, > although it is much larger than net/ today (0.5MB). > > What is the churn on lwip? E.g. would it be easy to add a commit every > few months to bring in upstream changes? > > > > > or > > > > 2. use git submodules. Size of the project will be lower. Submodule will > > not allow you to use local changes. I.e. change needs to be merged into > the > > upstream project and then you can update git HEAD for the submodule. > > I really don't want to work with a submodule project. I've just had > too many problems. > > > > > or > > > > 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. > Then > > use git submoule to connect this branch as a folder to the main U-Boot > code. > > It really needs to be properly part of U-Boot. > > Ok. Then more likely we don't need all the git history of lwip inside uboot.git. Then the option with a single commit is more preferable. Then we can use part 2 of this article, how to go with standard git commands: 1. <cmd> git remote add -f lwip https://git.savannah.nongnu.org/git/lwip.git git read-tree --prefix=net/lwip/lwip-external -u lwip/master git commit -m "lwip merge sha: xxxx" </cmd> this will create a git format-patch friendly commit. Then we send it to the mailing list and apply. I hope the mailing list will allow us to send a 7.8 MB patch. Then if for development you need to pull he history of lwip, you can do it with: git pull -s subtree lwip master --allow-unrelated-histories (but I think nobody will need this.) New update of the lwip net/lwip/lwip-external dir will be done with: git pull -s subtree lwip master --allow-unrelated-histories --squash Squash commit also has to be git format-patch friendly. If you are ok with that proposal I will send v9 with the first patch created with steps above. Thanks, Maxim. > > > > BR, > > Maxim. > > Regards, > Simon > > [1] https://www.atlassian.com/git/tutorials/git-subtree >
Hi Maxim, On Wed, 13 Sept 2023 at 10:32, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > > On Wed, 13 Sept 2023 at 01:27, Simon Glass <sjg@google.com> wrote: >> >> Hi Maxim, >> >> On Tue, 12 Sept 2023 at 05:42, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: >> > >> > On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini@konsulko.com> wrote: >> > >> > > On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: >> > > >> > > > Before apply these patches it is needed to create lwIP merge into >> > > U-Boot: >> > > > git subtree add --prefix net/lwip/lwip-external >> > > https://git.savannah.nongnu.org/git/lwip.git master --squash >> > > > or >> > > > create git submodule, depends how it's more easy to maintain external >> > > > library. >> > > >> > > So, I think we're going to go with subtree. Please work out how to >> > > integrate the above in to the build process automatically (and such that >> > > we can maintain it via upgrades moving forward). >> > > >> > > -- >> > > Tom >> > > >> > >> > I did not find a good way to friend git format-patch, git am and subtree. >> > And now with using subtree I can provide my thoughts, in general I do not >> > see any big advantages >> > with maintaining subtree code. >> > >> > Problem is that: >> > >> > 1. subtree does some root reset. So rebase looks like: >> > label onto >> > >> > >> > >> > # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2 >> > >> > reset [new root] >> > >> > pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit >> > 84fde1ebbf >> > label acbc0469a49de7055141cc730aa9c728e61b6de2-2 >> > >> > >> > >> > reset onto >> > >> > merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge >> > commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as >> > 'net/lwip/lwip-external' >> > pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst >> > >> > pick f0ecab85e0 net/lwip: integrate lwIP library >> > >> > 2. if --rebase-merges option was not provided to rebase, then rebase will >> > omit subtree directory and try to apply rebase patches to root directory. >> > I.e. in current case squashed commit for lwip, will be applied to uboot >> > root directory instead of ./net/lwip/lwip-external. >> > >> > 3. due to broken rebases without --rebase-merges more likely things like >> > git bisect also will not work. >> > >> > 4. change in subtree code ./net/lwip/lwip-external/../something.c will >> > create a git commit which looks like a standard U-Boot commit. I.e. path >> > starts with ./net/lwip/lwip-external/ >> >> I don't really understand most of the above, but I take it that >> subtree has some problems...I did find an article about subtree [1] >> >> > >> > 5. lwip maintains code with a mailing list. So we don't need to push >> > subtree git somewhere to create a pull request. >> > >> > 6. I rechecked the latest edk2 and they use submodules now. (openssl, >> > libfdt, berkeley-softfloat-3 and others). >> > >> > 7. dynamic download also looks horrible for me. I.e. create subtree in >> > Makefile on compilation process. I think maintaining that will give you a >> > bunch of problems. I think we should not touch git structure after cloning. >> > >> > So what I can here suggest: >> > 1. lwip source code is 9.4M. If we compare all code it will be 564M in >> > total. So just having 1 commit witn copy of lwip library will work here. >> >> So we add a 9.4MB patch for the code we need? I suppose that is OK, >> although it is much larger than net/ today (0.5MB). >> >> What is the churn on lwip? E.g. would it be easy to add a commit every >> few months to bring in upstream changes? >> >> > >> > or >> > >> > 2. use git submodules. Size of the project will be lower. Submodule will >> > not allow you to use local changes. I.e. change needs to be merged into the >> > upstream project and then you can update git HEAD for the submodule. >> >> I really don't want to work with a submodule project. I've just had >> too many problems. >> >> > >> > or >> > >> > 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. Then >> > use git submoule to connect this branch as a folder to the main U-Boot code. >> >> It really needs to be properly part of U-Boot. >> > > Ok. Then more likely we don't need all the git history of lwip inside uboot.git. Then the option with a single commit is more preferable. > Then we can use part 2 of this article, how to go with standard git commands: > > 1. > <cmd> > git remote add -f lwip https://git.savannah.nongnu.org/git/lwip.git > git read-tree --prefix=net/lwip/lwip-external -u lwip/master > git commit -m "lwip merge sha: xxxx" > </cmd> > this will create a git format-patch friendly commit. Then we send it to the mailing list and apply. > I hope the mailing list will allow us to send a 7.8 MB patch. > > > Then if for development you need to pull he history of lwip, you can do it with: > git pull -s subtree lwip master --allow-unrelated-histories > (but I think nobody will need this.) > > New update of the lwip net/lwip/lwip-external dir will be done with: > git pull -s subtree lwip master --allow-unrelated-histories --squash > Squash commit also has to be git format-patch friendly. > > If you are ok with that proposal I will send v9 with the first patch created with steps above. We've gone through this before. The whole purpose of this is not having to maintain patches. Simon, instead of "I had problems in the past", can you elaborate a bit more? Tom said he is fine with subtrees instead of submodules and I know for a fact EDK2 doesn't have too many issues with submodules. Their documentation is pretty clear on building and requires git clone https://github.com/tianocore/edk2.git cd edk2 git submodule update --init Perhaps the situation has improved since you had issues? Thanks /Ilias > > Thanks, > Maxim. > > > > > > >> >> > >> > BR, >> > Maxim. >> >> Regards, >> Simon >> >> [1] https://www.atlassian.com/git/tutorials/git-subtree
On 13.09.2023 09:53, Ilias Apalodimas wrote: > Hi Maxim, > > On Wed, 13 Sept 2023 at 10:32, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: >> >> >> >> On Wed, 13 Sept 2023 at 01:27, Simon Glass <sjg@google.com> wrote: >>> >>> Hi Maxim, >>> >>> On Tue, 12 Sept 2023 at 05:42, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: >>>> >>>> On Fri, 8 Sept 2023 at 19:59, Tom Rini <trini@konsulko.com> wrote: >>>> >>>>> On Fri, Sep 08, 2023 at 07:53:05PM +0600, Maxim Uvarov wrote: >>>>> >>>>>> Before apply these patches it is needed to create lwIP merge into >>>>> U-Boot: >>>>>> git subtree add --prefix net/lwip/lwip-external >>>>> https://git.savannah.nongnu.org/git/lwip.git master --squash >>>>>> or >>>>>> create git submodule, depends how it's more easy to maintain external >>>>>> library. >>>>> >>>>> So, I think we're going to go with subtree. Please work out how to >>>>> integrate the above in to the build process automatically (and such that >>>>> we can maintain it via upgrades moving forward). >>>>> >>>>> -- >>>>> Tom >>>>> >>>> >>>> I did not find a good way to friend git format-patch, git am and subtree. >>>> And now with using subtree I can provide my thoughts, in general I do not >>>> see any big advantages >>>> with maintaining subtree code. >>>> >>>> Problem is that: >>>> >>>> 1. subtree does some root reset. So rebase looks like: >>>> label onto >>>> >>>> >>>> >>>> # Branch acbc0469a49de7055141cc730aa9c728e61b6de2-2 >>>> >>>> reset [new root] >>>> >>>> pick acbc0469a4 Squashed 'net/lwip/lwip-external/' content from commit >>>> 84fde1ebbf >>>> label acbc0469a49de7055141cc730aa9c728e61b6de2-2 >>>> >>>> >>>> >>>> reset onto >>>> >>>> merge -C ec4a128c8d acbc0469a49de7055141cc730aa9c728e61b6de2-2 # Merge >>>> commit 'acbc0469a49de7055141cc730aa9c728e61b6de2' as >>>> 'net/lwip/lwip-external' >>>> pick 739681a6f5 net/lwip: add doc/develop/net_lwip.rst >>>> >>>> pick f0ecab85e0 net/lwip: integrate lwIP library >>>> >>>> 2. if --rebase-merges option was not provided to rebase, then rebase will >>>> omit subtree directory and try to apply rebase patches to root directory. >>>> I.e. in current case squashed commit for lwip, will be applied to uboot >>>> root directory instead of ./net/lwip/lwip-external. >>>> >>>> 3. due to broken rebases without --rebase-merges more likely things like >>>> git bisect also will not work. >>>> >>>> 4. change in subtree code ./net/lwip/lwip-external/../something.c will >>>> create a git commit which looks like a standard U-Boot commit. I.e. path >>>> starts with ./net/lwip/lwip-external/ >>> >>> I don't really understand most of the above, but I take it that >>> subtree has some problems...I did find an article about subtree [1] >>> >>>> >>>> 5. lwip maintains code with a mailing list. So we don't need to push >>>> subtree git somewhere to create a pull request. >>>> >>>> 6. I rechecked the latest edk2 and they use submodules now. (openssl, >>>> libfdt, berkeley-softfloat-3 and others). >>>> >>>> 7. dynamic download also looks horrible for me. I.e. create subtree in >>>> Makefile on compilation process. I think maintaining that will give you a >>>> bunch of problems. I think we should not touch git structure after cloning. >>>> >>>> So what I can here suggest: >>>> 1. lwip source code is 9.4M. If we compare all code it will be 564M in >>>> total. So just having 1 commit witn copy of lwip library will work here. >>> >>> So we add a 9.4MB patch for the code we need? I suppose that is OK, >>> although it is much larger than net/ today (0.5MB). >>> >>> What is the churn on lwip? E.g. would it be easy to add a commit every >>> few months to bring in upstream changes? >>> >>>> >>>> or >>>> >>>> 2. use git submodules. Size of the project will be lower. Submodule will >>>> not allow you to use local changes. I.e. change needs to be merged into the >>>> upstream project and then you can update git HEAD for the submodule. >>> >>> I really don't want to work with a submodule project. I've just had >>> too many problems. >>> >>>> >>>> or >>>> >>>> 3. inside u-boot.git create branch lib-lwip and clone lwip repo there. Then >>>> use git submoule to connect this branch as a folder to the main U-Boot code. >>> >>> It really needs to be properly part of U-Boot. >>> >> >> Ok. Then more likely we don't need all the git history of lwip inside uboot.git. Then the option with a single commit is more preferable. >> Then we can use part 2 of this article, how to go with standard git commands: >> >> 1. >> <cmd> >> git remote add -f lwip https://git.savannah.nongnu.org/git/lwip.git >> git read-tree --prefix=net/lwip/lwip-external -u lwip/master >> git commit -m "lwip merge sha: xxxx" >> </cmd> >> this will create a git format-patch friendly commit. Then we send it to the mailing list and apply. >> I hope the mailing list will allow us to send a 7.8 MB patch. >> >> >> Then if for development you need to pull he history of lwip, you can do it with: >> git pull -s subtree lwip master --allow-unrelated-histories >> (but I think nobody will need this.) >> >> New update of the lwip net/lwip/lwip-external dir will be done with: >> git pull -s subtree lwip master --allow-unrelated-histories --squash >> Squash commit also has to be git format-patch friendly. >> >> If you are ok with that proposal I will send v9 with the first patch created with steps above. > > We've gone through this before. The whole purpose of this is not > having to maintain patches. > Simon, instead of "I had problems in the past", can you elaborate a bit more? > > Tom said he is fine with subtrees instead of submodules and I know for > a fact EDK2 doesn't have too many issues with submodules. > Their documentation is pretty clear on building and requires > > git clone https://github.com/tianocore/edk2.git > cd edk2 > git submodule update --init > > Perhaps the situation has improved since you had issues? While I don't really care how you solve this technically, I'd *strongly* be interested for U-Boot to use *unmodified* lwIP sources where an explicit reference to an lwIP commit is used. I'd rather integrate bugfixes for U-Boot into lwIP than having the sources drift apart. Regards, Simon > > Thanks > /Ilias >> >> Thanks, >> Maxim. >> >> >> >> >> >> >>> >>>> >>>> BR, >>>> Maxim. >>> >>> Regards, >>> Simon >>> >>> [1] https://www.atlassian.com/git/tutorials/git-subtree
> >> Then if for development you need to pull he history of lwip, you can do it with: > >> git pull -s subtree lwip master --allow-unrelated-histories > >> (but I think nobody will need this.) > >> > >> New update of the lwip net/lwip/lwip-external dir will be done with: > >> git pull -s subtree lwip master --allow-unrelated-histories --squash > >> Squash commit also has to be git format-patch friendly. > >> > >> If you are ok with that proposal I will send v9 with the first patch created with steps above. > > > > We've gone through this before. The whole purpose of this is not > > having to maintain patches. > > Simon, instead of "I had problems in the past", can you elaborate a bit more? > > > > Tom said he is fine with subtrees instead of submodules and I know for > > a fact EDK2 doesn't have too many issues with submodules. > > Their documentation is pretty clear on building and requires > > > > git clone https://github.com/tianocore/edk2.git > > cd edk2 > > git submodule update --init > > > > Perhaps the situation has improved since you had issues? > > While I don't really care how you solve this technically, I'd *strongly* > be interested for U-Boot to use *unmodified* lwIP sources where an > explicit reference to an lwIP commit is used. I'd rather integrate > bugfixes for U-Boot into lwIP than having the sources drift apart. Strongly agree here, we want to use upstream and all the combined development and reviews etc rather than forking off and ending up with yet another slightly different IP stack. The whole advantage of adopting LWIP is the advantage of combined security, features and bugs from a wide range of projects :-)
On Wed, Sep 13, 2023 at 11:06:13AM +0100, Peter Robinson wrote: > > >> Then if for development you need to pull he history of lwip, you can do it with: > > >> git pull -s subtree lwip master --allow-unrelated-histories > > >> (but I think nobody will need this.) > > >> > > >> New update of the lwip net/lwip/lwip-external dir will be done with: > > >> git pull -s subtree lwip master --allow-unrelated-histories --squash > > >> Squash commit also has to be git format-patch friendly. > > >> > > >> If you are ok with that proposal I will send v9 with the first patch created with steps above. > > > > > > We've gone through this before. The whole purpose of this is not > > > having to maintain patches. > > > Simon, instead of "I had problems in the past", can you elaborate a bit more? > > > > > > Tom said he is fine with subtrees instead of submodules and I know for > > > a fact EDK2 doesn't have too many issues with submodules. > > > Their documentation is pretty clear on building and requires > > > > > > git clone https://github.com/tianocore/edk2.git > > > cd edk2 > > > git submodule update --init > > > > > > Perhaps the situation has improved since you had issues? > > > > While I don't really care how you solve this technically, I'd *strongly* > > be interested for U-Boot to use *unmodified* lwIP sources where an > > explicit reference to an lwIP commit is used. I'd rather integrate > > bugfixes for U-Boot into lwIP than having the sources drift apart. > > Strongly agree here, we want to use upstream and all the combined > development and reviews etc rather than forking off and ending up with > yet another slightly different IP stack. The whole advantage of > adopting LWIP is the advantage of combined security, features and bugs > from a wide range of projects :-) Yes, this is what I want as well, and why I'm perhaps more agreeable with the approaches where it's a lot harder for us to start forking things unintentionally. I gather submodule rather than subtree would be better for that case?
On Wed, 13 Sept 2023 at 19:14, Tom Rini <trini@konsulko.com> wrote: > On Wed, Sep 13, 2023 at 11:06:13AM +0100, Peter Robinson wrote: > > > >> Then if for development you need to pull he history of lwip, you > can do it with: > > > >> git pull -s subtree lwip master --allow-unrelated-histories > > > >> (but I think nobody will need this.) > > > >> > > > >> New update of the lwip net/lwip/lwip-external dir will be done with: > > > >> git pull -s subtree lwip master --allow-unrelated-histories > --squash > > > >> Squash commit also has to be git format-patch friendly. > > > >> > > > >> If you are ok with that proposal I will send v9 with the first > patch created with steps above. > > > > > > > > We've gone through this before. The whole purpose of this is not > > > > having to maintain patches. > > > > Simon, instead of "I had problems in the past", can you elaborate a > bit more? > > > > > > > > Tom said he is fine with subtrees instead of submodules and I know > for > > > > a fact EDK2 doesn't have too many issues with submodules. > > > > Their documentation is pretty clear on building and requires > > > > > > > > git clone https://github.com/tianocore/edk2.git > > > > cd edk2 > > > > git submodule update --init > > > > > > > > Perhaps the situation has improved since you had issues? > > > > > > While I don't really care how you solve this technically, I'd > *strongly* > > > be interested for U-Boot to use *unmodified* lwIP sources where an > > > explicit reference to an lwIP commit is used. I'd rather integrate > > > bugfixes for U-Boot into lwIP than having the sources drift apart. > > > > Strongly agree here, we want to use upstream and all the combined > > development and reviews etc rather than forking off and ending up with > > yet another slightly different IP stack. The whole advantage of > > adopting LWIP is the advantage of combined security, features and bugs > > from a wide range of projects :-) > > Yes, this is what I want as well, and why I'm perhaps more agreeable > with the approaches where it's a lot harder for us to start forking > things unintentionally. I gather submodule rather than subtree would be > better for that case? > > -- > Tom > Yes, submodule will be a much better solution for us. And I also don't think that today there are any issues with submodules. It works well of OE, RPM and DEB builds, distributions should not have problems with it. BR, Maxim.
Hi, On Wed, 13 Sept 2023 at 07:35, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > > On Wed, 13 Sept 2023 at 19:14, Tom Rini <trini@konsulko.com> wrote: >> >> On Wed, Sep 13, 2023 at 11:06:13AM +0100, Peter Robinson wrote: >> > > >> Then if for development you need to pull he history of lwip, you can do it with: >> > > >> git pull -s subtree lwip master --allow-unrelated-histories >> > > >> (but I think nobody will need this.) >> > > >> >> > > >> New update of the lwip net/lwip/lwip-external dir will be done with: >> > > >> git pull -s subtree lwip master --allow-unrelated-histories --squash >> > > >> Squash commit also has to be git format-patch friendly. >> > > >> >> > > >> If you are ok with that proposal I will send v9 with the first patch created with steps above. >> > > > >> > > > We've gone through this before. The whole purpose of this is not >> > > > having to maintain patches. >> > > > Simon, instead of "I had problems in the past", can you elaborate a bit more? >> > > > >> > > > Tom said he is fine with subtrees instead of submodules and I know for >> > > > a fact EDK2 doesn't have too many issues with submodules. >> > > > Their documentation is pretty clear on building and requires >> > > > >> > > > git clone https://github.com/tianocore/edk2.git >> > > > cd edk2 >> > > > git submodule update --init >> > > > >> > > > Perhaps the situation has improved since you had issues? Nope. >> > > >> > > While I don't really care how you solve this technically, I'd *strongly* >> > > be interested for U-Boot to use *unmodified* lwIP sources where an >> > > explicit reference to an lwIP commit is used. I'd rather integrate >> > > bugfixes for U-Boot into lwIP than having the sources drift apart. >> > >> > Strongly agree here, we want to use upstream and all the combined >> > development and reviews etc rather than forking off and ending up with >> > yet another slightly different IP stack. The whole advantage of >> > adopting LWIP is the advantage of combined security, features and bugs >> > from a wide range of projects :-) >> >> Yes, this is what I want as well, and why I'm perhaps more agreeable >> with the approaches where it's a lot harder for us to start forking >> things unintentionally. I gather submodule rather than subtree would be >> better for that case? >> >> -- >> Tom > > > Yes, submodule will be a much better solution for us. And I also don't think that today > there are any issues with submodules. It works well of OE, RPM and DEB builds, > distributions should not have problems with it. My particular experience is with coreboot. Some problems I have: 1. Updating the modules doesn't work and I need to reset, try the --init thing, fetch things manually, etc. etc. 2. In ChromiumOS coreboot we can't use submodules internally since each package has its own build script. E.g. we need to build coreboot separately from its blobs, fsp, external libraries, etc. At least there we can do this, but if U-Boot adopts a submodule for a core feature, this is going to create no end of problems. 3. It makes it impossible to patch lwip for any fix we need for a release 4. We still have to 'fast forward' to a new commit every now and then, which really is no easier than doing a merge commit for the changes since the last sync, is it? Really, we need a maintainer for the lwip piece, if we are to adopt it. Using submodules is not a substitute for that. Regards, Simon
On 21.09.2023 18:29, Simon Glass wrote: > Hi, > > On Wed, 13 Sept 2023 at 07:35, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: >> >> >> >> On Wed, 13 Sept 2023 at 19:14, Tom Rini <trini@konsulko.com> wrote: >>> >>> On Wed, Sep 13, 2023 at 11:06:13AM +0100, Peter Robinson wrote: >>>>>>> Then if for development you need to pull he history of lwip, you can do it with: >>>>>>> git pull -s subtree lwip master --allow-unrelated-histories >>>>>>> (but I think nobody will need this.) >>>>>>> >>>>>>> New update of the lwip net/lwip/lwip-external dir will be done with: >>>>>>> git pull -s subtree lwip master --allow-unrelated-histories --squash >>>>>>> Squash commit also has to be git format-patch friendly. >>>>>>> >>>>>>> If you are ok with that proposal I will send v9 with the first patch created with steps above. >>>>>> >>>>>> We've gone through this before. The whole purpose of this is not >>>>>> having to maintain patches. >>>>>> Simon, instead of "I had problems in the past", can you elaborate a bit more? >>>>>> >>>>>> Tom said he is fine with subtrees instead of submodules and I know for >>>>>> a fact EDK2 doesn't have too many issues with submodules. >>>>>> Their documentation is pretty clear on building and requires >>>>>> >>>>>> git clone https://github.com/tianocore/edk2.git >>>>>> cd edk2 >>>>>> git submodule update --init >>>>>> >>>>>> Perhaps the situation has improved since you had issues? > > Nope. > >>>>> >>>>> While I don't really care how you solve this technically, I'd *strongly* >>>>> be interested for U-Boot to use *unmodified* lwIP sources where an >>>>> explicit reference to an lwIP commit is used. I'd rather integrate >>>>> bugfixes for U-Boot into lwIP than having the sources drift apart. >>>> >>>> Strongly agree here, we want to use upstream and all the combined >>>> development and reviews etc rather than forking off and ending up with >>>> yet another slightly different IP stack. The whole advantage of >>>> adopting LWIP is the advantage of combined security, features and bugs >>>> from a wide range of projects :-) >>> >>> Yes, this is what I want as well, and why I'm perhaps more agreeable >>> with the approaches where it's a lot harder for us to start forking >>> things unintentionally. I gather submodule rather than subtree would be >>> better for that case? >>> >>> -- >>> Tom >> >> >> Yes, submodule will be a much better solution for us. And I also don't think that today >> there are any issues with submodules. It works well of OE, RPM and DEB builds, >> distributions should not have problems with it. > > My particular experience is with coreboot. Some problems I have: > > 1. Updating the modules doesn't work and I need to reset, try the > --init thing, fetch things manually, etc. etc. > 2. In ChromiumOS coreboot we can't use submodules internally since > each package has its own build script. E.g. we need to build coreboot > separately from its blobs, fsp, external libraries, etc. At least > there we can do this, but if U-Boot adopts a submodule for a core > feature, this is going to create no end of problems. > 3. It makes it impossible to patch lwip for any fix we need for a release > 4. We still have to 'fast forward' to a new commit every now and then, > which really is no easier than doing a merge commit for the changes > since the last sync, is it? > > Really, we need a maintainer for the lwip piece, if we are to adopt > it. Using submodules is not a substitute for that. As an lwIP maintainer, I cannot step up as a maintainer of lwIP in U-Boot, however, I can assure you I will do my best to work with you on integrating fixes into upstream lwIP if required. Without wanting to promote using submodules: all other examples of lwIP being copied into another repository have practically never resulted in bugfixes being sent back to us (ok, that's not 100% true, but we do get them only once in a while) and being like that, those projects are facing problems upgrading our stack in turn. I wouldn't want to be a maintainer of such code, either. Regards, Simon
On Fri, Sep 22, 2023 at 12:56:58PM +0200, Simon Goldschmidt wrote: > > > On 21.09.2023 18:29, Simon Glass wrote: > > Hi, > > > > On Wed, 13 Sept 2023 at 07:35, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > > > > > > > > > > On Wed, 13 Sept 2023 at 19:14, Tom Rini <trini@konsulko.com> wrote: > > > > > > > > On Wed, Sep 13, 2023 at 11:06:13AM +0100, Peter Robinson wrote: > > > > > > > > Then if for development you need to pull he history of lwip, you can do it with: > > > > > > > > git pull -s subtree lwip master --allow-unrelated-histories > > > > > > > > (but I think nobody will need this.) > > > > > > > > > > > > > > > > New update of the lwip net/lwip/lwip-external dir will be done with: > > > > > > > > git pull -s subtree lwip master --allow-unrelated-histories --squash > > > > > > > > Squash commit also has to be git format-patch friendly. > > > > > > > > > > > > > > > > If you are ok with that proposal I will send v9 with the first patch created with steps above. > > > > > > > > > > > > > > We've gone through this before. The whole purpose of this is not > > > > > > > having to maintain patches. > > > > > > > Simon, instead of "I had problems in the past", can you elaborate a bit more? > > > > > > > > > > > > > > Tom said he is fine with subtrees instead of submodules and I know for > > > > > > > a fact EDK2 doesn't have too many issues with submodules. > > > > > > > Their documentation is pretty clear on building and requires > > > > > > > > > > > > > > git clone https://github.com/tianocore/edk2.git > > > > > > > cd edk2 > > > > > > > git submodule update --init > > > > > > > > > > > > > > Perhaps the situation has improved since you had issues? > > > > Nope. > > > > > > > > > > > > > > While I don't really care how you solve this technically, I'd *strongly* > > > > > > be interested for U-Boot to use *unmodified* lwIP sources where an > > > > > > explicit reference to an lwIP commit is used. I'd rather integrate > > > > > > bugfixes for U-Boot into lwIP than having the sources drift apart. > > > > > > > > > > Strongly agree here, we want to use upstream and all the combined > > > > > development and reviews etc rather than forking off and ending up with > > > > > yet another slightly different IP stack. The whole advantage of > > > > > adopting LWIP is the advantage of combined security, features and bugs > > > > > from a wide range of projects :-) > > > > > > > > Yes, this is what I want as well, and why I'm perhaps more agreeable > > > > with the approaches where it's a lot harder for us to start forking > > > > things unintentionally. I gather submodule rather than subtree would be > > > > better for that case? > > > > > > > > -- > > > > Tom > > > > > > > > > Yes, submodule will be a much better solution for us. And I also don't think that today > > > there are any issues with submodules. It works well of OE, RPM and DEB builds, > > > distributions should not have problems with it. > > > > My particular experience is with coreboot. Some problems I have: > > > > 1. Updating the modules doesn't work and I need to reset, try the > > --init thing, fetch things manually, etc. etc. > > 2. In ChromiumOS coreboot we can't use submodules internally since > > each package has its own build script. E.g. we need to build coreboot > > separately from its blobs, fsp, external libraries, etc. At least > > there we can do this, but if U-Boot adopts a submodule for a core > > feature, this is going to create no end of problems. > > 3. It makes it impossible to patch lwip for any fix we need for a release > > 4. We still have to 'fast forward' to a new commit every now and then, > > which really is no easier than doing a merge commit for the changes > > since the last sync, is it? > > > > Really, we need a maintainer for the lwip piece, if we are to adopt > > it. Using submodules is not a substitute for that. > > As an lwIP maintainer, I cannot step up as a maintainer of lwIP in > U-Boot, however, I can assure you I will do my best to work with you on > integrating fixes into upstream lwIP if required. > > Without wanting to promote using submodules: all other examples of lwIP > being copied into another repository have practically never resulted in > bugfixes being sent back to us (ok, that's not 100% true, but we do get > them only once in a while) and being like that, those projects are > facing problems upgrading our stack in turn. I wouldn't want to be a > maintainer of such code, either. So, ideally once we've integrated lwip, I'd be having us track the stable tags, rather than top of tree. Is this what you'd recommend, first of all? And second, assuming we can take STABLE_2_2_0 once it's released how often do you envision updates? I ask since maybe things do move slow enough that if I make some of my local every-time test scripts stop and confirm it's OK to have some changes to net/ (or where ever...) I can make sure we don't diverge and then like the calendar entries I do have to remind me to check with a resync to X I could do that for lwip.
Before apply these patches it is needed to create lwIP merge into U-Boot: git subtree add --prefix net/lwip/lwip-external https://git.savannah.nongnu.org/git/lwip.git master --squash or create git submodule, depends how it's more easy to maintain external library. changelog: v8: - comments for previous review - removed lwip timeout callback pointer - made lwip timeouts works, that also allowed to remove static vars. - setenv for filesize tftp and wget has to be in hex. - Makefile changes always compile it tftp,dns,wget,ping due to it can be used not only by CONFIG_CMD_. - Kconfig changes - simplify lwIP settings and support only one configuration. - tested with mini debian.iso load over http or tftp, mount and boot it (qemu, arm64). v7: - more review fixes. - support of multiply eth devices, were "ethact" selects the active device. v6: - fixed review comments for v5 (thanks Ilias and Simon). - lwip is not under /net, so prior applying patch following commit is needed to create lwIP merge into U-Boot: git subtree add --prefix net/lwip/lwip-external https://git.savannah.nongnu.org/git/lwip.git master --squash v5: - fixed Iliases comments and split big patch on the small ones. You also need to issue command: git subtree add --prefix lib/lwip/lwip-external https://git.savannah.nongnu.org/git/lwip.git master --squash Which will create merge commit of lwip library placing sources into lib/lwip/lwip-external directory. I do not send it a patch due to 1. merges are not friendly with git format-patch and 2. the source code of lwip is 78kb. v4: - tested with tests/py/ did some minor fixes (out of tree build, variables set after downloads). - accounted review comments for documentation. - implemented dns command - corrected wget command to not use serverip variable and use just url string. v3: - use lwip commands for ping,tftp,wget,dhcp if this patch applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option. - docs: use rst variant and drop references to RFC. Maxim Uvarov (15): net/lwip: add doc/develop/net_lwip.rst net/lwip: integrate lwIP library net/lwip: implement dns cmd net/lwip: implement dhcp cmd net/lwip: implement tftp cmd net/lwip: implement wget cmd net/lwip: implement ping cmd net/lwip: add lwIP configuration net/lwip: implement lwIP port to U-Boot net/lwip: update .gitignore with lwIP net/lwip: connection between cmd and lwip apps net/lwip: replace original net commands with lwip net/lwip: split net.h to net.h, arp.h and eth.h net/lwip: drop old net/wget net/lwip/wget add port selection boot/bootmeth_efi.c | 18 +- boot/bootmeth_pxe.c | 21 +- cmd/Makefile | 1 + cmd/net-lwip.c | 286 +++++++++++++++++ cmd/net.c | 86 +---- cmd/pxe.c | 19 +- doc/develop/index.rst | 1 + doc/develop/net_lwip.rst | 75 +++++ include/net.h | 197 +----------- include/net/arp.h | 7 + include/net/eth.h | 190 +++++++++++ include/net/lwip.h | 73 +++++ include/net/ulwip.h | 64 ++++ include/net/wget.h | 22 -- net/Kconfig | 3 + net/Makefile | 2 +- net/eth-uclass.c | 8 + net/lwip/.gitignore | 8 + net/lwip/Kconfig | 25 ++ net/lwip/Makefile | 70 ++++ net/lwip/apps/dhcp/lwip-dhcp.c | 61 ++++ net/lwip/apps/dns/lwip-dns.c | 63 ++++ net/lwip/apps/http/Makefile | 6 + net/lwip/apps/http/lwip-wget.c | 126 ++++++++ net/lwip/apps/ping/Makefile | 12 + net/lwip/apps/ping/lwip_ping.c | 40 +++ net/lwip/apps/ping/lwip_ping.h | 15 + net/lwip/apps/ping/ping.h | 19 ++ net/lwip/apps/tftp/Makefile | 7 + net/lwip/apps/tftp/lwip-tftp.c | 129 ++++++++ net/lwip/lwipopts.h | 178 +++++++++++ net/lwip/port/if.c | 332 +++++++++++++++++++ net/lwip/port/include/arch/cc.h | 38 +++ net/lwip/port/include/arch/sys_arch.h | 10 + net/lwip/port/include/limits.h | 0 net/lwip/port/sys-arch.c | 13 + net/net.c | 26 +- net/wget.c | 440 -------------------------- 38 files changed, 1933 insertions(+), 758 deletions(-) create mode 100644 cmd/net-lwip.c create mode 100644 doc/develop/net_lwip.rst create mode 100644 include/net/arp.h create mode 100644 include/net/eth.h create mode 100644 include/net/lwip.h create mode 100644 include/net/ulwip.h delete mode 100644 include/net/wget.h create mode 100644 net/lwip/.gitignore create mode 100644 net/lwip/Kconfig create mode 100644 net/lwip/Makefile create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c create mode 100644 net/lwip/apps/dns/lwip-dns.c create mode 100644 net/lwip/apps/http/Makefile create mode 100644 net/lwip/apps/http/lwip-wget.c create mode 100644 net/lwip/apps/ping/Makefile create mode 100644 net/lwip/apps/ping/lwip_ping.c create mode 100644 net/lwip/apps/ping/lwip_ping.h create mode 100644 net/lwip/apps/ping/ping.h create mode 100644 net/lwip/apps/tftp/Makefile create mode 100644 net/lwip/apps/tftp/lwip-tftp.c create mode 100644 net/lwip/lwipopts.h create mode 100644 net/lwip/port/if.c create mode 100644 net/lwip/port/include/arch/cc.h create mode 100644 net/lwip/port/include/arch/sys_arch.h create mode 100644 net/lwip/port/include/limits.h create mode 100644 net/lwip/port/sys-arch.c delete mode 100644 net/wget.c