mbox series

[PATHv11,00/43] net/lwip: add lwip library for the network stack

Message ID 20231127125726.3735-1-maxim.uvarov@linaro.org
Headers show
Series net/lwip: add lwip library for the network stack | expand

Message

Maxim Uvarov Nov. 27, 2023, 12:56 p.m. UTC
Hello,

Please find updated version of lwip patches. Changes are in the
changelog bellow.

Thank you,
Maxim. 

changelog:
	v11: - v11 is mosly respin of v10 patches with CI error fixes.
                Gitlab CI: 
                https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
                Azure CI:
                https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
                (Azure CI, which is connected to github. Sometime I can see
                 tftp timeout after some part of download there, but that can not be
                 reproduced locally. While Gitblab CI is stable. Because of num tries in
                 CI I suspect this CI was not always reliable.)
                Azure and Gitlab also have different toolchains and I 
                would say Gitlab generates bigger code then Azure CI.
                
                Also many boards have a binary limit size of 800k (even
                qemu has limits). And increased limits to fit all the code. Specially did it
                patch by board config to show which boards are failing to build. There I have
                a question if we really want to support new functionality for old boards (mips,
                arm32 and etc...). I hope board owners can help me if
                it's valid to increase these limits.

		In this version I used git submodules and friend CI with
		submodules. But I don't mind if you decide to maintain it in a different
		way.


	v10: - fix ping with following tftp command issue with incorrect
		ping timeout clear.
	     - Makefile on make will init submodules and if needed will
	       do git clone.
	     - wget - some minor code style changes.
	v9: - added first patch describing git submodule for lwip. So
	      the build procedure is:
		git submodule init
		git submodule update
		make
	    - reworked a little bit dhcp cmd state polling
	    - fixed review comments for v8
	v8: - comments for previous review
	    - removed lwip timeout callback pointer
	    - made lwip timeouts works, that also allowed to remove
	      static vars.
	    - setenv for filesize tftp and wget has to be in hex.
	    - Makefile changes always compile it tftp,dns,wget,ping due
	      to it can be used not only by CONFIG_CMD_.
	    - Kconfig changes - simplify lwIP settings and support only
	      one configuration.
	    - tested with mini debian.iso load over http or tftp, mount
	      and boot it (qemu, arm64).
	v7: - more review fixes.
	    - support of multiply eth devices, were "ethact" selects the
	      active device.
	v6: - fixed review comments for v5 (thanks Ilias and Simon).
	v5: - fixed Iliases comments and split big patch on the small
		ones.
	v4: - tested with tests/py/ did some minor fixes (out of tree
		build, variables set after downloads).
	    - accounted review comments for documentation.
	    - implemented dns command
            - corrected wget command to not use serverip variable and use just
		url string.
	v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
	      applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
	    - docs: use rst variant and drop references to RFC.

Maxim Uvarov (43):
  submodule: add lwIP as git submodule
  net/lwip: add doc/develop/net_lwip.rst
  net/lwip: integrate lwIP library
  net/lwip: implement dns cmd
  net/lwip: implement dhcp cmd
  net/lwip: implement tftp cmd
  net/lwip: implement wget cmd
  net/lwip: implement ping cmd
  net/lwip: add lwIP configuration
  net/lwip: implement lwIP port to U-Boot
  net/lwip: update .gitignore with lwIP
  net/lwip: connection between cmd and lwip apps
  net/lwip: replace original net commands with lwip
  net/lwip: split net.h to net.h, arp.h and eth.h
  test_efi_loader.py: use $filesize var
  test_net: print out net list
  net: sandbox: fix NULL pointer derefences
  net/smc911x: fix return from smc911x_send
  sandbox: eth-raw-os: successful return code is 0
  driver/net/rtl8139: remove debug print
  mach-socfpga: do not overlap defines with lwip
  bcm_ns3: fix overlap define with lwip
  rcar3_salvator-x_defconfig: increase binary size limit
  lwip: omap3: rename mem_init
  configs/turris_omnia_defconfig set limit to 0xf6000
  configs/tbs2910_defconfig inc limit
  configs/socfpga_secu1_defconfig: enable LTO
  configs/turris_omnia_defconfig: enable LTO
  configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
    size
  configs/sheevaplug_defconfig: enable LTO and inc size
  configs/lschlv2_defconfig: enable LTO and inc size
  configs/lsxhl_defconfig: LTO + size
  configs/am335x_evm_defconfig: inc SPL size
  configs/bk4r1_defconfig: inc size
  configs/linkit-smart-7688_defconfig: increse size
  configs/gardena-smart-gateway-mt7688_defconfig: increase size
  configs/rcar3_ulcb_defconfig: increase size
  configs/qemu-x86_64_defconfig: increase ROM size
  Makefile: add dtbs to clean
  .azure-pipelines: init submodules
  mach-mtmips: inc SPL size limit
  configs/linkit-smart-7688_defconfig: increase board limit
  .gitlab-ci.yml: change ownership of the git files

 .azure-pipelines.yml                          |  10 +-
 .gitlab-ci.yml                                |  15 +-
 .gitmodules                                   |   4 +
 Makefile                                      |  12 +-
 arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
 arch/arm/mach-omap2/omap3/board.c             |   2 +-
 arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
 arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
 .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
 arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
 arch/mips/mach-mtmips/Kconfig                 |   2 +-
 arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
 boot/bootmeth_efi.c                           |  18 +-
 boot/bootmeth_pxe.c                           |  21 +-
 cmd/Makefile                                  |   1 +
 cmd/net-lwip.c                                | 307 ++++++++++++++++
 cmd/net.c                                     |  86 +----
 cmd/pxe.c                                     |  19 +-
 configs/am335x_boneblack_vboot_defconfig      |   3 +
 configs/am335x_evm_defconfig                  |   1 +
 configs/bk4r1_defconfig                       |   2 +-
 .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
 configs/linkit-smart-7688_defconfig           |   2 +-
 configs/lschlv2_defconfig                     |   3 +-
 configs/lsxhl_defconfig                       |   3 +-
 configs/qemu-x86_64_defconfig                 |   5 +-
 configs/rcar3_salvator-x_defconfig            |   3 +-
 configs/rcar3_ulcb_defconfig                  |   1 +
 configs/sheevaplug_defconfig                  |   3 +-
 configs/socfpga_secu1_defconfig               |   1 +
 configs/tbs2910_defconfig                     |   2 +-
 configs/turris_omnia_defconfig                |   3 +-
 doc/develop/index.rst                         |   1 +
 doc/develop/net_lwip.rst                      |  75 ++++
 drivers/net/rtl8139.c                         |   1 -
 drivers/net/sandbox.c                         |   5 +
 drivers/net/smc911x.c                         |   2 +-
 include/configs/bcm_ns3.h                     |   6 +-
 include/net.h                                 | 197 +----------
 include/net/arp.h                             |   9 +
 include/net/eth.h                             | 194 +++++++++++
 include/net/lwip.h                            |  73 ++++
 include/net/ulwip.h                           |  67 ++++
 lib/Kconfig                                   |   2 +-
 net/Kconfig                                   |   3 +
 net/Makefile                                  |   1 +
 net/eth-uclass.c                              |  37 +-
 net/lwip/.gitignore                           |   8 +
 net/lwip/Kconfig                              |  34 ++
 net/lwip/Makefile                             |  70 ++++
 net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
 net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
 net/lwip/apps/http/Makefile                   |   6 +
 net/lwip/apps/http/lwip-wget.c                | 105 ++++++
 net/lwip/apps/ping/Makefile                   |  12 +
 net/lwip/apps/ping/lwip_ping.c                |  39 +++
 net/lwip/apps/ping/lwip_ping.h                |  15 +
 net/lwip/apps/ping/ping.h                     |  28 ++
 net/lwip/apps/tftp/Makefile                   |   7 +
 net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
 net/lwip/lwip-external                        |   1 +
 net/lwip/lwipopts.h                           | 178 ++++++++++
 net/lwip/port/if.c                            | 327 ++++++++++++++++++
 net/lwip/port/include/arch/cc.h               |  44 +++
 net/lwip/port/include/arch/sys_arch.h         |  10 +
 net/lwip/port/include/limits.h                |   0
 net/lwip/port/sys-arch.c                      |  13 +
 net/net.c                                     |  44 ++-
 test/py/tests/test_efi_loader.py              |   4 +-
 test/py/tests/test_net.py                     |   2 +
 70 files changed, 2113 insertions(+), 348 deletions(-)
 create mode 100644 .gitmodules
 create mode 100644 cmd/net-lwip.c
 create mode 100644 doc/develop/net_lwip.rst
 create mode 100644 include/net/arp.h
 create mode 100644 include/net/eth.h
 create mode 100644 include/net/lwip.h
 create mode 100644 include/net/ulwip.h
 create mode 100644 net/lwip/.gitignore
 create mode 100644 net/lwip/Kconfig
 create mode 100644 net/lwip/Makefile
 create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
 create mode 100644 net/lwip/apps/dns/lwip-dns.c
 create mode 100644 net/lwip/apps/http/Makefile
 create mode 100644 net/lwip/apps/http/lwip-wget.c
 create mode 100644 net/lwip/apps/ping/Makefile
 create mode 100644 net/lwip/apps/ping/lwip_ping.c
 create mode 100644 net/lwip/apps/ping/lwip_ping.h
 create mode 100644 net/lwip/apps/ping/ping.h
 create mode 100644 net/lwip/apps/tftp/Makefile
 create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
 create mode 160000 net/lwip/lwip-external
 create mode 100644 net/lwip/lwipopts.h
 create mode 100644 net/lwip/port/if.c
 create mode 100644 net/lwip/port/include/arch/cc.h
 create mode 100644 net/lwip/port/include/arch/sys_arch.h
 create mode 100644 net/lwip/port/include/limits.h
 create mode 100644 net/lwip/port/sys-arch.c

Comments

Tom Rini Nov. 27, 2023, 6:37 p.m. UTC | #1
On Mon, Nov 27, 2023 at 06:56:43PM +0600, Maxim Uvarov wrote:

> Hello,
> 
> Please find updated version of lwip patches. Changes are in the
> changelog bellow.
> 
> Thank you,
> Maxim. 
> 
> changelog:
> 	v11: - v11 is mosly respin of v10 patches with CI error fixes.
>                 Gitlab CI: 
>                 https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
>                 Azure CI:
>                 https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
>                 (Azure CI, which is connected to github. Sometime I can see
>                  tftp timeout after some part of download there, but that can not be
>                  reproduced locally. While Gitblab CI is stable. Because of num tries in
>                  CI I suspect this CI was not always reliable.)
>                 Azure and Gitlab also have different toolchains and I 
>                 would say Gitlab generates bigger code then Azure CI.
>                 
>                 Also many boards have a binary limit size of 800k (even
>                 qemu has limits). And increased limits to fit all the code. Specially did it
>                 patch by board config to show which boards are failing to build. There I have
>                 a question if we really want to support new functionality for old boards (mips,
>                 arm32 and etc...). I hope board owners can help me if
>                 it's valid to increase these limits.
> 
> 		In this version I used git submodules and friend CI with
> 		submodules. But I don't mind if you decide to maintain it in a different
> 		way.

I just want to note that generally this is a good improvement on v10 and
I'll give things a spin in my lab (am335x_evm excluded ;)).
Tom Rini Nov. 27, 2023, 6:59 p.m. UTC | #2
On Mon, Nov 27, 2023 at 06:56:43PM +0600, Maxim Uvarov wrote:

> Hello,
> 
> Please find updated version of lwip patches. Changes are in the
> changelog bellow.
> 
> Thank you,
> Maxim. 
> 
> changelog:
> 	v11: - v11 is mosly respin of v10 patches with CI error fixes.

On a RPi 3:
U-Boot> dhcp
Waiting for Ethernet connection... done.
eth0: smsc95xx_eth b8:27:eb:fc:64:a6 active
dhcp_tmo 20/20
...
dhcp_tmo 0/20
DHCP client timeout
U-Boot>

And aside that those prints should be debug(), any ideas? This is
rpi_arm64_defconfig.
neil.armstrong@linaro.org Nov. 28, 2023, 10:37 a.m. UTC | #3
Hi,

On 27/11/2023 13:56, Maxim Uvarov wrote:
> Hello,
> 
> Please find updated version of lwip patches. Changes are in the
> changelog bellow.

I've ran it on the libretech-cc board, and tried to load grub over tftp,
and I got this strange EFI boot error:
========================================================================================
U-Boot 2024.01-rc3-00056-g10d85cb3e3 (Nov 28 2023 - 11:17:24 +0100) libretech-cc

Model: Libre Computer AML-S905X-CC
SoC:   Amlogic Meson GXL (S905X) Revision 21:d (84:2)
DRAM:  2 GiB

<snip>

Net:   eth0: ethernet@c9410000
Hit any key to stop autoboot:  0
=> setenv autoload no
=> dhcp
ethernet@c9410000 LPA corruption - aneg restart
ethernet@c9410000 Waiting for PHY auto negotiation to complete.... done
Speed: 100, full duplex
eth0: ethernet@c9410000 3e:a6:23:c0:39:4b active
dhcp_tmo 20/20
dhcp_tmo 19/20
DHCP client bound to address 10.34.56.105
=> tftpboot 8080000 grubaa64.efi
init already done for ethernet@c9410000
Speed: 100, full duplex
TFTP from server 10.34.56.1; our IP address is 10.34.56.105
Filename 'grubaa64.efi'.
Load address: 0x8080000
Loading:############
done
Bytes transferred = 4288512 (0x417000 hex)
=> crc32 8080000 0x417000
crc32 for 08080000 ... 08496fff ==> c79bc066
========================================================================================

- DHCP OK,
- transfer OK
- CRC32 value OK

but then trying to run the EFI binary:
========================================================================================
=> bootefi 8080000
No EFI system partition
No EFI system partition
Failed to persist EFI variables
No UEFI binary known at 8080000
========================================================================================

This is what I get on the current master without this patchset:
========================================================================================
U-Boot 2024.01-rc3-00013-gacae7eb5fe (Nov 28 2023 - 11:29:38 +0100) libretech-cc

Model: Libre Computer AML-S905X-CC
SoC:   Amlogic Meson GXL (S905X) Revision 21:d (84:2)
DRAM:  2 GiB

<snip>

Net:   eth0: ethernet@c9410000
Hit any key to stop autoboot:  0
=> setenv autoload no
=> dhcp
ethernet@c9410000 LPA corruption - aneg restart
ethernet@c9410000 Waiting for PHY auto negotiation to complete.... done
Speed: 100, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
DHCP client bound to address 10.34.56.105 (1008 ms)
=> tftpboot 8080000 grubaa64.efi
Speed: 100, full duplex
Using ethernet@c9410000 device
TFTP from server 10.34.56.1; our IP address is 10.34.56.105
Filename 'grubaa64.efi'.
Load address: 0x8080000
Loading: ##T #T T ##############################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 #################################################################
	 ##########################################################
	 199.2 KiB/s
done
Bytes transferred = 4288512 (417000 hex)
=> crc32 8080000 0x417000
crc32 for 08080000 ... 08496fff ==> c79bc066
=> bootefi 8080000
No EFI system partition
No EFI system partition
Failed to persist EFI variables
Booting /grubaa64.efi
Welcome to GRUB!

<snip>

grub> net_ls_addr
efinet0 3e:a6:23:c0:39:4b 10.34.56.105
========================================================================================

I don't see what's wrong, crc32 is good and env variables are the same:
fileaddr=8080000
filesize=417000

Neil


> 
> Thank you,
> Maxim.
> 
> changelog:
> 	v11: - v11 is mosly respin of v10 patches with CI error fixes.
>                  Gitlab CI:
>                  https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
>                  Azure CI:
>                  https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
>                  (Azure CI, which is connected to github. Sometime I can see
>                   tftp timeout after some part of download there, but that can not be
>                   reproduced locally. While Gitblab CI is stable. Because of num tries in
>                   CI I suspect this CI was not always reliable.)
>                  Azure and Gitlab also have different toolchains and I
>                  would say Gitlab generates bigger code then Azure CI.
>                  
>                  Also many boards have a binary limit size of 800k (even
>                  qemu has limits). And increased limits to fit all the code. Specially did it
>                  patch by board config to show which boards are failing to build. There I have
>                  a question if we really want to support new functionality for old boards (mips,
>                  arm32 and etc...). I hope board owners can help me if
>                  it's valid to increase these limits.
> 
> 		In this version I used git submodules and friend CI with
> 		submodules. But I don't mind if you decide to maintain it in a different
> 		way.
> 
> 
> 	v10: - fix ping with following tftp command issue with incorrect
> 		ping timeout clear.
> 	     - Makefile on make will init submodules and if needed will
> 	       do git clone.
> 	     - wget - some minor code style changes.
> 	v9: - added first patch describing git submodule for lwip. So
> 	      the build procedure is:
> 		git submodule init
> 		git submodule update
> 		make
> 	    - reworked a little bit dhcp cmd state polling
> 	    - fixed review comments for v8
> 	v8: - comments for previous review
> 	    - removed lwip timeout callback pointer
> 	    - made lwip timeouts works, that also allowed to remove
> 	      static vars.
> 	    - setenv for filesize tftp and wget has to be in hex.
> 	    - Makefile changes always compile it tftp,dns,wget,ping due
> 	      to it can be used not only by CONFIG_CMD_.
> 	    - Kconfig changes - simplify lwIP settings and support only
> 	      one configuration.
> 	    - tested with mini debian.iso load over http or tftp, mount
> 	      and boot it (qemu, arm64).
> 	v7: - more review fixes.
> 	    - support of multiply eth devices, were "ethact" selects the
> 	      active device.
> 	v6: - fixed review comments for v5 (thanks Ilias and Simon).
> 	v5: - fixed Iliases comments and split big patch on the small
> 		ones.
> 	v4: - tested with tests/py/ did some minor fixes (out of tree
> 		build, variables set after downloads).
> 	    - accounted review comments for documentation.
> 	    - implemented dns command
>              - corrected wget command to not use serverip variable and use just
> 		url string.
> 	v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
> 	      applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
> 	    - docs: use rst variant and drop references to RFC.
> 
> Maxim Uvarov (43):
>    submodule: add lwIP as git submodule
>    net/lwip: add doc/develop/net_lwip.rst
>    net/lwip: integrate lwIP library
>    net/lwip: implement dns cmd
>    net/lwip: implement dhcp cmd
>    net/lwip: implement tftp cmd
>    net/lwip: implement wget cmd
>    net/lwip: implement ping cmd
>    net/lwip: add lwIP configuration
>    net/lwip: implement lwIP port to U-Boot
>    net/lwip: update .gitignore with lwIP
>    net/lwip: connection between cmd and lwip apps
>    net/lwip: replace original net commands with lwip
>    net/lwip: split net.h to net.h, arp.h and eth.h
>    test_efi_loader.py: use $filesize var
>    test_net: print out net list
>    net: sandbox: fix NULL pointer derefences
>    net/smc911x: fix return from smc911x_send
>    sandbox: eth-raw-os: successful return code is 0
>    driver/net/rtl8139: remove debug print
>    mach-socfpga: do not overlap defines with lwip
>    bcm_ns3: fix overlap define with lwip
>    rcar3_salvator-x_defconfig: increase binary size limit
>    lwip: omap3: rename mem_init
>    configs/turris_omnia_defconfig set limit to 0xf6000
>    configs/tbs2910_defconfig inc limit
>    configs/socfpga_secu1_defconfig: enable LTO
>    configs/turris_omnia_defconfig: enable LTO
>    configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
>      size
>    configs/sheevaplug_defconfig: enable LTO and inc size
>    configs/lschlv2_defconfig: enable LTO and inc size
>    configs/lsxhl_defconfig: LTO + size
>    configs/am335x_evm_defconfig: inc SPL size
>    configs/bk4r1_defconfig: inc size
>    configs/linkit-smart-7688_defconfig: increse size
>    configs/gardena-smart-gateway-mt7688_defconfig: increase size
>    configs/rcar3_ulcb_defconfig: increase size
>    configs/qemu-x86_64_defconfig: increase ROM size
>    Makefile: add dtbs to clean
>    .azure-pipelines: init submodules
>    mach-mtmips: inc SPL size limit
>    configs/linkit-smart-7688_defconfig: increase board limit
>    .gitlab-ci.yml: change ownership of the git files
> 
>   .azure-pipelines.yml                          |  10 +-
>   .gitlab-ci.yml                                |  15 +-
>   .gitmodules                                   |   4 +
>   Makefile                                      |  12 +-
>   arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
>   arch/arm/mach-omap2/omap3/board.c             |   2 +-
>   arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
>   arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
>   .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
>   arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
>   arch/mips/mach-mtmips/Kconfig                 |   2 +-
>   arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
>   boot/bootmeth_efi.c                           |  18 +-
>   boot/bootmeth_pxe.c                           |  21 +-
>   cmd/Makefile                                  |   1 +
>   cmd/net-lwip.c                                | 307 ++++++++++++++++
>   cmd/net.c                                     |  86 +----
>   cmd/pxe.c                                     |  19 +-
>   configs/am335x_boneblack_vboot_defconfig      |   3 +
>   configs/am335x_evm_defconfig                  |   1 +
>   configs/bk4r1_defconfig                       |   2 +-
>   .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
>   configs/linkit-smart-7688_defconfig           |   2 +-
>   configs/lschlv2_defconfig                     |   3 +-
>   configs/lsxhl_defconfig                       |   3 +-
>   configs/qemu-x86_64_defconfig                 |   5 +-
>   configs/rcar3_salvator-x_defconfig            |   3 +-
>   configs/rcar3_ulcb_defconfig                  |   1 +
>   configs/sheevaplug_defconfig                  |   3 +-
>   configs/socfpga_secu1_defconfig               |   1 +
>   configs/tbs2910_defconfig                     |   2 +-
>   configs/turris_omnia_defconfig                |   3 +-
>   doc/develop/index.rst                         |   1 +
>   doc/develop/net_lwip.rst                      |  75 ++++
>   drivers/net/rtl8139.c                         |   1 -
>   drivers/net/sandbox.c                         |   5 +
>   drivers/net/smc911x.c                         |   2 +-
>   include/configs/bcm_ns3.h                     |   6 +-
>   include/net.h                                 | 197 +----------
>   include/net/arp.h                             |   9 +
>   include/net/eth.h                             | 194 +++++++++++
>   include/net/lwip.h                            |  73 ++++
>   include/net/ulwip.h                           |  67 ++++
>   lib/Kconfig                                   |   2 +-
>   net/Kconfig                                   |   3 +
>   net/Makefile                                  |   1 +
>   net/eth-uclass.c                              |  37 +-
>   net/lwip/.gitignore                           |   8 +
>   net/lwip/Kconfig                              |  34 ++
>   net/lwip/Makefile                             |  70 ++++
>   net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
>   net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
>   net/lwip/apps/http/Makefile                   |   6 +
>   net/lwip/apps/http/lwip-wget.c                | 105 ++++++
>   net/lwip/apps/ping/Makefile                   |  12 +
>   net/lwip/apps/ping/lwip_ping.c                |  39 +++
>   net/lwip/apps/ping/lwip_ping.h                |  15 +
>   net/lwip/apps/ping/ping.h                     |  28 ++
>   net/lwip/apps/tftp/Makefile                   |   7 +
>   net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
>   net/lwip/lwip-external                        |   1 +
>   net/lwip/lwipopts.h                           | 178 ++++++++++
>   net/lwip/port/if.c                            | 327 ++++++++++++++++++
>   net/lwip/port/include/arch/cc.h               |  44 +++
>   net/lwip/port/include/arch/sys_arch.h         |  10 +
>   net/lwip/port/include/limits.h                |   0
>   net/lwip/port/sys-arch.c                      |  13 +
>   net/net.c                                     |  44 ++-
>   test/py/tests/test_efi_loader.py              |   4 +-
>   test/py/tests/test_net.py                     |   2 +
>   70 files changed, 2113 insertions(+), 348 deletions(-)
>   create mode 100644 .gitmodules
>   create mode 100644 cmd/net-lwip.c
>   create mode 100644 doc/develop/net_lwip.rst
>   create mode 100644 include/net/arp.h
>   create mode 100644 include/net/eth.h
>   create mode 100644 include/net/lwip.h
>   create mode 100644 include/net/ulwip.h
>   create mode 100644 net/lwip/.gitignore
>   create mode 100644 net/lwip/Kconfig
>   create mode 100644 net/lwip/Makefile
>   create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
>   create mode 100644 net/lwip/apps/dns/lwip-dns.c
>   create mode 100644 net/lwip/apps/http/Makefile
>   create mode 100644 net/lwip/apps/http/lwip-wget.c
>   create mode 100644 net/lwip/apps/ping/Makefile
>   create mode 100644 net/lwip/apps/ping/lwip_ping.c
>   create mode 100644 net/lwip/apps/ping/lwip_ping.h
>   create mode 100644 net/lwip/apps/ping/ping.h
>   create mode 100644 net/lwip/apps/tftp/Makefile
>   create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
>   create mode 160000 net/lwip/lwip-external
>   create mode 100644 net/lwip/lwipopts.h
>   create mode 100644 net/lwip/port/if.c
>   create mode 100644 net/lwip/port/include/arch/cc.h
>   create mode 100644 net/lwip/port/include/arch/sys_arch.h
>   create mode 100644 net/lwip/port/include/limits.h
>   create mode 100644 net/lwip/port/sys-arch.c
>
Maxim Uvarov Nov. 28, 2023, 7:13 p.m. UTC | #4
On Tue, 28 Nov 2023 at 16:37, <neil.armstrong@linaro.org> wrote:

> Hi,
>
> On 27/11/2023 13:56, Maxim Uvarov wrote:
> > Hello,
> >
> > Please find updated version of lwip patches. Changes are in the
> > changelog bellow.
>
> I've ran it on the libretech-cc board, and tried to load grub over tftp,
> and I got this strange EFI boot error:
>
> ========================================================================================
> U-Boot 2024.01-rc3-00056-g10d85cb3e3 (Nov 28 2023 - 11:17:24 +0100)
> libretech-cc
>
> Model: Libre Computer AML-S905X-CC
> SoC:   Amlogic Meson GXL (S905X) Revision 21:d (84:2)
> DRAM:  2 GiB
>
> <snip>
>
> Net:   eth0: ethernet@c9410000
> Hit any key to stop autoboot:  0
> => setenv autoload no
> => dhcp
> ethernet@c9410000 LPA corruption - aneg restart
> ethernet@c9410000 Waiting for PHY auto negotiation to complete.... done
> Speed: 100, full duplex
> eth0: ethernet@c9410000 3e:a6:23:c0:39:4b active
> dhcp_tmo 20/20
> dhcp_tmo 19/20
> DHCP client bound to address 10.34.56.105
> => tftpboot 8080000 grubaa64.efi
> init already done for ethernet@c9410000
> Speed: 100, full duplex
> TFTP from server 10.34.56.1; our IP address is 10.34.56.105
> Filename 'grubaa64.efi'.
> Load address: 0x8080000
> Loading:############
> done
> Bytes transferred = 4288512 (0x417000 hex)
> => crc32 8080000 0x417000
> crc32 for 08080000 ... 08496fff ==> c79bc066
>
> ========================================================================================
>
> - DHCP OK,
> - transfer OK
> - CRC32 value OK
>
> but then trying to run the EFI binary:
>
> ========================================================================================
> => bootefi 8080000
> No EFI system partition
> No EFI system partition
> Failed to persist EFI variables
> No UEFI binary known at 8080000
>
> ========================================================================================
>
> This is what I get on the current master without this patchset:
>
> ========================================================================================
> U-Boot 2024.01-rc3-00013-gacae7eb5fe (Nov 28 2023 - 11:29:38 +0100)
> libretech-cc
>
> Model: Libre Computer AML-S905X-CC
> SoC:   Amlogic Meson GXL (S905X) Revision 21:d (84:2)
> DRAM:  2 GiB
>
> <snip>
>
> Net:   eth0: ethernet@c9410000
> Hit any key to stop autoboot:  0
> => setenv autoload no
> => dhcp
> ethernet@c9410000 LPA corruption - aneg restart
> ethernet@c9410000 Waiting for PHY auto negotiation to complete.... done
> Speed: 100, full duplex
> BOOTP broadcast 1
> BOOTP broadcast 2
> BOOTP broadcast 3
> DHCP client bound to address 10.34.56.105 (1008 ms)
> => tftpboot 8080000 grubaa64.efi
> Speed: 100, full duplex
> Using ethernet@c9410000 device
> TFTP from server 10.34.56.1; our IP address is 10.34.56.105
> Filename 'grubaa64.efi'.
> Load address: 0x8080000
> Loading: ##T #T T
> ##############################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          #################################################################
>          ##########################################################
>          199.2 KiB/s
> done
> Bytes transferred = 4288512 (417000 hex)
> => crc32 8080000 0x417000
> crc32 for 08080000 ... 08496fff ==> c79bc066
> => bootefi 8080000
> No EFI system partition
> No EFI system partition
> Failed to persist EFI variables
> Booting /grubaa64.efi
> Welcome to GRUB!
>
> <snip>
>
> grub> net_ls_addr
> efinet0 3e:a6:23:c0:39:4b 10.34.56.105
>
> ========================================================================================
>
> I don't see what's wrong, crc32 is good and env variables are the same:
> fileaddr=8080000
> filesize=417000
>
> Neil
>
>
Ok. You provided a small fix for this use case. Will take it into a new
version.



>
> >
> > Thank you,
> > Maxim.
> >
> > changelog:
> >       v11: - v11 is mosly respin of v10 patches with CI error fixes.
> >                  Gitlab CI:
> >
> https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
> >                  Azure CI:
> >
> https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
> >                  (Azure CI, which is connected to github. Sometime I can
> see
> >                   tftp timeout after some part of download there, but
> that can not be
> >                   reproduced locally. While Gitblab CI is stable.
> Because of num tries in
> >                   CI I suspect this CI was not always reliable.)
> >                  Azure and Gitlab also have different toolchains and I
> >                  would say Gitlab generates bigger code then Azure CI.
> >
> >                  Also many boards have a binary limit size of 800k (even
> >                  qemu has limits). And increased limits to fit all the
> code. Specially did it
> >                  patch by board config to show which boards are failing
> to build. There I have
> >                  a question if we really want to support new
> functionality for old boards (mips,
> >                  arm32 and etc...). I hope board owners can help me if
> >                  it's valid to increase these limits.
> >
> >               In this version I used git submodules and friend CI with
> >               submodules. But I don't mind if you decide to maintain it
> in a different
> >               way.
> >
> >
> >       v10: - fix ping with following tftp command issue with incorrect
> >               ping timeout clear.
> >            - Makefile on make will init submodules and if needed will
> >              do git clone.
> >            - wget - some minor code style changes.
> >       v9: - added first patch describing git submodule for lwip. So
> >             the build procedure is:
> >               git submodule init
> >               git submodule update
> >               make
> >           - reworked a little bit dhcp cmd state polling
> >           - fixed review comments for v8
> >       v8: - comments for previous review
> >           - removed lwip timeout callback pointer
> >           - made lwip timeouts works, that also allowed to remove
> >             static vars.
> >           - setenv for filesize tftp and wget has to be in hex.
> >           - Makefile changes always compile it tftp,dns,wget,ping due
> >             to it can be used not only by CONFIG_CMD_.
> >           - Kconfig changes - simplify lwIP settings and support only
> >             one configuration.
> >           - tested with mini debian.iso load over http or tftp, mount
> >             and boot it (qemu, arm64).
> >       v7: - more review fixes.
> >           - support of multiply eth devices, were "ethact" selects the
> >             active device.
> >       v6: - fixed review comments for v5 (thanks Ilias and Simon).
> >       v5: - fixed Iliases comments and split big patch on the small
> >               ones.
> >       v4: - tested with tests/py/ did some minor fixes (out of tree
> >               build, variables set after downloads).
> >           - accounted review comments for documentation.
> >           - implemented dns command
> >              - corrected wget command to not use serverip variable and
> use just
> >               url string.
> >       v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
> >             applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
> >           - docs: use rst variant and drop references to RFC.
> >
> > Maxim Uvarov (43):
> >    submodule: add lwIP as git submodule
> >    net/lwip: add doc/develop/net_lwip.rst
> >    net/lwip: integrate lwIP library
> >    net/lwip: implement dns cmd
> >    net/lwip: implement dhcp cmd
> >    net/lwip: implement tftp cmd
> >    net/lwip: implement wget cmd
> >    net/lwip: implement ping cmd
> >    net/lwip: add lwIP configuration
> >    net/lwip: implement lwIP port to U-Boot
> >    net/lwip: update .gitignore with lwIP
> >    net/lwip: connection between cmd and lwip apps
> >    net/lwip: replace original net commands with lwip
> >    net/lwip: split net.h to net.h, arp.h and eth.h
> >    test_efi_loader.py: use $filesize var
> >    test_net: print out net list
> >    net: sandbox: fix NULL pointer derefences
> >    net/smc911x: fix return from smc911x_send
> >    sandbox: eth-raw-os: successful return code is 0
> >    driver/net/rtl8139: remove debug print
> >    mach-socfpga: do not overlap defines with lwip
> >    bcm_ns3: fix overlap define with lwip
> >    rcar3_salvator-x_defconfig: increase binary size limit
> >    lwip: omap3: rename mem_init
> >    configs/turris_omnia_defconfig set limit to 0xf6000
> >    configs/tbs2910_defconfig inc limit
> >    configs/socfpga_secu1_defconfig: enable LTO
> >    configs/turris_omnia_defconfig: enable LTO
> >    configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
> >      size
> >    configs/sheevaplug_defconfig: enable LTO and inc size
> >    configs/lschlv2_defconfig: enable LTO and inc size
> >    configs/lsxhl_defconfig: LTO + size
> >    configs/am335x_evm_defconfig: inc SPL size
> >    configs/bk4r1_defconfig: inc size
> >    configs/linkit-smart-7688_defconfig: increse size
> >    configs/gardena-smart-gateway-mt7688_defconfig: increase size
> >    configs/rcar3_ulcb_defconfig: increase size
> >    configs/qemu-x86_64_defconfig: increase ROM size
> >    Makefile: add dtbs to clean
> >    .azure-pipelines: init submodules
> >    mach-mtmips: inc SPL size limit
> >    configs/linkit-smart-7688_defconfig: increase board limit
> >    .gitlab-ci.yml: change ownership of the git files
> >
> >   .azure-pipelines.yml                          |  10 +-
> >   .gitlab-ci.yml                                |  15 +-
> >   .gitmodules                                   |   4 +
> >   Makefile                                      |  12 +-
> >   arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
> >   arch/arm/mach-omap2/omap3/board.c             |   2 +-
> >   arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
> >   arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
> >   .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
> >   arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
> >   arch/mips/mach-mtmips/Kconfig                 |   2 +-
> >   arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
> >   boot/bootmeth_efi.c                           |  18 +-
> >   boot/bootmeth_pxe.c                           |  21 +-
> >   cmd/Makefile                                  |   1 +
> >   cmd/net-lwip.c                                | 307 ++++++++++++++++
> >   cmd/net.c                                     |  86 +----
> >   cmd/pxe.c                                     |  19 +-
> >   configs/am335x_boneblack_vboot_defconfig      |   3 +
> >   configs/am335x_evm_defconfig                  |   1 +
> >   configs/bk4r1_defconfig                       |   2 +-
> >   .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
> >   configs/linkit-smart-7688_defconfig           |   2 +-
> >   configs/lschlv2_defconfig                     |   3 +-
> >   configs/lsxhl_defconfig                       |   3 +-
> >   configs/qemu-x86_64_defconfig                 |   5 +-
> >   configs/rcar3_salvator-x_defconfig            |   3 +-
> >   configs/rcar3_ulcb_defconfig                  |   1 +
> >   configs/sheevaplug_defconfig                  |   3 +-
> >   configs/socfpga_secu1_defconfig               |   1 +
> >   configs/tbs2910_defconfig                     |   2 +-
> >   configs/turris_omnia_defconfig                |   3 +-
> >   doc/develop/index.rst                         |   1 +
> >   doc/develop/net_lwip.rst                      |  75 ++++
> >   drivers/net/rtl8139.c                         |   1 -
> >   drivers/net/sandbox.c                         |   5 +
> >   drivers/net/smc911x.c                         |   2 +-
> >   include/configs/bcm_ns3.h                     |   6 +-
> >   include/net.h                                 | 197 +----------
> >   include/net/arp.h                             |   9 +
> >   include/net/eth.h                             | 194 +++++++++++
> >   include/net/lwip.h                            |  73 ++++
> >   include/net/ulwip.h                           |  67 ++++
> >   lib/Kconfig                                   |   2 +-
> >   net/Kconfig                                   |   3 +
> >   net/Makefile                                  |   1 +
> >   net/eth-uclass.c                              |  37 +-
> >   net/lwip/.gitignore                           |   8 +
> >   net/lwip/Kconfig                              |  34 ++
> >   net/lwip/Makefile                             |  70 ++++
> >   net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
> >   net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
> >   net/lwip/apps/http/Makefile                   |   6 +
> >   net/lwip/apps/http/lwip-wget.c                | 105 ++++++
> >   net/lwip/apps/ping/Makefile                   |  12 +
> >   net/lwip/apps/ping/lwip_ping.c                |  39 +++
> >   net/lwip/apps/ping/lwip_ping.h                |  15 +
> >   net/lwip/apps/ping/ping.h                     |  28 ++
> >   net/lwip/apps/tftp/Makefile                   |   7 +
> >   net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
> >   net/lwip/lwip-external                        |   1 +
> >   net/lwip/lwipopts.h                           | 178 ++++++++++
> >   net/lwip/port/if.c                            | 327 ++++++++++++++++++
> >   net/lwip/port/include/arch/cc.h               |  44 +++
> >   net/lwip/port/include/arch/sys_arch.h         |  10 +
> >   net/lwip/port/include/limits.h                |   0
> >   net/lwip/port/sys-arch.c                      |  13 +
> >   net/net.c                                     |  44 ++-
> >   test/py/tests/test_efi_loader.py              |   4 +-
> >   test/py/tests/test_net.py                     |   2 +
> >   70 files changed, 2113 insertions(+), 348 deletions(-)
> >   create mode 100644 .gitmodules
> >   create mode 100644 cmd/net-lwip.c
> >   create mode 100644 doc/develop/net_lwip.rst
> >   create mode 100644 include/net/arp.h
> >   create mode 100644 include/net/eth.h
> >   create mode 100644 include/net/lwip.h
> >   create mode 100644 include/net/ulwip.h
> >   create mode 100644 net/lwip/.gitignore
> >   create mode 100644 net/lwip/Kconfig
> >   create mode 100644 net/lwip/Makefile
> >   create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
> >   create mode 100644 net/lwip/apps/dns/lwip-dns.c
> >   create mode 100644 net/lwip/apps/http/Makefile
> >   create mode 100644 net/lwip/apps/http/lwip-wget.c
> >   create mode 100644 net/lwip/apps/ping/Makefile
> >   create mode 100644 net/lwip/apps/ping/lwip_ping.c
> >   create mode 100644 net/lwip/apps/ping/lwip_ping.h
> >   create mode 100644 net/lwip/apps/ping/ping.h
> >   create mode 100644 net/lwip/apps/tftp/Makefile
> >   create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
> >   create mode 160000 net/lwip/lwip-external
> >   create mode 100644 net/lwip/lwipopts.h
> >   create mode 100644 net/lwip/port/if.c
> >   create mode 100644 net/lwip/port/include/arch/cc.h
> >   create mode 100644 net/lwip/port/include/arch/sys_arch.h
> >   create mode 100644 net/lwip/port/include/limits.h
> >   create mode 100644 net/lwip/port/sys-arch.c
> >
>
>
Tim Harvey Nov. 28, 2023, 11:05 p.m. UTC | #5
On Mon, Nov 27, 2023 at 5:00 AM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Hello,
>
> Please find updated version of lwip patches. Changes are in the
> changelog bellow.
>
> Thank you,
> Maxim.
>

Hi Maxim,

I've tried this series on my imx8mm-venice boards and find that basic
ping/dhcp/tftp breaks:
u-boot=> net list
eth0 : ethernet@30be0000 00:d0:12:78:f8:01 active
u-boot=> setenv ipaddr 192.168.1.1
u-boot=> ping 192.168.1.146
eth0: ethernet@30be0000 00:d0:12:78:f8:01 active
Using ethernet@30be0000 device
pinging addr: 192.168.1.146
ping_tmo: ping failed; host 192.168.1.146 is not alive
u-boot=> dhcp
init already done for ethernet@30be0000
dhcp_tmo 20/20
dhcp_tmo 19/20
dhcp_tmo 18/20
dhcp_tmo 17/20
dhcp_tmo 16/20
dhcp_tmo 15/20
dhcp_tmo 14/20
dhcp_tmo 13/20
dhcp_tmo 12/20
dhcp_tmo 11/20
dhcp_tmo 10/20
dhcp_tmo 9/20
dhcp_tmo 8/20
dhcp_tmo 7/20
dhcp_tmo 6/20
dhcp_tmo 5/20
dhcp_tmo 4/20
dhcp_tmo 3/20
dhcp_tmo 2/20
dhcp_tmo 1/20
dhcp_tmo 0/20
DHCP client timeout
u-boot=> tftpboot $loadaddr venice/Image
init already done for ethernet@30be0000
TFTP from server 192.168.1.146; our IP address is 192.168.1.1
Filename 'venice/Image'.
Load address: 0x48200000
Loading:
^ cntl-c after a minute or so of no output

Is there some basic config here that I may have wrong?

Also, while trying to see if I can determine where things broke I
noticed there is build breakage starting with 'net/lwip: integrate
lwIP library'
In file included from net/lwip/lwip-external/src/core/init.c:38:
./net/lwip/lwip-external/src/include/lwip/opt.h:51:10: fatal error: lwipopts.h:
No such file or directory
   51 | #include "lwipopts.h"
      |          ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [scripts/Makefile.build:256: net/lwip/lwip-external/src/core/init.o
] Error 1
make[1]: *** [scripts/Makefile.build:397: net/lwip] Error 2
make: *** [Makefile:1858: net] Error 2
make: *** Waiting for unfinished jobs....

We need to make sure that none of the patches within your series break
basic compilation or it makes it difficult for people to bisect
issues.

Best regards,

Tim
Fabio Estevam Nov. 28, 2023, 11:21 p.m. UTC | #6
Hi Tim and Maxim,

On Tue, Nov 28, 2023 at 8:05 PM Tim Harvey <tharvey@gateworks.com> wrote:

> Hi Maxim,
>
> I've tried this series on my imx8mm-venice boards and find that basic
> ping/dhcp/tftp breaks:
> u-boot=> net list
> eth0 : ethernet@30be0000 00:d0:12:78:f8:01 active
> u-boot=> setenv ipaddr 192.168.1.1
> u-boot=> ping 192.168.1.146
> eth0: ethernet@30be0000 00:d0:12:78:f8:01 active
> Using ethernet@30be0000 device
> pinging addr: 192.168.1.146
> ping_tmo: ping failed; host 192.168.1.146 is not alive
> u-boot=> dhcp
> init already done for ethernet@30be0000
> dhcp_tmo 20/20
> dhcp_tmo 19/20
> dhcp_tmo 18/20
> dhcp_tmo 17/20
> dhcp_tmo 16/20
> dhcp_tmo 15/20
> dhcp_tmo 14/20
> dhcp_tmo 13/20
> dhcp_tmo 12/20
> dhcp_tmo 11/20
> dhcp_tmo 10/20
> dhcp_tmo 9/20
> dhcp_tmo 8/20
> dhcp_tmo 7/20
> dhcp_tmo 6/20
> dhcp_tmo 5/20
> dhcp_tmo 4/20
> dhcp_tmo 3/20
> dhcp_tmo 2/20
> dhcp_tmo 1/20
> dhcp_tmo 0/20
> DHCP client timeout

I got the same result on a custom imx8mn-based board.

> u-boot=> tftpboot $loadaddr venice/Image
> init already done for ethernet@30be0000
> TFTP from server 192.168.1.146; our IP address is 192.168.1.1
> Filename 'venice/Image'.
> Load address: 0x48200000
> Loading:
> ^ cntl-c after a minute or so of no output
>
> Is there some basic config here that I may have wrong?
>
> Also, while trying to see if I can determine where things broke I
> noticed there is build breakage starting with 'net/lwip: integrate
> lwIP library'
> In file included from net/lwip/lwip-external/src/core/init.c:38:
> ./net/lwip/lwip-external/src/include/lwip/opt.h:51:10: fatal error: lwipopts.h:
> No such file or directory
>    51 | #include "lwipopts.h"
>       |          ^~~~~~~~~~~~
> compilation terminated.
> make[2]: *** [scripts/Makefile.build:256: net/lwip/lwip-external/src/core/init.o
> ] Error 1
> make[1]: *** [scripts/Makefile.build:397: net/lwip] Error 2
> make: *** [Makefile:1858: net] Error 2

I haven't noticed this build error here though. I applied Maxim's
series against master.
Tim Harvey Nov. 29, 2023, 12:10 a.m. UTC | #7
On Tue, Nov 28, 2023 at 3:21 PM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi Tim and Maxim,
>
> On Tue, Nov 28, 2023 at 8:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
>
> > Hi Maxim,
> >
> > I've tried this series on my imx8mm-venice boards and find that basic
> > ping/dhcp/tftp breaks:
> > u-boot=> net list
> > eth0 : ethernet@30be0000 00:d0:12:78:f8:01 active
> > u-boot=> setenv ipaddr 192.168.1.1
> > u-boot=> ping 192.168.1.146
> > eth0: ethernet@30be0000 00:d0:12:78:f8:01 active
> > Using ethernet@30be0000 device
> > pinging addr: 192.168.1.146
> > ping_tmo: ping failed; host 192.168.1.146 is not alive
> > u-boot=> dhcp
> > init already done for ethernet@30be0000
> > dhcp_tmo 20/20
> > dhcp_tmo 19/20
> > dhcp_tmo 18/20
> > dhcp_tmo 17/20
> > dhcp_tmo 16/20
> > dhcp_tmo 15/20
> > dhcp_tmo 14/20
> > dhcp_tmo 13/20
> > dhcp_tmo 12/20
> > dhcp_tmo 11/20
> > dhcp_tmo 10/20
> > dhcp_tmo 9/20
> > dhcp_tmo 8/20
> > dhcp_tmo 7/20
> > dhcp_tmo 6/20
> > dhcp_tmo 5/20
> > dhcp_tmo 4/20
> > dhcp_tmo 3/20
> > dhcp_tmo 2/20
> > dhcp_tmo 1/20
> > dhcp_tmo 0/20
> > DHCP client timeout
>
> I got the same result on a custom imx8mn-based board.
>
> > u-boot=> tftpboot $loadaddr venice/Image
> > init already done for ethernet@30be0000
> > TFTP from server 192.168.1.146; our IP address is 192.168.1.1
> > Filename 'venice/Image'.
> > Load address: 0x48200000
> > Loading:
> > ^ cntl-c after a minute or so of no output
> >
> > Is there some basic config here that I may have wrong?
> >
> > Also, while trying to see if I can determine where things broke I
> > noticed there is build breakage starting with 'net/lwip: integrate
> > lwIP library'
> > In file included from net/lwip/lwip-external/src/core/init.c:38:
> > ./net/lwip/lwip-external/src/include/lwip/opt.h:51:10: fatal error: lwipopts.h:
> > No such file or directory
> >    51 | #include "lwipopts.h"
> >       |          ^~~~~~~~~~~~
> > compilation terminated.
> > make[2]: *** [scripts/Makefile.build:256: net/lwip/lwip-external/src/core/init.o
> > ] Error 1
> > make[1]: *** [scripts/Makefile.build:397: net/lwip] Error 2
> > make: *** [Makefile:1858: net] Error 2
>
> I haven't noticed this build error here though. I applied Maxim's
> series against master.

Fabio,

You won't see this build issue with the entire series applied... only
with the patches up to and including 'net/lwip: integrate lwIP
library'. My point was there are points in the series where
compilation fails which is not ok.

Tim
Fabio Estevam Nov. 29, 2023, 12:16 a.m. UTC | #8
Hi Tim,

On Tue, Nov 28, 2023 at 9:10 PM Tim Harvey <tharvey@gateworks.com> wrote:

> Fabio,
>
> You won't see this build issue with the entire series applied... only
> with the patches up to and including 'net/lwip: integrate lwIP
> library'. My point was there are points in the series where
> compilation fails which is not ok.

You are right and I agree with your point.

I am able to reproduce the build failure at 'net/lwip: integrate lwIP library':

In file included from net/lwip/lwip-external/src/core/init.c:38:
./net/lwip/lwip-external/src/include/lwip/opt.h:51:10: fatal error:
lwipopts.h: No such file or directory
   51 | #include "lwipopts.h"
      |          ^~~~~~~~~~~~

Thanks
Maxim Uvarov Nov. 30, 2023, 8:39 a.m. UTC | #9
On Wed, 29 Nov 2023 at 06:16, Fabio Estevam <festevam@gmail.com> wrote:

> Hi Tim,
>
> On Tue, Nov 28, 2023 at 9:10 PM Tim Harvey <tharvey@gateworks.com> wrote:
>
> > Fabio,
> >
> > You won't see this build issue with the entire series applied... only
> > with the patches up to and including 'net/lwip: integrate lwIP
> > library'. My point was there are points in the series where
> > compilation fails which is not ok.
>
> You are right and I agree with your point.
>
> I am able to reproduce the build failure at 'net/lwip: integrate lwIP
> library':
>
> In file included from net/lwip/lwip-external/src/core/init.c:38:
> ./net/lwip/lwip-external/src/include/lwip/opt.h:51:10: fatal error:
> lwipopts.h: No such file or directory
>    51 | #include "lwipopts.h"
>       |          ^~~~~~~~~~~~
>
> Thanks
>

Ok, I will try to reorder patches.

If networking for some reason doesn't work on that board (I tested only
virtual board). Please try to compile with:
CONFIG_LWIP_LIB_DEBUG=y
#CONFIG_LWIP_LIB_NOASSERT is not set

It should make network activity more verbose. In many cases I saw issues
that physically MAC address do not match to lwip initialized value,
and lwip drops such packets.

BR,
Maxim.
Peter Robinson Nov. 30, 2023, 11:09 a.m. UTC | #10
Hi Maxim,

> Please find updated version of lwip patches. Changes are in the
> changelog bellow.
>
> Thank you,
> Maxim.
>
> changelog:
>         v11: - v11 is mosly respin of v10 patches with CI error fixes.
>                 Gitlab CI:
>                 https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
>                 Azure CI:
>                 https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
>                 (Azure CI, which is connected to github. Sometime I can see
>                  tftp timeout after some part of download there, but that can not be
>                  reproduced locally. While Gitblab CI is stable. Because of num tries in
>                  CI I suspect this CI was not always reliable.)
>                 Azure and Gitlab also have different toolchains and I
>                 would say Gitlab generates bigger code then Azure CI.
>
>                 Also many boards have a binary limit size of 800k (even
>                 qemu has limits). And increased limits to fit all the code. Specially did it
>                 patch by board config to show which boards are failing to build. There I have
>                 a question if we really want to support new functionality for old boards (mips,
>                 arm32 and etc...). I hope board owners can help me if
>                 it's valid to increase these limits.
>
>                 In this version I used git submodules and friend CI with
>                 submodules. But I don't mind if you decide to maintain it in a different
>                 way.
>
>
>         v10: - fix ping with following tftp command issue with incorrect
>                 ping timeout clear.
>              - Makefile on make will init submodules and if needed will
>                do git clone.
>              - wget - some minor code style changes.
>         v9: - added first patch describing git submodule for lwip. So
>               the build procedure is:
>                 git submodule init
>                 git submodule update
>                 make
>             - reworked a little bit dhcp cmd state polling
>             - fixed review comments for v8
>         v8: - comments for previous review
>             - removed lwip timeout callback pointer
>             - made lwip timeouts works, that also allowed to remove
>               static vars.
>             - setenv for filesize tftp and wget has to be in hex.
>             - Makefile changes always compile it tftp,dns,wget,ping due
>               to it can be used not only by CONFIG_CMD_.
>             - Kconfig changes - simplify lwIP settings and support only
>               one configuration.
>             - tested with mini debian.iso load over http or tftp, mount
>               and boot it (qemu, arm64).
>         v7: - more review fixes.
>             - support of multiply eth devices, were "ethact" selects the
>               active device.
>         v6: - fixed review comments for v5 (thanks Ilias and Simon).
>         v5: - fixed Iliases comments and split big patch on the small
>                 ones.
>         v4: - tested with tests/py/ did some minor fixes (out of tree
>                 build, variables set after downloads).
>             - accounted review comments for documentation.
>             - implemented dns command
>             - corrected wget command to not use serverip variable and use just
>                 url string.
>         v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
>               applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
>             - docs: use rst variant and drop references to RFC.
>
> Maxim Uvarov (43):
>   submodule: add lwIP as git submodule
>   net/lwip: add doc/develop/net_lwip.rst
>   net/lwip: integrate lwIP library
>   net/lwip: implement dns cmd
>   net/lwip: implement dhcp cmd
>   net/lwip: implement tftp cmd
>   net/lwip: implement wget cmd
>   net/lwip: implement ping cmd
>   net/lwip: add lwIP configuration
>   net/lwip: implement lwIP port to U-Boot
>   net/lwip: update .gitignore with lwIP
>   net/lwip: connection between cmd and lwip apps
>   net/lwip: replace original net commands with lwip
>   net/lwip: split net.h to net.h, arp.h and eth.h

I think the patches below likely make sense to be split out, either
into individual patches or a fixes/cleanup series that the LWIP series
depends upon. A bunch of the patches below look like they would even
be reasonable fixes for the current release cycle.

Peter

>   test_efi_loader.py: use $filesize var
>   test_net: print out net list
>   net: sandbox: fix NULL pointer derefences
>   net/smc911x: fix return from smc911x_send
>   sandbox: eth-raw-os: successful return code is 0
>   driver/net/rtl8139: remove debug print
>   mach-socfpga: do not overlap defines with lwip
>   bcm_ns3: fix overlap define with lwip
>   rcar3_salvator-x_defconfig: increase binary size limit
>   lwip: omap3: rename mem_init
>   configs/turris_omnia_defconfig set limit to 0xf6000
>   configs/tbs2910_defconfig inc limit
>   configs/socfpga_secu1_defconfig: enable LTO
>   configs/turris_omnia_defconfig: enable LTO
>   configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
>     size
>   configs/sheevaplug_defconfig: enable LTO and inc size
>   configs/lschlv2_defconfig: enable LTO and inc size
>   configs/lsxhl_defconfig: LTO + size
>   configs/am335x_evm_defconfig: inc SPL size
>   configs/bk4r1_defconfig: inc size
>   configs/linkit-smart-7688_defconfig: increse size
>   configs/gardena-smart-gateway-mt7688_defconfig: increase size
>   configs/rcar3_ulcb_defconfig: increase size
>   configs/qemu-x86_64_defconfig: increase ROM size
>   Makefile: add dtbs to clean
>   .azure-pipelines: init submodules
>   mach-mtmips: inc SPL size limit
>   configs/linkit-smart-7688_defconfig: increase board limit
>   .gitlab-ci.yml: change ownership of the git files
>
>  .azure-pipelines.yml                          |  10 +-
>  .gitlab-ci.yml                                |  15 +-
>  .gitmodules                                   |   4 +
>  Makefile                                      |  12 +-
>  arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
>  arch/arm/mach-omap2/omap3/board.c             |   2 +-
>  arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
>  arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
>  .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
>  arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
>  arch/mips/mach-mtmips/Kconfig                 |   2 +-
>  arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
>  boot/bootmeth_efi.c                           |  18 +-
>  boot/bootmeth_pxe.c                           |  21 +-
>  cmd/Makefile                                  |   1 +
>  cmd/net-lwip.c                                | 307 ++++++++++++++++
>  cmd/net.c                                     |  86 +----
>  cmd/pxe.c                                     |  19 +-
>  configs/am335x_boneblack_vboot_defconfig      |   3 +
>  configs/am335x_evm_defconfig                  |   1 +
>  configs/bk4r1_defconfig                       |   2 +-
>  .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
>  configs/linkit-smart-7688_defconfig           |   2 +-
>  configs/lschlv2_defconfig                     |   3 +-
>  configs/lsxhl_defconfig                       |   3 +-
>  configs/qemu-x86_64_defconfig                 |   5 +-
>  configs/rcar3_salvator-x_defconfig            |   3 +-
>  configs/rcar3_ulcb_defconfig                  |   1 +
>  configs/sheevaplug_defconfig                  |   3 +-
>  configs/socfpga_secu1_defconfig               |   1 +
>  configs/tbs2910_defconfig                     |   2 +-
>  configs/turris_omnia_defconfig                |   3 +-
>  doc/develop/index.rst                         |   1 +
>  doc/develop/net_lwip.rst                      |  75 ++++
>  drivers/net/rtl8139.c                         |   1 -
>  drivers/net/sandbox.c                         |   5 +
>  drivers/net/smc911x.c                         |   2 +-
>  include/configs/bcm_ns3.h                     |   6 +-
>  include/net.h                                 | 197 +----------
>  include/net/arp.h                             |   9 +
>  include/net/eth.h                             | 194 +++++++++++
>  include/net/lwip.h                            |  73 ++++
>  include/net/ulwip.h                           |  67 ++++
>  lib/Kconfig                                   |   2 +-
>  net/Kconfig                                   |   3 +
>  net/Makefile                                  |   1 +
>  net/eth-uclass.c                              |  37 +-
>  net/lwip/.gitignore                           |   8 +
>  net/lwip/Kconfig                              |  34 ++
>  net/lwip/Makefile                             |  70 ++++
>  net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
>  net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
>  net/lwip/apps/http/Makefile                   |   6 +
>  net/lwip/apps/http/lwip-wget.c                | 105 ++++++
>  net/lwip/apps/ping/Makefile                   |  12 +
>  net/lwip/apps/ping/lwip_ping.c                |  39 +++
>  net/lwip/apps/ping/lwip_ping.h                |  15 +
>  net/lwip/apps/ping/ping.h                     |  28 ++
>  net/lwip/apps/tftp/Makefile                   |   7 +
>  net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
>  net/lwip/lwip-external                        |   1 +
>  net/lwip/lwipopts.h                           | 178 ++++++++++
>  net/lwip/port/if.c                            | 327 ++++++++++++++++++
>  net/lwip/port/include/arch/cc.h               |  44 +++
>  net/lwip/port/include/arch/sys_arch.h         |  10 +
>  net/lwip/port/include/limits.h                |   0
>  net/lwip/port/sys-arch.c                      |  13 +
>  net/net.c                                     |  44 ++-
>  test/py/tests/test_efi_loader.py              |   4 +-
>  test/py/tests/test_net.py                     |   2 +
>  70 files changed, 2113 insertions(+), 348 deletions(-)
>  create mode 100644 .gitmodules
>  create mode 100644 cmd/net-lwip.c
>  create mode 100644 doc/develop/net_lwip.rst
>  create mode 100644 include/net/arp.h
>  create mode 100644 include/net/eth.h
>  create mode 100644 include/net/lwip.h
>  create mode 100644 include/net/ulwip.h
>  create mode 100644 net/lwip/.gitignore
>  create mode 100644 net/lwip/Kconfig
>  create mode 100644 net/lwip/Makefile
>  create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
>  create mode 100644 net/lwip/apps/dns/lwip-dns.c
>  create mode 100644 net/lwip/apps/http/Makefile
>  create mode 100644 net/lwip/apps/http/lwip-wget.c
>  create mode 100644 net/lwip/apps/ping/Makefile
>  create mode 100644 net/lwip/apps/ping/lwip_ping.c
>  create mode 100644 net/lwip/apps/ping/lwip_ping.h
>  create mode 100644 net/lwip/apps/ping/ping.h
>  create mode 100644 net/lwip/apps/tftp/Makefile
>  create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
>  create mode 160000 net/lwip/lwip-external
>  create mode 100644 net/lwip/lwipopts.h
>  create mode 100644 net/lwip/port/if.c
>  create mode 100644 net/lwip/port/include/arch/cc.h
>  create mode 100644 net/lwip/port/include/arch/sys_arch.h
>  create mode 100644 net/lwip/port/include/limits.h
>  create mode 100644 net/lwip/port/sys-arch.c
>
> --
> 2.30.2
>
Fabio Estevam Nov. 30, 2023, 1 p.m. UTC | #11
Hi Maxim,

On Thu, Nov 30, 2023 at 5:39 AM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:

> If networking for some reason doesn't work on that board (I tested only virtual board). Please try to compile with:

Could you also try on a real board?

Tom tested on a Raspberry Pi3 and it failed too:

https://lore.kernel.org/u-boot/20231127185953.GW2513409@bill-the-cat/

> CONFIG_LWIP_LIB_DEBUG=y
> #CONFIG_LWIP_LIB_NOASSERT is not set
>
> It should make network activity more verbose. In many cases I saw issues that physically MAC address do not match to lwip initialized value,
> and lwip drops such packets.

The ping response with:

CONFIG_LWIP_LIB_DEBUG=y
 #CONFIG_LWIP_LIB_NOASSERT is not set

u-boot=> ping 192.168.0.16
init already done for eth1
Using eth1 device
pinging addr: 192.168.0.16
ping: send 192.168.0.16
ip4_output_if: 0IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        60     | (v, hl, tos, len)
+-------------------------------+
|       40      |000|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    1  |    0x3994     | (ttl, proto, chksum)
+-------------------------------+
|  192  |  168  |    0  |  164  | (src)
+-------------------------------+
|  192  |  168  |    0  |   16  | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
etharp_find_entry: found matching entry 0
etharp_query: queued packet 00000000bdf03e70 on ARP entry 0
ping_tmo: ping failed; host 192.168.0.16 is not alive
u-boot=>
Maxim Uvarov Nov. 30, 2023, 1:05 p.m. UTC | #12
On Thu, 30 Nov 2023 at 17:09, Peter Robinson <pbrobinson@gmail.com> wrote:

> Hi Maxim,
>
> > Please find updated version of lwip patches. Changes are in the
> > changelog bellow.
> >
> > Thank you,
> > Maxim.
> >
> > changelog:
> >         v11: - v11 is mosly respin of v10 patches with CI error fixes.
> >                 Gitlab CI:
> >
> https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
> >                 Azure CI:
> >
> https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
> >                 (Azure CI, which is connected to github. Sometime I can
> see
> >                  tftp timeout after some part of download there, but
> that can not be
> >                  reproduced locally. While Gitblab CI is stable. Because
> of num tries in
> >                  CI I suspect this CI was not always reliable.)
> >                 Azure and Gitlab also have different toolchains and I
> >                 would say Gitlab generates bigger code then Azure CI.
> >
> >                 Also many boards have a binary limit size of 800k (even
> >                 qemu has limits). And increased limits to fit all the
> code. Specially did it
> >                 patch by board config to show which boards are failing
> to build. There I have
> >                 a question if we really want to support new
> functionality for old boards (mips,
> >                 arm32 and etc...). I hope board owners can help me if
> >                 it's valid to increase these limits.
> >
> >                 In this version I used git submodules and friend CI with
> >                 submodules. But I don't mind if you decide to maintain
> it in a different
> >                 way.
> >
> >
> >         v10: - fix ping with following tftp command issue with incorrect
> >                 ping timeout clear.
> >              - Makefile on make will init submodules and if needed will
> >                do git clone.
> >              - wget - some minor code style changes.
> >         v9: - added first patch describing git submodule for lwip. So
> >               the build procedure is:
> >                 git submodule init
> >                 git submodule update
> >                 make
> >             - reworked a little bit dhcp cmd state polling
> >             - fixed review comments for v8
> >         v8: - comments for previous review
> >             - removed lwip timeout callback pointer
> >             - made lwip timeouts works, that also allowed to remove
> >               static vars.
> >             - setenv for filesize tftp and wget has to be in hex.
> >             - Makefile changes always compile it tftp,dns,wget,ping due
> >               to it can be used not only by CONFIG_CMD_.
> >             - Kconfig changes - simplify lwIP settings and support only
> >               one configuration.
> >             - tested with mini debian.iso load over http or tftp, mount
> >               and boot it (qemu, arm64).
> >         v7: - more review fixes.
> >             - support of multiply eth devices, were "ethact" selects the
> >               active device.
> >         v6: - fixed review comments for v5 (thanks Ilias and Simon).
> >         v5: - fixed Iliases comments and split big patch on the small
> >                 ones.
> >         v4: - tested with tests/py/ did some minor fixes (out of tree
> >                 build, variables set after downloads).
> >             - accounted review comments for documentation.
> >             - implemented dns command
> >             - corrected wget command to not use serverip variable and
> use just
> >                 url string.
> >         v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
> >               applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
> >             - docs: use rst variant and drop references to RFC.
> >
> > Maxim Uvarov (43):
> >   submodule: add lwIP as git submodule
> >   net/lwip: add doc/develop/net_lwip.rst
> >   net/lwip: integrate lwIP library
> >   net/lwip: implement dns cmd
> >   net/lwip: implement dhcp cmd
> >   net/lwip: implement tftp cmd
> >   net/lwip: implement wget cmd
> >   net/lwip: implement ping cmd
> >   net/lwip: add lwIP configuration
> >   net/lwip: implement lwIP port to U-Boot
> >   net/lwip: update .gitignore with lwIP
> >   net/lwip: connection between cmd and lwip apps
> >   net/lwip: replace original net commands with lwip
> >   net/lwip: split net.h to net.h, arp.h and eth.h
>
> I think the patches below likely make sense to be split out, either
> into individual patches or a fixes/cleanup series that the LWIP series
> depends upon. A bunch of the patches below look like they would even
> be reasonable fixes for the current release cycle.
>
> Peter
>

Yes, Peter, some fixes are not related to lwip, but somehow they triggered
failures. I will send them as a separate patchset.
What I'm trying to do now is disable TCP for lwip and remove original
tftp,ping and dns code. In my calculations this configuration
should be ok for all boards limits. Original ping 2k, tftp and dns - 7k  -
that has to be enough for lwip ping, tftp and dns. If the sizing
question is resolved then other issues have to be easy.

BR,
Maxim.


>
> >   test_efi_loader.py: use $filesize var
> >   test_net: print out net list
> >   net: sandbox: fix NULL pointer derefences
> >   net/smc911x: fix return from smc911x_send
> >   sandbox: eth-raw-os: successful return code is 0
> >   driver/net/rtl8139: remove debug print
> >   mach-socfpga: do not overlap defines with lwip
> >   bcm_ns3: fix overlap define with lwip
> >   rcar3_salvator-x_defconfig: increase binary size limit
> >   lwip: omap3: rename mem_init
> >   configs/turris_omnia_defconfig set limit to 0xf6000
> >   configs/tbs2910_defconfig inc limit
> >   configs/socfpga_secu1_defconfig: enable LTO
> >   configs/turris_omnia_defconfig: enable LTO
> >   configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
> >     size
> >   configs/sheevaplug_defconfig: enable LTO and inc size
> >   configs/lschlv2_defconfig: enable LTO and inc size
> >   configs/lsxhl_defconfig: LTO + size
> >   configs/am335x_evm_defconfig: inc SPL size
> >   configs/bk4r1_defconfig: inc size
> >   configs/linkit-smart-7688_defconfig: increse size
> >   configs/gardena-smart-gateway-mt7688_defconfig: increase size
> >   configs/rcar3_ulcb_defconfig: increase size
> >   configs/qemu-x86_64_defconfig: increase ROM size
> >   Makefile: add dtbs to clean
> >   .azure-pipelines: init submodules
> >   mach-mtmips: inc SPL size limit
> >   configs/linkit-smart-7688_defconfig: increase board limit
> >   .gitlab-ci.yml: change ownership of the git files
> >
> >  .azure-pipelines.yml                          |  10 +-
> >  .gitlab-ci.yml                                |  15 +-
> >  .gitmodules                                   |   4 +
> >  Makefile                                      |  12 +-
> >  arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
> >  arch/arm/mach-omap2/omap3/board.c             |   2 +-
> >  arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
> >  arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
> >  .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
> >  arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
> >  arch/mips/mach-mtmips/Kconfig                 |   2 +-
> >  arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
> >  boot/bootmeth_efi.c                           |  18 +-
> >  boot/bootmeth_pxe.c                           |  21 +-
> >  cmd/Makefile                                  |   1 +
> >  cmd/net-lwip.c                                | 307 ++++++++++++++++
> >  cmd/net.c                                     |  86 +----
> >  cmd/pxe.c                                     |  19 +-
> >  configs/am335x_boneblack_vboot_defconfig      |   3 +
> >  configs/am335x_evm_defconfig                  |   1 +
> >  configs/bk4r1_defconfig                       |   2 +-
> >  .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
> >  configs/linkit-smart-7688_defconfig           |   2 +-
> >  configs/lschlv2_defconfig                     |   3 +-
> >  configs/lsxhl_defconfig                       |   3 +-
> >  configs/qemu-x86_64_defconfig                 |   5 +-
> >  configs/rcar3_salvator-x_defconfig            |   3 +-
> >  configs/rcar3_ulcb_defconfig                  |   1 +
> >  configs/sheevaplug_defconfig                  |   3 +-
> >  configs/socfpga_secu1_defconfig               |   1 +
> >  configs/tbs2910_defconfig                     |   2 +-
> >  configs/turris_omnia_defconfig                |   3 +-
> >  doc/develop/index.rst                         |   1 +
> >  doc/develop/net_lwip.rst                      |  75 ++++
> >  drivers/net/rtl8139.c                         |   1 -
> >  drivers/net/sandbox.c                         |   5 +
> >  drivers/net/smc911x.c                         |   2 +-
> >  include/configs/bcm_ns3.h                     |   6 +-
> >  include/net.h                                 | 197 +----------
> >  include/net/arp.h                             |   9 +
> >  include/net/eth.h                             | 194 +++++++++++
> >  include/net/lwip.h                            |  73 ++++
> >  include/net/ulwip.h                           |  67 ++++
> >  lib/Kconfig                                   |   2 +-
> >  net/Kconfig                                   |   3 +
> >  net/Makefile                                  |   1 +
> >  net/eth-uclass.c                              |  37 +-
> >  net/lwip/.gitignore                           |   8 +
> >  net/lwip/Kconfig                              |  34 ++
> >  net/lwip/Makefile                             |  70 ++++
> >  net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
> >  net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
> >  net/lwip/apps/http/Makefile                   |   6 +
> >  net/lwip/apps/http/lwip-wget.c                | 105 ++++++
> >  net/lwip/apps/ping/Makefile                   |  12 +
> >  net/lwip/apps/ping/lwip_ping.c                |  39 +++
> >  net/lwip/apps/ping/lwip_ping.h                |  15 +
> >  net/lwip/apps/ping/ping.h                     |  28 ++
> >  net/lwip/apps/tftp/Makefile                   |   7 +
> >  net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
> >  net/lwip/lwip-external                        |   1 +
> >  net/lwip/lwipopts.h                           | 178 ++++++++++
> >  net/lwip/port/if.c                            | 327 ++++++++++++++++++
> >  net/lwip/port/include/arch/cc.h               |  44 +++
> >  net/lwip/port/include/arch/sys_arch.h         |  10 +
> >  net/lwip/port/include/limits.h                |   0
> >  net/lwip/port/sys-arch.c                      |  13 +
> >  net/net.c                                     |  44 ++-
> >  test/py/tests/test_efi_loader.py              |   4 +-
> >  test/py/tests/test_net.py                     |   2 +
> >  70 files changed, 2113 insertions(+), 348 deletions(-)
> >  create mode 100644 .gitmodules
> >  create mode 100644 cmd/net-lwip.c
> >  create mode 100644 doc/develop/net_lwip.rst
> >  create mode 100644 include/net/arp.h
> >  create mode 100644 include/net/eth.h
> >  create mode 100644 include/net/lwip.h
> >  create mode 100644 include/net/ulwip.h
> >  create mode 100644 net/lwip/.gitignore
> >  create mode 100644 net/lwip/Kconfig
> >  create mode 100644 net/lwip/Makefile
> >  create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
> >  create mode 100644 net/lwip/apps/dns/lwip-dns.c
> >  create mode 100644 net/lwip/apps/http/Makefile
> >  create mode 100644 net/lwip/apps/http/lwip-wget.c
> >  create mode 100644 net/lwip/apps/ping/Makefile
> >  create mode 100644 net/lwip/apps/ping/lwip_ping.c
> >  create mode 100644 net/lwip/apps/ping/lwip_ping.h
> >  create mode 100644 net/lwip/apps/ping/ping.h
> >  create mode 100644 net/lwip/apps/tftp/Makefile
> >  create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
> >  create mode 160000 net/lwip/lwip-external
> >  create mode 100644 net/lwip/lwipopts.h
> >  create mode 100644 net/lwip/port/if.c
> >  create mode 100644 net/lwip/port/include/arch/cc.h
> >  create mode 100644 net/lwip/port/include/arch/sys_arch.h
> >  create mode 100644 net/lwip/port/include/limits.h
> >  create mode 100644 net/lwip/port/sys-arch.c
> >
> > --
> > 2.30.2
> >
>
Maxim Uvarov Nov. 30, 2023, 1:14 p.m. UTC | #13
On Thu, 30 Nov 2023 at 19:00, Fabio Estevam <festevam@gmail.com> wrote:

> Hi Maxim,
>
> On Thu, Nov 30, 2023 at 5:39 AM Maxim Uvarov <maxim.uvarov@linaro.org>
> wrote:
>
> > If networking for some reason doesn't work on that board (I tested only
> virtual board). Please try to compile with:
>
> Could you also try on a real board?
>
> Tom tested on a Raspberry Pi3 and it failed too:
>
> https://lore.kernel.org/u-boot/20231127185953.GW2513409@bill-the-cat/
>
> > CONFIG_LWIP_LIB_DEBUG=y
> > #CONFIG_LWIP_LIB_NOASSERT is not set
> >
> > It should make network activity more verbose. In many cases I saw issues
> that physically MAC address do not match to lwip initialized value,
> > and lwip drops such packets.
>
> The ping response with:
>
> CONFIG_LWIP_LIB_DEBUG=y
>  #CONFIG_LWIP_LIB_NOASSERT is not set
>
> u-boot=> ping 192.168.0.16
> init already done for eth1
> Using eth1 device
> pinging addr: 192.168.0.16
> ping: send 192.168.0.16
> ip4_output_if: 0IP header:
> +-------------------------------+
> | 4 | 5 |  0x00 |        60     | (v, hl, tos, len)
> +-------------------------------+
> |       40      |000|       0   | (id, flags, offset)
> +-------------------------------+
> |  255  |    1  |    0x3994     | (ttl, proto, chksum)
> +-------------------------------+
> |  192  |  168  |    0  |  164  | (src)
> +-------------------------------+
> |  192  |  168  |    0  |   16  | (dest)
> +-------------------------------+
> ip4_output_if: call netif->output()
> etharp_find_entry: found matching entry 0
> etharp_query: queued packet 00000000bdf03e70 on ARP entry 0
> ping_tmo: ping failed; host 192.168.0.16 is not alive
> u-boot=>
>

I will test that. smsc95xx_eth included in some CI tests and it worked. So
some differences on real hardware...
Log says that a ping packet was sent, but no packet on rx.
Might be the start function of  smsc95xx_eth called wrongly....

BR,
Maxim.
Fabio Estevam Nov. 30, 2023, 1:18 p.m. UTC | #14
On Thu, Nov 30, 2023 at 10:14 AM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:

> I will test that. smsc95xx_eth included in some CI tests and it worked. So some differences on real hardware...
> Log says that a ping packet was sent, but no packet on rx.
> Might be the start function of  smsc95xx_eth called wrongly....

Please note that on the board I tested there is no smsc95xx involved.

It is an i.MX8MN board with the FEC Ethernet controller and an 88E6320 switch.
Alexander Dahl Dec. 19, 2023, 10:12 a.m. UTC | #15
Hello Maxim,

Am Mon, Nov 27, 2023 at 06:56:43PM +0600 schrieb Maxim Uvarov:
> Hello,
> 
> Please find updated version of lwip patches. Changes are in the
> changelog bellow.
> 
> Thank you,
> Maxim. 
> 
> changelog:
> 	v11: - v11 is mosly respin of v10 patches with CI error fixes.
>                 Gitlab CI: 
>                 https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
>                 Azure CI:
>                 https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
>                 (Azure CI, which is connected to github. Sometime I can see
>                  tftp timeout after some part of download there, but that can not be
>                  reproduced locally. While Gitblab CI is stable. Because of num tries in
>                  CI I suspect this CI was not always reliable.)
>                 Azure and Gitlab also have different toolchains and I 
>                 would say Gitlab generates bigger code then Azure CI.
>                 
>                 Also many boards have a binary limit size of 800k (even
>                 qemu has limits). And increased limits to fit all the code. Specially did it
>                 patch by board config to show which boards are failing to build. There I have
>                 a question if we really want to support new functionality for old boards (mips,
>                 arm32 and etc...). I hope board owners can help me if
>                 it's valid to increase these limits.

In general one can not simply increase that limit without knowing
details on where U-Boot binary is supposed to be stored on a
particular board.  For example there are boards where U-Boot is stored
on NAND flash with fixed sized (mtd) partitions.  Changing the
partition layout on a running board is quite risky from my point of
view, so you can assume that partition sizes fixed at all times.
Those sizes determine the limit however.  That said: it has to be
checked board by board if such a limit can be increased at all.

Greets
Alex

> 
> 		In this version I used git submodules and friend CI with
> 		submodules. But I don't mind if you decide to maintain it in a different
> 		way.
> 
> 
> 	v10: - fix ping with following tftp command issue with incorrect
> 		ping timeout clear.
> 	     - Makefile on make will init submodules and if needed will
> 	       do git clone.
> 	     - wget - some minor code style changes.
> 	v9: - added first patch describing git submodule for lwip. So
> 	      the build procedure is:
> 		git submodule init
> 		git submodule update
> 		make
> 	    - reworked a little bit dhcp cmd state polling
> 	    - fixed review comments for v8
> 	v8: - comments for previous review
> 	    - removed lwip timeout callback pointer
> 	    - made lwip timeouts works, that also allowed to remove
> 	      static vars.
> 	    - setenv for filesize tftp and wget has to be in hex.
> 	    - Makefile changes always compile it tftp,dns,wget,ping due
> 	      to it can be used not only by CONFIG_CMD_.
> 	    - Kconfig changes - simplify lwIP settings and support only
> 	      one configuration.
> 	    - tested with mini debian.iso load over http or tftp, mount
> 	      and boot it (qemu, arm64).
> 	v7: - more review fixes.
> 	    - support of multiply eth devices, were "ethact" selects the
> 	      active device.
> 	v6: - fixed review comments for v5 (thanks Ilias and Simon).
> 	v5: - fixed Iliases comments and split big patch on the small
> 		ones.
> 	v4: - tested with tests/py/ did some minor fixes (out of tree
> 		build, variables set after downloads).
> 	    - accounted review comments for documentation.
> 	    - implemented dns command
>             - corrected wget command to not use serverip variable and use just
> 		url string.
> 	v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
> 	      applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
> 	    - docs: use rst variant and drop references to RFC.
> 
> Maxim Uvarov (43):
>   submodule: add lwIP as git submodule
>   net/lwip: add doc/develop/net_lwip.rst
>   net/lwip: integrate lwIP library
>   net/lwip: implement dns cmd
>   net/lwip: implement dhcp cmd
>   net/lwip: implement tftp cmd
>   net/lwip: implement wget cmd
>   net/lwip: implement ping cmd
>   net/lwip: add lwIP configuration
>   net/lwip: implement lwIP port to U-Boot
>   net/lwip: update .gitignore with lwIP
>   net/lwip: connection between cmd and lwip apps
>   net/lwip: replace original net commands with lwip
>   net/lwip: split net.h to net.h, arp.h and eth.h
>   test_efi_loader.py: use $filesize var
>   test_net: print out net list
>   net: sandbox: fix NULL pointer derefences
>   net/smc911x: fix return from smc911x_send
>   sandbox: eth-raw-os: successful return code is 0
>   driver/net/rtl8139: remove debug print
>   mach-socfpga: do not overlap defines with lwip
>   bcm_ns3: fix overlap define with lwip
>   rcar3_salvator-x_defconfig: increase binary size limit
>   lwip: omap3: rename mem_init
>   configs/turris_omnia_defconfig set limit to 0xf6000
>   configs/tbs2910_defconfig inc limit
>   configs/socfpga_secu1_defconfig: enable LTO
>   configs/turris_omnia_defconfig: enable LTO
>   configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
>     size
>   configs/sheevaplug_defconfig: enable LTO and inc size
>   configs/lschlv2_defconfig: enable LTO and inc size
>   configs/lsxhl_defconfig: LTO + size
>   configs/am335x_evm_defconfig: inc SPL size
>   configs/bk4r1_defconfig: inc size
>   configs/linkit-smart-7688_defconfig: increse size
>   configs/gardena-smart-gateway-mt7688_defconfig: increase size
>   configs/rcar3_ulcb_defconfig: increase size
>   configs/qemu-x86_64_defconfig: increase ROM size
>   Makefile: add dtbs to clean
>   .azure-pipelines: init submodules
>   mach-mtmips: inc SPL size limit
>   configs/linkit-smart-7688_defconfig: increase board limit
>   .gitlab-ci.yml: change ownership of the git files
> 
>  .azure-pipelines.yml                          |  10 +-
>  .gitlab-ci.yml                                |  15 +-
>  .gitmodules                                   |   4 +
>  Makefile                                      |  12 +-
>  arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
>  arch/arm/mach-omap2/omap3/board.c             |   2 +-
>  arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
>  arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
>  .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
>  arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
>  arch/mips/mach-mtmips/Kconfig                 |   2 +-
>  arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
>  boot/bootmeth_efi.c                           |  18 +-
>  boot/bootmeth_pxe.c                           |  21 +-
>  cmd/Makefile                                  |   1 +
>  cmd/net-lwip.c                                | 307 ++++++++++++++++
>  cmd/net.c                                     |  86 +----
>  cmd/pxe.c                                     |  19 +-
>  configs/am335x_boneblack_vboot_defconfig      |   3 +
>  configs/am335x_evm_defconfig                  |   1 +
>  configs/bk4r1_defconfig                       |   2 +-
>  .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
>  configs/linkit-smart-7688_defconfig           |   2 +-
>  configs/lschlv2_defconfig                     |   3 +-
>  configs/lsxhl_defconfig                       |   3 +-
>  configs/qemu-x86_64_defconfig                 |   5 +-
>  configs/rcar3_salvator-x_defconfig            |   3 +-
>  configs/rcar3_ulcb_defconfig                  |   1 +
>  configs/sheevaplug_defconfig                  |   3 +-
>  configs/socfpga_secu1_defconfig               |   1 +
>  configs/tbs2910_defconfig                     |   2 +-
>  configs/turris_omnia_defconfig                |   3 +-
>  doc/develop/index.rst                         |   1 +
>  doc/develop/net_lwip.rst                      |  75 ++++
>  drivers/net/rtl8139.c                         |   1 -
>  drivers/net/sandbox.c                         |   5 +
>  drivers/net/smc911x.c                         |   2 +-
>  include/configs/bcm_ns3.h                     |   6 +-
>  include/net.h                                 | 197 +----------
>  include/net/arp.h                             |   9 +
>  include/net/eth.h                             | 194 +++++++++++
>  include/net/lwip.h                            |  73 ++++
>  include/net/ulwip.h                           |  67 ++++
>  lib/Kconfig                                   |   2 +-
>  net/Kconfig                                   |   3 +
>  net/Makefile                                  |   1 +
>  net/eth-uclass.c                              |  37 +-
>  net/lwip/.gitignore                           |   8 +
>  net/lwip/Kconfig                              |  34 ++
>  net/lwip/Makefile                             |  70 ++++
>  net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
>  net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
>  net/lwip/apps/http/Makefile                   |   6 +
>  net/lwip/apps/http/lwip-wget.c                | 105 ++++++
>  net/lwip/apps/ping/Makefile                   |  12 +
>  net/lwip/apps/ping/lwip_ping.c                |  39 +++
>  net/lwip/apps/ping/lwip_ping.h                |  15 +
>  net/lwip/apps/ping/ping.h                     |  28 ++
>  net/lwip/apps/tftp/Makefile                   |   7 +
>  net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
>  net/lwip/lwip-external                        |   1 +
>  net/lwip/lwipopts.h                           | 178 ++++++++++
>  net/lwip/port/if.c                            | 327 ++++++++++++++++++
>  net/lwip/port/include/arch/cc.h               |  44 +++
>  net/lwip/port/include/arch/sys_arch.h         |  10 +
>  net/lwip/port/include/limits.h                |   0
>  net/lwip/port/sys-arch.c                      |  13 +
>  net/net.c                                     |  44 ++-
>  test/py/tests/test_efi_loader.py              |   4 +-
>  test/py/tests/test_net.py                     |   2 +
>  70 files changed, 2113 insertions(+), 348 deletions(-)
>  create mode 100644 .gitmodules
>  create mode 100644 cmd/net-lwip.c
>  create mode 100644 doc/develop/net_lwip.rst
>  create mode 100644 include/net/arp.h
>  create mode 100644 include/net/eth.h
>  create mode 100644 include/net/lwip.h
>  create mode 100644 include/net/ulwip.h
>  create mode 100644 net/lwip/.gitignore
>  create mode 100644 net/lwip/Kconfig
>  create mode 100644 net/lwip/Makefile
>  create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
>  create mode 100644 net/lwip/apps/dns/lwip-dns.c
>  create mode 100644 net/lwip/apps/http/Makefile
>  create mode 100644 net/lwip/apps/http/lwip-wget.c
>  create mode 100644 net/lwip/apps/ping/Makefile
>  create mode 100644 net/lwip/apps/ping/lwip_ping.c
>  create mode 100644 net/lwip/apps/ping/lwip_ping.h
>  create mode 100644 net/lwip/apps/ping/ping.h
>  create mode 100644 net/lwip/apps/tftp/Makefile
>  create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
>  create mode 160000 net/lwip/lwip-external
>  create mode 100644 net/lwip/lwipopts.h
>  create mode 100644 net/lwip/port/if.c
>  create mode 100644 net/lwip/port/include/arch/cc.h
>  create mode 100644 net/lwip/port/include/arch/sys_arch.h
>  create mode 100644 net/lwip/port/include/limits.h
>  create mode 100644 net/lwip/port/sys-arch.c
> 
> -- 
> 2.30.2
>
Maxim Uvarov Jan. 2, 2024, 5:30 p.m. UTC | #16
Small update here. I made changes without any board limit size changes.
Reordered patches to make them compile in sequence. I think I will do one
more clean up and more testing,
and after that will be able to send an updated version.
Working tree is here: https://github.com/u-boot/u-boot/pull/434

SPL might have a very limited network stack (current code for tftp and
bootp). While later stage bootloader can have complete protocol (lwip).
That model fits all current boards limits.

'ethrotate' variable is more likely to go away and be replaced with routing
tables and command line arguments ('dhcp all', "ping -i eth0 IP" ).

However, I have a strange issue with mips malta. Work around:
https://github.com/u-boot/u-boot/pull/434/commits/14717056115944f67eeb22571330d551a1b1ecb7
Issue is that if LWIP_TCP=0 generated code for network stack is less on few
kilobytes. In that case
qemu is unable to start and does not even print hello messages. I followed
this documentation ./doc/board/emulation/qemu-mips.rst
for qemu run. While if image size is bigger and TCP for example compiled
then qemu boots and operates fine. For now
this issue is a little bit strange for me. And more strange how that
related to lwip code, because the network works on much later
stage then initial messages print.

Thanks,
Maxim.


On Tue, 19 Dec 2023 at 16:12, Alexander Dahl <ada@thorsis.com> wrote:

> Hello Maxim,
>
> Am Mon, Nov 27, 2023 at 06:56:43PM +0600 schrieb Maxim Uvarov:
> > Hello,
> >
> > Please find updated version of lwip patches. Changes are in the
> > changelog bellow.
> >
> > Thank you,
> > Maxim.
> >
> > changelog:
> >       v11: - v11 is mosly respin of v10 patches with CI error fixes.
> >                 Gitlab CI:
> >
> https://source.denx.de/u-boot/custodians/u-boot-tpm/-/pipelines/18368
> >                 Azure CI:
> >
> https://dev.azure.com/u-boot/u-boot/_build/results?buildId=7366&view=results
> >                 (Azure CI, which is connected to github. Sometime I can
> see
> >                  tftp timeout after some part of download there, but
> that can not be
> >                  reproduced locally. While Gitblab CI is stable. Because
> of num tries in
> >                  CI I suspect this CI was not always reliable.)
> >                 Azure and Gitlab also have different toolchains and I
> >                 would say Gitlab generates bigger code then Azure CI.
> >
> >                 Also many boards have a binary limit size of 800k (even
> >                 qemu has limits). And increased limits to fit all the
> code. Specially did it
> >                 patch by board config to show which boards are failing
> to build. There I have
> >                 a question if we really want to support new
> functionality for old boards (mips,
> >                 arm32 and etc...). I hope board owners can help me if
> >                 it's valid to increase these limits.
>
> In general one can not simply increase that limit without knowing
> details on where U-Boot binary is supposed to be stored on a
> particular board.  For example there are boards where U-Boot is stored
> on NAND flash with fixed sized (mtd) partitions.  Changing the
> partition layout on a running board is quite risky from my point of
> view, so you can assume that partition sizes fixed at all times.
> Those sizes determine the limit however.  That said: it has to be
> checked board by board if such a limit can be increased at all.
>
> Greets
> Alex
>
> >
> >               In this version I used git submodules and friend CI with
> >               submodules. But I don't mind if you decide to maintain it
> in a different
> >               way.
> >
> >
> >       v10: - fix ping with following tftp command issue with incorrect
> >               ping timeout clear.
> >            - Makefile on make will init submodules and if needed will
> >              do git clone.
> >            - wget - some minor code style changes.
> >       v9: - added first patch describing git submodule for lwip. So
> >             the build procedure is:
> >               git submodule init
> >               git submodule update
> >               make
> >           - reworked a little bit dhcp cmd state polling
> >           - fixed review comments for v8
> >       v8: - comments for previous review
> >           - removed lwip timeout callback pointer
> >           - made lwip timeouts works, that also allowed to remove
> >             static vars.
> >           - setenv for filesize tftp and wget has to be in hex.
> >           - Makefile changes always compile it tftp,dns,wget,ping due
> >             to it can be used not only by CONFIG_CMD_.
> >           - Kconfig changes - simplify lwIP settings and support only
> >             one configuration.
> >           - tested with mini debian.iso load over http or tftp, mount
> >             and boot it (qemu, arm64).
> >       v7: - more review fixes.
> >           - support of multiply eth devices, were "ethact" selects the
> >             active device.
> >       v6: - fixed review comments for v5 (thanks Ilias and Simon).
> >       v5: - fixed Iliases comments and split big patch on the small
> >               ones.
> >       v4: - tested with tests/py/ did some minor fixes (out of tree
> >               build, variables set after downloads).
> >           - accounted review comments for documentation.
> >           - implemented dns command
> >             - corrected wget command to not use serverip variable and
> use just
> >               url string.
> >       v3: - use lwip commands for ping,tftp,wget,dhcp if this patch
> >             applied. Drop CONFIG_LIB_LWIP_REPLACE_<COMMAND> option.
> >           - docs: use rst variant and drop references to RFC.
> >
> > Maxim Uvarov (43):
> >   submodule: add lwIP as git submodule
> >   net/lwip: add doc/develop/net_lwip.rst
> >   net/lwip: integrate lwIP library
> >   net/lwip: implement dns cmd
> >   net/lwip: implement dhcp cmd
> >   net/lwip: implement tftp cmd
> >   net/lwip: implement wget cmd
> >   net/lwip: implement ping cmd
> >   net/lwip: add lwIP configuration
> >   net/lwip: implement lwIP port to U-Boot
> >   net/lwip: update .gitignore with lwIP
> >   net/lwip: connection between cmd and lwip apps
> >   net/lwip: replace original net commands with lwip
> >   net/lwip: split net.h to net.h, arp.h and eth.h
> >   test_efi_loader.py: use $filesize var
> >   test_net: print out net list
> >   net: sandbox: fix NULL pointer derefences
> >   net/smc911x: fix return from smc911x_send
> >   sandbox: eth-raw-os: successful return code is 0
> >   driver/net/rtl8139: remove debug print
> >   mach-socfpga: do not overlap defines with lwip
> >   bcm_ns3: fix overlap define with lwip
> >   rcar3_salvator-x_defconfig: increase binary size limit
> >   lwip: omap3: rename mem_init
> >   configs/turris_omnia_defconfig set limit to 0xf6000
> >   configs/tbs2910_defconfig inc limit
> >   configs/socfpga_secu1_defconfig: enable LTO
> >   configs/turris_omnia_defconfig: enable LTO
> >   configs/am335x_boneblack_vboot_defconfig: enable LTO and increase SPL
> >     size
> >   configs/sheevaplug_defconfig: enable LTO and inc size
> >   configs/lschlv2_defconfig: enable LTO and inc size
> >   configs/lsxhl_defconfig: LTO + size
> >   configs/am335x_evm_defconfig: inc SPL size
> >   configs/bk4r1_defconfig: inc size
> >   configs/linkit-smart-7688_defconfig: increse size
> >   configs/gardena-smart-gateway-mt7688_defconfig: increase size
> >   configs/rcar3_ulcb_defconfig: increase size
> >   configs/qemu-x86_64_defconfig: increase ROM size
> >   Makefile: add dtbs to clean
> >   .azure-pipelines: init submodules
> >   mach-mtmips: inc SPL size limit
> >   configs/linkit-smart-7688_defconfig: increase board limit
> >   .gitlab-ci.yml: change ownership of the git files
> >
> >  .azure-pipelines.yml                          |  10 +-
> >  .gitlab-ci.yml                                |  15 +-
> >  .gitmodules                                   |   4 +
> >  Makefile                                      |  12 +-
> >  arch/arm/include/asm/arch-omap3/mem.h         |   2 +-
> >  arch/arm/mach-omap2/omap3/board.c             |   2 +-
> >  arch/arm/mach-omap2/omap3/emif4.c             |   4 +-
> >  arch/arm/mach-omap2/omap3/sdrc.c              |   6 +-
> >  .../mach-socfpga/include/mach/handoff_soc64.h |   6 -
> >  arch/arm/mach-socfpga/wrap_handoff_soc64.c    |   9 +
> >  arch/mips/mach-mtmips/Kconfig                 |   2 +-
> >  arch/sandbox/cpu/eth-raw-os.c                 |   2 +-
> >  boot/bootmeth_efi.c                           |  18 +-
> >  boot/bootmeth_pxe.c                           |  21 +-
> >  cmd/Makefile                                  |   1 +
> >  cmd/net-lwip.c                                | 307 ++++++++++++++++
> >  cmd/net.c                                     |  86 +----
> >  cmd/pxe.c                                     |  19 +-
> >  configs/am335x_boneblack_vboot_defconfig      |   3 +
> >  configs/am335x_evm_defconfig                  |   1 +
> >  configs/bk4r1_defconfig                       |   2 +-
> >  .../gardena-smart-gateway-mt7688_defconfig    |   2 +-
> >  configs/linkit-smart-7688_defconfig           |   2 +-
> >  configs/lschlv2_defconfig                     |   3 +-
> >  configs/lsxhl_defconfig                       |   3 +-
> >  configs/qemu-x86_64_defconfig                 |   5 +-
> >  configs/rcar3_salvator-x_defconfig            |   3 +-
> >  configs/rcar3_ulcb_defconfig                  |   1 +
> >  configs/sheevaplug_defconfig                  |   3 +-
> >  configs/socfpga_secu1_defconfig               |   1 +
> >  configs/tbs2910_defconfig                     |   2 +-
> >  configs/turris_omnia_defconfig                |   3 +-
> >  doc/develop/index.rst                         |   1 +
> >  doc/develop/net_lwip.rst                      |  75 ++++
> >  drivers/net/rtl8139.c                         |   1 -
> >  drivers/net/sandbox.c                         |   5 +
> >  drivers/net/smc911x.c                         |   2 +-
> >  include/configs/bcm_ns3.h                     |   6 +-
> >  include/net.h                                 | 197 +----------
> >  include/net/arp.h                             |   9 +
> >  include/net/eth.h                             | 194 +++++++++++
> >  include/net/lwip.h                            |  73 ++++
> >  include/net/ulwip.h                           |  67 ++++
> >  lib/Kconfig                                   |   2 +-
> >  net/Kconfig                                   |   3 +
> >  net/Makefile                                  |   1 +
> >  net/eth-uclass.c                              |  37 +-
> >  net/lwip/.gitignore                           |   8 +
> >  net/lwip/Kconfig                              |  34 ++
> >  net/lwip/Makefile                             |  70 ++++
> >  net/lwip/apps/dhcp/lwip-dhcp.c                |  86 +++++
> >  net/lwip/apps/dns/lwip-dns.c                  |  63 ++++
> >  net/lwip/apps/http/Makefile                   |   6 +
> >  net/lwip/apps/http/lwip-wget.c                | 105 ++++++
> >  net/lwip/apps/ping/Makefile                   |  12 +
> >  net/lwip/apps/ping/lwip_ping.c                |  39 +++
> >  net/lwip/apps/ping/lwip_ping.h                |  15 +
> >  net/lwip/apps/ping/ping.h                     |  28 ++
> >  net/lwip/apps/tftp/Makefile                   |   7 +
> >  net/lwip/apps/tftp/lwip-tftp.c                | 132 +++++++
> >  net/lwip/lwip-external                        |   1 +
> >  net/lwip/lwipopts.h                           | 178 ++++++++++
> >  net/lwip/port/if.c                            | 327 ++++++++++++++++++
> >  net/lwip/port/include/arch/cc.h               |  44 +++
> >  net/lwip/port/include/arch/sys_arch.h         |  10 +
> >  net/lwip/port/include/limits.h                |   0
> >  net/lwip/port/sys-arch.c                      |  13 +
> >  net/net.c                                     |  44 ++-
> >  test/py/tests/test_efi_loader.py              |   4 +-
> >  test/py/tests/test_net.py                     |   2 +
> >  70 files changed, 2113 insertions(+), 348 deletions(-)
> >  create mode 100644 .gitmodules
> >  create mode 100644 cmd/net-lwip.c
> >  create mode 100644 doc/develop/net_lwip.rst
> >  create mode 100644 include/net/arp.h
> >  create mode 100644 include/net/eth.h
> >  create mode 100644 include/net/lwip.h
> >  create mode 100644 include/net/ulwip.h
> >  create mode 100644 net/lwip/.gitignore
> >  create mode 100644 net/lwip/Kconfig
> >  create mode 100644 net/lwip/Makefile
> >  create mode 100644 net/lwip/apps/dhcp/lwip-dhcp.c
> >  create mode 100644 net/lwip/apps/dns/lwip-dns.c
> >  create mode 100644 net/lwip/apps/http/Makefile
> >  create mode 100644 net/lwip/apps/http/lwip-wget.c
> >  create mode 100644 net/lwip/apps/ping/Makefile
> >  create mode 100644 net/lwip/apps/ping/lwip_ping.c
> >  create mode 100644 net/lwip/apps/ping/lwip_ping.h
> >  create mode 100644 net/lwip/apps/ping/ping.h
> >  create mode 100644 net/lwip/apps/tftp/Makefile
> >  create mode 100644 net/lwip/apps/tftp/lwip-tftp.c
> >  create mode 160000 net/lwip/lwip-external
> >  create mode 100644 net/lwip/lwipopts.h
> >  create mode 100644 net/lwip/port/if.c
> >  create mode 100644 net/lwip/port/include/arch/cc.h
> >  create mode 100644 net/lwip/port/include/arch/sys_arch.h
> >  create mode 100644 net/lwip/port/include/limits.h
> >  create mode 100644 net/lwip/port/sys-arch.c
> >
> > --
> > 2.30.2
> >
>
Fabio Estevam Jan. 3, 2024, 2:30 p.m. UTC | #17
Hi Maxim,

On Tue, Jan 2, 2024 at 2:31 PM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Small update here. I made changes without any board limit size changes.
> Reordered patches to make them compile in sequence. I think I will do one
> more clean up and more testing,
> and after that will be able to send an updated version.
> Working tree is here: https://github.com/u-boot/u-boot/pull/434

In which boards have you tested the series?

I built from https://github.com/muvarov/u-boot/tree/master_lwip_test_v10, but
it does not work reliably on an imx8mn-evk board. (Actualy tftp worked
only once):

u-boot=> ping 192.168.0.16
init already done for ethernet@30be0000
Using ethernet@30be0000 device
pinging addr: 192.168.0.16
host 192.168.0.16 is alive
 3 ms
u-boot=> dhcp Image
init already done for ethernet@30be0000
dhcp using: 0:ethernet@30be0000
dhcp_tmo 20/20
dhcp_tmo 19/20
dhcp_tmo 18/20
dhcp_tmo 17/20
dhcp_tmo 16/20
dhcp_tmo 15/20
dhcp_tmo 14/20
dhcp_tmo 13/20
dhcp_tmo 12/20
dhcp_tmo 11/20
dhcp_tmo 10/20
dhcp_tmo 9/20
dhcp_tmo 8/20
dhcp_tmo 7/20
dhcp_tmo 6/20
dhcp_tmo 5/20
dhcp_tmo 4/20
dhcp_tmo 3/20
dhcp_tmo 2/20
dhcp_tmo 1/20
dhcp_tmo 0/20
DHCP client timeout
u-boot=> tftp Image
init already done for ethernet@30be0000
TFTP from server 192.168.0.16; our IP address is <NULL>
Filename 'Image'.
Load address: 0x42000000
Loading:########################################################
done
Bytes transferred = 19808768 (0x12e4200 hex)
u-boot=> tftp Image
init already done for ethernet@30be0000
TFTP from server 192.168.0.16; our IP address is <NULL>
Filename 'Image'.
Load address: 0x42000000
Loading:eth_send error -22
tftp_get err=-13
u-boot=> tftp Image
init already done for ethernet@30be0000
TFTP from server 192.168.0.16; our IP address is <NULL>
Filename 'Image'.
Load address: 0x42000000
Loading:eth_send error -22
tftp_get err=-13
u-boot=> tftp Image
init already done for ethernet@30be0000
TFTP from server 192.168.0.16; our IP address is <NULL>
Filename 'Image'.
Load address: 0x42000000
Loading:eth_send error -22
tftp_get err=-13

Rebooted the board and tried again:

u-boot=> ping 192.168.0.16
eth0: ethernet@30be0000 00:04:9f:06:33:d3 active
Using ethernet@30be0000 device
pinging addr: 192.168.0.16
ping_tmo: ping failed; host 192.168.0.16 is not alive
u-boot=>
Maxim Uvarov Jan. 4, 2024, 3:02 p.m. UTC | #18
On Wed, 3 Jan 2024 at 20:30, Fabio Estevam <festevam@gmail.com> wrote:

> Hi Maxim,
>
> On Tue, Jan 2, 2024 at 2:31 PM Maxim Uvarov <maxim.uvarov@linaro.org>
> wrote:
> >
> > Small update here. I made changes without any board limit size changes.
> > Reordered patches to make them compile in sequence. I think I will do one
> > more clean up and more testing,
> > and after that will be able to send an updated version.
> > Working tree is here: https://github.com/u-boot/u-boot/pull/434
>
> In which boards have you tested the series?
>
>
CI tests this in all qemu emulated boards. I tested it locally on Raspberry
Pi - works well. This version did not
yet tested on imx8, but the previous version worked on eth0, did not work
on eth1.  So I plan to continue testing
these patches on different boards. But in general it's the upper layer then
drivers and theoretically has to work on any
board. But in fact different boards work in different ways and behaviour is
different with their qemu model. So I'm a little bit confused
how to test and debug these patches on board which I don't have physical
access...


> I built from https://github.com/muvarov/u-boot/tree/master_lwip_test_v10,
> but
> it does not work reliably on an imx8mn-evk board. (Actualy tftp worked
> only once):
>
> u-boot=> ping 192.168.0.16
> init already done for ethernet@30be0000
> Using ethernet@30be0000 device
> pinging addr: 192.168.0.16
> host 192.168.0.16 is alive
>  3 ms
>

It looks like it worked here.


> u-boot=> dhcp Image
>

and this command did not get a dhcp response and unset IP address.
Because there is no IP address then all tftp commands fail. The reasons
might be different:
- no MAC address
- device is not active
- remote DHCP server does not answer for this request

init already done for ethernet@30be0000
> dhcp using: 0:ethernet@30be0000
> dhcp_tmo 20/20
> dhcp_tmo 19/20
> dhcp_tmo 18/20
> dhcp_tmo 17/20
> dhcp_tmo 16/20
> dhcp_tmo 15/20
> dhcp_tmo 14/20
> dhcp_tmo 13/20
> dhcp_tmo 12/20
> dhcp_tmo 11/20
> dhcp_tmo 10/20
> dhcp_tmo 9/20
> dhcp_tmo 8/20
> dhcp_tmo 7/20
> dhcp_tmo 6/20
> dhcp_tmo 5/20
> dhcp_tmo 4/20
> dhcp_tmo 3/20
> dhcp_tmo 2/20
> dhcp_tmo 1/20
> dhcp_tmo 0/20
> DHCP client timeout
> u-boot=> tftp Image
> init already done for ethernet@30be0000
> TFTP from server 192.168.0.16; our IP address is <NULL>
> Filename 'Image'.
> Load address: 0x42000000
> Loading:########################################################
> done
> Bytes transferred = 19808768 (0x12e4200 hex)
> u-boot=> tftp Image
> init already done for ethernet@30be0000
> TFTP from server 192.168.0.16; our IP address is <NULL>
> Filename 'Image'.
> Load address: 0x42000000
> Loading:eth_send error -22
>

ok. eth_send() is a function which actually sends a packet. It checks if
the device is active or not.
        if (!eth_is_active(current))
                return -EINVAL;
I think eth_init() or something else is missing this specific board. But I
don't have any idea how to fix it
without physical access to this board or a way to reproduce it on some
other board.
'ethact' - is a variable to choose an active net device.


> tftp_get err=-13
> u-boot=> tftp Image
> init already done for ethernet@30be0000
> TFTP from server 192.168.0.16; our IP address is <NULL>
> Filename 'Image'.
> Load address: 0x42000000
> Loading:eth_send error -22
> tftp_get err=-13
> u-boot=> tftp Image
> init already done for ethernet@30be0000
> TFTP from server 192.168.0.16; our IP address is <NULL>
> Filename 'Image'.
> Load address: 0x42000000
> Loading:eth_send error -22
> tftp_get err=-13
>
> Rebooted the board and tried again:
>
> u-boot=> ping 192.168.0.16
>

that is really strange, after reboot you should get the initial stage.
You can type 'lwip list' and 'net list' to check if there is an IP address
for
active device set.

eth0: ethernet@30be0000 00:04:9f:06:33:d3 active
> Using ethernet@30be0000 device
> pinging addr: 192.168.0.16
> ping_tmo: ping failed; host 192.168.0.16 is not alive
> u-boot=>
>
Fabio Estevam Jan. 4, 2024, 8:45 p.m. UTC | #19
On Thu, Jan 4, 2024 at 12:02 PM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:

> CI tests this in all qemu emulated boards. I tested it locally on Raspberry Pi - works well. This version did not
> yet tested on imx8, but the previous version worked on eth0, did not work on eth1.  So I plan to continue testing
> these patches on different boards. But in general it's the upper layer then drivers and theoretically has to work on any
> board. But in fact different boards work in different ways and behaviour is different with their qemu model. So I'm a little bit confused
> how to test and debug these patches on board which I don't have physical access...

Can you try to get access to any i.MX board via Linaro?

> that is really strange, after reboot you should get the initial stage.
> You can type 'lwip list' and 'net list' to check if there is an IP address for
> active device set.

Nothing strange there:

u-boot=> lwip list
1:  eth0 UP IP: 192.168.0.122/255.255.255.0, GW: 192.168.0.1, 00:04:9f:06:33:d3
done.
u-boot=> net list
eth0 : ethernet@30be0000 00:04:9f:06:12:98 active
Maxim Uvarov Jan. 8, 2024, 8:41 a.m. UTC | #20
On Fri, 5 Jan 2024 at 02:46, Fabio Estevam <festevam@gmail.com> wrote:

> On Thu, Jan 4, 2024 at 12:02 PM Maxim Uvarov <maxim.uvarov@linaro.org>
> wrote:
>
> > CI tests this in all qemu emulated boards. I tested it locally on
> Raspberry Pi - works well. This version did not
> > yet tested on imx8, but the previous version worked on eth0, did not
> work on eth1.  So I plan to continue testing
> > these patches on different boards. But in general it's the upper layer
> then drivers and theoretically has to work on any
> > board. But in fact different boards work in different ways and behaviour
> is different with their qemu model. So I'm a little bit confused
> > how to test and debug these patches on board which I don't have physical
> access...
>
> Can you try to get access to any i.MX board via Linaro?
>


I will try, but for some reason there is no i.MX board in the lab.


>
> > that is really strange, after reboot you should get the initial stage.
> > You can type 'lwip list' and 'net list' to check if there is an IP
> address for
> > active device set.
>
> Nothing strange there:
>
> u-boot=> lwip list
> 1:  eth0 UP IP: 192.168.0.122/255.255.255.0, GW: 192.168.0.1,
> 00:04:9f:06:33:d3
> done.
> u-boot=> net list
> eth0 : ethernet@30be0000 00:04:9f:06:12:98 active
>

MAC address is different in two commands, it has to be the same.

BR,
Maxim.
Fabio Estevam Jan. 8, 2024, 11:39 a.m. UTC | #21
On Mon, Jan 8, 2024 at 5:43 AM Maxim Uvarov <maxim.uvarov@linaro.org> wrote:

> I will try, but for some reason there is no i.MX board in the lab.

Yes, please. Try to get it.

> MAC address is different in two commands, it has to be the same.

They are the same actually:

u-boot=> net list
eth0 : ethernet@30be0000 00:04:9f:06:33:d3 active
u-boot=> lwip list
1:  eth0 UP IP: 192.168.0.122/255.255.255.0, GW: 0.0.0.0, 00:04:9f:06:33:d3
done.
u-boot=>