| Message ID | 20210331200857.3274425-1-olteanv@gmail.com |
|---|---|
| Headers | show |
| Series | XDP for NXP ENETC | expand |
Vladimir Oltean <olteanv@gmail.com> writes: > On Wed, Mar 31, 2021 at 10:20:18PM +0000, patchwork-bot+netdevbpf@kernel.org wrote: >> Hello: >> >> This series was applied to netdev/net-next.git (refs/heads/master): >> >> On Wed, 31 Mar 2021 23:08:48 +0300 you wrote: >> > From: Vladimir Oltean <vladimir.oltean@nxp.com> >> > >> > This series adds support to the enetc driver for the basic XDP primitives. >> > The ENETC is a network controller found inside the NXP LS1028A SoC, >> > which is a dual-core Cortex A72 device for industrial networking, >> > with the CPUs clocked at up to 1.3 GHz. On this platform, there are 4 >> > ENETC ports and a 6-port embedded DSA switch, in a topology that looks >> > like this: >> > >> > [...] >> >> Here is the summary with links: >> - [net-next,1/9] net: enetc: consume the error RX buffer descriptors in a dedicated function >> https://git.kernel.org/netdev/net-next/c/2fa423f5f0c6 >> - [net-next,2/9] net: enetc: move skb creation into enetc_build_skb >> https://git.kernel.org/netdev/net-next/c/a800abd3ecb9 >> - [net-next,3/9] net: enetc: add a dedicated is_eof bit in the TX software BD >> https://git.kernel.org/netdev/net-next/c/d504498d2eb3 >> - [net-next,4/9] net: enetc: clean the TX software BD on the TX confirmation path >> https://git.kernel.org/netdev/net-next/c/1ee8d6f3bebb >> - [net-next,5/9] net: enetc: move up enetc_reuse_page and enetc_page_reusable >> https://git.kernel.org/netdev/net-next/c/65d0cbb414ce >> - [net-next,6/9] net: enetc: add support for XDP_DROP and XDP_PASS >> https://git.kernel.org/netdev/net-next/c/d1b15102dd16 >> - [net-next,7/9] net: enetc: add support for XDP_TX >> https://git.kernel.org/netdev/net-next/c/7ed2bc80074e >> - [net-next,8/9] net: enetc: increase RX ring default size >> https://git.kernel.org/netdev/net-next/c/d6a2829e82cf >> - [net-next,9/9] net: enetc: add support for XDP_REDIRECT >> https://git.kernel.org/netdev/net-next/c/9d2b68cc108d >> >> You are awesome, thank you! >> -- >> Deet-doot-dot, I am a bot. >> https://korg.docs.kernel.org/patchwork/pwbot.html > > Let's play a drinking game, the winner is who doesn't get drunk every > time Dave merges a 9-patch series with no review in less than two hours > after it was posted :D No thanks! I value my liver too much for that ;) (I was wondering about whether there was some black magic going on here that I had missed; glad to see it's not just me) > Now in all seriousness, I'm very much open to receiving feedback > still. Good - sent you one comment on the REDIRECT support to start with :) -Toke
From: Vladimir Oltean <vladimir.oltean@nxp.com> This series adds support to the enetc driver for the basic XDP primitives. The ENETC is a network controller found inside the NXP LS1028A SoC, which is a dual-core Cortex A72 device for industrial networking, with the CPUs clocked at up to 1.3 GHz. On this platform, there are 4 ENETC ports and a 6-port embedded DSA switch, in a topology that looks like this: +-------------------------------------------------------------------------+ | +--------+ 1 Gbps (typically disabled) | | ENETC PCI | ENETC |--------------------------+ | | Root Complex | port 3 |-----------------------+ | | | Integrated +--------+ | | | | Endpoint | | | | +--------+ 2.5 Gbps | | | | | ENETC |--------------+ | | | | | port 2 |-----------+ | | | | | +--------+ | | | | | | | | | | | | +------------------------------------------------+ | | | Felix | | Felix | | | | Switch | port 4 | | port 5 | | | | +--------+ +--------+ | | | | | +--------+ +--------+ | +--------+ +--------+ +--------+ +--------+ | | | ENETC | | ENETC | | | Felix | | Felix | | Felix | | Felix | | | | port 0 | | port 1 | | | port 0 | | port 1 | | port 2 | | port 3 | | +-------------------------------------------------------------------------+ | | | | | | v v v v v v Up to Up to Up to 4x 2.5Gbps 2.5Gbps 1Gbps The ENETC ports 2 and 3 can act as DSA masters for the embedded switch. Because 4 out of the 6 externally-facing ports of the SoC are switch ports, the most interesting use case for XDP on this device is in fact XDP_TX on the 2.5Gbps DSA master. Nonetheless, the results presented below are for IPv4 forwarding between ENETC port 0 (eno0) and port 1 (eno1) both configured for 1Gbps. There are two streams of IPv4/UDP datagrams with a frame length of 64 octets delivered at 100% port load to eno0 and to eno1. eno0 has a flow steering rule to process the traffic on RX ring 0 (CPU 0), and eno1 has a flow steering rule towards RX ring 1 (CPU 1). For the IPFWD test, standard IP routing was enabled in the netns. For the XDP_DROP test, the samples/bpf/xdp1 program was attached to both eno0 and to eno1. For the XDP_TX test, the samples/bpf/xdp2 program was attached to both eno0 and to eno1. For the XDP_REDIRECT test, the samples/bpf/xdp_redirect program was attached once to the input of eno0/output of eno1, and twice to the input of eno1/output of eno0. Finally, the preliminary results are as follows: | IPFWD | XDP_TX | XDP_REDIRECT | XDP_DROP --------+-------+--------+------------------------- fps | 761 | 2535 | 1735 | 2783 Gbps | 0.51 | 1.71 | 1.17 | n/a There is a strange phenomenon in my testing sistem where it appears that one CPU is processing more than the other. I have not investigated this too much. Also, the code might not be very well optimized (for example dma_sync_for_device is called with the full ENETC_RXB_DMA_SIZE_XDP). Design wise, the ENETC is a PCI device with BD rings, so it uses the MEM_TYPE_PAGE_SHARED memory model, as can typically be seen in Intel devices. The strategy was to build upon the existing model that the driver uses, and not change it too much. So you will see things like a separate NAPI poll function for XDP. I have only tested with PAGE_SIZE=4096, and since we split pages in half, it means that MTU-sized frames are scatter/gather (the XDP headroom + skb_shared_info only leaves us 1476 bytes of data per buffer). This is sub-optimal, but I would rather keep it this way and help speed up Lorenzo's series for S/G support through testing, rather than change the enetc driver to use some other memory model like page_pool. My code is already structured for S/G, and that works fine for XDP_DROP and XDP_TX, just not for XDP_REDIRECT, even between two enetc ports. So the S/G XDP_REDIRECT is stubbed out (the frames are dropped), but obviously I would like to remove that limitation soon. Please note that I am rather new to this kind of stuff, I am more of a control path person, so I would appreciate feedback. Enough talking, on to the patches. Vladimir Oltean (9): net: enetc: consume the error RX buffer descriptors in a dedicated function net: enetc: move skb creation into enetc_build_skb net: enetc: add a dedicated is_eof bit in the TX software BD net: enetc: clean the TX software BD on the TX confirmation path net: enetc: move up enetc_reuse_page and enetc_page_reusable net: enetc: add support for XDP_DROP and XDP_PASS net: enetc: add support for XDP_TX net: enetc: increase RX ring default size net: enetc: add support for XDP_REDIRECT drivers/net/ethernet/freescale/enetc/enetc.c | 826 +++++++++++++++--- drivers/net/ethernet/freescale/enetc/enetc.h | 53 +- .../ethernet/freescale/enetc/enetc_ethtool.c | 19 +- .../net/ethernet/freescale/enetc/enetc_pf.c | 2 + 4 files changed, 796 insertions(+), 104 deletions(-)