| Message ID | 20250325121624.523258-1-guoren@kernel.org |
|---|---|
| Headers | show |
| Series | rv64ilp32_abi: Build CONFIG_64BIT kernel-self with ILP32 ABI | expand |
On Tue, Mar 25, 2025 at 9:18 PM Arnd Bergmann <arnd@arndb.de> wrote: > > On Tue, Mar 25, 2025, at 13:26, Peter Zijlstra wrote: > > On Tue, Mar 25, 2025 at 08:15:41AM -0400, guoren@kernel.org wrote: > >> From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org> > >> > >> Since 2001, the CONFIG_64BIT kernel has been built with the LP64 ABI, > >> but this patchset allows the CONFIG_64BIT kernel to use an ILP32 ABI > > > > Please, don't do this. This adds a significant maintenance burden on all > > of us. > > It would be easier to this with CONFIG_64BIT disabled and continue > treating CONFIG_64BIT to be the same as BITS_PER_LONG=64, but I still > think it's fundamentally a bad idea to support this in mainline > kernels in any variation, other than supporting regular 32-bit > compat mode tasks on a regular 64-bit kernel. > > >> The patchset targets RISC-V and is built on the RV64ILP32 ABI, which > >> was introduced into RISC-V's psABI in January 2025 [1]. This patchset > >> equips an rv64ilp32-abi kernel with all the functionalities of a > >> traditional lp64-abi kernel, yet restricts the address space to 2GiB. > >> Hence, the rv64ilp32-abi kernel simultaneously supports lp64-abi > >> userspace and ilp32-abi (compat) userspace, the same as the > >> traditional lp64-abi kernel. > > You declare the syscall ABI to be the native 64-bit ABI, but this > is fundamentally not true because a many uapi structures are > defined in terms of 'long' or pointer values, in particular in > the ioctl call. I modified uapi with void __user *msg_name; -> union {void __user *msg_name; u64 __msg_name;}; to make native 64-bit ABI. I would look at compat stuff instead of using __riscv_xlen macro. > This might work for an rv64ilp32 userspace that > uses the same headers and the same types, but you explicitly > say that the goal is to run native rv64 or compat rv32 tasks, > not rv64ilp32 (thanks!). It's not for rv64ilp32-abi userspace, no rv64ilp32-abi userspace introduced in the patch set. It's for native lp64-abi. Let's discuss this in the first patch thread: uapi: Reuse lp64 ABI interface > > As far as I can tell, there is no way to rectify this design flaw > other than to drop support for 64-bit userspace and only support > regular rv32 userspace. I'm also skeptical that supporting rv64 > userspace helps in practice other than for testing, since > generally most memory overhead is in userspace rather than the > kernel, and there is much more to gain from shrinking the larger > userspace by running rv32 compat mode binaries on a 64-bit kernel > than the other way round. The lp64-abi userspace rootfs works fine in this patch set, which proves the technique is valid. But the modification on uapi is raw, and I'm looking at compat stuff. Supporting lp64-abi userspace is essential because riscv lp64-abi and ilp32-abi userspace are hybrid deployments when the target is ilp32-abi userspace. The lp64-abi provides a good supplement to ilp32-abi which eases the development. > > If you remove the CONFIG_64BIT changes that Peter mentioned and > the support for ilp64 userland from your series, you end up > with a kernel that is very similar to a native rv32 kernel > but executes as rv64ilp32 and runs rv32 userspace. I don't have > any objections to that approach, and the same thing has come > up on arm64 as a possible idea as well, but I don't know if > that actually brings any notable advantage over an rv32 kernel. > > Are there CPUs that can run rv64 kernels and rv32 userspace > but not rv32 kernels, similar to what we have on Arm Cortex-A76 > and Cortex-A510? Yes, there is, and it only supports rv32 userspace, not rv32 kernel. https://www.xrvm.com/product/xuantie/C908 Here are the products: https://developer.canaan-creative.com/k230_canmv/en/dev/userguide/boards/canmv_k230d.html http://riscv.org/ecosystem-news/2024/07/unpacking-the-canmv-k230-risc-v-board/
On Wed, Mar 26, 2025 at 2:56 PM Arnd Bergmann <arnd@arndb.de> wrote: > > On Wed, Mar 26, 2025, at 07:07, Guo Ren wrote: > > On Tue, Mar 25, 2025 at 9:18 PM Arnd Bergmann <arnd@arndb.de> wrote: > >> On Tue, Mar 25, 2025, at 13:26, Peter Zijlstra wrote: > >> > On Tue, Mar 25, 2025 at 08:15:41AM -0400, guoren@kernel.org wrote: > >> > >> You declare the syscall ABI to be the native 64-bit ABI, but this > >> is fundamentally not true because a many uapi structures are > >> defined in terms of 'long' or pointer values, in particular in > >> the ioctl call. > > > > I modified uapi with > > void __user *msg_name; > > -> > > union {void __user *msg_name; u64 __msg_name;}; > > to make native 64-bit ABI. > > > > I would look at compat stuff instead of using __riscv_xlen macro. > > The problem I see here is that there are many more drivers > that you did not modify than drivers that you did change this > way. The union is particularly ugly, but even if you find > a nicer method of doing this, you now also put the burden > on future driver writers to do this right for your platform. Got it. > > >> As far as I can tell, there is no way to rectify this design flaw > >> other than to drop support for 64-bit userspace and only support > >> regular rv32 userspace. I'm also skeptical that supporting rv64 > >> userspace helps in practice other than for testing, since > >> generally most memory overhead is in userspace rather than the > >> kernel, and there is much more to gain from shrinking the larger > >> userspace by running rv32 compat mode binaries on a 64-bit kernel > >> than the other way round. > > > > The lp64-abi userspace rootfs works fine in this patch set, which > > proves the technique is valid. But the modification on uapi is raw, > > and I'm looking at compat stuff. > > There is a big difference between making it work for a particular > set of userspace binaries and making it correct for the entire > kernel ABI. > > I agree that limiting the hacks to the compat side while keeping > the native ABI as ilp32 as in your previous versions is better, > but I also don't think this can be easily done without major > changes to how compat mode works in general, and that still > seems like a show-stopper for two reasons: > > - it still puts the burden on driver writers to get it right > for your platform. The scope is a bit smaller than in the > current version because that would be limited to the compat > handlers and not change the native codepath, but that's > still a lot of drivers. > > - the way that I would imagine this to be implemented in > practice would require changing the compat code in a way that > allows multiple compat ABIs, so drivers can separate the > normal 32-on-64 handling from the 64-on-32 version you need. > We have discussed something like this in the past, but Linus > has already made it very clear that he doesn't want it done > that way. Whichever way you do it, this is unlikely to > find consensus. Got it, thanks for analysing. > > > Supporting lp64-abi userspace is essential because riscv lp64-abi and > > ilp32-abi userspace are hybrid deployments when the target is > > ilp32-abi userspace. The lp64-abi provides a good supplement to > > ilp32-abi which eases the development. > > I'm not following here, please clarify. I do understand that > having a mixed 32/64 userspace can help for development, but > that can already be done on a 64-bit kernel and it doesn't > seem to be useful for deployment because having two sets of > support libraries makes this counterproductive for the goal > of saving RAM. In my case, most binaries and libraries are based on 32-bit, but a small part would remain on 64-bit, which may be statically linked. For RISC-V, the rv64 ecosystem is more complete than the rv32's. So, rv64-abi is always necessary, and rv32-abi is a supplement. > > >> If you remove the CONFIG_64BIT changes that Peter mentioned and > >> the support for ilp64 userland from your series, you end up > >> with a kernel that is very similar to a native rv32 kernel > >> but executes as rv64ilp32 and runs rv32 userspace. I don't have > >> any objections to that approach, and the same thing has come > >> up on arm64 as a possible idea as well, but I don't know if > >> that actually brings any notable advantage over an rv32 kernel. > >> > >> Are there CPUs that can run rv64 kernels and rv32 userspace > >> but not rv32 kernels, similar to what we have on Arm Cortex-A76 > >> and Cortex-A510? > > > > Yes, there is, and it only supports rv32 userspace, not rv32 kernel. > > https://www.xrvm.com/product/xuantie/C908 > > Ok, thanks for the link. > > Arnd >
On Tue, 25 Mar 2025 08:15:41 -0400 guoren@kernel.org wrote: > From: "Guo Ren (Alibaba DAMO Academy)" <guoren@kernel.org> > > Since 2001, the CONFIG_64BIT kernel has been built with the LP64 ABI, > but this patchset allows the CONFIG_64BIT kernel to use an ILP32 ABI > for construction to reduce cache & memory footprint (Compared to > kernel-lp64-abi, kernel-rv64ilp32-abi decreased the used memory by > about 20%, as shown in "free -h" in the following demo.) ... Why on earth would you want to run a 64bit application on a 32bit kernel. IIRC the main justification for 64bit was to get a larger address space. Now you might want to compile a 32bit (ILP32) system that actually runs in 64bit mode (c/f x32) so that 64bit maths (long long) is more efficient - but that is a different issue. (I suspect you'd need to change the process switch code to save all 64bits of the registers - but maybe not much else??) David