| Message ID | 20240502090547.87824-1-philmd@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | migration/rdma: Allow building without on-demand paging support | expand |
On Thu, May 02, 2024 at 11:05:47AM +0200, Philippe Mathieu-Daudé wrote: > On-demand paging support was added in libibverbs v1.2.0 in > commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 That is 9 years old, so I'm surprised any distro we target still is so outdated. Can you say what distro you're seeing a problem on and what version it has ? With regards, Daniel
On 2/5/24 11:10, Daniel P. Berrangé wrote: > On Thu, May 02, 2024 at 11:05:47AM +0200, Philippe Mathieu-Daudé wrote: >> On-demand paging support was added in libibverbs v1.2.0 in >> commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 > > That is 9 years old, so I'm surprised any distro we target still > is so outdated. Can you say what distro you're seeing a problem > on and what version it has ? This is Oracle Solaris 11.4 SRU, released 2 weeks ago: https://support.oracle.com/knowledge/Sun%20Microsystems/2433412_1.html I'm not sure how to detect the version, I'm seeing downstream patches applied on top.
On 2/5/24 11:19, Philippe Mathieu-Daudé wrote: > On 2/5/24 11:10, Daniel P. Berrangé wrote: >> On Thu, May 02, 2024 at 11:05:47AM +0200, Philippe Mathieu-Daudé wrote: >>> On-demand paging support was added in libibverbs v1.2.0 in >>> commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 >> >> That is 9 years old, so I'm surprised any distro we target still >> is so outdated. Can you say what distro you're seeing a problem >> on and what version it has ? > > This is Oracle Solaris 11.4 SRU, released 2 weeks ago: > https://support.oracle.com/knowledge/Sun%20Microsystems/2433412_1.html > > I'm not sure how to detect the version, I'm seeing downstream > patches applied on top. Regardless, we can't expect any rdma library version to work, either we ask for some version, or we check that symbol is there and reject if not as "your rdma is too old", so the user can choose to build with --disable-rdma.
On Thu, May 02, 2024 at 11:21:31AM +0200, Philippe Mathieu-Daudé wrote: > On 2/5/24 11:19, Philippe Mathieu-Daudé wrote: > > On 2/5/24 11:10, Daniel P. Berrangé wrote: > > > On Thu, May 02, 2024 at 11:05:47AM +0200, Philippe Mathieu-Daudé wrote: > > > > On-demand paging support was added in libibverbs v1.2.0 in > > > > commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 > > > > > > That is 9 years old, so I'm surprised any distro we target still > > > is so outdated. Can you say what distro you're seeing a problem > > > on and what version it has ? > > > > This is Oracle Solaris 11.4 SRU, released 2 weeks ago: > > https://support.oracle.com/knowledge/Sun%20Microsystems/2433412_1.html > > > > I'm not sure how to detect the version, I'm seeing downstream > > patches applied on top. > > Regardless, we can't expect any rdma library version to work, > either we ask for some version, or we check that symbol is there > and reject if not as "your rdma is too old", so the user can > choose to build with --disable-rdma. The RDMA libs appear to have pkg-config files, so I agree that we should be requesting a min version in meson that matches what our code actually requires. With regards, Daniel
On Thu, May 02, 2024 at 11:19:28AM +0200, Philippe Mathieu-Daudé wrote: > On 2/5/24 11:10, Daniel P. Berrangé wrote: > > On Thu, May 02, 2024 at 11:05:47AM +0200, Philippe Mathieu-Daudé wrote: > > > On-demand paging support was added in libibverbs v1.2.0 in > > > commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 > > > > That is 9 years old, so I'm surprised any distro we target still > > is so outdated. Can you say what distro you're seeing a problem > > on and what version it has ? > > This is Oracle Solaris 11.4 SRU, released 2 weeks ago: > https://support.oracle.com/knowledge/Sun%20Microsystems/2433412_1.html Solaris 11.4 was originally from 2018 IIUC, so that explains why it is so ancient. > I'm not sure how to detect the version, I'm seeing downstream > patches applied on top. When this was first introduced there was no pkg-config files present, nor was it using cmake, or even library versioning afaics. The pkgconfig files were added 3 years later in commit df5fe3c2fa9d7dbb38fb7b4878955330620211ed Author: Luca Boccassi <bluca@debian.org> Date: Tue Aug 28 19:46:07 2018 +0100 Generate and install pkgconfig files for libs which at that point had defined: CMakeLists.txt:set(PACKAGE_VERSION "20.0") librdmacm/CMakeLists.txt: 1 1.1.${PACKAGE_VERSION} libibverbs/CMakeLists.txt: 1 1.5.${PACKAGE_VERSION} So I think we can try checking libibverbs >= 1.5 rdmacm >= 1.1 in meson.build, and see if that solves the Solaris problem....depends what the scope of their downstream patches is. With regards, Daniel
diff --git a/meson.build b/meson.build index 837a2bdb56..7c6436ac9e 100644 --- a/meson.build +++ b/meson.build @@ -2410,6 +2410,9 @@ if rdma.found() cc.has_function('ibv_advise_mr', dependencies: rdma, prefix: '#include <infiniband/verbs.h>')) + config_host_data.set('HAVE_IBV_ACCESS_ON_DEMAND', + cc.has_header_symbol('infiniband/verbs.h', + 'IBV_ACCESS_ON_DEMAND')) endif have_asan_fiber = false diff --git a/migration/rdma.c b/migration/rdma.c index 855753c671..4717fb3143 100644 --- a/migration/rdma.c +++ b/migration/rdma.c @@ -1127,9 +1127,14 @@ static int qemu_rdma_alloc_qp(RDMAContext *rdma) return 0; } +#ifndef HAVE_IBV_ACCESS_ON_DEMAND +#define IBV_ACCESS_ON_DEMAND 0 +#endif + /* Check whether On-Demand Paging is supported by RDAM device */ static bool rdma_support_odp(struct ibv_context *dev) { +#ifdef HAVE_IBV_ACCESS_ON_DEMAND struct ibv_device_attr_ex attr = {0}; if (ibv_query_device_ex(dev, NULL, &attr)) { @@ -1139,6 +1144,7 @@ static bool rdma_support_odp(struct ibv_context *dev) if (attr.odp_caps.general_caps & IBV_ODP_SUPPORT) { return true; } +#endif return false; }
On-demand paging support was added in libibverbs v1.2.0 in commit https://github.com/linux-rdma/rdma-core/commit/e500adc7b1 We don't check the libibverbs, so add a meson check on the IBV_ACCESS_ON_DEMAND symbol, and define HAVE_IBV_ACCESS_ON_DEMAND if found. Restrict rdma_support_odp() so it returns %false when on-demand paging is not supported. This fixes: migration/rdma.c: In function 'rdma_support_odp': migration/rdma.c:1133:12: error: variable 'attr' has initializer but incomplete type 1133 | struct ibv_device_attr_ex attr = {0}; | ^~~~~~~~~~~~~~~~~~ migration/rdma.c:1135:9: warning: implicit declaration of function 'ibv_query_device_ex'; did you mean 'ibv_query_device'? [-Wimplicit-function-declaration] 1135 | if (ibv_query_device_ex(dev, NULL, &attr)) { | ^~~~~~~~~~~~~~~~~~~ | ibv_query_device migration/rdma.c:1135:9: warning: nested extern declaration of 'ibv_query_device_ex' [-Wnested-externs] migration/rdma.c:1139:38: error: 'IBV_ODP_SUPPORT' undeclared (first use in this function); did you mean 'IBV_QP_PORT'? 1139 | if (attr.odp_caps.general_caps & IBV_ODP_SUPPORT) { | ^~~~~~~~~~~~~~~ | IBV_QP_PORT migration/rdma.c: In function 'qemu_rdma_reg_whole_ram_blocks': migration/rdma.c:1189:27: error: 'IBV_ACCESS_ON_DEMAND' undeclared (first use in this function); did you mean 'IBV_ACCESS_MW_BIND'? 1189 | access |= IBV_ACCESS_ON_DEMAND; | ^~~~~~~~~~~~~~~~~~~~ | IBV_ACCESS_MW_BIND ninja: build stopped: subcommand failed. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> --- meson.build | 3 +++ migration/rdma.c | 6 ++++++ 2 files changed, 9 insertions(+)