| Message ID | 20180625130814.29771-1-julien.grall@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | [Xen-devel,RFC] tools/libxl: Switch Arm guest type to PVH | expand |
On Mon, Jun 25, 2018 at 02:08:14PM +0100, Julien Grall wrote: > Currently, the toolstack is considering Arm guest always PV. However, > they are very similar to PVH because HW virtualization extension are used > and QEMU is not started. So switch Arm guest type to PVH. > > Furthermore, the default type for Arm in xl will now be PVH to allow > smooth transition for user. > > Signed-off-by: Julien Grall <julien.grall@arm.com> > > --- > > This was discussed at Xen Summit and also in various thread on > xen-devel. The latest one was when Andrew sent a patch to deny guest creation > on Arm with XEN_DOMCTL_CDF_hap unset. > > I suspect we first implemented Arm guest as PV in libxl because PVH was > non-existent and the type was easier to avoid spawning QEMU. Note that > Linux and Xen are already considering Arm guest as PVH. > > This patch is denying the creation of PV guest on Arm. This will have an > impact on any toolstack selecting PV for Arm (e.g libvirt) or anyone > using 'type="pv"' in libxl configuration file. > > I am not entirely sure whether this is the best solution, any opinions? > --- > docs/man/xl.cfg.pod.5.in | 3 ++- > tools/libxl/libxl_arm.c | 10 +++++++++- > tools/xl/xl_parse.c | 4 ++++ > 3 files changed, 15 insertions(+), 2 deletions(-) > > diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in > index 47d88243b1..2c2af01515 100644 > --- a/docs/man/xl.cfg.pod.5.in > +++ b/docs/man/xl.cfg.pod.5.in > @@ -86,7 +86,8 @@ guest operating systems. This is the default. > > Specifies that this is to be an PVH domain. That is a lightweight HVM-like > guest without a device model and without many of the emulated devices > -available to HVM guests. Note that this mode requires a PVH aware kernel. > +available to HVM guests. Note that this mode requires a PVH aware kernel on > +x86. Have we ever advertised on ARM which guest type users should set? Are they aware of the fact that it is PV in toolstack and HVM in Xen? If users are oblivious to the underlying implementation, I think we can skip telling them PVH in ARM world. Wei.
Hi Wei, On 25/06/18 14:19, Wei Liu wrote: > On Mon, Jun 25, 2018 at 02:08:14PM +0100, Julien Grall wrote: >> Currently, the toolstack is considering Arm guest always PV. However, >> they are very similar to PVH because HW virtualization extension are used >> and QEMU is not started. So switch Arm guest type to PVH. >> >> Furthermore, the default type for Arm in xl will now be PVH to allow >> smooth transition for user. >> >> Signed-off-by: Julien Grall <julien.grall@arm.com> >> >> --- >> >> This was discussed at Xen Summit and also in various thread on >> xen-devel. The latest one was when Andrew sent a patch to deny guest creation >> on Arm with XEN_DOMCTL_CDF_hap unset. >> >> I suspect we first implemented Arm guest as PV in libxl because PVH was >> non-existent and the type was easier to avoid spawning QEMU. Note that >> Linux and Xen are already considering Arm guest as PVH. >> >> This patch is denying the creation of PV guest on Arm. This will have an >> impact on any toolstack selecting PV for Arm (e.g libvirt) or anyone >> using 'type="pv"' in libxl configuration file. >> >> I am not entirely sure whether this is the best solution, any opinions? >> --- >> docs/man/xl.cfg.pod.5.in | 3 ++- >> tools/libxl/libxl_arm.c | 10 +++++++++- >> tools/xl/xl_parse.c | 4 ++++ >> 3 files changed, 15 insertions(+), 2 deletions(-) >> >> diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in >> index 47d88243b1..2c2af01515 100644 >> --- a/docs/man/xl.cfg.pod.5.in >> +++ b/docs/man/xl.cfg.pod.5.in >> @@ -86,7 +86,8 @@ guest operating systems. This is the default. >> >> Specifies that this is to be an PVH domain. That is a lightweight HVM-like >> guest without a device model and without many of the emulated devices >> -available to HVM guests. Note that this mode requires a PVH aware kernel. >> +available to HVM guests. Note that this mode requires a PVH aware kernel on >> +x86. > > Have we ever advertised on ARM which guest type users should set? Are > they aware of the fact that it is PV in toolstack and HVM in Xen? We always advertised "ARM guest" and not a specific type of guest. So most of the users of xl will likely not specified the type of guest. > > If users are oblivious to the underlying implementation, I think we can > skip telling them PVH in ARM world. PVH is quite confusing for Arm. We only used the term to make a comparison with x86. This is the closest to Arm guest but I am not too sure whether they would be the exactly the same. At the moment, I would rather avoid to say 'PVH' for the end user and just let the different toolstack to set the type to PVH. Another solution is for libxl to force the guest type to PVH on Arm. I am not sure what would be the impact here. Cheers,
On Mon, Jun 25, 2018 at 05:18:31PM +0100, Julien Grall wrote: > Hi Wei, > > On 25/06/18 14:19, Wei Liu wrote: > > On Mon, Jun 25, 2018 at 02:08:14PM +0100, Julien Grall wrote: > > > Currently, the toolstack is considering Arm guest always PV. However, > > > they are very similar to PVH because HW virtualization extension are used > > > and QEMU is not started. So switch Arm guest type to PVH. > > > > > > Furthermore, the default type for Arm in xl will now be PVH to allow > > > smooth transition for user. > > > > > > Signed-off-by: Julien Grall <julien.grall@arm.com> > > > > > > --- > > > > > > This was discussed at Xen Summit and also in various thread on > > > xen-devel. The latest one was when Andrew sent a patch to deny guest creation > > > on Arm with XEN_DOMCTL_CDF_hap unset. > > > > > > I suspect we first implemented Arm guest as PV in libxl because PVH was > > > non-existent and the type was easier to avoid spawning QEMU. Note that > > > Linux and Xen are already considering Arm guest as PVH. > > > > > > This patch is denying the creation of PV guest on Arm. This will have an > > > impact on any toolstack selecting PV for Arm (e.g libvirt) or anyone > > > using 'type="pv"' in libxl configuration file. > > > > > > I am not entirely sure whether this is the best solution, any opinions? > > > --- > > > docs/man/xl.cfg.pod.5.in | 3 ++- > > > tools/libxl/libxl_arm.c | 10 +++++++++- > > > tools/xl/xl_parse.c | 4 ++++ > > > 3 files changed, 15 insertions(+), 2 deletions(-) > > > > > > diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in > > > index 47d88243b1..2c2af01515 100644 > > > --- a/docs/man/xl.cfg.pod.5.in > > > +++ b/docs/man/xl.cfg.pod.5.in > > > @@ -86,7 +86,8 @@ guest operating systems. This is the default. > > > Specifies that this is to be an PVH domain. That is a lightweight HVM-like > > > guest without a device model and without many of the emulated devices > > > -available to HVM guests. Note that this mode requires a PVH aware kernel. > > > +available to HVM guests. Note that this mode requires a PVH aware kernel on > > > +x86. > > > > Have we ever advertised on ARM which guest type users should set? Are > > they aware of the fact that it is PV in toolstack and HVM in Xen? > > We always advertised "ARM guest" and not a specific type of guest. So most > of the users of xl will likely not specified the type of guest. > > > > > If users are oblivious to the underlying implementation, I think we can > > skip telling them PVH in ARM world. > PVH is quite confusing for Arm. We only used the term to make a comparison > with x86. This is the closest to Arm guest but I am not too sure whether > they would be the exactly the same. > > At the moment, I would rather avoid to say 'PVH' for the end user and just > let the different toolstack to set the type to PVH. > > Another solution is for libxl to force the guest type to PVH on Arm. I am > not sure what would be the impact here. IMO I would remove the 'type' option from xl.cfg (so that it's basically ignored) in the ARM case and force it internally to PVH (if that's the best route for current ARM guests). Iff ARM even gains more guest types the option can be added and the guest types can be set to meaningful ARM guest types. In any case, this requires changes to xl.cfg man page. Roger.
Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest type to PVH"):
> IMO I would remove the 'type' option from xl.cfg (so that it's
> basically ignored) in the ARM case and force it internally to PVH (if
> that's the best route for current ARM guests).
What about libvirt users ? I haven't seen what a libvirt Xen ARM
guest config looks like but we need to meak sure that existing guests
don't break.
Ian.
On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest type to PVH"): > > IMO I would remove the 'type' option from xl.cfg (so that it's > > basically ignored) in the ARM case and force it internally to PVH (if > > that's the best route for current ARM guests). > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > guest config looks like but we need to meak sure that existing guests > don't break. For livbirt (or users of libxl library) we could force the type to pvh, regardless of the value set by the client, but I guess that would make adding types later on quite complicated. Roger.
On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest type to PVH"): > > IMO I would remove the 'type' option from xl.cfg (so that it's > > basically ignored) in the ARM case and force it internally to PVH (if > > that's the best route for current ARM guests). > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > guest config looks like but we need to meak sure that existing guests > don't break. It may set type to pv. The following file is generated by the config converter. logs.test-lab.xenproject.org/osstest/logs/124566/test-armhf-armhf-libvirt-xsm/arndale-metrocentre---var-log-libvirt-libxl-debian.guest.osstest.log Wei.
On 06/26/2018 07:49 AM, Roger Pau Monné wrote: > On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: >> Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest type to PVH"): >>> IMO I would remove the 'type' option from xl.cfg (so that it's >>> basically ignored) in the ARM case and force it internally to PVH (if >>> that's the best route for current ARM guests). >> >> What about libvirt users ? I haven't seen what a libvirt Xen ARM >> guest config looks like but we need to meak sure that existing guests >> don't break. > > For livbirt (or users of libxl library) we could force the type to > pvh, regardless of the value set by the client, but I guess that would > make adding types later on quite complicated. I am fairly confident we will never have PV guest on Arm. So one solution would be to alias PV to PVH for Arm. This still give us the liberty to add more guest type in the future. Any opinions? Cheers,
On Mon, Jul 02, 2018 at 03:13:38PM +0100, Julien Grall wrote: > > > On 06/26/2018 07:49 AM, Roger Pau Monné wrote: > > On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > > > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest type to PVH"): > > > > IMO I would remove the 'type' option from xl.cfg (so that it's > > > > basically ignored) in the ARM case and force it internally to PVH (if > > > > that's the best route for current ARM guests). > > > > > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > > > guest config looks like but we need to meak sure that existing guests > > > don't break. > > > > For livbirt (or users of libxl library) we could force the type to > > pvh, regardless of the value set by the client, but I guess that would > > make adding types later on quite complicated. > > I am fairly confident we will never have PV guest on Arm. So one solution > would be to alias PV to PVH for Arm. This still give us the liberty to add > more guest type in the future. > > Any opinions? Yes, we could make PV synonym to PVH (but don't talk about it, since Xen Arm never emphasises it guest type). Whatever new guest type we're going to add can't be called PV or PVH. I don't think that would be a big problem -- if you want a different guest type, you definitely want a different name. Wei.
On Mon, 2 Jul 2018, Julien Grall wrote: > On 06/26/2018 07:49 AM, Roger Pau Monné wrote: > > On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > > > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest > > > type to PVH"): > > > > IMO I would remove the 'type' option from xl.cfg (so that it's > > > > basically ignored) in the ARM case and force it internally to PVH (if > > > > that's the best route for current ARM guests). > > > > > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > > > guest config looks like but we need to meak sure that existing guests > > > don't break. > > > > For livbirt (or users of libxl library) we could force the type to > > pvh, regardless of the value set by the client, but I guess that would > > make adding types later on quite complicated. > > I am fairly confident we will never have PV guest on Arm. So one solution > would be to alias PV to PVH for Arm. This still give us the liberty to add > more guest type in the future. > > Any opinions? Roger, what is the plan for x86? Wasn't there an idea to silently and transparently "upgrade" PV guests to PVH when possible (when hardware support is available)? If that is the case, basically we could do the same for ARM. We could have an hardware features check, that would always return true on ARM because without virtualization extensions Xen cannot even boot, then upgrade PV to PVH. On x86 the upgrade would only happen when the required features are present.
On Mon, Jul 02, 2018 at 10:18:17AM -0700, Stefano Stabellini wrote: > On Mon, 2 Jul 2018, Julien Grall wrote: > > On 06/26/2018 07:49 AM, Roger Pau Monné wrote: > > > On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > > > > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest > > > > type to PVH"): > > > > > IMO I would remove the 'type' option from xl.cfg (so that it's > > > > > basically ignored) in the ARM case and force it internally to PVH (if > > > > > that's the best route for current ARM guests). > > > > > > > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > > > > guest config looks like but we need to meak sure that existing guests > > > > don't break. > > > > > > For livbirt (or users of libxl library) we could force the type to > > > pvh, regardless of the value set by the client, but I guess that would > > > make adding types later on quite complicated. > > > > I am fairly confident we will never have PV guest on Arm. So one solution > > would be to alias PV to PVH for Arm. This still give us the liberty to add > > more guest type in the future. > > > > Any opinions? > > Roger, what is the plan for x86? Wasn't there an idea to silently and > transparently "upgrade" PV guests to PVH when possible (when hardware > support is available)? This could only be done for xl config files that don't specify a 'type' option IMO, for libvirt and other users of libxl I don't think this is feasible. > If that is the case, basically we could do the same for ARM. We could > have an hardware features check, that would always return true on ARM > because without virtualization extensions Xen cannot even boot, then > upgrade PV to PVH. On x86 the upgrade would only happen when the > required features are present. It's slightly more convoluted on x86 because apart from the hardware features you also need to parse the kernel in order to figure out if it has the PVH entry point before setting the type to PVH. Roger.
On Tue, 3 Jul 2018, Roger Pau Monné wrote: > On Mon, Jul 02, 2018 at 10:18:17AM -0700, Stefano Stabellini wrote: > > On Mon, 2 Jul 2018, Julien Grall wrote: > > > On 06/26/2018 07:49 AM, Roger Pau Monné wrote: > > > > On Mon, Jun 25, 2018 at 05:39:12PM +0100, Ian Jackson wrote: > > > > > Roger Pau Monné writes ("Re: [PATCH RFC] tools/libxl: Switch Arm guest > > > > > type to PVH"): > > > > > > IMO I would remove the 'type' option from xl.cfg (so that it's > > > > > > basically ignored) in the ARM case and force it internally to PVH (if > > > > > > that's the best route for current ARM guests). > > > > > > > > > > What about libvirt users ? I haven't seen what a libvirt Xen ARM > > > > > guest config looks like but we need to meak sure that existing guests > > > > > don't break. > > > > > > > > For livbirt (or users of libxl library) we could force the type to > > > > pvh, regardless of the value set by the client, but I guess that would > > > > make adding types later on quite complicated. > > > > > > I am fairly confident we will never have PV guest on Arm. So one solution > > > would be to alias PV to PVH for Arm. This still give us the liberty to add > > > more guest type in the future. > > > > > > Any opinions? > > > > Roger, what is the plan for x86? Wasn't there an idea to silently and > > transparently "upgrade" PV guests to PVH when possible (when hardware > > support is available)? > > This could only be done for xl config files that don't specify a > 'type' option IMO, for libvirt and other users of libxl I don't think > this is feasible. > > > If that is the case, basically we could do the same for ARM. We could > > have an hardware features check, that would always return true on ARM > > because without virtualization extensions Xen cannot even boot, then > > upgrade PV to PVH. On x86 the upgrade would only happen when the > > required features are present. > > It's slightly more convoluted on x86 because apart from the hardware > features you also need to parse the kernel in order to figure out if > it has the PVH entry point before setting the type to PVH. I see. Too bad. In that case, if we have to go with an ARM specific solution and nobody has a better suggestion, then I think it is OK to go ahead with aliasing PV to PVH on ARM as Julien mentioned.
diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in index 47d88243b1..2c2af01515 100644 --- a/docs/man/xl.cfg.pod.5.in +++ b/docs/man/xl.cfg.pod.5.in @@ -86,7 +86,8 @@ guest operating systems. This is the default. Specifies that this is to be an PVH domain. That is a lightweight HVM-like guest without a device model and without many of the emulated devices -available to HVM guests. Note that this mode requires a PVH aware kernel. +available to HVM guests. Note that this mode requires a PVH aware kernel on +x86. =item B<type="hvm"> diff --git a/tools/libxl/libxl_arm.c b/tools/libxl/libxl_arm.c index 8af9f6f992..3acac7fa93 100644 --- a/tools/libxl/libxl_arm.c +++ b/tools/libxl/libxl_arm.c @@ -978,7 +978,15 @@ int libxl__arch_domain_init_hw_description(libxl__gc *gc, int rc; uint64_t val; - assert(info->type == LIBXL_DOMAIN_TYPE_PV); + if (info->type == LIBXL_DOMAIN_TYPE_PV) { + LOG(ERROR, "PV guest not supported. Please use PVH.\n"); + return ERROR_FEATURE_REMOVED; + } + else if (info->type != LIBXL_DOMAIN_TYPE_PVH) { + LOG(ERROR, "Unsupported Arm guest type %s\n", + libxl_domain_type_to_string(info->type)); + return ERROR_FAIL; + } /* Set the value of domain param HVM_PARAM_CALLBACK_IRQ. */ val = MASK_INSR(HVM_PARAM_CALLBACK_TYPE_PPI, diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index e6c54483e0..b705f27277 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -937,7 +937,11 @@ void parse_config_data(const char *config_source, } if (c_info->type == LIBXL_DOMAIN_TYPE_INVALID) +#if defined(__arm__) || defined(__aarch64__) + c_info->type = LIBXL_DOMAIN_TYPE_PVH; +#else c_info->type = LIBXL_DOMAIN_TYPE_PV; +#endif xlu_cfg_get_defbool(config, "hap", &c_info->hap, 0);
Currently, the toolstack is considering Arm guest always PV. However, they are very similar to PVH because HW virtualization extension are used and QEMU is not started. So switch Arm guest type to PVH. Furthermore, the default type for Arm in xl will now be PVH to allow smooth transition for user. Signed-off-by: Julien Grall <julien.grall@arm.com> --- This was discussed at Xen Summit and also in various thread on xen-devel. The latest one was when Andrew sent a patch to deny guest creation on Arm with XEN_DOMCTL_CDF_hap unset. I suspect we first implemented Arm guest as PV in libxl because PVH was non-existent and the type was easier to avoid spawning QEMU. Note that Linux and Xen are already considering Arm guest as PVH. This patch is denying the creation of PV guest on Arm. This will have an impact on any toolstack selecting PV for Arm (e.g libvirt) or anyone using 'type="pv"' in libxl configuration file. I am not entirely sure whether this is the best solution, any opinions? --- docs/man/xl.cfg.pod.5.in | 3 ++- tools/libxl/libxl_arm.c | 10 +++++++++- tools/xl/xl_parse.c | 4 ++++ 3 files changed, 15 insertions(+), 2 deletions(-)