| Message ID | 20200805091640.11134-1-drjones@redhat.com |
|---|---|
| Headers | show |
| Series | hw/arm/virt: Introduce kvm-steal-time | expand |
FYI, the new KVM cap is on its way into the kernel and Peter has submit patches to remove kvm32 from qemu. So I'll be refreshing this series to incorporate both those changes. Thanks, drew On Wed, Aug 05, 2020 at 11:16:35AM +0200, Andrew Jones wrote: > v2: > - Changed the introduction of the feature to 5.2 from 5.1 > (The 5.2 machine type patch posted by Cornelia was thrown > in for completeness, but I suppose that'll get picked up > separately.) > - Added a patch adding g_assert_not_reached() to many KVM > stubs. (This isn't exactly related to the series, but the > series does add two more stubs that can now be added in > the same way.) > - Fixed a patch that wasn't suppose to have a functional > change, but did (fdt_add_gic_node() shouldn't generate > the node without the PMU) [Peter] > - Pass sysmem to virt_cpu_post_init() [Peter] > - Introduced a define for the pvtime struct size > - Calculate the pvtime memory region size based on max-cpus > and host-page-size [Beata] > - Renamed kvm_no_steal_time to no_kvm_steal_time [Peter] > - Fixed a parameter misordering with object_property_set_bool() > - Added a comment explaining why the feature isn't supported > for AArch32 guests > - Changed a !kvm_steal_time to a kvm_steal_time == ON_OFF_AUTO_OFF > as it should be > - Picked up one r-b from Peter > > > KVM supports the ability to publish the amount of time that VCPUs > were runnable, but not running due to other host threads running > instead, to the guest. The guest scheduler may use that information > when making decisions and the guest may expose it to its userspace > (Linux publishes this information in /proc/stat). This feature is > called "steal time" as it represents the amount of time stolen from > a guest by scheduling out its VCPUs. To enable this feature KVM > userspace must provide a memory region that will be used to publish > the information to the guest. The memory region is typical migratable > region. The GPA of the region is given to KVM through a VCPU device > ioctl interface. This feature is only available for 64-bit hosts > running 64-bit guests. > > This series provides the QEMU support of this feature. It will > be enabled by default for 5.2 machine types and later, but may > be disabled with a new CPU property "kvm-steal-time". > > While testing migration it was observed that the amount of > steal time as viewed by the guest was getting reset on each > migration. Patch 4/6 of a pvtime fix series posted[*] for KVM > should fix that. Also, we may still want to change the way we > probe KVM for the feature in this QEMU series to the new KVM > cap proposed in that KVM series. > > Migration testing: > > * virt-5.1 can migrate as usual, no steal-time enabled > > * virt-5.2 can migrate between hosts with steal-time enabled > (the default) and disabled when both hosts support steal-time > > * virt-5.2 with steal-time disabled can migrate to a host that > does not support steal-time > > * virt-5.2 with steal-time enabled will cleanly fail when migrating > to a host that does not support steal-time > > * virt-5.2 without the kvm-steal-time property specified can > boot on a host that does not support the feature - the feature > will be disabled. However, if the guest is migrated to a host > that does support the feature, then after the guest has reboot > the feature will become available with the guest kernel supports > it (this is the nature of migrating guests that use '-cpu host'). > Additionally, once this guest has been migrated to a host that > does have the feature, then it will fail (cleanly) to migrate to > a host that does not have the feature. If this behavior isn't > desired, then the user should explicitly disable steal-time with > the kvm-steal-time property if they want to boot a 5.2 guest on > a host that doesn't support steal-time. > > [*] https://lists.cs.columbia.edu/pipermail/kvmarm/2020-August/041823.html > > Thanks, > drew > > > Andrew Jones (4): > target/arm/kvm: Make uncalled stubs explicitly unreachable > hw/arm/virt: Move post cpu realize check into its own function > hw/arm/virt: Move kvm pmu setup to virt_cpu_post_init > hw/arm/virt: Implement kvm-steal-time > > Cornelia Huck (1): > hw: add compat machines for 5.2 > > docs/system/arm/cpu-features.rst | 11 +++ > hw/arm/virt.c | 119 +++++++++++++++++++++++-------- > hw/core/machine.c | 3 + > hw/i386/pc.c | 6 +- > hw/i386/pc_piix.c | 14 +++- > hw/i386/pc_q35.c | 13 +++- > hw/ppc/spapr.c | 15 +++- > hw/s390x/s390-virtio-ccw.c | 14 +++- > include/hw/arm/virt.h | 5 ++ > include/hw/boards.h | 3 + > include/hw/i386/pc.h | 3 + > target/arm/cpu.c | 10 +++ > target/arm/cpu.h | 4 ++ > target/arm/kvm.c | 20 ++++++ > target/arm/kvm32.c | 5 ++ > target/arm/kvm64.c | 77 ++++++++++++++++++-- > target/arm/kvm_arm.h | 74 ++++++++++++++----- > target/arm/monitor.c | 2 +- > tests/qtest/arm-cpu-features.c | 25 +++++-- > 19 files changed, 358 insertions(+), 65 deletions(-) > > -- > 2.25.4 >
v2: - Changed the introduction of the feature to 5.2 from 5.1 (The 5.2 machine type patch posted by Cornelia was thrown in for completeness, but I suppose that'll get picked up separately.) - Added a patch adding g_assert_not_reached() to many KVM stubs. (This isn't exactly related to the series, but the series does add two more stubs that can now be added in the same way.) - Fixed a patch that wasn't suppose to have a functional change, but did (fdt_add_gic_node() shouldn't generate the node without the PMU) [Peter] - Pass sysmem to virt_cpu_post_init() [Peter] - Introduced a define for the pvtime struct size - Calculate the pvtime memory region size based on max-cpus and host-page-size [Beata] - Renamed kvm_no_steal_time to no_kvm_steal_time [Peter] - Fixed a parameter misordering with object_property_set_bool() - Added a comment explaining why the feature isn't supported for AArch32 guests - Changed a !kvm_steal_time to a kvm_steal_time == ON_OFF_AUTO_OFF as it should be - Picked up one r-b from Peter KVM supports the ability to publish the amount of time that VCPUs were runnable, but not running due to other host threads running instead, to the guest. The guest scheduler may use that information when making decisions and the guest may expose it to its userspace (Linux publishes this information in /proc/stat). This feature is called "steal time" as it represents the amount of time stolen from a guest by scheduling out its VCPUs. To enable this feature KVM userspace must provide a memory region that will be used to publish the information to the guest. The memory region is typical migratable region. The GPA of the region is given to KVM through a VCPU device ioctl interface. This feature is only available for 64-bit hosts running 64-bit guests. This series provides the QEMU support of this feature. It will be enabled by default for 5.2 machine types and later, but may be disabled with a new CPU property "kvm-steal-time". While testing migration it was observed that the amount of steal time as viewed by the guest was getting reset on each migration. Patch 4/6 of a pvtime fix series posted[*] for KVM should fix that. Also, we may still want to change the way we probe KVM for the feature in this QEMU series to the new KVM cap proposed in that KVM series. Migration testing: * virt-5.1 can migrate as usual, no steal-time enabled * virt-5.2 can migrate between hosts with steal-time enabled (the default) and disabled when both hosts support steal-time * virt-5.2 with steal-time disabled can migrate to a host that does not support steal-time * virt-5.2 with steal-time enabled will cleanly fail when migrating to a host that does not support steal-time * virt-5.2 without the kvm-steal-time property specified can boot on a host that does not support the feature - the feature will be disabled. However, if the guest is migrated to a host that does support the feature, then after the guest has reboot the feature will become available with the guest kernel supports it (this is the nature of migrating guests that use '-cpu host'). Additionally, once this guest has been migrated to a host that does have the feature, then it will fail (cleanly) to migrate to a host that does not have the feature. If this behavior isn't desired, then the user should explicitly disable steal-time with the kvm-steal-time property if they want to boot a 5.2 guest on a host that doesn't support steal-time. [*] https://lists.cs.columbia.edu/pipermail/kvmarm/2020-August/041823.html Thanks, drew Andrew Jones (4): target/arm/kvm: Make uncalled stubs explicitly unreachable hw/arm/virt: Move post cpu realize check into its own function hw/arm/virt: Move kvm pmu setup to virt_cpu_post_init hw/arm/virt: Implement kvm-steal-time Cornelia Huck (1): hw: add compat machines for 5.2 docs/system/arm/cpu-features.rst | 11 +++ hw/arm/virt.c | 119 +++++++++++++++++++++++-------- hw/core/machine.c | 3 + hw/i386/pc.c | 6 +- hw/i386/pc_piix.c | 14 +++- hw/i386/pc_q35.c | 13 +++- hw/ppc/spapr.c | 15 +++- hw/s390x/s390-virtio-ccw.c | 14 +++- include/hw/arm/virt.h | 5 ++ include/hw/boards.h | 3 + include/hw/i386/pc.h | 3 + target/arm/cpu.c | 10 +++ target/arm/cpu.h | 4 ++ target/arm/kvm.c | 20 ++++++ target/arm/kvm32.c | 5 ++ target/arm/kvm64.c | 77 ++++++++++++++++++-- target/arm/kvm_arm.h | 74 ++++++++++++++----- target/arm/monitor.c | 2 +- tests/qtest/arm-cpu-features.c | 25 +++++-- 19 files changed, 358 insertions(+), 65 deletions(-)