From patchwork Thu Dec 3 17:11:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 338017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED171C4361A for ; Thu, 3 Dec 2020 17:13:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8559320663 for ; Thu, 3 Dec 2020 17:13:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731447AbgLCRNc (ORCPT ); Thu, 3 Dec 2020 12:13:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49681 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730591AbgLCRNc (ORCPT ); Thu, 3 Dec 2020 12:13:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607015524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jmr/JFRgIc7BBDuWcf7Mzpm7NP0GuJrUZZKAzRa6tmY=; b=fMpEV63+/c6wxpttVp6Cjy4Ap9DG+zeKPRebLYa2P8P0pxo1aAL3K3D93uBocvF4jc++Mx 3SzwcTtIYP0mzWh3vfDw0lioUJznSrCyPa+qx3ASvWT/SEwKOC41wa5tT895N/QIMKFWUm 48Q22jM3nwoj38AKmX0J05CPUOw5Tbo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-345-z18ZtV5cMmaOU8BVu94eGQ-1; Thu, 03 Dec 2020 12:12:02 -0500 X-MC-Unique: z18ZtV5cMmaOU8BVu94eGQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E174E1016CB2; Thu, 3 Dec 2020 17:11:34 +0000 (UTC) Received: from localhost.localdomain (unknown [10.35.206.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0E1560854; Thu, 3 Dec 2020 17:11:27 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: "H. Peter Anvin" , Paolo Bonzini , Jonathan Corbet , Jim Mattson , Wanpeng Li , linux-kselftest@vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK), Vitaly Kuznetsov , Marcelo Tosatti , Thomas Gleixner , Sean Christopherson , linux-kernel@vger.kernel.org (open list), Ingo Molnar , x86@kernel.org (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)), Joerg Roedel , Borislav Petkov , Shuah Khan , Maxim Levitsky , Andrew Jones , Oliver Upton , linux-doc@vger.kernel.org (open list:DOCUMENTATION) Subject: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE Date: Thu, 3 Dec 2020 19:11:16 +0200 Message-Id: <20201203171118.372391-2-mlevitsk@redhat.com> In-Reply-To: <20201203171118.372391-1-mlevitsk@redhat.com> References: <20201203171118.372391-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org These two new ioctls allow to more precisly capture and restore guest's TSC state. Both ioctls are meant to be used to accurately migrate guest TSC even when there is a significant downtime during the migration. Suggested-by: Paolo Bonzini Signed-off-by: Maxim Levitsky --- Documentation/virt/kvm/api.rst | 65 ++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 73 ++++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 15 +++++++ 3 files changed, 153 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 70254eaa5229f..ebecfe4b414ce 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -4826,6 +4826,71 @@ If a vCPU is in running state while this ioctl is invoked, the vCPU may experience inconsistent filtering behavior on MSR accesses. +4.127 KVM_GET_TSC_STATE +---------------------------- + +:Capability: KVM_CAP_PRECISE_TSC +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_tsc_state +:Returns: 0 on success, < 0 on error + +:: + + #define KVM_TSC_STATE_TIMESTAMP_VALID 1 + #define KVM_TSC_STATE_TSC_ADJUST_VALID 2 + struct kvm_tsc_state { + __u32 flags; + __u64 nsec; + __u64 tsc; + __u64 tsc_adjust; + }; + +flags values for ``struct kvm_tsc_state``: + +``KVM_TSC_STATE_TIMESTAMP_VALID`` + + ``nsec`` contains nanoseconds from unix epoch. + Always set by KVM_GET_TSC_STATE, might be omitted in KVM_SET_TSC_STATE + +``KVM_TSC_STATE_TSC_ADJUST_VALID`` + + ``tsc_adjust`` contains valid IA32_TSC_ADJUST value + + +This ioctl allows the user space to read the guest's IA32_TSC,IA32_TSC_ADJUST, +and the current value of host's CLOCK_REALTIME clock in nanoseconds since unix +epoch. + + +4.128 KVM_SET_TSC_STATE +---------------------------- + +:Capability: KVM_CAP_PRECISE_TSC +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_tsc_state +:Returns: 0 on success, < 0 on error + +:: + +This ioctl allows to reconstruct the guest's IA32_TSC and TSC_ADJUST value +from the state obtained in the past by KVM_GET_TSC_STATE on the same vCPU. + +If 'KVM_TSC_STATE_TIMESTAMP_VALID' is set in flags, +KVM will adjust the guest TSC value by the time that passed since the moment +CLOCK_REALTIME timestamp was saved in the struct and current value of +CLOCK_REALTIME, and set the guest's TSC to the new value. + +Otherwise KVM will set the guest TSC value to the exact value as given +in the struct. + +if KVM_TSC_STATE_TSC_ADJUST_VALID is set, and guest supports IA32_MSR_TSC_ADJUST, +then its value will be set to the given value from the struct. + +It is assumed that either both ioctls will be run on the same machine, +or that source and destination machines have synchronized clocks. + 5. The kvm_run structure ======================== diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a3fdc16cfd6f3..9b8a2fe3a2398 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2438,6 +2438,21 @@ static bool kvm_get_walltime_and_clockread(struct timespec64 *ts, return gtod_is_based_on_tsc(do_realtime(ts, tsc_timestamp)); } + + +static void kvm_get_walltime(u64 *walltime_ns, u64 *host_tsc) +{ + struct timespec64 ts; + + if (kvm_get_walltime_and_clockread(&ts, host_tsc)) { + *walltime_ns = timespec64_to_ns(&ts); + return; + } + + *host_tsc = rdtsc(); + *walltime_ns = ktime_get_real_ns(); +} + #endif /* @@ -3757,6 +3772,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_X86_USER_SPACE_MSR: case KVM_CAP_X86_MSR_FILTER: case KVM_CAP_ENFORCE_PV_FEATURE_CPUID: +#ifdef CONFIG_X86_64 + case KVM_CAP_PRECISE_TSC: +#endif r = 1; break; case KVM_CAP_SYNC_REGS: @@ -4999,6 +5017,61 @@ long kvm_arch_vcpu_ioctl(struct file *filp, case KVM_GET_SUPPORTED_HV_CPUID: r = kvm_ioctl_get_supported_hv_cpuid(vcpu, argp); break; +#ifdef CONFIG_X86_64 + case KVM_GET_TSC_STATE: { + struct kvm_tsc_state __user *user_tsc_state = argp; + u64 host_tsc; + + struct kvm_tsc_state tsc_state = { + .flags = KVM_TSC_STATE_TIMESTAMP_VALID + }; + + kvm_get_walltime(&tsc_state.nsec, &host_tsc); + tsc_state.tsc = kvm_read_l1_tsc(vcpu, host_tsc); + + if (guest_cpuid_has(vcpu, X86_FEATURE_TSC_ADJUST)) { + tsc_state.tsc_adjust = vcpu->arch.ia32_tsc_adjust_msr; + tsc_state.flags |= KVM_TSC_STATE_TSC_ADJUST_VALID; + } + + r = -EFAULT; + if (copy_to_user(user_tsc_state, &tsc_state, sizeof(tsc_state))) + goto out; + r = 0; + break; + } + case KVM_SET_TSC_STATE: { + struct kvm_tsc_state __user *user_tsc_state = argp; + struct kvm_tsc_state tsc_state; + u64 host_tsc, wall_nsec; + + u64 new_guest_tsc, new_guest_tsc_offset; + + r = -EFAULT; + if (copy_from_user(&tsc_state, user_tsc_state, sizeof(tsc_state))) + goto out; + + kvm_get_walltime(&wall_nsec, &host_tsc); + new_guest_tsc = tsc_state.tsc; + + if (tsc_state.flags & KVM_TSC_STATE_TIMESTAMP_VALID) { + s64 diff = wall_nsec - tsc_state.nsec; + if (diff >= 0) + new_guest_tsc += nsec_to_cycles(vcpu, diff); + else + new_guest_tsc -= nsec_to_cycles(vcpu, -diff); + } + + new_guest_tsc_offset = new_guest_tsc - kvm_scale_tsc(vcpu, host_tsc); + kvm_vcpu_write_tsc_offset(vcpu, new_guest_tsc_offset); + + if (tsc_state.flags & KVM_TSC_STATE_TSC_ADJUST_VALID) + if (guest_cpuid_has(vcpu, X86_FEATURE_TSC_ADJUST)) + vcpu->arch.ia32_tsc_adjust_msr = tsc_state.tsc_adjust; + r = 0; + break; + } +#endif default: r = -EINVAL; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 886802b8ffba3..bf4c38fd58291 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1056,6 +1056,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ENFORCE_PV_FEATURE_CPUID 190 #define KVM_CAP_SYS_HYPERV_CPUID 191 #define KVM_CAP_DIRTY_LOG_RING 192 +#define KVM_CAP_PRECISE_TSC 193 #ifdef KVM_CAP_IRQ_ROUTING @@ -1169,6 +1170,16 @@ struct kvm_clock_data { __u32 pad[9]; }; + +#define KVM_TSC_STATE_TIMESTAMP_VALID 1 +#define KVM_TSC_STATE_TSC_ADJUST_VALID 2 +struct kvm_tsc_state { + __u32 flags; + __u64 nsec; + __u64 tsc; + __u64 tsc_adjust; +}; + /* For KVM_CAP_SW_TLB */ #define KVM_MMU_FSL_BOOKE_NOHV 0 @@ -1563,6 +1574,10 @@ struct kvm_pv_cmd { /* Available with KVM_CAP_DIRTY_LOG_RING */ #define KVM_RESET_DIRTY_RINGS _IO(KVMIO, 0xc7) +/* Available with KVM_CAP_PRECISE_TSC*/ +#define KVM_SET_TSC_STATE _IOW(KVMIO, 0xc8, struct kvm_tsc_state) +#define KVM_GET_TSC_STATE _IOR(KVMIO, 0xc9, struct kvm_tsc_state) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ From patchwork Thu Dec 3 17:11:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 337270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8596C0018C for ; Thu, 3 Dec 2020 17:13:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 87F78206D5 for ; Thu, 3 Dec 2020 17:13:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2501950AbgLCRNg (ORCPT ); Thu, 3 Dec 2020 12:13:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:51354 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728925AbgLCRNf (ORCPT ); Thu, 3 Dec 2020 12:13:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607015528; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6hX5d7i1onM6KJjcxjt4YgWrUzJK2cIDgrBlA7uN5Mk=; b=MDzMptkIb51QzHUWu7AbFpA5KAhqkqvqW634w553OfQFwFEE3HK7Tzzn69XFElrD0CcikO Nos/r6ApkyvlF4CGhPALJqbyri9yDAPtH1f/bQFFVnMwtB1QuCNu43yGDXLvc/C8KPmCgf sMCNourTkI1D2+kWc7Fm4K5xVQFl8tI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-270-mzLNlnliPHCl5vKZ2vvgKg-1; Thu, 03 Dec 2020 12:12:03 -0500 X-MC-Unique: mzLNlnliPHCl5vKZ2vvgKg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 053AD805EE3; Thu, 3 Dec 2020 17:11:42 +0000 (UTC) Received: from localhost.localdomain (unknown [10.35.206.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6431F6085A; Thu, 3 Dec 2020 17:11:35 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: "H. Peter Anvin" , Paolo Bonzini , Jonathan Corbet , Jim Mattson , Wanpeng Li , linux-kselftest@vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK), Vitaly Kuznetsov , Marcelo Tosatti , Thomas Gleixner , Sean Christopherson , linux-kernel@vger.kernel.org (open list), Ingo Molnar , x86@kernel.org (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)), Joerg Roedel , Borislav Petkov , Shuah Khan , Maxim Levitsky , Andrew Jones , Oliver Upton , linux-doc@vger.kernel.org (open list:DOCUMENTATION) Subject: [PATCH v2 2/3] KVM: x86: introduce KVM_X86_QUIRK_TSC_HOST_ACCESS Date: Thu, 3 Dec 2020 19:11:17 +0200 Message-Id: <20201203171118.372391-3-mlevitsk@redhat.com> In-Reply-To: <20201203171118.372391-1-mlevitsk@redhat.com> References: <20201203171118.372391-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org This quirk reflects the fact that we currently treat MSR_IA32_TSC and MSR_TSC_ADJUST access by the host (e.g qemu) in a way that is different compared to an access from the guest. For host's MSR_IA32_TSC read we currently always return L1 TSC value, and for host's write we do the tsc synchronization. For host's MSR_TSC_ADJUST write, we don't make the tsc 'jump' as we should for this msr. When the hypervisor uses the new TSC GET/SET state ioctls, all of this is no longer needed, thus leave this enabled only with a quirk which the hypervisor can disable. Suggested-by: Paolo Bonzini Signed-off-by: Maxim Levitsky --- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/x86.c | 19 ++++++++++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 8e76d3701db3f..2a60fc6674164 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -404,6 +404,7 @@ struct kvm_sync_regs { #define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2) #define KVM_X86_QUIRK_OUT_7E_INC_RIP (1 << 3) #define KVM_X86_QUIRK_MISC_ENABLE_NO_MWAIT (1 << 4) +#define KVM_X86_QUIRK_TSC_HOST_ACCESS (1 << 5) #define KVM_STATE_NESTED_FORMAT_VMX 0 #define KVM_STATE_NESTED_FORMAT_SVM 1 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9b8a2fe3a2398..aabded17abae4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3091,7 +3091,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) break; case MSR_IA32_TSC_ADJUST: if (guest_cpuid_has(vcpu, X86_FEATURE_TSC_ADJUST)) { - if (!msr_info->host_initiated) { + if (!msr_info->host_initiated || + !kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_TSC_HOST_ACCESS)) { s64 adj = data - vcpu->arch.ia32_tsc_adjust_msr; adjust_tsc_offset_guest(vcpu, adj); } @@ -3118,7 +3119,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vcpu->arch.msr_ia32_power_ctl = data; break; case MSR_IA32_TSC: - if (msr_info->host_initiated) { + if (msr_info->host_initiated && + kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_TSC_HOST_ACCESS)) { kvm_synchronize_tsc(vcpu, data); } else { u64 adj = kvm_compute_tsc_offset(vcpu, data) - vcpu->arch.l1_tsc_offset; @@ -3409,17 +3411,24 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = vcpu->arch.msr_ia32_power_ctl; break; case MSR_IA32_TSC: { + u64 tsc_offset; + /* * Intel SDM states that MSR_IA32_TSC read adds the TSC offset * even when not intercepted. AMD manual doesn't explicitly * state this but appears to behave the same. * - * On userspace reads and writes, however, we unconditionally + * On userspace reads and writes, when KVM_X86_QUIRK_SPECIAL_TSC_READ + * is present, however, we unconditionally * return L1's TSC value to ensure backwards-compatible * behavior for migration. */ - u64 tsc_offset = msr_info->host_initiated ? vcpu->arch.l1_tsc_offset : - vcpu->arch.tsc_offset; + + if (msr_info->host_initiated && + kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_TSC_HOST_ACCESS)) + tsc_offset = vcpu->arch.l1_tsc_offset; + else + tsc_offset = vcpu->arch.tsc_offset; msr_info->data = kvm_scale_tsc(vcpu, rdtsc()) + tsc_offset; break; From patchwork Thu Dec 3 17:11:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Levitsky X-Patchwork-Id: 338016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0059C1B0D8 for ; Thu, 3 Dec 2020 17:13:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 792D8206B6 for ; Thu, 3 Dec 2020 17:13:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2501959AbgLCRNl (ORCPT ); Thu, 3 Dec 2020 12:13:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50279 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731525AbgLCRNk (ORCPT ); Thu, 3 Dec 2020 12:13:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1607015534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wd+lJPFWy0m+3K/46pw2A3DNxUcmHii9mZMdRl39SGs=; b=bBFU6PULQjuNJRlMMmpSD84lbvrCk4t2JW1k678BxsxHPp78VCha/a3N+K86lYTM+5jBYj 3wo5quafbMikl2umtH8J8HduHF4hEVbuJlnWli5gKjBXWb19BXrObBo8Mkze4TukwS0ft9 R5xczATcYVSV3XxeyJKWI8iqL5oQTRE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-463-P_LlYtaeNaCLjF2S9EoQQw-1; Thu, 03 Dec 2020 12:12:12 -0500 X-MC-Unique: P_LlYtaeNaCLjF2S9EoQQw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 740CA1926DBD; Thu, 3 Dec 2020 17:11:49 +0000 (UTC) Received: from localhost.localdomain (unknown [10.35.206.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 895EB60854; Thu, 3 Dec 2020 17:11:42 +0000 (UTC) From: Maxim Levitsky To: kvm@vger.kernel.org Cc: "H. Peter Anvin" , Paolo Bonzini , Jonathan Corbet , Jim Mattson , Wanpeng Li , linux-kselftest@vger.kernel.org (open list:KERNEL SELFTEST FRAMEWORK), Vitaly Kuznetsov , Marcelo Tosatti , Thomas Gleixner , Sean Christopherson , linux-kernel@vger.kernel.org (open list), Ingo Molnar , x86@kernel.org (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)), Joerg Roedel , Borislav Petkov , Shuah Khan , Maxim Levitsky , Andrew Jones , Oliver Upton , linux-doc@vger.kernel.org (open list:DOCUMENTATION) Subject: [PATCH v2 3/3] kvm/selftests: update tsc_msrs_test to cover KVM_X86_QUIRK_TSC_HOST_ACCESS Date: Thu, 3 Dec 2020 19:11:18 +0200 Message-Id: <20201203171118.372391-4-mlevitsk@redhat.com> In-Reply-To: <20201203171118.372391-1-mlevitsk@redhat.com> References: <20201203171118.372391-1-mlevitsk@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Run the test once with quirk enabled and once disabled, and adjust the expected values accordingly. Signed-off-by: Maxim Levitsky --- .../selftests/kvm/x86_64/tsc_msrs_test.c | 79 ++++++++++++++++--- 1 file changed, 69 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c index e357d8e222d47..3900c543a7ee1 100644 --- a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c +++ b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c @@ -79,8 +79,6 @@ static void run_vcpu(struct kvm_vm *vm, uint32_t vcpuid, int stage) { struct ucall uc; - vcpu_args_set(vm, vcpuid, 1, vcpuid); - vcpu_ioctl(vm, vcpuid, KVM_RUN, NULL); switch (get_ucall(vm, vcpuid, &uc)) { @@ -101,7 +99,7 @@ static void run_vcpu(struct kvm_vm *vm, uint32_t vcpuid, int stage) } } -int main(void) +void run_test(bool quirk_disabled) { struct kvm_vm *vm; uint64_t val; @@ -109,6 +107,14 @@ int main(void) vm = vm_create_default(VCPU_ID, 0, guest_code); val = 0; + if (quirk_disabled) { + struct kvm_enable_cap cap = { + .cap = KVM_CAP_DISABLE_QUIRKS, + .args[0] = KVM_X86_QUIRK_TSC_HOST_ACCESS, + }; + vm_enable_cap(vm, &cap); + } + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), val); ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val); @@ -124,20 +130,67 @@ int main(void) ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), val); ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val); - /* - * Host: writes to MSR_IA32_TSC set the host-side offset - * and therefore do not change MSR_IA32_TSC_ADJUST. - */ - vcpu_set_msr(vm, 0, MSR_IA32_TSC, HOST_ADJUST + val); + if (quirk_disabled) { + struct kvm_tsc_state state = { + .tsc = HOST_ADJUST + val, + .flags = 0 + }; + vcpu_ioctl(vm, VCPU_ID, KVM_SET_TSC_STATE, &state); + } else { + /* + * Host: writes to MSR_IA32_TSC set the host-side offset + * and therefore do not change MSR_IA32_TSC_ADJUST + */ + vcpu_set_msr(vm, 0, MSR_IA32_TSC, HOST_ADJUST + val); + } + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), HOST_ADJUST + val); ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val); + + if (quirk_disabled) { + /* + * Host: writes to MSR_IA32_TSC work like in the guest + * when quirk is disabled + */ + vcpu_set_msr(vm, 0, MSR_IA32_TSC, val); + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), val); + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val - HOST_ADJUST); + + /* Restore the value */ + vcpu_set_msr(vm, 0, MSR_IA32_TSC, HOST_ADJUST + val); + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val); + } + run_vcpu(vm, VCPU_ID, 3); - /* Host: writes to MSR_IA32_TSC_ADJUST do not modify the TSC. */ + /* + * Host: writes to MSR_IA32_TSC_ADJUST do not modify the TSC, + * (unless the quirk is disabled) + */ vcpu_set_msr(vm, 0, MSR_IA32_TSC_ADJUST, UNITY * 123456); - ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), HOST_ADJUST + val); + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), + quirk_disabled ? UNITY * 123456 + HOST_ADJUST : HOST_ADJUST + val); ASSERT_EQ(vcpu_get_msr(vm, 0, MSR_IA32_TSC_ADJUST), UNITY * 123456); + if (quirk_disabled) { + /* + * Host: writes via KVM_SET_TSC_STATE + * to MSR_IA32_TSC and MSR_IA32_TSC_ADJUST can be done + * independently + */ + struct kvm_tsc_state state = { + .tsc = UNITY * 42, + .tsc_adjust = UNITY * 42 - HOST_ADJUST, + .flags = KVM_TSC_STATE_TSC_ADJUST_VALID + }; + + vcpu_ioctl(vm, VCPU_ID, KVM_SET_TSC_STATE, &state); + + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), UNITY * 42); + ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), + UNITY * 42 - HOST_ADJUST); + } + /* Restore previous value. */ vcpu_set_msr(vm, 0, MSR_IA32_TSC_ADJUST, val); ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC), HOST_ADJUST + val); @@ -162,6 +215,12 @@ int main(void) ASSERT_EQ(rounded_host_rdmsr(MSR_IA32_TSC_ADJUST), val - HOST_ADJUST); kvm_vm_free(vm); +} + +int main(void) +{ + run_test(false); + run_test(true); return 0; }