From patchwork Wed Feb 26 18:15:11 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 25384 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ob0-f199.google.com (mail-ob0-f199.google.com [209.85.214.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id B3C5C20636 for ; Wed, 26 Feb 2014 18:15:51 +0000 (UTC) Received: by mail-ob0-f199.google.com with SMTP id uy5sf4239333obc.2 for ; Wed, 26 Feb 2014 10:15:51 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=amLVY0LhLTVa/X/JGNcybzBBBWul8Wp1Kpr0WfGxqLk=; b=DR4tF2MT7c3cw9Fog8PZZc6AZM5HPn9eoBaGnGU9NBDQ1OtlsQuUvM0p+BF57bdZlY vQNDH8cSdtMXJ5eotm7kmvHko4gMbsbZBVCEMoXP5ISeSriWYD18S+NGtkcMlhSXXCGh hI/6hQ8GFCpAXkjyrSRFSDbEuFnev9CgyvUzIQ6P1OdxaOJgqiUxp0DFIHQAydxlOc7V 6YYCM3G2xENR8uTWv90Vs4omyoTlJiq4u7QJwEQZ9LvH4UVMKMMeZAl+ckqzi0NoKdzr Vt/qrmZgvF5rCOp9d711/2wzm7PIgKwujckgwnh9XZSTpDjY+zGmL722F8A4I8WkaMqh etMQ== X-Gm-Message-State: ALoCoQk/8/3XrCzTbFSGikqfbtYhtispoEvwd70Zo5iGEJjoOvWGm+NaMfqW/RpHwyzfAlnlJX4w X-Received: by 10.42.115.129 with SMTP id k1mr256667icq.25.1393438551190; Wed, 26 Feb 2014 10:15:51 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.109.137 with SMTP id l9ls330191qgf.40.gmail; Wed, 26 Feb 2014 10:15:51 -0800 (PST) X-Received: by 10.220.68.5 with SMTP id t5mr23045vci.63.1393438551040; Wed, 26 Feb 2014 10:15:51 -0800 (PST) Received: from mail-vc0-f174.google.com (mail-vc0-f174.google.com [209.85.220.174]) by mx.google.com with ESMTPS id x7si436818vel.76.2014.02.26.10.15.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Feb 2014 10:15:51 -0800 (PST) Received-SPF: neutral (google.com: 209.85.220.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.174; Received: by mail-vc0-f174.google.com with SMTP id im17so1331393vcb.33 for ; Wed, 26 Feb 2014 10:15:51 -0800 (PST) X-Received: by 10.221.66.132 with SMTP id xq4mr26331vcb.57.1393438550950; Wed, 26 Feb 2014 10:15:50 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.174.196 with SMTP id u4csp45254vcz; Wed, 26 Feb 2014 10:15:47 -0800 (PST) X-Received: by 10.66.150.69 with SMTP id ug5mr10241806pab.55.1393438530461; Wed, 26 Feb 2014 10:15:30 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id uc9si1807882pac.181.2014.02.26.10.15.29; Wed, 26 Feb 2014 10:15:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753231AbaBZSP0 (ORCPT + 26 others); Wed, 26 Feb 2014 13:15:26 -0500 Received: from mx1.redhat.com ([209.132.183.28]:24260 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751557AbaBZSPV (ORCPT ); Wed, 26 Feb 2014 13:15:21 -0500 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s1QIFJsK028488 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 26 Feb 2014 13:15:21 -0500 Received: from hawk.usersys.redhat.com.com (dhcp-1-243.brq.redhat.com [10.34.1.243]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s1QIFGDN017815; Wed, 26 Feb 2014 13:15:18 -0500 From: Andrew Jones To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mtosatti@redhat.com, pbonzini@redhat.com Subject: [PATCH 1/2] x86: kvm: rate-limit global clock updates Date: Wed, 26 Feb 2014 19:15:11 +0100 Message-Id: <1393438512-21273-2-git-send-email-drjones@redhat.com> In-Reply-To: <1393438512-21273-1-git-send-email-drjones@redhat.com> References: <1393438512-21273-1-git-send-email-drjones@redhat.com> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: drjones@redhat.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , When we update a vcpu's local clock it may pick up an NTP correction. We can't wait an indeterminate amount of time for other vcpus to pick up that correction, so commit 0061d53daf26f introduced a global clock update. However, we can't request a global clock update on every vcpu load either (which is what happens if the tsc is marked as unstable). The solution is to rate-limit the global clock updates. Marcelo calculated that we should delay the global clock updates no more than 0.1s as follows: Assume an NTP correction c is applied to one vcpu, but not the other, then in n seconds the delta of the vcpu system_timestamps will be c * n. If we assume a correction of 500ppm (worst-case), then the two vcpus will diverge 100us in 0.1s, which is a considerable amount. Signed-off-by: Andrew Jones --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 33 +++++++++++++++++++++++++++++---- 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e714f8c08ccf2..9aa09d330a4b5 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -598,6 +598,7 @@ struct kvm_arch { bool use_master_clock; u64 master_kernel_ns; cycle_t master_cycle_now; + struct delayed_work kvmclock_update_work; struct kvm_xen_hvm_config xen_hvm_config; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4cca45853dfeb..a2d30de597b7d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1628,20 +1628,43 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) * the others. * * So in those cases, request a kvmclock update for all vcpus. - * The worst case for a remote vcpu to update its kvmclock - * is then bounded by maximum nohz sleep latency. + * We need to rate-limit these requests though, as they can + * considerably slow guests that have a large number of vcpus. + * The time for a remote vcpu to update its kvmclock is bound + * by the delay we use to rate-limit the updates. */ -static void kvm_gen_kvmclock_update(struct kvm_vcpu *v) +#define KVMCLOCK_UPDATE_DELAY msecs_to_jiffies(100) + +static void kvmclock_update_fn(struct work_struct *work) { int i; - struct kvm *kvm = v->kvm; + struct delayed_work *dwork = to_delayed_work(work); + struct kvm_arch *ka = container_of(dwork, struct kvm_arch, + kvmclock_update_work); + struct kvm *kvm = container_of(ka, struct kvm, arch); struct kvm_vcpu *vcpu; kvm_for_each_vcpu(i, vcpu, kvm) { set_bit(KVM_REQ_CLOCK_UPDATE, &vcpu->requests); kvm_vcpu_kick(vcpu); } + kvm_put_kvm(kvm); +} + +static void kvm_schedule_kvmclock_update(struct kvm *kvm) +{ + kvm_get_kvm(kvm); + schedule_delayed_work(&kvm->arch.kvmclock_update_work, + KVMCLOCK_UPDATE_DELAY); +} + +static void kvm_gen_kvmclock_update(struct kvm_vcpu *v) +{ + struct kvm *kvm = v->kvm; + + set_bit(KVM_REQ_CLOCK_UPDATE, &v->requests); + kvm_schedule_kvmclock_update(kvm); } static bool msr_mtrr_valid(unsigned msr) @@ -7019,6 +7042,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) pvclock_update_vm_gtod_copy(kvm); + INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn); + return 0; }