From patchwork Tue Apr 21 03:22:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 220887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 231F2C2BA19 for ; Tue, 21 Apr 2020 03:23:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C9DC206F9 for ; Tue, 21 Apr 2020 03:23:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728082AbgDUDXc (ORCPT ); Mon, 20 Apr 2020 23:23:32 -0400 Received: from foss.arm.com ([217.140.110.172]:57324 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725989AbgDUDXb (ORCPT ); Mon, 20 Apr 2020 23:23:31 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 03367C14; Mon, 20 Apr 2020 20:23:31 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D44B43F6CF; Mon, 20 Apr 2020 20:23:23 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v11 1/9] psci: export psci conduit get helper. Date: Tue, 21 Apr 2020 11:22:56 +0800 Message-Id: <20200421032304.26300-2-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421032304.26300-1-jianyong.wu@arm.com> References: <20200421032304.26300-1-jianyong.wu@arm.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Export arm_smccc_1_1_get_conduit then modules can use smccc helper which adopts it. Signed-off-by: Jianyong Wu --- drivers/firmware/psci/psci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index 2937d44b5df4..fd3c88f21b6a 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -64,6 +64,7 @@ enum arm_smccc_conduit arm_smccc_1_1_get_conduit(void) return psci_ops.conduit; } +EXPORT_SYMBOL(arm_smccc_1_1_get_conduit); typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); From patchwork Tue Apr 21 03:22:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 220886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07F26C55183 for ; Tue, 21 Apr 2020 03:23:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE3442084D for ; Tue, 21 Apr 2020 03:23:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728219AbgDUDXm (ORCPT ); Mon, 20 Apr 2020 23:23:42 -0400 Received: from foss.arm.com ([217.140.110.172]:57360 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725989AbgDUDXj (ORCPT ); Mon, 20 Apr 2020 23:23:39 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C883D31B; Mon, 20 Apr 2020 20:23:37 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 835A83F6CF; Mon, 20 Apr 2020 20:23:31 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v11 2/9] ptp: Reorganize ptp_kvm modules to make it arch-independent. Date: Tue, 21 Apr 2020 11:22:57 +0800 Message-Id: <20200421032304.26300-3-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421032304.26300-1-jianyong.wu@arm.com> References: <20200421032304.26300-1-jianyong.wu@arm.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Currently, ptp_kvm modules implementation is only for x86 which includs large part of arch-specific code. This patch move all of those code into new arch related file in the same directory. Signed-off-by: Jianyong Wu --- drivers/ptp/Makefile | 1 + drivers/ptp/ptp_kvm.h | 11 +++ drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 80 +++++------------- drivers/ptp/ptp_kvm_x86.c | 89 +++++++++++++++++++++ 4 files changed, 122 insertions(+), 59 deletions(-) create mode 100644 drivers/ptp/ptp_kvm.h rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (63%) create mode 100644 drivers/ptp/ptp_kvm_x86.c diff --git a/drivers/ptp/Makefile b/drivers/ptp/Makefile index 7aff75f745dc..baac6f5b243b 100644 --- a/drivers/ptp/Makefile +++ b/drivers/ptp/Makefile @@ -4,6 +4,7 @@ # ptp-y := ptp_clock.o ptp_chardev.o ptp_sysfs.o +ptp_kvm-y := ptp_kvm_$(ARCH).o ptp_kvm_common.o obj-$(CONFIG_PTP_1588_CLOCK) += ptp.o obj-$(CONFIG_PTP_1588_CLOCK_DTE) += ptp_dte.o obj-$(CONFIG_PTP_1588_CLOCK_INES) += ptp_ines.o diff --git a/drivers/ptp/ptp_kvm.h b/drivers/ptp/ptp_kvm.h new file mode 100644 index 000000000000..4bf1802bbeb8 --- /dev/null +++ b/drivers/ptp/ptp_kvm.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Virtual PTP 1588 clock for use with KVM guests + * + * Copyright (C) 2017 Red Hat Inc. + */ + +int kvm_arch_ptp_init(void); +int kvm_arch_ptp_get_clock(struct timespec64 *ts); +int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, + struct timespec64 *tspec, void *cs); diff --git a/drivers/ptp/ptp_kvm.c b/drivers/ptp/ptp_kvm_common.c similarity index 63% rename from drivers/ptp/ptp_kvm.c rename to drivers/ptp/ptp_kvm_common.c index fc7d0b77e118..4fdd8ab11a28 100644 --- a/drivers/ptp/ptp_kvm.c +++ b/drivers/ptp/ptp_kvm_common.c @@ -8,15 +8,16 @@ #include #include #include +#include #include #include #include -#include -#include #include #include +#include "ptp_kvm.h" + struct kvm_ptp_clock { struct ptp_clock *ptp_clock; struct ptp_clock_info caps; @@ -24,56 +25,29 @@ struct kvm_ptp_clock { DEFINE_SPINLOCK(kvm_ptp_lock); -static struct pvclock_vsyscall_time_info *hv_clock; - -static struct kvm_clock_pairing clock_pair; -static phys_addr_t clock_pair_gpa; - static int ptp_kvm_get_time_fn(ktime_t *device_time, struct system_counterval_t *system_counter, void *ctx) { - unsigned long ret; + unsigned long ret, cycle; struct timespec64 tspec; - unsigned version; - int cpu; - struct pvclock_vcpu_time_info *src; + struct clocksource *cs; spin_lock(&kvm_ptp_lock); preempt_disable_notrace(); - cpu = smp_processor_id(); - src = &hv_clock[cpu].pvti; - - do { - /* - * We are using a TSC value read in the hosts - * kvm_hc_clock_pairing handling. - * So any changes to tsc_to_system_mul - * and tsc_shift or any other pvclock - * data invalidate that measurement. - */ - version = pvclock_read_begin(src); - - ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, - clock_pair_gpa, - KVM_CLOCK_PAIRING_WALLCLOCK); - if (ret != 0) { - pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret); - spin_unlock(&kvm_ptp_lock); - preempt_enable_notrace(); - return -EOPNOTSUPP; - } - - tspec.tv_sec = clock_pair.sec; - tspec.tv_nsec = clock_pair.nsec; - ret = __pvclock_read_cycles(src, clock_pair.tsc); - } while (pvclock_read_retry(src, version)); + ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs); + if (ret != 0) { + pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret); + spin_unlock(&kvm_ptp_lock); + preempt_enable_notrace(); + return -EOPNOTSUPP; + } preempt_enable_notrace(); - system_counter->cycles = ret; - system_counter->cs = &kvm_clock; + system_counter->cycles = cycle; + system_counter->cs = cs; *device_time = timespec64_to_ktime(tspec); @@ -116,17 +90,13 @@ static int ptp_kvm_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts) spin_lock(&kvm_ptp_lock); - ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, - clock_pair_gpa, - KVM_CLOCK_PAIRING_WALLCLOCK); + ret = kvm_arch_ptp_get_clock(&tspec); if (ret != 0) { pr_err_ratelimited("clock offset hypercall ret %lu\n", ret); spin_unlock(&kvm_ptp_lock); return -EOPNOTSUPP; } - tspec.tv_sec = clock_pair.sec; - tspec.tv_nsec = clock_pair.nsec; spin_unlock(&kvm_ptp_lock); memcpy(ts, &tspec, sizeof(struct timespec64)); @@ -166,21 +136,13 @@ static void __exit ptp_kvm_exit(void) static int __init ptp_kvm_init(void) { - long ret; - - if (!kvm_para_available()) - return -ENODEV; - - clock_pair_gpa = slow_virt_to_phys(&clock_pair); - hv_clock = pvclock_get_pvti_cpu0_va(); + int ret; - if (!hv_clock) - return -ENODEV; - - ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, clock_pair_gpa, - KVM_CLOCK_PAIRING_WALLCLOCK); - if (ret == -KVM_ENOSYS || ret == -KVM_EOPNOTSUPP) - return -ENODEV; + ret = kvm_arch_ptp_init(); + if (ret) { + pr_err("fail to initialize ptp_kvm"); + return -EOPNOTSUPP; + } kvm_ptp_clock.caps = ptp_kvm_caps; diff --git a/drivers/ptp/ptp_kvm_x86.c b/drivers/ptp/ptp_kvm_x86.c new file mode 100644 index 000000000000..55417b3b282d --- /dev/null +++ b/drivers/ptp/ptp_kvm_x86.c @@ -0,0 +1,89 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Virtual PTP 1588 clock for use with KVM guests + * + * Copyright (C) 2017 Red Hat Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +phys_addr_t clock_pair_gpa; +struct kvm_clock_pairing clock_pair; +struct pvclock_vsyscall_time_info *hv_clock; + +int kvm_arch_ptp_init(void) +{ + int ret; + + if (!kvm_para_available()) + return -ENODEV; + + clock_pair_gpa = slow_virt_to_phys(&clock_pair); + hv_clock = pvclock_get_pvti_cpu0_va(); + if (!hv_clock) + return -ENODEV; + + ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, clock_pair_gpa, + KVM_CLOCK_PAIRING_WALLCLOCK); + if (ret == -KVM_ENOSYS || ret == -KVM_EOPNOTSUPP) + return -ENODEV; + + return 0; +} + +int kvm_arch_ptp_get_clock(struct timespec64 *ts) +{ + long ret; + + ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, + clock_pair_gpa, + KVM_CLOCK_PAIRING_WALLCLOCK); + if (ret != 0) + return -EOPNOTSUPP; + + ts->tv_sec = clock_pair.sec; + ts->tv_nsec = clock_pair.nsec; + + return 0; +} + +int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *tspec, + struct clocksource **cs) +{ + unsigned long ret; + unsigned int version; + int cpu; + struct pvclock_vcpu_time_info *src; + + cpu = smp_processor_id(); + src = &hv_clock[cpu].pvti; + + do { + /* + * We are using a TSC value read in the hosts + * kvm_hc_clock_pairing handling. + * So any changes to tsc_to_system_mul + * and tsc_shift or any other pvclock + * data invalidate that measurement. + */ + version = pvclock_read_begin(src); + + ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, + clock_pair_gpa, + KVM_CLOCK_PAIRING_WALLCLOCK); + tspec->tv_sec = clock_pair.sec; + tspec->tv_nsec = clock_pair.nsec; + *cycle = __pvclock_read_cycles(src, clock_pair.tsc); + } while (pvclock_read_retry(src, version)); + + *cs = &kvm_clock; + + return 0; +} From patchwork Tue Apr 21 03:22:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 220885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77E99C55183 for ; Tue, 21 Apr 2020 03:23:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F5E020679 for ; Tue, 21 Apr 2020 03:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728311AbgDUDXy (ORCPT ); Mon, 20 Apr 2020 23:23:54 -0400 Received: from foss.arm.com ([217.140.110.172]:57430 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728285AbgDUDXw (ORCPT ); Mon, 20 Apr 2020 23:23:52 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E0C9031B; Mon, 20 Apr 2020 20:23:51 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D7F043F6CF; Mon, 20 Apr 2020 20:23:44 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v11 4/9] clocksource: Add clocksource id for arm arch counter Date: Tue, 21 Apr 2020 11:22:59 +0800 Message-Id: <20200421032304.26300-5-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421032304.26300-1-jianyong.wu@arm.com> References: <20200421032304.26300-1-jianyong.wu@arm.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add clocksource id for arm arch counter to let it be identified easily and elegantly in ptp_kvm implementation for arm. Signed-off-by: Jianyong Wu --- drivers/clocksource/arm_arch_timer.c | 2 ++ include/linux/clocksource_ids.h | 1 + 2 files changed, 3 insertions(+) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 2204a444e801..0f44f296ed17 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -191,6 +192,7 @@ static u64 arch_counter_read_cc(const struct cyclecounter *cc) static struct clocksource clocksource_counter = { .name = "arch_sys_counter", + .id = CSID_ARM_ARCH_COUNTER, .rating = 400, .read = arch_counter_read, .mask = CLOCKSOURCE_MASK(56), diff --git a/include/linux/clocksource_ids.h b/include/linux/clocksource_ids.h index 4d8e19e05328..16775d7d8f8d 100644 --- a/include/linux/clocksource_ids.h +++ b/include/linux/clocksource_ids.h @@ -5,6 +5,7 @@ /* Enum to give clocksources a unique identifier */ enum clocksource_ids { CSID_GENERIC = 0, + CSID_ARM_ARCH_COUNTER, CSID_MAX, }; From patchwork Tue Apr 21 03:23:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 220884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CD78C55185 for ; Tue, 21 Apr 2020 03:24:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0B995206F9 for ; Tue, 21 Apr 2020 03:24:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728376AbgDUDYK (ORCPT ); Mon, 20 Apr 2020 23:24:10 -0400 Received: from foss.arm.com ([217.140.110.172]:57492 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728284AbgDUDYI (ORCPT ); Mon, 20 Apr 2020 23:24:08 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5DF7CC14; Mon, 20 Apr 2020 20:24:07 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 892543F6CF; Mon, 20 Apr 2020 20:23:58 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v11 6/9] ptp: arm64: Enable ptp_kvm for arm/arm64 Date: Tue, 21 Apr 2020 11:23:01 +0800 Message-Id: <20200421032304.26300-7-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421032304.26300-1-jianyong.wu@arm.com> References: <20200421032304.26300-1-jianyong.wu@arm.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Currently, there is no mechanism to keep time sync between guest and host in arm64 virtualization environment. Time in guest will drift compared with host after boot up as they may both use third party time sources to correct their time respectively. The time deviation will be in order of milliseconds. But in some scenarios,like in cloud envirenment, we ask for higher time precision. kvm ptp clock, which choose the host clock source clock as a reference clock to sync time clock between guest and host has been adopted by x86 which makes the time sync order from milliseconds to nanoseconds. This patch enables kvm ptp on arm64 and improve clock sync precison significantly. Test result comparison between with kvm ptp and without it in arm64 is as follows. This test derived from the result of command 'chronyc sources'. we should take more care of the last sample column which shows the offset between the local clock and the source at the last measurement. no kvm ptp in guest: MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== ^* dns1.synet.edu.cn 2 6 377 13 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 21 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 29 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 37 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 45 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 53 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 61 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 4 -130us[ +796us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 12 -130us[ +796us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 20 -130us[ +796us] +/- 21ms in host: MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== ^* 120.25.115.20 2 7 377 72 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 92 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 112 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 2 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 22 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 43 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 63 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 83 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 103 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 123 +872ns[-6808ns] +/- 17ms The dns1.synet.edu.cn is the network reference clock for guest and 120.25.115.20 is the network reference clock for host. we can't get the clock error between guest and host directly, but a roughly estimated value will be in order of hundreds of us to ms. with kvm ptp in guest: chrony has been disabled in host to remove the disturb by network clock. MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== * PHC0 0 3 377 8 -7ns[ +1ns] +/- 3ns * PHC0 0 3 377 8 +1ns[ +16ns] +/- 3ns * PHC0 0 3 377 6 -4ns[ -0ns] +/- 6ns * PHC0 0 3 377 6 -8ns[ -12ns] +/- 5ns * PHC0 0 3 377 5 +2ns[ +4ns] +/- 4ns * PHC0 0 3 377 13 +2ns[ +4ns] +/- 4ns * PHC0 0 3 377 12 -4ns[ -6ns] +/- 4ns * PHC0 0 3 377 11 -8ns[ -11ns] +/- 6ns * PHC0 0 3 377 10 -14ns[ -20ns] +/- 4ns * PHC0 0 3 377 8 +4ns[ +5ns] +/- 4ns The PHC0 is the ptp clock which choose the host clock as its source clock. So we can be sure to say that the clock error between host and guest is in order of ns. Signed-off-by: Jianyong Wu --- drivers/clocksource/arm_arch_timer.c | 22 ++++++++++++ drivers/ptp/Kconfig | 2 +- drivers/ptp/ptp_kvm_arm64.c | 53 ++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 drivers/ptp/ptp_kvm_arm64.c diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 0f44f296ed17..7a33993c0d05 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -1641,3 +1641,25 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table) } TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init); #endif + +#if IS_ENABLED(CONFIG_PTP_1588_CLOCK_KVM) +#include +int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts, + struct clocksource **cs) +{ + struct arm_smccc_res hvc_res; + ktime_t ktime_overall; + + arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, &hvc_res); + if ((int)(hvc_res.a0) < 0) + return -EOPNOTSUPP; + + ktime_overall = (long long)hvc_res.a0 << 32 | hvc_res.a1; + *ts = ktime_to_timespec64(ktime_overall); + *cycle = (long long)hvc_res.a2 << 32 | hvc_res.a3; + *cs = &clocksource_counter; + + return 0; +} +EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp); +#endif diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig index 86400c708150..0733c8c61541 100644 --- a/drivers/ptp/Kconfig +++ b/drivers/ptp/Kconfig @@ -106,7 +106,7 @@ config PTP_1588_CLOCK_PCH config PTP_1588_CLOCK_KVM tristate "KVM virtual PTP clock" depends on PTP_1588_CLOCK - depends on KVM_GUEST && X86 + depends on KVM_GUEST && X86 || ARM64 && ARM_ARCH_TIMER && ARM_PSCI_FW default y help This driver adds support for using kvm infrastructure as a PTP diff --git a/drivers/ptp/ptp_kvm_arm64.c b/drivers/ptp/ptp_kvm_arm64.c new file mode 100644 index 000000000000..9752e849e0ae --- /dev/null +++ b/drivers/ptp/ptp_kvm_arm64.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Virtual PTP 1588 clock for use with KVM guests + * Copyright (C) 2019 ARM Ltd. + * All Rights Reserved + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +int kvm_arch_ptp_init(void) +{ + struct arm_smccc_res hvc_res; + + arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, + &hvc_res); + if ((int)(hvc_res.a0) < 0) + return -EOPNOTSUPP; + + return 0; +} + +int kvm_arch_ptp_get_clock_generic(struct timespec64 *ts, + struct arm_smccc_res *hvc_res) +{ + ktime_t ktime_overall; + + arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, + hvc_res); + if ((int)(hvc_res->a0) < 0) + return -EOPNOTSUPP; + + ktime_overall = (long long)hvc_res->a0 << 32 | hvc_res->a1; + *ts = ktime_to_timespec64(ktime_overall); + + return 0; +} + +int kvm_arch_ptp_get_clock(struct timespec64 *ts) +{ + struct arm_smccc_res hvc_res; + + kvm_arch_ptp_get_clock_generic(ts, &hvc_res); + + return 0; +} From patchwork Tue Apr 21 03:23:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 220883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ACEFC54FC9 for ; Tue, 21 Apr 2020 03:24:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F4202084D for ; Tue, 21 Apr 2020 03:24:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728464AbgDUDYX (ORCPT ); Mon, 20 Apr 2020 23:24:23 -0400 Received: from foss.arm.com ([217.140.110.172]:57560 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728444AbgDUDYW (ORCPT ); Mon, 20 Apr 2020 23:24:22 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 96733C14; Mon, 20 Apr 2020 20:24:21 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6A4193F6CF; Mon, 20 Apr 2020 20:24:15 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v11 8/9] arm64: add mechanism to let user choose which counter to return Date: Tue, 21 Apr 2020 11:23:03 +0800 Message-Id: <20200421032304.26300-9-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200421032304.26300-1-jianyong.wu@arm.com> References: <20200421032304.26300-1-jianyong.wu@arm.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In general, vm inside will use virtual counter compered with host use phyical counter. But in some special scenarios, like nested virtualization, phyical counter maybe used by vm. A interface added in ptp_kvm driver to offer a mechanism to let user choose which counter should be return from host. To use this feature, you should call PTP_EXTTS_REQUEST(2) ioctl with flag set bit PTP_KVM_ARM_PHY_COUNTER in its argument then call PTP_SYS_OFFSET_PRECISE(2) ioctl to get the cross timestamp and phyical counter will return. If the bit not set or no call for PTP_EXTTS_REQUEST2, virtual counter will return by default. Signed-off-by: Jianyong Wu Suggested-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 11 ++++++++++- drivers/ptp/ptp_chardev.c | 8 +++++++- drivers/ptp/ptp_kvm_common.c | 7 ++++--- include/uapi/linux/ptp_clock.h | 4 +++- 4 files changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 47d69b3f2d9a..52f629d9c561 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -1650,7 +1650,16 @@ int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts, struct arm_smccc_res hvc_res; ktime_t ktime_overall; - arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, &hvc_res); + /* + * an argument will be passed by a0 to determine weather virtual + * counter or phyical counter should be passed back. + */ + if (*ctx) + arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, + ARM_SMCCC_HYP_KVM_PTP_PHY, &hvc_res); + else + arm_smccc_1_1_invoke(ARM_SMCCC_HYP_KVM_PTP_FUNC_ID, + ARM_SMCCC_HYP_KVM_PTP_VIRT, &hvc_res); if ((int)(hvc_res.a0) < 0) return -EOPNOTSUPP; diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c index fef72f29f3c8..505ed7eb0ca5 100644 --- a/drivers/ptp/ptp_chardev.c +++ b/drivers/ptp/ptp_chardev.c @@ -122,6 +122,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg) struct ptp_pin_desc pd; struct timespec64 ts; int enable, err = 0; + static long flag; switch (cmd) { @@ -149,6 +150,11 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg) err = -EFAULT; break; } + /* + * Tell driver to call for physical counter. + * This is only for arm. + */ + flag = req.extts.flags & PTP_KVM_ARM_PHY_COUNTER; if (cmd == PTP_EXTTS_REQUEST2) { /* Tell the drivers to check the flags carefully. */ req.extts.flags |= PTP_STRICT_FLAGS; @@ -235,7 +241,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd, unsigned long arg) err = -EOPNOTSUPP; break; } - err = ptp->info->getcrosststamp(ptp->info, &xtstamp, NULL); + err = ptp->info->getcrosststamp(ptp->info, &xtstamp, &flag); if (err) break; diff --git a/drivers/ptp/ptp_kvm_common.c b/drivers/ptp/ptp_kvm_common.c index 4fdd8ab11a28..39367e230176 100644 --- a/drivers/ptp/ptp_kvm_common.c +++ b/drivers/ptp/ptp_kvm_common.c @@ -36,7 +36,7 @@ static int ptp_kvm_get_time_fn(ktime_t *device_time, spin_lock(&kvm_ptp_lock); preempt_disable_notrace(); - ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs); + ret = kvm_arch_ptp_get_crosststamp(&cycle, &tspec, &cs, ctx); if (ret != 0) { pr_err_ratelimited("clock pairing hypercall ret %lu\n", ret); spin_unlock(&kvm_ptp_lock); @@ -57,9 +57,10 @@ static int ptp_kvm_get_time_fn(ktime_t *device_time, } static int ptp_kvm_getcrosststamp(struct ptp_clock_info *ptp, - struct system_device_crosststamp *xtstamp) + struct system_device_crosststamp *xtstamp, + long *flag) { - return get_device_system_crosststamp(ptp_kvm_get_time_fn, NULL, + return get_device_system_crosststamp(ptp_kvm_get_time_fn, flag, NULL, xtstamp); } diff --git a/include/uapi/linux/ptp_clock.h b/include/uapi/linux/ptp_clock.h index 9dc9d0079e98..71e388a82244 100644 --- a/include/uapi/linux/ptp_clock.h +++ b/include/uapi/linux/ptp_clock.h @@ -32,6 +32,7 @@ #define PTP_RISING_EDGE (1<<1) #define PTP_FALLING_EDGE (1<<2) #define PTP_STRICT_FLAGS (1<<3) +#define PTP_KVM_ARM_PHY_COUNTER (1<<4) #define PTP_EXTTS_EDGES (PTP_RISING_EDGE | PTP_FALLING_EDGE) /* @@ -40,7 +41,8 @@ #define PTP_EXTTS_VALID_FLAGS (PTP_ENABLE_FEATURE | \ PTP_RISING_EDGE | \ PTP_FALLING_EDGE | \ - PTP_STRICT_FLAGS) + PTP_STRICT_FLAGS | \ + PTP_KVM_ARM_PHY_COUNTER) /* * flag fields valid for the original PTP_EXTTS_REQUEST ioctl.