From patchwork Thu Jul 10 15:50:07 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 33464 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ob0-f198.google.com (mail-ob0-f198.google.com [209.85.214.198]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 6335B203F4 for ; Thu, 10 Jul 2014 23:44:42 +0000 (UTC) Received: by mail-ob0-f198.google.com with SMTP id wp18sf1648726obc.5 for ; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list:content-type :content-transfer-encoding; bh=sSsEQt+R0OLh0Ukp09bTyN7tPMpbJXSonVeACWMNtZs=; b=c5PWDqk4jtV1da5iUhoE0pai54l+SIY7FZrowi/lH7fd7q5FHbp8or0tnZH+rUwfBu N9mZv1Mou01q3mUB3A/3vlA8K/3bk9bJo5qPd7I6Z2SjlX+RFaLGr6Jud45D2dxhZSLm 7HJAquPUxNZFslCjg3PILwWumSfhSC/wx0My3TLXZNZknJ9cAjlUrTXAD9pG4WhfEkyH fP15Be/YCkTnOEGAK9gbxVnim7PdhiUDUBJg8sB/+OVcI1lfOF3uj7vZCE052T0e+jH4 Ovn2GJc1+SHh/x83Dhfcm504ejDqLGG7e/mOQIFxd4NVo0iCrdqUzp7c+n5ig8o+ReMY cevA== X-Gm-Message-State: ALoCoQlapdKB3NjG7qHrTMLtkQSwu5dlj1f407tECiPr7Y3C5yEOO5gRLplb2mWd/kH2Nhs/RHou X-Received: by 10.182.125.4 with SMTP id mm4mr17193384obb.49.1405035881968; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.50.77 with SMTP id r71ls247784qga.71.gmail; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-Received: by 10.58.198.201 with SMTP id je9mr3494061vec.33.1405035881871; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) Received: from mail-vc0-f169.google.com (mail-vc0-f169.google.com [209.85.220.169]) by mx.google.com with ESMTPS id vf3si441675veb.100.2014.07.10.16.44.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Jul 2014 16:44:41 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.169 as permitted sender) client-ip=209.85.220.169; Received: by mail-vc0-f169.google.com with SMTP id la4so631679vcb.14 for ; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-Received: by 10.58.243.170 with SMTP id wz10mr3181068vec.55.1405035881774; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp176446vcb; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) X-Received: by 10.224.42.196 with SMTP id t4mr84147962qae.48.1405035881309; Thu, 10 Jul 2014 16:44:41 -0700 (PDT) Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t96si956055qgd.15.2014.07.10.16.44.41 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 10 Jul 2014 16:44:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Received: from localhost ([::1]:40953 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X5O12-0007GL-Qv for patch@linaro.org; Thu, 10 Jul 2014 19:44:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56465) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X5Gt1-0001Se-In for qemu-devel@nongnu.org; Thu, 10 Jul 2014 12:08:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X5Gsv-0005mq-Rt for qemu-devel@nongnu.org; Thu, 10 Jul 2014 12:07:55 -0400 Received: from static.88-198-71-155.clients.your-server.de ([88.198.71.155]:35750 helo=socrates.bennee.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X5Gsv-0005mg-Hv for qemu-devel@nongnu.org; Thu, 10 Jul 2014 12:07:49 -0400 Received: from localhost ([127.0.0.1] helo=zen.linaro.local) by socrates.bennee.com with esmtp (Exim 4.80) (envelope-from ) id 1X5GgM-0007L4-JB; Thu, 10 Jul 2014 17:54:50 +0200 From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Thu, 10 Jul 2014 16:50:07 +0100 Message-Id: <1405007407-23549-11-git-send-email-alex.bennee@linaro.org> X-Mailer: git-send-email 2.0.1 In-Reply-To: <1405007407-23549-1-git-send-email-alex.bennee@linaro.org> References: <1405007407-23549-1-git-send-email-alex.bennee@linaro.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: alex.bennee@linaro.org X-SA-Exim-Scanned: No (on socrates.bennee.com); SAEximRunCond expanded to false X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 88.198.71.155 Cc: Peter Maydell , =?UTF-8?q?Alex=20Benn=C3=A9e?= , "open list:Overall" , Paolo Bonzini Subject: [Qemu-devel] [PATCH v2 10/10] target-arm/kvm: make reg sync code common between kvm32/64 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: alex.bennee@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.169 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Before we launch a guest we query KVM for the list of "co-processor" registers it knows about which is used later for save/restore of machine state. The logic is identical for both 32-bit and 64-bit so I've moved it all into the common code and simplified the exit paths (as failure => exit). This list may well have more registers than are known by the TCG emulation which is not necessarily a problem but it does stop us from migrating between KVM and TCG hosted guests. I've added some additional checking to report those registers under -d unimp. Signed-off-by: Alex Bennée diff --git a/target-arm/kvm.c b/target-arm/kvm.c index 72e242d..a2895dc 100644 --- a/target-arm/kvm.c +++ b/target-arm/kvm.c @@ -21,6 +21,7 @@ #include "sysemu/kvm.h" #include "kvm_arm.h" #include "cpu.h" +#include "internals.h" #include "hw/arm/arm.h" const KVMCapabilityInfo kvm_arch_required_capabilities[] = { @@ -289,6 +290,130 @@ static void failed_cpreg_operation(ARMCPU *cpu, uint64_t regidx, int ret, func, ret, regidx, cpreg ? cpreg->name : "unknown"); } +static int compare_u64(const void *a, const void *b) +{ + if (*(uint64_t *)a > *(uint64_t *)b) { + return 1; + } + if (*(uint64_t *)a < *(uint64_t *)b) { + return -1; + } + return 0; +} + +static bool reg_syncs_via_tuple_list(uint64_t regidx) +{ + /* Return true if the regidx is a register we should synchronize + * via the cpreg_tuples array (ie is not a core reg we sync by + * hand in kvm_arch_get/put_registers()) + */ + switch (regidx & KVM_REG_ARM_COPROC_MASK) { + case KVM_REG_ARM_CORE: +#ifdef KVM_REG_ARM_VFP + case KVM_REG_ARM_VFP: +#endif + return false; + default: + return true; + } +} + +/* + * Fetch a list of registers from KVM that we will need to be able to + * migrate the state. These registers may or may not map onto real + * hardware registers but either way QEMU uses the KVM_GET/SET_ONE_REG + * api to copy their state back and forth when required. + * + * For migration between KVM and TCG both models need to understand + * the same set of registers. + * + * If we exit due to failure we would leak memory but we'll be exiting + * anyway so the return path is kept simple. + */ +bool kvm_arm_sync_register_list(CPUState *cs) +{ + struct kvm_reg_list rl; + struct kvm_reg_list *rlp; + int i, j, ret, arraylen; + ARMCPU *cpu = ARM_CPU(cs); + + /* Populate the cpreg list based on the kernel's idea + * of what registers exist (and throw away the TCG-created list). + */ + rl.n = 0; + ret = kvm_vcpu_ioctl(cs, KVM_GET_REG_LIST, &rl); + if (ret != -E2BIG) { + return FALSE; + } + + rlp = g_malloc(sizeof(struct kvm_reg_list) + (rl.n * sizeof(uint64_t))); + rlp->n = rl.n; + ret = kvm_vcpu_ioctl(cs, KVM_GET_REG_LIST, rlp); + if (ret) { + fprintf(stderr, "%s: failed to get register list\n", __func__); + return FALSE; + } + /* Sort the list we get back from the kernel, since cpreg_tuples + * must be in strictly ascending order. + */ + qsort(&rlp->reg, rlp->n, sizeof(rlp->reg[0]), compare_u64); + + /* Count how many of these registers we'll actually sync through + * the cpreg_indexes mechanism and overwrite the existing TCG + * built array of registers. + */ + for (i = 0, arraylen = 0; i < rlp->n; i++) { + uint64_t regidx = rlp->reg[i]; + if (reg_syncs_via_tuple_list(regidx)) { + gboolean found = FALSE; + arraylen++; + for (j = 0; j < cpu->cpreg_array_len; j++) { + if (regidx == cpu->cpreg_indexes[j]) { + found = TRUE; + break; + } + } + if (!found) { + qemu_log_mask(LOG_UNIMP, + "%s: TCG missing definition of %"PRIx64"\n", + __func__, regidx); + } + } + } + + cpu->cpreg_indexes = g_renew(uint64_t, cpu->cpreg_indexes, arraylen); + cpu->cpreg_values = g_renew(uint64_t, cpu->cpreg_values, arraylen); + cpu->cpreg_vmstate_indexes = g_renew(uint64_t, cpu->cpreg_vmstate_indexes, + arraylen); + cpu->cpreg_vmstate_values = g_renew(uint64_t, cpu->cpreg_vmstate_values, + arraylen); + cpu->cpreg_array_len = arraylen; + cpu->cpreg_vmstate_array_len = arraylen; + + for (i = 0, arraylen = 0; i < rlp->n; i++) { + uint64_t regidx = rlp->reg[i]; + if (!reg_syncs_via_tuple_list(regidx)) { + continue; + } + switch (regidx & KVM_REG_SIZE_MASK) { + case KVM_REG_SIZE_U32: + case KVM_REG_SIZE_U64: + break; + default: + fprintf(stderr, + "%s: un-handled register size (%"PRIx64") in kernel list\n", + __func__, regidx); + return FALSE; + } + cpu->cpreg_indexes[arraylen] = regidx; + arraylen++; + } + + g_assert(cpu->cpreg_array_len == arraylen); + + return TRUE; +} + bool write_kvmstate_to_list(ARMCPU *cpu) { CPUState *cs = CPU(cpu); diff --git a/target-arm/kvm32.c b/target-arm/kvm32.c index 39117c7..adfc902 100644 --- a/target-arm/kvm32.c +++ b/target-arm/kvm32.c @@ -138,39 +138,11 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUClass *ahcc) return true; } -static bool reg_syncs_via_tuple_list(uint64_t regidx) -{ - /* Return true if the regidx is a register we should synchronize - * via the cpreg_tuples array (ie is not a core reg we sync by - * hand in kvm_arch_get/put_registers()) - */ - switch (regidx & KVM_REG_ARM_COPROC_MASK) { - case KVM_REG_ARM_CORE: - case KVM_REG_ARM_VFP: - return false; - default: - return true; - } -} - -static int compare_u64(const void *a, const void *b) -{ - if (*(uint64_t *)a > *(uint64_t *)b) { - return 1; - } - if (*(uint64_t *)a < *(uint64_t *)b) { - return -1; - } - return 0; -} - int kvm_arch_init_vcpu(CPUState *cs) { - int i, ret, arraylen; + int i, ret; uint64_t v; struct kvm_one_reg r; - struct kvm_reg_list rl; - struct kvm_reg_list *rlp; ARMCPU *cpu = ARM_CPU(cs); if (cpu->kvm_target == QEMU_KVM_ARM_TARGET_NONE) { @@ -206,73 +178,17 @@ int kvm_arch_init_vcpu(CPUState *cs) return -EINVAL; } - /* Populate the cpreg list based on the kernel's idea - * of what registers exist (and throw away the TCG-created list). - */ - rl.n = 0; - ret = kvm_vcpu_ioctl(cs, KVM_GET_REG_LIST, &rl); - if (ret != -E2BIG) { - return ret; - } - rlp = g_malloc(sizeof(struct kvm_reg_list) + rl.n * sizeof(uint64_t)); - rlp->n = rl.n; - ret = kvm_vcpu_ioctl(cs, KVM_GET_REG_LIST, rlp); - if (ret) { - goto out; - } - /* Sort the list we get back from the kernel, since cpreg_tuples - * must be in strictly ascending order. - */ - qsort(&rlp->reg, rlp->n, sizeof(rlp->reg[0]), compare_u64); - - for (i = 0, arraylen = 0; i < rlp->n; i++) { - if (!reg_syncs_via_tuple_list(rlp->reg[i])) { - continue; - } - switch (rlp->reg[i] & KVM_REG_SIZE_MASK) { - case KVM_REG_SIZE_U32: - case KVM_REG_SIZE_U64: - break; - default: - fprintf(stderr, "Can't handle size of register in kernel list\n"); - ret = -EINVAL; - goto out; - } - - arraylen++; - } - - cpu->cpreg_indexes = g_renew(uint64_t, cpu->cpreg_indexes, arraylen); - cpu->cpreg_values = g_renew(uint64_t, cpu->cpreg_values, arraylen); - cpu->cpreg_vmstate_indexes = g_renew(uint64_t, cpu->cpreg_vmstate_indexes, - arraylen); - cpu->cpreg_vmstate_values = g_renew(uint64_t, cpu->cpreg_vmstate_values, - arraylen); - cpu->cpreg_array_len = arraylen; - cpu->cpreg_vmstate_array_len = arraylen; - - for (i = 0, arraylen = 0; i < rlp->n; i++) { - uint64_t regidx = rlp->reg[i]; - if (!reg_syncs_via_tuple_list(regidx)) { - continue; - } - cpu->cpreg_indexes[arraylen] = regidx; - arraylen++; + if (!kvm_arm_sync_register_list(cpu)) { + return -EINVAL; } - assert(cpu->cpreg_array_len == arraylen); if (!write_kvmstate_to_list(cpu)) { /* Shouldn't happen unless kernel is inconsistent about * what registers exist. */ fprintf(stderr, "Initial read of kernel register state failed\n"); - ret = -EINVAL; - goto out; + return -EINVAL; } - -out: - g_free(rlp); - return ret; } typedef struct Reg { diff --git a/target-arm/kvm64.c b/target-arm/kvm64.c index 7a022a6..0e28901 100644 --- a/target-arm/kvm64.c +++ b/target-arm/kvm64.c @@ -102,7 +102,7 @@ int kvm_arch_init_vcpu(CPUState *cs) return ret; } - /* TODO : support for save/restore/reset of system regs via tuple list */ + kvm_arm_sync_register_list(cs); return 0; } diff --git a/target-arm/kvm_arm.h b/target-arm/kvm_arm.h index af93105..2efd0b7 100644 --- a/target-arm/kvm_arm.h +++ b/target-arm/kvm_arm.h @@ -47,6 +47,18 @@ void kvm_arm_register_device(MemoryRegion *mr, uint64_t devid, uint64_t group, uint64_t attr, int dev_fd); /** + * kvm_arm_sync_register_list: + * @cs: CPUState + * + * Before migration can occur we need to sync the list of additional + * registers that KVM knows about which we can then use when we start + * doing migration. It's OK for the TCG side not to know about + * registers exposed by the KVM side although this will break + * migration between the two VM types. + */ +bool kvm_arm_sync_register_list(CPUState *cs); + +/** * write_list_to_kvmstate: * @cpu: ARMCPU *