From patchwork Tue Oct 6 07:29:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 271997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 372C3C41604 for ; Tue, 6 Oct 2020 07:39:14 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7216620760 for ; Tue, 6 Oct 2020 07:39:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KAwtw5e8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7216620760 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36600 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kPhZ2-0006ID-FA for qemu-devel@archiver.kernel.org; Tue, 06 Oct 2020 03:39:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39502) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kPhQ3-0002LV-7B for qemu-devel@nongnu.org; Tue, 06 Oct 2020 03:29:55 -0400 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]:38413) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kPhQ1-0001Nk-9V for qemu-devel@nongnu.org; Tue, 06 Oct 2020 03:29:54 -0400 Received: by mail-wr1-x42e.google.com with SMTP id n18so4414724wrs.5 for ; Tue, 06 Oct 2020 00:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XKuqPTs8be++xZGyvL7rQyJ9PUjeBH7cxBsMq1MMFoo=; b=KAwtw5e8iOJ5mamSKH4eEwV418VOO21t7V8aM5zrfZ9rnx3dV8L6nvjJdbcOOv+w6E 58j/+UceMV5zvnmz89fYomJquCZwh2mLEepumiIUFQOrLAt1EAdxsQONMw6xM2MjNwiK owQia7Eq42mgSiIxX5rVp+fh7DRQrW/uD4rXpexy7WUlAS8quahL+aCNtFaSqxrvqhC2 8Dda1aM9VaoUPUuRx3KI3Nkt1HOrMsyuIGSKZvZLUuoF6ncbYT+pwKtVWQwzrWuN60MX xoFEH44UWyr7Yfro4S/E6+rcGjyA5boA1QBzPkO9Ii08DCAmq6sH/CJYcJrSanGERi+z vsKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=XKuqPTs8be++xZGyvL7rQyJ9PUjeBH7cxBsMq1MMFoo=; b=mGjyR8D8VNesrDnsb00AT6XY4jPyUg4mbn4P1Bv3N+vPxHXCFgF4SSGfldQ3q8CUVM Nt9wx19II+WNpHOqumLHCYFr5Z5H3QtV1wkpaPNPscKiPUhe7D33BOZlDtNbx10Eimub MeBb23RcN7IEGep/7+GHNKTWjzmmmMDh9kH/IVRAroiiauLh4WgG/mLu4naHexOa9ep5 R2LpfEqfau5530/sDRieNb02NKSjhvqcOnTSfvxnqWfm0SeB2Vy3P65XwVHv1zNNg6d3 czxDFhr93dD/OSk4J+vv5pMNIme52Wyo0C0TwaQhw8tFCKV/eowe4LU/V19aYF0p6G5f Ihpw== X-Gm-Message-State: AOAM5330yY1FR4dKw4KLXYTHBG4zzF/RvfQkNV7+3nkTDie575175sMw mLb3N71kT6jxsOv585zxA8mvJ278TWg= X-Google-Smtp-Source: ABdhPJwidL0VhdJBOl6aHNG5w1I79tqZ1w/Q0t4nSPklQWpJaIFyzMYp5zZEODxc7TNFKsW3myktzw== X-Received: by 2002:adf:dd89:: with SMTP id x9mr3555444wrl.25.1601969391559; Tue, 06 Oct 2020 00:29:51 -0700 (PDT) Received: from localhost.localdomain ([2001:b07:6468:f312:5aca:cd0b:c4a1:9c2e]) by smtp.gmail.com with ESMTPSA id i9sm2576329wma.47.2020.10.06.00.29.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Oct 2020 00:29:51 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Subject: [PULL 06/37] cpus: extract out kvm-specific code to accel/kvm Date: Tue, 6 Oct 2020 09:29:16 +0200 Message-Id: <20201006072947.487729-7-pbonzini@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20201006072947.487729-1-pbonzini@redhat.com> References: <20201006072947.487729-1-pbonzini@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::42e; envelope-from=paolo.bonzini@gmail.com; helo=mail-wr1-x42e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.248, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Richard Henderson , =?utf-8?q?Philippe_M?= =?utf-8?q?athieu-Daud=C3=A9?= , Claudio Fontana Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Claudio Fontana register a "CpusAccel" interface for KVM as well. Signed-off-by: Claudio Fontana Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé [added const] Signed-off-by: Claudio Fontana Reviewed-by: Richard Henderson Signed-off-by: Paolo Bonzini --- accel/kvm/kvm-all.c | 14 ++++++- accel/kvm/kvm-cpus.c | 88 ++++++++++++++++++++++++++++++++++++++++++ accel/kvm/kvm-cpus.h | 17 ++++++++ accel/kvm/meson.build | 5 ++- accel/stubs/kvm-stub.c | 3 +- include/sysemu/kvm.h | 2 +- softmmu/cpus.c | 77 ------------------------------------ 7 files changed, 124 insertions(+), 82 deletions(-) create mode 100644 accel/kvm/kvm-cpus.c create mode 100644 accel/kvm/kvm-cpus.h diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index e4bbf78366..784e9dad55 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -44,6 +44,9 @@ #include "qapi/qapi-types-common.h" #include "qapi/qapi-visit-common.h" #include "sysemu/reset.h" +#include "qemu/guest-random.h" +#include "sysemu/hw_accel.h" +#include "kvm-cpus.h" #include "hw/boards.h" @@ -378,7 +381,7 @@ err: return ret; } -int kvm_destroy_vcpu(CPUState *cpu) +static int do_kvm_destroy_vcpu(CPUState *cpu) { KVMState *s = kvm_state; long mmap_size; @@ -412,6 +415,14 @@ err: return ret; } +void kvm_destroy_vcpu(CPUState *cpu) +{ + if (do_kvm_destroy_vcpu(cpu) < 0) { + error_report("kvm_destroy_vcpu failed"); + exit(EXIT_FAILURE); + } +} + static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id) { struct KVMParkedVcpu *cpu; @@ -2232,6 +2243,7 @@ static int kvm_init(MachineState *ms) assert(!ret); } + cpus_register_accel(&kvm_cpus); return 0; err: diff --git a/accel/kvm/kvm-cpus.c b/accel/kvm/kvm-cpus.c new file mode 100644 index 0000000000..a120601564 --- /dev/null +++ b/accel/kvm/kvm-cpus.c @@ -0,0 +1,88 @@ +/* + * QEMU KVM support + * + * Copyright IBM, Corp. 2008 + * Red Hat, Inc. 2008 + * + * Authors: + * Anthony Liguori + * Glauber Costa + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "sysemu/kvm_int.h" +#include "sysemu/runstate.h" +#include "sysemu/cpus.h" +#include "qemu/guest-random.h" + +#include "kvm-cpus.h" + +static void *kvm_vcpu_thread_fn(void *arg) +{ + CPUState *cpu = arg; + int r; + + rcu_register_thread(); + + qemu_mutex_lock_iothread(); + qemu_thread_get_self(cpu->thread); + cpu->thread_id = qemu_get_thread_id(); + cpu->can_do_io = 1; + current_cpu = cpu; + + r = kvm_init_vcpu(cpu); + if (r < 0) { + error_report("kvm_init_vcpu failed: %s", strerror(-r)); + exit(1); + } + + kvm_init_cpu_signals(cpu); + + /* signal CPU creation */ + cpu_thread_signal_created(cpu); + qemu_guest_random_seed_thread_part2(cpu->random_seed); + + do { + if (cpu_can_run(cpu)) { + r = kvm_cpu_exec(cpu); + if (r == EXCP_DEBUG) { + cpu_handle_guest_debug(cpu); + } + } + qemu_wait_io_event(cpu); + } while (!cpu->unplug || cpu_can_run(cpu)); + + kvm_destroy_vcpu(cpu); + cpu_thread_signal_destroyed(cpu); + qemu_mutex_unlock_iothread(); + rcu_unregister_thread(); + return NULL; +} + +static void kvm_start_vcpu_thread(CPUState *cpu) +{ + char thread_name[VCPU_THREAD_NAME_SIZE]; + + cpu->thread = g_malloc0(sizeof(QemuThread)); + cpu->halt_cond = g_malloc0(sizeof(QemuCond)); + qemu_cond_init(cpu->halt_cond); + snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/KVM", + cpu->cpu_index); + qemu_thread_create(cpu->thread, thread_name, kvm_vcpu_thread_fn, + cpu, QEMU_THREAD_JOINABLE); +} + +const CpusAccel kvm_cpus = { + .create_vcpu_thread = kvm_start_vcpu_thread, + + .synchronize_post_reset = kvm_cpu_synchronize_post_reset, + .synchronize_post_init = kvm_cpu_synchronize_post_init, + .synchronize_state = kvm_cpu_synchronize_state, + .synchronize_pre_loadvm = kvm_cpu_synchronize_pre_loadvm, +}; diff --git a/accel/kvm/kvm-cpus.h b/accel/kvm/kvm-cpus.h new file mode 100644 index 0000000000..547fbee111 --- /dev/null +++ b/accel/kvm/kvm-cpus.h @@ -0,0 +1,17 @@ +/* + * Accelerator CPUS Interface + * + * Copyright 2020 SUSE LLC + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef KVM_CPUS_H +#define KVM_CPUS_H + +#include "sysemu/cpus.h" + +extern const CpusAccel kvm_cpus; + +#endif /* KVM_CPUS_H */ diff --git a/accel/kvm/meson.build b/accel/kvm/meson.build index 4db2388e2f..7e9dafe24c 100644 --- a/accel/kvm/meson.build +++ b/accel/kvm/meson.build @@ -1,5 +1,8 @@ kvm_ss = ss.source_set() -kvm_ss.add(files('kvm-all.c')) +kvm_ss.add(files( + 'kvm-all.c', + 'kvm-cpus.c', +)) kvm_ss.add(when: 'CONFIG_SEV', if_false: files('sev-stub.c')) specific_ss.add_all(when: 'CONFIG_KVM', if_true: kvm_ss) diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c index 82f118d2df..69f8a842da 100644 --- a/accel/stubs/kvm-stub.c +++ b/accel/stubs/kvm-stub.c @@ -32,9 +32,8 @@ bool kvm_readonly_mem_allowed; bool kvm_ioeventfd_any_length_allowed; bool kvm_msi_use_devid; -int kvm_destroy_vcpu(CPUState *cpu) +void kvm_destroy_vcpu(CPUState *cpu) { - return -ENOSYS; } int kvm_init_vcpu(CPUState *cpu) diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 5bbea53883..07937454d2 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -225,7 +225,7 @@ int kvm_has_intx_set_mask(void); int kvm_init_vcpu(CPUState *cpu); int kvm_cpu_exec(CPUState *cpu); -int kvm_destroy_vcpu(CPUState *cpu); +void kvm_destroy_vcpu(CPUState *cpu); /** * kvm_arm_supports_user_irq diff --git a/softmmu/cpus.c b/softmmu/cpus.c index b14b22161f..9fa73735a2 100644 --- a/softmmu/cpus.c +++ b/softmmu/cpus.c @@ -179,9 +179,6 @@ void cpu_synchronize_state(CPUState *cpu) if (cpus_accel && cpus_accel->synchronize_state) { cpus_accel->synchronize_state(cpu); } - if (kvm_enabled()) { - kvm_cpu_synchronize_state(cpu); - } if (hax_enabled()) { hax_cpu_synchronize_state(cpu); } @@ -195,9 +192,6 @@ void cpu_synchronize_post_reset(CPUState *cpu) if (cpus_accel && cpus_accel->synchronize_post_reset) { cpus_accel->synchronize_post_reset(cpu); } - if (kvm_enabled()) { - kvm_cpu_synchronize_post_reset(cpu); - } if (hax_enabled()) { hax_cpu_synchronize_post_reset(cpu); } @@ -211,9 +205,6 @@ void cpu_synchronize_post_init(CPUState *cpu) if (cpus_accel && cpus_accel->synchronize_post_init) { cpus_accel->synchronize_post_init(cpu); } - if (kvm_enabled()) { - kvm_cpu_synchronize_post_init(cpu); - } if (hax_enabled()) { hax_cpu_synchronize_post_init(cpu); } @@ -227,9 +218,6 @@ void cpu_synchronize_pre_loadvm(CPUState *cpu) if (cpus_accel && cpus_accel->synchronize_pre_loadvm) { cpus_accel->synchronize_pre_loadvm(cpu); } - if (kvm_enabled()) { - kvm_cpu_synchronize_pre_loadvm(cpu); - } if (hax_enabled()) { hax_cpu_synchronize_pre_loadvm(cpu); } @@ -384,14 +372,6 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data) do_run_on_cpu(cpu, func, data, &qemu_global_mutex); } -static void qemu_kvm_destroy_vcpu(CPUState *cpu) -{ - if (kvm_destroy_vcpu(cpu) < 0) { - error_report("kvm_destroy_vcpu failed"); - exit(EXIT_FAILURE); - } -} - static void qemu_cpu_stop(CPUState *cpu, bool exit) { g_assert(qemu_cpu_is_self(cpu)); @@ -436,48 +416,6 @@ void qemu_wait_io_event(CPUState *cpu) qemu_wait_io_event_common(cpu); } -static void *qemu_kvm_cpu_thread_fn(void *arg) -{ - CPUState *cpu = arg; - int r; - - rcu_register_thread(); - - qemu_mutex_lock_iothread(); - qemu_thread_get_self(cpu->thread); - cpu->thread_id = qemu_get_thread_id(); - cpu->can_do_io = 1; - current_cpu = cpu; - - r = kvm_init_vcpu(cpu); - if (r < 0) { - error_report("kvm_init_vcpu failed: %s", strerror(-r)); - exit(1); - } - - kvm_init_cpu_signals(cpu); - - /* signal CPU creation */ - cpu_thread_signal_created(cpu); - qemu_guest_random_seed_thread_part2(cpu->random_seed); - - do { - if (cpu_can_run(cpu)) { - r = kvm_cpu_exec(cpu); - if (r == EXCP_DEBUG) { - cpu_handle_guest_debug(cpu); - } - } - qemu_wait_io_event(cpu); - } while (!cpu->unplug || cpu_can_run(cpu)); - - qemu_kvm_destroy_vcpu(cpu); - cpu_thread_signal_destroyed(cpu); - qemu_mutex_unlock_iothread(); - rcu_unregister_thread(); - return NULL; -} - static void *qemu_hax_cpu_thread_fn(void *arg) { CPUState *cpu = arg; @@ -801,19 +739,6 @@ static void qemu_hax_start_vcpu(CPUState *cpu) #endif } -static void qemu_kvm_start_vcpu(CPUState *cpu) -{ - char thread_name[VCPU_THREAD_NAME_SIZE]; - - cpu->thread = g_malloc0(sizeof(QemuThread)); - cpu->halt_cond = g_malloc0(sizeof(QemuCond)); - qemu_cond_init(cpu->halt_cond); - snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/KVM", - cpu->cpu_index); - qemu_thread_create(cpu->thread, thread_name, qemu_kvm_cpu_thread_fn, - cpu, QEMU_THREAD_JOINABLE); -} - static void qemu_hvf_start_vcpu(CPUState *cpu) { char thread_name[VCPU_THREAD_NAME_SIZE]; @@ -875,8 +800,6 @@ void qemu_init_vcpu(CPUState *cpu) if (cpus_accel) { /* accelerator already implements the CpusAccel interface */ cpus_accel->create_vcpu_thread(cpu); - } else if (kvm_enabled()) { - qemu_kvm_start_vcpu(cpu); } else if (hax_enabled()) { qemu_hax_start_vcpu(cpu); } else if (hvf_enabled()) {