From patchwork Sat May 23 23:20:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91785C433E0 for ; Sat, 23 May 2020 23:21:41 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F8E820735 for ; Sat, 23 May 2020 23:21:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iZHzwRL5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F8E820735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdSW-00008a-C8 for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:21:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45182) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRe-0006sU-Lp for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:46 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:57447 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRd-0000Ow-MS for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276044; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7Kvi6xzK9MWs5xCm4IUNKTRO2aVL0Wwl9B3a3YZXML8=; b=iZHzwRL5hLE+M9pUAI8ObeMeyFyCIdcrSk5gln10SfcyiTT8gAtbU7XN82GnAQSKzRj4C4 NIvuss3LR+Lrr+3WCZcnq7z4WuIbW9hTrUF3GqRloPngiR8MZXO3whD86tIcJD2BYGhEZo pedAIXWBdEqAc4qNlzqu1jnP23YvW9g= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-187-10Paq6OFMhWdwIzcGiw_MA-1; Sat, 23 May 2020 19:20:43 -0400 X-MC-Unique: 10Paq6OFMhWdwIzcGiw_MA-1 Received: by mail-qt1-f200.google.com with SMTP id o16so15947879qto.12 for ; Sat, 23 May 2020 16:20:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7Kvi6xzK9MWs5xCm4IUNKTRO2aVL0Wwl9B3a3YZXML8=; b=tMZ26sOg9zGdXVHREK3f3FLa+tUjBJsXe92v/Lo+HS6QX9/ISgz5ktPuLMuj6FU8Zw Q/MLT4GvLabHusOWAviHnQyn8KPpmgTzFZqEiLBVlHJYAg1u6ROIsztlR3u2ZbzbdVlk eTGNGVkB44V3ytzLa7qYV9X6zY5fq/dl2BIwY7J53CblEqz64LmxP3p8xC4LqsFDEUuj StvJxmo5IqDSJPSiW+C+qisnA/tTpnRQnlNGyWQ8wi6rwU08epGZJIiG+pa/vWRJftDy Tj0itH2fknU0j8lSBniOnpysJg0NntyGGAZypcDqd0sR9SnwrywmGNN+fpxpwZkUtM/O xMzw== X-Gm-Message-State: AOAM531HhjlhAdUEK1cUbh4Ei1f1xsfm5hz+SI2N3+A1oLDVrIxhJmnV ZtDb3wFqqvQl/o61WdahEXfscd81ICSQdnpxkzyP3Ott4Il3NBZ9YRlxPCK3EJHjg6J1MHYH1jd 1NLAIBu53kbnv+qA= X-Received: by 2002:aed:3009:: with SMTP id 9mr22069073qte.191.1590276042202; Sat, 23 May 2020 16:20:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwWRrzStb2z+qPHYVXam3uw39Xic1d5lduo5dqWB5YefEOq6B+ogqyKtb3yeYFp/cj3VE21ZQ== X-Received: by 2002:aed:3009:: with SMTP id 9mr22069057qte.191.1590276041921; Sat, 23 May 2020 16:20:41 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:41 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 02/11] memory: Introduce log_sync_global() to memory listener Date: Sat, 23 May 2020 19:20:26 -0400 Message-Id: <20200523232035.1029349-3-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 19:20:44 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Some of the memory listener may want to do log synchronization without being able to specify a range of memory to sync but always globally. Such a memory listener should provide this new method instead of the log_sync() method. Obviously we can also achieve similar thing when we put the global sync logic into a log_sync() handler. However that's not efficient enough because otherwise memory_global_dirty_log_sync() may do the global sync N times, where N is the number of flat ranges in the address space. Make this new method be exclusive to log_sync(). Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- include/exec/memory.h | 12 ++++++++++++ memory.c | 33 +++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 10 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index e000bd2f97..c0c6155ca0 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -533,6 +533,18 @@ struct MemoryListener { */ void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section); + /** + * @log_sync_global: + * + * This is the global version of @log_sync when the listener does + * not have a way to synchronize the log with finer granularity. + * When the listener registers with @log_sync_global defined, then + * its @log_sync must be NULL. Vice versa. + * + * @listener: The #MemoryListener. + */ + void (*log_sync_global)(MemoryListener *listener); + /** * @log_clear: * diff --git a/memory.c b/memory.c index 92fb8b80d7..a77c884e8e 100644 --- a/memory.c +++ b/memory.c @@ -2047,6 +2047,10 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr, memory_region_get_dirty_log_mask(mr)); } +/* + * If memory region `mr' is NULL, do global sync. Otherwise, sync + * dirty bitmap for the specified memory region. + */ static void memory_region_sync_dirty_bitmap(MemoryRegion *mr) { MemoryListener *listener; @@ -2060,18 +2064,24 @@ static void memory_region_sync_dirty_bitmap(MemoryRegion *mr) * address space once. */ QTAILQ_FOREACH(listener, &memory_listeners, link) { - if (!listener->log_sync) { - continue; - } - as = listener->address_space; - view = address_space_get_flatview(as); - FOR_EACH_FLAT_RANGE(fr, view) { - if (fr->dirty_log_mask && (!mr || fr->mr == mr)) { - MemoryRegionSection mrs = section_from_flat_range(fr, view); - listener->log_sync(listener, &mrs); + if (listener->log_sync) { + as = listener->address_space; + view = address_space_get_flatview(as); + FOR_EACH_FLAT_RANGE(fr, view) { + if (fr->dirty_log_mask && (!mr || fr->mr == mr)) { + MemoryRegionSection mrs = section_from_flat_range(fr, view); + listener->log_sync(listener, &mrs); + } } + flatview_unref(view); + } else if (listener->log_sync_global) { + /* + * No matter whether MR is specified, what we can do here + * is to do a global sync, because we are not capable to + * sync in a finer granularity. + */ + listener->log_sync_global(listener); } - flatview_unref(view); } } @@ -2758,6 +2768,9 @@ void memory_listener_register(MemoryListener *listener, AddressSpace *as) { MemoryListener *other = NULL; + /* Only one of them can be defined for a listener */ + assert(!(listener->log_sync && listener->log_sync_global)); + listener->address_space = as; if (QTAILQ_EMPTY(&memory_listeners) || listener->priority >= QTAILQ_LAST(&memory_listeners)->priority) { From patchwork Sat May 23 23:20:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C779C433DF for ; Sat, 23 May 2020 23:23:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6778320735 for ; Sat, 23 May 2020 23:23:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BxbeeTMQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6778320735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:57736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdUB-0003hB-Iv for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:23:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRf-0006tQ-Gc for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:47 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:31136 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRe-0000P5-M2 for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZWuKxqHpTHrg9vNDgSvtlSiGvO4lTbyQf2mpVceh26Y=; b=BxbeeTMQF1a2xSwrElNVNTbIz6Vz523MhouSpEStZwFDxBkn0uVADBx4hbrQeYN/NQnJYE YXELD/NDjcACu1/xYXrh+MbbpJ/+KyQnrQorbK/neXHk39ZAnzn6ivhlK7Y5qqeEc5uU9v sD3xhyJK+ZDzxRUsSguPTGowG0/KxWQ= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-GpL6XNN8Pn6QwpFKifHthQ-1; Sat, 23 May 2020 19:20:44 -0400 X-MC-Unique: GpL6XNN8Pn6QwpFKifHthQ-1 Received: by mail-qt1-f200.google.com with SMTP id 19so15963253qtp.8 for ; Sat, 23 May 2020 16:20:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZWuKxqHpTHrg9vNDgSvtlSiGvO4lTbyQf2mpVceh26Y=; b=YHj/a4gdfhEl1F2lDT4A/W4tir+bWI3DtFgpHjUmPUyb/rZPW6h4pjZvHgL1eavP2t LaACZIGNZj9VOnIHEITkBfaZ9eYpdgOOLdHoDfFGdLe+Hl/bbCK4N4QQ032P9tZfXn6V gtp+RdLEYU/eii3TB9tgFjPcoYJqKcS+iYtrxJSy0ihTAlBsyRfpXTl2fM7nGvCVLRjB v/jHHH8mZd8Ji1nNQmD63O1Lz3mTXz153dmDWPX9FGqV0MCHXAOFjMm9QqXfwhpoOxBw Bn0nZwaGpkLDowMh5W3VJHGpFuIf8dBQCo5ibvk05HAZF3vr/gfAJIDXleyVv6x/64c+ CEGQ== X-Gm-Message-State: AOAM532wX9vB53R9n0ykn6FcI/2wH70zQ1HH04YBsHQuqjffYsz+Mt2y nTSWpZdcZUKn9ZxpRTQk4bjUV514EPhPE4sXhiorkVYA7Zo9neFJaIJqPoLBnzu4Wz9TPoA5aF1 mrZv1lM8VVvVSPxs= X-Received: by 2002:a37:a949:: with SMTP id s70mr614400qke.111.1590276043647; Sat, 23 May 2020 16:20:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJygS5o3xF5CKmnHw37C6T1jF25S7GLCcJ78nmO6rTD1dm3nysTeM7J1xR4LmOAvJLjRQOFGwg== X-Received: by 2002:a37:a949:: with SMTP id s70mr614384qke.111.1590276043426; Sat, 23 May 2020 16:20:43 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:42 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 03/11] KVM: Fixup kvm_log_clear_one_slot() ioctl return check Date: Sat, 23 May 2020 19:20:27 -0400 Message-Id: <20200523232035.1029349-4-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 18:49:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" kvm_vm_ioctl() handles the errno trick already for ioctl() on returning -1 for errors. Fix this. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index d06cc04079..6e015aa2d4 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -699,14 +699,13 @@ static int kvm_log_clear_one_slot(KVMSlot *mem, int as_id, uint64_t start, d.num_pages = bmap_npages; d.slot = mem->slot | (as_id << 16); - if (kvm_vm_ioctl(s, KVM_CLEAR_DIRTY_LOG, &d) == -1) { - ret = -errno; + ret = kvm_vm_ioctl(s, KVM_CLEAR_DIRTY_LOG, &d); + if (ret) { error_report("%s: KVM_CLEAR_DIRTY_LOG failed, slot=%d, " "start=0x%"PRIx64", size=0x%"PRIx32", errno=%d", __func__, d.slot, (uint64_t)d.first_page, (uint32_t)d.num_pages, ret); } else { - ret = 0; trace_kvm_clear_dirty_log(d.slot, d.first_page, d.num_pages); } From patchwork Sat May 23 23:20:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A62AC433E0 for ; Sat, 23 May 2020 23:21:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16BE020735 for ; Sat, 23 May 2020 23:21:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DJiiDCWJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16BE020735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:49608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdSa-0000IY-6k for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:21:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45194) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRi-0006y3-7f for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:50 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:59514 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRg-0000PI-97 for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uLJPlWIV69MwDMrC4NVA6RBO4QfOmhlpk5oUjM+e9os=; b=DJiiDCWJVaIVZohFLo8qlk6hrBwehafyxlR2t8CHmcEMDl6U7XybSV9f/evqVlV6TLZqLO QflZ8lE5lpm/peFKdbnD/QzfvxdfTGI/PW8mhqhwswQL+mhAVRJLX2NmsUFOrRxGm25NHP T9g/LJWRVu5hErnC3SPTSRPNlG18ImQ= Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-443-xph7hmGgPlGcGnR3FQrwKQ-1; Sat, 23 May 2020 19:20:46 -0400 X-MC-Unique: xph7hmGgPlGcGnR3FQrwKQ-1 Received: by mail-qv1-f69.google.com with SMTP id da20so4443907qvb.3 for ; Sat, 23 May 2020 16:20:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uLJPlWIV69MwDMrC4NVA6RBO4QfOmhlpk5oUjM+e9os=; b=F1S9XOjWqifccTmQ7qjYh84YU5VDYytadSw5oUxQ/PueOQ8Y/I8PzD+TyhZV0tu2Gd Vp5rQfhxSSwjj+ba7vBn7Fu2x2sjTcRKq1z7y6J6hl9DB1qs/4R96nN4FmKq34bftyya 7aoShTEPnnM9YErEWrLBEKieMIbRFARrH662zr/nlNpEusG6nDwcI/P54Kx6bxCOnqmI WXuqDeFN1Ea43FKQ7/CEzV3MwkoXz+mRRJdff7a641tL77Dt49bcpTX1ol2kVbsJjhtf LJpRP6cBBhOLZt2UcVCkCIFmkTGBbQNzFuCy1pNSttZnFMjVtstPlZ2BNSIriiv9HB8E nHQA== X-Gm-Message-State: AOAM530njQt7++syLdiiTbG1/JXIuMZL9ytouPzuHs3dEn/HtHJcQO/z lSoLCT7aPvGbZ52GPKnT+yoMByvyJ8h9fjTXFVR02DShsh7b8TP+TAig4jtunb3rTPrb1jcX49p 2M+UMfWLH0DHvcX0= X-Received: by 2002:a37:6c8:: with SMTP id 191mr21519363qkg.358.1590276045293; Sat, 23 May 2020 16:20:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw4LpgduQANLxqX9mmMtxBNboKeKQci9AtyBZlTrl8YgxSh8xq2fmGgbZkozs844g/TbxDhcg== X-Received: by 2002:a37:6c8:: with SMTP id 191mr21519353qkg.358.1590276045049; Sat, 23 May 2020 16:20:45 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:44 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 04/11] KVM: Use a big lock to replace per-kml slots_lock Date: Sat, 23 May 2020 19:20:28 -0400 Message-Id: <20200523232035.1029349-5-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 18:49:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Per-kml slots_lock will bring some trouble if we want to take all slots_lock of all the KMLs, especially when we're in a context that we could have taken some of the KML slots_lock, then we even need to figure out what we've taken and what we need to take. Make this simple by merging all KML slots_lock into a single slots lock. Per-kml slots_lock isn't anything that helpful anyway - so far only x86 has two address spaces (so, two slots_locks). All the rest archs will be having one address space always, which means there's actually one slots_lock so it will be the same as before. Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 32 +++++++++++++++++--------------- include/sysemu/kvm_int.h | 2 -- 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 6e015aa2d4..6bdb7909cc 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -160,8 +160,10 @@ static const KVMCapabilityInfo kvm_required_capabilites[] = { static NotifierList kvm_irqchip_change_notifiers = NOTIFIER_LIST_INITIALIZER(kvm_irqchip_change_notifiers); -#define kvm_slots_lock(kml) qemu_mutex_lock(&(kml)->slots_lock) -#define kvm_slots_unlock(kml) qemu_mutex_unlock(&(kml)->slots_lock) +static QemuMutex kml_slots_lock; + +#define kvm_slots_lock() qemu_mutex_lock(&kml_slots_lock) +#define kvm_slots_unlock() qemu_mutex_unlock(&kml_slots_lock) int kvm_get_max_memslots(void) { @@ -211,9 +213,9 @@ bool kvm_has_free_slot(MachineState *ms) bool result; KVMMemoryListener *kml = &s->memory_listener; - kvm_slots_lock(kml); + kvm_slots_lock(); result = !!kvm_get_free_slot(kml); - kvm_slots_unlock(kml); + kvm_slots_unlock(); return result; } @@ -279,7 +281,7 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, KVMMemoryListener *kml = &s->memory_listener; int i, ret = 0; - kvm_slots_lock(kml); + kvm_slots_lock(); for (i = 0; i < s->nr_slots; i++) { KVMSlot *mem = &kml->slots[i]; @@ -289,7 +291,7 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, break; } } - kvm_slots_unlock(kml); + kvm_slots_unlock(); return ret; } @@ -468,7 +470,7 @@ static int kvm_section_update_flags(KVMMemoryListener *kml, return 0; } - kvm_slots_lock(kml); + kvm_slots_lock(); while (size && !ret) { slot_size = MIN(kvm_max_slot_size, size); @@ -484,7 +486,7 @@ static int kvm_section_update_flags(KVMMemoryListener *kml, } out: - kvm_slots_unlock(kml); + kvm_slots_unlock(); return ret; } @@ -754,7 +756,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *kml, return ret; } - kvm_slots_lock(kml); + kvm_slots_lock(); for (i = 0; i < s->nr_slots; i++) { mem = &kml->slots[i]; @@ -780,7 +782,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *kml, } } - kvm_slots_unlock(kml); + kvm_slots_unlock(); return ret; } @@ -1085,7 +1087,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, ram = memory_region_get_ram_ptr(mr) + section->offset_within_region + (start_addr - section->offset_within_address_space); - kvm_slots_lock(kml); + kvm_slots_lock(); if (!add) { do { @@ -1143,7 +1145,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, } while (size); out: - kvm_slots_unlock(kml); + kvm_slots_unlock(); } static void kvm_region_add(MemoryListener *listener, @@ -1170,9 +1172,9 @@ static void kvm_log_sync(MemoryListener *listener, KVMMemoryListener *kml = container_of(listener, KVMMemoryListener, listener); int r; - kvm_slots_lock(kml); + kvm_slots_lock(); r = kvm_physical_sync_dirty_bitmap(kml, section); - kvm_slots_unlock(kml); + kvm_slots_unlock(); if (r < 0) { abort(); } @@ -1272,7 +1274,7 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, { int i; - qemu_mutex_init(&kml->slots_lock); + qemu_mutex_init(&kml_slots_lock); kml->slots = g_malloc0(s->nr_slots * sizeof(KVMSlot)); kml->as_id = as_id; diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index c660a70c51..f143b28671 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -27,8 +27,6 @@ typedef struct KVMSlot typedef struct KVMMemoryListener { MemoryListener listener; - /* Protects the slots and all inside them */ - QemuMutex slots_lock; KVMSlot *slots; int as_id; } KVMMemoryListener; From patchwork Sat May 23 23:20:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3687C433E0 for ; Sat, 23 May 2020 23:23:53 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ADCA520735 for ; Sat, 23 May 2020 23:23:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Tq9cfdWS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADCA520735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:60762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdUe-0004yz-TM for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:23:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRr-0007Jj-2F for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:59 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:42089 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRp-0000Tp-Uk for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cdBTNtboGBy9lg3+b3bykeoaluEQ3dTDeb6ikWfO2cs=; b=Tq9cfdWSX2f8bX5plZNInXH5xjthRzNTJIEpAWf8UZIsLs+tk6YAAN66amxByJ6ReZvElX 3gKSX/HR3gb04knXDB6HQf+a/rm6KWc/iWJ7V12id649m+bQz3OF5SZtcdQR6wZs7R8M+9 cUdWljHqaSGdpt1awzZh1/E+cr7VBNk= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-468-brDNya00PIC1RE-g0x6E9g-1; Sat, 23 May 2020 19:20:53 -0400 X-MC-Unique: brDNya00PIC1RE-g0x6E9g-1 Received: by mail-qk1-f199.google.com with SMTP id n187so2391791qkd.10 for ; Sat, 23 May 2020 16:20:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cdBTNtboGBy9lg3+b3bykeoaluEQ3dTDeb6ikWfO2cs=; b=XBkTBhUyjSU039iSZS46wTaQOwuC0gymeMKfAU4y5NI9vce4ePb0SLGRLVwJ+dDBOr pdmv9X1ZlJfrEqbrgBkrscnMUx7hbk8h7sHyhRHphysIYI2h0w1VZxl6DpPzQ2wAALdi LCMzR5OX8woe/BreY1Ji9+w0JvyaBhe3n0FFSyTzkb6XeHZJ22xT99uT1J4Sbabk3QkW t3PYLfb5RDrQUZFPLiz1D9KQwaNQlM72uGM0dcFRb+tLh93hMqzPgmwUKWr7lkUcgA9p hcWgMcflRAn+VhEzV1HjzitSXOtKoMXI2xFKWuaKctnpVsMGswBxnsNs3BIeeXun3zyU o9dQ== X-Gm-Message-State: AOAM533ZICviqVxLf1n5TF0x4Wcni4InJGV/62Vf0dzQ0sLY5SvCkgYM I14HYg+S3t+gFXkJzbHmJobrAbYfOWWz3TCZYgx61k6Ln9Qn77qcy/3dECCo9bja3apX/yy3UdB mjVsesH2GXqK1AhU= X-Received: by 2002:ac8:768c:: with SMTP id g12mr22221482qtr.51.1590276052794; Sat, 23 May 2020 16:20:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwx/bv8Q9ugZF4DKvnROLq/0qMBy+phWv7UBncJ9ATO7JN5oaUOA3iHvOKy4x0NXOQsTEjwRw== X-Received: by 2002:ac8:768c:: with SMTP id g12mr22221464qtr.51.1590276052564; Sat, 23 May 2020 16:20:52 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:51 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 09/11] KVM: Cache kvm slot dirty bitmap size Date: Sat, 23 May 2020 19:20:33 -0400 Message-Id: <20200523232035.1029349-10-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 19:07:45 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Cache it too because we'll reference it more frequently in the future. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 1 + include/sysemu/kvm_int.h | 1 + 2 files changed, 2 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f7c8e6bebe..b9aaa7912c 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -561,6 +561,7 @@ static void kvm_slot_init_dirty_bitmap(KVMSlot *mem) hwaddr bitmap_size = ALIGN(((mem->memory_size) >> TARGET_PAGE_BITS), /*HOST_LONG_BITS*/ 64) / 8; mem->dirty_bmap = g_malloc0(bitmap_size); + mem->dirty_bmap_size = bitmap_size; } /* Sync dirty bitmap from kernel to KVMSlot.dirty_bmap */ diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 4a35d04478..b4d2886e26 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -23,6 +23,7 @@ typedef struct KVMSlot int old_flags; /* Dirty bitmap cache for the slot */ unsigned long *dirty_bmap; + unsigned long dirty_bmap_size; /* Cache of the address space ID */ int as_id; /* Cache of the offset in ram address space */ From patchwork Sat May 23 23:20:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B74BCC433E0 for ; Sat, 23 May 2020 23:24:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 87B0E20735 for ; Sat, 23 May 2020 23:24:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A0ZPINbM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87B0E20735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36850 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdVZ-0006hD-Oi for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:24:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45230) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRs-0007MX-2V for qemu-devel@nongnu.org; Sat, 23 May 2020 19:21:00 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:51919 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRq-0000UI-MX for qemu-devel@nongnu.org; Sat, 23 May 2020 19:20:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Vf79eVB2XK0J+Fvwx282tsbcI9o5vEmo7tTYQwHKag=; b=A0ZPINbMq3Hh2BWNBblA+xcPmJoKrmjpPAYjPthyaU7L4WnSM/A2qMqvZw3suj4TMq+cre Y2SKnJzxH1mVmpS0/PZBgMXuOg873VD1g2qkOo4RifXS0l2W1XEQmexgqi1RnamUY05Xqq RfPAET8Bv+crUFUSljgkg7Sv5VW2j74= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-377-wwS8s7vsPaSnSq1WOl0ksg-1; Sat, 23 May 2020 19:20:55 -0400 X-MC-Unique: wwS8s7vsPaSnSq1WOl0ksg-1 Received: by mail-qt1-f198.google.com with SMTP id l11so15927963qti.19 for ; Sat, 23 May 2020 16:20:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4Vf79eVB2XK0J+Fvwx282tsbcI9o5vEmo7tTYQwHKag=; b=AgE9W60nxhznuNMkhjWk4qokiyRsU4PxsJV4F3Ocbx1BHwSEDC82RHOw0GOiFVtMeg 8ojCPqoy+NCy5M2AeeztNHB2OpZyJXyetG0LHL6060tyZ2YiRArKnlXztplvsV0wqg2U WjfGPdSq6nwE96N2xlc7Vkowdsk01z7YK8r0wMFs2c4f1t35TZ0TwLWCzf0CV/eVZj3N +UyxM4TY1K/1oEW8/+CdatvK88JdT9CA4XowcDBcwLiYjVN73YIxHwUSSTMdhgo6dfEt h+UKdTlO+UtSuxSLIy0yBGrP980bZPuRQkc7areqCSlRCbhhHq0jlgjKT6JgVkAqb/kl 2Y6w== X-Gm-Message-State: AOAM53254aqaP1l0NR002LAeAuuM7nkHD7Af0zLkgNxAvfyDqAWhv3pa CjXn8asaIv1O41FEZoQFqbXXAYsXHHu6MxiOwTwS+B5TeZ7DUe9H/DXnEsjIBOi+AMhZG09m2dC dGw8mrUxVwZrhTKA= X-Received: by 2002:a0c:b5c1:: with SMTP id o1mr3467975qvf.9.1590276054667; Sat, 23 May 2020 16:20:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyc97PaUnm7AfdbPF9CtfOXDQKZkAKXDj+yOCG8MhfQJFPTEHJ2oyry6t0vcPLK2kwbTTgQnA== X-Received: by 2002:a0c:b5c1:: with SMTP id o1mr3467965qvf.9.1590276054424; Sat, 23 May 2020 16:20:54 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:53 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 10/11] KVM: Add dirty-gfn-count property Date: Sat, 23 May 2020 19:20:34 -0400 Message-Id: <20200523232035.1029349-11-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=peterx@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 18:49:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add a parameter for dirty gfn count for dirty rings. If zero, dirty ring is disabled. Otherwise dirty ring will be enabled with the per-vcpu gfn count as specified. If dirty ring cannot be enabled due to unsupported kernel or illegal parameter, it'll fallback to dirty logging. By default, dirty ring is not enabled (dirty-gfn-count default to 0). Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ qemu-options.hx | 5 ++++ 2 files changed, 77 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index b9aaa7912c..dd017d0720 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -128,6 +128,9 @@ struct KVMState KVMMemoryListener *ml; AddressSpace *as; } *as; + bool kvm_dirty_ring_enabled; /* Whether KVM dirty ring is enabled */ + uint64_t kvm_dirty_ring_size; /* Size of the per-vcpu dirty ring */ + uint32_t kvm_dirty_gfn_count; /* Number of dirty GFNs per ring */ }; KVMState *kvm_state; @@ -2129,6 +2132,40 @@ static int kvm_init(MachineState *ms) s->memory_listener.listener.coalesced_io_add = kvm_coalesce_mmio_region; s->memory_listener.listener.coalesced_io_del = kvm_uncoalesce_mmio_region; + /* + * Enable KVM dirty ring if supported, otherwise fall back to + * dirty logging mode + */ + if (s->kvm_dirty_gfn_count > 0) { + uint64_t ring_size; + + ring_size = s->kvm_dirty_gfn_count * sizeof(struct kvm_dirty_gfn); + + /* Read the max supported pages */ + ret = kvm_vm_check_extension(kvm_state, KVM_CAP_DIRTY_LOG_RING); + if (ret > 0) { + if (ring_size > ret) { + error_report("KVM dirty GFN count %" PRIu32 " too big " + "(maximum is %ld). Please use a smaller value.", + s->kvm_dirty_gfn_count, + ret / sizeof(struct kvm_dirty_gfn)); + ret = -EINVAL; + goto err; + } + + ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_size); + if (ret) { + error_report("Enabling of KVM dirty ring failed: %d. " + "Suggested mininum value is 1024. " + "Please also make sure it's a power of two.", ret); + goto err; + } + + s->kvm_dirty_ring_size = ring_size; + s->kvm_dirty_ring_enabled = true; + } + } + kvm_memory_listener_register(s, &s->memory_listener, &address_space_memory, 0); memory_listener_register(&kvm_io_listener, @@ -3089,6 +3126,33 @@ bool kvm_kernel_irqchip_split(void) return kvm_state->kernel_irqchip_split == ON_OFF_AUTO_ON; } +static void kvm_get_dirty_gfn_count(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + KVMState *s = KVM_STATE(obj); + uint32_t value = s->kvm_dirty_gfn_count; + + visit_type_uint32(v, name, &value, errp); +} + +static void kvm_set_dirty_gfn_count(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + KVMState *s = KVM_STATE(obj); + Error *error = NULL; + uint32_t value; + + visit_type_uint32(v, name, &value, &error); + if (error) { + error_propagate(errp, error); + return; + } + + s->kvm_dirty_gfn_count = value; +} + static void kvm_accel_instance_init(Object *obj) { KVMState *s = KVM_STATE(obj); @@ -3096,6 +3160,8 @@ static void kvm_accel_instance_init(Object *obj) s->kvm_shadow_mem = -1; s->kernel_irqchip_allowed = true; s->kernel_irqchip_split = ON_OFF_AUTO_AUTO; + /* KVM dirty ring is by default off */ + s->kvm_dirty_gfn_count = 0; } static void kvm_accel_class_init(ObjectClass *oc, void *data) @@ -3117,6 +3183,12 @@ static void kvm_accel_class_init(ObjectClass *oc, void *data) NULL, NULL); object_class_property_set_description(oc, "kvm-shadow-mem", "KVM shadow MMU size"); + + object_class_property_add(oc, "dirty-gfn-count", "uint32", + kvm_get_dirty_gfn_count, kvm_set_dirty_gfn_count, + NULL, NULL); + object_class_property_set_description(oc, "dirty-gfn-count", + "KVM dirty GFN count (=0 to disable dirty ring)"); } static const TypeInfo kvm_accel_type = { diff --git a/qemu-options.hx b/qemu-options.hx index 93bde2bbc8..b59d47473e 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -124,6 +124,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel, " kernel-irqchip=on|off|split controls accelerated irqchip support (default=on)\n" " kvm-shadow-mem=size of KVM shadow MMU in bytes\n" " tb-size=n (TCG translation block cache size)\n" + " dirty-gfn-count=n (KVM dirty ring GFN count, default 0)\n" " thread=single|multi (enable multi-threaded TCG)\n", QEMU_ARCH_ALL) SRST ``-accel name[,prop=value[,...]]`` @@ -158,6 +159,10 @@ SRST where both the back-end and front-ends support it and no incompatible TCG features have been enabled (e.g. icount/replay). + + ``dirty-gfn-count=n`` + Controls the per-vcpu KVM dirty ring GFN count (=0 to disable). + ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, From patchwork Sat May 23 23:20:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 282003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0D4BC433E0 for ; Sat, 23 May 2020 23:26:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9035020735 for ; Sat, 23 May 2020 23:26:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HoaUu0BW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9035020735 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcdWh-0000KS-PD for qemu-devel@archiver.kernel.org; Sat, 23 May 2020 19:25:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45236) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcdRu-0007Rc-4I for qemu-devel@nongnu.org; Sat, 23 May 2020 19:21:02 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:52607 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jcdRs-0000Vb-5u for qemu-devel@nongnu.org; Sat, 23 May 2020 19:21:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590276059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=odsL73+Q7FzWnr/03F1vy25OTOWAkSShgRJYyp0wnik=; b=HoaUu0BWt2SgZ+sahbytQxux6/YcQ/lrp+lEQR2sbvvrt5R5M38/VWLGQicOLRUsYj08kD kD5t1V51KPyUg5ikkpkfiI2Iei10BMKz2X1BJqv701ufl3cwChYTORSJDNjvIPd2gxCsGR YnMFQtAxrNOrfokLstVX1Rm+ODSFT/c= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-272-uxiqqkBaO_m9ykJ_L1C8NQ-1; Sat, 23 May 2020 19:20:57 -0400 X-MC-Unique: uxiqqkBaO_m9ykJ_L1C8NQ-1 Received: by mail-qv1-f72.google.com with SMTP id g15so14139207qvx.6 for ; Sat, 23 May 2020 16:20:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=odsL73+Q7FzWnr/03F1vy25OTOWAkSShgRJYyp0wnik=; b=Cyi+JPELLJZGLaf2oA7IVrIZooqQ5TFoFNjZXN18Y9Lc6VY+r/p4UckEkt3ux2gEwT pdNNQBINlB9nNzozHVb250u1LEy8/4YkM78lW1jU9jXRB5RgLw4hWc3zwRSUosXql76a xxQu8ynBQ/k9wznG5neH2DOfkwY5z98iIlPDwKcxrLzbOKjljy/psfsmTv6B1pD2SgqA TxowkVX/PhfHQkDlmjqcd3QReuefavBddEOSUh0WdMBSZgLMVAKTVBVRKNccXIm/9QQY vOfm5f3wyQth4H8L83h/n29FGtQEheAkU1Y0AEYvUt+7ZkcQ+/a1aDsHV8F0mV+26m+7 J5DA== X-Gm-Message-State: AOAM533c19LqwzutKe/91C0TLdO/ovtxyoVeA4NWb6oG71Huf8fPn7sR kf13/rhS2VxLaBJHQBVTEkWzAlAeY2MEFuSMt2NoXfJ3+34JgHtOU/ibfX0JJoXD5gH65wL1ORW CmkBJ1c22JcZMBuw= X-Received: by 2002:a37:5fc4:: with SMTP id t187mr19917305qkb.484.1590276056501; Sat, 23 May 2020 16:20:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzEevV2OsO28OC8loTw1kMaASS5gPyKN6m4v/j1BfJtleDw39WnffPV7aPC3YbNzJ4Xnoq59g== X-Received: by 2002:a37:5fc4:: with SMTP id t187mr19917287qkb.484.1590276056127; Sat, 23 May 2020 16:20:56 -0700 (PDT) Received: from xz-x1.hitronhub.home ([2607:9880:19c0:32::2]) by smtp.gmail.com with ESMTPSA id m33sm11974235qtb.88.2020.05.23.16.20.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 16:20:55 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Subject: [PATCH RFC v3 11/11] KVM: Dirty ring support Date: Sat, 23 May 2020 19:20:35 -0400 Message-Id: <20200523232035.1029349-12-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200523232035.1029349-1-peterx@redhat.com> References: <20200523232035.1029349-1-peterx@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/23 19:20:49 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , "Dr . David Alan Gilbert" , peterx@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" KVM dirty ring is a new interface to pass over dirty bits from kernel to the userspace. Instead of using a bitmap for each memory region, the dirty ring contains an array of dirtied GPAs to fetch (in the form of offset in slots). For each vcpu there will be one dirty ring that binds to it. kvm_dirty_ring_reap() is the major function to collect dirty rings. It can be called either by a standalone reaper thread that runs in the background, collecting dirty pages for the whole VM. It can also be called directly by any thread that has BQL taken. Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 331 ++++++++++++++++++++++++++++++++++++++++- accel/kvm/trace-events | 7 + include/hw/core/cpu.h | 8 + 3 files changed, 343 insertions(+), 3 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index dd017d0720..b064e68040 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -15,6 +15,7 @@ #include "qemu/osdep.h" #include +#include #include @@ -76,6 +77,25 @@ struct KVMParkedVcpu { QLIST_ENTRY(KVMParkedVcpu) node; }; +enum KVMDirtyRingReaperState { + KVM_DIRTY_RING_REAPER_NONE = 0, + /* The reaper is sleeping */ + KVM_DIRTY_RING_REAPER_WAIT, + /* The reaper is reaping for dirty pages */ + KVM_DIRTY_RING_REAPER_REAPING, +}; + +/* + * KVM reaper instance, responsible for collecting the KVM dirty bits + * via the dirty ring. + */ +struct KVMDirtyRingReaper { + /* The reaper thread */ + QemuThread reaper_thr; + volatile uint64_t reaper_iteration; /* iteration number of reaper thr */ + volatile enum KVMDirtyRingReaperState reaper_state; /* reap thr state */ +}; + struct KVMState { AccelState parent_obj; @@ -131,6 +151,7 @@ struct KVMState bool kvm_dirty_ring_enabled; /* Whether KVM dirty ring is enabled */ uint64_t kvm_dirty_ring_size; /* Size of the per-vcpu dirty ring */ uint32_t kvm_dirty_gfn_count; /* Number of dirty GFNs per ring */ + struct KVMDirtyRingReaper reaper; }; KVMState *kvm_state; @@ -362,6 +383,13 @@ int kvm_destroy_vcpu(CPUState *cpu) goto err; } + if (cpu->kvm_dirty_gfns) { + ret = munmap(cpu->kvm_dirty_gfns, s->kvm_dirty_ring_size); + if (ret < 0) { + goto err; + } + } + vcpu = g_malloc0(sizeof(*vcpu)); vcpu->vcpu_id = kvm_arch_vcpu_id(cpu); vcpu->kvm_fd = cpu->kvm_fd; @@ -426,6 +454,19 @@ int kvm_init_vcpu(CPUState *cpu) (void *)cpu->kvm_run + s->coalesced_mmio * PAGE_SIZE; } + if (s->kvm_dirty_ring_enabled) { + /* Use MAP_SHARED to share pages with the kernel */ + cpu->kvm_dirty_gfns = mmap(NULL, s->kvm_dirty_ring_size, + PROT_READ | PROT_WRITE, MAP_SHARED, + cpu->kvm_fd, + PAGE_SIZE * KVM_DIRTY_LOG_PAGE_OFFSET); + if (cpu->kvm_dirty_gfns == MAP_FAILED) { + ret = -errno; + DPRINTF("mmap'ing vcpu dirty gfns failed: %d\n", ret); + goto err; + } + } + ret = kvm_arch_init_vcpu(cpu); err: return ret; @@ -539,6 +580,11 @@ static void kvm_slot_sync_dirty_pages(KVMSlot *slot) cpu_physical_memory_set_dirty_lebitmap(slot->dirty_bmap, start, pages); } +static void kvm_slot_reset_dirty_pages(KVMSlot *slot) +{ + memset(slot->dirty_bmap, 0, slot->dirty_bmap_size); +} + #define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1)) /* Allocate the dirty bitmap for a slot */ @@ -582,6 +628,170 @@ static void kvm_slot_get_dirty_log(KVMState *s, KVMSlot *slot) } } +/* Should be with all slots_lock held for the address spaces. */ +static void kvm_dirty_ring_mark_page(KVMState *s, uint32_t as_id, + uint32_t slot_id, uint64_t offset) +{ + KVMMemoryListener *kml; + KVMSlot *mem; + + if (as_id >= s->nr_as) { + return; + } + + kml = s->as[as_id].ml; + mem = &kml->slots[slot_id]; + + if (!mem->memory_size || offset >= (mem->memory_size / TARGET_PAGE_SIZE)) { + return; + } + + set_bit(offset, mem->dirty_bmap); +} + +static bool dirty_gfn_is_dirtied(struct kvm_dirty_gfn *gfn) +{ + return gfn->flags == KVM_DIRTY_GFN_F_DIRTY; +} + +static void dirty_gfn_set_collected(struct kvm_dirty_gfn *gfn) +{ + gfn->flags = KVM_DIRTY_GFN_F_RESET; +} + +/* + * Should be with all slots_lock held for the address spaces. It returns the + * dirty page we've collected on this dirty ring. + */ +static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu) +{ + struct kvm_dirty_gfn *dirty_gfns = cpu->kvm_dirty_gfns, *cur; + uint32_t gfn_count = s->kvm_dirty_gfn_count; + uint32_t count = 0, fetch = cpu->kvm_fetch_index; + + assert(dirty_gfns && gfn_count); + trace_kvm_dirty_ring_reap_vcpu(cpu->cpu_index); + + while (true) { + cur = &dirty_gfns[fetch % gfn_count]; + if (!dirty_gfn_is_dirtied(cur)) { + break; + } + kvm_dirty_ring_mark_page(s, cur->slot >> 16, cur->slot & 0xffff, + cur->offset); + dirty_gfn_set_collected(cur); + trace_kvm_dirty_ring_page(cpu->cpu_index, fetch, cur->offset); + fetch++; + count++; + } + cpu->kvm_fetch_index = fetch; + + return count; +} + +/* Must be with slots_lock held */ +static uint64_t kvm_dirty_ring_reap_locked(KVMState *s) +{ + int ret; + CPUState *cpu; + uint64_t total = 0; + int64_t stamp; + + stamp = get_clock(); + + CPU_FOREACH(cpu) { + total += kvm_dirty_ring_reap_one(s, cpu); + } + + if (total) { + ret = kvm_vm_ioctl(s, KVM_RESET_DIRTY_RINGS); + assert(ret == total); + } + + stamp = get_clock() - stamp; + + if (total) { + trace_kvm_dirty_ring_reap(total, stamp / 1000); + } + + return total; +} + +/* + * Currently for simplicity, we must hold BQL before calling this. We can + * consider to drop the BQL if we're clear with all the race conditions. + */ +static uint64_t kvm_dirty_ring_reap(KVMState *s) +{ + uint64_t total; + + /* + * We need to lock all kvm slots for all address spaces here, + * because: + * + * (1) We need to mark dirty for dirty bitmaps in multiple slots + * and for tons of pages, so it's better to take the lock here + * once rather than once per page. And more importantly, + * + * (2) We must _NOT_ publish dirty bits to the other threads + * (e.g., the migration thread) via the kvm memory slot dirty + * bitmaps before correctly re-protect those dirtied pages. + * Otherwise we can have potential risk of data corruption if + * the page data is read in the other thread before we do + * reset below. + */ + kvm_slots_lock(); + total = kvm_dirty_ring_reap_locked(s); + kvm_slots_unlock(); + + return total; +} + +static void do_kvm_cpu_synchronize_kick(CPUState *cpu, run_on_cpu_data arg) +{ + /* No need to do anything */ +} + +/* + * Kick all vcpus out in a synchronized way. When returned, we + * guarantee that every vcpu has been kicked and at least returned to + * userspace once. + */ +static void kvm_cpu_synchronize_kick_all(void) +{ + CPUState *cpu; + + CPU_FOREACH(cpu) { + run_on_cpu(cpu, do_kvm_cpu_synchronize_kick, RUN_ON_CPU_NULL); + } +} + +/* + * Flush all the existing dirty pages to the KVM slot buffers. When + * this call returns, we guarantee that all the touched dirty pages + * before calling this function have been put into the per-kvmslot + * dirty bitmap. + * + * This function must be called with BQL held. + */ +static void kvm_dirty_ring_flush(struct KVMDirtyRingReaper *r) +{ + trace_kvm_dirty_ring_flush(0); + /* + * The function needs to be serialized. Since this function + * should always be with BQL held, serialization is guaranteed. + * However, let's be sure of it. + */ + assert(qemu_mutex_iothread_locked()); + /* + * First make sure to flush the hardware buffers by kicking all + * vcpus out in a synchronous way. + */ + kvm_cpu_synchronize_kick_all(); + kvm_dirty_ring_reap(kvm_state); + trace_kvm_dirty_ring_flush(1); +} + /** * kvm_physical_sync_dirty_bitmap - Sync dirty bitmap from kernel space * @@ -1103,7 +1313,24 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, goto out; } if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { - kvm_slot_get_dirty_log(kvm_state, mem); + /* + * NOTE: We should be aware of the fact that here we're only + * doing a best effort to sync dirty bits. No matter whether + * we're using dirty log or dirty ring, we ignored two facts: + * + * (1) dirty bits can reside in hardware buffers (PML) + * + * (2) after we collected dirty bits here, pages can be dirtied + * again before we do the final KVM_SET_USER_MEMORY_REGION to + * remove the slot. + * + * Not easy. Let's cross the fingers until it's fixed. + */ + if (kvm_state->kvm_dirty_ring_enabled) { + kvm_dirty_ring_reap_locked(kvm_state); + } else { + kvm_slot_get_dirty_log(kvm_state, mem); + } kvm_slot_sync_dirty_pages(mem); } @@ -1151,6 +1378,51 @@ out: kvm_slots_unlock(); } +static void *kvm_dirty_ring_reaper_thread(void *data) +{ + KVMState *s = data; + struct KVMDirtyRingReaper *r = &s->reaper; + + rcu_register_thread(); + + trace_kvm_dirty_ring_reaper("init"); + + while (true) { + r->reaper_state = KVM_DIRTY_RING_REAPER_WAIT; + trace_kvm_dirty_ring_reaper("wait"); + /* + * TODO: provide a smarter timeout rather than a constant? + */ + sleep(1); + + trace_kvm_dirty_ring_reaper("wakeup"); + r->reaper_state = KVM_DIRTY_RING_REAPER_REAPING; + + qemu_mutex_lock_iothread(); + kvm_dirty_ring_reap(s); + qemu_mutex_unlock_iothread(); + + r->reaper_iteration++; + } + + trace_kvm_dirty_ring_reaper("exit"); + + rcu_unregister_thread(); + + return NULL; +} + +static int kvm_dirty_ring_reaper_init(KVMState *s) +{ + struct KVMDirtyRingReaper *r = &s->reaper; + + qemu_thread_create(&r->reaper_thr, "kvm-reaper", + kvm_dirty_ring_reaper_thread, + s, QEMU_THREAD_JOINABLE); + + return 0; +} + static void kvm_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -1179,6 +1451,36 @@ static void kvm_log_sync(MemoryListener *listener, kvm_slots_unlock(); } +static void kvm_log_sync_global(MemoryListener *l) +{ + KVMMemoryListener *kml = container_of(l, KVMMemoryListener, listener); + KVMState *s = kvm_state; + KVMSlot *mem; + int i; + + /* Flush all kernel dirty addresses into KVMSlot dirty bitmap */ + kvm_dirty_ring_flush(&s->reaper); + + /* + * TODO: make this faster when nr_slots is big while there are + * only a few used slots (small VMs). + */ + kvm_slots_lock(); + for (i = 0; i < s->nr_slots; i++) { + mem = &kml->slots[i]; + if (mem->memory_size && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { + kvm_slot_sync_dirty_pages(mem); + /* + * This is not needed by KVM_GET_DIRTY_LOG because the + * ioctl will unconditionally overwrite the whole region. + * However kvm dirty ring has no such side effect. + */ + kvm_slot_reset_dirty_pages(mem); + } + } + kvm_slots_unlock(); +} + static void kvm_log_clear(MemoryListener *listener, MemoryRegionSection *section) { @@ -1285,10 +1587,15 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, kml->listener.region_del = kvm_region_del; kml->listener.log_start = kvm_log_start; kml->listener.log_stop = kvm_log_stop; - kml->listener.log_sync = kvm_log_sync; - kml->listener.log_clear = kvm_log_clear; kml->listener.priority = 10; + if (s->kvm_dirty_ring_enabled) { + kml->listener.log_sync_global = kvm_log_sync_global; + } else { + kml->listener.log_sync = kvm_log_sync; + kml->listener.log_clear = kvm_log_clear; + } + memory_listener_register(&kml->listener, as); for (i = 0; i < s->nr_as; ++i) { @@ -2180,6 +2487,13 @@ static int kvm_init(MachineState *ms) qemu_balloon_inhibit(true); } + if (s->kvm_dirty_ring_enabled) { + ret = kvm_dirty_ring_reaper_init(s); + if (ret) { + goto err; + } + } + return 0; err: @@ -2487,6 +2801,17 @@ int kvm_cpu_exec(CPUState *cpu) case KVM_EXIT_INTERNAL_ERROR: ret = kvm_handle_internal_error(cpu, run); break; + case KVM_EXIT_DIRTY_RING_FULL: + /* + * We shouldn't continue if the dirty ring of this vcpu is + * still full. Got kicked by KVM_RESET_DIRTY_RINGS. + */ + trace_kvm_dirty_ring_full(cpu->cpu_index); + qemu_mutex_lock_iothread(); + kvm_dirty_ring_reap(kvm_state); + qemu_mutex_unlock_iothread(); + ret = 0; + break; case KVM_EXIT_SYSTEM_EVENT: switch (run->system_event.type) { case KVM_SYSTEM_EVENT_SHUTDOWN: diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events index 4fb6e59d19..89ef99569f 100644 --- a/accel/kvm/trace-events +++ b/accel/kvm/trace-events @@ -16,4 +16,11 @@ kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_ kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 +kvm_dirty_ring_full(int id) "vcpu %d" +kvm_dirty_ring_reap_vcpu(int id) "vcpu %d" +kvm_dirty_ring_page(int vcpu, uint32_t slot, uint64_t offset) "vcpu %d fetch %"PRIu32" offset 0x%"PRIx64 +kvm_dirty_ring_reaper(const char *s) "%s" +kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)" +kvm_dirty_ring_reaper_kick(const char *reason) "%s" +kvm_dirty_ring_flush(int finished) "%d" diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index a998dc2620..ed8ea59eb5 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -340,6 +340,11 @@ struct qemu_work_item; * @ignore_memory_transaction_failures: Cached copy of the MachineState * flag of the same name: allows the board to suppress calling of the * CPU do_transaction_failed hook function. + * @kvm_dirty_ring_full: + * Whether the kvm dirty ring of this vcpu is soft-full. + * @kvm_dirty_ring_avail: + * Semaphore to be posted when the kvm dirty ring of the vcpu is + * available again. * * State of one CPU core or thread. */ @@ -407,9 +412,12 @@ struct CPUState { */ uintptr_t mem_io_pc; + /* Only used in KVM */ int kvm_fd; struct KVMState *kvm_state; struct kvm_run *kvm_run; + struct kvm_dirty_gfn *kvm_dirty_gfns; + uint32_t kvm_fetch_index; /* Used for events with 'vcpu' and *without* the 'disabled' properties */ DECLARE_BITMAP(trace_dstate_delayed, CPU_TRACE_DSTATE_MAX_EVENTS);