From patchwork Fri Jul 10 05:22:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 347DFC433DF for ; Fri, 10 Jul 2020 05:25:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F053A2077D for ; Fri, 10 Jul 2020 05:25:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WEsolbU+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F053A2077D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:47644 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlXF-00049p-4w for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:25:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUU-0007BU-FW for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:30 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:47384) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUT-0008Bv-1I for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sStIApWiVtCBKiexrQL7OTQwVFLoTBgkZqfp2zgj6T8=; b=WEsolbU+zTby9fE1dGMWwii6mS0jB19RJKYOZBkK2iyPiys4YhGHFp4cNOsU77N/Z/z8ze Z+nUfabT7twCYgc86bfqwEP0K4DX04YMEu/ylr3qxlGPQN21vl7nEUhShrxvEsExiN1rVX RjtyLfsCXmnE2Ua6ofzr6qxXX/sw3hw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-1-5s-UYu8YM7WDSjolnyX09g-1; Fri, 10 Jul 2020 01:22:24 -0400 X-MC-Unique: 5s-UYu8YM7WDSjolnyX09g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 484958027E5; Fri, 10 Jul 2020 05:22:23 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 483AC920CA; Fri, 10 Jul 2020 05:22:22 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 01/16] python/qmp.py: Define common types Date: Fri, 10 Jul 2020 01:22:05 -0400 Message-Id: <20200710052220.3306-2-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/09 23:35:58 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Define some common types that we'll need to annotate a lot of other functions going forward. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/qmp.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index e64b6b5faa..8388c7b603 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -12,13 +12,31 @@ import socket import logging from typing import ( + Any, + Dict, Optional, TextIO, Type, + Tuple, + Union, ) from types import TracebackType +# QMPMessage is a QMP Message of any kind. +# e.g. {'yee': 'haw'} +# +# QMPReturnValue is the inner value of return values only. +# {'return': {}} is the QMPMessage, +# {} is the QMPReturnValue. +QMPMessage = Dict[str, Any] +QMPReturnValue = Dict[str, Any] + +InternetAddrT = Tuple[str, str] +UnixAddrT = str +SocketAddrT = Union[InternetAddrT, UnixAddrT] + + class QMPError(Exception): """ QMP base exception From patchwork Fri Jul 10 05:22:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA24AC433DF for ; Fri, 10 Jul 2020 05:23:47 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A60C220720 for ; Fri, 10 Jul 2020 05:23:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LZ9kWNkF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A60C220720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38984 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlVi-0000dv-VA for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:23:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUW-0007Ew-A2 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:32 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:53037 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUU-0008CO-DV for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z8yNpM8ZFzLT1bZ25F2U4O7aip1EBVa97Zsuy0QTJgw=; b=LZ9kWNkFiDYk69y8+F45tMmhEPuH4HXb8g2/xIr73Ccl9ywvEOSpFNhGFMua43oSIXart6 7FtiOGpobpNJIbmHjjUQYu5la45oIKSbL8Z8aD1cjalWi5Ai0se9JOSzBrPqmOP12pu6QY I1MqaT8udmBrhgdd4ayIn50RDaa+zg8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-452-nFLNDyOnMtmuWAkUzks-WQ-1; Fri, 10 Jul 2020 01:22:25 -0400 X-MC-Unique: nFLNDyOnMtmuWAkUzks-WQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 83EAD91C955; Fri, 10 Jul 2020 05:22:24 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 81A42920CF; Fri, 10 Jul 2020 05:22:23 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 02/16] iotests.py: use qemu.qmp type aliases Date: Fri, 10 Jul 2020 01:22:06 -0400 Message-Id: <20200710052220.3306-3-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:55:31 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" iotests.py should use the type definitions from qmp.py instead of its own. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index f1e0733dda..4457786f09 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -35,13 +35,10 @@ # pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest +from qemu.qmp import QMPMessage assert sys.version_info >= (3, 6) -# Type Aliases -QMPResponse = Dict[str, Any] - - # Use this logger for logging messages directly from the iotests module logger = logging.getLogger('qemu.iotests') logger.addHandler(logging.NullHandler()) @@ -554,7 +551,7 @@ def add_incoming(self, addr): self._args.append(addr) return self - def hmp(self, command_line: str, use_log: bool = False) -> QMPResponse: + def hmp(self, command_line: str, use_log: bool = False) -> QMPMessage: cmd = 'human-monitor-command' kwargs = {'command-line': command_line} if use_log: @@ -575,7 +572,7 @@ def resume_drive(self, drive: str) -> None: self.hmp(f'qemu-io {drive} "remove_break bp_{drive}"') def hmp_qemu_io(self, drive: str, cmd: str, - use_log: bool = False) -> QMPResponse: + use_log: bool = False) -> QMPMessage: """Write to a given drive using an HMP command""" return self.hmp(f'qemu-io {drive} "{cmd}"', use_log=use_log) From patchwork Fri Jul 10 05:22:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0AA7C433E0 for ; Fri, 10 Jul 2020 05:23:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9B85E2076A for ; Fri, 10 Jul 2020 05:23:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ud58ojQL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B85E2076A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlVq-0000zO-Pe for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:23:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUX-0007ID-KQ for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:33 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:22082 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUW-0008DD-30 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5G6lYDMDML75Gjj0qFiqI+XPtpp1YQeLwcWxPKSlt5U=; b=Ud58ojQLLKlyEx+za8mJvLUoHWsf59xAwNL33+bkcEiEE1Jb8CnqdD/h5cXzQVGXqX/Sj6 GkGbHaTxjD0Kj01aoVbahxNa84x9NPGQZDVMXCIDUColvH+r3Sho6YZfgIPgJnnBWP32li M633glyx7SXxdbjbVq5LY/JbWRMDHfw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-325-h40SkHSZMxGIe2XPQOfLvg-1; Fri, 10 Jul 2020 01:22:26 -0400 X-MC-Unique: h40SkHSZMxGIe2XPQOfLvg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B08D08027E7; Fri, 10 Jul 2020 05:22:25 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id AEBF0920CA; Fri, 10 Jul 2020 05:22:24 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 03/16] python/qmp.py: re-absorb MonitorResponseError Date: Fri, 10 Jul 2020 01:22:07 -0400 Message-Id: <20200710052220.3306-4-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:36:21 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When I initially split this out, I considered this more of a machine error than a QMP protocol error, but I think that's misguided. Move this back to qmp.py and name it QMPResponseError. Convert qmp.command() to use this exception type. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf --- python/qemu/machine.py | 15 +-------------- python/qemu/qmp.py | 17 +++++++++++++++-- scripts/render_block_graph.py | 7 +++++-- 3 files changed, 21 insertions(+), 18 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index d08a8e4a6e..8ca3d508df 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -55,19 +55,6 @@ class AbnormalShutdown(QEMUMachineError): """ -class MonitorResponseError(qmp.QMPError): - """ - Represents erroneous QMP monitor reply - """ - def __init__(self, reply): - try: - desc = reply["error"]["desc"] - except KeyError: - desc = reply - super().__init__(desc) - self.reply = reply - - class QEMUMachine: """ A QEMU VM @@ -519,7 +506,7 @@ def command(self, cmd, conv_keys=True, **args): if reply is None: raise qmp.QMPError("Monitor is closed") if "error" in reply: - raise MonitorResponseError(reply) + raise qmp.QMPResponseError(reply) return reply["return"] def get_qmp_event(self, wait=False): diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 8388c7b603..aa8a666b8a 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -61,6 +61,19 @@ class QMPTimeoutError(QMPError): """ +class QMPResponseError(QMPError): + """ + Represents erroneous QMP monitor reply + """ + def __init__(self, reply: QMPMessage): + try: + desc = reply['error']['desc'] + except KeyError: + desc = reply + super().__init__(desc) + self.reply = reply + + class QEMUMonitorProtocol: """ Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then @@ -251,8 +264,8 @@ def command(self, cmd, **kwds): Build and send a QMP command to the monitor, report errors if any """ ret = self.cmd(cmd, kwds) - if "error" in ret: - raise Exception(ret['error']['desc']) + if 'error' in ret: + raise QMPResponseError(ret) return ret['return'] def pull_event(self, wait=False): diff --git a/scripts/render_block_graph.py b/scripts/render_block_graph.py index 409b4321f2..da6acf050d 100755 --- a/scripts/render_block_graph.py +++ b/scripts/render_block_graph.py @@ -25,7 +25,10 @@ from graphviz import Digraph sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'python')) -from qemu.machine import MonitorResponseError +from qemu.qmp import ( + QEMUMonitorProtocol, + QMPResponseError, +) def perm(arr): @@ -102,7 +105,7 @@ def command(self, cmd): reply = json.loads(subprocess.check_output(ar)) if 'error' in reply: - raise MonitorResponseError(reply) + raise QMPResponseError(reply) return reply['return'] From patchwork Fri Jul 10 05:22:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCF0FC433DF for ; Fri, 10 Jul 2020 05:26:45 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 88C8B20720 for ; Fri, 10 Jul 2020 05:26:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H4snuNdr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 88C8B20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlYa-0007d1-PL for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:26:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50660) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUV-0007E9-TR for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:31 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:31704 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUU-0008CQ-71 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358549; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oNhPhaxByI3BTfZupYYGTKC0C0PPGE+d9nTXAs0uWH4=; b=H4snuNdr+thasC9wHjVs1dUWbf3JKdmWbd4UrtnIQHTXzuicMwwRUQzZhj8s83kwrASC0W z9DX6B5S2+lPpuBuwEmZXtmXEiGZoe8aCME/K5+4ekqC2hejEfner6pqW9bSjHwOzho79N pYV8HB+eGRJx1sl8ylKq28pXmxjEiG8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-148-9kKgvxRmMNqybuXdkor2Rg-1; Fri, 10 Jul 2020 01:22:27 -0400 X-MC-Unique: 9kKgvxRmMNqybuXdkor2Rg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB7C9100A614; Fri, 10 Jul 2020 05:22:26 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id DCE65920C2; Fri, 10 Jul 2020 05:22:25 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 04/16] python/qmp.py: Do not return None from cmd_obj Date: Fri, 10 Jul 2020 01:22:08 -0400 Message-Id: <20200710052220.3306-5-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/09 22:08:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This makes typing the qmp library difficult, as it necessitates wrapping Optional[] around the type for every return type up the stack. At some point, it becomes difficult to discern or remember why it's None instead of the expected object. Use the python exception system to tell us exactly why we didn't get an object. Remove this special-cased return. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/qmp.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index aa8a666b8a..ef3c919b76 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -225,22 +225,18 @@ def accept(self, timeout=15.0): self.__sockfile = self.__sock.makefile(mode='r') return self.__negotiate_capabilities() - def cmd_obj(self, qmp_cmd): + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: """ Send a QMP command to the QMP Monitor. @param qmp_cmd: QMP command to be sent as a Python dict - @return QMP response as a Python dict or None if the connection has - been closed + @return QMP response as a Python dict """ self.logger.debug(">>> %s", qmp_cmd) - try: - self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) - except OSError as err: - if err.errno == errno.EPIPE: - return None - raise err + self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) resp = self.__json_read() + if resp is None: + raise QMPConnectError("Unexpected empty reply from server") self.logger.debug("<<< %s", resp) return resp From patchwork Fri Jul 10 05:22:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10671C433E0 for ; Fri, 10 Jul 2020 05:28:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC3FC20720 for ; Fri, 10 Jul 2020 05:28:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="h9JkeUwX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC3FC20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:36064 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlZr-0002bH-3t for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:28:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUY-0007Jv-7v for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:34 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:32321 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUW-0008DW-NT for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358552; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JbxQguf5/Qcyyylen1TvxegiZi7WDnWU8Sf2ZEcWEVg=; b=h9JkeUwXICuDYIwpva5W+CGdk5+45o/e4A8unpEQTWTqPHJKj0XI4JFO42mgChwOWSBmf1 oq3U+TGOxXZFtYykAQMcB5sp8fl3mzD8mAdRSp2o63GnsqDjKgQnGHNc1tBVBXp9G1+Tqc 9kSovEkbczr+JXOnpZFs5H70yqRCq2c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-414-aSmCELV9M82IrMm8XLPCpw-1; Fri, 10 Jul 2020 01:22:30 -0400 X-MC-Unique: aSmCELV9M82IrMm8XLPCpw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 445BF1009440; Fri, 10 Jul 2020 05:22:29 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 401A5920C2; Fri, 10 Jul 2020 05:22:28 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 06/16] python/qmp.py: add QMPProtocolError Date: Fri, 10 Jul 2020 01:22:10 -0400 Message-Id: <20200710052220.3306-7-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:55:31 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" In the case that we receive a reply but are unable to understand it, use this exception name to indicate that case. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf --- python/qemu/qmp.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 1ae36050a4..7935dababb 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -62,6 +62,12 @@ class QMPTimeoutError(QMPError): """ +class QMPProtocolError(QMPError): + """ + QMP protocol error; unexpected response + """ + + class QMPResponseError(QMPError): """ Represents erroneous QMP monitor reply @@ -266,6 +272,10 @@ def command(self, cmd, **kwds): ret = self.cmd(cmd, kwds) if 'error' in ret: raise QMPResponseError(ret) + if 'return' not in ret: + raise QMPProtocolError( + "'return' key not found in QMP response '{}'".format(str(ret)) + ) return cast(QMPReturnValue, ret['return']) def pull_event(self, wait=False): From patchwork Fri Jul 10 05:22:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E826C433E0 for ; Fri, 10 Jul 2020 05:27:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EA62020720 for ; Fri, 10 Jul 2020 05:27:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CSwnUW6i" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EA62020720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:59398 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlZ7-0000cJ-4n for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:27:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUZ-0007OM-L7 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:25359 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUY-0008Dt-08 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358553; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wP4LFX/0gcJwxqvf0SL4qgNKSCoBFyojS/IqLf5vTxE=; b=CSwnUW6islwd7wqqbpoH1AOYIxMjGc7fBWToYNP8uXbm2TMoxNRbZYKbaJUWSiL0pd7G0S 8GbOHW061ZM9LSYnDJ7ZXO2DZRDfXGcy0+93BS4lFKpRkhtKvhkDzSvZFR9i2P0/E6w91j sHBBHzmJClSIpfxyOCf0E6vRZC++1Pc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-398-HTLTiBh5PMiL8I0WoeU8gQ-1; Fri, 10 Jul 2020 01:22:31 -0400 X-MC-Unique: HTLTiBh5PMiL8I0WoeU8gQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 71634800EB6; Fri, 10 Jul 2020 05:22:30 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 702AA920CA; Fri, 10 Jul 2020 05:22:29 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 07/16] python/machine.py: Fix monitor address typing Date: Fri, 10 Jul 2020 01:22:11 -0400 Message-Id: <20200710052220.3306-8-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:55:31 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Prior to this, it's difficult for mypy to intuit what the concrete type of the monitor address is; it has difficulty inferring the type across two variables. Create _monitor_address as a property that always returns a valid address to simplify static type analysis. To preserve our ability to clean up, use a simple boolean to indicate whether or not we should try to clean up the sock file after execution. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/machine.py | 45 +++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 8ca3d508df..bab1b1921f 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -25,10 +25,14 @@ import signal import socket import tempfile -from typing import Optional, Type +from typing import ( + Optional, + Type, +) from types import TracebackType from . import qmp +from .qmp import SocketAddrT LOG = logging.getLogger(__name__) @@ -68,7 +72,8 @@ class QEMUMachine: """ def __init__(self, binary, args=None, wrapper=None, name=None, - test_dir="/var/tmp", monitor_address=None, + test_dir="/var/tmp", + monitor_address: Optional[SocketAddrT] = None, socket_scm_helper=None, sock_dir=None): ''' Initialize a QEMUMachine @@ -91,8 +96,14 @@ def __init__(self, binary, args=None, wrapper=None, name=None, if sock_dir is None: sock_dir = test_dir self._name = name - self._monitor_address = monitor_address - self._vm_monitor = None + if monitor_address is not None: + self._monitor_address = monitor_address + self._remove_monitor_sockfile = False + else: + self._monitor_address = os.path.join( + sock_dir, f"{name}-monitor.sock" + ) + self._remove_monitor_sockfile = True self._qemu_log_path = None self._qemu_log_file = None self._popen = None @@ -231,15 +242,17 @@ def _load_io_log(self): def _base_args(self): args = ['-display', 'none', '-vga', 'none'] + if self._qmp_set: if isinstance(self._monitor_address, tuple): - moncdev = "socket,id=mon,host=%s,port=%s" % ( - self._monitor_address[0], - self._monitor_address[1]) + moncdev = "socket,id=mon,host={},port={}".format( + *self._monitor_address + ) else: - moncdev = 'socket,id=mon,path=%s' % self._vm_monitor + moncdev = f"socket,id=mon,path={self._monitor_address}" args.extend(['-chardev', moncdev, '-mon', 'chardev=mon,mode=control']) + if self._machine is not None: args.extend(['-machine', self._machine]) for _ in range(self._console_index): @@ -264,14 +277,14 @@ def _pre_launch(self): self._qemu_log_file = open(self._qemu_log_path, 'wb') if self._qmp_set: - if self._monitor_address is not None: - self._vm_monitor = self._monitor_address - else: - self._vm_monitor = os.path.join(self._sock_dir, - self._name + "-monitor.sock") - self._remove_files.append(self._vm_monitor) - self._qmp = qmp.QEMUMonitorProtocol(self._vm_monitor, server=True, - nickname=self._name) + if self._remove_monitor_sockfile: + assert isinstance(self._monitor_address, str) + self._remove_files.append(self._monitor_address) + self._qmp = qmp.QEMUMonitorProtocol( + self._monitor_address, + server=True, + nickname=self._name + ) def _post_launch(self): if self._qmp: From patchwork Fri Jul 10 05:22:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62906C433E0 for ; Fri, 10 Jul 2020 05:25:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2C69B20720 for ; Fri, 10 Jul 2020 05:25:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jWRHCCoy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C69B20720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50426 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlXi-0005MN-CJ for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:25:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUe-0007de-8R for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:40 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:45901 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUb-0008Ej-PV for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gt5bdR8o0JAhInkTgPVMvDF+0bcSYyhERKJO3jGGaJE=; b=jWRHCCoyqnUbVX+y+zNo+OPko/mz7EXovWBw9K7/aHrWdAe5S+BYlfM5BDCdVoSn5JwQja Nq31LqKir4CZngVgCyaTyHlmGzrCWK2hLc0k4mqXeF4qUhHnNzLqIiOWk03nUGSdX5HYcx Ea58Z1/InVt0xxPoRjNYoeQyun/Vvd4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-494-vRjlN3tHNc235PiY77egTA-1; Fri, 10 Jul 2020 01:22:34 -0400 X-MC-Unique: vRjlN3tHNc235PiY77egTA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 07B9019057A4; Fri, 10 Jul 2020 05:22:34 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02FC3920CA; Fri, 10 Jul 2020 05:22:32 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 10/16] python/machine.py: Handle None events in events_wait Date: Fri, 10 Jul 2020 01:22:14 -0400 Message-Id: <20200710052220.3306-11-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/09 22:08:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" If the timeout is 0, we can get None back. Handle this explicitly. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/machine.py | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 37e859f6d2..48e70253fa 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -33,7 +33,7 @@ from types import TracebackType from . import qmp -from .qmp import SocketAddrT +from .qmp import SocketAddrT, QMPMessage LOG = logging.getLogger(__name__) @@ -594,13 +594,20 @@ def event_wait(self, name, timeout=60.0, match=None): def events_wait(self, events, timeout=60.0): """ - events_wait waits for and returns a named event - from QMP with a timeout. + events_wait waits for and returns a single named event from QMP. + In the case of multiple qualifying events, this function returns the + first one. - events: a sequence of (name, match_criteria) tuples. - The match criteria are optional and may be None. - See event_match for details. - timeout: QEMUMonitorProtocol.pull_event timeout parameter. + :param events: A sequence of (name, match_criteria) tuples. + The match criteria are optional and may be None. + See event_match for details. + :param timeout: Optional timeout, in seconds. + See QEMUMonitorProtocol.pull_event. + + :raise QMPTimeoutError: If timeout was non-zero and no matching events + were found. + :return: A QMP event matching the filter criteria. + If timeout was 0 and no event matched, None. """ def _match(event): for name, match in events: @@ -608,6 +615,8 @@ def _match(event): return True return False + event: Optional[QMPMessage] + # Search cached events for event in self._events: if _match(event): @@ -617,6 +626,10 @@ def _match(event): # Poll for new events while True: event = self._qmp.pull_event(wait=timeout) + if event is None: + # NB: None is only returned when timeout is false-ish. + # Timeouts raise QMPTimeoutError instead! + break if _match(event): return event self._events.append(event) From patchwork Fri Jul 10 05:22:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81E1AC433E3 for ; Fri, 10 Jul 2020 05:29:11 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51B722077D for ; Fri, 10 Jul 2020 05:29:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IbviGaNe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51B722077D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:40924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlaw-0004cA-L9 for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:29:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50894) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUh-0007mK-4B for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:43 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:47132 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUf-0008Fg-D4 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xy9s+1wdx+fQwdHHZY7/5KndW5zbjdDLC4YYERX1j7M=; b=IbviGaNeC4uIN8A9t9Gk7V6plrQR+UGOyzzdfdYBtnxtZ2AAYbjy9cJMHwPAApNs0sfHkd JUtMklDabAnRjNC+hRVnrZ7E9A62ZMt36RlhLzVjtOMPQ4dxmF6jKhrZ+hS5wL53Vaw8nl HHyi+uWgVR+fSm4H5v66EH46a6RlA7I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-467-279Qz5BWPhC5Lm3wr7NrFw-1; Fri, 10 Jul 2020 01:22:36 -0400 X-MC-Unique: 279Qz5BWPhC5Lm3wr7NrFw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 37FF91082; Fri, 10 Jul 2020 05:22:35 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 33E0379231; Fri, 10 Jul 2020 05:22:34 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 11/16] python/machine.py: use qmp.command Date: Fri, 10 Jul 2020 01:22:15 -0400 Message-Id: <20200710052220.3306-12-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:36:21 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" machine.py and qmp.py both do the same thing here; refactor machine.py to use qmp.py's functionality more directly. Signed-off-by: John Snow Reviewed-by: Kevin Wolf --- python/qemu/machine.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 48e70253fa..0df94c3211 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,8 @@ import socket import tempfile from typing import ( + Any, + Dict, List, Optional, Type, @@ -505,17 +507,23 @@ def set_qmp_monitor(self, enabled=True): self._qmp_set = False self._qmp = None - def qmp(self, cmd, conv_keys=True, **args): - """ - Invoke a QMP command and return the response dict - """ + @classmethod + def _qmp_args(cls, _conv_keys: bool = True, **args: Any) -> Dict[str, Any]: qmp_args = dict() for key, value in args.items(): - if conv_keys: + if _conv_keys: qmp_args[key.replace('_', '-')] = value else: qmp_args[key] = value + return qmp_args + def qmp(self, cmd: str, + conv_keys: bool = True, + **args: Any) -> QMPMessage: + """ + Invoke a QMP command and return the response dict + """ + qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.cmd(cmd, args=qmp_args) def command(self, cmd, conv_keys=True, **args): @@ -524,12 +532,8 @@ def command(self, cmd, conv_keys=True, **args): On success return the response dict. On failure raise an exception. """ - reply = self.qmp(cmd, conv_keys, **args) - if reply is None: - raise qmp.QMPError("Monitor is closed") - if "error" in reply: - raise qmp.QMPResponseError(reply) - return reply["return"] + qmp_args = self._qmp_args(conv_keys, **args) + return self._qmp.command(cmd, **qmp_args) def get_qmp_event(self, wait=False): """ From patchwork Fri Jul 10 05:22:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 278139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7C76C433DF for ; Fri, 10 Jul 2020 05:30:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 734152063A for ; Fri, 10 Jul 2020 05:30:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YBgdpibh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 734152063A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45550 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jtlcJ-0006Yg-NS for qemu-devel@archiver.kernel.org; Fri, 10 Jul 2020 01:30:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50970) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jtlUm-00080O-8O for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:48 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:60308 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jtlUj-0008Gg-04 for qemu-devel@nongnu.org; Fri, 10 Jul 2020 01:22:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594358564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1jl5lTHg0WxX2qj39Z2IQcVuSWcvw32aB6MG4lf3ToQ=; b=YBgdpibhexlUgUSXQJTFpUut4SlNz0a4wvwH41qkE5wekbTp/BywpJhVUUtSTRTN6P3D7b vpU6rr6J8kcffUy0pE2Q8UJeMwhiuniB3Y8goNqp1vYE4pmCFFXUfglwwXEJgHONb7zkTQ fAkgBMD0M7SpEm8zmQx92hS8sLkiLk8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-212-7hQRj5NbNMqR1crxS9Gy_g-1; Fri, 10 Jul 2020 01:22:42 -0400 X-MC-Unique: 7hQRj5NbNMqR1crxS9Gy_g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7B0BC1080; Fri, 10 Jul 2020 05:22:41 +0000 (UTC) Received: from probe.redhat.com (ovpn-118-196.rdu2.redhat.com [10.10.118.196]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5B27E920CA; Fri, 10 Jul 2020 05:22:40 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v5 16/16] python/qemu: Add mypy type annotations Date: Fri, 10 Jul 2020 01:22:20 -0400 Message-Id: <20200710052220.3306-17-jsnow@redhat.com> In-Reply-To: <20200710052220.3306-1-jsnow@redhat.com> References: <20200710052220.3306-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/10 00:55:31 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -40 X-Spam_score: -4.1 X-Spam_bar: ---- X-Spam_report: (-4.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" These should all be purely annotations with no changes in behavior at all. You need to be in the python folder, but you should be able to confirm that these annotations are correct (or at least self-consistent) by running `mypy --strict qemu`. Signed-off-by: John Snow --- python/qemu/accel.py | 8 ++-- python/qemu/machine.py | 94 ++++++++++++++++++++++++------------------ python/qemu/qmp.py | 44 +++++++++++--------- python/qemu/qtest.py | 26 +++++++----- 4 files changed, 98 insertions(+), 74 deletions(-) diff --git a/python/qemu/accel.py b/python/qemu/accel.py index 7fabe62920..4325114e51 100644 --- a/python/qemu/accel.py +++ b/python/qemu/accel.py @@ -17,6 +17,7 @@ import logging import os import subprocess +from typing import List, Optional LOG = logging.getLogger(__name__) @@ -29,7 +30,7 @@ } -def list_accel(qemu_bin): +def list_accel(qemu_bin: str) -> List[str]: """ List accelerators enabled in the QEMU binary. @@ -49,7 +50,8 @@ def list_accel(qemu_bin): return [acc.strip() for acc in out.splitlines()[1:]] -def kvm_available(target_arch=None, qemu_bin=None): +def kvm_available(target_arch: Optional[str] = None, + qemu_bin: Optional[str] = None) -> bool: """ Check if KVM is available using the following heuristic: - Kernel module is present in the host; @@ -72,7 +74,7 @@ def kvm_available(target_arch=None, qemu_bin=None): return True -def tcg_available(qemu_bin): +def tcg_available(qemu_bin: str) -> bool: """ Check if TCG is available. diff --git a/python/qemu/machine.py b/python/qemu/machine.py index f7239a249b..3f0c98987f 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -28,6 +28,7 @@ import tempfile from typing import ( Any, + BinaryIO, Dict, List, Optional, @@ -38,7 +39,7 @@ from types import TracebackType from . import qmp -from .qmp import SocketAddrT, QMPMessage +from .qmp import QMPMessage, QMPReturnValue, SocketAddrT LOG = logging.getLogger(__name__) @@ -67,7 +68,7 @@ class AbnormalShutdown(QEMUMachineError): class QEMUMachine: """ - A QEMU VM + A QEMU VM. Use this object as a context manager to ensure the QEMU process terminates:: @@ -84,7 +85,8 @@ def __init__(self, name: Optional[str] = None, test_dir: str = "/var/tmp", monitor_address: Optional[SocketAddrT] = None, - socket_scm_helper=None, sock_dir=None): + socket_scm_helper: Optional[str] = None, + sock_dir: Optional[str] = None): ''' Initialize a QEMUMachine @@ -118,28 +120,28 @@ def __init__(self, self._remove_monitor_sockfile = True # Runstate - self._qemu_log_path = None - self._qemu_log_file = None + self._qemu_log_path: Optional[str] = None + self._qemu_log_file: Optional[BinaryIO] = None self._popen: Optional['subprocess.Popen[bytes]'] = None - self._events = [] - self._iolog = None + self._events: List[QMPMessage] = [] + self._iolog: Optional[str] = None self._qmp_set = True # Enable QMP monitor by default. self._qmp_connection: Optional[qmp.QEMUMonitorProtocol] = None self._qemu_full_args: Tuple[str, ...] = () - self._temp_dir = None + self._temp_dir: Optional[str] = None self._launched = False - self._machine = None + self._machine: Optional[str] = None self._console_index = 0 self._console_set = False - self._console_device_type = None + self._console_device_type: Optional[str] = None self._console_address = os.path.join( self._sock_dir, f"{self._name}-console.sock" ) - self._console_socket = None - self._remove_files = [] + self._console_socket: Optional[socket.socket] = None + self._remove_files: List[str] = [] self._user_killed = False - def __enter__(self): + def __enter__(self) -> 'QEMUMachine': return self def __exit__(self, @@ -148,14 +150,15 @@ def __exit__(self, exc_tb: Optional[TracebackType]) -> None: self.shutdown() - def add_monitor_null(self): + def add_monitor_null(self) -> None: """ This can be used to add an unused monitor instance. """ self._args.append('-monitor') self._args.append('null') - def add_fd(self, fd, fdset, opaque, opts=''): + def add_fd(self, fd: int, fdset: int, + opaque: str, opts: str = '') -> 'QEMUMachine': """ Pass a file descriptor to the VM """ @@ -174,7 +177,8 @@ def add_fd(self, fd, fdset, opaque, opts=''): self._args.append(','.join(options)) return self - def send_fd_scm(self, fd=None, file_path=None): + def send_fd_scm(self, fd: Optional[int] = None, + file_path: Optional[str] = None) -> int: """ Send an fd or file_path to socket_scm_helper. @@ -218,7 +222,7 @@ def send_fd_scm(self, fd=None, file_path=None): return proc.returncode @staticmethod - def _remove_if_exists(path): + def _remove_if_exists(path: str) -> None: """ Remove file object at path if it exists """ @@ -229,7 +233,7 @@ def _remove_if_exists(path): return raise - def is_running(self): + def is_running(self) -> bool: """Returns true if the VM is running.""" return self._popen is not None and self._popen.poll() is None @@ -239,19 +243,19 @@ def _subp(self) -> 'subprocess.Popen[bytes]': raise QEMUMachineError('Subprocess pipe not present') return self._popen - def exitcode(self): + def exitcode(self) -> Optional[int]: """Returns the exit code if possible, or None.""" if self._popen is None: return None return self._popen.poll() - def get_pid(self): + def get_pid(self) -> Optional[int]: """Returns the PID of the running process, or None.""" if not self.is_running(): return None return self._subp.pid - def _load_io_log(self): + def _load_io_log(self) -> None: if self._qemu_log_path is not None: with open(self._qemu_log_path, "r") as iolog: self._iolog = iolog.read() @@ -285,7 +289,7 @@ def _base_args(self) -> List[str]: args.extend(['-device', device]) return args - def _pre_launch(self): + def _pre_launch(self) -> None: self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) self._qemu_log_path = os.path.join(self._temp_dir, self._name + ".log") self._qemu_log_file = open(self._qemu_log_path, 'wb') @@ -303,11 +307,11 @@ def _pre_launch(self): nickname=self._name ) - def _post_launch(self): + def _post_launch(self) -> None: if self._qmp_connection: self._qmp.accept() - def _post_shutdown(self): + def _post_shutdown(self) -> None: """ Called to cleanup the VM instance after the process has exited. May also be called after a failed launch. @@ -347,7 +351,7 @@ def _post_shutdown(self): self._user_killed = False self._launched = False - def launch(self): + def launch(self) -> None: """ Launch the VM and make sure we cleanup and expose the command line/output in case of exception @@ -371,7 +375,7 @@ def launch(self): LOG.debug('Output: %r', self._iolog) raise - def _launch(self): + def _launch(self) -> None: """ Launch the VM and establish a QMP connection """ @@ -487,7 +491,7 @@ def shutdown(self, has_quit: bool = False, finally: self._post_shutdown() - def kill(self): + def kill(self) -> None: """ Terminate the VM forcefully, wait for it to exit, and perform cleanup. """ @@ -502,7 +506,7 @@ def wait(self, timeout: Optional[int] = 3) -> None: """ self.shutdown(has_quit=True, timeout=timeout) - def set_qmp_monitor(self, enabled=True): + def set_qmp_monitor(self, enabled: bool = True) -> None: """ Set the QMP monitor. @@ -538,7 +542,9 @@ def qmp(self, cmd: str, qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.cmd(cmd, args=qmp_args) - def command(self, cmd, conv_keys=True, **args): + def command(self, cmd: str, + conv_keys: bool = True, + **args: Any) -> QMPReturnValue: """ Invoke a QMP command. On success return the response dict. @@ -547,7 +553,7 @@ def command(self, cmd, conv_keys=True, **args): qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.command(cmd, **qmp_args) - def get_qmp_event(self, wait=False): + def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: """ Poll for one queued QMP events and return it """ @@ -555,7 +561,7 @@ def get_qmp_event(self, wait=False): return self._events.pop(0) return self._qmp.pull_event(wait=wait) - def get_qmp_events(self, wait=False): + def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: """ Poll for queued QMP events and return a list of dicts """ @@ -566,7 +572,7 @@ def get_qmp_events(self, wait=False): return events @staticmethod - def event_match(event, match=None): + def event_match(event: Any, match: Optional[Any]) -> bool: """ Check if an event matches optional match criteria. @@ -596,9 +602,11 @@ def event_match(event, match=None): return True except TypeError: # either match or event wasn't iterable (not a dict) - return match == event + return bool(match == event) - def event_wait(self, name, timeout=60.0, match=None): + def event_wait(self, name: str, + timeout: float = 60.0, + match: Optional[QMPMessage] = None) -> Optional[QMPMessage]: """ event_wait waits for and returns a named event from QMP with a timeout. @@ -608,7 +616,9 @@ def event_wait(self, name, timeout=60.0, match=None): """ return self.events_wait([(name, match)], timeout) - def events_wait(self, events, timeout=60.0): + def events_wait(self, + events: Sequence[Tuple[str, Any]], + timeout: float = 60.0) -> Optional[QMPMessage]: """ events_wait waits for and returns a single named event from QMP. In the case of multiple qualifying events, this function returns the @@ -625,7 +635,7 @@ def events_wait(self, events, timeout=60.0): :return: A QMP event matching the filter criteria. If timeout was 0 and no event matched, None. """ - def _match(event): + def _match(event: QMPMessage) -> bool: for name, match in events: if event['event'] == name and self.event_match(event, match): return True @@ -652,20 +662,20 @@ def _match(event): return None - def get_log(self): + def get_log(self) -> Optional[str]: """ After self.shutdown or failed qemu execution, this returns the output of the qemu process. """ return self._iolog - def add_args(self, *args): + def add_args(self, *args: str) -> None: """ Adds to the list of extra arguments to be given to the QEMU binary """ self._args.extend(args) - def set_machine(self, machine_type): + def set_machine(self, machine_type: str) -> None: """ Sets the machine type @@ -674,7 +684,9 @@ def set_machine(self, machine_type): """ self._machine = machine_type - def set_console(self, device_type=None, console_index=0): + def set_console(self, + device_type: Optional[str] = None, + console_index: int = 0) -> None: """ Sets the device type for a console device @@ -705,7 +717,7 @@ def set_console(self, device_type=None, console_index=0): self._console_index = console_index @property - def console_socket(self): + def console_socket(self) -> socket.socket: """ Returns a socket connected to the console """ diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 7935dababb..303e82ee6b 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -15,6 +15,7 @@ Any, cast, Dict, + List, Optional, TextIO, Type, @@ -90,7 +91,9 @@ class QEMUMonitorProtocol: #: Logger object for debugging messages logger = logging.getLogger('QMP') - def __init__(self, address, server=False, nickname=None): + def __init__(self, address: SocketAddrT, + server: bool = False, + nickname: Optional[str] = None): """ Create a QEMUMonitorProtocol class. @@ -102,7 +105,7 @@ def __init__(self, address, server=False, nickname=None): @note No connection is established, this is done by the connect() or accept() methods """ - self.__events = [] + self.__events: List[QMPMessage] = [] self.__address = address self.__sock = self.__get_sock() self.__sockfile: Optional[TextIO] = None @@ -114,14 +117,14 @@ def __init__(self, address, server=False, nickname=None): self.__sock.bind(self.__address) self.__sock.listen(1) - def __get_sock(self): + def __get_sock(self) -> socket.socket: if isinstance(self.__address, tuple): family = socket.AF_INET else: family = socket.AF_UNIX return socket.socket(family, socket.SOCK_STREAM) - def __negotiate_capabilities(self): + def __negotiate_capabilities(self) -> QMPMessage: greeting = self.__json_read() if greeting is None or "QMP" not in greeting: raise QMPConnectError @@ -131,7 +134,7 @@ def __negotiate_capabilities(self): return greeting raise QMPCapabilitiesError - def __json_read(self, only_event=False): + def __json_read(self, only_event: bool = False) -> Optional[QMPMessage]: assert self.__sockfile is not None while True: data = self.__sockfile.readline() @@ -148,7 +151,7 @@ def __json_read(self, only_event=False): continue return resp - def __get_events(self, wait=False): + def __get_events(self, wait: Union[bool, float] = False) -> None: """ Check for new events in the stream and cache them in __events. @@ -186,7 +189,7 @@ def __get_events(self, wait=False): raise QMPConnectError("Error while reading from socket") self.__sock.settimeout(None) - def __enter__(self): + def __enter__(self) -> 'QEMUMonitorProtocol': # Implement context manager enter function. return self @@ -199,7 +202,7 @@ def __exit__(self, # Implement context manager exit function. self.close() - def connect(self, negotiate=True): + def connect(self, negotiate: bool = True) -> Optional[QMPMessage]: """ Connect to the QMP Monitor and perform capabilities negotiation. @@ -214,7 +217,7 @@ def connect(self, negotiate=True): return self.__negotiate_capabilities() return None - def accept(self, timeout=15.0): + def accept(self, timeout: float = 15.0) -> QMPMessage: """ Await connection from QMP Monitor and perform capabilities negotiation. @@ -250,7 +253,9 @@ def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: self.logger.debug("<<< %s", resp) return resp - def cmd(self, name, args=None, cmd_id=None): + def cmd(self, name: str, + args: Optional[Dict[str, Any]] = None, + cmd_id: Optional[Any] = None) -> QMPMessage: """ Build a QMP command and send it to the QMP Monitor. @@ -258,14 +263,14 @@ def cmd(self, name, args=None, cmd_id=None): @param args: command arguments (dict) @param cmd_id: command id (dict, list, string or int) """ - qmp_cmd = {'execute': name} + qmp_cmd: QMPMessage = {'execute': name} if args: qmp_cmd['arguments'] = args if cmd_id: qmp_cmd['id'] = cmd_id return self.cmd_obj(qmp_cmd) - def command(self, cmd, **kwds): + def command(self, cmd: str, **kwds: Any) -> QMPReturnValue: """ Build and send a QMP command to the monitor, report errors if any """ @@ -278,7 +283,8 @@ def command(self, cmd, **kwds): ) return cast(QMPReturnValue, ret['return']) - def pull_event(self, wait=False): + def pull_event(self, + wait: Union[bool, float] = False) -> Optional[QMPMessage]: """ Pulls a single event. @@ -298,7 +304,7 @@ def pull_event(self, wait=False): return self.__events.pop(0) return None - def get_events(self, wait=False): + def get_events(self, wait: bool = False) -> List[QMPMessage]: """ Get a list of available QMP events. @@ -315,13 +321,13 @@ def get_events(self, wait=False): self.__get_events(wait) return self.__events - def clear_events(self): + def clear_events(self) -> None: """ Clear current list of pending events. """ self.__events = [] - def close(self): + def close(self) -> None: """ Close the socket and socket file. """ @@ -330,7 +336,7 @@ def close(self): if self.__sockfile: self.__sockfile.close() - def settimeout(self, timeout): + def settimeout(self, timeout: float) -> None: """ Set the socket timeout. @@ -339,7 +345,7 @@ def settimeout(self, timeout): """ self.__sock.settimeout(timeout) - def get_sock_fd(self): + def get_sock_fd(self) -> int: """ Get the socket file descriptor. @@ -347,7 +353,7 @@ def get_sock_fd(self): """ return self.__sock.fileno() - def is_scm_available(self): + def is_scm_available(self) -> bool: """ Check if the socket allows for SCM_RIGHTS. diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index ae4661d4d3..49c04bc98d 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -27,6 +27,7 @@ ) from .machine import QEMUMachine +from .qmp import SocketAddrT class QEMUQtestProtocol: @@ -43,7 +44,8 @@ class QEMUQtestProtocol: No conection is estabalished by __init__(), this is done by the connect() or accept() methods. """ - def __init__(self, address, server=False): + def __init__(self, address: SocketAddrT, + server: bool = False): self._address = address self._sock = self._get_sock() self._sockfile: Optional[TextIO] = None @@ -51,14 +53,14 @@ def __init__(self, address, server=False): self._sock.bind(self._address) self._sock.listen(1) - def _get_sock(self): + def _get_sock(self) -> socket.socket: if isinstance(self._address, tuple): family = socket.AF_INET else: family = socket.AF_UNIX return socket.socket(family, socket.SOCK_STREAM) - def connect(self): + def connect(self) -> None: """ Connect to the qtest socket. @@ -67,7 +69,7 @@ def connect(self): self._sock.connect(self._address) self._sockfile = self._sock.makefile(mode='r') - def accept(self): + def accept(self) -> None: """ Await connection from QEMU. @@ -76,7 +78,7 @@ def accept(self): self._sock, _ = self._sock.accept() self._sockfile = self._sock.makefile(mode='r') - def cmd(self, qtest_cmd): + def cmd(self, qtest_cmd: str) -> str: """ Send a qtest command on the wire. @@ -87,14 +89,16 @@ def cmd(self, qtest_cmd): resp = self._sockfile.readline() return resp - def close(self): - """Close this socket.""" + def close(self) -> None: + """ + Close this socket. + """ self._sock.close() if self._sockfile: self._sockfile.close() self._sockfile = None - def settimeout(self, timeout): + def settimeout(self, timeout: Optional[float]) -> None: """Set a timeout, in seconds.""" self._sock.settimeout(timeout) @@ -118,7 +122,7 @@ def __init__(self, super().__init__(binary, args, name=name, test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir) - self._qtest = None + self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") @property @@ -130,7 +134,7 @@ def _base_args(self) -> List[str]: ]) return args - def _pre_launch(self): + def _pre_launch(self) -> None: super()._pre_launch() self._qtest = QEMUQtestProtocol(self._qtest_path, server=True) @@ -139,7 +143,7 @@ def _post_launch(self) -> None: super()._post_launch() self._qtest.accept() - def _post_shutdown(self): + def _post_shutdown(self) -> None: super()._post_shutdown() self._remove_if_exists(self._qtest_path)