From patchwork Thu Sep 14 15:54:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 722547 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp469816wro; Thu, 14 Sep 2023 08:55:26 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHMyBNNEftOXcLiMk9kDzVlyQ2nXVINFH3Hi8uflzsMrjUS1r+WRlWrBk7IMoP0uGydkjRe X-Received: by 2002:ac8:7dd5:0:b0:40f:ba4c:6222 with SMTP id c21-20020ac87dd5000000b0040fba4c6222mr6735379qte.3.1694706926691; Thu, 14 Sep 2023 08:55:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694706926; cv=none; d=google.com; s=arc-20160816; b=OQIYQKVKC+rPvkz01is0y8PgDYvx83bxSyM+8Cc73PkC3Up5ZXs8CCSodHGM7j743e xi+frXUDj8kbyg+yIvJuonGXL3K0eu1+fd1GN2Tayn82MzzU++VVOYt3Wav7VX4F2aaq OQOipXDcTC7hPurDInnId+qcOUBlYz02x1mygAg32xOAUFJgBNdMBnYK4AtlzkiRuJSn lK7pHDE6JbU5S4qoFSuxlv1zC4BGfgDhicrM9mkw74FiXw4NYDxEps3+z/hiKtwrklMM 0bKGLDRbD3f3NZBWZ1leNP+X7URRQFjurmbRokYrCeNqmGaG//GHHXhDMoUbPwCQPKqF 0ckw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Aq8NmlF/vcFodf5p9FXdk6/zlLfm5raaYFWHiGtXyA8=; fh=WYeIlKG36NqBdj35jV78K5CKcUBEmxD++xP1r1Z3+mk=; b=An0YCaGwSnPc39GH8kEyK8OoCD3C33uy8f8jdQ8ytfmKUeOTMdUTG599/bBvXwkxlK s6slazT746XsoO9ltv43JC2fVCNjVCHUpwqK9W6blLG+OJovqlrKd55MdpqVHj6XT+Zy +trKI5iFhEHvx60R2erq4KvwtAnNJxHNU10g+DexNGVzbFzFx3cRc9itFPVm5ZiSDWH3 iNCemuOG1SrnSdhN9wQzoNmbCfYtWXMt977HvpsjO29pzCTeLXB+1Ef9jQkH3pwcOnnJ 56AIZMRugARQg+H9NuwO2+IQ2Bvtyb9K8MY4y+zWMJUnKG5MYNrmKPXTB8mNLPNxHVBK qDyw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jCzCbeZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f22-20020ac859d6000000b00403b21c69c3si1103820qtf.214.2023.09.14.08.55.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Sep 2023 08:55:26 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jCzCbeZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qgofz-0001hn-Bd; Thu, 14 Sep 2023 11:54:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qgoft-0001VA-A6 for qemu-devel@nongnu.org; Thu, 14 Sep 2023 11:54:39 -0400 Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qgofl-00088P-VH for qemu-devel@nongnu.org; Thu, 14 Sep 2023 11:54:36 -0400 Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-402d499580dso11976005e9.1 for ; Thu, 14 Sep 2023 08:54:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694706868; x=1695311668; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Aq8NmlF/vcFodf5p9FXdk6/zlLfm5raaYFWHiGtXyA8=; b=jCzCbeZtub8O++9prSZ3QgpCZ/PF/ajpHTGmuuKIR959ZzDGngVy3aQLyYRuED8Y0X p910tFDcQm0b2yfIVNfKo2jj+bm/Y7fz/C5WnBtKnoVIZDqxreMPqLfAzzceZC7PY4uf 1fyAXAFtWcyY2oOpWTQRk83ueP662Ah0DZo0p6ClfdRmaLZQh5GDxBfFkRT8ZxO2i4tS LNJd41BLEToIf7/f+jB28yecxH23gGNlA/W03ObF8o9dMocBve4YiEgyX2bYFApqdEli xNwAe7qwRU/CgrvYX3eqQKIrb+R8WZFjkzMFor/ItlElnbRGv3crwsrtItzUXN1ZmMq3 FPlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694706868; x=1695311668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Aq8NmlF/vcFodf5p9FXdk6/zlLfm5raaYFWHiGtXyA8=; b=mK2PDYhkbxsmdE0pm1IpsEqdSxGpkjyq2XsCsB43JHW9exBUW7WLsu1MHNNAUpJjeF Gjx4eBnnjz5lbH3EmaDHX1A4Pj2/QJv54V0mdmq6688oY6/jjhkVTWPWkT9gWbwghy9l A/Jfddee7r8aoa7Q0MbIqBpo9p0LrZ+FzjDhFrplJTxm4M0UIXwbCe4/+sVJuvLp0t8d dc7BJDHFHk2UW5pUkZkw5zBRGEbac7C2GlA6vj2CevJhyIyqabo7NRKawSTEItckv6Ug Rfd5Z9vivmiy/Uz4aPePMMrWNe5dQYfE8bBDmuB6W63j/BPSSqwv2CWWnQ01s+YrVb1j RDpQ== X-Gm-Message-State: AOJu0YxRxmcQQ2HRyaAq6lHrO/9HuJREUy7t/LXe6PM9YzB3CY4o8Ybz VELkfKPirp8+/ZTPEdtwfe0UIw== X-Received: by 2002:a05:600c:210f:b0:403:be16:baa with SMTP id u15-20020a05600c210f00b00403be160baamr4972026wml.29.1694706867810; Thu, 14 Sep 2023 08:54:27 -0700 (PDT) Received: from zen.linaroharston ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id s23-20020a7bc397000000b00403bbe69629sm2299139wmj.31.2023.09.14.08.54.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Sep 2023 08:54:26 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 4F3A11FFC3; Thu, 14 Sep 2023 16:54:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , Cleber Rosa , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Joel Stanley , Wainer dos Santos Moschetta , Beraldo Leal , Richard Henderson , Pavel Dovgalyuk , qemu-arm@nongnu.org, Thomas Huth , Jiaxun Yang , John Snow , Mark Cave-Ayland , Peter Maydell , Laurent Vivier , Aurelien Jarno , =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , Paolo Bonzini , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Markus Armbruster , Nicholas Piggin Subject: [PATCH 8/9] tests/avocado: Fix console data loss Date: Thu, 14 Sep 2023 16:54:21 +0100 Message-Id: <20230914155422.426639-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230914155422.426639-1-alex.bennee@linaro.org> References: <20230914155422.426639-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org From: Nicholas Piggin Occasionally some avocado tests will fail waiting for console line despite the machine running correctly. Console data goes missing, as can be seen in the console log. This is due to _console_interaction calling makefile() on the console socket each time it is invoked, which must be losing old buffer contents when going out of scope. It is not enough to makefile() with buffered=0. That helps significantly but data loss is still possible. My guess is that readline() has a line buffer even when the file is in unbuffered mode, that can eat data. Fix this by providing a console file that persists for the life of the console. Signed-off-by: Nicholas Piggin Reviewed-by: "Daniel P. Berrangé" Message-Id: <20230912131340.405619-1-npiggin@gmail.com> Signed-off-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Acked-by: John Snow --- python/qemu/machine/machine.py | 19 +++++++++++++++++++ tests/avocado/avocado_qemu/__init__.py | 2 +- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py index c16a0b6fed..35d5a672db 100644 --- a/python/qemu/machine/machine.py +++ b/python/qemu/machine/machine.py @@ -191,6 +191,7 @@ def __init__(self, self.sock_dir, f"{self._name}.con" ) self._console_socket: Optional[socket.socket] = None + self._console_file: Optional[socket.SocketIO] = None self._remove_files: List[str] = [] self._user_killed = False self._quit_issued = False @@ -509,6 +510,11 @@ def _early_cleanup(self) -> None: # If we keep the console socket open, we may deadlock waiting # for QEMU to exit, while QEMU is waiting for the socket to # become writable. + if self._console_file is not None: + LOG.debug("Closing console file") + self._console_file.close() + self._console_file = None + if self._console_socket is not None: LOG.debug("Closing console socket") self._console_socket.close() @@ -874,12 +880,25 @@ def console_socket(self) -> socket.socket: Returns a socket connected to the console """ if self._console_socket is None: + LOG.debug("Opening console socket") self._console_socket = console_socket.ConsoleSocket( self._console_address, file=self._console_log_path, drain=self._drain_console) return self._console_socket + @property + def console_file(self) -> socket.SocketIO: + """ + Returns a file associated with the console socket + """ + if self._console_file is None: + LOG.debug("Opening console file") + self._console_file = self.console_socket.makefile(mode='rb', + buffering=0, + encoding='utf-8') + return self._console_file + @property def temp_dir(self) -> str: """ diff --git a/tests/avocado/avocado_qemu/__init__.py b/tests/avocado/avocado_qemu/__init__.py index 33090903f1..0172a359b7 100644 --- a/tests/avocado/avocado_qemu/__init__.py +++ b/tests/avocado/avocado_qemu/__init__.py @@ -137,7 +137,7 @@ def _console_interaction(test, success_message, failure_message, assert not keep_sending or send_string if vm is None: vm = test.vm - console = vm.console_socket.makefile(mode='rb', encoding='utf-8') + console = vm.console_file console_logger = logging.getLogger('console') while True: if send_string: