From patchwork Fri Jul 31 18:20:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90DF6C433E0 for ; Fri, 31 Jul 2020 18:22:43 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61FEE21744 for ; Fri, 31 Jul 2020 18:22:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ta8bOuOv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61FEE21744 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1Zg2-0003J1-H6 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:22:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeK-0001gD-64 for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50682) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeI-0005JZ-2d for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:55 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VID0M1110645; Fri, 31 Jul 2020 18:20:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2020-01-29; bh=jNZcu3DA7/vi8mi2Vu4keoe60eioLTWhHHzQEBhuJZs=; b=ta8bOuOvAb+su29Z6fe+plHbVcohpsz8fOPc+BMjTe8NV+tZ2v7Ko3ivBZJ3DSiZrfqt fTByAXGwfg1sL4JF8E0Oct+o355c5bP6h2267qbaMRrrdgptqPeqUGXKFcOx26SfmOom nmcgFQNSDAEJ88Pr9AudcAgdK6TTsyU9zR1t+ex9m+OuNkU8xynwrrikIN09zZaVSzF6 jK9N1OPRdA4IEx1WqfVaxhLZR/dt3glzWUZUzDeZlFO2WNy+dX6MykmQvb0TPGiWUhut 4f6Ok4KA47N08goYdOfTUSEa6TiRu/7x+kim++inIXHA0qZZbfKw8QqQKczoyjYVQtnX KA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 32hu1jtfxx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:42 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIjus080444; Fri, 31 Jul 2020 18:20:41 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 32mf70vdud-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:41 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIKdYX013221; Fri, 31 Jul 2020 18:20:39 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:39 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 02/20] multi-process: Add config option for multi-process QEMU Date: Fri, 31 Jul 2020 14:20:09 -0400 Message-Id: <77b128d3739ddef84e1e964c027767893f57c89a.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=1 spamscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 suspectscore=1 bulkscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.86; envelope-from=jag.raman@oracle.com; helo=userp2130.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 12:51:59 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -63 X-Spam_score: -6.4 X-Spam_bar: ------ X-Spam_report: (-6.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add a configuration option to separate multi-process code Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- configure | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/configure b/configure index 2acc4d1..7b615c5 100755 --- a/configure +++ b/configure @@ -519,6 +519,7 @@ fuzzing="no" rng_none="no" secret_keyring="" libdaxctl="" +mpqemu="no" supported_cpu="no" supported_os="no" @@ -1640,6 +1641,10 @@ for opt do ;; --disable-libdaxctl) libdaxctl=no ;; + --enable-mpqemu) mpqemu=yes + ;; + --disable-mpqemu) mpqemu=no + ;; *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information" @@ -1943,6 +1948,8 @@ disabled with --disable-FEATURE, default is enabled if available: xkbcommon xkbcommon support rng-none dummy RNG, avoid using /dev/(u)random and getrandom() libdaxctl libdaxctl support + mpqemu multi-process QEMU support + NOTE: The object files are built at the place where configure is launched EOF @@ -7019,6 +7026,7 @@ echo "fuzzing support $fuzzing" echo "gdb $gdb_bin" echo "rng-none $rng_none" echo "Linux keyring $secret_keyring" +echo "multiprocess QEMU $mpqemu" if test "$supported_cpu" = "no"; then echo @@ -7891,6 +7899,9 @@ fi if test "$have_mlockall" = "yes" ; then echo "HAVE_MLOCKALL=y" >> $config_host_mak fi +if test "$mpqemu" = "yes" ; then + echo "CONFIG_MPQEMU=y" >> $config_host_mak +fi if test "$fuzzing" = "yes" ; then if test "$have_fuzzer" = "yes"; then FUZZ_LDFLAGS=" -fsanitize=fuzzer" From patchwork Fri Jul 31 18:20:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88751C433DF for ; Fri, 31 Jul 2020 18:22:46 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61FC222B3F for ; Fri, 31 Jul 2020 18:22:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Hxd85FVU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61FC222B3F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:44796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1Zg5-0003Sx-L5 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:22:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58052) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeN-0001k8-9b for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:59 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50684) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeI-0005JX-9E for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:58 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VID2kN110735; Fri, 31 Jul 2020 18:20:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=esnuSUsw+zutTCesko7PBcHnqXd9qZSK/UE6b2zHDZA=; b=Hxd85FVUOy5pQ9zjstVOVcMYmhTbwB9hKb/742BenP+sjJ1gmFtcbsZKZ+4wwEEmRgYC pSCTACb1lXN/IP78ZyUEDWYJdaLwrM/sXQZ6lDi1jjDPmNgbbX1guE9R2wBqoO7GJdpg EG1csCWptAKJCzxtQgOzikSogV9HYH0m7jeNspp6DOruj93FWyuGc4431Aq1OqwZoKKp WA27NZ0berMU105sj60jJX5WiK5m+Xyc1EGss2KEA2A4wzLwTUBFrL6kvoShePaUyQFc oMRP9hjDpaAkcjMeZF/RKkJHb4hk/4dW3TBBo+vWeZVxYLqcdTm7fa1GFQxgd1zK5biW Rw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 32hu1jtfy0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:42 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIjVe080443; Fri, 31 Jul 2020 18:20:42 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 32mf70vdug-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:42 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIKfj7013242; Fri, 31 Jul 2020 18:20:41 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:40 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 03/20] multi-process: setup PCI host bridge for remote device Date: Fri, 31 Jul 2020 14:20:10 -0400 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=1 spamscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 suspectscore=1 bulkscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.86; envelope-from=jag.raman@oracle.com; helo=userp2130.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 12:51:59 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -63 X-Spam_score: -6.4 X-Spam_bar: ------ X-Spam_report: (-6.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" PCI host bridge is setup for the remote device process. It is implemented using remote-pcihost object. It is an extension of the PCI host bridge setup by QEMU. Remote-pcihost configures a PCI bus which could be used by the remote PCI device to latch on to. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva --- MAINTAINERS | 8 +++++ hw/pci-host/Makefile.objs | 1 + hw/pci-host/remote.c | 76 ++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci-host/remote.h | 30 +++++++++++++++++ 4 files changed, 115 insertions(+) create mode 100644 hw/pci-host/remote.c create mode 100644 include/hw/pci-host/remote.h diff --git a/MAINTAINERS b/MAINTAINERS index 0886eb3..d781a28 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3035,6 +3035,14 @@ S: Maintained F: hw/semihosting/ F: include/hw/semihosting/ +Multi-process QEMU +M: Jagannathan Raman +M: Elena Ufimtseva +M: John G Johnson +S: Maintained +F: hw/pci-host/remote.c +F: include/hw/pci-host/remote.h + Build and test automation ------------------------- Build and test automation diff --git a/hw/pci-host/Makefile.objs b/hw/pci-host/Makefile.objs index e422e0a..daf9007 100644 --- a/hw/pci-host/Makefile.objs +++ b/hw/pci-host/Makefile.objs @@ -18,6 +18,7 @@ common-obj-$(CONFIG_XEN_IGD_PASSTHROUGH) += xen_igd_pt.o common-obj-$(CONFIG_PCI_EXPRESS_Q35) += q35.o common-obj-$(CONFIG_PCI_EXPRESS_GENERIC_BRIDGE) += gpex.o common-obj-$(CONFIG_PCI_EXPRESS_XILINX) += xilinx-pcie.o +common-obj-$(CONFIG_MPQEMU) += remote.o common-obj-$(CONFIG_PCI_EXPRESS_DESIGNWARE) += designware.o obj-$(CONFIG_POWERNV) += pnv_phb4.o pnv_phb4_pec.o diff --git a/hw/pci-host/remote.c b/hw/pci-host/remote.c new file mode 100644 index 0000000..03c8e99 --- /dev/null +++ b/hw/pci-host/remote.c @@ -0,0 +1,76 @@ +/* + * Remote PCI host device + * + * Unlike PCI host devices that model physical hardware, the purpose + * of this PCI host is to host multi-process QEMU devices. + * + * Multi-process QEMU extends the PCI host of a QEMU machine into a + * remote process. Any PCI device attached to the remote process is + * visible in the QEMU guest. This allows existing QEMU device models + * to be reused in the remote process. + * + * This PCI host is purely a container for PCI devices. It's fake in the + * sense that the guest never sees this PCI host and has no way of + * accessing it. It's job is just to provide the environment that QEMU + * PCI device models need when running in a remote process. + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "hw/pci/pci.h" +#include "hw/pci/pci_host.h" +#include "hw/pci/pcie_host.h" +#include "hw/qdev-properties.h" +#include "hw/pci-host/remote.h" +#include "exec/memory.h" + +static const char *remote_pcihost_root_bus_path(PCIHostState *host_bridge, + PCIBus *rootbus) +{ + return "0000:00"; +} + +static void remote_pcihost_realize(DeviceState *dev, Error **errp) +{ + char *busname = g_strdup_printf("remote-pci-%ld", (unsigned long)getpid()); + PCIHostState *pci = PCI_HOST_BRIDGE(dev); + RemotePCIHost *s = REMOTE_HOST_DEVICE(dev); + + pci->bus = pci_root_bus_new(DEVICE(s), busname, + s->mr_pci_mem, s->mr_sys_io, + 0, TYPE_PCIE_BUS); +} + +static void remote_pcihost_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PCIHostBridgeClass *hc = PCI_HOST_BRIDGE_CLASS(klass); + + hc->root_bus_path = remote_pcihost_root_bus_path; + dc->realize = remote_pcihost_realize; + + dc->user_creatable = false; + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); + dc->fw_name = "pci"; +} + +static const TypeInfo remote_pcihost_info = { + .name = TYPE_REMOTE_HOST_DEVICE, + .parent = TYPE_PCIE_HOST_BRIDGE, + .instance_size = sizeof(RemotePCIHost), + .class_init = remote_pcihost_class_init, +}; + +static void remote_pcihost_register(void) +{ + type_register_static(&remote_pcihost_info); +} + +type_init(remote_pcihost_register) diff --git a/include/hw/pci-host/remote.h b/include/hw/pci-host/remote.h new file mode 100644 index 0000000..bab6d3c --- /dev/null +++ b/include/hw/pci-host/remote.h @@ -0,0 +1,30 @@ +/* + * PCI Host for remote device + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef REMOTE_PCIHOST_H +#define REMOTE_PCIHOST_H + +#include "exec/memory.h" +#include "hw/pci/pcie_host.h" + +#define TYPE_REMOTE_HOST_DEVICE "remote-pcihost" +#define REMOTE_HOST_DEVICE(obj) \ + OBJECT_CHECK(RemotePCIHost, (obj), TYPE_REMOTE_HOST_DEVICE) + +typedef struct RemotePCIHost { + /*< private >*/ + PCIExpressHost parent_obj; + /*< public >*/ + + MemoryRegion *mr_pci_mem; + MemoryRegion *mr_sys_io; +} RemotePCIHost; + +#endif From patchwork Fri Jul 31 18:20:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFF27C433DF for ; Fri, 31 Jul 2020 18:25:02 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A31BC22B42 for ; Fri, 31 Jul 2020 18:25:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="a/d4D1Ne" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A31BC22B42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53072 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1ZiH-0006tc-Rf for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:25:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58026) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeL-0001h0-GJ for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:57 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeI-0005Jz-KK for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:57 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VICxHE110610; Fri, 31 Jul 2020 18:20:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2020-01-29; bh=vnIr8ft5TgP2wkeLfVoE3u7eHFCMSMjH1MvfeVIsqfs=; b=a/d4D1Ne4Z09g2Sm+BXSZJ1mRd7nDtQ2hKLK5lAv7ognhGlchY1M67d3LLGsukj9Ykc0 mPWKXB0jhLIfeH7JpfnD1UdVlU+3D6eQDxqJJOuGhtNpgGJ+zU5GuMU1nWTAFLR1ZXNy AVMjLk/bB9t+W05mvdjbD29mOxRiOfJuUKCGIBsTD/Zs5UcAg8oBAqtlHphe3tI56nmQ X7pocnseypeT7ypa0p0XQZKfiK1OgDoQLURncOluzikMhxKYcBccbbB0hEkDoCr6+Jv2 b7IN2+YVmrCc8H+rN6xuBFfO+ZX7aVih3jxYqOgFdlVXCHyLPKPDYee34Rhdm7JAmJVJ dA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 32hu1jtfya-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:46 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIdPP137711; Fri, 31 Jul 2020 18:20:46 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3020.oracle.com with ESMTP id 32hu64r70q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:46 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIKixq013464; Fri, 31 Jul 2020 18:20:44 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:43 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 05/20] multi-process: add qio channel function to transmit Date: Fri, 31 Jul 2020 14:20:12 -0400 Message-Id: <8458f12d90a0bc1a8242e4b4a445c2bf0beab003.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 suspectscore=3 bulkscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.86; envelope-from=jag.raman@oracle.com; helo=userp2130.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 12:51:59 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -63 X-Spam_score: -6.4 X-Spam_bar: ------ X-Spam_report: (-6.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva The entire array of the memory regions and file handlers. Will be used in the next patch. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- include/io/channel.h | 24 ++++++++++++++++++++++++ io/channel.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/include/io/channel.h b/include/io/channel.h index d4557f0..c2e3eae 100644 --- a/include/io/channel.h +++ b/include/io/channel.h @@ -779,5 +779,29 @@ void qio_channel_set_aio_fd_handler(QIOChannel *ioc, IOHandler *io_read, IOHandler *io_write, void *opaque); +/** + * qio_channel_writev_full_all: + * @ioc: the channel object + * @iov: the array of memory regions to write data from + * @niov: the length of the @iov array + * @fds: an array of file handles to send + * @nfds: number of file handles in @fds + * @errp: pointer to a NULL-initialized error object + * + * + * Behaves like qio_channel_writev_full but will attempt + * to send all data passed (file handles and memory regions). + * The function will wait for all requested data + * to be written, yielding from the current coroutine + * if required. + * + * Returns: 0 if all bytes were written, or -1 on error + */ + +int qio_channel_writev_full_all(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + int *fds, size_t nfds, + Error **errp); #endif /* QIO_CHANNEL_H */ diff --git a/io/channel.c b/io/channel.c index e4376eb..22c10c5 100644 --- a/io/channel.c +++ b/io/channel.c @@ -190,6 +190,51 @@ int qio_channel_writev_all(QIOChannel *ioc, return ret; } +int qio_channel_writev_full_all(QIOChannel *ioc, + const struct iovec *iov, + size_t niov, + int *fds, size_t nfds, + Error **errp) +{ + int ret = -1; + struct iovec *local_iov = g_new(struct iovec, niov); + struct iovec *local_iov_head = local_iov; + unsigned int nlocal_iov = niov; + + nlocal_iov = iov_copy(local_iov, nlocal_iov, + iov, niov, + 0, iov_size(iov, niov)); + + while (nlocal_iov > 0) { + ssize_t len; + len = qio_channel_writev_full(ioc, local_iov, nlocal_iov, fds, + nfds, errp); + if (len == QIO_CHANNEL_ERR_BLOCK) { + if (qemu_in_coroutine()) { + qio_channel_yield(ioc, G_IO_OUT); + } else { + qio_channel_wait(ioc, G_IO_OUT); + } + continue; + } + if (len < 0) { + goto cleanup; + } + + iov_discard_front(&local_iov, &nlocal_iov, len); + + if (len > 0) { + fds = NULL; + nfds = 0; + } + } + + ret = 0; + cleanup: + g_free(local_iov_head); + return ret; +} + ssize_t qio_channel_readv(QIOChannel *ioc, const struct iovec *iov, size_t niov, From patchwork Fri Jul 31 18:20:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B5B7C433DF for ; Fri, 31 Jul 2020 18:33:25 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C2E821744 for ; Fri, 31 Jul 2020 18:33:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="oO3jk8e4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C2E821744 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:51334 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1ZqO-0001Cy-H7 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:33:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58552) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZgI-0004Oc-1W for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:22:58 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:56466) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZgD-0005bd-NJ for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:22:57 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIMFt4092729; Fri, 31 Jul 2020 18:22:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=1+MgqF0N0d8dXHSe7a/SxoPOyeQy5/CjjiuwlEa+YeA=; b=oO3jk8e4o8eYErd8anDQPwbAG2Fn528y/vn6pjRsHYUnsZqv4bU1sH/FkzbJjZGvF/G4 71Q9kyZphRSZXeOYae796oRnhndeoA0aH/KqN+nu8L4bWspbh7wds0XOI0KKQTKuLLUt WogqwhmdXnCGUeP3d/aDlimbIL657/BjmLqcB07PQ3zSUdv1RlMmDQVwNRl0QYULgdOv OlyLpk9T43z5uVc9UgSuA5m32ewQT/N9RPaONywmwcwTgrrSMPOG4raN0kioQYuxCYND AXfckF4FNYjDi3s5EQI42EAMNnUsDRHNqweaUxaiEBACEGj7g6Pw3yUMTsOC4JcfC7eH tw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 32mf702uj0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:22:48 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIId2Y137682; Fri, 31 Jul 2020 18:20:47 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 32hu64r724-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:47 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 06VIKjnM020161; Fri, 31 Jul 2020 18:20:45 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:45 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 06/20] multi-process: define MPQemuMsg format and transmission functions Date: Fri, 31 Jul 2020 14:20:13 -0400 Message-Id: <50f84ecac23889920aeed397adb9317b8ea5a54b.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=3 spamscore=0 impostorscore=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 Received-SPF: pass client-ip=156.151.31.85; envelope-from=jag.raman@oracle.com; helo=userp2120.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 14:20:51 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva Defines MPQemuMsg, which is the message that is sent to the remote process. This message is sent over QIOChannel and is used to command the remote process to perform various tasks. Also defined the helper functions to send and receive messages over the QIOChannel Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva --- MAINTAINERS | 2 + include/io/mpqemu-link.h | 73 ++++++++++++++++++++ io/Makefile.objs | 2 + io/mpqemu-link.c | 175 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 252 insertions(+) create mode 100644 include/io/mpqemu-link.h create mode 100644 io/mpqemu-link.c diff --git a/MAINTAINERS b/MAINTAINERS index 32e264e..41fe809 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3044,6 +3044,8 @@ F: hw/pci-host/remote.c F: include/hw/pci-host/remote.h F: hw/i386/remote.c F: include/hw/i386/remote.h +F: io/mpqemu-link.c +F: include/io/mpqemu-link.h Build and test automation ------------------------- diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h new file mode 100644 index 0000000..ae7008e --- /dev/null +++ b/include/io/mpqemu-link.h @@ -0,0 +1,73 @@ +/* + * Communication channel between QEMU and remote device process + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef MPQEMU_LINK_H +#define MPQEMU_LINK_H + +#include "qom/object.h" +#include "qemu/thread.h" +#include "io/channel.h" + +#define REMOTE_MAX_FDS 8 + +#define MPQEMU_MSG_HDR_SIZE offsetof(MPQemuMsg, data1.u64) + +/** + * MPQemuCmd: + * + * MPQemuCmd enum type to specify the command to be executed on the remote + * device. + */ +typedef enum { + INIT = 0, + MAX = INT_MAX, +} MPQemuCmd; + +/** + * Maximum size of data2 field in the message to be transmitted. + */ +#define MPQEMU_MSG_DATA_MAX 256 + +/** + * MPQemuMsg: + * @cmd: The remote command + * @bytestream: Indicates if the data to be shared is structured (data1) + * or unstructured (data2) + * @size: Size of the data to be shared + * @data1: Structured data + * @fds: File descriptors to be shared with remote device + * @data2: Unstructured data + * + * MPQemuMsg Format of the message sent to the remote device from QEMU. + * + */ +typedef struct { + int cmd; + int bytestream; + size_t size; + + union { + uint64_t u64; + } data1; + + int fds[REMOTE_MAX_FDS]; + int num_fds; + + /* Max size of data2 is MPQEMU_MSG_DATA_MAX */ + uint8_t *data2; +} MPQemuMsg; + +void mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp); +void mpqemu_msg_recv(MPQemuMsg *msg, QIOChannel *ioc, Error **errp); + +bool mpqemu_msg_valid(MPQemuMsg *msg); +void mpqemu_msg_cleanup(MPQemuMsg *msg); + +#endif diff --git a/io/Makefile.objs b/io/Makefile.objs index 9a20fce..5875ab0 100644 --- a/io/Makefile.objs +++ b/io/Makefile.objs @@ -10,3 +10,5 @@ io-obj-y += channel-util.o io-obj-y += dns-resolver.o io-obj-y += net-listener.o io-obj-y += task.o + +io-obj-$(CONFIG_MPQEMU) += mpqemu-link.o diff --git a/io/mpqemu-link.c b/io/mpqemu-link.c new file mode 100644 index 0000000..dcefa42 --- /dev/null +++ b/io/mpqemu-link.c @@ -0,0 +1,175 @@ +/* + * Communication channel between QEMU and remote device process + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "qemu/module.h" +#include "io/mpqemu-link.h" +#include "qapi/error.h" +#include "qemu/iov.h" +#include "qemu/error-report.h" + +void mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp) +{ + Error *local_err = NULL; + struct iovec send[2]; + int *fds = NULL; + size_t nfds = 0; + + send[0].iov_base = msg; + send[0].iov_len = MPQEMU_MSG_HDR_SIZE; + + send[1].iov_base = msg->bytestream ? msg->data2 : (void *)&msg->data1; + send[1].iov_len = msg->size; + + if (msg->num_fds) { + nfds = msg->num_fds; + fds = msg->fds; + } + + (void)qio_channel_writev_full_all(ioc, send, G_N_ELEMENTS(send), fds, nfds, + &local_err); + if (local_err) { + error_propagate(errp, local_err); + } + + return; +} + +static int mpqemu_readv(QIOChannel *ioc, struct iovec *iov, int **fds, + size_t *nfds, Error **errp) +{ + struct iovec *l_iov = iov; + size_t *l_nfds = nfds; + unsigned int niov = 1; + int **l_fds = fds; + size_t size, len; + Error *local_err = NULL; + + size = iov->iov_len; + + while (size > 0) { + len = qio_channel_readv_full(ioc, l_iov, niov, l_fds, l_nfds, + &local_err); + + if (len == QIO_CHANNEL_ERR_BLOCK) { + if (qemu_in_coroutine()) { + qio_channel_yield(ioc, G_IO_IN); + } else { + qio_channel_wait(ioc, G_IO_IN); + } + continue; + } + + if (len <= 0) { + error_propagate(errp, local_err); + return -EIO; + } + + l_fds = NULL; + l_nfds = NULL; + + size -= len; + + (void)iov_discard_front(&l_iov, &niov, len); + } + + return iov->iov_len; +} + +void mpqemu_msg_recv(MPQemuMsg *msg, QIOChannel *ioc, Error **errp) +{ + Error *local_err = NULL; + int *fds = NULL; + struct iovec hdr, data; + size_t nfds = 0; + + hdr.iov_base = msg; + hdr.iov_len = MPQEMU_MSG_HDR_SIZE; + + if (mpqemu_readv(ioc, &hdr, &fds, &nfds, &local_err) < 0) { + error_propagate(errp, local_err); + return; + } + + if (msg->size > MPQEMU_MSG_DATA_MAX) { + error_setg(errp, "The message size is more than MPQEMU_MSG_DATA_MAX %d", + MPQEMU_MSG_DATA_MAX); + return; + } + + data.iov_base = g_malloc0(msg->size); + data.iov_len = msg->size; + + if (mpqemu_readv(ioc, &data, NULL, NULL, &local_err) < 0) { + error_propagate(errp, local_err); + g_free(data.iov_base); + return; + } + + if (msg->bytestream) { + msg->data2 = data.iov_base; + data.iov_base = NULL; + } else if (msg->size > sizeof(msg->data1)) { + error_setg(errp, "Invalid size for message"); + g_free(data.iov_base); + } else { + memcpy((void *)&msg->data1, data.iov_base, msg->size); + g_free(data.iov_base); + } + + msg->num_fds = nfds; + if (nfds) { + memcpy(msg->fds, fds, nfds * sizeof(int)); + } +} + +bool mpqemu_msg_valid(MPQemuMsg *msg) +{ + if (msg->cmd >= MAX && msg->cmd < 0) { + return false; + } + + if (msg->bytestream) { + if (!msg->data2) { + return false; + } + if (msg->data1.u64 != 0) { + return false; + } + } else { + if (msg->data2) { + return false; + } + } + + /* Verify FDs. */ + if (msg->num_fds >= REMOTE_MAX_FDS) { + return false; + } + + if (msg->num_fds > 0) { + for (int i = 0; i < msg->num_fds; i++) { + if (fcntl(msg->fds[i], F_GETFL) == -1) { + return false; + } + } + } + + return true; +} + +void mpqemu_msg_cleanup(MPQemuMsg *msg) +{ + if (msg->data2) { + free(msg->data2); + } +} From patchwork Fri Jul 31 18:20:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 132E2C433DF for ; Fri, 31 Jul 2020 18:27:32 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DFA1922B42 for ; Fri, 31 Jul 2020 18:27:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ijBD3K1j" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFA1922B42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1Zkh-0002BG-3R for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:27:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58036) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeL-0001hY-OA for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:57 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:54838) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeJ-0005LZ-Hz for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:20:57 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIBmft069809; Fri, 31 Jul 2020 18:20:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2020-01-29; bh=1l67bn+gCumzFGqkgymzvlntK4IEFqBqvaSg3HO6MtE=; b=ijBD3K1jNpms2CG0XjRlzRrdvgu8I9A4mDzpDQRNwkDrLQerCR+EM1pR0VU8U/zySSUi Vb7tZbkLP6ZeoW5LsJJLfiELJbnK6hbLkRH1bA1qGBeIruwzrBltuByJfqGuJWypE5Wu rzmc8XgwiAu5w10CAytqwP4iZiU+RFDHPaDdnR06PikJhmVzOhObjFEuWoIJoXEmyCaE Q0vOrsW0zsgAFWZt/gjRFddGSIawtsITITYMMkbV5zlxz4RZ3cwlGOkefs1je0s4Oesp YelPIgVJx/z16YzyD4CVo21BCsnWHgFkiphpQrqvtRXvm668BuczXqlw6re6mxGR3cKz jA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 32mf702u7x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:48 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIi8c080286; Fri, 31 Jul 2020 18:20:48 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 32mf70vdwy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:48 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIKldb013260; Fri, 31 Jul 2020 18:20:47 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:46 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 07/20] multi-process: add co-routines to communicate with remote Date: Fri, 31 Jul 2020 14:20:14 -0400 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=0 spamscore=0 impostorscore=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.85; envelope-from=jag.raman@oracle.com; helo=userp2120.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 14:20:51 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva process to avoid blocking the main loop during the message exchanges. To be used by proxy device. Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson --- include/io/mpqemu-link.h | 15 +++++++++ io/mpqemu-link.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 97 insertions(+) diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h index ae7008e..8591ad2 100644 --- a/include/io/mpqemu-link.h +++ b/include/io/mpqemu-link.h @@ -14,6 +14,7 @@ #include "qom/object.h" #include "qemu/thread.h" #include "io/channel.h" +#include "io/channel-socket.h" #define REMOTE_MAX_FDS 8 @@ -27,6 +28,7 @@ */ typedef enum { INIT = 0, + RET_MSG, MAX = INT_MAX, } MPQemuCmd; @@ -64,6 +66,19 @@ typedef struct { uint8_t *data2; } MPQemuMsg; +struct MPQemuRequest { + MPQemuMsg *msg; + QIOChannel *ioc; + Coroutine *co; + bool finished; + int error; + long ret; +}; + +typedef struct MPQemuRequest MPQemuRequest; + +uint64_t mpqemu_msg_send_and_await_reply(MPQemuMsg *msg, QIOChannel *ioc, + Error **errp); void mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp); void mpqemu_msg_recv(MPQemuMsg *msg, QIOChannel *ioc, Error **errp); diff --git a/io/mpqemu-link.c b/io/mpqemu-link.c index dcefa42..d4dd0fe 100644 --- a/io/mpqemu-link.c +++ b/io/mpqemu-link.c @@ -16,6 +16,8 @@ #include "qapi/error.h" #include "qemu/iov.h" #include "qemu/error-report.h" +#include "qemu/main-loop.h" +#include "io/channel-socket.h" void mpqemu_msg_send(MPQemuMsg *msg, QIOChannel *ioc, Error **errp) { @@ -132,6 +134,86 @@ void mpqemu_msg_recv(MPQemuMsg *msg, QIOChannel *ioc, Error **errp) } } +/* Use in proxy only as it clobbers fd handlers. */ +static void coroutine_fn mpqemu_msg_send_co(void *data) +{ + MPQemuRequest *req = (MPQemuRequest *)data; + MPQemuMsg msg_reply = {0}; + Error *local_err = NULL; + + if (!req->ioc) { + error_report("No channel available to send command %d", + req->msg->cmd); + req->finished = true; + req->error = -EINVAL; + return; + } + + req->co = qemu_coroutine_self(); + mpqemu_msg_send(req->msg, req->ioc, &local_err); + if (local_err) { + error_report("ERROR: failed to send command to remote %d, ", + req->msg->cmd); + req->finished = true; + req->error = -EINVAL; + return; + } + + mpqemu_msg_recv(&msg_reply, req->ioc, &local_err); + if (local_err) { + error_report("ERROR: failed to get a reply for command %d, " + "errno %s", + req->msg->cmd, strerror(errno)); + req->error = -EIO; + } else { + if (!mpqemu_msg_valid(&msg_reply) || msg_reply.cmd != RET_MSG) { + error_report("ERROR: Invalid reply received for command %d", + req->msg->cmd); + req->error = -EINVAL; + } else { + req->ret = msg_reply.data1.u64; + } + } + req->finished = true; +} + +/* + * Create if needed and enter co-routine to send the message to the + * remote channel ioc and wait for the reply. + * Returns the value from the reply message, sets the error on failure. + */ + +uint64_t mpqemu_msg_send_and_await_reply(MPQemuMsg *msg, QIOChannel *ioc, + Error **errp) +{ + MPQemuRequest req = {0}; + uint64_t ret = UINT64_MAX; + + req.ioc = ioc; + if (!req.ioc) { + error_setg(errp, "Channel is set to NULL"); + return ret; + } + + req.msg = msg; + req.ret = 0; + req.finished = false; + + req.co = qemu_coroutine_create(mpqemu_msg_send_co, &req); + qemu_coroutine_enter(req.co); + + while (!req.finished) { + aio_poll(qemu_get_aio_context(), true); + } + if (req.error) { + error_setg(errp, "Error exchanging message with remote process, " + "error %d", req.error); + } + ret = req.ret; + + return ret; +} + bool mpqemu_msg_valid(MPQemuMsg *msg) { if (msg->cmd >= MAX && msg->cmd < 0) { From patchwork Fri Jul 31 18:20:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0FFFC433E0 for ; Fri, 31 Jul 2020 18:25:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB19A22B42 for ; Fri, 31 Jul 2020 18:25:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Pvpjej4T" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB19A22B42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:53446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1ZiL-00072X-U8 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:25:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58094) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeR-0001tr-P1 for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:03 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:54934) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeP-0005PJ-Nd for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:03 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIBmov069799; Fri, 31 Jul 2020 18:20:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=dxPLklm6WT/PsavEZFX/Da2B98UHd7xtUPB9I7sAaN8=; b=Pvpjej4TR1AxwKmj7NT7KeGJzqLJY9A3UCH5U/qbjor/mab/qQEwPDC7DBeRoAqmP5pI eVn4hz9Y76p4iqPE2YGM5eI0FMQ6wpDhEul7ncL/NDyGgAYJEjVgxwoEWoV3hpytkFsi XLM5aSO+KwBkibQEDEvjZ5xbjaKPeRVe0Q/eXpurKIZgS+0KJkqV+gAlh28QdjO4cCdV EKGwBTkvUipCCUKY8JIV47GnK4zXVSVgO3KBqHYTgFs/ZhhHpAAbcYGUkiXzWpW2biq+ gaEmcbc/vBCMET1zmmVlc/X++3xj0a4Cx/YCF0G68C3ZQu2Ri6aFa0kaZBJntTyVrFb7 Yw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 32mf702u8a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:55 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIt5a086448; Fri, 31 Jul 2020 18:20:55 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 32hu608m1x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:54 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIKrlH013498; Fri, 31 Jul 2020 18:20:53 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:52 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 11/20] multi-process: introduce proxy object Date: Fri, 31 Jul 2020 14:20:18 -0400 Message-Id: <735bfb2360481fcb576d7da1d00179b074edd3b6.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 mlxscore=0 adultscore=0 spamscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=1 spamscore=0 impostorscore=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.85; envelope-from=jag.raman@oracle.com; helo=userp2120.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 14:20:51 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva Defines a PCI Device proxy object as a child of TYPE_PCI_DEVICE. Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson --- MAINTAINERS | 2 ++ hw/pci/Makefile.objs | 1 + hw/pci/proxy.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/proxy.h | 26 ++++++++++++++++ 4 files changed, 109 insertions(+) create mode 100644 hw/pci/proxy.c create mode 100644 include/hw/pci/proxy.h diff --git a/MAINTAINERS b/MAINTAINERS index 2da7680..a8f097a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3051,6 +3051,8 @@ F: include/hw/i386/remote-obj.h F: hw/i386/remote-obj.c F: include/hw/i386/remote-memory.h F: hw/i386/remote-memory.c +F: hw/pci/proxy.c +F: include/hw/pci/proxy.h Build and test automation ------------------------- diff --git a/hw/pci/Makefile.objs b/hw/pci/Makefile.objs index c78f2fb..515dda5 100644 --- a/hw/pci/Makefile.objs +++ b/hw/pci/Makefile.objs @@ -12,3 +12,4 @@ common-obj-$(CONFIG_PCI_EXPRESS) += pcie_port.o pcie_host.o common-obj-$(call lnot,$(CONFIG_PCI)) += pci-stub.o common-obj-$(CONFIG_ALL) += pci-stub.o +obj-$(CONFIG_MPQEMU) += proxy.o diff --git a/hw/pci/proxy.c b/hw/pci/proxy.c new file mode 100644 index 0000000..014c1cb --- /dev/null +++ b/hw/pci/proxy.c @@ -0,0 +1,80 @@ +/* + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "hw/pci/proxy.h" +#include "hw/pci/pci.h" +#include "qapi/error.h" +#include "io/channel-util.h" +#include "hw/qdev-properties.h" +#include "monitor/monitor.h" + +static void proxy_set_socket(PCIProxyDev *pdev, int fd, Error **errp) +{ + pdev->ioc = qio_channel_new_fd(fd, errp); +} + +static Property proxy_properties[] = { + DEFINE_PROP_STRING("fd", PCIProxyDev, fd), + DEFINE_PROP_END_OF_LIST(), +}; + +static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) +{ + PCIProxyDev *dev = PCI_PROXY_DEV(device); + int fd; + + if (dev->fd) { + fd = monitor_fd_param(cur_mon, dev->fd, errp); + if (fd == -1) { + error_prepend(errp, "proxy: unable to parse fd: "); + return; + } + proxy_set_socket(dev, fd, errp); + } else { + error_setg(errp, "fd parameter not specified for %s", + DEVICE(device)->id); + } +} + +static void pci_proxy_dev_exit(PCIDevice *pdev) +{ + PCIProxyDev *dev = PCI_PROXY_DEV(pdev); + + qio_channel_close(dev->ioc, NULL); +} + +static void pci_proxy_dev_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PCIDeviceClass *k = PCI_DEVICE_CLASS(klass); + + k->realize = pci_proxy_dev_realize; + k->exit = pci_proxy_dev_exit; + device_class_set_props(dc, proxy_properties); +} + +static const TypeInfo pci_proxy_dev_type_info = { + .name = TYPE_PCI_PROXY_DEV, + .parent = TYPE_PCI_DEVICE, + .instance_size = sizeof(PCIProxyDev), + .class_init = pci_proxy_dev_class_init, + .interfaces = (InterfaceInfo[]) { + { INTERFACE_CONVENTIONAL_PCI_DEVICE }, + { }, + }, +}; + +static void pci_proxy_dev_register_types(void) +{ + type_register_static(&pci_proxy_dev_type_info); +} + +type_init(pci_proxy_dev_register_types) diff --git a/include/hw/pci/proxy.h b/include/hw/pci/proxy.h new file mode 100644 index 0000000..0a66ddc --- /dev/null +++ b/include/hw/pci/proxy.h @@ -0,0 +1,26 @@ +/* + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef PROXY_H +#define PROXY_H + +#include "hw/pci/pci.h" +#include "io/channel.h" + +#define TYPE_PCI_PROXY_DEV "pci-proxy-dev" + +#define PCI_PROXY_DEV(obj) \ + OBJECT_CHECK(PCIProxyDev, (obj), TYPE_PCI_PROXY_DEV) + +typedef struct PCIProxyDev { + PCIDevice parent_dev; + char *fd; + QIOChannel *ioc; +} PCIProxyDev; + +#endif /* PROXY_H */ From patchwork Fri Jul 31 18:20:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0D9EC433E0 for ; Fri, 31 Jul 2020 18:27:37 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AAC1B22B42 for ; Fri, 31 Jul 2020 18:27:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Og8uFqsM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAC1B22B42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:34128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1Zkm-0002QI-T2 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:27:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58138) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeW-000266-HV for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:50936) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeT-0005Ps-VB for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:08 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VICxN7110597; Fri, 31 Jul 2020 18:20:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=8Ome06812C8bafzlnnYF2jRWfGFZyOpxST3ApEtTdUY=; b=Og8uFqsMxLF+h4jnbfG7HUHSdG9CG+L4F50VlpJkl3rikBPs90aIbWZ6lQ/5REsnhVuh oNyLgLyW2YAxFTIAsJsmZwl8d2aMapY2aJgQwhCWYO4EpiHehv09nL/rPzsaVBGP0+k/ bZG/RMTLwCBNmkSZIsQRKclQr1kbobvuWRztBeMMCLyAyO/Zy18tzdS7G3NUiHIDGvD+ GILhve03hcYm0grSJBlM9Hnv6Ogd5oN/BpCGdwF16NFyOOD2RZxCsEfKwEK+AcBnHTY9 HF851b0Ja7mGqbcAoeDRxbHjtlHDbqzsqxkjKa9Zz2lyr579kmdrMpYcuZYc6yCZSht3 sQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 32hu1jtg0s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:20:59 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIiUt080288; Fri, 31 Jul 2020 18:20:58 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 32mf70ve40-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:20:58 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 06VIKv3m020219; Fri, 31 Jul 2020 18:20:57 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:57 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 14/20] multi-process: Synchronize remote memory Date: Fri, 31 Jul 2020 14:20:21 -0400 Message-Id: <9e2064df688ff10c0094427dcdae58ff2c3ada34.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=3 spamscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 suspectscore=3 bulkscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.86; envelope-from=jag.raman@oracle.com; helo=userp2130.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 12:51:59 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -63 X-Spam_score: -6.4 X-Spam_bar: ------ X-Spam_report: (-6.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Add memory-listener object which is used to keep the view of the RAM in sync between QEMU and remote process. A MemoryListener is registered for system-memory AddressSpace. The listener sends SYNC_SYSMEM message to the remote process when memory listener commits the changes to memory, the remote process receives the message and processes it in the handler for SYNC_SYSMEM message. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva --- MAINTAINERS | 2 + hw/i386/remote-msg.c | 4 + hw/pci/Makefile.objs | 1 + hw/pci/memory-sync.c | 211 +++++++++++++++++++++++++++++++++++++++++++ hw/pci/proxy.c | 6 ++ include/hw/pci/memory-sync.h | 27 ++++++ include/hw/pci/proxy.h | 3 + 7 files changed, 254 insertions(+) create mode 100644 hw/pci/memory-sync.c create mode 100644 include/hw/pci/memory-sync.h diff --git a/MAINTAINERS b/MAINTAINERS index a8f097a..bd4f4ec 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3053,6 +3053,8 @@ F: include/hw/i386/remote-memory.h F: hw/i386/remote-memory.c F: hw/pci/proxy.c F: include/hw/pci/proxy.h +F: hw/pci/memory-sync.c +F: include/hw/pci/memory-sync.h Build and test automation ------------------------- diff --git a/hw/i386/remote-msg.c b/hw/i386/remote-msg.c index 7ccdd63..584498d 100644 --- a/hw/i386/remote-msg.c +++ b/hw/i386/remote-msg.c @@ -17,6 +17,7 @@ #include "sysemu/runstate.h" #include "hw/pci/pci.h" #include "exec/memattrs.h" +#include "hw/i386/remote-memory.h" static void process_config_write(QIOChannel *ioc, PCIDevice *dev, MPQemuMsg *msg); @@ -67,6 +68,9 @@ gboolean mpqemu_process_msg(QIOChannel *ioc, GIOCondition cond, case BAR_READ: process_bar_read(ioc, &msg, &local_err); break; + case SYNC_SYSMEM: + remote_sysmem_reconfig(&msg, &local_err); + break; default: error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)", diff --git a/hw/pci/Makefile.objs b/hw/pci/Makefile.objs index 515dda5..c90acd5 100644 --- a/hw/pci/Makefile.objs +++ b/hw/pci/Makefile.objs @@ -13,3 +13,4 @@ common-obj-$(CONFIG_PCI_EXPRESS) += pcie_port.o pcie_host.o common-obj-$(call lnot,$(CONFIG_PCI)) += pci-stub.o common-obj-$(CONFIG_ALL) += pci-stub.o obj-$(CONFIG_MPQEMU) += proxy.o +obj-$(CONFIG_MPQEMU) += memory-sync.o diff --git a/hw/pci/memory-sync.c b/hw/pci/memory-sync.c new file mode 100644 index 0000000..1af28395 --- /dev/null +++ b/hw/pci/memory-sync.c @@ -0,0 +1,211 @@ +/* + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "qemu/compiler.h" +#include "qemu/int128.h" +#include "qemu/range.h" +#include "exec/memory.h" +#include "exec/cpu-common.h" +#include "cpu.h" +#include "exec/ram_addr.h" +#include "exec/address-spaces.h" +#include "io/mpqemu-link.h" +#include "hw/pci/memory-sync.h" + +static void proxy_ml_begin(MemoryListener *listener) +{ + RemoteMemSync *sync = container_of(listener, RemoteMemSync, listener); + int mrs; + + for (mrs = 0; mrs < sync->n_mr_sections; mrs++) { + memory_region_unref(sync->mr_sections[mrs].mr); + } + + g_free(sync->mr_sections); + sync->mr_sections = NULL; + sync->n_mr_sections = 0; +} + +static int get_fd_from_hostaddr(uint64_t host, ram_addr_t *offset) +{ + MemoryRegion *mr; + ram_addr_t off; + + /** + * Assumes that the host address is a valid address as it's + * coming from the MemoryListener system. In the case host + * address is not valid, the following call would return + * the default subregion of "system_memory" region, and + * not NULL. So it's not possible to check for NULL here. + */ + mr = memory_region_from_host((void *)(uintptr_t)host, &off); + + if (offset) { + *offset = off; + } + + return memory_region_get_fd(mr); +} + +static bool proxy_mrs_can_merge(uint64_t host, uint64_t prev_host, size_t size) +{ + bool merge; + int fd1, fd2; + + fd1 = get_fd_from_hostaddr(host, NULL); + + fd2 = get_fd_from_hostaddr(prev_host, NULL); + + merge = (fd1 == fd2); + + merge &= ((prev_host + size) == host); + + return merge; +} + +static bool try_merge(RemoteMemSync *sync, MemoryRegionSection *section) +{ + uint64_t mrs_size, mrs_gpa, mrs_page; + MemoryRegionSection *prev_sec; + bool merged = false; + uintptr_t mrs_host; + RAMBlock *mrs_rb; + + if (!sync->n_mr_sections) { + return false; + } + + mrs_rb = section->mr->ram_block; + mrs_page = (uint64_t)qemu_ram_pagesize(mrs_rb); + mrs_size = int128_get64(section->size); + mrs_gpa = section->offset_within_address_space; + mrs_host = (uintptr_t)memory_region_get_ram_ptr(section->mr) + + section->offset_within_region; + + if (get_fd_from_hostaddr(mrs_host, NULL) < 0) { + return true; + } + + mrs_host = mrs_host & ~(mrs_page - 1); + mrs_gpa = mrs_gpa & ~(mrs_page - 1); + mrs_size = ROUND_UP(mrs_size, mrs_page); + + prev_sec = sync->mr_sections + (sync->n_mr_sections - 1); + uint64_t prev_gpa_start = prev_sec->offset_within_address_space; + uint64_t prev_size = int128_get64(prev_sec->size); + uint64_t prev_gpa_end = range_get_last(prev_gpa_start, prev_size); + uint64_t prev_host_start = + (uintptr_t)memory_region_get_ram_ptr(prev_sec->mr) + + prev_sec->offset_within_region; + uint64_t prev_host_end = range_get_last(prev_host_start, prev_size); + + if (mrs_gpa <= (prev_gpa_end + 1)) { + g_assert(mrs_gpa > prev_gpa_start); + + if ((section->mr == prev_sec->mr) && + proxy_mrs_can_merge(mrs_host, prev_host_start, + (mrs_gpa - prev_gpa_start))) { + uint64_t max_end = MAX(prev_host_end, mrs_host + mrs_size); + merged = true; + prev_sec->offset_within_address_space = + MIN(prev_gpa_start, mrs_gpa); + prev_sec->offset_within_region = + MIN(prev_host_start, mrs_host) - + (uintptr_t)memory_region_get_ram_ptr(prev_sec->mr); + prev_sec->size = int128_make64(max_end - MIN(prev_host_start, + mrs_host)); + } + } + + return merged; +} + +static void proxy_ml_region_addnop(MemoryListener *listener, + MemoryRegionSection *section) +{ + RemoteMemSync *sync = container_of(listener, RemoteMemSync, listener); + + if (!(memory_region_is_ram(section->mr) && + !memory_region_is_rom(section->mr))) { + return; + } + + if (try_merge(sync, section)) { + return; + } + + ++sync->n_mr_sections; + sync->mr_sections = g_renew(MemoryRegionSection, sync->mr_sections, + sync->n_mr_sections); + sync->mr_sections[sync->n_mr_sections - 1] = *section; + sync->mr_sections[sync->n_mr_sections - 1].fv = NULL; + memory_region_ref(section->mr); +} + +static void proxy_ml_commit(MemoryListener *listener) +{ + RemoteMemSync *sync = container_of(listener, RemoteMemSync, listener); + MPQemuMsg msg; + MemoryRegionSection *section; + ram_addr_t offset; + uintptr_t host_addr; + int region; + Error *local_err = NULL; + + memset(&msg, 0, sizeof(MPQemuMsg)); + + msg.cmd = SYNC_SYSMEM; + msg.bytestream = 0; + msg.num_fds = sync->n_mr_sections; + msg.size = sizeof(msg.data1); + if (msg.num_fds > REMOTE_MAX_FDS) { + error_report("Number of fds is more than %d", REMOTE_MAX_FDS); + return; + } + + for (region = 0; region < sync->n_mr_sections; region++) { + section = &sync->mr_sections[region]; + msg.data1.sync_sysmem.gpas[region] = + section->offset_within_address_space; + msg.data1.sync_sysmem.sizes[region] = int128_get64(section->size); + host_addr = (uintptr_t)memory_region_get_ram_ptr(section->mr) + + section->offset_within_region; + msg.fds[region] = get_fd_from_hostaddr(host_addr, &offset); + msg.data1.sync_sysmem.offsets[region] = offset; + } + mpqemu_msg_send(&msg, sync->ioc, &local_err); + if (local_err) { + error_report("Error in sending command %d", msg.cmd); + } +} + +void deconfigure_memory_sync(RemoteMemSync *sync) +{ + memory_listener_unregister(&sync->listener); + + proxy_ml_begin(&sync->listener); +} + +void configure_memory_sync(RemoteMemSync *sync, QIOChannel *ioc) +{ + sync->n_mr_sections = 0; + sync->mr_sections = NULL; + + sync->ioc = ioc; + + sync->listener.begin = proxy_ml_begin; + sync->listener.commit = proxy_ml_commit; + sync->listener.region_add = proxy_ml_region_addnop; + sync->listener.region_nop = proxy_ml_region_addnop; + sync->listener.priority = 10; + + memory_listener_register(&sync->listener, &address_space_memory); +} diff --git a/hw/pci/proxy.c b/hw/pci/proxy.c index 179f0c7..28b8c76 100644 --- a/hw/pci/proxy.c +++ b/hw/pci/proxy.c @@ -17,6 +17,8 @@ #include "monitor/monitor.h" #include "io/mpqemu-link.h" #include "qemu/error-report.h" +#include "hw/pci/memory-sync.h" +#include "qom/object.h" static void proxy_set_socket(PCIProxyDev *pdev, int fd, Error **errp) { @@ -44,6 +46,8 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) error_setg(errp, "fd parameter not specified for %s", DEVICE(device)->id); } + + configure_memory_sync(&dev->sync, dev->ioc); } static void pci_proxy_dev_exit(PCIDevice *pdev) @@ -51,6 +55,8 @@ static void pci_proxy_dev_exit(PCIDevice *pdev) PCIProxyDev *dev = PCI_PROXY_DEV(pdev); qio_channel_close(dev->ioc, NULL); + + deconfigure_memory_sync(&dev->sync); } static int config_op_send(PCIProxyDev *pdev, uint32_t addr, uint32_t *val, diff --git a/include/hw/pci/memory-sync.h b/include/hw/pci/memory-sync.h new file mode 100644 index 0000000..785f76a --- /dev/null +++ b/include/hw/pci/memory-sync.h @@ -0,0 +1,27 @@ +/* + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef MEMORY_SYNC_H +#define MEMORY_SYNC_H + +#include "exec/memory.h" +#include "io/channel.h" + +typedef struct RemoteMemSync { + MemoryListener listener; + + int n_mr_sections; + MemoryRegionSection *mr_sections; + + QIOChannel *ioc; +} RemoteMemSync; + +void configure_memory_sync(RemoteMemSync *sync, QIOChannel *ioc); +void deconfigure_memory_sync(RemoteMemSync *sync); + +#endif diff --git a/include/hw/pci/proxy.h b/include/hw/pci/proxy.h index 9d57483..3ff79f4 100644 --- a/include/hw/pci/proxy.h +++ b/include/hw/pci/proxy.h @@ -11,6 +11,7 @@ #include "hw/pci/pci.h" #include "io/channel.h" +#include "hw/pci/memory-sync.h" #define TYPE_PCI_PROXY_DEV "pci-proxy-dev" @@ -32,6 +33,8 @@ struct PCIProxyDev { char *fd; QIOChannel *ioc; + RemoteMemSync sync; + ProxyMemoryRegion region[PCI_NUM_REGIONS]; }; From patchwork Fri Jul 31 18:20:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E95D2C433E0 for ; Fri, 31 Jul 2020 18:31:08 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC41821744 for ; Fri, 31 Jul 2020 18:31:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="qaKCS81+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC41821744 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:45176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1ZoB-00072V-Uh for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:31:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58156) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeX-00028R-Gm for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:09 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:55024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZeU-0005Py-Pd for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:09 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIBmox069799; Fri, 31 Jul 2020 18:21:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=N1KyQJYew0usvujXlnKqIzHSk1mUFJmy66eWFBKdl1M=; b=qaKCS81+OmUJJF01vhMb+ZkHIxid7kRTFvpJKq5BPJX2JM4Hm5lnbsaL0nv28h7p19MN TaOdsqKv8g8dohSlJtjvaq/nCH1bR4UN0mEDvOUx2jKiSWKIHiuDTkkqAImthjcMk2YN 7aVJfqqykUI2OW4mID42h1GEtTTZSKKoTmAiy0yDDuBTklidDhPDHoyZ9AB15nz6SJQa iqmYLbKFV0c04gBeJKXMdr8hmEGC0SkqkConpT/186PosOX5xKGbN+USRNS6tOfmLB76 9612yiczxIaK7H3XL+YP/xzwrYFvm/aiCXdJ4u+37mJEwoJTKEXBYjop9G6a5O8s6uLk Pg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 32mf702u8v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:21:00 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIjUp080307; Fri, 31 Jul 2020 18:21:00 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 32mf70ve4t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:21:00 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 06VIKwxW020290; Fri, 31 Jul 2020 18:20:58 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:20:58 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 15/20] multi-process: create IOHUB object to handle irq Date: Fri, 31 Jul 2020 14:20:22 -0400 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=1 spamscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=1 spamscore=0 impostorscore=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.85; envelope-from=jag.raman@oracle.com; helo=userp2120.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 14:20:51 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" IOHUB object is added to manage PCI IRQs. It uses KVM_IRQFD ioctl to create irqfd to injecting PCI interrupts to the guest. IOHUB object forwards the irqfd to the remote process. Remote process uses this fd to directly send interrupts to the guest, bypassing QEMU. Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva --- MAINTAINERS | 2 + hw/Makefile.objs | 1 + hw/i386/remote-msg.c | 4 ++ hw/i386/remote.c | 10 ++++ hw/pci/proxy.c | 58 ++++++++++++++++++++++ hw/remote/Makefile.objs | 1 + hw/remote/iohub.c | 123 ++++++++++++++++++++++++++++++++++++++++++++++ include/hw/i386/remote.h | 3 ++ include/hw/pci/pci_ids.h | 3 ++ include/hw/pci/proxy.h | 5 ++ include/hw/remote/iohub.h | 42 ++++++++++++++++ include/io/mpqemu-link.h | 1 + io/mpqemu-link.c | 5 ++ 13 files changed, 258 insertions(+) create mode 100644 hw/remote/Makefile.objs create mode 100644 hw/remote/iohub.c create mode 100644 include/hw/remote/iohub.h diff --git a/MAINTAINERS b/MAINTAINERS index bd4f4ec..e75d010 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3055,6 +3055,8 @@ F: hw/pci/proxy.c F: include/hw/pci/proxy.h F: hw/pci/memory-sync.c F: include/hw/pci/memory-sync.h +F: hw/remote/iohub.c +F: include/hw/remote/iohub.h Build and test automation ------------------------- diff --git a/hw/Makefile.objs b/hw/Makefile.objs index 14b7ea4..4b2fa39 100644 --- a/hw/Makefile.objs +++ b/hw/Makefile.objs @@ -40,6 +40,7 @@ devices-dirs-$(CONFIG_MEM_DEVICE) += mem/ devices-dirs-$(CONFIG_NUBUS) += nubus/ devices-dirs-y += semihosting/ devices-dirs-y += smbios/ +devices-dirs-y += remote/ endif common-obj-y += $(devices-dirs-y) diff --git a/hw/i386/remote-msg.c b/hw/i386/remote-msg.c index 584498d..756b710 100644 --- a/hw/i386/remote-msg.c +++ b/hw/i386/remote-msg.c @@ -18,6 +18,7 @@ #include "hw/pci/pci.h" #include "exec/memattrs.h" #include "hw/i386/remote-memory.h" +#include "hw/remote/iohub.h" static void process_config_write(QIOChannel *ioc, PCIDevice *dev, MPQemuMsg *msg); @@ -71,6 +72,9 @@ gboolean mpqemu_process_msg(QIOChannel *ioc, GIOCondition cond, case SYNC_SYSMEM: remote_sysmem_reconfig(&msg, &local_err); break; + case SET_IRQFD: + process_set_irqfd_msg(pci_dev, &msg); + break; default: error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)", diff --git a/hw/i386/remote.c b/hw/i386/remote.c index a67be33..d4b5e33 100644 --- a/hw/i386/remote.c +++ b/hw/i386/remote.c @@ -20,12 +20,15 @@ #include "exec/address-spaces.h" #include "exec/memory.h" #include "qapi/error.h" +#include "hw/pci/pci_host.h" +#include "hw/remote/iohub.h" static void remote_machine_init(MachineState *machine) { MemoryRegion *system_memory, *system_io, *pci_memory; RemoteMachineState *s = REMOTE_MACHINE(machine); RemotePCIHost *rem_host; + PCIHostState *pci_host; system_memory = get_system_memory(); system_io = get_system_io(); @@ -45,6 +48,13 @@ static void remote_machine_init(MachineState *machine) memory_region_add_subregion_overlap(system_memory, 0x0, pci_memory, -1); qdev_realize(DEVICE(rem_host), sysbus_get_default(), &error_fatal); + + pci_host = PCI_HOST_BRIDGE(rem_host); + + remote_iohub_init(&s->iohub); + + pci_bus_irqs(pci_host->bus, remote_iohub_set_irq, remote_iohub_map_irq, + &s->iohub, REMOTE_IOHUB_NB_PIRQS); } static void remote_machine_class_init(ObjectClass *oc, void *data) diff --git a/hw/pci/proxy.c b/hw/pci/proxy.c index 28b8c76..1c1b682 100644 --- a/hw/pci/proxy.c +++ b/hw/pci/proxy.c @@ -19,6 +19,9 @@ #include "qemu/error-report.h" #include "hw/pci/memory-sync.h" #include "qom/object.h" +#include "qemu/event_notifier.h" +#include "sysemu/kvm.h" +#include "util/event_notifier-posix.c" static void proxy_set_socket(PCIProxyDev *pdev, int fd, Error **errp) { @@ -30,6 +33,56 @@ static Property proxy_properties[] = { DEFINE_PROP_END_OF_LIST(), }; +static void proxy_intx_update(PCIDevice *pci_dev) +{ + PCIProxyDev *dev = PCI_PROXY_DEV(pci_dev); + PCIINTxRoute route; + int pin = pci_get_byte(pci_dev->config + PCI_INTERRUPT_PIN) - 1; + + if (dev->virq != -1) { + kvm_irqchip_remove_irqfd_notifier_gsi(kvm_state, &dev->intr, dev->virq); + dev->virq = -1; + } + + route = pci_device_route_intx_to_irq(pci_dev, pin); + + dev->virq = route.irq; + + if (dev->virq != -1) { + kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, &dev->intr, + &dev->resample, dev->virq); + } +} + +static void setup_irqfd(PCIProxyDev *dev) +{ + PCIDevice *pci_dev = PCI_DEVICE(dev); + MPQemuMsg msg; + Error *local_err = NULL; + + event_notifier_init(&dev->intr, 0); + event_notifier_init(&dev->resample, 0); + + memset(&msg, 0, sizeof(MPQemuMsg)); + msg.cmd = SET_IRQFD; + msg.num_fds = 2; + msg.fds[0] = event_notifier_get_fd(&dev->intr); + msg.fds[1] = event_notifier_get_fd(&dev->resample); + msg.size = 0; + + mpqemu_msg_send(&msg, dev->ioc, &local_err); + if (local_err) { + error_report("Error to send cmd to remote process %d", + msg.cmd); + } + + dev->virq = -1; + + proxy_intx_update(pci_dev); + + pci_device_set_intx_routing_notifier(pci_dev, proxy_intx_update); +} + static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) { PCIProxyDev *dev = PCI_PROXY_DEV(device); @@ -48,6 +101,8 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) } configure_memory_sync(&dev->sync, dev->ioc); + + setup_irqfd(dev); } static void pci_proxy_dev_exit(PCIDevice *pdev) @@ -57,6 +112,9 @@ static void pci_proxy_dev_exit(PCIDevice *pdev) qio_channel_close(dev->ioc, NULL); deconfigure_memory_sync(&dev->sync); + + event_notifier_cleanup(&dev->intr); + event_notifier_cleanup(&dev->resample); } static int config_op_send(PCIProxyDev *pdev, uint32_t addr, uint32_t *val, diff --git a/hw/remote/Makefile.objs b/hw/remote/Makefile.objs new file mode 100644 index 0000000..635ce5e --- /dev/null +++ b/hw/remote/Makefile.objs @@ -0,0 +1 @@ +common-obj-$(CONFIG_MPQEMU) += iohub.o diff --git a/hw/remote/iohub.c b/hw/remote/iohub.c new file mode 100644 index 0000000..751e6d8 --- /dev/null +++ b/hw/remote/iohub.c @@ -0,0 +1,123 @@ +/* + * Remote IO Hub + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "qemu-common.h" + +#include "hw/pci/pci.h" +#include "hw/pci/pci_ids.h" +#include "hw/pci/pci_bus.h" +#include "hw/remote/iohub.h" +#include "qemu/thread.h" +#include "hw/boards.h" +#include "hw/i386/remote.h" +#include "qemu/main-loop.h" + +void remote_iohub_init(RemoteIOHubState *iohub) +{ + int pirq; + + memset(&iohub->irqfds, 0, sizeof(iohub->irqfds)); + memset(&iohub->resamplefds, 0, sizeof(iohub->resamplefds)); + + for (pirq = 0; pirq < REMOTE_IOHUB_NB_PIRQS; pirq++) { + qemu_mutex_init(&iohub->irq_level_lock[pirq]); + iohub->irq_level[pirq] = 0; + event_notifier_init_fd(&iohub->irqfds[pirq], -1); + event_notifier_init_fd(&iohub->resamplefds[pirq], -1); + } +} + +void remote_iohub_finalize(RemoteIOHubState *iohub) +{ + int pirq; + + for (pirq = 0; pirq < REMOTE_IOHUB_NB_PIRQS; pirq++) { + qemu_set_fd_handler(event_notifier_get_fd(&iohub->resamplefds[pirq]), + NULL, NULL, NULL); + event_notifier_cleanup(&iohub->irqfds[pirq]); + event_notifier_cleanup(&iohub->resamplefds[pirq]); + qemu_mutex_destroy(&iohub->irq_level_lock[pirq]); + } +} + +int remote_iohub_map_irq(PCIDevice *pci_dev, int intx) +{ + return pci_dev->devfn; +} + +void remote_iohub_set_irq(void *opaque, int pirq, int level) +{ + RemoteIOHubState *iohub = opaque; + + assert(pirq >= 0); + assert(pirq < PCI_DEVFN_MAX); + + qemu_mutex_lock(&iohub->irq_level_lock[pirq]); + + if (level) { + if (++iohub->irq_level[pirq] == 1) { + event_notifier_set(&iohub->irqfds[pirq]); + } + } else if (iohub->irq_level[pirq] > 0) { + iohub->irq_level[pirq]--; + } + + qemu_mutex_unlock(&iohub->irq_level_lock[pirq]); +} + +static void intr_resample_handler(void *opaque) +{ + ResampleToken *token = opaque; + RemoteIOHubState *iohub = token->iohub; + int pirq, s; + + pirq = token->pirq; + + s = event_notifier_test_and_clear(&iohub->resamplefds[pirq]); + + assert(s >= 0); + + qemu_mutex_lock(&iohub->irq_level_lock[pirq]); + + if (iohub->irq_level[pirq]) { + event_notifier_set(&iohub->irqfds[pirq]); + } + + qemu_mutex_unlock(&iohub->irq_level_lock[pirq]); +} + +void process_set_irqfd_msg(PCIDevice *pci_dev, MPQemuMsg *msg) +{ + RemoteMachineState *machine = REMOTE_MACHINE(current_machine); + RemoteIOHubState *iohub = &machine->iohub; + int pirq, intx; + + intx = pci_get_byte(pci_dev->config + PCI_INTERRUPT_PIN) - 1; + + pirq = remote_iohub_map_irq(pci_dev, intx); + + if (event_notifier_get_fd(&iohub->irqfds[pirq]) != -1) { + qemu_set_fd_handler(event_notifier_get_fd(&iohub->resamplefds[pirq]), + NULL, NULL, NULL); + event_notifier_cleanup(&iohub->irqfds[pirq]); + event_notifier_cleanup(&iohub->resamplefds[pirq]); + memset(&iohub->token[pirq], 0, sizeof(ResampleToken)); + } + + event_notifier_init_fd(&iohub->irqfds[pirq], msg->fds[0]); + event_notifier_init_fd(&iohub->resamplefds[pirq], msg->fds[1]); + + iohub->token[pirq].iohub = iohub; + iohub->token[pirq].pirq = pirq; + + qemu_set_fd_handler(msg->fds[1], intr_resample_handler, NULL, + &iohub->token[pirq]); +} diff --git a/include/hw/i386/remote.h b/include/hw/i386/remote.h index 7f9028f..01deccd 100644 --- a/include/hw/i386/remote.h +++ b/include/hw/i386/remote.h @@ -15,11 +15,14 @@ #include "hw/boards.h" #include "hw/pci-host/remote.h" #include "io/channel.h" +#include "hw/remote/iohub.h" typedef struct RemoteMachineState { MachineState parent_obj; RemotePCIHost *host; + + RemoteIOHubState iohub; } RemoteMachineState; #define TYPE_REMOTE_MACHINE "remote-machine" diff --git a/include/hw/pci/pci_ids.h b/include/hw/pci/pci_ids.h index 11f8ab7..bd0c17d 100644 --- a/include/hw/pci/pci_ids.h +++ b/include/hw/pci/pci_ids.h @@ -192,6 +192,9 @@ #define PCI_DEVICE_ID_SUN_SIMBA 0x5000 #define PCI_DEVICE_ID_SUN_SABRE 0xa000 +#define PCI_VENDOR_ID_ORACLE 0x108e +#define PCI_DEVICE_ID_REMOTE_IOHUB 0xb000 + #define PCI_VENDOR_ID_CMD 0x1095 #define PCI_DEVICE_ID_CMD_646 0x0646 diff --git a/include/hw/pci/proxy.h b/include/hw/pci/proxy.h index 3ff79f4..15cc381 100644 --- a/include/hw/pci/proxy.h +++ b/include/hw/pci/proxy.h @@ -12,6 +12,7 @@ #include "hw/pci/pci.h" #include "io/channel.h" #include "hw/pci/memory-sync.h" +#include "qemu/event_notifier.h" #define TYPE_PCI_PROXY_DEV "pci-proxy-dev" @@ -35,6 +36,10 @@ struct PCIProxyDev { RemoteMemSync sync; + int virq; + EventNotifier intr; + EventNotifier resample; + ProxyMemoryRegion region[PCI_NUM_REGIONS]; }; diff --git a/include/hw/remote/iohub.h b/include/hw/remote/iohub.h new file mode 100644 index 0000000..cd59476 --- /dev/null +++ b/include/hw/remote/iohub.h @@ -0,0 +1,42 @@ +/* + * IO Hub for remote device + * + * Copyright © 2018, 2020 Oracle and/or its affiliates. + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef REMOTE_IOHUB_H +#define REMOTE_IOHUB_H + +#include "hw/pci/pci.h" +#include "qemu/event_notifier.h" +#include "qemu/thread-posix.h" +#include "io/mpqemu-link.h" + +#define REMOTE_IOHUB_NB_PIRQS PCI_DEVFN_MAX + +typedef struct ResampleToken { + void *iohub; + int pirq; +} ResampleToken; + +typedef struct RemoteIOHubState { + PCIDevice d; + EventNotifier irqfds[REMOTE_IOHUB_NB_PIRQS]; + EventNotifier resamplefds[REMOTE_IOHUB_NB_PIRQS]; + unsigned int irq_level[REMOTE_IOHUB_NB_PIRQS]; + ResampleToken token[REMOTE_IOHUB_NB_PIRQS]; + QemuMutex irq_level_lock[REMOTE_IOHUB_NB_PIRQS]; +} RemoteIOHubState; + +int remote_iohub_map_irq(PCIDevice *pci_dev, int intx); +void remote_iohub_set_irq(void *opaque, int pirq, int level); +void process_set_irqfd_msg(PCIDevice *pci_dev, MPQemuMsg *msg); + +void remote_iohub_init(RemoteIOHubState *iohub); +void remote_iohub_finalize(RemoteIOHubState *iohub); + +#endif diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h index ee3b44f..6625de6 100644 --- a/include/io/mpqemu-link.h +++ b/include/io/mpqemu-link.h @@ -38,6 +38,7 @@ typedef enum { PCI_CONFIG_READ, BAR_WRITE, BAR_READ, + SET_IRQFD, MAX = INT_MAX, } MPQemuCmd; diff --git a/io/mpqemu-link.c b/io/mpqemu-link.c index 82b8465..6fa4665 100644 --- a/io/mpqemu-link.c +++ b/io/mpqemu-link.c @@ -275,6 +275,11 @@ bool mpqemu_msg_valid(MPQemuMsg *msg) return false; } break; + case SET_IRQFD: + if (msg->size || (msg->num_fds != 2)) { + return false; + } + break; default: break; } From patchwork Fri Jul 31 18:20:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB064C433E0 for ; Fri, 31 Jul 2020 18:36:38 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F00E208E4 for ; Fri, 31 Jul 2020 18:36:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="E6go9cFL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F00E208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:32988 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1ZtV-0005NP-S3 for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:36:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZgX-0004bX-Lw for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:23:13 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:52808) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1ZgV-0005cp-JI for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:23:13 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIN38i119524; Fri, 31 Jul 2020 18:23:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2020-01-29; bh=9l9pS4Sinj0utCiirwhSm+iEc4fU8TW4CMeiAbDwD2I=; b=E6go9cFL4DqMmpI4TYkXKYfCFN4ltAa8B/1ZV3EWPd8ycxsGOp9+yAROtkf7knvzvBoy 8SUMS8le//dlzYU81zEfn7FwJl69Ik0tPzVQciB3QLdGP/4sgjjqtIlElLshtYNexQUl CMiEWdsU3NvMh68tThhmf9smInVXtft4Y8U8PkCk7P+QdBwu5PCYG2qylFQWHoRLP/bm 9nzBtNJkGiwolavx1wracGh4LSBtHGerOA85k0f9gEbwWaFfKrf4EgbKfJ3IiG+Dwz9N 87g9D7NApmuFZwc3cUegYdUOAyYyLBFuR1q7XM8uWUe6wtsTGOBT3MM1RUikYndVlE+g Hw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 32hu1jtgbq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:23:03 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIcTL137654; Fri, 31 Jul 2020 18:21:02 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 32hu64r7jn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:21:02 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 06VIL11w023774; Fri, 31 Jul 2020 18:21:01 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:21:01 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 17/20] multi-process: heartbeat messages to remote Date: Fri, 31 Jul 2020 14:20:24 -0400 Message-Id: <93b7566e5d565b9e5d8127849bb5be65057e25cc.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 suspectscore=3 bulkscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 lowpriorityscore=0 impostorscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 Received-SPF: pass client-ip=156.151.31.86; envelope-from=jag.raman@oracle.com; helo=userp2130.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 12:51:59 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -63 X-Spam_score: -6.4 X-Spam_bar: ------ X-Spam_report: (-6.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva In order to detect remote processes which are hung, the proxy periodically sends heartbeat messages to confirm if the remote process is alive. The remote process responds to this heartbeat message to confirm it is alive. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva --- hw/i386/remote-msg.c | 19 ++++++++++++++++++ hw/pci/proxy.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/proxy.h | 2 ++ include/io/mpqemu-link.h | 1 + io/mpqemu-link.c | 8 ++++++++ 5 files changed, 82 insertions(+) diff --git a/hw/i386/remote-msg.c b/hw/i386/remote-msg.c index 756b710..2a4d7f1 100644 --- a/hw/i386/remote-msg.c +++ b/hw/i386/remote-msg.c @@ -26,6 +26,7 @@ static void process_config_read(QIOChannel *ioc, PCIDevice *dev, MPQemuMsg *msg); static void process_bar_write(QIOChannel *ioc, MPQemuMsg *msg, Error **errp); static void process_bar_read(QIOChannel *ioc, MPQemuMsg *msg, Error **errp); +static void process_proxy_ping_msg(QIOChannel *ioc, Error **errp); gboolean mpqemu_process_msg(QIOChannel *ioc, GIOCondition cond, gpointer opaque) @@ -75,6 +76,9 @@ gboolean mpqemu_process_msg(QIOChannel *ioc, GIOCondition cond, case SET_IRQFD: process_set_irqfd_msg(pci_dev, &msg); break; + case PROXY_PING: + process_proxy_ping_msg(ioc, &local_err); + break; default: error_setg(&local_err, "Unknown command (%d) received for device %s (pid=%d)", @@ -223,3 +227,18 @@ fail: "in remote process pid=%d", getpid()); } } + +static void process_proxy_ping_msg(QIOChannel *ioc, Error **errp) +{ + MPQemuMsg ret = { 0 }; + Error *local_err = NULL; + + ret.cmd = RET_MSG; + ret.size = sizeof(ret.data1); + + mpqemu_msg_send(&ret, ioc, &local_err); + if (local_err) { + error_setg(errp, "Error while sending message to proxy " + "in remote process pid=%d", getpid()); + } +} diff --git a/hw/pci/proxy.c b/hw/pci/proxy.c index 50a806c..490093c 100644 --- a/hw/pci/proxy.c +++ b/hw/pci/proxy.c @@ -24,6 +24,8 @@ #include "util/event_notifier-posix.c" static void probe_pci_info(PCIDevice *dev, Error **errp); +static void start_hb_timer(PCIProxyDev *dev); +static void stop_hb_timer(PCIProxyDev *dev); static void proxy_set_socket(PCIProxyDev *pdev, int fd, Error **errp) { @@ -111,6 +113,8 @@ static void pci_proxy_dev_realize(PCIDevice *device, Error **errp) setup_irqfd(dev); probe_pci_info(PCI_DEVICE(dev), errp); + + start_hb_timer(dev); } static void pci_proxy_dev_exit(PCIDevice *pdev) @@ -123,6 +127,8 @@ static void pci_proxy_dev_exit(PCIDevice *pdev) event_notifier_cleanup(&dev->intr); event_notifier_cleanup(&dev->resample); + + stop_hb_timer(dev); } static int config_op_send(PCIProxyDev *pdev, uint32_t addr, uint32_t *val, @@ -343,3 +349,49 @@ static void probe_pci_info(PCIDevice *dev, Error **errp) } } } + +static void hb_msg(PCIProxyDev *dev) +{ + DeviceState *ds = DEVICE(dev); + Error *local_err = NULL; + MPQemuMsg msg = { 0 }; + + msg.cmd = PROXY_PING; + msg.bytestream = 0; + msg.size = 0; + + (void)mpqemu_msg_send_and_await_reply(&msg, dev->ioc, &local_err); + if (local_err) { + error_report_err(local_err); + qio_channel_close(dev->ioc, &local_err); + error_setg(&error_fatal, "Lost contact with device %s", ds->id); + } +} + +#define NOP_INTERVAL 1000 + +static void remote_ping(void *opaque) +{ + PCIProxyDev *dev = opaque; + + hb_msg(dev); + + timer_mod(dev->hb_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + NOP_INTERVAL); +} + +static void start_hb_timer(PCIProxyDev *dev) +{ + dev->hb_timer = timer_new_ms(QEMU_CLOCK_VIRTUAL, + remote_ping, + dev); + + timer_mod(dev->hb_timer, + qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + NOP_INTERVAL); +} + +static void stop_hb_timer(PCIProxyDev *dev) +{ + timer_del(dev->hb_timer); + timer_free(dev->hb_timer); +} diff --git a/include/hw/pci/proxy.h b/include/hw/pci/proxy.h index 15cc381..d784328 100644 --- a/include/hw/pci/proxy.h +++ b/include/hw/pci/proxy.h @@ -40,6 +40,8 @@ struct PCIProxyDev { EventNotifier intr; EventNotifier resample; + QEMUTimer *hb_timer; + ProxyMemoryRegion region[PCI_NUM_REGIONS]; }; diff --git a/include/io/mpqemu-link.h b/include/io/mpqemu-link.h index a3844e1..5f2913f 100644 --- a/include/io/mpqemu-link.h +++ b/include/io/mpqemu-link.h @@ -40,6 +40,7 @@ typedef enum { BAR_READ, SET_IRQFD, GET_PCI_INFO, + PROXY_PING, MAX = INT_MAX, } MPQemuCmd; diff --git a/io/mpqemu-link.c b/io/mpqemu-link.c index 6fa4665..0222e81 100644 --- a/io/mpqemu-link.c +++ b/io/mpqemu-link.c @@ -280,6 +280,14 @@ bool mpqemu_msg_valid(MPQemuMsg *msg) return false; } break; + case PROXY_PING: + if (msg->bytestream || msg->num_fds) { + return false; + } + if (msg->size) { + return false; + } + break; default: break; } From patchwork Fri Jul 31 18:20:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jag Raman X-Patchwork-Id: 277212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73318C433DF for ; Fri, 31 Jul 2020 18:29:51 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 447D822B42 for ; Fri, 31 Jul 2020 18:29:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="niq9l6xU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 447D822B42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:41062 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k1Zmw-0005Gp-Hu for qemu-devel@archiver.kernel.org; Fri, 31 Jul 2020 14:29:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58240) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1Zef-0002S2-61 for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:17 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:55140) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k1Zec-0005Rq-PA for qemu-devel@nongnu.org; Fri, 31 Jul 2020 14:21:16 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIBwu3069937; Fri, 31 Jul 2020 18:21:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : in-reply-to : references; s=corp-2020-01-29; bh=z+B+7nLzsZ06MKfna63YXrq7L1SIE6BY4AOI+9Toi4I=; b=niq9l6xUi5/EEfdXzssoZ4DeT/ZVFtwnDxBnHXIUzY9qQPwYvrjINAHsGv3OY1wZVoaS tUjipY68xqOuWZOoJJCOsrrDwepV4Fss0BfPqgXdCO+bkBwDgW06OEcYR6+nEI+vxayJ UVqospUO7YtJ9oxs5CtX6ADqrhXXKgrQ9Ot5ts8YBvml8ojnJB5LlyFxaW4DQb4iHTu9 ZzyfnRS4yYg/xFfPKR34gmzA7kLNMyZLx0l/+jJPqN41zWTbxpNKvVzjlE4l1NYu6wTT 0Ot7usznukllUPckgsYsBF7qg6o9a7I5cg+H69C9ewZnKor4BgUvzSa+s7PMmhcmVFwk qQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 32mf702u9c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 31 Jul 2020 18:21:07 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 06VIIccS137659; Fri, 31 Jul 2020 18:21:07 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 32hu64r7rm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 31 Jul 2020 18:21:06 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 06VIL67Y013536; Fri, 31 Jul 2020 18:21:06 GMT Received: from jaraman-bur-1.us.oracle.com (/10.152.33.39) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 31 Jul 2020 11:21:05 -0700 From: Jagannathan Raman To: qemu-devel@nongnu.org Subject: [PATCH v8 20/20] multi-process: add configure and usage information Date: Fri, 31 Jul 2020 14:20:27 -0400 Message-Id: <4d3d55b69bfb5a7a10c9b4bc5b1b2dcbf160084b.1596217462.git.jag.raman@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 adultscore=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310137 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9699 signatures=668679 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 bulkscore=0 suspectscore=1 spamscore=0 impostorscore=0 clxscore=1015 phishscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2007310136 Received-SPF: pass client-ip=156.151.31.85; envelope-from=jag.raman@oracle.com; helo=userp2120.oracle.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/31 14:20:51 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: elena.ufimtseva@oracle.com, fam@euphon.net, swapnil.ingle@nutanix.com, john.g.johnson@oracle.com, kraxel@redhat.com, jag.raman@oracle.com, quintela@redhat.com, mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com, felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com, konrad.wilk@oracle.com, dgilbert@redhat.com, alex.williamson@redhat.com, stefanha@redhat.com, thanos.makatos@nutanix.com, rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com, mreitz@redhat.com, ross.lagerwall@citrix.com, marcandre.lureau@gmail.com, pbonzini@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Elena Ufimtseva Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson --- MAINTAINERS | 2 ++ docs/multi-process.rst | 67 ++++++++++++++++++++++++++++++++++++++++++++++ scripts/mpqemu-launcher.py | 49 +++++++++++++++++++++++++++++++++ 3 files changed, 118 insertions(+) create mode 100644 docs/multi-process.rst create mode 100755 scripts/mpqemu-launcher.py diff --git a/MAINTAINERS b/MAINTAINERS index f41b54c..3df9441 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3058,6 +3058,8 @@ F: include/hw/pci/memory-sync.h F: hw/remote/iohub.c F: include/hw/remote/iohub.h F: docs/devel/multi-process.rst +F: scripts/mpqemu-launcher.py +F: scripts/mpqemu-launcher-perf-mode.py Build and test automation ------------------------- diff --git a/docs/multi-process.rst b/docs/multi-process.rst new file mode 100644 index 0000000..9dc55e3 --- /dev/null +++ b/docs/multi-process.rst @@ -0,0 +1,67 @@ +Multi-process QEMU +================== + +This document describes how to configure and use multi-process qemu. +For the design document refer to docs/devel/qemu-multiprocess. + +1) Configuration +---------------- + +To enable support for multi-process add --enable-mpqemu +to the list of options for the "configure" script. + + +2) Usage +-------- + +Multi-process QEMU requires an orchestrator to launch. Please refer to a +light-weight python based orchestrator for mpqemu in +scripts/mpqemu-launcher.py to lauch QEMU in multi-process mode. + +Following is a description of command-line used to launch mpqemu. + +* Orchestrator: + + - The Orchestrator creates a unix socketpair + + - It launches the remote process and passes one of the + sockets to it via command-line. + + - It then launches QEMU and specifies the other socket as an option + to the Proxy device object + +* Remote Process: + + - QEMU can enter remote process mode by using the "remote" machine + option. + + - The orchestrator creates a "remote-object" with details about + the device and the file descriptor for the device + + - The remaining options are no different from how one launches QEMU with + devices. + + - Example command-line for the remote process is as follows: + + /usr/bin/qemu-system-x86_64 \ + -machine remote \ + -device lsi53c895a,id=lsi0 \ + -drive id=drive_image2,file=/build/ol7-nvme-test-1.qcow2 \ + -device scsi-hd,id=drive2,drive=drive_image2,bus=lsi0.0,scsi-id=0 \ + -object remote-object,id=robj1,devid=lsi1,fd=4, + +* QEMU: + + - Since parts of the RAM are shared between QEMU & remote process, a + memory-backend-memfd is required to facilitate this, as follows: + + -object memory-backend-memfd,id=mem,size=2G + + - A "pci-proxy-dev" device is created for each of the PCI devices emulated + in the remote process. A "socket" sub-option specifies the other end of + unix channel created by orchestrator. The "id" sub-option must be specified + and should be the same as the "id" specified for the remote PCI device + + - Example commandline for QEMU is as follows: + + -device pci-proxy-dev,id=lsi0,socket=3 diff --git a/scripts/mpqemu-launcher.py b/scripts/mpqemu-launcher.py new file mode 100755 index 0000000..07b11c6 --- /dev/null +++ b/scripts/mpqemu-launcher.py @@ -0,0 +1,49 @@ +#!/usr/bin/env python3 +import socket +import os +import subprocess +import time + +PROC_QEMU='/usr/bin/qemu-system-x86_64' + +proxy, remote = socket.socketpair(socket.AF_UNIX, socket.SOCK_STREAM) + +remote_cmd = [ PROC_QEMU, \ + '-machine', 'remote', \ + '-device', 'lsi53c895a,id=lsi1', \ + '-drive', 'id=drive_image1,file=/build/ol7-nvme-test-1.qcow2', \ + '-device', 'scsi-hd,id=drive1,drive=drive_image1,bus=lsi1.0,' \ + 'scsi-id=0', \ + '-object', \ + 'remote-object,id=robj1,devid=lsi1,fd='+str(remote.fileno()), \ + '-nographic', \ + ] + +proxy_cmd = [ PROC_QEMU, \ + '-name', 'OL7.4', \ + '-machine', 'q35,accel=kvm', \ + '-smp', 'sockets=1,cores=1,threads=1', \ + '-m', '2048', \ + '-object', 'memory-backend-memfd,id=sysmem-file,size=2G', \ + '-numa', 'node,memdev=sysmem-file', \ + '-device', 'virtio-scsi-pci,id=virtio_scsi_pci0', \ + '-drive', 'id=drive_image1,if=none,format=qcow2,' \ + 'file=/home/ol7-hdd-1.qcow2', \ + '-device', 'scsi-hd,id=image1,drive=drive_image1,' \ + 'bus=virtio_scsi_pci0.0', \ + '-boot', 'd', \ + '-vnc', ':0', \ + '-device', 'pci-proxy-dev,id=lsi1,fd='+str(proxy.fileno()), \ + ] + + +pid = os.fork(); + +if pid: + # In Proxy + print('Launching QEMU with Proxy object'); + process = subprocess.Popen(proxy_cmd, pass_fds=[proxy.fileno()]) +else: + # In remote + print('Launching Remote process'); + process = subprocess.Popen(remote_cmd, pass_fds=[remote.fileno()])