mbox series

[RFC,v3,0/7] Use ACPI PCI hot-plug for Q35

Message ID 20200924070013.165026-1-jusual@redhat.com
Headers show
Series Use ACPI PCI hot-plug for Q35 | expand

Message

Julia Suvorova Sept. 24, 2020, 7 a.m. UTC
The patch set consists of two parts:
patches 1-4: introduce new feature
             'acpi-pci-hotplug-with-bridge-support' on Q35
patches 5-7: make the feature default along with changes in ACPI tables

This way maintainers can decide which way to choose without breaking
the patch set.

With the feature disabled Q35 falls back to the native hot-plug.

Pros
    * no racy behavior during boot (see 110c477c2ed)
    * eject is possible - according to PCIe spec, attention button
      press should lead to power off, and then the adapter should be
      removed manually. As there is no power down state exists in QEMU,
      we cannot distinguish between an eject and a power down
      request.
    * no delay during deleting - after the actual power off software
      must wait at least 1 second before indicating about it. This case
      is quite important for users, it even has its own bug:
          https://bugzilla.redhat.com/show_bug.cgi?id=1594168
    * no timer-based behavior - in addition to the previous example,
      the attention button has a 5-second waiting period, during which
      the operation can be canceled with a second press. While this
      looks fine for manual button control, automation will result in
      the need to queue or drop events, and the software receiving
      events in all sort of unspecified combinations of attention/power
      indicator states, which is racy and uppredictable.
    * fixes:
        * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
        * https://bugzilla.redhat.com/show_bug.cgi?id=1690256

Cons:
    * lose per-port control over hot-plug (can be resolved)
    * no access to possible features presented in slot capabilities
      (this is only surprise removal AFAIK)

v3:
    * drop change of _OSC to allow SHPC on hotplugged bridges
    * use 'acpi-root-pci-hotplug'
    * add migration states [Igor]
    * minor style changes

v2:
    * new ioport range for acpiphp [Gerd]
    * drop find_pci_host() [Igor]
    * explain magic numbers in _OSC [Igor]
    * drop build_q35_pci_hotplug() wrapper [Igor]

Julia Suvorova (7):
  hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
  hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
  hw/pci/pcie: Do not initialize slot capability if acpihp is used
  hw/acpi/ich9: Enable ACPI PCI hot-plug
  bios-tables-test: Allow changes in DSDT ACPI tables
  hw/acpi/ich9: Set ACPI PCI hot-plug as default
  bios-tables-test: Update golden binaries

 hw/i386/acpi-build.h              |   7 ++++
 include/hw/acpi/ich9.h            |   5 +++
 include/hw/acpi/pcihp.h           |   3 +-
 hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
 hw/acpi/pcihp.c                   |  16 ++++---
 hw/acpi/piix4.c                   |   4 +-
 hw/i386/acpi-build.c              |  31 ++++++++------
 hw/i386/pc.c                      |   1 +
 hw/pci/pcie.c                     |  16 +++++++
 tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
 tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
 tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
 tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
 tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
 tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
 tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
 tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
 tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
 tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
 19 files changed, 129 insertions(+), 21 deletions(-)

Comments

Michael S. Tsirkin Sept. 24, 2020, 7:36 a.m. UTC | #1
On Thu, Sep 24, 2020 at 09:00:09AM +0200, Julia Suvorova wrote:
> Instead of changing the hot-plug type in _OSC register, do not
> initialize the slot capability or set the 'Slot Implemented' flag.
> This way guest will choose ACPI hot-plug if it is preferred and leave
> the option to use SHPC with pcie-pci-bridge.
> 
> Signed-off-by: Julia Suvorova <jusual@redhat.com>
> ---
>  hw/i386/acpi-build.h |  2 ++
>  hw/i386/acpi-build.c |  2 +-
>  hw/pci/pcie.c        | 16 ++++++++++++++++
>  3 files changed, 19 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h
> index 487ec7710f..4c5bfb3d0b 100644
> --- a/hw/i386/acpi-build.h
> +++ b/hw/i386/acpi-build.h
> @@ -11,4 +11,6 @@ extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;
>  
>  void acpi_setup(void);
>  
> +Object *object_resolve_type_unambiguous(const char *typename);
> +
>  #endif
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index cf503b16af..b7811a8912 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -174,7 +174,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,
>      *data = fadt;
>  }
>  
> -static Object *object_resolve_type_unambiguous(const char *typename)
> +Object *object_resolve_type_unambiguous(const char *typename)
>  {
>      bool ambig;
>      Object *o = object_resolve_path_type("", typename, &ambig);
> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
> index 5b48bae0f6..c1a082e8b9 100644
> --- a/hw/pci/pcie.c
> +++ b/hw/pci/pcie.c
> @@ -27,6 +27,8 @@
>  #include "hw/pci/pci_bus.h"
>  #include "hw/pci/pcie_regs.h"
>  #include "hw/pci/pcie_port.h"
> +#include "hw/i386/ich9.h"
> +#include "hw/i386/acpi-build.h"
>  #include "qemu/range.h"
>  
>  //#define DEBUG_PCIE


Not really happy with pcie.c getting an i386 dependency.



> @@ -515,12 +517,26 @@ void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev,
>      pcie_cap_slot_push_attention_button(hotplug_pdev);
>  }
>  
> +static bool acpi_pcihp_enabled(void)
> +{
> +    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);
> +
> +    return lpc &&
> +           object_property_get_bool(lpc, "acpi-pci-hotplug-with-bridge-support",
> +                                    NULL);
> +
> +}
> +

Why not just check the property unconditionally?


>  /* pci express slot for pci express root/downstream port
>     PCI express capability slot registers */
>  void pcie_cap_slot_init(PCIDevice *dev, PCIESlot *s)
>  {
>      uint32_t pos = dev->exp.exp_cap;
>  
> +    if (acpi_pcihp_enabled()) {
> +        return;
> +    }
> +

I think I would rather not teach pcie about acpi. How about we
change the polarity, name the property
"pci-native-hotplug" or whatever makes sense.

>      pci_word_test_and_set_mask(dev->config + pos + PCI_EXP_FLAGS,
>                                 PCI_EXP_FLAGS_SLOT);
>  
> -- 
> 2.25.4
Julia Suvorova Sept. 24, 2020, 8:23 a.m. UTC | #2
On Thu, Sep 24, 2020 at 9:36 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>

> On Thu, Sep 24, 2020 at 09:00:09AM +0200, Julia Suvorova wrote:

> > Instead of changing the hot-plug type in _OSC register, do not

> > initialize the slot capability or set the 'Slot Implemented' flag.

> > This way guest will choose ACPI hot-plug if it is preferred and leave

> > the option to use SHPC with pcie-pci-bridge.

> >

> > Signed-off-by: Julia Suvorova <jusual@redhat.com>

> > ---

> >  hw/i386/acpi-build.h |  2 ++

> >  hw/i386/acpi-build.c |  2 +-

> >  hw/pci/pcie.c        | 16 ++++++++++++++++

> >  3 files changed, 19 insertions(+), 1 deletion(-)

> >

> > diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h

> > index 487ec7710f..4c5bfb3d0b 100644

> > --- a/hw/i386/acpi-build.h

> > +++ b/hw/i386/acpi-build.h

> > @@ -11,4 +11,6 @@ extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;

> >

> >  void acpi_setup(void);

> >

> > +Object *object_resolve_type_unambiguous(const char *typename);

> > +

> >  #endif

> > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c

> > index cf503b16af..b7811a8912 100644

> > --- a/hw/i386/acpi-build.c

> > +++ b/hw/i386/acpi-build.c

> > @@ -174,7 +174,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,

> >      *data = fadt;

> >  }

> >

> > -static Object *object_resolve_type_unambiguous(const char *typename)

> > +Object *object_resolve_type_unambiguous(const char *typename)

> >  {

> >      bool ambig;

> >      Object *o = object_resolve_path_type("", typename, &ambig);

> > diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c

> > index 5b48bae0f6..c1a082e8b9 100644

> > --- a/hw/pci/pcie.c

> > +++ b/hw/pci/pcie.c

> > @@ -27,6 +27,8 @@

> >  #include "hw/pci/pci_bus.h"

> >  #include "hw/pci/pcie_regs.h"

> >  #include "hw/pci/pcie_port.h"

> > +#include "hw/i386/ich9.h"

> > +#include "hw/i386/acpi-build.h"

> >  #include "qemu/range.h"

> >

> >  //#define DEBUG_PCIE

>

>

> Not really happy with pcie.c getting an i386 dependency.

>

>

>

> > @@ -515,12 +517,26 @@ void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev,

> >      pcie_cap_slot_push_attention_button(hotplug_pdev);

> >  }

> >

> > +static bool acpi_pcihp_enabled(void)

> > +{

> > +    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);

> > +

> > +    return lpc &&

> > +           object_property_get_bool(lpc, "acpi-pci-hotplug-with-bridge-support",

> > +                                    NULL);

> > +

> > +}

> > +

>

> Why not just check the property unconditionally?


Ok.

> >  /* pci express slot for pci express root/downstream port

> >     PCI express capability slot registers */

> >  void pcie_cap_slot_init(PCIDevice *dev, PCIESlot *s)

> >  {

> >      uint32_t pos = dev->exp.exp_cap;

> >

> > +    if (acpi_pcihp_enabled()) {

> > +        return;

> > +    }

> > +

>

> I think I would rather not teach pcie about acpi. How about we

> change the polarity, name the property

> "pci-native-hotplug" or whatever makes sense.


I'd prefer not to change the property name since the common code in
hw/i386/acpi-build.c depends on it, but I can add a new one if it
makes any sense.

> >      pci_word_test_and_set_mask(dev->config + pos + PCI_EXP_FLAGS,

> >                                 PCI_EXP_FLAGS_SLOT);

> >

> > --

> > 2.25.4

>
no-reply@patchew.org Sept. 24, 2020, 8:57 a.m. UTC | #3
Patchew URL: https://patchew.org/QEMU/20200924070013.165026-1-jusual@redhat.com/



Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===

Host machine cpu: x86_64
Target machine cpu family: x86
Target machine cpu: x86_64
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
The manual pages are in docs.
/usr/lib/gcc/x86_64-w64-mingw32/9.2.1/../../../../x86_64-w64-mingw32/bin/ld: libcommon.fa.p/hw_pci_pcie.c.obj: in function `acpi_pcihp_enabled':
/tmp/qemu-test/build/../src/hw/pci/pcie.c:522: undefined reference to `object_resolve_type_unambiguous'
collect2: error: ld returned 1 exit status
make: *** [Makefile.ninja:1878: qemu-system-aarch64w.exe] Error 1
make: *** Waiting for unfinished jobs....
/usr/lib/gcc/x86_64-w64-mingw32/9.2.1/../../../../x86_64-w64-mingw32/bin/ld: libcommon.fa.p/hw_pci_pcie.c.obj: in function `acpi_pcihp_enabled':
/tmp/qemu-test/build/../src/hw/pci/pcie.c:522: undefined reference to `object_resolve_type_unambiguous'
collect2: error: ld returned 1 exit status
make: *** [Makefile.ninja:1876: qemu-system-aarch64.exe] Error 1
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 709, in <module>
    sys.exit(main())
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--rm', '--label', 'com.qemu.instance.uuid=342bc147a50847f886da9b01c0ca5ec9', '-u', '1003', '--security-opt', 'seccomp=unconfined', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-vmsaoknh/src/docker-src.2020-09-24-04.53.10.30016:/var/tmp/qemu:z,ro', 'qemu/fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=342bc147a50847f886da9b01c0ca5ec9
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-vmsaoknh/src'
make: *** [docker-run-test-mingw@fedora] Error 2

real    4m42.837s
user    0m18.829s


The full log is available at
http://patchew.org/logs/20200924070013.165026-1-jusual@redhat.com/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Michael S. Tsirkin Sept. 24, 2020, 9:02 a.m. UTC | #4
On Thu, Sep 24, 2020 at 10:23:13AM +0200, Julia Suvorova wrote:
> On Thu, Sep 24, 2020 at 9:36 AM Michael S. Tsirkin <mst@redhat.com> wrote:

> >

> > On Thu, Sep 24, 2020 at 09:00:09AM +0200, Julia Suvorova wrote:

> > > Instead of changing the hot-plug type in _OSC register, do not

> > > initialize the slot capability or set the 'Slot Implemented' flag.

> > > This way guest will choose ACPI hot-plug if it is preferred and leave

> > > the option to use SHPC with pcie-pci-bridge.

> > >

> > > Signed-off-by: Julia Suvorova <jusual@redhat.com>

> > > ---

> > >  hw/i386/acpi-build.h |  2 ++

> > >  hw/i386/acpi-build.c |  2 +-

> > >  hw/pci/pcie.c        | 16 ++++++++++++++++

> > >  3 files changed, 19 insertions(+), 1 deletion(-)

> > >

> > > diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h

> > > index 487ec7710f..4c5bfb3d0b 100644

> > > --- a/hw/i386/acpi-build.h

> > > +++ b/hw/i386/acpi-build.h

> > > @@ -11,4 +11,6 @@ extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;

> > >

> > >  void acpi_setup(void);

> > >

> > > +Object *object_resolve_type_unambiguous(const char *typename);

> > > +

> > >  #endif

> > > diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c

> > > index cf503b16af..b7811a8912 100644

> > > --- a/hw/i386/acpi-build.c

> > > +++ b/hw/i386/acpi-build.c

> > > @@ -174,7 +174,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,

> > >      *data = fadt;

> > >  }

> > >

> > > -static Object *object_resolve_type_unambiguous(const char *typename)

> > > +Object *object_resolve_type_unambiguous(const char *typename)

> > >  {

> > >      bool ambig;

> > >      Object *o = object_resolve_path_type("", typename, &ambig);

> > > diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c

> > > index 5b48bae0f6..c1a082e8b9 100644

> > > --- a/hw/pci/pcie.c

> > > +++ b/hw/pci/pcie.c

> > > @@ -27,6 +27,8 @@

> > >  #include "hw/pci/pci_bus.h"

> > >  #include "hw/pci/pcie_regs.h"

> > >  #include "hw/pci/pcie_port.h"

> > > +#include "hw/i386/ich9.h"

> > > +#include "hw/i386/acpi-build.h"

> > >  #include "qemu/range.h"

> > >

> > >  //#define DEBUG_PCIE

> >

> >

> > Not really happy with pcie.c getting an i386 dependency.

> >

> >

> >

> > > @@ -515,12 +517,26 @@ void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev,

> > >      pcie_cap_slot_push_attention_button(hotplug_pdev);

> > >  }

> > >

> > > +static bool acpi_pcihp_enabled(void)

> > > +{

> > > +    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);

> > > +

> > > +    return lpc &&

> > > +           object_property_get_bool(lpc, "acpi-pci-hotplug-with-bridge-support",

> > > +                                    NULL);

> > > +

> > > +}

> > > +

> >

> > Why not just check the property unconditionally?

> 

> Ok.

> 

> > >  /* pci express slot for pci express root/downstream port

> > >     PCI express capability slot registers */

> > >  void pcie_cap_slot_init(PCIDevice *dev, PCIESlot *s)

> > >  {

> > >      uint32_t pos = dev->exp.exp_cap;

> > >

> > > +    if (acpi_pcihp_enabled()) {

> > > +        return;

> > > +    }

> > > +

> >

> > I think I would rather not teach pcie about acpi. How about we

> > change the polarity, name the property

> > "pci-native-hotplug" or whatever makes sense.

> 

> I'd prefer not to change the property name since the common code in

> hw/i386/acpi-build.c depends on it, but I can add a new one if it

> makes any sense.


And maybe prefix with "x-" so we don't commit to it as an
external API.


> > >      pci_word_test_and_set_mask(dev->config + pos + PCI_EXP_FLAGS,

> > >                                 PCI_EXP_FLAGS_SLOT);

> > >

> > > --

> > > 2.25.4

> >
no-reply@patchew.org Sept. 24, 2020, 9:03 a.m. UTC | #5
Patchew URL: https://patchew.org/QEMU/20200924070013.165026-1-jusual@redhat.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1
=== TEST SCRIPT END ===

C linker for the host machine: cc ld.bfd 2.27-43
Host machine cpu family: x86_64
Host machine cpu: x86_64
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
Linking target tests/test-replication
libcommon.fa.p/hw_pci_pcie.c.o: In function `acpi_pcihp_enabled':
/tmp/qemu-test/build/../src/hw/pci/pcie.c:522: undefined reference to `object_resolve_type_unambiguous'
collect2: error: ld returned 1 exit status
make: *** [qemu-system-aarch64] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 709, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--rm', '--label', 'com.qemu.instance.uuid=38644690a9934226b7e7108ea4390530', '-u', '1003', '--security-opt', 'seccomp=unconfined', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-2jqqex0j/src/docker-src.2020-09-24-04.59.06.4662:/var/tmp/qemu:z,ro', 'qemu/centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=38644690a9934226b7e7108ea4390530
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-2jqqex0j/src'
make: *** [docker-run-test-quick@centos7] Error 2

real    4m43.237s
user    0m20.603s


The full log is available at
http://patchew.org/logs/20200924070013.165026-1-jusual@redhat.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Michael S. Tsirkin Sept. 24, 2020, 9:20 a.m. UTC | #6
On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:
> The patch set consists of two parts:
> patches 1-4: introduce new feature
>              'acpi-pci-hotplug-with-bridge-support' on Q35
> patches 5-7: make the feature default along with changes in ACPI tables
> 
> This way maintainers can decide which way to choose without breaking
> the patch set.
> 
> With the feature disabled Q35 falls back to the native hot-plug.
> 
> Pros
>     * no racy behavior during boot (see 110c477c2ed)
>     * eject is possible - according to PCIe spec, attention button
>       press should lead to power off, and then the adapter should be
>       removed manually. As there is no power down state exists in QEMU,
>       we cannot distinguish between an eject and a power down
>       request.
>     * no delay during deleting - after the actual power off software
>       must wait at least 1 second before indicating about it. This case
>       is quite important for users, it even has its own bug:
>           https://bugzilla.redhat.com/show_bug.cgi?id=1594168
>     * no timer-based behavior - in addition to the previous example,
>       the attention button has a 5-second waiting period, during which
>       the operation can be canceled with a second press. While this
>       looks fine for manual button control, automation will result in
>       the need to queue or drop events, and the software receiving
>       events in all sort of unspecified combinations of attention/power
>       indicator states, which is racy and uppredictable.
>     * fixes:
>         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
>         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256
> 
> Cons:
>     * lose per-port control over hot-plug (can be resolved)
>     * no access to possible features presented in slot capabilities
>       (this is only surprise removal AFAIK)

something I don't quite get is whether with this one can still add
new pcie bridges and then hotplug into these with native
hotplug.

the challenge to answering this with certainty is that right now qemu
does not allow hotplugging complex devices such as bridges at all,
we only have a hack for multifunction devices.
Maybe adding a bridge as function 1 on command line, then hotplugging another device as
function 0 will work to test this.



> v3:
>     * drop change of _OSC to allow SHPC on hotplugged bridges
>     * use 'acpi-root-pci-hotplug'
>     * add migration states [Igor]
>     * minor style changes
> 
> v2:
>     * new ioport range for acpiphp [Gerd]
>     * drop find_pci_host() [Igor]
>     * explain magic numbers in _OSC [Igor]
>     * drop build_q35_pci_hotplug() wrapper [Igor]
> 
> Julia Suvorova (7):
>   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
>   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
>   hw/pci/pcie: Do not initialize slot capability if acpihp is used
>   hw/acpi/ich9: Enable ACPI PCI hot-plug
>   bios-tables-test: Allow changes in DSDT ACPI tables
>   hw/acpi/ich9: Set ACPI PCI hot-plug as default
>   bios-tables-test: Update golden binaries
> 
>  hw/i386/acpi-build.h              |   7 ++++
>  include/hw/acpi/ich9.h            |   5 +++
>  include/hw/acpi/pcihp.h           |   3 +-
>  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
>  hw/acpi/pcihp.c                   |  16 ++++---
>  hw/acpi/piix4.c                   |   4 +-
>  hw/i386/acpi-build.c              |  31 ++++++++------
>  hw/i386/pc.c                      |   1 +
>  hw/pci/pcie.c                     |  16 +++++++
>  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
>  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
>  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
>  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
>  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
>  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
>  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
>  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
>  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
>  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
>  19 files changed, 129 insertions(+), 21 deletions(-)
> 
> -- 
> 2.25.4
Igor Mammedov Sept. 24, 2020, 9:30 a.m. UTC | #7
On Thu, 24 Sep 2020 09:00:06 +0200
Julia Suvorova <jusual@redhat.com> wrote:

> The patch set consists of two parts:
> patches 1-4: introduce new feature
>              'acpi-pci-hotplug-with-bridge-support' on Q35
> patches 5-7: make the feature default along with changes in ACPI tables
> 
> This way maintainers can decide which way to choose without breaking
> the patch set.
> 
> With the feature disabled Q35 falls back to the native hot-plug.
> 
> Pros
>     * no racy behavior during boot (see 110c477c2ed)
>     * eject is possible - according to PCIe spec, attention button
>       press should lead to power off, and then the adapter should be
>       removed manually. As there is no power down state exists in QEMU,
>       we cannot distinguish between an eject and a power down
>       request.

if I recall right, you had a idea about
 keeping pending removal request to distinguish between eject and poweroff
 (i.e. eject if mgmt asked for removal and otherwise it could be poweroff|nop)
 Why it didn't work out in the end?

>     * no delay during deleting - after the actual power off software
>       must wait at least 1 second before indicating about it. This case
>       is quite important for users, it even has its own bug:
>           https://bugzilla.redhat.com/show_bug.cgi?id=1594168
>     * no timer-based behavior - in addition to the previous example,
>       the attention button has a 5-second waiting period, during which
>       the operation can be canceled with a second press. While this
>       looks fine for manual button control, automation will result in
>       the need to queue or drop events, and the software receiving
>       events in all sort of unspecified combinations of attention/power
>       indicator states, which is racy and uppredictable.
>     * fixes:
>         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
>         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256
> 
> Cons:
>     * lose per-port control over hot-plug (can be resolved)
>     * no access to possible features presented in slot capabilities
>       (this is only surprise removal AFAIK)
> 
> v3:
>     * drop change of _OSC to allow SHPC on hotplugged bridges
>     * use 'acpi-root-pci-hotplug'
>     * add migration states [Igor]
>     * minor style changes
> 
> v2:
>     * new ioport range for acpiphp [Gerd]
>     * drop find_pci_host() [Igor]
>     * explain magic numbers in _OSC [Igor]
>     * drop build_q35_pci_hotplug() wrapper [Igor]
> 
> Julia Suvorova (7):
>   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
>   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
>   hw/pci/pcie: Do not initialize slot capability if acpihp is used
>   hw/acpi/ich9: Enable ACPI PCI hot-plug
>   bios-tables-test: Allow changes in DSDT ACPI tables
>   hw/acpi/ich9: Set ACPI PCI hot-plug as default
>   bios-tables-test: Update golden binaries
> 
>  hw/i386/acpi-build.h              |   7 ++++
>  include/hw/acpi/ich9.h            |   5 +++
>  include/hw/acpi/pcihp.h           |   3 +-
>  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
>  hw/acpi/pcihp.c                   |  16 ++++---
>  hw/acpi/piix4.c                   |   4 +-
>  hw/i386/acpi-build.c              |  31 ++++++++------
>  hw/i386/pc.c                      |   1 +
>  hw/pci/pcie.c                     |  16 +++++++
>  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
>  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
>  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
>  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
>  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
>  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
>  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
>  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
>  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
>  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
>  19 files changed, 129 insertions(+), 21 deletions(-)
>
Michael S. Tsirkin Sept. 24, 2020, 9:39 a.m. UTC | #8
On Thu, Sep 24, 2020 at 11:30:00AM +0200, Igor Mammedov wrote:
> On Thu, 24 Sep 2020 09:00:06 +0200
> Julia Suvorova <jusual@redhat.com> wrote:
> 
> > The patch set consists of two parts:
> > patches 1-4: introduce new feature
> >              'acpi-pci-hotplug-with-bridge-support' on Q35
> > patches 5-7: make the feature default along with changes in ACPI tables
> > 
> > This way maintainers can decide which way to choose without breaking
> > the patch set.
> > 
> > With the feature disabled Q35 falls back to the native hot-plug.
> > 
> > Pros
> >     * no racy behavior during boot (see 110c477c2ed)
> >     * eject is possible - according to PCIe spec, attention button
> >       press should lead to power off, and then the adapter should be
> >       removed manually. As there is no power down state exists in QEMU,
> >       we cannot distinguish between an eject and a power down
> >       request.
> 
> if I recall right, you had a idea about
>  keeping pending removal request to distinguish between eject and poweroff
>  (i.e. eject if mgmt asked for removal and otherwise it could be poweroff|nop)
>  Why it didn't work out in the end?

For better or worse guests expect to eject and have it happen,
without also asking management to do it ...

> >     * no delay during deleting - after the actual power off software
> >       must wait at least 1 second before indicating about it. This case
> >       is quite important for users, it even has its own bug:
> >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168
> >     * no timer-based behavior - in addition to the previous example,
> >       the attention button has a 5-second waiting period, during which
> >       the operation can be canceled with a second press. While this
> >       looks fine for manual button control, automation will result in
> >       the need to queue or drop events, and the software receiving
> >       events in all sort of unspecified combinations of attention/power
> >       indicator states, which is racy and uppredictable.
> >     * fixes:
> >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
> >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256
> > 
> > Cons:
> >     * lose per-port control over hot-plug (can be resolved)
> >     * no access to possible features presented in slot capabilities
> >       (this is only surprise removal AFAIK)
> > 
> > v3:
> >     * drop change of _OSC to allow SHPC on hotplugged bridges
> >     * use 'acpi-root-pci-hotplug'
> >     * add migration states [Igor]
> >     * minor style changes
> > 
> > v2:
> >     * new ioport range for acpiphp [Gerd]
> >     * drop find_pci_host() [Igor]
> >     * explain magic numbers in _OSC [Igor]
> >     * drop build_q35_pci_hotplug() wrapper [Igor]
> > 
> > Julia Suvorova (7):
> >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
> >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
> >   hw/pci/pcie: Do not initialize slot capability if acpihp is used
> >   hw/acpi/ich9: Enable ACPI PCI hot-plug
> >   bios-tables-test: Allow changes in DSDT ACPI tables
> >   hw/acpi/ich9: Set ACPI PCI hot-plug as default
> >   bios-tables-test: Update golden binaries
> > 
> >  hw/i386/acpi-build.h              |   7 ++++
> >  include/hw/acpi/ich9.h            |   5 +++
> >  include/hw/acpi/pcihp.h           |   3 +-
> >  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
> >  hw/acpi/pcihp.c                   |  16 ++++---
> >  hw/acpi/piix4.c                   |   4 +-
> >  hw/i386/acpi-build.c              |  31 ++++++++------
> >  hw/i386/pc.c                      |   1 +
> >  hw/pci/pcie.c                     |  16 +++++++
> >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
> >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
> >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
> >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
> >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
> >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
> >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
> >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
> >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
> >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
> >  19 files changed, 129 insertions(+), 21 deletions(-)
> >
Ani Sinha Sept. 24, 2020, 10:57 a.m. UTC | #9
On Thu, Sep 24, 2020 at 12:30 PM Julia Suvorova <jusual@redhat.com> wrote:
>
> All DSDT Q35 tables will be modified because ACPI hot-plug is enabled
> by default.
>
> Signed-off-by: Julia Suvorova <jusual@redhat.com>
Acked-by: Ani Sinha <ani@anisinha.ca>

> ---
>  tests/qtest/bios-tables-test-allowed-diff.h | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
> index dfb8523c8b..84f56b14db 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1 +1,11 @@
>  /* List of comma-separated changed AML files to ignore */
> +"tests/data/acpi/q35/DSDT",
> +"tests/data/acpi/q35/DSDT.tis",
> +"tests/data/acpi/q35/DSDT.bridge",
> +"tests/data/acpi/q35/DSDT.mmio64",
> +"tests/data/acpi/q35/DSDT.ipmibt",
> +"tests/data/acpi/q35/DSDT.cphp",
> +"tests/data/acpi/q35/DSDT.memhp",
> +"tests/data/acpi/q35/DSDT.acpihmat",
> +"tests/data/acpi/q35/DSDT.numamem",
> +"tests/data/acpi/q35/DSDT.dimmpxm",
> --
> 2.25.4
>
Igor Mammedov Sept. 24, 2020, 11:04 a.m. UTC | #10
On Thu, 24 Sep 2020 09:00:08 +0200
Julia Suvorova <jusual@redhat.com> wrote:

> Implement notifications and gpe to support q35 ACPI PCI hot-plug.
> Use 0xcc4 - 0xcd7 range for 'acpi-pci-hotplug' io ports.

Gerd,

does picked IO range looks acceptable to you?


> Signed-off-by: Julia Suvorova <jusual@redhat.com>
with minor nits below fixed:

 Reviewed-by: Igor Mammedov <imammedo@redhat.com>


> ---
>  hw/i386/acpi-build.h    |  4 ++++
>  include/hw/acpi/ich9.h  |  2 ++
>  include/hw/acpi/pcihp.h |  3 ++-
>  hw/acpi/pcihp.c         |  8 ++++----
>  hw/acpi/piix4.c         |  4 +++-
>  hw/i386/acpi-build.c    | 27 ++++++++++++++++-----------
>  6 files changed, 31 insertions(+), 17 deletions(-)
> 
> diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h
> index 74df5fc612..487ec7710f 100644
> --- a/hw/i386/acpi-build.h
> +++ b/hw/i386/acpi-build.h
> @@ -5,6 +5,10 @@
>  
>  extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;
>  
> +/* PCI Hot-plug registers bases. See docs/spec/acpi_pci_hotplug.txt */
> +#define ACPI_PCIHP_SEJ_BASE 0x8
> +#define ACPI_PCIHP_BNMR_BASE 0x10
> +
>  void acpi_setup(void);
>  
>  #endif
> diff --git a/include/hw/acpi/ich9.h b/include/hw/acpi/ich9.h
> index 28a53181cb..4d19571ed7 100644
> --- a/include/hw/acpi/ich9.h
> +++ b/include/hw/acpi/ich9.h
> @@ -28,6 +28,8 @@
>  #include "hw/acpi/acpi_dev_interface.h"
>  #include "hw/acpi/tco.h"
>  
> +#define ACPI_PCIHP_ADDR_ICH9 0x0cc4
> +
>  typedef struct ICH9LPCPMRegs {
>      /*
>       * In ich9 spec says that pm1_cnt register is 32bit width and
> diff --git a/include/hw/acpi/pcihp.h b/include/hw/acpi/pcihp.h
> index 02f4665767..ce49fb03b9 100644
> --- a/include/hw/acpi/pcihp.h
> +++ b/include/hw/acpi/pcihp.h
> @@ -54,7 +54,8 @@ typedef struct AcpiPciHpState {
>  } AcpiPciHpState;
>  
>  void acpi_pcihp_init(Object *owner, AcpiPciHpState *, PCIBus *root,
> -                     MemoryRegion *address_space_io, bool bridges_enabled);
> +                     MemoryRegion *address_space_io, bool bridges_enabled,
> +                     uint16_t io_base);
>  
>  void acpi_pcihp_device_pre_plug_cb(HotplugHandler *hotplug_dev,
>                                     DeviceState *dev, Error **errp);
> diff --git a/hw/acpi/pcihp.c b/hw/acpi/pcihp.c
> index ff23104aea..bb457bc279 100644
> --- a/hw/acpi/pcihp.c
> +++ b/hw/acpi/pcihp.c
> @@ -38,7 +38,6 @@
>  #include "qom/qom-qobject.h"
>  #include "trace.h"
>  
> -#define ACPI_PCIHP_ADDR 0xae00
>  #define ACPI_PCIHP_SIZE 0x0014
>  #define PCI_UP_BASE 0x0000
>  #define PCI_DOWN_BASE 0x0004
> @@ -381,12 +380,13 @@ static const MemoryRegionOps acpi_pcihp_io_ops = {
>  };
>  
>  void acpi_pcihp_init(Object *owner, AcpiPciHpState *s, PCIBus *root_bus,
> -                     MemoryRegion *address_space_io, bool bridges_enabled)
> +                     MemoryRegion *address_space_io, bool bridges_enabled,
> +                     uint16_t io_base)
>  {
>      s->io_len = ACPI_PCIHP_SIZE;
> -    s->io_base = ACPI_PCIHP_ADDR;
> +    s->io_base = io_base;
>  
> -    s->root= root_bus;
> +    s->root = root_bus;
>      s->legacy_piix = !bridges_enabled;
>  
>      memory_region_init_io(&s->io, owner, &acpi_pcihp_io_ops, s,
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 832f8fba82..a505ab5bcf 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -50,6 +50,8 @@
>  #define GPE_BASE 0xafe0
>  #define GPE_LEN 4
>  
> +#define ACPI_PCIHP_ADDR_PIIX4 0xae00
> +
>  struct pci_status {
>      uint32_t up; /* deprecated, maintained for migration compatibility */
>      uint32_t down;
> @@ -597,7 +599,7 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      memory_region_add_subregion(parent, GPE_BASE, &s->io_gpe);
>  
>      acpi_pcihp_init(OBJECT(s), &s->acpi_pci_hotplug, bus, parent,
> -                    s->use_acpi_hotplug_bridge);
> +                    s->use_acpi_hotplug_bridge, ACPI_PCIHP_ADDR_PIIX4);
>  
>      s->cpu_hotplug_legacy = true;
>      object_property_add_bool(OBJECT(s), "cpu-hotplug-legacy",
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 0e0535d2e3..cf503b16af 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -201,10 +201,6 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
>          /* w2k requires FADT(rev1) or it won't boot, keep PC compatible */
>          pm->fadt.rev = 1;
>          pm->cpu_hp_io_base = PIIX4_CPU_HOTPLUG_IO_BASE;
> -        pm->pcihp_io_base =
> -            object_property_get_uint(obj, ACPI_PCIHP_IO_BASE_PROP, NULL);
> -        pm->pcihp_io_len =
> -            object_property_get_uint(obj, ACPI_PCIHP_IO_LEN_PROP, NULL);
>      }
>      if (lpc) {
>          struct AcpiGenericAddress r = { .space_id = AML_AS_SYSTEM_IO,
> @@ -214,6 +210,10 @@ static void acpi_get_pm_info(MachineState *machine, AcpiPmInfo *pm)
>          pm->fadt.flags |= 1 << ACPI_FADT_F_RESET_REG_SUP;
>          pm->cpu_hp_io_base = ICH9_CPU_HOTPLUG_IO_BASE;
>      }
> +    pm->pcihp_io_base =
> +        object_property_get_uint(obj, ACPI_PCIHP_IO_BASE_PROP, NULL);
> +    pm->pcihp_io_len =
> +        object_property_get_uint(obj, ACPI_PCIHP_IO_LEN_PROP, NULL);
>  
>      /* The above need not be conditional on machine type because the reset port
>       * happens to be the same on PIIX (pc) and ICH9 (q35). */
> @@ -472,7 +472,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus,
>          QLIST_FOREACH(sec, &bus->child, sibling) {
>              int32_t devfn = sec->parent_dev->devfn;
>  
> -            if (pci_bus_is_root(sec) || pci_bus_is_express(sec)) {
> +            if (pci_bus_is_root(sec)) {
>                  continue;
>              }
>  
> @@ -1368,7 +1368,7 @@ static void build_piix4_isa_bridge(Aml *table)
>      aml_append(table, scope);
>  }
>  
> -static void build_piix4_pci_hotplug(Aml *table)
> +static void build_x86_pci_hotplug(Aml *table, uint64_t pcihp_addr)
maybe
  s/_pci_/_acpi_pci_/
otherwise it could be a bit confusing 

>  {
>      Aml *scope;
>      Aml *field;
> @@ -1377,20 +1377,22 @@ static void build_piix4_pci_hotplug(Aml *table)
>      scope =  aml_scope("_SB.PCI0");
>  
>      aml_append(scope,
> -        aml_operation_region("PCST", AML_SYSTEM_IO, aml_int(0xae00), 0x08));
> +        aml_operation_region("PCST", AML_SYSTEM_IO, aml_int(pcihp_addr), 0x08));
>      field = aml_field("PCST", AML_DWORD_ACC, AML_NOLOCK, AML_WRITE_AS_ZEROS);
>      aml_append(field, aml_named_field("PCIU", 32));
>      aml_append(field, aml_named_field("PCID", 32));
>      aml_append(scope, field);
>  
>      aml_append(scope,
> -        aml_operation_region("SEJ", AML_SYSTEM_IO, aml_int(0xae08), 0x04));
> +    aml_operation_region("SEJ", AML_SYSTEM_IO,
not aligned properly

> +                         aml_int(pcihp_addr + ACPI_PCIHP_SEJ_BASE), 0x04));
>      field = aml_field("SEJ", AML_DWORD_ACC, AML_NOLOCK, AML_WRITE_AS_ZEROS);
>      aml_append(field, aml_named_field("B0EJ", 32));
>      aml_append(scope, field);
>  
>      aml_append(scope,
> -        aml_operation_region("BNMR", AML_SYSTEM_IO, aml_int(0xae10), 0x04));
> +        aml_operation_region("BNMR", AML_SYSTEM_IO,
> +                             aml_int(pcihp_addr + ACPI_PCIHP_BNMR_BASE), 0x04));
>      field = aml_field("BNMR", AML_DWORD_ACC, AML_NOLOCK, AML_WRITE_AS_ZEROS);
>      aml_append(field, aml_named_field("BNUM", 32));
>      aml_append(scope, field);
> @@ -1504,7 +1506,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
>          build_hpet_aml(dsdt);
>          build_piix4_isa_bridge(dsdt);
>          build_isa_devices_aml(dsdt);
> -        build_piix4_pci_hotplug(dsdt);
> +        build_x86_pci_hotplug(dsdt, pm->pcihp_io_base);
>          build_piix4_pci0_int(dsdt);
>      } else {
>          sb_scope = aml_scope("_SB");
> @@ -1520,6 +1522,9 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
>          build_hpet_aml(dsdt);
>          build_q35_isa_bridge(dsdt);
>          build_isa_devices_aml(dsdt);
> +        if (pm->pcihp_bridge_en) {
> +            build_x86_pci_hotplug(dsdt, pm->pcihp_io_base);
> +        }
>          build_q35_pci0_int(dsdt);
>          if (pcms->smbus && !pcmc->do_not_add_smb_acpi) {
>              build_smb0(dsdt, pcms->smbus, ICH9_SMB_DEV, ICH9_SMB_FUNC);
> @@ -1546,7 +1551,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
>      {
>          aml_append(scope, aml_name_decl("_HID", aml_string("ACPI0006")));
>  
> -        if (misc->is_piix4) {
> +        if (misc->is_piix4 || pm->pcihp_bridge_en) {
>              method = aml_method("_E01", 0, AML_NOTSERIALIZED);
>              aml_append(method,
>                  aml_acquire(aml_name("\\_SB.PCI0.BLCK"), 0xFFFF));
Ani Sinha Sept. 24, 2020, 11:37 a.m. UTC | #11
On Thu, 24 Sep 2020, Igor Mammedov wrote:

> On Thu, 24 Sep 2020 09:00:09 +0200
> Julia Suvorova <jusual@redhat.com> wrote:
>
>> Instead of changing the hot-plug type in _OSC register, do not
>> initialize the slot capability or set the 'Slot Implemented' flag.
>> This way guest will choose ACPI hot-plug if it is preferred and leave
>> the option to use SHPC with pcie-pci-bridge.
>>
>> Signed-off-by: Julia Suvorova <jusual@redhat.com>
>> ---
>>  hw/i386/acpi-build.h |  2 ++
>>  hw/i386/acpi-build.c |  2 +-
>>  hw/pci/pcie.c        | 16 ++++++++++++++++
>>  3 files changed, 19 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h
>> index 487ec7710f..4c5bfb3d0b 100644
>> --- a/hw/i386/acpi-build.h
>> +++ b/hw/i386/acpi-build.h
>> @@ -11,4 +11,6 @@ extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;
>>
>>  void acpi_setup(void);
>>
>> +Object *object_resolve_type_unambiguous(const char *typename);
>> +
>>  #endif
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index cf503b16af..b7811a8912 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -174,7 +174,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,
>>      *data = fadt;
>>  }
>>
>> -static Object *object_resolve_type_unambiguous(const char *typename)
>> +Object *object_resolve_type_unambiguous(const char *typename)
>>  {
>>      bool ambig;
>>      Object *o = object_resolve_path_type("", typename, &ambig);
>> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
>> index 5b48bae0f6..c1a082e8b9 100644
>> --- a/hw/pci/pcie.c
>> +++ b/hw/pci/pcie.c
>> @@ -27,6 +27,8 @@
>>  #include "hw/pci/pci_bus.h"
>>  #include "hw/pci/pcie_regs.h"
>>  #include "hw/pci/pcie_port.h"
>> +#include "hw/i386/ich9.h"
>> +#include "hw/i386/acpi-build.h"
>>  #include "qemu/range.h"
>>
>>  //#define DEBUG_PCIE
>> @@ -515,12 +517,26 @@ void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev,
>>      pcie_cap_slot_push_attention_button(hotplug_pdev);
>>  }
>>
>> +static bool acpi_pcihp_enabled(void)
>> +{
>> +    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);
>> +
>> +    return lpc &&
>> +           object_property_get_bool(lpc, "acpi-pci-hotplug-with-bridge-support",
>> +                                    NULL);
>> +
>> +}
>> +
>>  /* pci express slot for pci express root/downstream port
>>     PCI express capability slot registers */
>>  void pcie_cap_slot_init(PCIDevice *dev, PCIESlot *s)
>>  {
>>      uint32_t pos = dev->exp.exp_cap;
>>
>> +    if (acpi_pcihp_enabled()) {
>> +        return;
>> +    }
>> +
>>      pci_word_test_and_set_mask(dev->config + pos + PCI_EXP_FLAGS,
>>                                 PCI_EXP_FLAGS_SLOT);
>>
> why do you drop all slot caps instead of disabling just "if (s->hotplug) {" branch?

+1 to Igor's suggestion.


>
>
Ani Sinha Sept. 24, 2020, 11:54 a.m. UTC | #12
On Thu, 24 Sep 2020, Julia Suvorova wrote:

> On Thu, Sep 24, 2020 at 9:36 AM Michael S. Tsirkin <mst@redhat.com> wrote:

>>

>> On Thu, Sep 24, 2020 at 09:00:09AM +0200, Julia Suvorova wrote:

>>> Instead of changing the hot-plug type in _OSC register, do not

>>> initialize the slot capability or set the 'Slot Implemented' flag.

>>> This way guest will choose ACPI hot-plug if it is preferred and leave

>>> the option to use SHPC with pcie-pci-bridge.

>>>

>>> Signed-off-by: Julia Suvorova <jusual@redhat.com>

>>> ---

>>>  hw/i386/acpi-build.h |  2 ++

>>>  hw/i386/acpi-build.c |  2 +-

>>>  hw/pci/pcie.c        | 16 ++++++++++++++++

>>>  3 files changed, 19 insertions(+), 1 deletion(-)

>>>

>>> diff --git a/hw/i386/acpi-build.h b/hw/i386/acpi-build.h

>>> index 487ec7710f..4c5bfb3d0b 100644

>>> --- a/hw/i386/acpi-build.h

>>> +++ b/hw/i386/acpi-build.h

>>> @@ -11,4 +11,6 @@ extern const struct AcpiGenericAddress x86_nvdimm_acpi_dsmio;

>>>

>>>  void acpi_setup(void);

>>>

>>> +Object *object_resolve_type_unambiguous(const char *typename);

>>> +

>>>  #endif

>>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c

>>> index cf503b16af..b7811a8912 100644

>>> --- a/hw/i386/acpi-build.c

>>> +++ b/hw/i386/acpi-build.c

>>> @@ -174,7 +174,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,

>>>      *data = fadt;

>>>  }

>>>

>>> -static Object *object_resolve_type_unambiguous(const char *typename)

>>> +Object *object_resolve_type_unambiguous(const char *typename)

>>>  {

>>>      bool ambig;

>>>      Object *o = object_resolve_path_type("", typename, &ambig);

>>> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c

>>> index 5b48bae0f6..c1a082e8b9 100644

>>> --- a/hw/pci/pcie.c

>>> +++ b/hw/pci/pcie.c

>>> @@ -27,6 +27,8 @@

>>>  #include "hw/pci/pci_bus.h"

>>>  #include "hw/pci/pcie_regs.h"

>>>  #include "hw/pci/pcie_port.h"

>>> +#include "hw/i386/ich9.h"

>>> +#include "hw/i386/acpi-build.h"

>>>  #include "qemu/range.h"

>>>

>>>  //#define DEBUG_PCIE

>>

>>

>> Not really happy with pcie.c getting an i386 dependency.

>>

>>

>>

>>> @@ -515,12 +517,26 @@ void pcie_cap_slot_unplug_request_cb(HotplugHandler *hotplug_dev,

>>>      pcie_cap_slot_push_attention_button(hotplug_pdev);

>>>  }

>>>

>>> +static bool acpi_pcihp_enabled(void)

>>> +{

>>> +    Object *lpc = object_resolve_type_unambiguous(TYPE_ICH9_LPC_DEVICE);

>>> +

>>> +    return lpc &&

>>> +           object_property_get_bool(lpc, "acpi-pci-hotplug-with-bridge-support",

>>> +                                    NULL);

>>> +

>>> +}

>>> +

>>

>> Why not just check the property unconditionally?

>

> Ok.


I do not see how that would work if lpc is NULL.
object_resolve_type_unambiguous() can return NULL, at least in theory.

>

>>>  /* pci express slot for pci express root/downstream port

>>>     PCI express capability slot registers */

>>>  void pcie_cap_slot_init(PCIDevice *dev, PCIESlot *s)

>>>  {

>>>      uint32_t pos = dev->exp.exp_cap;

>>>

>>> +    if (acpi_pcihp_enabled()) {

>>> +        return;

>>> +    }

>>> +

>>

>> I think I would rather not teach pcie about acpi. How about we

>> change the polarity, name the property

>> "pci-native-hotplug" or whatever makes sense.

>

> I'd prefer not to change the property name since the common code in

> hw/i386/acpi-build.c depends on it, but I can add a new one if it

> makes any sense.

>

>>>      pci_word_test_and_set_mask(dev->config + pos + PCI_EXP_FLAGS,

>>>                                 PCI_EXP_FLAGS_SLOT);

>>>

>>> --

>>> 2.25.4

>>

>

>
Gerd Hoffmann Sept. 25, 2020, 8:20 a.m. UTC | #13
On Thu, Sep 24, 2020 at 01:04:19PM +0200, Igor Mammedov wrote:
> On Thu, 24 Sep 2020 09:00:08 +0200

> Julia Suvorova <jusual@redhat.com> wrote:

> 

> > Implement notifications and gpe to support q35 ACPI PCI hot-plug.

> > Use 0xcc4 - 0xcd7 range for 'acpi-pci-hotplug' io ports.

> 

> Gerd,

> 

> does picked IO range looks acceptable to you?


Yes, that is fine.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Julia Suvorova Oct. 1, 2020, 8:55 a.m. UTC | #14
On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>

> On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:

> > The patch set consists of two parts:

> > patches 1-4: introduce new feature

> >              'acpi-pci-hotplug-with-bridge-support' on Q35

> > patches 5-7: make the feature default along with changes in ACPI tables

> >

> > This way maintainers can decide which way to choose without breaking

> > the patch set.

> >

> > With the feature disabled Q35 falls back to the native hot-plug.

> >

> > Pros

> >     * no racy behavior during boot (see 110c477c2ed)

> >     * eject is possible - according to PCIe spec, attention button

> >       press should lead to power off, and then the adapter should be

> >       removed manually. As there is no power down state exists in QEMU,

> >       we cannot distinguish between an eject and a power down

> >       request.

> >     * no delay during deleting - after the actual power off software

> >       must wait at least 1 second before indicating about it. This case

> >       is quite important for users, it even has its own bug:

> >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168

> >     * no timer-based behavior - in addition to the previous example,

> >       the attention button has a 5-second waiting period, during which

> >       the operation can be canceled with a second press. While this

> >       looks fine for manual button control, automation will result in

> >       the need to queue or drop events, and the software receiving

> >       events in all sort of unspecified combinations of attention/power

> >       indicator states, which is racy and uppredictable.

> >     * fixes:

> >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465

> >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256

> >

> > Cons:

> >     * lose per-port control over hot-plug (can be resolved)

> >     * no access to possible features presented in slot capabilities

> >       (this is only surprise removal AFAIK)

>

> something I don't quite get is whether with this one can still add

> new pcie bridges and then hotplug into these with native

> hotplug.


Right now I disable native if there is acpihp anywhere, but even if
you enable it for hotplugged devices, native hot-plug will not work.

> the challenge to answering this with certainty is that right now qemu

> does not allow hotplugging complex devices such as bridges at all,

> we only have a hack for multifunction devices.

> Maybe adding a bridge as function 1 on command line, then hotplugging another device as

> function 0 will work to test this.

>

>

>

> > v3:

> >     * drop change of _OSC to allow SHPC on hotplugged bridges

> >     * use 'acpi-root-pci-hotplug'

> >     * add migration states [Igor]

> >     * minor style changes

> >

> > v2:

> >     * new ioport range for acpiphp [Gerd]

> >     * drop find_pci_host() [Igor]

> >     * explain magic numbers in _OSC [Igor]

> >     * drop build_q35_pci_hotplug() wrapper [Igor]

> >

> > Julia Suvorova (7):

> >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35

> >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35

> >   hw/pci/pcie: Do not initialize slot capability if acpihp is used

> >   hw/acpi/ich9: Enable ACPI PCI hot-plug

> >   bios-tables-test: Allow changes in DSDT ACPI tables

> >   hw/acpi/ich9: Set ACPI PCI hot-plug as default

> >   bios-tables-test: Update golden binaries

> >

> >  hw/i386/acpi-build.h              |   7 ++++

> >  include/hw/acpi/ich9.h            |   5 +++

> >  include/hw/acpi/pcihp.h           |   3 +-

> >  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++

> >  hw/acpi/pcihp.c                   |  16 ++++---

> >  hw/acpi/piix4.c                   |   4 +-

> >  hw/i386/acpi-build.c              |  31 ++++++++------

> >  hw/i386/pc.c                      |   1 +

> >  hw/pci/pcie.c                     |  16 +++++++

> >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes

> >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes

> >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes

> >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes

> >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes

> >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes

> >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes

> >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes

> >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes

> >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes

> >  19 files changed, 129 insertions(+), 21 deletions(-)

> >

> > --

> > 2.25.4

>
Michael S. Tsirkin Oct. 1, 2020, 11:40 a.m. UTC | #15
On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:
> On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com> wrote:

> >

> > On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:

> > > The patch set consists of two parts:

> > > patches 1-4: introduce new feature

> > >              'acpi-pci-hotplug-with-bridge-support' on Q35

> > > patches 5-7: make the feature default along with changes in ACPI tables

> > >

> > > This way maintainers can decide which way to choose without breaking

> > > the patch set.

> > >

> > > With the feature disabled Q35 falls back to the native hot-plug.

> > >

> > > Pros

> > >     * no racy behavior during boot (see 110c477c2ed)

> > >     * eject is possible - according to PCIe spec, attention button

> > >       press should lead to power off, and then the adapter should be

> > >       removed manually. As there is no power down state exists in QEMU,

> > >       we cannot distinguish between an eject and a power down

> > >       request.

> > >     * no delay during deleting - after the actual power off software

> > >       must wait at least 1 second before indicating about it. This case

> > >       is quite important for users, it even has its own bug:

> > >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168

> > >     * no timer-based behavior - in addition to the previous example,

> > >       the attention button has a 5-second waiting period, during which

> > >       the operation can be canceled with a second press. While this

> > >       looks fine for manual button control, automation will result in

> > >       the need to queue or drop events, and the software receiving

> > >       events in all sort of unspecified combinations of attention/power

> > >       indicator states, which is racy and uppredictable.

> > >     * fixes:

> > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465

> > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256

> > >

> > > Cons:

> > >     * lose per-port control over hot-plug (can be resolved)

> > >     * no access to possible features presented in slot capabilities

> > >       (this is only surprise removal AFAIK)

> >

> > something I don't quite get is whether with this one can still add

> > new pcie bridges and then hotplug into these with native

> > hotplug.

> 

> Right now I disable native if there is acpihp anywhere, but even if

> you enable it for hotplugged devices, native hot-plug will not work.


So that's a minor regression in functionality, right?
Why is that the case? Because you disable it in ACPI?
What if we don't?

> > the challenge to answering this with certainty is that right now qemu

> > does not allow hotplugging complex devices such as bridges at all,

> > we only have a hack for multifunction devices.

> > Maybe adding a bridge as function 1 on command line, then hotplugging another device as

> > function 0 will work to test this.

> >

> >

> >

> > > v3:

> > >     * drop change of _OSC to allow SHPC on hotplugged bridges

> > >     * use 'acpi-root-pci-hotplug'

> > >     * add migration states [Igor]

> > >     * minor style changes

> > >

> > > v2:

> > >     * new ioport range for acpiphp [Gerd]

> > >     * drop find_pci_host() [Igor]

> > >     * explain magic numbers in _OSC [Igor]

> > >     * drop build_q35_pci_hotplug() wrapper [Igor]

> > >

> > > Julia Suvorova (7):

> > >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35

> > >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35

> > >   hw/pci/pcie: Do not initialize slot capability if acpihp is used

> > >   hw/acpi/ich9: Enable ACPI PCI hot-plug

> > >   bios-tables-test: Allow changes in DSDT ACPI tables

> > >   hw/acpi/ich9: Set ACPI PCI hot-plug as default

> > >   bios-tables-test: Update golden binaries

> > >

> > >  hw/i386/acpi-build.h              |   7 ++++

> > >  include/hw/acpi/ich9.h            |   5 +++

> > >  include/hw/acpi/pcihp.h           |   3 +-

> > >  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++

> > >  hw/acpi/pcihp.c                   |  16 ++++---

> > >  hw/acpi/piix4.c                   |   4 +-

> > >  hw/i386/acpi-build.c              |  31 ++++++++------

> > >  hw/i386/pc.c                      |   1 +

> > >  hw/pci/pcie.c                     |  16 +++++++

> > >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes

> > >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes

> > >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes

> > >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes

> > >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes

> > >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes

> > >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes

> > >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes

> > >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes

> > >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes

> > >  19 files changed, 129 insertions(+), 21 deletions(-)

> > >

> > > --

> > > 2.25.4

> >
Ani Sinha Oct. 1, 2020, 1:01 p.m. UTC | #16
On Thu, Oct 1, 2020 at 17:10 Michael S. Tsirkin <mst@redhat.com> wrote:

> On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:

> > On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com>

> wrote:

> > >

> > > On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:

> > > > The patch set consists of two parts:

> > > > patches 1-4: introduce new feature

> > > >              'acpi-pci-hotplug-with-bridge-support' on Q35

> > > > patches 5-7: make the feature default along with changes in ACPI

> tables

> > > >

> > > > This way maintainers can decide which way to choose without breaking

> > > > the patch set.

> > > >

> > > > With the feature disabled Q35 falls back to the native hot-plug.

> > > >

> > > > Pros

> > > >     * no racy behavior during boot (see 110c477c2ed)

> > > >     * eject is possible - according to PCIe spec, attention button

> > > >       press should lead to power off, and then the adapter should be

> > > >       removed manually. As there is no power down state exists in

> QEMU,

> > > >       we cannot distinguish between an eject and a power down

> > > >       request.

> > > >     * no delay during deleting - after the actual power off software

> > > >       must wait at least 1 second before indicating about it. This

> case

> > > >       is quite important for users, it even has its own bug:

> > > >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168

> > > >     * no timer-based behavior - in addition to the previous example,

> > > >       the attention button has a 5-second waiting period, during

> which

> > > >       the operation can be canceled with a second press. While this

> > > >       looks fine for manual button control, automation will result in

> > > >       the need to queue or drop events, and the software receiving

> > > >       events in all sort of unspecified combinations of

> attention/power

> > > >       indicator states, which is racy and uppredictable.

> > > >     * fixes:

> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465

> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256

> > > >

> > > > Cons:

> > > >     * lose per-port control over hot-plug (can be resolved)

> > > >     * no access to possible features presented in slot capabilities

> > > >       (this is only surprise removal AFAIK)

> > >

> > > something I don't quite get is whether with this one can still add

> > > new pcie bridges and then hotplug into these with native

> > > hotplug.

> >

> > Right now I disable native if there is acpihp anywhere, but even if

> > you enable it for hotplugged devices, native hot-plug will not work.

>

> So that's a minor regression in functionality, right?

> Why is that the case? Because you disable it in ACPI?

> What if we don't?



Stupid question. If both native and acpi is enabled which one does OS
chose? Does this choice vary between OSes and different flavours of the
same OS like Windows?


>

> > > the challenge to answering this with certainty is that right now qemu

> > > does not allow hotplugging complex devices such as bridges at all,

> > > we only have a hack for multifunction devices.

> > > Maybe adding a bridge as function 1 on command line, then hotplugging

> another device as

> > > function 0 will work to test this.

> > >

> > >

> > >

> > > > v3:

> > > >     * drop change of _OSC to allow SHPC on hotplugged bridges

> > > >     * use 'acpi-root-pci-hotplug'

> > > >     * add migration states [Igor]

> > > >     * minor style changes

> > > >

> > > > v2:

> > > >     * new ioport range for acpiphp [Gerd]

> > > >     * drop find_pci_host() [Igor]

> > > >     * explain magic numbers in _OSC [Igor]

> > > >     * drop build_q35_pci_hotplug() wrapper [Igor]

> > > >

> > > > Julia Suvorova (7):

> > > >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35

> > > >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35

> > > >   hw/pci/pcie: Do not initialize slot capability if acpihp is used

> > > >   hw/acpi/ich9: Enable ACPI PCI hot-plug

> > > >   bios-tables-test: Allow changes in DSDT ACPI tables

> > > >   hw/acpi/ich9: Set ACPI PCI hot-plug as default

> > > >   bios-tables-test: Update golden binaries

> > > >

> > > >  hw/i386/acpi-build.h              |   7 ++++

> > > >  include/hw/acpi/ich9.h            |   5 +++

> > > >  include/hw/acpi/pcihp.h           |   3 +-

> > > >  hw/acpi/ich9.c                    |  67

> ++++++++++++++++++++++++++++++

> > > >  hw/acpi/pcihp.c                   |  16 ++++---

> > > >  hw/acpi/piix4.c                   |   4 +-

> > > >  hw/i386/acpi-build.c              |  31 ++++++++------

> > > >  hw/i386/pc.c                      |   1 +

> > > >  hw/pci/pcie.c                     |  16 +++++++

> > > >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes

> > > >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes

> > > >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes

> > > >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes

> > > >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes

> > > >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes

> > > >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes

> > > >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes

> > > >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes

> > > >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes

> > > >  19 files changed, 129 insertions(+), 21 deletions(-)

> > > >

> > > > --

> > > > 2.25.4

> > >

>

>
<div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 1, 2020 at 17:10 Michael S. Tsirkin &lt;<a href="mailto:mst@redhat.com">mst@redhat.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:<br>
&gt; On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin &lt;<a href="mailto:mst@redhat.com" target="_blank">mst@redhat.com</a>&gt; wrote:<br>
&gt; &gt;<br>
&gt; &gt; On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:<br>
&gt; &gt; &gt; The patch set consists of two parts:<br>
&gt; &gt; &gt; patches 1-4: introduce new feature<br>
&gt; &gt; &gt;              &#39;acpi-pci-hotplug-with-bridge-support&#39; on Q35<br>
&gt; &gt; &gt; patches 5-7: make the feature default along with changes in ACPI tables<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; This way maintainers can decide which way to choose without breaking<br>
&gt; &gt; &gt; the patch set.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; With the feature disabled Q35 falls back to the native hot-plug.<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Pros<br>
&gt; &gt; &gt;     * no racy behavior during boot (see 110c477c2ed)<br>
&gt; &gt; &gt;     * eject is possible - according to PCIe spec, attention button<br>
&gt; &gt; &gt;       press should lead to power off, and then the adapter should be<br>
&gt; &gt; &gt;       removed manually. As there is no power down state exists in QEMU,<br>
&gt; &gt; &gt;       we cannot distinguish between an eject and a power down<br>
&gt; &gt; &gt;       request.<br>
&gt; &gt; &gt;     * no delay during deleting - after the actual power off software<br>
&gt; &gt; &gt;       must wait at least 1 second before indicating about it. This case<br>
&gt; &gt; &gt;       is quite important for users, it even has its own bug:<br>
&gt; &gt; &gt;           <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1594168" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1594168</a><br>
&gt; &gt; &gt;     * no timer-based behavior - in addition to the previous example,<br>
&gt; &gt; &gt;       the attention button has a 5-second waiting period, during which<br>
&gt; &gt; &gt;       the operation can be canceled with a second press. While this<br>
&gt; &gt; &gt;       looks fine for manual button control, automation will result in<br>
&gt; &gt; &gt;       the need to queue or drop events, and the software receiving<br>
&gt; &gt; &gt;       events in all sort of unspecified combinations of attention/power<br>
&gt; &gt; &gt;       indicator states, which is racy and uppredictable.<br>
&gt; &gt; &gt;     * fixes:<br>
&gt; &gt; &gt;         * <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1752465" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1752465</a><br>
&gt; &gt; &gt;         * <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1690256" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1690256</a><br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Cons:<br>
&gt; &gt; &gt;     * lose per-port control over hot-plug (can be resolved)<br>
&gt; &gt; &gt;     * no access to possible features presented in slot capabilities<br>
&gt; &gt; &gt;       (this is only surprise removal AFAIK)<br>
&gt; &gt;<br>
&gt; &gt; something I don&#39;t quite get is whether with this one can still add<br>
&gt; &gt; new pcie bridges and then hotplug into these with native<br>
&gt; &gt; hotplug.<br>
&gt; <br>
&gt; Right now I disable native if there is acpihp anywhere, but even if<br>
&gt; you enable it for hotplugged devices, native hot-plug will not work.<br>
<br>
So that&#39;s a minor regression in functionality, right?<br>
Why is that the case? Because you disable it in ACPI?<br>
What if we don&#39;t?</blockquote><div dir="auto"><br></div><div dir="auto">Stupid question. If both native and acpi is enabled which one does OS chose? Does this choice vary between OSes and different flavours of the same OS like Windows?</div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)" dir="auto"><br>
<br>
&gt; &gt; the challenge to answering this with certainty is that right now qemu<br>
&gt; &gt; does not allow hotplugging complex devices such as bridges at all,<br>
&gt; &gt; we only have a hack for multifunction devices.<br>
&gt; &gt; Maybe adding a bridge as function 1 on command line, then hotplugging another device as<br>
&gt; &gt; function 0 will work to test this.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; &gt; v3:<br>
&gt; &gt; &gt;     * drop change of _OSC to allow SHPC on hotplugged bridges<br>
&gt; &gt; &gt;     * use &#39;acpi-root-pci-hotplug&#39;<br>
&gt; &gt; &gt;     * add migration states [Igor]<br>
&gt; &gt; &gt;     * minor style changes<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; v2:<br>
&gt; &gt; &gt;     * new ioport range for acpiphp [Gerd]<br>
&gt; &gt; &gt;     * drop find_pci_host() [Igor]<br>
&gt; &gt; &gt;     * explain magic numbers in _OSC [Igor]<br>
&gt; &gt; &gt;     * drop build_q35_pci_hotplug() wrapper [Igor]<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; Julia Suvorova (7):<br>
&gt; &gt; &gt;   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35<br>
&gt; &gt; &gt;   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35<br>
&gt; &gt; &gt;   hw/pci/pcie: Do not initialize slot capability if acpihp is used<br>
&gt; &gt; &gt;   hw/acpi/ich9: Enable ACPI PCI hot-plug<br>
&gt; &gt; &gt;   bios-tables-test: Allow changes in DSDT ACPI tables<br>
&gt; &gt; &gt;   hw/acpi/ich9: Set ACPI PCI hot-plug as default<br>
&gt; &gt; &gt;   bios-tables-test: Update golden binaries<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt;  hw/i386/acpi-build.h              |   7 ++++<br>
&gt; &gt; &gt;  include/hw/acpi/ich9.h            |   5 +++<br>
&gt; &gt; &gt;  include/hw/acpi/pcihp.h           |   3 +-<br>
&gt; &gt; &gt;  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++<br>
&gt; &gt; &gt;  hw/acpi/pcihp.c                   |  16 ++++---<br>
&gt; &gt; &gt;  hw/acpi/piix4.c                   |   4 +-<br>
&gt; &gt; &gt;  hw/i386/acpi-build.c              |  31 ++++++++------<br>
&gt; &gt; &gt;  hw/i386/pc.c                      |   1 +<br>
&gt; &gt; &gt;  hw/pci/pcie.c                     |  16 +++++++<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT          | Bin 7678 -&gt; 7950 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -&gt; 9274 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -&gt; 9865 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -&gt; 8413 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -&gt; 9603 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -&gt; 8025 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -&gt; 9309 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -&gt; 9080 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -&gt; 7956 bytes<br>
&gt; &gt; &gt;  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -&gt; 8555 bytes<br>
&gt; &gt; &gt;  19 files changed, 129 insertions(+), 21 deletions(-)<br>
&gt; &gt; &gt;<br>
&gt; &gt; &gt; --<br>
&gt; &gt; &gt; 2.25.4<br>
&gt; &gt;<br>
<br>
</blockquote></div></div>
Julia Suvorova Oct. 1, 2020, 3:25 p.m. UTC | #17
On Thu, Oct 1, 2020 at 3:02 PM Ani Sinha <ani@anisinha.ca> wrote:
>
>
>
> On Thu, Oct 1, 2020 at 17:10 Michael S. Tsirkin <mst@redhat.com> wrote:
>>
>> On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:
>> > On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>> > >
>> > > On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:
>> > > > The patch set consists of two parts:
>> > > > patches 1-4: introduce new feature
>> > > >              'acpi-pci-hotplug-with-bridge-support' on Q35
>> > > > patches 5-7: make the feature default along with changes in ACPI tables
>> > > >
>> > > > This way maintainers can decide which way to choose without breaking
>> > > > the patch set.
>> > > >
>> > > > With the feature disabled Q35 falls back to the native hot-plug.
>> > > >
>> > > > Pros
>> > > >     * no racy behavior during boot (see 110c477c2ed)
>> > > >     * eject is possible - according to PCIe spec, attention button
>> > > >       press should lead to power off, and then the adapter should be
>> > > >       removed manually. As there is no power down state exists in QEMU,
>> > > >       we cannot distinguish between an eject and a power down
>> > > >       request.
>> > > >     * no delay during deleting - after the actual power off software
>> > > >       must wait at least 1 second before indicating about it. This case
>> > > >       is quite important for users, it even has its own bug:
>> > > >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168
>> > > >     * no timer-based behavior - in addition to the previous example,
>> > > >       the attention button has a 5-second waiting period, during which
>> > > >       the operation can be canceled with a second press. While this
>> > > >       looks fine for manual button control, automation will result in
>> > > >       the need to queue or drop events, and the software receiving
>> > > >       events in all sort of unspecified combinations of attention/power
>> > > >       indicator states, which is racy and uppredictable.
>> > > >     * fixes:
>> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
>> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256
>> > > >
>> > > > Cons:
>> > > >     * lose per-port control over hot-plug (can be resolved)
>> > > >     * no access to possible features presented in slot capabilities
>> > > >       (this is only surprise removal AFAIK)
>> > >
>> > > something I don't quite get is whether with this one can still add
>> > > new pcie bridges and then hotplug into these with native
>> > > hotplug.
>> >
>> > Right now I disable native if there is acpihp anywhere, but even if
>> > you enable it for hotplugged devices, native hot-plug will not work.
>>
>> So that's a minor regression in functionality, right?
>> Why is that the case? Because you disable it in ACPI?
>> What if we don't?
>
>
> Stupid question. If both native and acpi is enabled which one does OS chose? Does this choice vary between OSes and different flavours of the same OS like Windows?

It will always choose native.

>>
>>
>> > > the challenge to answering this with certainty is that right now qemu
>> > > does not allow hotplugging complex devices such as bridges at all,
>> > > we only have a hack for multifunction devices.
>> > > Maybe adding a bridge as function 1 on command line, then hotplugging another device as
>> > > function 0 will work to test this.
>> > >
>> > >
>> > >
>> > > > v3:
>> > > >     * drop change of _OSC to allow SHPC on hotplugged bridges
>> > > >     * use 'acpi-root-pci-hotplug'
>> > > >     * add migration states [Igor]
>> > > >     * minor style changes
>> > > >
>> > > > v2:
>> > > >     * new ioport range for acpiphp [Gerd]
>> > > >     * drop find_pci_host() [Igor]
>> > > >     * explain magic numbers in _OSC [Igor]
>> > > >     * drop build_q35_pci_hotplug() wrapper [Igor]
>> > > >
>> > > > Julia Suvorova (7):
>> > > >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
>> > > >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
>> > > >   hw/pci/pcie: Do not initialize slot capability if acpihp is used
>> > > >   hw/acpi/ich9: Enable ACPI PCI hot-plug
>> > > >   bios-tables-test: Allow changes in DSDT ACPI tables
>> > > >   hw/acpi/ich9: Set ACPI PCI hot-plug as default
>> > > >   bios-tables-test: Update golden binaries
>> > > >
>> > > >  hw/i386/acpi-build.h              |   7 ++++
>> > > >  include/hw/acpi/ich9.h            |   5 +++
>> > > >  include/hw/acpi/pcihp.h           |   3 +-
>> > > >  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
>> > > >  hw/acpi/pcihp.c                   |  16 ++++---
>> > > >  hw/acpi/piix4.c                   |   4 +-
>> > > >  hw/i386/acpi-build.c              |  31 ++++++++------
>> > > >  hw/i386/pc.c                      |   1 +
>> > > >  hw/pci/pcie.c                     |  16 +++++++
>> > > >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
>> > > >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
>> > > >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
>> > > >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
>> > > >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
>> > > >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
>> > > >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
>> > > >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
>> > > >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
>> > > >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
>> > > >  19 files changed, 129 insertions(+), 21 deletions(-)
>> > > >
>> > > > --
>> > > > 2.25.4
>> > >
>>
Julia Suvorova Oct. 1, 2020, 3:54 p.m. UTC | #18
On Thu, Oct 1, 2020 at 1:40 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:
> > On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:
> > > > The patch set consists of two parts:
> > > > patches 1-4: introduce new feature
> > > >              'acpi-pci-hotplug-with-bridge-support' on Q35
> > > > patches 5-7: make the feature default along with changes in ACPI tables
> > > >
> > > > This way maintainers can decide which way to choose without breaking
> > > > the patch set.
> > > >
> > > > With the feature disabled Q35 falls back to the native hot-plug.
> > > >
> > > > Pros
> > > >     * no racy behavior during boot (see 110c477c2ed)
> > > >     * eject is possible - according to PCIe spec, attention button
> > > >       press should lead to power off, and then the adapter should be
> > > >       removed manually. As there is no power down state exists in QEMU,
> > > >       we cannot distinguish between an eject and a power down
> > > >       request.
> > > >     * no delay during deleting - after the actual power off software
> > > >       must wait at least 1 second before indicating about it. This case
> > > >       is quite important for users, it even has its own bug:
> > > >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168
> > > >     * no timer-based behavior - in addition to the previous example,
> > > >       the attention button has a 5-second waiting period, during which
> > > >       the operation can be canceled with a second press. While this
> > > >       looks fine for manual button control, automation will result in
> > > >       the need to queue or drop events, and the software receiving
> > > >       events in all sort of unspecified combinations of attention/power
> > > >       indicator states, which is racy and uppredictable.
> > > >     * fixes:
> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465
> > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256
> > > >
> > > > Cons:
> > > >     * lose per-port control over hot-plug (can be resolved)
> > > >     * no access to possible features presented in slot capabilities
> > > >       (this is only surprise removal AFAIK)
> > >
> > > something I don't quite get is whether with this one can still add
> > > new pcie bridges and then hotplug into these with native
> > > hotplug.
> >
> > Right now I disable native if there is acpihp anywhere, but even if
> > you enable it for hotplugged devices, native hot-plug will not work.
>
> So that's a minor regression in functionality, right?
> Why is that the case? Because you disable it in ACPI?
> What if we don't?

I meant that I disable slot hotplug capabilities, nothing in ACPI
prevents native from working. Actually, I don't see if there's any
regression at all. Configurations like hot-plugging downstream port or
switch to another downstream port haven't worked before, and they
don't work now. I can enable native for hotplugged bridges, but that
doesn't make sense, because you won't be able to hot-plug anything to
it. It's not an issue of ACPI, it's PCIe behaviour. Also, native-acpi
combination may seem bizarre to os (slot enumeration is independent,
that's why I suggested disabling pcie slot flags).

> > > the challenge to answering this with certainty is that right now qemu
> > > does not allow hotplugging complex devices such as bridges at all,
> > > we only have a hack for multifunction devices.
> > > Maybe adding a bridge as function 1 on command line, then hotplugging another device as
> > > function 0 will work to test this.
> > >
> > >
> > >
> > > > v3:
> > > >     * drop change of _OSC to allow SHPC on hotplugged bridges
> > > >     * use 'acpi-root-pci-hotplug'
> > > >     * add migration states [Igor]
> > > >     * minor style changes
> > > >
> > > > v2:
> > > >     * new ioport range for acpiphp [Gerd]
> > > >     * drop find_pci_host() [Igor]
> > > >     * explain magic numbers in _OSC [Igor]
> > > >     * drop build_q35_pci_hotplug() wrapper [Igor]
> > > >
> > > > Julia Suvorova (7):
> > > >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35
> > > >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35
> > > >   hw/pci/pcie: Do not initialize slot capability if acpihp is used
> > > >   hw/acpi/ich9: Enable ACPI PCI hot-plug
> > > >   bios-tables-test: Allow changes in DSDT ACPI tables
> > > >   hw/acpi/ich9: Set ACPI PCI hot-plug as default
> > > >   bios-tables-test: Update golden binaries
> > > >
> > > >  hw/i386/acpi-build.h              |   7 ++++
> > > >  include/hw/acpi/ich9.h            |   5 +++
> > > >  include/hw/acpi/pcihp.h           |   3 +-
> > > >  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++
> > > >  hw/acpi/pcihp.c                   |  16 ++++---
> > > >  hw/acpi/piix4.c                   |   4 +-
> > > >  hw/i386/acpi-build.c              |  31 ++++++++------
> > > >  hw/i386/pc.c                      |   1 +
> > > >  hw/pci/pcie.c                     |  16 +++++++
> > > >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes
> > > >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes
> > > >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes
> > > >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes
> > > >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes
> > > >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes
> > > >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes
> > > >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes
> > > >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes
> > > >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes
> > > >  19 files changed, 129 insertions(+), 21 deletions(-)
> > > >
> > > > --
> > > > 2.25.4
> > >
>
Ani Sinha Oct. 1, 2020, 4:11 p.m. UTC | #19
On Thu, Oct 1, 2020 at 9:24 PM Julia Suvorova <jusual@redhat.com> wrote:

> On Thu, Oct 1, 2020 at 1:40 PM Michael S. Tsirkin <mst@redhat.com> wrote:

>

> >

>

> > On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:

>

> > > On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin <mst@redhat.com>

> wrote:

>

> > > >

>

> > > > On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:

>

> > > > > The patch set consists of two parts:

>

> > > > > patches 1-4: introduce new feature

>

> > > > >              'acpi-pci-hotplug-with-bridge-support' on Q35

>

> > > > > patches 5-7: make the feature default along with changes in ACPI

> tables

>

> > > > >

>

> > > > > This way maintainers can decide which way to choose without

> breaking

>

> > > > > the patch set.

>

> > > > >

>

> > > > > With the feature disabled Q35 falls back to the native hot-plug.

>

> > > > >

>

> > > > > Pros

>

> > > > >     * no racy behavior during boot (see 110c477c2ed)

>

> > > > >     * eject is possible - according to PCIe spec, attention button

>

> > > > >       press should lead to power off, and then the adapter should

> be

>

> > > > >       removed manually. As there is no power down state exists in

> QEMU,

>

> > > > >       we cannot distinguish between an eject and a power down

>

> > > > >       request.

>

> > > > >     * no delay during deleting - after the actual power off

> software

>

> > > > >       must wait at least 1 second before indicating about it. This

> case

>

> > > > >       is quite important for users, it even has its own bug:

>

> > > > >           https://bugzilla.redhat.com/show_bug.cgi?id=1594168

>

> > > > >     * no timer-based behavior - in addition to the previous

> example,

>

> > > > >       the attention button has a 5-second waiting period, during

> which

>

> > > > >       the operation can be canceled with a second press. While this

>

> > > > >       looks fine for manual button control, automation will result

> in

>

> > > > >       the need to queue or drop events, and the software receiving

>

> > > > >       events in all sort of unspecified combinations of

> attention/power

>

> > > > >       indicator states, which is racy and uppredictable.

>

> > > > >     * fixes:

>

> > > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1752465

>

> > > > >         * https://bugzilla.redhat.com/show_bug.cgi?id=1690256

>

> > > > >

>

> > > > > Cons:

>

> > > > >     * lose per-port control over hot-plug (can be resolved)

>

> > > > >     * no access to possible features presented in slot capabilities

>

> > > > >       (this is only surprise removal AFAIK)

>

> > > >

>

> > > > something I don't quite get is whether with this one can still add

>

> > > > new pcie bridges and then hotplug into these with native

>

> > > > hotplug.

>

> > >

>

> > > Right now I disable native if there is acpihp anywhere, but even if

>

> > > you enable it for hotplugged devices, native hot-plug will not work.

>

> >

>

> > So that's a minor regression in functionality, right?

>

> > Why is that the case? Because you disable it in ACPI?

>

> > What if we don't?

>

>

>

> I meant that I disable slot hotplug capabilities, nothing in ACPI

>

> prevents native from working. Actually, I don't see if there's any

>

> regression at all. Configurations like hot-plugging downstream port or

>

> switch to another downstream port haven't worked before, and they

>

> don't work now. I can enable native for hotplugged bridges, but that

>

> doesn't make sense, because you won't be able to hot-plug anything to

>

> it.



Why is that?

It's not an issue of ACPI, it's PCIe behaviour. Also, native-acpi
>

> combination may seem bizarre to os (slot enumeration is independent,

>

> that's why I suggested disabling pcie slot flags).

>

>

>

> > > > the challenge to answering this with certainty is that right now qemu

>

> > > > does not allow hotplugging complex devices such as bridges at all,

>

> > > > we only have a hack for multifunction devices.

>

> > > > Maybe adding a bridge as function 1 on command line, then

> hotplugging another device as

>

> > > > function 0 will work to test this.

>

> > > >

>

> > > >

>

> > > >

>

> > > > > v3:

>

> > > > >     * drop change of _OSC to allow SHPC on hotplugged bridges

>

> > > > >     * use 'acpi-root-pci-hotplug'

>

> > > > >     * add migration states [Igor]

>

> > > > >     * minor style changes

>

> > > > >

>

> > > > > v2:

>

> > > > >     * new ioport range for acpiphp [Gerd]

>

> > > > >     * drop find_pci_host() [Igor]

>

> > > > >     * explain magic numbers in _OSC [Igor]

>

> > > > >     * drop build_q35_pci_hotplug() wrapper [Igor]

>

> > > > >

>

> > > > > Julia Suvorova (7):

>

> > > > >   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support

> Q35

>

> > > > >   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35

>

> > > > >   hw/pci/pcie: Do not initialize slot capability if acpihp is used

>

> > > > >   hw/acpi/ich9: Enable ACPI PCI hot-plug

>

> > > > >   bios-tables-test: Allow changes in DSDT ACPI tables

>

> > > > >   hw/acpi/ich9: Set ACPI PCI hot-plug as default

>

> > > > >   bios-tables-test: Update golden binaries

>

> > > > >

>

> > > > >  hw/i386/acpi-build.h              |   7 ++++

>

> > > > >  include/hw/acpi/ich9.h            |   5 +++

>

> > > > >  include/hw/acpi/pcihp.h           |   3 +-

>

> > > > >  hw/acpi/ich9.c                    |  67

> ++++++++++++++++++++++++++++++

>

> > > > >  hw/acpi/pcihp.c                   |  16 ++++---

>

> > > > >  hw/acpi/piix4.c                   |   4 +-

>

> > > > >  hw/i386/acpi-build.c              |  31 ++++++++------

>

> > > > >  hw/i386/pc.c                      |   1 +

>

> > > > >  hw/pci/pcie.c                     |  16 +++++++

>

> > > > >  tests/data/acpi/q35/DSDT          | Bin 7678 -> 7950 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -> 9274 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -> 9865 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -> 8413 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -> 9603 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -> 8025 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -> 9309 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -> 9080 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -> 7956 bytes

>

> > > > >  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -> 8555 bytes

>

> > > > >  19 files changed, 129 insertions(+), 21 deletions(-)

>

> > > > >

>

> > > > > --

>

> > > > > 2.25.4

>

> > > >

>

> >

>

>

>

>
<div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 1, 2020 at 9:24 PM Julia Suvorova &lt;<a href="mailto:jusual@redhat.com">jusual@redhat.com</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">On Thu, Oct 1, 2020 at 1:40 PM Michael S. Tsirkin &lt;<a href="mailto:mst@redhat.com" target="_blank">mst@redhat.com</a>&gt; wrote:<br><br>&gt;<br><br>&gt; On Thu, Oct 01, 2020 at 10:55:35AM +0200, Julia Suvorova wrote:<br><br>&gt; &gt; On Thu, Sep 24, 2020 at 11:20 AM Michael S. Tsirkin &lt;<a href="mailto:mst@redhat.com" target="_blank">mst@redhat.com</a>&gt; wrote:<br><br>&gt; &gt; &gt;<br><br>&gt; &gt; &gt; On Thu, Sep 24, 2020 at 09:00:06AM +0200, Julia Suvorova wrote:<br><br>&gt; &gt; &gt; &gt; The patch set consists of two parts:<br><br>&gt; &gt; &gt; &gt; patches 1-4: introduce new feature<br><br>&gt; &gt; &gt; &gt;              &#39;acpi-pci-hotplug-with-bridge-support&#39; on Q35<br><br>&gt; &gt; &gt; &gt; patches 5-7: make the feature default along with changes in ACPI tables<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; This way maintainers can decide which way to choose without breaking<br><br>&gt; &gt; &gt; &gt; the patch set.<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; With the feature disabled Q35 falls back to the native hot-plug.<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; Pros<br><br>&gt; &gt; &gt; &gt;     * no racy behavior during boot (see 110c477c2ed)<br><br>&gt; &gt; &gt; &gt;     * eject is possible - according to PCIe spec, attention button<br><br>&gt; &gt; &gt; &gt;       press should lead to power off, and then the adapter should be<br><br>&gt; &gt; &gt; &gt;       removed manually. As there is no power down state exists in QEMU,<br><br>&gt; &gt; &gt; &gt;       we cannot distinguish between an eject and a power down<br><br>&gt; &gt; &gt; &gt;       request.<br><br>&gt; &gt; &gt; &gt;     * no delay during deleting - after the actual power off software<br><br>&gt; &gt; &gt; &gt;       must wait at least 1 second before indicating about it. This case<br><br>&gt; &gt; &gt; &gt;       is quite important for users, it even has its own bug:<br><br>&gt; &gt; &gt; &gt;           <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1594168" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1594168</a><br><br>&gt; &gt; &gt; &gt;     * no timer-based behavior - in addition to the previous example,<br><br>&gt; &gt; &gt; &gt;       the attention button has a 5-second waiting period, during which<br><br>&gt; &gt; &gt; &gt;       the operation can be canceled with a second press. While this<br><br>&gt; &gt; &gt; &gt;       looks fine for manual button control, automation will result in<br><br>&gt; &gt; &gt; &gt;       the need to queue or drop events, and the software receiving<br><br>&gt; &gt; &gt; &gt;       events in all sort of unspecified combinations of attention/power<br><br>&gt; &gt; &gt; &gt;       indicator states, which is racy and uppredictable.<br><br>&gt; &gt; &gt; &gt;     * fixes:<br><br>&gt; &gt; &gt; &gt;         * <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1752465" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1752465</a><br><br>&gt; &gt; &gt; &gt;         * <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1690256" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1690256</a><br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; Cons:<br><br>&gt; &gt; &gt; &gt;     * lose per-port control over hot-plug (can be resolved)<br><br>&gt; &gt; &gt; &gt;     * no access to possible features presented in slot capabilities<br><br>&gt; &gt; &gt; &gt;       (this is only surprise removal AFAIK)<br><br>&gt; &gt; &gt;<br><br>&gt; &gt; &gt; something I don&#39;t quite get is whether with this one can still add<br><br>&gt; &gt; &gt; new pcie bridges and then hotplug into these with native<br><br>&gt; &gt; &gt; hotplug.<br><br>&gt; &gt;<br><br>&gt; &gt; Right now I disable native if there is acpihp anywhere, but even if<br><br>&gt; &gt; you enable it for hotplugged devices, native hot-plug will not work.<br><br>&gt;<br><br>&gt; So that&#39;s a minor regression in functionality, right?<br><br>&gt; Why is that the case? Because you disable it in ACPI?<br><br>&gt; What if we don&#39;t?<br><br><br><br>I meant that I disable slot hotplug capabilities, nothing in ACPI<br><br>prevents native from working. Actually, I don&#39;t see if there&#39;s any<br><br>regression at all. Configurations like hot-plugging downstream port or<br><br>switch to another downstream port haven&#39;t worked before, and they<br><br>don&#39;t work now. I can enable native for hotplugged bridges, but that<br><br>doesn&#39;t make sense, because you won&#39;t be able to hot-plug anything to<br><br>it. </blockquote><div dir="auto"><br></div><div dir="auto">Why is that?</div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)" dir="auto">It&#39;s not an issue of ACPI, it&#39;s PCIe behaviour. Also, native-acpi<br><br>combination may seem bizarre to os (slot enumeration is independent,<br><br>that&#39;s why I suggested disabling pcie slot flags).<br><br><br><br>&gt; &gt; &gt; the challenge to answering this with certainty is that right now qemu<br><br>&gt; &gt; &gt; does not allow hotplugging complex devices such as bridges at all,<br><br>&gt; &gt; &gt; we only have a hack for multifunction devices.<br><br>&gt; &gt; &gt; Maybe adding a bridge as function 1 on command line, then hotplugging another device as<br><br>&gt; &gt; &gt; function 0 will work to test this.<br><br>&gt; &gt; &gt;<br><br>&gt; &gt; &gt;<br><br>&gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; v3:<br><br>&gt; &gt; &gt; &gt;     * drop change of _OSC to allow SHPC on hotplugged bridges<br><br>&gt; &gt; &gt; &gt;     * use &#39;acpi-root-pci-hotplug&#39;<br><br>&gt; &gt; &gt; &gt;     * add migration states [Igor]<br><br>&gt; &gt; &gt; &gt;     * minor style changes<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; v2:<br><br>&gt; &gt; &gt; &gt;     * new ioport range for acpiphp [Gerd]<br><br>&gt; &gt; &gt; &gt;     * drop find_pci_host() [Igor]<br><br>&gt; &gt; &gt; &gt;     * explain magic numbers in _OSC [Igor]<br><br>&gt; &gt; &gt; &gt;     * drop build_q35_pci_hotplug() wrapper [Igor]<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; Julia Suvorova (7):<br><br>&gt; &gt; &gt; &gt;   hw/acpi/pcihp: Enhance acpi_pcihp_disable_root_bus() to support Q35<br><br>&gt; &gt; &gt; &gt;   hw/i386/acpi-build: Add ACPI PCI hot-plug methods to Q35<br><br>&gt; &gt; &gt; &gt;   hw/pci/pcie: Do not initialize slot capability if acpihp is used<br><br>&gt; &gt; &gt; &gt;   hw/acpi/ich9: Enable ACPI PCI hot-plug<br><br>&gt; &gt; &gt; &gt;   bios-tables-test: Allow changes in DSDT ACPI tables<br><br>&gt; &gt; &gt; &gt;   hw/acpi/ich9: Set ACPI PCI hot-plug as default<br><br>&gt; &gt; &gt; &gt;   bios-tables-test: Update golden binaries<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt;  hw/i386/acpi-build.h              |   7 ++++<br><br>&gt; &gt; &gt; &gt;  include/hw/acpi/ich9.h            |   5 +++<br><br>&gt; &gt; &gt; &gt;  include/hw/acpi/pcihp.h           |   3 +-<br><br>&gt; &gt; &gt; &gt;  hw/acpi/ich9.c                    |  67 ++++++++++++++++++++++++++++++<br><br>&gt; &gt; &gt; &gt;  hw/acpi/pcihp.c                   |  16 ++++---<br><br>&gt; &gt; &gt; &gt;  hw/acpi/piix4.c                   |   4 +-<br><br>&gt; &gt; &gt; &gt;  hw/i386/acpi-build.c              |  31 ++++++++------<br><br>&gt; &gt; &gt; &gt;  hw/i386/pc.c                      |   1 +<br><br>&gt; &gt; &gt; &gt;  hw/pci/pcie.c                     |  16 +++++++<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT          | Bin 7678 -&gt; 7950 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.acpihmat | Bin 9002 -&gt; 9274 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.bridge   | Bin 7695 -&gt; 9865 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.cphp     | Bin 8141 -&gt; 8413 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.dimmpxm  | Bin 9331 -&gt; 9603 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.ipmibt   | Bin 7753 -&gt; 8025 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.memhp    | Bin 9037 -&gt; 9309 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.mmio64   | Bin 8808 -&gt; 9080 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.numamem  | Bin 7684 -&gt; 7956 bytes<br><br>&gt; &gt; &gt; &gt;  tests/data/acpi/q35/DSDT.tis      | Bin 8283 -&gt; 8555 bytes<br><br>&gt; &gt; &gt; &gt;  19 files changed, 129 insertions(+), 21 deletions(-)<br><br>&gt; &gt; &gt; &gt;<br><br>&gt; &gt; &gt; &gt; --<br><br>&gt; &gt; &gt; &gt; 2.25.4<br><br>&gt; &gt; &gt;<br><br>&gt;<br><br><br><br></blockquote></div></div>
Michael S. Tsirkin Oct. 6, 2020, 6:44 a.m. UTC | #20
On Thu, Oct 01, 2020 at 05:54:39PM +0200, Julia Suvorova wrote:
> > > Right now I disable native if there is acpihp anywhere, but even if
> > > you enable it for hotplugged devices, native hot-plug will not work.
> >
> > So that's a minor regression in functionality, right?
> > Why is that the case? Because you disable it in ACPI?
> > What if we don't?
> 
> I meant that I disable slot hotplug capabilities, nothing in ACPI
> prevents native from working. Actually, I don't see if there's any
> regression at all. Configurations like hot-plugging downstream port or
> switch to another downstream port haven't worked before, and they
> don't work now. I can enable native for hotplugged bridges, but that
> doesn't make sense, because you won't be able to hot-plug anything to
> it.


You can do the following hack right now:
1- add an upstream port as function 1
2- add a downstream port behind it
3- add some other device (e.g. another upstream port?) as function 0

As this point both ports should be detected.

Going forward we can consider support for adding ports in a hidden state
(not visible to guest) so one won't need an extra function.

> It's not an issue of ACPI, it's PCIe behaviour. Also, native-acpi
> combination may seem bizarre to os

Maybe, maybe not ...
Worth testing whether this works with existing guests.

> (slot enumeration is independent,
> that's why I suggested disabling pcie slot flags).

Yes that part makes sense imho.