mbox series

[v4,0/5] s390x/pci: Accomodate vfio DMA limiting

Message ID 1600352445-21110-1-git-send-email-mjrosato@linux.ibm.com
Headers show
Series s390x/pci: Accomodate vfio DMA limiting | expand

Message

Matthew Rosato Sept. 17, 2020, 2:20 p.m. UTC 
Kernel commit 492855939bdb added a limit to the number of outstanding DMA
requests for a type1 vfio container.  However, lazy unmapping in s390 can 
in fact cause quite a large number of outstanding DMA requests to build up
prior to being purged, potentially the entire guest DMA space.  This
results in unexpected 'VFIO_MAP_DMA failed: No space left on device'
conditions seen in QEMU.

This patchset adds support to qemu to retrieve the number of allowable DMA
requests via the VFIO_IOMMU_GET_INFO ioctl.  The patches are separated into
vfio hits which add support for reading in VFIO_IOMMU_GET_INFO capability
chains and getting the per-container dma_avail value, and s390 hits to 
track DMA usage on a per-container basis.

Associated kernel patch:
https://marc.info/?l=kvm&m=160019703922812&w=2


Changes from v3:
- Patch 2,3: Added RBs provided (thank you!)
- Patch 4: retry the ioctl in the same manner as vfio_get_region_info
- Patch 5: switch to static inline functions
- Patch 5: relocate more code to s390-pci-vfio.*


Matthew Rosato (5):
  linux-headers: update against 5.9-rc5
  vfio: Create shared routine for scanning info capabilities
  vfio: Find DMA available capability
  s390x/pci: Add routine to get the vfio dma available count
  s390x/pci: Honor DMA limits set by vfio

 hw/s390x/meson.build                               |   1 +
 hw/s390x/s390-pci-bus.c                            |  16 ++-
 hw/s390x/s390-pci-bus.h                            |   9 ++
 hw/s390x/s390-pci-inst.c                           |  45 ++++++-
 hw/s390x/s390-pci-inst.h                           |   3 +
 hw/s390x/s390-pci-vfio.c                           |  96 ++++++++++++++
 hw/s390x/s390-pci-vfio.h                           |  22 ++++
 hw/vfio/common.c                                   |  52 ++++++--
 include/hw/vfio/vfio-common.h                      |   2 +
 include/standard-headers/drm/drm_fourcc.h          | 140 +++++++++++++++++++++
 include/standard-headers/linux/ethtool.h           |  87 +++++++++++++
 include/standard-headers/linux/input-event-codes.h |   3 +-
 include/standard-headers/linux/vhost_types.h       |  11 ++
 include/standard-headers/linux/virtio_9p.h         |   4 +-
 include/standard-headers/linux/virtio_blk.h        |  26 ++--
 include/standard-headers/linux/virtio_config.h     |   8 +-
 include/standard-headers/linux/virtio_console.h    |   8 +-
 include/standard-headers/linux/virtio_net.h        |   6 +-
 include/standard-headers/linux/virtio_scsi.h       |  20 +--
 linux-headers/asm-generic/unistd.h                 |   6 +-
 linux-headers/asm-mips/unistd_n32.h                |   1 +
 linux-headers/asm-mips/unistd_n64.h                |   1 +
 linux-headers/asm-mips/unistd_o32.h                |   1 +
 linux-headers/asm-powerpc/kvm.h                    |   5 +
 linux-headers/asm-powerpc/unistd_32.h              |   1 +
 linux-headers/asm-powerpc/unistd_64.h              |   1 +
 linux-headers/asm-s390/kvm.h                       |   7 +-
 linux-headers/asm-s390/unistd_32.h                 |   1 +
 linux-headers/asm-s390/unistd_64.h                 |   1 +
 linux-headers/asm-x86/unistd_32.h                  |   1 +
 linux-headers/asm-x86/unistd_64.h                  |   1 +
 linux-headers/asm-x86/unistd_x32.h                 |   1 +
 linux-headers/linux/kvm.h                          |  10 +-
 linux-headers/linux/vfio.h                         |  17 ++-
 linux-headers/linux/vhost.h                        |   2 +
 35 files changed, 554 insertions(+), 62 deletions(-)
 create mode 100644 hw/s390x/s390-pci-vfio.c
 create mode 100644 hw/s390x/s390-pci-vfio.h

Comments

no-reply@patchew.org Sept. 17, 2020, 2:46 p.m. UTC | #1 
Patchew URL: https://patchew.org/QEMU/1600352445-21110-1-git-send-email-mjrosato@linux.ibm.com/



Hi,

This series failed the docker-quick@centos7 build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.






The full log is available at
http://patchew.org/logs/1600352445-21110-1-git-send-email-mjrosato@linux.ibm.com/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Matthew Rosato Sept. 17, 2020, 3:05 p.m. UTC | #2 
On 9/17/20 10:46 AM, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/1600352445-21110-1-git-send-email-mjrosato@linux.ibm.com/

> 

> 

> 

> Hi,

> 

> This series failed the docker-quick@centos7 build test. Please find the testing commands and

> their output below. If you have Docker installed, you can probably reproduce it

> locally.

> 

> 


Errors seem to be of the flavor

'N/A. Internal error while reading log file'
and
'No space left on device'

seems like a patchew disk issue rather than something with this patchset?

> 

> 

> 

> 

> The full log is available at

> http://patchew.org/logs/1600352445-21110-1-git-send-email-mjrosato@linux.ibm.com/testing.docker-quick@centos7/?type=message.

> ---

> Email generated automatically by Patchew [https://patchew.org/].

> Please send your feedback to patchew-devel@redhat.com

>
Cornelia Huck Sept. 22, 2020, 10:16 a.m. UTC | #3 
On Thu, 17 Sep 2020 10:20:45 -0400
Matthew Rosato <mjrosato@linux.ibm.com> wrote:

> When an s390 guest is using lazy unmapping, it can result in a very
> large number of oustanding DMA requests, far beyond the default
> limit configured for vfio.  Let's track DMA usage similar to vfio
> in the host, and trigger the guest to flush their DMA mappings
> before vfio runs out.
> 
> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
> ---
>  hw/s390x/s390-pci-bus.c  | 16 +++++++++++-----
>  hw/s390x/s390-pci-bus.h  |  9 +++++++++
>  hw/s390x/s390-pci-inst.c | 45 +++++++++++++++++++++++++++++++++++++++------
>  hw/s390x/s390-pci-inst.h |  3 +++
>  hw/s390x/s390-pci-vfio.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>  hw/s390x/s390-pci-vfio.h |  5 +++++
>  6 files changed, 109 insertions(+), 11 deletions(-)

Reviewed-by: Cornelia Huck <cohuck@redhat.com>