mbox series

[v2,00/38] qapi: static typing conversion, pt1

Message ID 20200922210101.4081073-1-jsnow@redhat.com
Headers show
Series qapi: static typing conversion, pt1 | expand

Message

John Snow Sept. 22, 2020, 9 p.m. UTC 
Hi, this series adds static type hints to the QAPI module.
This is part one!

Part 1: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt1
Everything: https://gitlab.com/jsnow/qemu/-/tree/python-qapi-cleanup-pt6

- Requires Python 3.6+
- Requires mypy 0.770 or newer (for type analysis only)
- Requires pylint 2.6.0 or newer (for lint checking only)

In general, this series tackles the cleanup of one individual QAPI
module at a time. Once it passes pylint or mypy checks, those checks are
enabled for that file.

Type hints are added in patches that add *only* type hints and change no
other behavior. Any necessary changes to behavior to accommodate typing
are split out into their own tiny patches.

- Patches 1-7: generic, qapi-wide cleanups, flake8 and pylint support
- Patches 8-16: common.py cleanup (with mypy support added at the end)
- Patches 17-18: events.py cleanup
- Patches 19-21: commands.py cleanup
- Patches 22-23: source.py cleanup
- Patches 24-29: gen.py cleanup
- Patches 30-33: introspect.py cleanup
- Patches 34-35: types.py cleanup
- Patches 36-37: visit.py cleanup

Notes:
- After patch 6, `flake8 qapi/` should pass 100% on this and every
  future commit.

- After patch 7, `pylint --rcfile=3Dqapi/pylintrc qapi/` should pass 100%
  on this and every future commit.

- After patch 16, `mypy --config-file=3Dqapi/mypy.ini qapi/` should pass
  100% on this and every future commit.

V2:
 - Removed Python3.6 patch in favor of Thomas Huth's
 - Addressed (most) feedback from Markus
 - Renamed type hint annotation commits

Changelog:

003/38:[0082] [FC] 'qapi: move generator entrypoint into module'
004/38:[down] 'qapi: Prefer explicit relative imports'
006/38:[0027] [FC] 'qapi: delint using flake8'
007/38:[0006] [FC] 'qapi: add pylintrc'
008/38:[0002] [FC] 'qapi/common.py: Remove python compatibility workaround'
009/38:[0037] [FC] 'qapi/common.py: Add indent manager'
010/38:[0008] [FC] 'qapi/common.py: delint with pylint'
011/38:[0002] [FC] 'qapi/common.py: Replace one-letter 'c' variable'
012/38:[----] [-C] 'qapi/common.py: check with pylint'
013/38:[down] 'qapi/common.py: add type hint annotations'
014/38:[down] 'qapi/common.py: Convert comments into docstrings, and elaborat=
e'
015/38:[down] 'qapi/common.py: move build_params into gen.py'
017/38:[down] 'qapi/events.py: add type hint annotations'
020/38:[0006] [FC] 'qapi/commands.py: add notational type hints'
022/38:[down] 'qapi/source.py: add type hint annotations'
025/38:[down] 'qapi/gen.py: add type hint annotations'
033/38:[down] 'qapi/introspect.py: add type hint annotations'
034/38:[down] 'qapi/types.py: add type hint annotations'
036/38:[down] 'qapi/visit.py: assert tag_member contains a QAPISchemaEnumType'
037/38:[----] [-C] 'qapi/visit.py: remove unused parameters from gen_visit_ob=
ject'
038/38:[down] 'qapi/visit.py: add type hint annotations'

John Snow (38):
  [DO-NOT-MERGE] qapi: add debugging tools
  qapi-gen: Separate arg-parsing from generation
  qapi: move generator entrypoint into module
  qapi: Prefer explicit relative imports
  qapi: Remove wildcard includes
  qapi: delint using flake8
  qapi: add pylintrc
  qapi/common.py: Remove python compatibility workaround
  qapi/common.py: Add indent manager
  qapi/common.py: delint with pylint
  qapi/common.py: Replace one-letter 'c' variable
  qapi/common.py: check with pylint
  qapi/common.py: add type hint annotations
  qapi/common.py: Convert comments into docstrings, and elaborate
  qapi/common.py: move build_params into gen.py
  qapi: establish mypy type-checking baseline
  qapi/events.py: add type hint annotations
  qapi/events.py: Move comments into docstrings
  qapi/commands.py: Don't re-bind to variable of different type
  qapi/commands.py: add notational type hints
  qapi/commands.py: enable checking with mypy
  qapi/source.py: add type hint annotations
  qapi/source.py: delint with pylint
  qapi/gen.py: Fix edge-case of _is_user_module
  qapi/gen.py: add type hint annotations
  qapi/gen.py: Enable checking with mypy
  qapi/gen.py: Remove unused parameter
  qapi/gen.py: update write() to be more idiomatic
  qapi/gen.py: delint with pylint
  qapi/introspect.py: Add a typed 'extra' structure
  qapi/introspect.py: add _gen_features helper
  qapi/introspect.py: create a typed 'Node' data structure
  qapi/introspect.py: add type hint annotations
  qapi/types.py: add type hint annotations
  qapi/types.py: remove one-letter variables
  qapi/visit.py: assert tag_member contains a QAPISchemaEnumType
  qapi/visit.py: remove unused parameters from gen_visit_object
  qapi/visit.py: add type hint annotations

 scripts/qapi-gen.py        |  59 ++---------
 scripts/qapi/.flake8       |   2 +
 scripts/qapi/commands.py   |  85 +++++++++++----
 scripts/qapi/common.py     | 163 +++++++++++++++-------------
 scripts/qapi/debug.py      |  78 ++++++++++++++
 scripts/qapi/doc.py        |   3 +-
 scripts/qapi/events.py     |  64 ++++++++---
 scripts/qapi/expr.py       |   4 +-
 scripts/qapi/gen.py        | 176 ++++++++++++++++++------------
 scripts/qapi/introspect.py | 212 +++++++++++++++++++++++++------------
 scripts/qapi/main.py       |  91 ++++++++++++++++
 scripts/qapi/mypy.ini      |  30 ++++++
 scripts/qapi/parser.py     |   4 +-
 scripts/qapi/pylintrc      |  71 +++++++++++++
 scripts/qapi/schema.py     |  33 +++---
 scripts/qapi/source.py     |  34 +++---
 scripts/qapi/types.py      | 125 +++++++++++++++-------
 scripts/qapi/visit.py      | 116 ++++++++++++++------
 18 files changed, 952 insertions(+), 398 deletions(-)
 create mode 100644 scripts/qapi/.flake8
 create mode 100644 scripts/qapi/debug.py
 create mode 100644 scripts/qapi/main.py
 create mode 100644 scripts/qapi/mypy.ini
 create mode 100644 scripts/qapi/pylintrc

--=20
2.26.2

Comments

John Snow Sept. 24, 2020, 8:37 p.m. UTC | #1 
On 9/22/20 5:00 PM, John Snow wrote:
> Hi, this series adds static type hints to the QAPI module.

> This is part one!

> 


Hi, I will be sending out a v3 shortly.

The status of this series is presently as follows:

+ [01] do-not-merge-qapi-add          #
+ [02] qapi-gen-separate-arg-parsing  # [SOB] JS [RB] EH
+ [03] qapi-move-generator-entrypoint # [SOB] JS [RB] CR,EH [TB] CR
+ [04] qapi-prefer-explicit-relative  # [SOB] JS [RB] CR,EH
+ [05] qapi-remove-wildcard-includes  # [SOB] JS [RB] CR,EH
+ [06] qapi-enforce-import-order      # [SOB] JS
+ [07] qapi-delint-using-flake8       # [SOB] JS [RB] CR,EH
+ [08] qapi-add-pylintrc              # [SOB] JS [TB] CR,EH [RB] CR
+ [09] qapi-common-py-remove-python   # [SOB] JS [RB] CR,EH
+ [10] qapi-common-add-indent-manager # [SOB] JS [RB] CR,EH
+ [11] qapi-common-py-delint-with     # [SOB] JS [RB] CR,EH
+ [12] replace-c-by-char              # [SOB] JS [RB] CR,EH
+ [13] qapi-common-py-check-with      # [SOB] JS [RB] CR [TB] CR,EH
+ [14] qapi-common-py-add-notational  # [SOB] JS [RB] CR,EH
+ [15] qapi-common-move-comments-into # [SOB] JS [RB] CR,EH
+ [16] qapi-split-build_params-into   # [SOB] JS [RB] CR,EH
+ [17] qapi-establish-mypy-type       # [SOB] JS [TB] EH
+ [18] qapi-events-py-add-notational  # [SOB] JS [RB] CR,EH
+ [19] qapi-events-move-comments-into # [SOB] JS [RB] CR,EH
+ [20] qapi-commands-py-don-t-re-bind # [SOB] JS [RB] CR,EH
+ [21] qapi-commands-py-add           # [SOB] JS [RB] CR,EH
+ [22] qapi-commands-py-enable        # [SOB] JS [RB] CR,EH
+ [23] qapi-source-py-add-notational  # [SOB] JS [RB] EH
+ [24] qapi-source-py-delint-with     # [SOB] JS [RB] CR,EH [TB] CR
+ [25] qapi-gen-py-fix-edge-case-of   #
+ [26] qapi-gen-py-add-notational     # [SOB] JS [RB] CR,EH
+ [27] qapi-gen-py-enable-checking    # [SOB] JS [RB] CR,EH [TB] CR
+ [28] qapi-gen-py-remove-unused      # [SOB] JS [RB] CR,EH
+ [29] qapi-gen-py-update-write-to-be # [SOB] JS [RB] CR,EH
+ [30] qapi-gen-py-delint-with-pylint # [SOB] JS [RB] CR,EH
+ [31] qapi-introspect-py-assert-obj  #
+ [32] qapi-introspect-py-create-a    # [SOB] EH,JS
+ [33] qapi-introspect-py-add         #
+ [34] qapi-introspect-py-unify       #
+ [35] qapi-introspect-py-replace     #
+ [36] qapi-introspect-py-create-a-0  #
+ [37] qapi-types-py-add-type-hint    # [SOB] JS [RB] CR,EH
+ [38] qapi-types-py-remove-one       # [SOB] JS [RB] CR,EH
+ [39] qapi-visit-py-assert           # [SOB] JS [RB] CR,EH
+ [40] qapi-visit-py-remove-unused    # [SOB] JS [RB] CR,EH [TB] CR
 > [41] qapi-visit-py-add-notational   # [SOB] JS [RB] CR,EH [TB] CR



Thank you for your help so far, everyone!

--js