mbox series

[v4,00/16] Add a General Virtual Device Fuzzer

Message ID 20201015134137.205958-1-alxndr@bu.edu
Headers show
Series Add a General Virtual Device Fuzzer | expand

Message

Alexander Bulekov Oct. 15, 2020, 1:41 p.m. UTC
v4:
    - Replace yaml + c template-based oss-fuzz configs, with C code to
      register a FuzzTarget for each config (as suggested by Paolo)
    - Replicate the functionality of address_space_write_rom to ensure
      matching behavior when QTEST_LOG is enabled
    - Improve code documentation/comments
    - Small formatting changes
v3:
	- Use flatviews to help select regions for fuzzing 
	- Meson-related changes
    - Add some documentation
	- Improve minimalization script to trim write{bwlq} commands
v2:
	- Remove QOS dependency.
	- Add a custom crossover function
	- Fix broken minimization scripts
	- Fixes to the IO region and DMA handling code

This is a general virtual-device fuzzer, designed to fuzz devices over Port IO,
MMIO, and DMA.

To get started with this:
 1. Build the fuzzers (see docs/devel/fuzzing.txt)
    Note: Build with --enable-sanitizers, or create a "dictionary file":
    echo kw1=\"FUZZ\" > dict
    and pass it as an argument to libFuzzer with -dict=./dict
    This magic value is a command separator that lets the fuzzer perform
    multiple IO actions with a single input.

 2. Pick the qemu arguments you wish to fuzz:
    export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon"

 3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the
 "info qom-tree", "info qtree" and "info mtree" commands useful for identifying
 these. Supports globbing. Here I will try to simultaneously fuzz(for no good
 reason) virtio-balloon and e1000e, which is included by default in the q35:
    export QEMU_FUZZ_OBJECTS='virtio* e1000*'
    You can also try to fuzz the whole machine:
    export QEMU_FUZZ_OBJECTS='*'

 4. Run the fuzzer for 0 inputs. The fuzzer should output a list of
 MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your
 expectations.
    ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0

 5. Run the fuzzer:
    ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz 


Basically, at the core, this fuzzer is an interpreter that splits the input
into a series of commands, such as mmio_write, pio_write, etc. We structure
these commands to hit only MemoryRegions that are associated with the devices
specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to
functions that are typically used by virtual-devices to read from RAM (DMA).
These hooks attempt to populate these DMA regions with fuzzed data, just in
time.

Some of the issues I have found or reproduced with this fuzzer:
https://bugs.launchpad.net/bugs/1525123
https://bugs.launchpad.net/bugs/1681439
https://bugs.launchpad.net/bugs/1777315
https://bugs.launchpad.net/bugs/1878034
https://bugs.launchpad.net/bugs/1878043
https://bugs.launchpad.net/bugs/1878054
https://bugs.launchpad.net/bugs/1878057
https://bugs.launchpad.net/bugs/1878067
https://bugs.launchpad.net/bugs/1878134
https://bugs.launchpad.net/bugs/1878136
https://bugs.launchpad.net/bugs/1878253
https://bugs.launchpad.net/bugs/1878255
https://bugs.launchpad.net/bugs/1878259
https://bugs.launchpad.net/bugs/1878263
https://bugs.launchpad.net/bugs/1878323
https://bugs.launchpad.net/bugs/1878641
https://bugs.launchpad.net/bugs/1878642
https://bugs.launchpad.net/bugs/1878645
https://bugs.launchpad.net/bugs/1878651
https://bugs.launchpad.net/bugs/1879223
https://bugs.launchpad.net/bugs/1879227
https://bugs.launchpad.net/bugs/1879531
https://bugs.launchpad.net/bugs/1880355
https://bugs.launchpad.net/bugs/1880539
https://bugs.launchpad.net/bugs/1884693
https://bugs.launchpad.net/bugs/1886362
https://bugs.launchpad.net/bugs/1887303
https://bugs.launchpad.net/bugs/1887309
https://bugs.launchpad.net/bugs/697510

Alexander Bulekov (16):
  memory: Add FlatView foreach function
  fuzz: Add general virtual-device fuzzer
  fuzz: Add PCI features to the general fuzzer
  fuzz: Add DMA support to the generic-fuzzer
  fuzz: Declare DMA Read callback function
  fuzz: Add fuzzer callbacks to DMA-read functions
  fuzz: Add support for custom crossover functions
  fuzz: add a DISABLE_PCI op to general-fuzzer
  fuzz: add a crossover function to generic-fuzzer
  scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
  scripts/oss-fuzz: Add crash trace minimization script
  fuzz: Add instructions for using general-fuzz
  fuzz: add an "opaque" to the FuzzTarget struct
  fuzz: add general-fuzz configs for oss-fuzz
  fuzz: register predefined general-fuzz configs
  scripts/oss-fuzz: remove the general-fuzz target

 docs/devel/fuzzing.txt                        |  39 +
 include/exec/memory.h                         |  21 +
 include/exec/memory_ldst_cached.h.inc         |   3 +
 memory_ldst.c.inc                             |   4 +
 scripts/oss-fuzz/build.sh                     |   6 +
 scripts/oss-fuzz/minimize_qtest_trace.py      | 157 +++
 .../oss-fuzz/reorder_fuzzer_qtest_trace.py    | 103 ++
 softmmu/memory.c                              |  23 +
 softmmu/physmem.c                             |   2 +
 tests/qtest/fuzz/fuzz.c                       |  13 +
 tests/qtest/fuzz/fuzz.h                       |  28 +
 tests/qtest/fuzz/general_fuzz.c               | 948 ++++++++++++++++++
 tests/qtest/fuzz/general_fuzz_configs.c       | 140 +++
 tests/qtest/fuzz/general_fuzz_configs.h       |  24 +
 tests/qtest/fuzz/meson.build                  |   1 +
 15 files changed, 1512 insertions(+)
 create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py
 create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py
 create mode 100644 tests/qtest/fuzz/general_fuzz.c
 create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c
 create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h

Comments

no-reply@patchew.org Oct. 15, 2020, 2:03 p.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20201015134137.205958-1-alxndr@bu.edu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201015134137.205958-1-alxndr@bu.edu
Subject: [PATCH v4 00/16] Add a General Virtual Device Fuzzer

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com -> patchew/1602767767-32713-1-git-send-email-mjrosato@linux.ibm.com
 * [new tag]         patchew/20201015134137.205958-1-alxndr@bu.edu -> patchew/20201015134137.205958-1-alxndr@bu.edu
Switched to a new branch 'test'
03323c2 scripts/oss-fuzz: remove the general-fuzz target
3c59bd5 fuzz: register predefined general-fuzz configs
0d80db3 fuzz: add general-fuzz configs for oss-fuzz
cea76d4 fuzz: add an "opaque" to the FuzzTarget struct
10cf373 fuzz: Add instructions for using general-fuzz
540fed2 scripts/oss-fuzz: Add crash trace minimization script
0550db0 scripts/oss-fuzz: Add script to reorder a general-fuzzer trace
106e830 fuzz: add a crossover function to generic-fuzzer
90837f5 fuzz: add a DISABLE_PCI op to general-fuzzer
adf044f fuzz: Add support for custom crossover functions
0851cb4 fuzz: Add fuzzer callbacks to DMA-read functions
4acf44b fuzz: Declare DMA Read callback function
86fec34 fuzz: Add DMA support to the generic-fuzzer
237e9d6 fuzz: Add PCI features to the general fuzzer
41e6d7b fuzz: Add general virtual-device fuzzer
ba68f35 memory: Add FlatView foreach function

=== OUTPUT BEGIN ===
1/16 Checking commit ba68f35ed358 (memory: Add FlatView foreach function)
2/16 Checking commit 41e6d7bb40d3 (fuzz: Add general virtual-device fuzzer)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#19: 
new file mode 100644

WARNING: line over 80 characters
#322: FILE: tests/qtest/fuzz/general_fuzz.c:299:
+ * Some commands can be variable-width, so we use a separator, SEPARATOR, to specify

WARNING: line over 80 characters
#330: FILE: tests/qtest/fuzz/general_fuzz.c:307:
+ *      Simply, by removing the first byte, we end up with a very different sequence:

total: 0 errors, 3 warnings, 516 lines checked

Patch 2/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/16 Checking commit 237e9d6e734d (fuzz: Add PCI features to the general fuzzer)
4/16 Checking commit 86fec343b37a (fuzz: Add DMA support to the generic-fuzzer)
ERROR: externs should be avoided in .c files
#84: FILE: tests/qtest/fuzz/general_fuzz.c:120:
+void fuzz_dma_read_cb(size_t addr, size_t len, MemoryRegion *mr, bool is_write);

WARNING: Block comments use a leading /* on a separate line
#109: FILE: tests/qtest/fuzz/general_fuzz.c:145:
+    /* Regions are assumed to support 1-4 byte accesses unless

WARNING: Block comments use * on subsequent lines
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+    /* Regions are assumed to support 1-4 byte accesses unless
+       otherwise specified.  */

WARNING: Block comments use a trailing */ on a separate line
#110: FILE: tests/qtest/fuzz/general_fuzz.c:146:
+       otherwise specified.  */

total: 1 errors, 3 warnings, 299 lines checked

Patch 4/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/16 Checking commit 4acf44ba5630 (fuzz: Declare DMA Read callback function)
6/16 Checking commit 0851cb4b787d (fuzz: Add fuzzer callbacks to DMA-read functions)
7/16 Checking commit adf044f2b9d2 (fuzz: Add support for custom crossover functions)
8/16 Checking commit 90837f5a8315 (fuzz: add a DISABLE_PCI op to general-fuzzer)
9/16 Checking commit 106e83015d43 (fuzz: add a crossover function to generic-fuzzer)
10/16 Checking commit 0550db09152f (scripts/oss-fuzz: Add script to reorder a general-fuzzer trace)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#22: 
new file mode 100755

WARNING: line over 80 characters
#117: FILE: scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py:91:
+                    sys.stderr.write("Warning: Likely double fetch on line {}.\n"

total: 0 errors, 2 warnings, 103 lines checked

Patch 10/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/16 Checking commit 540fed26dc40 (scripts/oss-fuzz: Add crash trace minimization script)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#17: 
new file mode 100755

total: 0 errors, 1 warnings, 157 lines checked

Patch 11/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/16 Checking commit 10cf3731768d (fuzz: Add instructions for using general-fuzz)
13/16 Checking commit cea76d423400 (fuzz: add an "opaque" to the FuzzTarget struct)
14/16 Checking commit 0d80db351b77 (fuzz: add general-fuzz configs for oss-fuzz)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#15: 
new file mode 100644

total: 0 errors, 1 warnings, 172 lines checked

Patch 14/16 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
15/16 Checking commit 3c59bd5b67df (fuzz: register predefined general-fuzz configs)
16/16 Checking commit 03323c23616b (scripts/oss-fuzz: remove the general-fuzz target)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201015134137.205958-1-alxndr@bu.edu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Paolo Bonzini Oct. 16, 2020, 1:33 p.m. UTC | #2
If you have to do a v5, the correct word is generic not general. :)

Otherwise looks great.

Paolo

On 15/10/20 15:41, Alexander Bulekov wrote:
> v4:

>     - Replace yaml + c template-based oss-fuzz configs, with C code to

>       register a FuzzTarget for each config (as suggested by Paolo)

>     - Replicate the functionality of address_space_write_rom to ensure

>       matching behavior when QTEST_LOG is enabled

>     - Improve code documentation/comments

>     - Small formatting changes

> v3:

> 	- Use flatviews to help select regions for fuzzing 

> 	- Meson-related changes

>     - Add some documentation

> 	- Improve minimalization script to trim write{bwlq} commands

> v2:

> 	- Remove QOS dependency.

> 	- Add a custom crossover function

> 	- Fix broken minimization scripts

> 	- Fixes to the IO region and DMA handling code

> 

> This is a general virtual-device fuzzer, designed to fuzz devices over Port IO,

> MMIO, and DMA.

> 

> To get started with this:

>  1. Build the fuzzers (see docs/devel/fuzzing.txt)

>     Note: Build with --enable-sanitizers, or create a "dictionary file":

>     echo kw1=\"FUZZ\" > dict

>     and pass it as an argument to libFuzzer with -dict=./dict

>     This magic value is a command separator that lets the fuzzer perform

>     multiple IO actions with a single input.

> 

>  2. Pick the qemu arguments you wish to fuzz:

>     export QEMU_FUZZ_ARGS="-M q35 -device virtio-balloon"

> 

>  3. Tell the fuzzer which QOM objects or MemoryRegion names to fuzz. I find the

>  "info qom-tree", "info qtree" and "info mtree" commands useful for identifying

>  these. Supports globbing. Here I will try to simultaneously fuzz(for no good

>  reason) virtio-balloon and e1000e, which is included by default in the q35:

>     export QEMU_FUZZ_OBJECTS='virtio* e1000*'

>     You can also try to fuzz the whole machine:

>     export QEMU_FUZZ_OBJECTS='*'

> 

>  4. Run the fuzzer for 0 inputs. The fuzzer should output a list of

>  MemoryRegions/PCI Devices it will try to fuzz. Confirm that these match your

>  expectations.

>     ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz -runs=0

> 

>  5. Run the fuzzer:

>     ./i386-softmmu/qemu-fuzz-i386 --fuzz-target=general-fuzz 

> 

> 

> Basically, at the core, this fuzzer is an interpreter that splits the input

> into a series of commands, such as mmio_write, pio_write, etc. We structure

> these commands to hit only MemoryRegions that are associated with the devices

> specified in QEMU_FUZZ_OBJECTS. Additionally, these patches add "hooks" to

> functions that are typically used by virtual-devices to read from RAM (DMA).

> These hooks attempt to populate these DMA regions with fuzzed data, just in

> time.

> 

> Some of the issues I have found or reproduced with this fuzzer:

> https://bugs.launchpad.net/bugs/1525123

> https://bugs.launchpad.net/bugs/1681439

> https://bugs.launchpad.net/bugs/1777315

> https://bugs.launchpad.net/bugs/1878034

> https://bugs.launchpad.net/bugs/1878043

> https://bugs.launchpad.net/bugs/1878054

> https://bugs.launchpad.net/bugs/1878057

> https://bugs.launchpad.net/bugs/1878067

> https://bugs.launchpad.net/bugs/1878134

> https://bugs.launchpad.net/bugs/1878136

> https://bugs.launchpad.net/bugs/1878253

> https://bugs.launchpad.net/bugs/1878255

> https://bugs.launchpad.net/bugs/1878259

> https://bugs.launchpad.net/bugs/1878263

> https://bugs.launchpad.net/bugs/1878323

> https://bugs.launchpad.net/bugs/1878641

> https://bugs.launchpad.net/bugs/1878642

> https://bugs.launchpad.net/bugs/1878645

> https://bugs.launchpad.net/bugs/1878651

> https://bugs.launchpad.net/bugs/1879223

> https://bugs.launchpad.net/bugs/1879227

> https://bugs.launchpad.net/bugs/1879531

> https://bugs.launchpad.net/bugs/1880355

> https://bugs.launchpad.net/bugs/1880539

> https://bugs.launchpad.net/bugs/1884693

> https://bugs.launchpad.net/bugs/1886362

> https://bugs.launchpad.net/bugs/1887303

> https://bugs.launchpad.net/bugs/1887309

> https://bugs.launchpad.net/bugs/697510

> 

> Alexander Bulekov (16):

>   memory: Add FlatView foreach function

>   fuzz: Add general virtual-device fuzzer

>   fuzz: Add PCI features to the general fuzzer

>   fuzz: Add DMA support to the generic-fuzzer

>   fuzz: Declare DMA Read callback function

>   fuzz: Add fuzzer callbacks to DMA-read functions

>   fuzz: Add support for custom crossover functions

>   fuzz: add a DISABLE_PCI op to general-fuzzer

>   fuzz: add a crossover function to generic-fuzzer

>   scripts/oss-fuzz: Add script to reorder a general-fuzzer trace

>   scripts/oss-fuzz: Add crash trace minimization script

>   fuzz: Add instructions for using general-fuzz

>   fuzz: add an "opaque" to the FuzzTarget struct

>   fuzz: add general-fuzz configs for oss-fuzz

>   fuzz: register predefined general-fuzz configs

>   scripts/oss-fuzz: remove the general-fuzz target

> 

>  docs/devel/fuzzing.txt                        |  39 +

>  include/exec/memory.h                         |  21 +

>  include/exec/memory_ldst_cached.h.inc         |   3 +

>  memory_ldst.c.inc                             |   4 +

>  scripts/oss-fuzz/build.sh                     |   6 +

>  scripts/oss-fuzz/minimize_qtest_trace.py      | 157 +++

>  .../oss-fuzz/reorder_fuzzer_qtest_trace.py    | 103 ++

>  softmmu/memory.c                              |  23 +

>  softmmu/physmem.c                             |   2 +

>  tests/qtest/fuzz/fuzz.c                       |  13 +

>  tests/qtest/fuzz/fuzz.h                       |  28 +

>  tests/qtest/fuzz/general_fuzz.c               | 948 ++++++++++++++++++

>  tests/qtest/fuzz/general_fuzz_configs.c       | 140 +++

>  tests/qtest/fuzz/general_fuzz_configs.h       |  24 +

>  tests/qtest/fuzz/meson.build                  |   1 +

>  15 files changed, 1512 insertions(+)

>  create mode 100755 scripts/oss-fuzz/minimize_qtest_trace.py

>  create mode 100755 scripts/oss-fuzz/reorder_fuzzer_qtest_trace.py

>  create mode 100644 tests/qtest/fuzz/general_fuzz.c

>  create mode 100644 tests/qtest/fuzz/general_fuzz_configs.c

>  create mode 100644 tests/qtest/fuzz/general_fuzz_configs.h

>