| Message ID | 20190519203726.20729-1-richard.henderson@linaro.org |
|---|---|
| Headers | show |
| Series | linux-user: Split do_syscall | expand |
On 19/05/2019 22:36, Richard Henderson wrote: > Based-on: <20190519201953.20161-1-richard.henderson@linaro.org> > Aka "linux-user: path, clone, sparc, shmat fixes" > > Version 6 was > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html > > Since v5, I've fixed the ioctl failure that Laurent found, and > have done a few more syscalls. > > I've tried to do more testing with LTP, but it's a really annoying > testsuite. There are at least 3 tests that hang forever (before > and after this patch set), and the host kernel's OOM handler kills > my login session, ending the test run. This seems to be different > behaviour than I was seeing in January, when I did have test runs > that completed successfully. > > OTOH, looking back at the v6 cover letter, I now see that I was > testing armhf and i386 as guests, whereas this time I was trying > aarch64 and ppc64 as guests. 32-bit guests unable to trigger the > host OOM killer? > > Laurent, is there anything special that you're doing to see these > tests run to completion? I'm running them in a container ('unshare chroot') and aarch64 needs a patch that hasn't been merged: "[PATCH] linux-user: check valid address in access_ok()" https://patchew.org/QEMU/20190208173520.15007-1-remi@remlab.net/ I have a skipfile that contains: futex_wait03 mmap13 mmap16 msgctl10 msgctl11 ptrace01 ptrace02 ptrace03 ptrace05 remap_file_pages01 remap_file_pages02 sendmsg01 fork13 fork14 creat07 mremap03 but I didn't update it for a while. I will test as soon as possible your series in my environment. Thanks, Laurent
On Sun, 19 May 2019 at 21:39, Richard Henderson <richard.henderson@linaro.org> wrote: > > Based-on: <20190519201953.20161-1-richard.henderson@linaro.org> > Aka "linux-user: path, clone, sparc, shmat fixes" > > Version 6 was > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html > > Since v5, I've fixed the ioctl failure that Laurent found, and > have done a few more syscalls. > > I've tried to do more testing with LTP, but it's a really annoying > testsuite. There are at least 3 tests that hang forever (before > and after this patch set), and the host kernel's OOM handler kills > my login session, ending the test run. This seems to be different > behaviour than I was seeing in January, when I did have test runs > that completed successfully. If these are different from the ones we're already skipping in the skiplist documented at https://wiki.qemu.org/Testing/LTP we should update that wiki page to also skip these new hanging-tests (and whatever it is that eats all the memory). thanks -- PMM
On May 20, 2019 11:43 AM, "Peter Maydell" <peter.maydell@linaro.org> wrote: > > On Sun, 19 May 2019 at 21:39, Richard Henderson > <richard.henderson@linaro.org> wrote: > > > > Based-on: <20190519201953.20161-1-richard.henderson@linaro.org> > > Aka "linux-user: path, clone, sparc, shmat fixes" > > > > Version 6 was > > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html > > > > Since v5, I've fixed the ioctl failure that Laurent found, and > > have done a few more syscalls. > > > > I've tried to do more testing with LTP, but it's a really annoying > > testsuite. There are at least 3 tests that hang forever (before > > and after this patch set), and the host kernel's OOM handler kills > > my login session, ending the test run. This seems to be different > > behaviour than I was seeing in January, when I did have test runs > > that completed successfully. > > If these are different from the ones we're already skipping > in the skiplist documented at https://wiki.qemu.org/Testing/LTP > we should update that wiki page to also skip these new > hanging-tests (and whatever it is that eats all the memory). > Peter, yes, it is definitely different. It has been a moving target lately. May I suggest that we use a stable version of QEMU, and a stable version of LTP, and create a skiplist for such stable configuration, include it in the docs, explicitely noting used versions? We can continue doing that for each QEMU release and whatever the latest LTP would be at the moment of that QEMU release (maybe even keeping the previous skiplists - which could be useful for debugging). Regards, Aleksandar > thanks > -- PMM >
On May 19, 2019 10:38 PM, "Richard Henderson" <richard.henderson@linaro.org> wrote: > > Based-on: <20190519201953.20161-1-richard.henderson@linaro.org> > Aka "linux-user: path, clone, sparc, shmat fixes" > > Version 6 was > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html > > Since v5, I've fixed the ioctl failure that Laurent found, and > have done a few more syscalls. > > I've tried to do more testing with LTP, but it's a really annoying > testsuite. There are at least 3 tests that hang forever (before > and after this patch set), and the host kernel's OOM handler kills > my login session, ending the test run. This seems to be different > behaviour than I was seeing in January, when I did have test runs > that completed successfully. > > OTOH, looking back at the v6 cover letter, I now see that I was > testing armhf and i386 as guests, whereas this time I was trying > aarch64 and ppc64 as guests. 32-bit guests unable to trigger the > host OOM killer? > > Laurent, is there anything special that you're doing to see these > tests run to completion? > On surface, this series looks to be a refactoring series. However, this is not the case: burried in various patches, are removals of some features visible to the end user. Can you please list all such cases in the cover letter, so that they can be easier identified and evaluated by reviewers? Thanks, Aleksandar > > r~ > > > Richard Henderson (74): > linux-user: Setup split syscall infrastructure > linux-user: Split out open, open_at > linux-user: Share more code for open and openat > linux-user: Tidy do_openat loop over fakes > linux-user: Split out readlink, readlinkat > linux-user: Split out close > linux-user: Split out read, write > linux-user: Reduce regpairs_aligned & target_offset64 ifdefs > linux-user: Split out readv, writev > linux-user: Split out pread64, pwrite64 > linux-user: Split out preadv, pwritev > linux-user: Split out name_to_handle_at, open_by_handle_at > linux-user: Split out ipc syscalls > linux-user: Split out memory syscalls > linux-user: Split out exit > linux-user: Split out brk > linux-user: Split out clone, fork, vfork > linux-user: Split out wait4, waitid, waitpid > linux-user: Implement rusage argument to waitid > linux-user: Split out creat > linux-user: Split out link, linkat > linux-user: Split out unlink, unlinkat, rmdir > linux-user: Split out execve > linux-user: Implement execveat > linux-user: Split out chdir > linux-user: Split out time > linux-user: Split out mknod, mknodat > linux-user: Split out chmod, fchmod, fchmodat > linux-user: Split out lseek, llseek > linux-user: Split out getpid, getppid, getxpid > linux-user: Split out mount > linux-user: Split out umount, umount2 > linux-user: Split out stime > linux-user: Split out alarm, pause > linux-user: Split out utime, utimes, futimesat > linux-user: Split out access, faccessat > linux-user: Split out nice > linux-user: Split out sync, syncfs > linux-user: Split out kill > linux-user: Split out rename, renameat, renameat2 > linux-user: Split out mkdir, mkdirat > linux-user: Split out dup, dup2, dup3 > linux-user: Split out pipe, pipe2 > linux-user: Split out times > linux-user: Split out acct > linux-user: Move syscall_init to the end > linux-user: Split out ioctl > linux-user: Fix types in ioctl logging > linux-user: Remove sentinel from ioctl_entries > linux-user: Split out fcntl, fcntl64 > linux-user: Split out setpgid > linux-user: Split out umask > linux-user: Split out chroot > linux-user: Split out getpgid, getpgrp > linux-user: Split out getsid, setsid > linux-user: Split out sigaction, rt_sigaction > linux-user: Split out sgetmask, ssetmask > linux-user: Split out sigprocmask, rt_sigprocmask > linux-user: Split out sigpending, rt_sigpending > linux-user: Split out sigsuspend, rt_sigsuspend > linux-user: Split out rt_sigtimedwait > linux-user: Split out rt_sigqueueinfo, rt_tgsigqueueinfo > linux-user: Split out sigreturn, rt_sigreturn > linux-user: Split out gethostname, sethostname > linux-user: Split out getrlimit, setrlimit > linux-user: Split out getrusage > linux-user: Split out gettimeofday, settimeofday > linux-user: Split out select, _newselect > linux-user: Split out pselect6 > linux-user: Split out symlink, symlinkat > linux-user: Split out swapon, swapoff > linux-user: Split out reboot > linux-user: Split out truncate, truncate64, ftruncate, ftruncate64 > linux-user: Split out getpriority, setpriority > > linux-user/syscall-defs.h | 338 +++ > linux-user/syscall.h | 103 + > linux-user/strace.c | 1569 +++------- > linux-user/syscall-fcntl.inc.c | 322 ++ > linux-user/syscall-file.inc.c | 1526 ++++++++++ > linux-user/syscall-ioctl.inc.c | 873 ++++++ > linux-user/syscall-ipc.inc.c | 1088 +++++++ > linux-user/syscall-mem.inc.c | 244 ++ > linux-user/syscall-proc.inc.c | 758 +++++ > linux-user/syscall-sig.inc.c | 549 ++++ > linux-user/syscall-time.inc.c | 77 + > linux-user/syscall.c | 5045 ++------------------------------ > configure | 20 - > linux-user/strace.list | 414 --- > 14 files changed, 6512 insertions(+), 6414 deletions(-) > create mode 100644 linux-user/syscall-defs.h > create mode 100644 linux-user/syscall.h > create mode 100644 linux-user/syscall-fcntl.inc.c > create mode 100644 linux-user/syscall-file.inc.c > create mode 100644 linux-user/syscall-ioctl.inc.c > create mode 100644 linux-user/syscall-ipc.inc.c > create mode 100644 linux-user/syscall-mem.inc.c > create mode 100644 linux-user/syscall-proc.inc.c > create mode 100644 linux-user/syscall-sig.inc.c > create mode 100644 linux-user/syscall-time.inc.c > > -- > 2.17.1 > >
[Adding the mailing list back in.] On 5/22/19 2:47 AM, Aleksandar Markovic wrote: >> > Version 6 was >> > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html >> > >> > Since v5, I've fixed the ioctl failure that Laurent found, and >> > have done a few more syscalls. >> > >> > I've tried to do more testing with LTP.... > > Running LTP is certainly a very important way of testing this series. However, > it is not sufficient. This is because this series completely replaces internal > QEMU strace mechanism for involved system calls, What you missed because of the very large time lag between iterations, and because I was lazy and only referenced v6 in this cover letter, is that in an early revision the major objection was that I *wasn't* replacing strace, and thus we were retaining lots of duplicate logic in multiple places. > and any bug in such > replacement would not be captured by LTP results. How did you test new strace > bits and peaces? Can you provide logs of such tests (for example, before/after > comparisons of strace output for relevant system calls, or similar)? My strace testing is totally ad-hoc. Run something with -strace and eyeball it. Here are before/after logs for busybox ls (aka linux-user-test-0.3, which has for some reason vanished from the qemu wiki?) for alpha-linux-user. 657 uname(0x40008014b8) = 0 657 brk(NULL) = 0x000000012006a000 657 access("/etc/ld.so.nohwcap",F_OK) = -1 errno=2 (No such file or directory) 657 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x000000400082e000 657 access("/etc/ld.so.preload",R_OK) = -1 errno=2 (No such file or directory) 657 open("/etc/ld.so.cache",O_RDONLY) = 3 657 fstat64(3,0x0000004000800c30) = 0 657 close(3) = 0 657 open("/lib/tls/libc.so.6.1",O_RDONLY) = -1 errno=2 (No such file or directory) 657 stat64("/lib/tls",0x0000004000800bb0) = -1 errno=2 (No such file or directory) 657 open("/lib/libc.so.6.1",O_RDONLY) = 3 657 read(3,0x800da8,640) = 640 657 fstat64(3,0x0000004000800c30) = 0 657 mmap(NULL,927824,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE,3,0) = 0x0000004000830000 657 mprotect(0x00000040008f2000,133200,PROT_NONE) = 0 657 mmap(0x0000004000902000,65536,PROT_EXEC|PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0xc2000) = 0x0000004000902000 657 mmap(0x0000004000912000,2128,PROT_EXEC|PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x0000004000912000 657 close(3) = 0 657 mprotect(0x000000400082a000,8192,PROT_READ) = 0 657 getxuid(274886302098,4832112143,77,24,4831859744,274886356864) = 1000 657 getxgid(274886302098,4832112143,77,0,1000,274886356864) = 1000 657 setgid(1000,4832112143,77,0,1000,274886356864) = 0 657 setuid(1000,4832112143,77,0,1000,274886356864) = 0 657 ioctl(1,1074295912,274886301072,0,1000,274886356864) = 0 657 lstat64("dummyfile",0x0000004000801510) = 0 657 brk(NULL) = 0x000000012006a000 657 brk(0x000000012008c000) = 0x000000012008c000 657 open("/etc/localtime",O_RDONLY) = 3 657 fstat64(3,0x0000004000801360) = 0 657 fstat64(3,0x00000040008011f0) = 0 657 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000004000914000 657 read(3,0x914000,4096) = 3545 657 close(3) = 0 657 munmap(0x0000004000914000,8192) = 0 657 fstat64(1,0x0000004000800c70) = 0 657 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000004000916000 657 open("/etc/passwd",O_RDONLY) = 3 657 fstat64(3,0x0000004000801290) = 0 657 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000004000918000 657 read(3,0x918000,4096) = 2727 657 close(3) = 0 657 munmap(0x0000004000918000,8192) = 0 657 open("/etc/group",O_RDONLY) = 3 657 fstat64(3,0x0000004000801290) = 0 657 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x000000400091a000 657 read(3,0x91a000,4096) = 1043 657 close(3) = 0 657 munmap(0x000000400091a000,8192) = 0 657 gettimeofday(274886300912,0,1,0,103,1024) = 0 657 write(1,0x916000,66) = 66 657 munmap(0x0000004000916000,8192) = 0 657 exit_group(0) 17862 uname(0x40008014b8) = 0 17862 brk(NULL) = 0x000000012006a000 17862 access("/etc/ld.so.nohwcap", F_OK) = -1 errno=2 (No such file or directory) 17862 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x000000400082e000 17862 access("/etc/ld.so.preload", R_OK) = -1 errno=2 (No such file or directory) 17862 open("/etc/ld.so.cache", O_RDONLY, 01) = 3 17862 fstat64(3,0x0000004000800c30) = 0 17862 close(3) = 0 17862 open("/lib/tls/libc.so.6.1", O_RDONLY, 014) = -1 errno=2 (No such file or directory) 17862 stat64("/lib/tls",0x0000004000800bb0) = -1 errno=2 (No such file or directory) 17862 open("/lib/libc.so.6.1", O_RDONLY, 014) = 3 17862 read(3, 0x0000004000800da8, 640) = 640 17862 fstat64(3,0x0000004000800c30) = 0 17862 mmap(NULL, 927824, PROT_EXEC|PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x0000004000830000 17862 mprotect(0x00000040008f2000, 133200, PROT_NONE) = 0 17862 mmap(0x0000004000902000, 65536, PROT_EXEC|PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED, 3, 794624) = 0x0000004000902000 17862 mmap(0x0000004000912000, 2128, PROT_EXEC|PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, -1, 0) = 0x0000004000912000 17862 close(3) = 0 17862 mprotect(0x000000400082a000, 8192, PROT_READ) = 0 17862 getxuid(274886302098,4832112143,77,24,4831859744,274886356864) = 1000 17862 getxgid(274886302098,4832112143,77,0,1000,274886356864) = 1000 17862 setgid(1000,4832112143,77,0,1000,274886356864) = 0 17862 setuid(1000,4832112143,77,0,1000,274886356864) = 0 17862 ioctl(1, 40087468) = 0 17862 lstat64("dummyfile",0x0000004000801510) = 0 17862 brk(NULL) = 0x000000012006a000 17862 brk(0x000000012008c000) = 0x000000012008c000 17862 open("/etc/localtime", O_RDONLY, 0666) = 3 17862 fstat64(3,0x0000004000801360) = 0 17862 fstat64(3,0x00000040008011f0) = 0 17862 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x0000004000914000 17862 read(3, 0x0000004000914000, 4096) = 3545 17862 close(3) = 0 17862 munmap(0x0000004000914000, 8192) = 0 17862 fstat64(1,0x0000004000800c70) = 0 17862 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x0000004000916000 17862 open("/etc/passwd", O_RDONLY, 0666) = 3 17862 fstat64(3,0x0000004000801290) = 0 17862 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x0000004000918000 17862 read(3, 0x0000004000918000, 4096) = 2727 17862 close(3) = 0 17862 munmap(0x0000004000918000, 8192) = 0 17862 open("/etc/group", O_RDONLY, 0666) = 3 17862 fstat64(3,0x0000004000801290) = 0 17862 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x000000400091a000 17862 read(3, 0x000000400091a000, 4096) = 1043 17862 close(3) = 0 17862 munmap(0x000000400091a000, 8192) = 0 17862 gettimeofday(0x00000040008014f0) = 0 17862 write(1, 0x0000004000916000, 66) = 66 17862 munmap(0x0000004000916000, 8192) = 0 17862 exit_group(0)