| Message ID | 1596122076-341293-16-git-send-email-steven.sistare@oracle.com |
|---|---|
| State | New |
| Headers | show |
| Series | [V1,01/32] savevm: add vmstate handler iterators | expand |
On 9/11/2020 2:49 PM, Dr. David Alan Gilbert wrote: > * Steve Sistare (steven.sistare@oracle.com) wrote: >> For qemu upgrade and restart, we will re-exec() qemu with the same argv. >> However, qemu must start in a paused state and wait for the cprload command, >> and the original argv might not contain the -S option. To avoid modifying >> argv, provide the QEMU_START_FREEZE environment variable. If >> QEMU_START_FREEZE is set, then set autostart=0, like the -S option. >> >> Signed-off-by: Steve Sistare <steven.sistare@oracle.com> > > What's wrong with modifying the argv? > > Note, also the trick -incoming defer uses; the whole point here is that > we start qemu with -incoming defer and then we can issue commands > to modify the QEMU configuration before we actually reload state. > > Note, even without CPR there might be reasons that you need to modify > the argv; for example, imagine that since it was originally booted > someone had hotplug added an extra CPU or RAM or a disk; the new QEMU > must be started in a state that reflects the state in which the VM was > at the point when it was saved, not the point at which it was started > long ago. The code is simpler if we do not need to parse and massage the argv, and that is sufficient for many use cases. QEMU_START_FREEZE adds only a few lines of code, and it's nice to have that choice. For hot plug, we rely on the management layer to know what devices were plugged after the initial startup, and re-plug them after restart. cprsave restarts qemu, which creates command-line devices. At this point the manager would send the hotplug commands (just like -incoming defer), then send cprload. Having said that, if the management layer sometimes performs live migration, and sometimes performs cpr restart, then we need to strip out any -incoming args from argv before restart. This can be done in the vendor-specific qemu-exec helper (patch 20). - Steve >> --- >> softmmu/vl.c | 5 +++++ >> 1 file changed, 5 insertions(+) >> >> diff --git a/softmmu/vl.c b/softmmu/vl.c >> index 951994f..7016e39 100644 >> --- a/softmmu/vl.c >> +++ b/softmmu/vl.c >> @@ -4501,6 +4501,11 @@ void qemu_init(int argc, char **argv, char **envp) >> exit(0); >> } >> >> + if (getenv("QEMU_START_FREEZE")) { >> + unsetenv("QEMU_START_FREEZE"); >> + autostart = 0; >> + } >> + >> if (incoming) { >> Error *local_err = NULL; >> qemu_start_incoming_migration(incoming, &local_err); >> -- >> 1.8.3.1 >>
* Steven Sistare (steven.sistare@oracle.com) wrote: > On 9/11/2020 2:49 PM, Dr. David Alan Gilbert wrote: > > * Steve Sistare (steven.sistare@oracle.com) wrote: > >> For qemu upgrade and restart, we will re-exec() qemu with the same argv. > >> However, qemu must start in a paused state and wait for the cprload command, > >> and the original argv might not contain the -S option. To avoid modifying > >> argv, provide the QEMU_START_FREEZE environment variable. If > >> QEMU_START_FREEZE is set, then set autostart=0, like the -S option. > >> > >> Signed-off-by: Steve Sistare <steven.sistare@oracle.com> > > > > What's wrong with modifying the argv? > > > > Note, also the trick -incoming defer uses; the whole point here is that > > we start qemu with -incoming defer and then we can issue commands > > to modify the QEMU configuration before we actually reload state. > > > > Note, even without CPR there might be reasons that you need to modify > > the argv; for example, imagine that since it was originally booted > > someone had hotplug added an extra CPU or RAM or a disk; the new QEMU > > must be started in a state that reflects the state in which the VM was > > at the point when it was saved, not the point at which it was started > > long ago. > > The code is simpler if we do not need to parse and massage the argv, and that is > sufficient for many use cases. QEMU_START_FREEZE adds only a few lines of code, and > it's nice to have that choice. > > For hot plug, we rely on the management layer to know what devices were plugged > after the initial startup, and re-plug them after restart. cprsave restarts qemu, > which creates command-line devices. At this point the manager would send the hotplug > commands (just like -incoming defer), then send cprload. > > Having said that, if the management layer sometimes performs live migration, and sometimes > performs cpr restart, then we need to strip out any -incoming args from argv before restart. > This can be done in the vendor-specific qemu-exec helper (patch 20). My problem is I can see a whole bunch of places that reusing the original argv breaks, so I don't think this is a useful general solution: a) The -incoming example b) The management app has to reply the hotplug sequence c) ...even if it did there's no guarantee that the original pre-hotplug commandline works: i) e.g. an original block device file was deleted ii) One of the endpoints for a network device is gone. Any part of (c) could cause the exec'd qemu to fail before it gets as far as allowing you to issue the hotplug commands. It's also plain dangerous, since the exec'd qemu shouldn't be accessing a file or device that has been hot-unplugged and might now be part of a different VM. So I think you really should pass another command line option here rather than setting an environment variable; and then I think you should consider two separate things: a) You could easily strip out options of the form --cpr-freeze b) Consider something more general; e.g. allow the management layer to specify a new set of argv to be used by the exec. Dave > - Steve > > >> --- > >> softmmu/vl.c | 5 +++++ > >> 1 file changed, 5 insertions(+) > >> > >> diff --git a/softmmu/vl.c b/softmmu/vl.c > >> index 951994f..7016e39 100644 > >> --- a/softmmu/vl.c > >> +++ b/softmmu/vl.c > >> @@ -4501,6 +4501,11 @@ void qemu_init(int argc, char **argv, char **envp) > >> exit(0); > >> } > >> > >> + if (getenv("QEMU_START_FREEZE")) { > >> + unsetenv("QEMU_START_FREEZE"); > >> + autostart = 0; > >> + } > >> + > >> if (incoming) { > >> Error *local_err = NULL; > >> qemu_start_incoming_migration(incoming, &local_err); > >> -- > >> 1.8.3.1 > >> >
diff --git a/softmmu/vl.c b/softmmu/vl.c index 951994f..7016e39 100644 --- a/softmmu/vl.c +++ b/softmmu/vl.c @@ -4501,6 +4501,11 @@ void qemu_init(int argc, char **argv, char **envp) exit(0); } + if (getenv("QEMU_START_FREEZE")) { + unsetenv("QEMU_START_FREEZE"); + autostart = 0; + } + if (incoming) { Error *local_err = NULL; qemu_start_incoming_migration(incoming, &local_err);
For qemu upgrade and restart, we will re-exec() qemu with the same argv. However, qemu must start in a paused state and wait for the cprload command, and the original argv might not contain the -S option. To avoid modifying argv, provide the QEMU_START_FREEZE environment variable. If QEMU_START_FREEZE is set, then set autostart=0, like the -S option. Signed-off-by: Steve Sistare <steven.sistare@oracle.com> --- softmmu/vl.c | 5 +++++ 1 file changed, 5 insertions(+)