| Message ID | 20231212140043.714303-1-haokexin@gmail.com |
|---|---|
| State | Accepted |
| Commit | 4bbf0b6a64455c95586caf130e374586caef9986 |
| Headers | show |
| Series | docs: power: Adjust freezing-of-tasks.rst due to the freezer logic changes | expand |
On Tue, Dec 12, 2023 at 3:01 PM Kevin Hao <haokexin@gmail.com> wrote: > > Peter has rewritten core freezer logic in commit f5d39b020809 > ("freezer,sched: Rewrite core freezer logic"), adjust the > freezing-of-tasks.rst according to this commit. The main changes > include: > - Drop the mention of PF_FROZEN and PF_FREEZER_SKIP > - Introduce TASK_FROZEN, TASK_FREEZABLE and __TASK_FREEZABLE_UNSAFE > - Replace system_freezing_cnt with freezer_active > - Use another example for the loop of a freezable kernel thread since > the old codes are already gone > > Signed-off-by: Kevin Hao <haokexin@gmail.com> > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: "Rafael J. Wysocki" <rafael@kernel.org> > Cc: Pavel Machek <pavel@ucw.cz> > Cc: Len Brown <len.brown@intel.com> > --- > Documentation/power/freezing-of-tasks.rst | 83 +++++++++++++---------- > 1 file changed, 47 insertions(+), 36 deletions(-) > > diff --git a/Documentation/power/freezing-of-tasks.rst b/Documentation/power/freezing-of-tasks.rst > index 53b6a56c4635..8969ed244b20 100644 > --- a/Documentation/power/freezing-of-tasks.rst > +++ b/Documentation/power/freezing-of-tasks.rst > @@ -14,27 +14,28 @@ architectures). > II. How does it work? > ===================== > > -There are three per-task flags used for that, PF_NOFREEZE, PF_FROZEN > -and PF_FREEZER_SKIP (the last one is auxiliary). The tasks that have > -PF_NOFREEZE unset (all user space processes and some kernel threads) are > -regarded as 'freezable' and treated in a special way before the system enters a > -suspend state as well as before a hibernation image is created (in what follows > -we only consider hibernation, but the description also applies to suspend). > +There are one per-task flag (PF_NOFREEZE) and three per-task states > +(TASK_FROZEN, TASK_FREEZABLE and __TASK_FREEZABLE_UNSAFE) used for that. > +The tasks that have PF_NOFREEZE unset (all user space processes and some kernel > +threads) are regarded as 'freezable' and treated in a special way before the > +system enters a suspend state as well as before a hibernation image is created > +(in what follows we only consider hibernation, but the description also applies > +to suspend). > > Namely, as the first step of the hibernation procedure the function > freeze_processes() (defined in kernel/power/process.c) is called. A system-wide > -variable system_freezing_cnt (as opposed to a per-task flag) is used to indicate > -whether the system is to undergo a freezing operation. And freeze_processes() > -sets this variable. After this, it executes try_to_freeze_tasks() that sends a > -fake signal to all user space processes, and wakes up all the kernel threads. > -All freezable tasks must react to that by calling try_to_freeze(), which > -results in a call to __refrigerator() (defined in kernel/freezer.c), which sets > -the task's PF_FROZEN flag, changes its state to TASK_UNINTERRUPTIBLE and makes > -it loop until PF_FROZEN is cleared for it. Then, we say that the task is > -'frozen' and therefore the set of functions handling this mechanism is referred > -to as 'the freezer' (these functions are defined in kernel/power/process.c, > -kernel/freezer.c & include/linux/freezer.h). User space processes are generally > -frozen before kernel threads. > +static key freezer_active (as opposed to a per-task flag or state) is used to > +indicate whether the system is to undergo a freezing operation. And > +freeze_processes() sets this variable. After this, it executes > +try_to_freeze_tasks() that sends a fake signal to all user space processes, and > +wakes up all the kernel threads. All freezable tasks must react to that by > +calling try_to_freeze(), which results in a call to __refrigerator() (defined > +in kernel/freezer.c), which sets the task's state to TASK_FROZEN, and makes it > +loop until it is woken by an explicit TASK_FROZEN wakeup. Then, we say that the > +task is 'frozen' and therefore the set of functions handling this mechanism is > +referred to as 'the freezer' (these functions are defined in > +kernel/power/process.c, kernel/freezer.c & include/linux/freezer.h). User space > +processes are generally frozen before kernel threads. > > __refrigerator() must not be called directly. Instead, use the > try_to_freeze() function (defined in include/linux/freezer.h), that checks > @@ -43,31 +44,40 @@ if the task is to be frozen and makes the task enter __refrigerator(). > For user space processes try_to_freeze() is called automatically from the > signal-handling code, but the freezable kernel threads need to call it > explicitly in suitable places or use the wait_event_freezable() or > -wait_event_freezable_timeout() macros (defined in include/linux/freezer.h) > -that combine interruptible sleep with checking if the task is to be frozen and > -calling try_to_freeze(). The main loop of a freezable kernel thread may look > -like the following one:: > +wait_event_freezable_timeout() macros (defined in include/linux/wait.h) > +that puts the task to sleep (TASK_INTERRUPTIBLE) or frozen (TASK_FROZEN) if the > +task is freezing. The main loop of a freezable kernel thread may look like the > +following one:: > > set_freezable(); > - do { > - hub_events(); > - wait_event_freezable(khubd_wait, > - !list_empty(&hub_event_list) || > - kthread_should_stop()); > - } while (!kthread_should_stop() || !list_empty(&hub_event_list)); > > -(from drivers/usb/core/hub.c::hub_thread()). > + while (true) { > + struct task_struct *tsk = NULL; > > -If a freezable kernel thread fails to call try_to_freeze() after the freezer has > -initiated a freezing operation, the freezing of tasks will fail and the entire > -hibernation operation will be cancelled. For this reason, freezable kernel > -threads must call try_to_freeze() somewhere or use one of the > + wait_event_freezable(oom_reaper_wait, oom_reaper_list != NULL); > + spin_lock_irq(&oom_reaper_lock); > + if (oom_reaper_list != NULL) { > + tsk = oom_reaper_list; > + oom_reaper_list = tsk->oom_reaper_list; > + } > + spin_unlock_irq(&oom_reaper_lock); > + > + if (tsk) > + oom_reap_task(tsk); > + } > + > +(from mm/oom_kill.c::oom_reaper()). > + > +If a freezable kernel thread fails to be put to frozen state after the freezer > +has initiated a freezing operation, the freezing of tasks will fail and the > +entire hibernation operation will be cancelled. For this reason, freezable > +kernel threads must call try_to_freeze() somewhere or use one of the > wait_event_freezable() and wait_event_freezable_timeout() macros. > > After the system memory state has been restored from a hibernation image and > devices have been reinitialized, the function thaw_processes() is called in > -order to clear the PF_FROZEN flag for each frozen task. Then, the tasks that > -have been frozen leave __refrigerator() and continue running. > +order to wake up each frozen task. Then, the tasks that have been frozen leave > +__refrigerator() and continue running. > > > Rationale behind the functions dealing with freezing and thawing of tasks > @@ -96,7 +106,8 @@ III. Which kernel threads are freezable? > Kernel threads are not freezable by default. However, a kernel thread may clear > PF_NOFREEZE for itself by calling set_freezable() (the resetting of PF_NOFREEZE > directly is not allowed). From this point it is regarded as freezable > -and must call try_to_freeze() in a suitable place. > +and must call try_to_freeze() or variants of wait_event_freezable() in a > +suitable place. > > IV. Why do we do that? > ====================== > -- Applied as 6.8 material with some edits in the subject and changelog. I have also edited some new documentation language added by the patch for clarity etc. Thanks!
On Tue, Dec 19, 2023 at 09:16:34PM +0100, Rafael J. Wysocki wrote: > Applied as 6.8 material with some edits in the subject and changelog. > > I have also edited some new documentation language added by the patch > for clarity etc. Thanks Rafael. Kevin
diff --git a/Documentation/power/freezing-of-tasks.rst b/Documentation/power/freezing-of-tasks.rst index 53b6a56c4635..8969ed244b20 100644 --- a/Documentation/power/freezing-of-tasks.rst +++ b/Documentation/power/freezing-of-tasks.rst @@ -14,27 +14,28 @@ architectures). II. How does it work? ===================== -There are three per-task flags used for that, PF_NOFREEZE, PF_FROZEN -and PF_FREEZER_SKIP (the last one is auxiliary). The tasks that have -PF_NOFREEZE unset (all user space processes and some kernel threads) are -regarded as 'freezable' and treated in a special way before the system enters a -suspend state as well as before a hibernation image is created (in what follows -we only consider hibernation, but the description also applies to suspend). +There are one per-task flag (PF_NOFREEZE) and three per-task states +(TASK_FROZEN, TASK_FREEZABLE and __TASK_FREEZABLE_UNSAFE) used for that. +The tasks that have PF_NOFREEZE unset (all user space processes and some kernel +threads) are regarded as 'freezable' and treated in a special way before the +system enters a suspend state as well as before a hibernation image is created +(in what follows we only consider hibernation, but the description also applies +to suspend). Namely, as the first step of the hibernation procedure the function freeze_processes() (defined in kernel/power/process.c) is called. A system-wide -variable system_freezing_cnt (as opposed to a per-task flag) is used to indicate -whether the system is to undergo a freezing operation. And freeze_processes() -sets this variable. After this, it executes try_to_freeze_tasks() that sends a -fake signal to all user space processes, and wakes up all the kernel threads. -All freezable tasks must react to that by calling try_to_freeze(), which -results in a call to __refrigerator() (defined in kernel/freezer.c), which sets -the task's PF_FROZEN flag, changes its state to TASK_UNINTERRUPTIBLE and makes -it loop until PF_FROZEN is cleared for it. Then, we say that the task is -'frozen' and therefore the set of functions handling this mechanism is referred -to as 'the freezer' (these functions are defined in kernel/power/process.c, -kernel/freezer.c & include/linux/freezer.h). User space processes are generally -frozen before kernel threads. +static key freezer_active (as opposed to a per-task flag or state) is used to +indicate whether the system is to undergo a freezing operation. And +freeze_processes() sets this variable. After this, it executes +try_to_freeze_tasks() that sends a fake signal to all user space processes, and +wakes up all the kernel threads. All freezable tasks must react to that by +calling try_to_freeze(), which results in a call to __refrigerator() (defined +in kernel/freezer.c), which sets the task's state to TASK_FROZEN, and makes it +loop until it is woken by an explicit TASK_FROZEN wakeup. Then, we say that the +task is 'frozen' and therefore the set of functions handling this mechanism is +referred to as 'the freezer' (these functions are defined in +kernel/power/process.c, kernel/freezer.c & include/linux/freezer.h). User space +processes are generally frozen before kernel threads. __refrigerator() must not be called directly. Instead, use the try_to_freeze() function (defined in include/linux/freezer.h), that checks @@ -43,31 +44,40 @@ if the task is to be frozen and makes the task enter __refrigerator(). For user space processes try_to_freeze() is called automatically from the signal-handling code, but the freezable kernel threads need to call it explicitly in suitable places or use the wait_event_freezable() or -wait_event_freezable_timeout() macros (defined in include/linux/freezer.h) -that combine interruptible sleep with checking if the task is to be frozen and -calling try_to_freeze(). The main loop of a freezable kernel thread may look -like the following one:: +wait_event_freezable_timeout() macros (defined in include/linux/wait.h) +that puts the task to sleep (TASK_INTERRUPTIBLE) or frozen (TASK_FROZEN) if the +task is freezing. The main loop of a freezable kernel thread may look like the +following one:: set_freezable(); - do { - hub_events(); - wait_event_freezable(khubd_wait, - !list_empty(&hub_event_list) || - kthread_should_stop()); - } while (!kthread_should_stop() || !list_empty(&hub_event_list)); -(from drivers/usb/core/hub.c::hub_thread()). + while (true) { + struct task_struct *tsk = NULL; -If a freezable kernel thread fails to call try_to_freeze() after the freezer has -initiated a freezing operation, the freezing of tasks will fail and the entire -hibernation operation will be cancelled. For this reason, freezable kernel -threads must call try_to_freeze() somewhere or use one of the + wait_event_freezable(oom_reaper_wait, oom_reaper_list != NULL); + spin_lock_irq(&oom_reaper_lock); + if (oom_reaper_list != NULL) { + tsk = oom_reaper_list; + oom_reaper_list = tsk->oom_reaper_list; + } + spin_unlock_irq(&oom_reaper_lock); + + if (tsk) + oom_reap_task(tsk); + } + +(from mm/oom_kill.c::oom_reaper()). + +If a freezable kernel thread fails to be put to frozen state after the freezer +has initiated a freezing operation, the freezing of tasks will fail and the +entire hibernation operation will be cancelled. For this reason, freezable +kernel threads must call try_to_freeze() somewhere or use one of the wait_event_freezable() and wait_event_freezable_timeout() macros. After the system memory state has been restored from a hibernation image and devices have been reinitialized, the function thaw_processes() is called in -order to clear the PF_FROZEN flag for each frozen task. Then, the tasks that -have been frozen leave __refrigerator() and continue running. +order to wake up each frozen task. Then, the tasks that have been frozen leave +__refrigerator() and continue running. Rationale behind the functions dealing with freezing and thawing of tasks @@ -96,7 +106,8 @@ III. Which kernel threads are freezable? Kernel threads are not freezable by default. However, a kernel thread may clear PF_NOFREEZE for itself by calling set_freezable() (the resetting of PF_NOFREEZE directly is not allowed). From this point it is regarded as freezable -and must call try_to_freeze() in a suitable place. +and must call try_to_freeze() or variants of wait_event_freezable() in a +suitable place. IV. Why do we do that? ======================
Peter has rewritten core freezer logic in commit f5d39b020809 ("freezer,sched: Rewrite core freezer logic"), adjust the freezing-of-tasks.rst according to this commit. The main changes include: - Drop the mention of PF_FROZEN and PF_FREEZER_SKIP - Introduce TASK_FROZEN, TASK_FREEZABLE and __TASK_FREEZABLE_UNSAFE - Replace system_freezing_cnt with freezer_active - Use another example for the loop of a freezable kernel thread since the old codes are already gone Signed-off-by: Kevin Hao <haokexin@gmail.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Pavel Machek <pavel@ucw.cz> Cc: Len Brown <len.brown@intel.com> --- Documentation/power/freezing-of-tasks.rst | 83 +++++++++++++---------- 1 file changed, 47 insertions(+), 36 deletions(-)