| Message ID | 20221116102659.70287-9-david@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | mm/gup: remove FOLL_FORCE usage from drivers (reliable R/O long-term pinning) | expand |
On 11/16/22 11:26, David Hildenbrand wrote: > Extend FAULT_FLAG_UNSHARE to break COW on anything mapped into a > COW (i.e., private writable) mapping and adjust the documentation > accordingly. > > FAULT_FLAG_UNSHARE will now also break COW when encountering the shared > zeropage, a pagecache page, a PFNMAP, ... inside a COW mapping, by > properly replacing the mapped page/pfn by a private copy (an exclusive > anonymous page). > > Note that only do_wp_page() needs care: hugetlb_wp() already handles > FAULT_FLAG_UNSHARE correctly. wp_huge_pmd()/wp_huge_pud() also handles it > correctly, for example, splitting the huge zeropage on FAULT_FLAG_UNSHARE > such that we can handle FAULT_FLAG_UNSHARE on the PTE level. > > This change is a requirement for reliable long-term R/O pinning in > COW mappings. > > Signed-off-by: David Hildenbrand <david@redhat.com> Reviewed-by: Vlastimil Babka <vbabka@suse.cz> > --- > include/linux/mm_types.h | 8 ++++---- > mm/memory.c | 4 ---- > 2 files changed, 4 insertions(+), 8 deletions(-) > > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index 5e7f4fac1e78..5e9aaad8c7b2 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -1037,9 +1037,9 @@ typedef struct { > * @FAULT_FLAG_REMOTE: The fault is not for current task/mm. > * @FAULT_FLAG_INSTRUCTION: The fault was during an instruction fetch. > * @FAULT_FLAG_INTERRUPTIBLE: The fault can be interrupted by non-fatal signals. > - * @FAULT_FLAG_UNSHARE: The fault is an unsharing request to unshare (and mark > - * exclusive) a possibly shared anonymous page that is > - * mapped R/O. > + * @FAULT_FLAG_UNSHARE: The fault is an unsharing request to break COW in a > + * COW mapping, making sure that an exclusive anon page is > + * mapped after the fault. > * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. > * We should only access orig_pte if this flag set. > * > @@ -1064,7 +1064,7 @@ typedef struct { > * > * The combination FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE is illegal. > * FAULT_FLAG_UNSHARE is ignored and treated like an ordinary read fault when > - * no existing R/O-mapped anonymous page is encountered. > + * applied to mappings that are not COW mappings. > */ > enum fault_flag { > FAULT_FLAG_WRITE = 1 << 0, > diff --git a/mm/memory.c b/mm/memory.c > index d47ad33c6487..56b21ab1e4d2 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3432,10 +3432,6 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) > } > wp_page_reuse(vmf); > return 0; > - } else if (unshare) { > - /* No anonymous page -> nothing to do. */ > - pte_unmap_unlock(vmf->pte, vmf->ptl); > - return 0; > } > copy: > /*
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5e7f4fac1e78..5e9aaad8c7b2 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1037,9 +1037,9 @@ typedef struct { * @FAULT_FLAG_REMOTE: The fault is not for current task/mm. * @FAULT_FLAG_INSTRUCTION: The fault was during an instruction fetch. * @FAULT_FLAG_INTERRUPTIBLE: The fault can be interrupted by non-fatal signals. - * @FAULT_FLAG_UNSHARE: The fault is an unsharing request to unshare (and mark - * exclusive) a possibly shared anonymous page that is - * mapped R/O. + * @FAULT_FLAG_UNSHARE: The fault is an unsharing request to break COW in a + * COW mapping, making sure that an exclusive anon page is + * mapped after the fault. * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. * We should only access orig_pte if this flag set. * @@ -1064,7 +1064,7 @@ typedef struct { * * The combination FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE is illegal. * FAULT_FLAG_UNSHARE is ignored and treated like an ordinary read fault when - * no existing R/O-mapped anonymous page is encountered. + * applied to mappings that are not COW mappings. */ enum fault_flag { FAULT_FLAG_WRITE = 1 << 0, diff --git a/mm/memory.c b/mm/memory.c index d47ad33c6487..56b21ab1e4d2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3432,10 +3432,6 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) } wp_page_reuse(vmf); return 0; - } else if (unshare) { - /* No anonymous page -> nothing to do. */ - pte_unmap_unlock(vmf->pte, vmf->ptl); - return 0; } copy: /*
Extend FAULT_FLAG_UNSHARE to break COW on anything mapped into a COW (i.e., private writable) mapping and adjust the documentation accordingly. FAULT_FLAG_UNSHARE will now also break COW when encountering the shared zeropage, a pagecache page, a PFNMAP, ... inside a COW mapping, by properly replacing the mapped page/pfn by a private copy (an exclusive anonymous page). Note that only do_wp_page() needs care: hugetlb_wp() already handles FAULT_FLAG_UNSHARE correctly. wp_huge_pmd()/wp_huge_pud() also handles it correctly, for example, splitting the huge zeropage on FAULT_FLAG_UNSHARE such that we can handle FAULT_FLAG_UNSHARE on the PTE level. This change is a requirement for reliable long-term R/O pinning in COW mappings. Signed-off-by: David Hildenbrand <david@redhat.com> --- include/linux/mm_types.h | 8 ++++---- mm/memory.c | 4 ---- 2 files changed, 4 insertions(+), 8 deletions(-)