| Message ID | 20240930200831.1669010-1-adhemerval.zanella@linaro.org |
|---|---|
| Headers | show |
| Series | Add support for memory sealing | expand |
On Mon, Sep 30, 2024 at 1:08 PM Adhemerval Zanella <adhemerval.zanella@linaro.org> wrote: > > The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the > mseal syscall that allows blocking some memory operations on the VMA > range: > > * Unmapping, moving to another location, extending or shrinking the > size, munmap, and mremap. > * Moving or expanding a different VMA into the current location, via > mremap. > * Modifying the memory range with mmap along with flag MAP_FIXED. > * Expanding the size with mremap. > * Change the protection flags with mprotect or pkey_mprotect. > * Destructive behaviors on anonymous memory, such as madvice with > MADV_DONTNEED. > > Memory sealing is useful as a hardening mechanism to avoid either > remapping the memory segments or changing the memory protection segments > layout by the dynamic loader (for instance, the RELRO hardening). A > similar hardening is done by OpenBSD with the mimmutable syscall [1]. > > The sealing is an opt-in security feature that requires a new GNU > property GNU_PROPERTY_MEMORY_SEAL to indicate that the ELF module > supports and should use memory sealing if the loader supports it. > Previous versions [2] had the sealing as an opt-out feature, however, it > has some drawbacks where the backport is not straightforward, there is > no clear semantic if memory sealing is a hint or requirement, some > programs bypass the loader to apply relocation themselves and are > incompatible with an opt-out feature [3], and it deviates from how other > security hardening was added on Linux ecosystem (such as RELRO and > non-executable stacks). > > A GNU property is used instead of a new dynamic section tag (like the > one proposed for DT_GNU_FLAGS_1) because the memory sealing should be > selectable for ET_EXEC and not only for ET_DYN. It also fits new opt-in > security features like x86 CET or AArch64 BTI. > > The first patch adds the mseal support for Linux. Although most > programs will not use it directly, some specific ones, like Chrome, > intend to use it. > > The second and third patches are requirements to enable memory sealing > to work on executables, where they add gnu property parsing on the > loader and static binaries. > > The fourth patch moves 'call_init_paths' after gnu attribute parsing, so > the loader can seal the rtld_malloc pages (since they are meant to be > immutable over process execution). > > The fifth patch propagates the RTLD_NODELETE flag in case of dlopen. It > will be used to extend memory sealing for the object dependencies. > > The sixth patch adds the memory sealing supports in multiple places > where the page is supposed to be immutable over program execution: > * All shared library dependencies from the binary, including the > read-only segments after PT_GNU_RELRO setup. > * The binary itself, including dynamic and static links. In both cases, > it is up either to binary or the loader to set up the sealing. > * Any preload libraries. > * Any library loaded with dlopen with RTLD_NODELETE flag (including > libgcc.so loaded to enable process unwind and thread cancellation). > * Audit modules. > * The loader bump allocator. > > The seventh patch makes glibc enable memory sealing as default if the > linker supports the option (-Wl,memory-seal). A new configure option, > --disable-default-memory-seal, disable it. > > The eighth patch adds memory sealing tests, they are enabled if the > linker supports it. > > The last patch adds a new tunable, glibc.rtld.seal, which can be used to > enforce memory sealing even if the programs or dependencies do not have > the GNU_PROPERTY_MEMORY_SEAL. The tunable accepts two different values: > > * '0': where loaders follow the GNU_PROPERTY_MEMORY_SEAL attribute if > * present. This is the default and no sealing would be applied if the > * object does not have the memory sealing attribute. > > * '1': where sealing is enforced even if the object does not have the > * GNU_PROPERTY_MEMORY_SEAL. Also, any syscall failure on memory sealing > * aborts the programs. > > This patchset does not delay RELRO activation until after their ELF > constructors have been executed, as suggested on the previous RFC for > mseal support. It is not strictly required, and it requires extensive > changes on_dl_start_user to either make _dl_init call RELRO/sealing > setup > after ctor/initarray is done, or call it after _dl_init. There is also > the > question of whether to apply RELRO/sealing per module after > dtor/initarray or in bulk after _dt_init. > > I tested on both x86_64-linux-gnu and aarch64-linux-gnu with Linux > 6.11, along with some testing on a powerpc64le-linux-gnu VM. > I have tested the binutil and glibc patches in a Debian VM. My testing confirms that the functionality aligns with the descriptions provided in the patch series. As reference: here is how the mapping looks like, with the default configuration of binutil and glibc. For example: #test_glibc_sealed.out => sealed #\_ lib4.so dlopen(lib4.so, RT_LAZY|RTLD_NODELETE) => sealed #| \_ lib4_1.so => sealed. #\_ lib1.so => sealed #\_ libc.so => sealed #\_ ld.so => sealed. test_glibc_sealed.out [557a3d5db000-557a3d5dc000 ], [rd mr mw me sl ] [/root/workdir3/test2/test_glibc_sealed.out] [557a3d5dc000-557a3d5dd000 ], [rd ex mr mw me sl ] [/root/workdir3/test2/test_glibc_sealed.out] [557a3d5dd000-557a3d5de000 ], [rd mr mw me sl ] [/root/workdir3/test2/test_glibc_sealed.out] [557a3d5de000-557a3d5df000 ], [rd mr mw me ac sl ] [/root/workdir3/test2/test_glibc_sealed.out] [557a3d5df000-557a3d5e0000 ], [rd wr mr mw me ac sl] [/root/workdir3/test2/test_glibc_sealed.out] [557a49b0a000-557a49b2b000 ], [rd wr mr mw me ac ] [[heap]] [7f98f6690000-7f98f6691000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib4_1.so] [7f98f6691000-7f98f6692000 ], [rd ex mr mw me sl ] [/root/workdir3/test2/lib4_1.so] [7f98f6692000-7f98f6693000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib4_1.so] [7f98f6693000-7f98f6694000 ], [rd mr mw me ac sl ] [/root/workdir3/test2/lib4_1.so] [7f98f6694000-7f98f6695000 ], [rd wr mr mw me ac sl] [/root/workdir3/test2/lib4_1.so] [7f98f6695000-7f98f6696000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib4.so] [7f98f6696000-7f98f6697000 ], [rd ex mr mw me sl ] [/root/workdir3/test2/lib4.so] [7f98f6697000-7f98f6698000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib4.so] [7f98f6698000-7f98f6699000 ], [rd mr mw me ac sl ] [/root/workdir3/test2/lib4.so] [7f98f6699000-7f98f669a000 ], [rd wr mr mw me ac sl] [/root/workdir3/test2/lib4.so] [7f98f669a000-7f98f669d000 ], [rd wr mr mw me ac sl] [[anonymous]] [7f98f669d000-7f98f66c1000 ], [rd mr mw me sl ] [/root/workdir3/target/lib/libc.so.6] [7f98f66c1000-7f98f6824000 ], [rd ex mr mw me sl ] [/root/workdir3/target/lib/libc.so.6] [7f98f6824000-7f98f687a000 ], [rd mr mw me sl ] [/root/workdir3/target/lib/libc.so.6] [7f98f687a000-7f98f687e000 ], [rd mr mw me ac sl ] [/root/workdir3/target/lib/libc.so.6] [7f98f687e000-7f98f6880000 ], [rd wr mr mw me ac sl] [/root/workdir3/target/lib/libc.so.6] [7f98f6880000-7f98f688d000 ], [rd wr mr mw me ac sl] [[anonymous]] [7f98f688d000-7f98f688e000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib1.so] [7f98f688e000-7f98f688f000 ], [rd ex mr mw me sl ] [/root/workdir3/test2/lib1.so] [7f98f688f000-7f98f6890000 ], [rd mr mw me sl ] [/root/workdir3/test2/lib1.so] [7f98f6890000-7f98f6891000 ], [rd mr mw me ac sl ] [/root/workdir3/test2/lib1.so] [7f98f6891000-7f98f6892000 ], [rd wr mr mw me ac sl] [/root/workdir3/test2/lib1.so] [7f98f6892000-7f98f6894000 ], [rd wr mr mw me ac sl] [[anonymous]] [7f98f6894000-7f98f6898000 ], [rd mr pf io de dd ] [[vvar]] [7f98f6898000-7f98f689a000 ], [rd ex mr mw me de ] [[vdso]] [7f98f689a000-7f98f689b000 ], [rd mr mw me sl ] [/root/workdir3/target/lib/ld-linux-x86-64.so.2] [7f98f689b000-7f98f68c3000 ], [rd ex mr mw me sl ] [/root/workdir3/target/lib/ld-linux-x86-64.so.2] [7f98f68c3000-7f98f68ce000 ], [rd mr mw me sl ] [/root/workdir3/target/lib/ld-linux-x86-64.so.2] [7f98f68ce000-7f98f68d0000 ], [rd mr mw me ac sl ] [/root/workdir3/target/lib/ld-linux-x86-64.so.2] [7f98f68d0000-7f98f68d2000 ], [rd wr mr mw me ac sl] [/root/workdir3/target/lib/ld-linux-x86-64.so.2] [7fff9c392000-7fff9c3b3000 ], [rd wr mr mw me gd ac] [[stack]] [ffffffffff600000-ffffffffff601000], [ex ] [[vsyscall]] Except vvar/vdso/stack/heap/vsyscall, all mappings are sealed. (The vvar/vdso/stack/vsyscall will be sealed by kernel, which I am working on) -Jeff > [1] https://man.openbsd.org/mimmutable.2 > [2] https://sourceware.org/pipermail/libc-alpha/2024-August/158836.html > [3] https://glandium.org/blog/?p=4297 > > Changes v2->v3: > * Make the option opt-int instead of opt-out. > > Adhemerval Zanella (9): > linux: Add mseal syscall support > elf: Parse gnu properties for static linked binaries > elf: Parse gnu properties for the loader > rtld: Move call_init_paths after _dl_process_pt_gnu_property > elf: Use RTLD_NODELETE for dependencies > elf: Add support to memory sealing > Enable memory sealing automatically > linux: Add memory sealing tests > elf: Add glibc.rtld.seal tunable > > INSTALL | 5 + > Makeconfig | 17 ++ > Makerules | 2 + > NEWS | 20 ++ > configure | 57 ++++ > configure.ac | 19 ++ > elf/Makefile | 1 + > elf/dl-load.c | 7 + > elf/dl-map-segments.h | 6 + > elf/dl-minimal-malloc.c | 3 + > elf/dl-mseal-mode.h | 28 ++ > elf/dl-open.c | 7 +- > elf/dl-reloc.c | 64 ++++ > elf/dl-support.c | 22 ++ > elf/dl-tunables.list | 6 + > elf/elf.h | 2 + > elf/rtld.c | 27 +- > elf/setup-vdso.h | 2 + > elf/tst-rtld-list-tunables.exp | 1 + > include/link.h | 8 + > manual/install.texi | 5 + > manual/memory.texi | 66 +++++ > manual/tunables.texi | 35 +++ > sysdeps/aarch64/dl-prop.h | 5 + > sysdeps/generic/dl-mseal.h | 23 ++ > sysdeps/generic/dl-prop-mseal.h | 36 +++ > sysdeps/generic/dl-prop.h | 5 + > sysdeps/generic/ldsodefs.h | 14 + > sysdeps/unix/sysv/linux/Makefile | 107 +++++++ > sysdeps/unix/sysv/linux/Versions | 1 + > sysdeps/unix/sysv/linux/aarch64/libc.abilist | 1 + > sysdeps/unix/sysv/linux/alpha/libc.abilist | 1 + > sysdeps/unix/sysv/linux/arc/libc.abilist | 1 + > sysdeps/unix/sysv/linux/arm/be/libc.abilist | 1 + > sysdeps/unix/sysv/linux/arm/le/libc.abilist | 1 + > sysdeps/unix/sysv/linux/bits/mman-shared.h | 8 + > sysdeps/unix/sysv/linux/csky/libc.abilist | 1 + > sysdeps/unix/sysv/linux/dl-mseal.c | 48 +++ > sysdeps/unix/sysv/linux/dl-mseal.h | 27 ++ > sysdeps/unix/sysv/linux/hppa/libc.abilist | 1 + > sysdeps/unix/sysv/linux/i386/libc.abilist | 1 + > sysdeps/unix/sysv/linux/kernel-features.h | 8 + > .../sysv/linux/loongarch/lp64/libc.abilist | 1 + > .../sysv/linux/m68k/coldfire/libc.abilist | 1 + > .../unix/sysv/linux/m68k/m680x0/libc.abilist | 1 + > .../sysv/linux/microblaze/be/libc.abilist | 1 + > .../sysv/linux/microblaze/le/libc.abilist | 1 + > .../sysv/linux/mips/mips32/fpu/libc.abilist | 1 + > .../sysv/linux/mips/mips64/n32/libc.abilist | 1 + > .../sysv/linux/mips/mips64/n64/libc.abilist | 1 + > sysdeps/unix/sysv/linux/nios2/libc.abilist | 1 + > sysdeps/unix/sysv/linux/or1k/libc.abilist | 1 + > .../linux/powerpc/powerpc32/fpu/libc.abilist | 1 + > .../powerpc/powerpc32/nofpu/libc.abilist | 1 + > .../linux/powerpc/powerpc64/be/libc.abilist | 1 + > .../linux/powerpc/powerpc64/le/libc.abilist | 1 + > .../unix/sysv/linux/riscv/rv32/libc.abilist | 1 + > .../unix/sysv/linux/riscv/rv64/libc.abilist | 1 + > .../unix/sysv/linux/s390/s390-32/libc.abilist | 1 + > .../unix/sysv/linux/s390/s390-64/libc.abilist | 1 + > sysdeps/unix/sysv/linux/sh/be/libc.abilist | 1 + > sysdeps/unix/sysv/linux/sh/le/libc.abilist | 1 + > .../sysv/linux/sparc/sparc32/libc.abilist | 1 + > .../sysv/linux/sparc/sparc64/libc.abilist | 1 + > sysdeps/unix/sysv/linux/syscalls.list | 1 + > .../sysv/linux/tst-dl_mseal-auditmod-noseal.c | 1 + > .../unix/sysv/linux/tst-dl_mseal-auditmod.c | 23 ++ > .../unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c | 19 ++ > .../unix/sysv/linux/tst-dl_mseal-dlopen-1.c | 19 ++ > .../linux/tst-dl_mseal-dlopen-2-1-noseal.c | 19 ++ > .../unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c | 19 ++ > .../sysv/linux/tst-dl_mseal-dlopen-2-noseal.c | 19 ++ > .../unix/sysv/linux/tst-dl_mseal-dlopen-2.c | 19 ++ > .../sysv/linux/tst-dl_mseal-mod-1-noseal.c | 19 ++ > sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c | 19 ++ > .../sysv/linux/tst-dl_mseal-mod-2-noseal.c | 19 ++ > sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c | 19 ++ > sysdeps/unix/sysv/linux/tst-dl_mseal-noseal.c | 74 +++++ > .../sysv/linux/tst-dl_mseal-preload-noseal.c | 1 + > .../unix/sysv/linux/tst-dl_mseal-preload.c | 19 ++ > .../unix/sysv/linux/tst-dl_mseal-skeleton.c | 278 ++++++++++++++++++ > .../sysv/linux/tst-dl_mseal-static-noseal.c | 45 +++ > sysdeps/unix/sysv/linux/tst-dl_mseal-static.c | 42 +++ > .../unix/sysv/linux/tst-dl_mseal-tunable.c | 76 +++++ > sysdeps/unix/sysv/linux/tst-dl_mseal.c | 72 +++++ > sysdeps/unix/sysv/linux/tst-mseal.c | 67 +++++ > .../unix/sysv/linux/x86_64/64/libc.abilist | 1 + > .../unix/sysv/linux/x86_64/x32/libc.abilist | 1 + > sysdeps/x86/dl-prop.h | 4 + > 89 files changed, 1611 insertions(+), 6 deletions(-) > create mode 100644 elf/dl-mseal-mode.h > create mode 100644 sysdeps/generic/dl-mseal.h > create mode 100644 sysdeps/generic/dl-prop-mseal.h > create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.c > create mode 100644 sysdeps/unix/sysv/linux/dl-mseal.h > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-auditmod-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-auditmod.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1-1.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-1.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2-1-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2-1.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-dlopen-2.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-1.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-mod-2.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-preload-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-preload.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-skeleton.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static-noseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-static.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal-tunable.c > create mode 100644 sysdeps/unix/sysv/linux/tst-dl_mseal.c > create mode 100644 sysdeps/unix/sysv/linux/tst-mseal.c > > -- > 2.34.1 >
On Tue, Oct 1, 2024 at 9:21 PM Jeff Xu <jeffxu@google.com> wrote: > On Mon, Sep 30, 2024 at 1:08 PM Adhemerval Zanella > <adhemerval.zanella@linaro.org> wrote: > > > > The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the > > mseal syscall that allows blocking some memory operations on the VMA > > range: > > > This is a very nice & neccesary improvement.. what is the status of this patchset?
On 31/10/24 12:39, Cristian Rodríguez wrote: > > > On Tue, Oct 1, 2024 at 9:21 PM Jeff Xu <jeffxu@google.com <mailto:jeffxu@google.com>> wrote: > > On Mon, Sep 30, 2024 at 1:08 PM Adhemerval Zanella > <adhemerval.zanella@linaro.org <mailto:adhemerval.zanella@linaro.org>> wrote: > > > > The Linux 6.10 (8be7258aad44b5e25977a98db136f677fa6f4370) added the > > mseal syscall that allows blocking some memory operations on the VMA > > range: > > > > This is a very nice & neccesary improvement.. what is the status of this patchset? I still need to sort out the binutils patchset, H.J has asked to re-do my initial approach of how to merge .gnu.attributes. The glibc patch did not had much review though.
On Mon, 30 Sep 2024, Adhemerval Zanella <adhemerval.zanella@linaro.org> wrote: [...] > The sixth patch adds the memory sealing supports in multiple places > where the page is supposed to be immutable over program execution: > * All shared library dependencies from the binary, including the > read-only segments after PT_GNU_RELRO setup. > * The binary itself, including dynamic and static links. In both cases, > it is up either to binary or the loader to set up the sealing. > * Any preload libraries. > * Any library loaded with dlopen with RTLD_NODELETE flag (including > libgcc.so loaded to enable process unwind and thread cancellation). > * Audit modules. > * The loader bump allocator. I have a concern of applying mseal(2) to whole executable regions on a per-module granularity. There are cases where some executable regions might want to not be sealed by glibc or not sealed at all. For example, I'm porting the Linux static keys [0] to user-space. This can be used for optimizing branches where the conditions are expected to not change often or at all. For examples: - Static tracepoints - Load time configurations - Enabling runtime assertions in production - Enabling loggings A template for static key would look like this at the assembly level: program Out of line section inline asm goto jmp a -> a: nop load test cond branch l_yes (asm goto) fallthrough to l_no l_no: no callbacks l_yes: callbacks The idea would be to patch the `nop' at label `a' to branch directly to `l_no' or `l_yes'. If code patching is not possible (e.g. due to hardening), then a load/test/cond-branch is executed. The level of indirection introduced by the out of line section separates the program from the code patching. This allows to keep the memory seal on the program without impacting the static key patching opportunity. I see two different patching requirements. For load-time configuration, the patching is done once and then the seal can be applied on the out of line section by the application itself. For tracepoints, mseal should not be applied because tracepoints can be enabled/disabled during the execution of the program. It is possible to bypass the mseal protection using `/proc/self/mem' However this comes with some downsides. First, a file descriptor is required and all the concerns that it implies. Second, accesses to `/proc/self/mem' are made with calls to `access_remote_vm()' which are basically memcpys from/to user-space to/from kernel-space. Thus, nothing atomic. This imposes restriction on the type of patching that can be done since we want to avoid a "stop the world" approach, i.e. we want to do asynchronous cross-modifying code without stopping threads in the program. There are other cases outside of static keys. What about patchable function entries (-fpatchable-function-entry) or profiling (-mfentry, -mnop-count, -minstrument-return)? Should users use the tunable to disable mseal on program when instrumentation/profiling is wanted? Or maybe we ought to implement something similar to static keys, i.e. a level of indirection where mseal is relaxed for allowing patching. Maybe mseal is not the good kind of protection for that and other hardening mechanisms are better suited to prevent patchable code? [...] Thanks, Olivier [0] https://www.kernel.org/doc/html/latest/staging/static-keys.html