0dfd96ad52
oom-reaper and process_mrelease system call should protect against races with exit_mmap which can destroy page tables while they walk the VMA tree. oom-reaper protects from that race by setting MMF_OOM_VICTIM and by relying on exit_mmap to set MMF_OOM_SKIP before taking and releasing mmap_write_lock. process_mrelease has to elevate mm->mm_users to prevent such race. Both oom-reaper and process_mrelease hold mmap_read_lock when walking the VMA tree. The locking rules and mechanisms could be simpler if exit_mmap takes mmap_write_lock while executing destructive operations such as free_pgtables. Change exit_mmap to hold the mmap_write_lock when calling free_pgtables. Operations like unmap_vmas() and unlock_range() are not destructive and could run under mmap_read_lock but for simplicity we take one mmap_write_lock during almost the entire operation. Note also that because oom-reaper checks VM_LOCKED flag, unlock_range() should not be allowed to race with it. In most cases this lock should be uncontended. Previously, Kirill reported ~4% regression caused by a similar change [1]. We reran the same test and although the individual results are quite noisy, the percentiles show lower regression with 1.6% being the worst case [2]. The change allows oom-reaper and process_mrelease to execute safely under mmap_read_lock without worries that exit_mmap might destroy page tables from under them. [1] https://lore.kernel.org/all/20170725141723.ivukwhddk2voyhuc@node.shutemov.name/ [2] https://lore.kernel.org/all/CAJuCfpGC9-c9P40x7oy=jy5SphMcd0o0G_6U1-+JAziGKG6dGA@mail.gmail.com/ Signed-off-by: Suren Baghdasaryan <surenb@google.com> Link: https://lore.kernel.org/all/20211124235906.14437-1-surenb@google.com/ Bug: 130172058 Bug: 189803002 Change-Id: Ic87272d09a0b68a1b0e968e8f1a1510fd6fc776a Git-commit: 28358ebf2adb31117893813992fefcfd359a6a16 Git-repo: https://android.googlesource.com/kernel/common/ [quic_gkohli@quicinc.com: Resolved cherry-pick conflict in mm/mmap.c due to mmap lock was implemented differently in older kernel, and Although process_mrelease is not applicable in older kernel, but this patch is required to take exclusive lock in exit_mmap path so that SPF knows an isolated vma was freed from this path] Signed-off-by: Gaurav Kohli <quic_gkohli@quicinc.com> Signed-off-by: Srinivasarao Pathipati <quic_c_spathi@quicinc.com> |
||
---|---|---|
.. | ||
kasan | ||
backing-dev.c | ||
balloon_compaction.c | ||
bootmem.c | ||
cleancache.c | ||
cma.c | ||
cma.h | ||
cma_debug.c | ||
compaction.c | ||
debug.c | ||
debug_page_ref.c | ||
dmapool.c | ||
early_ioremap.c | ||
fadvise.c | ||
failslab.c | ||
filemap.c | ||
frame_vector.c | ||
frontswap.c | ||
gup.c | ||
gup_benchmark.c | ||
highmem.c | ||
hmm.c | ||
huge_memory.c | ||
hugetlb.c | ||
hugetlb_cgroup.c | ||
hwpoison-inject.c | ||
init-mm.c | ||
internal.h | ||
interval_tree.c | ||
Kconfig | ||
Kconfig.debug | ||
khugepaged.c | ||
kmemleak-test.c | ||
kmemleak.c | ||
ksm.c | ||
list_lru.c | ||
maccess.c | ||
madvise.c | ||
Makefile | ||
memblock.c | ||
memcontrol.c | ||
memfd.c | ||
memory-failure.c | ||
memory.c | ||
memory_hotplug.c | ||
mempolicy.c | ||
mempool.c | ||
memtest.c | ||
migrate.c | ||
mincore.c | ||
mlock.c | ||
mm_event.c | ||
mm_init.c | ||
mmap.c | ||
mmu_context.c | ||
mmu_notifier.c | ||
mmzone.c | ||
mprotect.c | ||
mremap.c | ||
msync.c | ||
nobootmem.c | ||
nommu.c | ||
oom_kill.c | ||
page-writeback.c | ||
page_alloc.c | ||
page_counter.c | ||
page_ext.c | ||
page_idle.c | ||
page_io.c | ||
page_isolation.c | ||
page_owner.c | ||
page_poison.c | ||
page_vma_mapped.c | ||
pagewalk.c | ||
percpu-internal.h | ||
percpu-km.c | ||
percpu-stats.c | ||
percpu-vm.c | ||
percpu.c | ||
pgtable-generic.c | ||
process_reclaim.c | ||
process_vm_access.c | ||
quicklist.c | ||
readahead.c | ||
rmap.c | ||
rodata_test.c | ||
shmem.c | ||
showmem.c | ||
slab.c | ||
slab.h | ||
slab_common.c | ||
slob.c | ||
slub.c | ||
sparse-vmemmap.c | ||
sparse.c | ||
swap.c | ||
swap_cgroup.c | ||
swap_ratio.c | ||
swap_slots.c | ||
swap_state.c | ||
swapfile.c | ||
truncate.c | ||
usercopy.c | ||
userfaultfd.c | ||
util.c | ||
vmacache.c | ||
vmalloc.c | ||
vmpressure.c | ||
vmscan.c | ||
vmstat.c | ||
workingset.c | ||
z3fold.c | ||
zbud.c | ||
zpool.c | ||
zsmalloc.c | ||
zswap.c |