6718389514
* refs/heads/tmp-a483478: UPSTREAM: arm64: vdso: Build vDSO with -ffixed-x18 Revert "drm/dsi: Fix byte order of DCS set/get brightness" Reverting below patches from android-4.19-stable.125 Linux 4.19.125 rxrpc: Fix ack discard rxrpc: Trace discarded ACKs iio: adc: stm32-dfsdm: fix device used to request dma iio: adc: stm32-dfsdm: Use dma_request_chan() instead dma_request_slave_channel() iio: adc: stm32-adc: fix device used to request dma iio: adc: stm32-adc: Use dma_request_chan() instead dma_request_slave_channel() x86/unwind/orc: Fix unwind_get_return_address_ptr() for inactive tasks rxrpc: Fix a memory leak in rxkad_verify_response() rapidio: fix an error in get_user_pages_fast() error handling ipack: tpci200: fix error return code in tpci200_register() mei: release me_cl object reference misc: rtsx: Add short delay after exit from ASPM iio: dac: vf610: Fix an error handling path in 'vf610_dac_probe()' iio: sca3000: Remove an erroneous 'get_device()' staging: greybus: Fix uninitialized scalar variable staging: iio: ad2s1210: Fix SPI reading Revert "gfs2: Don't demote a glock until its revokes are written" brcmfmac: abort and release host after error tty: serial: qcom_geni_serial: Fix wrap around of TX buffer cxgb4/cxgb4vf: Fix mac_hlist initialization and free cxgb4: free mac_hlist properly net: bcmgenet: abort suspend on error net: bcmgenet: code movement Revert "net/ibmvnic: Fix EOI when running in XIVE mode" media: fdp1: Fix R-Car M3-N naming in debug message thunderbolt: Drop duplicated get_switch_at_route() staging: most: core: replace strcpy() by strscpy() libnvdimm/btt: Fix LBA masking during 'free list' population libnvdimm/btt: Remove unnecessary code in btt_freelist_init nfit: Add Hyper-V NVDIMM DSM command set to white list powerpc/64s: Disable STRICT_KERNEL_RWX powerpc: Remove STRICT_KERNEL_RWX incompatibility with RELOCATABLE drm/i915/gvt: Init DPLL/DDI vreg for virtual display instead of inheritance. dmaengine: owl: Use correct lock in owl_dma_get_pchan() dmaengine: tegra210-adma: Fix an error handling path in 'tegra_adma_probe()' apparmor: Fix aa_label refcnt leak in policy_update apparmor: fix potential label refcnt leak in aa_change_profile apparmor: Fix use-after-free in aa_audit_rule_init drm/etnaviv: fix perfmon domain interation ALSA: hda/realtek - Add more fixup entries for Clevo machines ALSA: hda/realtek - Fix silent output on Gigabyte X570 Aorus Xtreme ALSA: pcm: fix incorrect hw_base increase ALSA: iec1712: Initialize STDSP24 properly when using the model=staudio option padata: purge get_cpu and reorder_via_wq from padata_do_serial padata: initialize pd->cpu with effective cpumask padata: Replace delayed timer with immediate workqueue in padata_reorder ARM: futex: Address build warning platform/x86: asus-nb-wmi: Do not load on Asus T100TA and T200TA USB: core: Fix misleading driver bug report stmmac: fix pointer check after utilization in stmmac_interrupt ceph: fix double unlock in handle_cap_export() HID: quirks: Add HID_QUIRK_NO_INIT_REPORTS quirk for Dell K12A keyboard-dock gtp: set NLM_F_MULTI flag in gtp_genl_dump_pdp() x86/apic: Move TSC deadline timer debug printk HID: i2c-hid: reset Synaptics SYNA2393 on resume scsi: ibmvscsi: Fix WARN_ON during event pool release component: Silence bind error on -EPROBE_DEFER aquantia: Fix the media type of AQC100 ethernet controller in the driver vhost/vsock: fix packet delivery order to monitoring devices configfs: fix config_item refcnt leak in configfs_rmdir() scsi: qla2xxx: Delete all sessions before unregister local nvme port scsi: qla2xxx: Fix hang when issuing nvme disconnect-all in NPIV HID: alps: ALPS_1657 is too specific; use U1_UNICORN_LEGACY instead HID: alps: Add AUI1657 device ID HID: multitouch: add eGalaxTouch P80H84 support gcc-common.h: Update for GCC 10 ubi: Fix seq_file usage in detailed_erase_block_info debugfs file i2c: mux: demux-pinctrl: Fix an error handling path in 'i2c_demux_pinctrl_probe()' iommu/amd: Fix over-read of ACPI UID from IVRS table ubifs: remove broken lazytime support fix multiplication overflow in copy_fdtable() mtd: spinand: Propagate ECC information to the MTD structure ima: Fix return value of ima_write_policy() evm: Check also if *tfm is an error pointer in init_desc() ima: Set file->f_mode instead of file->f_flags in ima_calc_file_hash() riscv: set max_pfn to the PFN of the last page KVM: SVM: Fix potential memory leak in svm_cpu_init() i2c: dev: Fix the race between the release of i2c_dev and cdev ubsan: build ubsan.c more conservatively x86/uaccess, ubsan: Fix UBSAN vs. SMAP ANDROID: scsi: ufs: Handle clocks when lrbp fails ANDROID: fscrypt: handle direct I/O with IV_INO_LBLK_32 BACKPORT: FROMLIST: fscrypt: add support for IV_INO_LBLK_32 policies ANDROID: Update the ABI xml and qcom whitelist ANDROID: Fix build.config.gki-debug Linux 4.19.124 Makefile: disallow data races on gcc-10 as well KVM: x86: Fix off-by-one error in kvm_vcpu_ioctl_x86_setup_mce ARM: dts: r8a7740: Add missing extal2 to CPG node arm64: dts: renesas: r8a77980: Fix IPMMU VIP[01] nodes ARM: dts: r8a73a4: Add missing CMT1 interrupts arm64: dts: rockchip: Rename dwc3 device nodes on rk3399 to make dtc happy arm64: dts: rockchip: Replace RK805 PMIC node name with "pmic" on rk3328 boards clk: Unlink clock if failed to prepare or enable Revert "ALSA: hda/realtek: Fix pop noise on ALC225" usb: gadget: legacy: fix error return code in cdc_bind() usb: gadget: legacy: fix error return code in gncm_bind() usb: gadget: audio: Fix a missing error return value in audio_bind() usb: gadget: net2272: Fix a memory leak in an error handling path in 'net2272_plat_probe()' dwc3: Remove check for HWO flag in dwc3_gadget_ep_reclaim_trb_sg() clk: rockchip: fix incorrect configuration of rk3228 aclk_gpu* clocks exec: Move would_dump into flush_old_exec x86/unwind/orc: Fix error handling in __unwind_start() x86: Fix early boot crash on gcc-10, third try cifs: fix leaked reference on requeued write ARM: dts: imx27-phytec-phycard-s-rdk: Fix the I2C1 pinctrl entries ARM: dts: dra7: Fix bus_dma_limit for PCIe usb: xhci: Fix NULL pointer dereference when enqueuing trbs from urb sg list USB: gadget: fix illegal array access in binding with UDC usb: host: xhci-plat: keep runtime active when removing host usb: core: hub: limit HUB_QUIRK_DISABLE_AUTOSUSPEND to USB5534B ALSA: usb-audio: Add control message quirk delay for Kingston HyperX headset ALSA: rawmidi: Fix racy buffer resize under concurrent accesses ALSA: hda/realtek - Limit int mic boost for Thinkpad T530 gcc-10: avoid shadowing standard library 'free()' in crypto gcc-10: disable 'restrict' warning for now gcc-10: disable 'stringop-overflow' warning for now gcc-10: disable 'array-bounds' warning for now gcc-10: disable 'zero-length-bounds' warning for now Stop the ad-hoc games with -Wno-maybe-initialized kbuild: compute false-positive -Wmaybe-uninitialized cases in Kconfig gcc-10 warnings: fix low-hanging fruit pnp: Use list_for_each_entry() instead of open coding hwmon: (da9052) Synchronize access with mfd IB/mlx4: Test return value of calls to ib_get_cached_pkey netfilter: nft_set_rbtree: Introduce and use nft_rbtree_interval_start() arm64: fix the flush_icache_range arguments in machine_kexec netfilter: conntrack: avoid gcc-10 zero-length-bounds warning NFSv4: Fix fscache cookie aux_data to ensure change_attr is included nfs: fscache: use timespec64 in inode auxdata NFS: Fix fscache super_cookie index_key from changing after umount mmc: block: Fix request completion in the CQE timeout path mmc: core: Check request type before completing the request i40iw: Fix error handling in i40iw_manage_arp_cache() pinctrl: cherryview: Add missing spinlock usage in chv_gpio_irq_handler pinctrl: baytrail: Enable pin configuration setting for GPIO chip gfs2: Another gfs2_walk_metadata fix ALSA: hda/realtek - Fix S3 pop noise on Dell Wyse ipc/util.c: sysvipc_find_ipc() incorrectly updates position index drm/qxl: lost qxl_bo_kunmap_atomic_page in qxl_image_init_helper() ALSA: hda/hdmi: fix race in monitor detection during probe cpufreq: intel_pstate: Only mention the BIOS disabling turbo mode once dmaengine: mmp_tdma: Reset channel error on release dmaengine: pch_dma.c: Avoid data race between probe and irq handler riscv: fix vdso build with lld tcp: fix SO_RCVLOWAT hangs with fat skbs net: tcp: fix rx timestamp behavior for tcp_recvmsg netprio_cgroup: Fix unlimited memory leak of v2 cgroups net: ipv4: really enforce backoff for redirects net: dsa: loop: Add module soft dependency hinic: fix a bug of ndo_stop virtio_net: fix lockdep warning on 32 bit tcp: fix error recovery in tcp_zerocopy_receive() Revert "ipv6: add mtu lock check in __ip6_rt_update_pmtu" pppoe: only process PADT targeted at local interfaces net: phy: fix aneg restart in phy_ethtool_set_eee netlabel: cope with NULL catmap net: fix a potential recursive NETDEV_FEAT_CHANGE mmc: sdhci-acpi: Add SDHCI_QUIRK2_BROKEN_64_BIT_DMA for AMDI0040 scsi: sg: add sg_remove_request in sg_write virtio-blk: handle block_device_operations callbacks after hot unplug drop_monitor: work around gcc-10 stringop-overflow warning net: moxa: Fix a potential double 'free_irq()' net/sonic: Fix a resource leak in an error handling path in 'jazz_sonic_probe()' shmem: fix possible deadlocks on shmlock_user_lock net: dsa: Do not make user port errors fatal ANDROID: rtc: class: call hctosys in resource managed registration ANDROID: GKI: Update the ABI xml and whitelist ANDROID: power_supply: Add RTX power-supply property f2fs: flush dirty meta pages when flushing them f2fs: fix checkpoint=disable:%u%% f2fs: rework filename handling f2fs: split f2fs_d_compare() from f2fs_match_name() f2fs: don't leak filename in f2fs_try_convert_inline_dir() ANDROID: clang: update to 11.0.1 FROMLIST: x86_64: fix jiffies ODR violation ANDROID: arm64: vdso: Fix removing SCS flags ANDROID: GKI: Update the ABI xml and whitelist ANDROID: Incremental fs: wake up log pollers less often ANDROID: Incremental fs: Fix scheduling while atomic error ANDROID: Incremental fs: Avoid continually recalculating hashes ANDROID: export: Disable symbol trimming on modules ANDROID: GKI: Update the ABI xml and whitelist ANDROID: fscrypt: set dun_bytes more precisely ANDROID: dm-default-key: set dun_bytes more precisely ANDROID: block: backport the ability to specify max_dun_bytes ANDROID: Revert "ANDROID: GKI: gki_defconfig: CONFIG_DM_DEFAULT_KEY=m" Linux 4.19.123 ipc/mqueue.c: change __do_notify() to bypass check_kill_permission() scripts/decodecode: fix trapping instruction formatting objtool: Fix stack offset tracking for indirect CFAs netfilter: nf_osf: avoid passing pointer to local var netfilter: nat: never update the UDP checksum when it's 0 x86/unwind/orc: Fix premature unwind stoppage due to IRET frames x86/unwind/orc: Fix error path for bad ORC entry type x86/unwind/orc: Prevent unwinding before ORC initialization x86/unwind/orc: Don't skip the first frame for inactive tasks x86/entry/64: Fix unwind hints in rewind_stack_do_exit() x86/entry/64: Fix unwind hints in kernel exit path x86/entry/64: Fix unwind hints in register clearing code batman-adv: Fix refcnt leak in batadv_v_ogm_process batman-adv: Fix refcnt leak in batadv_store_throughput_override batman-adv: Fix refcnt leak in batadv_show_throughput_override batman-adv: fix batadv_nc_random_weight_tq KVM: VMX: Mark RCX, RDX and RSI as clobbered in vmx_vcpu_run()'s asm blob KVM: VMX: Explicitly reference RCX as the vmx_vcpu pointer in asm blobs coredump: fix crash when umh is disabled staging: gasket: Check the return value of gasket_get_bar_index() mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous() arm64: hugetlb: avoid potential NULL dereference KVM: arm64: Fix 32bit PC wrap-around KVM: arm: vgic: Fix limit condition when writing to GICD_I[CS]ACTIVER tracing: Add a vmalloc_sync_mappings() for safe measure USB: serial: garmin_gps: add sanity checking for data length USB: uas: add quirk for LaCie 2Big Quadra HID: usbhid: Fix race between usbhid_close() and usbhid_stop() sctp: Fix bundling of SHUTDOWN with COOKIE-ACK HID: wacom: Read HID_DG_CONTACTMAX directly for non-generic devices net: stricter validation of untrusted gso packets bnxt_en: Fix VF anti-spoof filter setup. bnxt_en: Improve AER slot reset. net/mlx5: Fix command entry leak in Internal Error State net/mlx5: Fix forced completion access non initialized command entry bnxt_en: Fix VLAN acceleration handling in bnxt_fix_features(). tipc: fix partial topology connection closure sch_sfq: validate silly quantum values sch_choke: avoid potential panic in choke_reset() net: usb: qmi_wwan: add support for DW5816e net_sched: sch_skbprio: add message validation to skbprio_change() net/mlx4_core: Fix use of ENOSPC around mlx4_counter_alloc() net: macsec: preserve ingress frame ordering fq_codel: fix TCA_FQ_CODEL_DROP_BATCH_SIZE sanity checks dp83640: reverse arguments to list_add_tail vt: fix unicode console freeing with a common interface tracing/kprobes: Fix a double initialization typo USB: serial: qcserial: Add DW5816e support ANDROID: usb: gadget: Add missing inline qualifier to stub functions ANDROID: Drop ABI monitoring from KASAN build config ANDROID: Rename build.config.gki.arch_kasan ANDROID: GKI: Enable CONFIG_STATIC_USERMODEHELPER ANDROID: dm-default-key: Update key size for wrapped keys ANDROID: gki_defconfig: enable CONFIG_MMC_CRYPTO ANDROID: mmc: MMC crypto API ANDROID: GKI: Update the ABI xml and whitelist ANDROID: GKI: add missing exports for cam_smmu_api.ko Linux 4.19.122 drm/atomic: Take the atomic toys away from X cgroup, netclassid: remove double cond_resched mac80211: add ieee80211_is_any_nullfunc() platform/x86: GPD pocket fan: Fix error message when temp-limits are out of range ALSA: hda: Match both PCI ID and SSID for driver blacklist hexagon: define ioremap_uc hexagon: clean up ioremap mfd: intel-lpss: Use devm_ioremap_uc for MMIO lib: devres: add a helper function for ioremap_uc drm/amdgpu: Fix oops when pp_funcs is unset in ACPI event sctp: Fix SHUTDOWN CTSN Ack in the peer restart case net: systemport: suppress warnings on failed Rx SKB allocations net: bcmgenet: suppress warnings on failed Rx SKB allocations lib/mpi: Fix building for powerpc with clang scripts/config: allow colons in option strings for sed s390/ftrace: fix potential crashes when switching tracers cifs: protect updating server->dstaddr with a spinlock ASoC: rsnd: Fix "status check failed" spam for multi-SSI ASoC: rsnd: Don't treat master SSI in multi SSI setup as parent net: stmmac: Fix sub-second increment net: stmmac: fix enabling socfpga's ptp_ref_clock wimax/i2400m: Fix potential urb refcnt leak drm/amdgpu: Correctly initialize thermal controller for GPUs with Powerplay table v0 (e.g Hawaii) ASoC: codecs: hdac_hdmi: Fix incorrect use of list_for_each_entry ASoC: rsnd: Fix HDMI channel mapping for multi-SSI mode ASoC: rsnd: Fix parent SSI start/stop in multi-SSI mode usb: dwc3: gadget: Properly set maxpacket limit ASoC: sgtl5000: Fix VAG power-on handling selftests/ipc: Fix test failure seen after initial test run ASoC: topology: Check return value of pcm_new_ver powerpc/pci/of: Parse unassigned resources vhost: vsock: kick send_pkt worker once device is started ANDROID: GKI: fix build warning on 32bits due to ASoC msm change ANDROID: GKI: fix build error on 32bits due to ASoC msm change ANDROID: GKI: update abi definition due to FAIR_GROUP_SCHED removal ANDROID: GKI: Remove FAIR_GROUP_SCHED ANDROID: GKI: BULK update ABI XML representation and qcom whitelist ANDROID: build.config.gki.aarch64: Enable WHITELIST_STRICT_MODE ANDROID: GKI: Update the ABI xml and qcom whitelist ANDROID: remove unused variable ANDROID: Drop ABI monitoring from KASAN build config Linux 4.19.121 mmc: meson-mx-sdio: remove the broken ->card_busy() op mmc: meson-mx-sdio: Set MMC_CAP_WAIT_WHILE_BUSY mmc: sdhci-msm: Enable host capabilities pertains to R1b response mmc: sdhci-pci: Fix eMMC driver strength for BYT-based controllers mmc: sdhci-xenon: fix annoying 1.8V regulator warning mmc: cqhci: Avoid false "cqhci: CQE stuck on" by not open-coding timeout loop btrfs: transaction: Avoid deadlock due to bad initialization timing of fs_info::journal_info btrfs: fix partial loss of prealloc extent past i_size after fsync selinux: properly handle multiple messages in selinux_netlink_send() dmaengine: dmatest: Fix iteration non-stop logic nfs: Fix potential posix_acl refcnt leak in nfs3_set_acl ALSA: opti9xx: shut up gcc-10 range warning iommu/amd: Fix legacy interrupt remapping for x2APIC-enabled system scsi: target/iblock: fix WRITE SAME zeroing iommu/qcom: Fix local_base status check vfio/type1: Fix VA->PA translation for PFNMAP VMAs in vaddr_get_pfn() vfio: avoid possible overflow in vfio_iommu_type1_pin_pages RDMA/core: Fix race between destroy and release FD object RDMA/core: Prevent mixed use of FDs between shared ufiles RDMA/mlx4: Initialize ib_spec on the stack RDMA/mlx5: Set GRH fields in query QP on RoCE scsi: qla2xxx: check UNLOADING before posting async work scsi: qla2xxx: set UNLOADING before waiting for session deletion dm multipath: use updated MPATHF_QUEUE_IO on mapping for bio-based mpath dm writecache: fix data corruption when reloading the target dm verity fec: fix hash block number in verity_fec_decode PM: hibernate: Freeze kernel threads in software_resume() PM: ACPI: Output correct message on target power state ALSA: pcm: oss: Place the plugin buffer overflow checks correctly ALSA: hda/hdmi: fix without unlocked before return ALSA: usb-audio: Correct a typo of NuPrime DAC-10 USB ID ALSA: hda/realtek - Two front mics on a Lenovo ThinkCenter btrfs: fix block group leak when removing fails drm/qxl: qxl_release use after free drm/qxl: qxl_release leak in qxl_hw_surface_alloc() drm/qxl: qxl_release leak in qxl_draw_dirty_fb() drm/edid: Fix off-by-one in DispID DTD pixel clock ANDROID: GKI: Bulk update ABI XML representation ANDROID: GKI: Enable net testing options ANDROID: gki_defconfig: Enable CONFIG_REMOTEPROC ANDROID: Rename build.config.gki.arch_kasan ANDROID: GKI: Update ABI for IOMMU ANDROID: Incremental fs: Fix issues with very large files ANDROID: Correct build.config branch name ANDROID: GKI: Bulk update ABI XML representation and whitelist. UPSTREAM: vdso: Fix clocksource.h macro detection ANDROID: GKI: update abi definition due to added padding ANDROID: GKI: networking: add Android ABI padding to a lot of networking structures ANDROID: GKI: dma-mapping.h: add Android ABI padding to a structure ANDROID: GKI: ioport.h: add Android ABI padding to a structure ANDROID: GKI: iomap.h: add Android ABI padding to a structure ANDROID: GKI: genhd.h: add Android ABI padding to some structures ANDROID: GKI: hrtimer.h: add Android ABI padding to a structure ANDROID: GKI: ethtool.h: add Android ABI padding to a structure ANDROID: GKI: sched: add Android ABI padding to some structures ANDROID: GKI: kernfs.h: add Android ABI padding to some structures ANDROID: GKI: kobject.h: add Android ABI padding to some structures ANDROID: GKI: mm.h: add Android ABI padding to a structure ANDROID: GKI: mmu_notifier.h: add Android ABI padding to some structures ANDROID: GKI: pci: add Android ABI padding to some structures ANDROID: GKI: irqdomain.h: add Android ABI padding to a structure ANDROID: GKI: blk_types.h: add Android ABI padding to a structure ANDROID: GKI: scsi.h: add Android ABI padding to a structure ANDROID: GKI: quota.h: add Android ABI padding to some structures ANDROID: GKI: timer.h: add Android ABI padding to a structure ANDROID: GKI: user_namespace.h: add Android ABI padding to a structure FROMGIT: f2fs: fix missing check for f2fs_unlock_op Linux 4.19.120 propagate_one(): mnt_set_mountpoint() needs mount_lock ext4: check for non-zero journal inum in ext4_calculate_overhead qed: Fix use after free in qed_chain_free bpf, x86_32: Fix clobbering of dst for BPF_JSET hwmon: (jc42) Fix name to have no illegal characters ext4: convert BUG_ON's to WARN_ON's in mballoc.c ext4: increase wait time needed before reuse of deleted inode numbers ext4: use matching invalidatepage in ext4_writepage arm64: Delete the space separator in __emit_inst ALSA: hda: call runtime_allow() for all hda controllers xen/xenbus: ensure xenbus_map_ring_valloc() returns proper grant status objtool: Support Clang non-section symbols in ORC dump objtool: Fix CONFIG_UBSAN_TRAP unreachable warnings scsi: target: tcmu: reset_ring should reset TCMU_DEV_BIT_BROKEN scsi: target: fix PR IN / READ FULL STATUS for FC ALSA: hda: Explicitly permit using autosuspend if runtime PM is supported ALSA: hda: Keep the controller initialization even if no codecs found xfs: fix partially uninitialized structure in xfs_reflink_remap_extent x86: hyperv: report value of misc_features net: fec: set GPR bit on suspend by DT configuration. bpf, x86: Fix encoding for lower 8-bit registers in BPF_STX BPF_B xfs: clear PF_MEMALLOC before exiting xfsaild thread mm: shmem: disable interrupt when acquiring info->lock in userfaultfd_copy path bpf, x86_32: Fix incorrect encoding in BPF_LDX zero-extension perf/core: fix parent pid/tid in task exit events net/mlx5: Fix failing fw tracer allocation on s390 cpumap: Avoid warning when CONFIG_DEBUG_PER_CPU_MAPS is enabled ARM: dts: bcm283x: Disable dsi0 node PCI: Move Apex Edge TPU class quirk to fix BAR assignment PCI: Avoid ASMedia XHCI USB PME# from D0 defect svcrdma: Fix leak of svc_rdma_recv_ctxt objects svcrdma: Fix trace point use-after-free race xfs: acquire superblock freeze protection on eofblocks scans net/cxgb4: Check the return from t4_query_params properly rxrpc: Fix DATA Tx to disable nofrag for UDP on AF_INET6 socket i2c: altera: use proper variable to hold errno nfsd: memory corruption in nfsd4_lock() ASoC: wm8960: Fix wrong clock after suspend & resume ASoC: tas571x: disable regulators on failed probe ASoC: q6dsp6: q6afe-dai: add missing channels to MI2S DAIs iio:ad7797: Use correct attribute_group usb: gadget: udc: bdc: Remove unnecessary NULL checks in bdc_req_complete usb: dwc3: gadget: Do link recovery for SS and SSP binder: take read mode of mmap_sem in binder_alloc_free_page() include/uapi/linux/swab.h: fix userspace breakage, use __BITS_PER_LONG for swap mtd: cfi: fix deadloop in cfi_cmdset_0002.c do_write_buffer remoteproc: Fix wrong rvring index computation FROMLIST: PM / devfreq: Restart previous governor if new governor fails to start ANDROID: GKI: arm64: Enable GZIP and LZ4 kernel compression modes ANDROID: GKI: arm64: gki_defconfig: Set arm_smmu configuration ANDROID: GKI: iommu/arm-smmu: Modularize ARM SMMU driver ANDROID: GKI: iommu: Snapshot of vendor changes ANDROID: GKI: Additions to ARM SMMU register definitions ANDROID: GKI: iommu/io-pgtable-arm: LPAE related updates by vendor ANDROID: GKI: common: dma-mapping: make dma_common_contiguous_remap more robust ANDROID: GKI: dma-coherent: Expose device base address and size ANDROID: GKI: arm64: add support for NO_KERNEL_MAPPING and STRONGLY_ORDERED ANDROID: GKI: dma-mapping: Add dma_remap functions ANDROID: GKI: arm64: Support early fixup for CMA ANDROID: GKI: iommu: dma-mapping-fast: Fast ARMv7/v8 Long Descriptor Format ANDROID: GKI: arm64: dma-mapping: add support for IOMMU mapper ANDROID: GKI: add ARCH_NR_GPIO for ABI match ANDROID: GKI: kernel: Export symbol of `cpu_do_idle` ANDROID: GKI: kernel: Export symbols needed by msm_minidump.ko and minidump_log.ko (again) ANDROID: GKI: add missing exports for __flush_dcache_area ANDROID: GKI: arm64: Export caching APIs ANDROID: GKI: arm64: provide dma cache routines with same API as 32 bit ANDROID: gki_defconfig: add FORTIFY_SOURCE, remove SPMI_MSM_PMIC_ARB Revert "ANDROID: GKI: spmi: pmic-arb: don't enable SPMI_MSM_PMIC_ARB by default" ANDROID: GKI: update abi definitions after adding padding ANDROID: GKI: elevator: add Android ABI padding to some structures ANDROID: GKI: dentry: add Android ABI padding to some structures ANDROID: GKI: bio: add Android ABI padding to some structures ANDROID: GKI: scsi: add Android ABI padding to some structures ANDROID: GKI: ufs: add Android ABI padding to some structures ANDROID: GKI: workqueue.h: add Android ABI padding to some structures ANDROID: GKI: fs.h: add Android ABI padding to some structures ANDROID: GKI: USB: add Android ABI padding to some structures ANDROID: GKI: mm: add Android ABI padding to some structures ANDROID: GKI: mount.h: add Android ABI padding to some structures ANDROID: GKI: sched.h: add Android ABI padding to some structures ANDROID: GKI: sock.h: add Android ABI padding to some structures ANDROID: GKI: module.h: add Android ABI padding to some structures ANDROID: GKI: device.h: add Android ABI padding to some structures ANDROID: GKI: phy: add Android ABI padding to some structures ANDROID: GKI: add android_kabi.h ANDROID: ABI: update due to previous changes in the tree BACKPORT: sched/core: Fix reset-on-fork from RT with uclamp ANDROID: GKI: Add support for missing V4L2 symbols ANDROID: GKI: Bulk update ABI XML representation ANDROID: GKI: arm64: psci: Support for OS initiated scheme ANDROID: GKI: net: add counter for number of frames coalesced in GRO ANDROID: GKI: cfg80211: Include length of kek in rekey data BACKPORT: loop: change queue block size to match when using DIO ANDROID: Incremental fs: Add setattr call ANDROID: GKI: enable CONFIG_RTC_SYSTOHC ANDROID: GKI: ipv4: add vendor padding to __IPV4_DEVCONF_* enums Revert "ANDROID: GKI: ipv4: increase __IPV4_DEVCONF_MAX to 64" ANDROID: driver: gpu: drm: fix export symbol types ANDROID: SoC: core: fix export symbol type ANDROID: ufshcd-crypto: fix export symbol type ANDROID: GKI: drivers: mailbox: fix race resulting in multiple message submission ANDROID: GKI: arm64: gki_defconfig: Enable a few thermal configs Revert "ANDROID: GKI: add base.h include to match MODULE_VERSIONS" FROMLIST: thermal: Make cooling device trip point writable from sysfs ANDROID: GKI: drivers: thermal: cpu_cooling: Use CPU ID as cooling device ID ANDROID: GKI: PM / devfreq: Allow min freq to be 0 ANDROID: GKI: arm64: gki_defconfig: Enable REGULATOR_PROXY_CONSUMER ANDROID: GKI: Bulk Update ABI XML representation ANDROID: KASAN support for GKI remove CONFIG_CC_WERROR ANDROID: KASAN support for GKI ANDROID: virt_wifi: fix export symbol types ANDROID: vfs: fix export symbol type ANDROID: vfs: fix export symbol types ANDROID: fscrypt: fix export symbol type ANDROID: cfi: fix export symbol types ANDROID: bpf: fix export symbol type Linux 4.19.119 s390/mm: fix page table upgrade vs 2ndary address mode accesses xfs: Fix deadlock between AGI and AGF with RENAME_WHITEOUT serial: sh-sci: Make sure status register SCxSR is read in correct sequence xhci: prevent bus suspend if a roothub port detected a over-current condition usb: f_fs: Clear OS Extended descriptor counts to zero in ffs_data_reset() usb: dwc3: gadget: Fix request completion check UAS: fix deadlock in error handling and PM flushing work UAS: no use logging any details in case of ENODEV cdc-acm: introduce a cool down cdc-acm: close race betrween suspend() and acm_softint staging: vt6656: Power save stop wake_up_count wrap around. staging: vt6656: Fix pairwise key entry save. staging: vt6656: Fix drivers TBTT timing counter. staging: vt6656: Fix calling conditions of vnt_set_bss_mode staging: vt6656: Don't set RCR_MULTICAST or RCR_BROADCAST by default. vt: don't use kmalloc() for the unicode screen buffer vt: don't hardcode the mem allocation upper bound staging: comedi: Fix comedi_device refcnt leak in comedi_open staging: comedi: dt2815: fix writing hi byte of analog output powerpc/setup_64: Set cache-line-size based on cache-block-size ARM: imx: provide v7_cpu_resume() only on ARM_CPU_SUSPEND=y iwlwifi: mvm: beacon statistics shouldn't go backwards iwlwifi: pcie: actually release queue memory in TVQM ASoC: dapm: fixup dapm kcontrol widget audit: check the length of userspace generated audit records usb-storage: Add unusual_devs entry for JMicron JMS566 tty: rocket, avoid OOB access tty: hvc: fix buffer overflow during hvc_alloc(). KVM: VMX: Enable machine check support for 32bit targets KVM: Check validity of resolved slot when searching memslots KVM: s390: Return last valid slot if approx index is out-of-bounds tpm: ibmvtpm: retry on H_CLOSED in tpm_ibmvtpm_send() tpm/tpm_tis: Free IRQ if probing fails ALSA: usb-audio: Filter out unsupported sample rates on Focusrite devices ALSA: usb-audio: Fix usb audio refcnt leak when getting spdif ALSA: hda/realtek - Add new codec supported for ALC245 ALSA: hda/realtek - Fix unexpected init_amp override ALSA: usx2y: Fix potential NULL dereference tools/vm: fix cross-compile build mm/ksm: fix NULL pointer dereference when KSM zero page is enabled mm/hugetlb: fix a addressing exception caused by huge_pte_offset vmalloc: fix remap_vmalloc_range() bounds checks USB: hub: Fix handling of connect changes during sleep USB: core: Fix free-while-in-use bug in the USB S-Glibrary USB: early: Handle AMD's spec-compliant identifiers, too USB: Add USB_QUIRK_DELAY_CTRL_MSG and USB_QUIRK_DELAY_INIT for Corsair K70 RGB RAPIDFIRE USB: sisusbvga: Change port variable from signed to unsigned fs/namespace.c: fix mountpoint reference counter race iio: xilinx-xadc: Make sure not exceed maximum samplerate iio: xilinx-xadc: Fix sequencer configuration for aux channels in simultaneous mode iio: xilinx-xadc: Fix clearing interrupt when enabling trigger iio: xilinx-xadc: Fix ADC-B powerdown iio: adc: stm32-adc: fix sleep in atomic context iio: st_sensors: rely on odr mask to know if odr can be set iio: core: remove extra semi-colon from devm_iio_device_register() macro ALSA: usb-audio: Add connector notifier delegation ALSA: usb-audio: Add static mapping table for ALC1220-VB-based mobos ALSA: hda: Remove ASUS ROG Zenith from the blacklist KEYS: Avoid false positive ENOMEM error on key read mlxsw: Fix some IS_ERR() vs NULL bugs vrf: Check skb for XFRM_TRANSFORMED flag xfrm: Always set XFRM_TRANSFORMED in xfrm{4,6}_output_finish net: dsa: b53: b53_arl_rw_op() needs to select IVL or SVL net: dsa: b53: Rework ARL bin logic net: dsa: b53: Fix ARL register definitions net: dsa: b53: Lookup VID in ARL searches when VLAN is enabled vrf: Fix IPv6 with qdisc and xfrm team: fix hang in team_mode_get() tcp: cache line align MAX_TCP_HEADER sched: etf: do not assume all sockets are full blown net/x25: Fix x25_neigh refcnt leak when receiving frame net: stmmac: dwmac-meson8b: Add missing boundary to RGMII TX clock array net: netrom: Fix potential nr_neigh refcnt leak in nr_add_node net: bcmgenet: correct per TX/RX ring statistics macvlan: fix null dereference in macvlan_device_event() macsec: avoid to set wrong mtu ipv6: fix restrict IPV6_ADDRFORM operation cxgb4: fix large delays in PTP synchronization cxgb4: fix adapter crash due to wrong MC size x86/KVM: Clean up host's steal time structure x86/KVM: Make sure KVM_VCPU_FLUSH_TLB flag is not missed x86/kvm: Cache gfn to pfn translation x86/kvm: Introduce kvm_(un)map_gfn() KVM: Properly check if "page" is valid in kvm_vcpu_unmap kvm: fix compile on s390 part 2 kvm: fix compilation on s390 kvm: fix compilation on aarch64 KVM: Introduce a new guest mapping API KVM: nVMX: Always sync GUEST_BNDCFGS when it comes from vmcs01 KVM: VMX: Zero out *all* general purpose registers after VM-Exit f2fs: fix to avoid memory leakage in f2fs_listxattr blktrace: fix dereference after null check blktrace: Protect q->blk_trace with RCU net: ipv6_stub: use ip6_dst_lookup_flow instead of ip6_dst_lookup net: ipv6: add net argument to ip6_dst_lookup_flow PCI/ASPM: Allow re-enabling Clock PM scsi: smartpqi: fix call trace in device discovery virtio-blk: improve virtqueue error to BLK_STS tracing/selftests: Turn off timeout setting drm/amd/display: Not doing optimize bandwidth if flip pending. xhci: Ensure link state is U3 after setting USB_SS_PORT_LS_U3 ASoC: Intel: bytcr_rt5640: Add quirk for MPMAN MPWIN895CL tablet perf/core: Disable page faults when getting phys address pwm: bcm2835: Dynamically allocate base pwm: renesas-tpu: Fix late Runtime PM enablement Revert "powerpc/64: irq_work avoid interrupt when called with hardware irqs enabled" loop: Better discard support for block devices s390/cio: avoid duplicated 'ADD' uevents kconfig: qconf: Fix a few alignment issues ipc/util.c: sysvipc_find_ipc() should increase position index selftests: kmod: fix handling test numbers above 9 kernel/gcov/fs.c: gcov_seq_next() should increase position index nvme: fix deadlock caused by ANA update wrong locking ASoC: Intel: atom: Take the drv->lock mutex before calling sst_send_slot_map() scsi: iscsi: Report unbind session event when the target has been removed pwm: rcar: Fix late Runtime PM enablement ceph: don't skip updating wanted caps when cap is stale ceph: return ceph_mdsc_do_request() errors from __get_parent() scsi: lpfc: Fix crash in target side cable pulls hitting WAIT_FOR_UNREG scsi: lpfc: Fix kasan slab-out-of-bounds error in lpfc_unreg_login watchdog: reset last_hw_keepalive time at start arm64: Silence clang warning on mismatched value/register sizes arm64: compat: Workaround Neoverse-N1 #1542419 for compat user-space arm64: Fake the IminLine size on systems affected by Neoverse-N1 #1542419 arm64: errata: Hide CTR_EL0.DIC on systems affected by Neoverse-N1 #1542419 arm64: Add part number for Neoverse N1 vti4: removed duplicate log message. crypto: mxs-dcp - make symbols 'sha1_null_hash' and 'sha256_null_hash' static bpftool: Fix printing incorrect pointer in btf_dump_ptr drm/msm: Use the correct dma_sync calls harder ext4: fix extent_status fragmentation for plain files ANDROID: abi_gki_aarch64_cuttlefish_whitelist: remove stale symbols ANDROID: GKI: ipv4: increase __IPV4_DEVCONF_MAX to 64 ANDROID: GKI: power: add missing export for POWER_RESET_QCOM=m BACKPORT: cfg80211: Support key configuration for Beacon protection (BIGTK) BACKPORT: cfg80211: Enhance the AKM advertizement to support per interface. UPSTREAM: sysrq: Use panic() to force a crash ANDROID: GKI: kernel: sound: update codec options with block size ANDROID: add compat cross compiler ANDROID: x86/vdso: disable LTO only for VDSO BACKPORT: arm64: vdso32: Enable Clang Compilation UPSTREAM: arm64: compat: vdso: Expose BUILD_VDSO32 BACKPORT: lib/vdso: Enable common headers BACKPORT: arm: vdso: Enable arm to use common headers BACKPORT: x86/vdso: Enable x86 to use common headers BACKPORT: mips: vdso: Enable mips to use common headers UPSTREAM: arm64: vdso32: Include common headers in the vdso library UPSTREAM: arm64: vdso: Include common headers in the vdso library UPSTREAM: arm64: Introduce asm/vdso/processor.h BACKPORT: arm64: vdso32: Code clean up UPSTREAM: linux/elfnote.h: Replace elf.h with UAPI equivalent UPSTREAM: scripts: Fix the inclusion order in modpost UPSTREAM: common: Introduce processor.h UPSTREAM: linux/ktime.h: Extract common header for vDSO UPSTREAM: linux/jiffies.h: Extract common header for vDSO UPSTREAM: linux/time64.h: Extract common header for vDSO BACKPORT: linux/time32.h: Extract common header for vDSO BACKPORT: linux/time.h: Extract common header for vDSO UPSTREAM: linux/math64.h: Extract common header for vDSO BACKPORT: linux/clocksource.h: Extract common header for vDSO BACKPORT: mips: Introduce asm/vdso/clocksource.h BACKPORT: arm64: Introduce asm/vdso/clocksource.h BACKPORT: arm: Introduce asm/vdso/clocksource.h BACKPORT: x86: Introduce asm/vdso/clocksource.h UPSTREAM: linux/limits.h: Extract common header for vDSO BACKPORT: linux/kernel.h: split *_MAX and *_MIN macros into <linux/limits.h> BACKPORT: linux/bits.h: Extract common header for vDSO UPSTREAM: linux/const.h: Extract common header for vDSO BACKPORT: arm64: vdso: fix flip/flop vdso build bug UPSTREAM: lib/vdso: Allow the high resolution parts to be compiled out UPSTREAM: lib/vdso: Only read hrtimer_res when needed in __cvdso_clock_getres() UPSTREAM: lib/vdso: Mark do_hres() and do_coarse() as __always_inline UPSTREAM: lib/vdso: Avoid duplication in __cvdso_clock_getres() UPSTREAM: lib/vdso: Let do_coarse() return 0 to simplify the callsite UPSTREAM: lib/vdso: Remove checks on return value for 32 bit vDSO UPSTREAM: lib/vdso: Build 32 bit specific functions in the right context UPSTREAM: lib/vdso: Make __cvdso_clock_getres() static UPSTREAM: lib/vdso: Make clock_getres() POSIX compliant again UPSTREAM: lib/vdso/32: Provide legacy syscall fallbacks UPSTREAM: lib/vdso: Move fallback invocation to the callers UPSTREAM: lib/vdso/32: Remove inconsistent NULL pointer checks UPSTREAM: lib/vdso: Make delta calculation work correctly UPSTREAM: arm64: compat: Fix syscall number of compat_clock_getres BACKPORT: arm64: lse: Fix LSE atomics with LLVM UPSTREAM: mips: Fix gettimeofday() in the vdso library UPSTREAM: mips: vdso: Fix __arch_get_hw_counter() BACKPORT: arm64: Kconfig: Make CONFIG_COMPAT_VDSO a proper Kconfig option UPSTREAM: arm64: vdso32: Rename COMPATCC to CC_COMPAT UPSTREAM: arm64: vdso32: Pass '--target' option to clang via VDSO_CAFLAGS UPSTREAM: arm64: vdso32: Don't use KBUILD_CPPFLAGS unconditionally UPSTREAM: arm64: vdso32: Move definition of COMPATCC into vdso32/Makefile UPSTREAM: arm64: Default to building compat vDSO with clang when CONFIG_CC_IS_CLANG UPSTREAM: lib: vdso: Remove CROSS_COMPILE_COMPAT_VDSO UPSTREAM: arm64: vdso32: Remove jump label config option in Makefile UPSTREAM: arm64: vdso32: Detect binutils support for dmb ishld BACKPORT: arm64: vdso: Remove stale files from old assembly implementation UPSTREAM: arm64: vdso32: Fix broken compat vDSO build warnings UPSTREAM: mips: compat: vdso: Use legacy syscalls as fallback BACKPORT: arm64: Relax Documentation/arm64/tagged-pointers.rst BACKPORT: arm64: Add tagged-address-abi.rst to index.rst UPSTREAM: arm64: vdso: Fix Makefile regression UPSTREAM: mips: vdso: Fix flip/flop vdso building bug UPSTREAM: mips: vdso: Fix source path UPSTREAM: mips: Add clock_gettime64 entry point UPSTREAM: mips: Add clock_getres entry point BACKPORT: mips: Add support for generic vDSO BACKPORT: arm64: vdso: Explicitly add build-id option BACKPORT: arm64: vdso: use $(LD) instead of $(CC) to link VDSO BACKPORT: arm64: vdso: Cleanup Makefiles UPSTREAM: arm64: vdso: Fix population of AT_SYSINFO_EHDR for compat vdso UPSTREAM: arm64: vdso: Fix compilation with clang older than 8 UPSTREAM: arm64: compat: Fix __arch_get_hw_counter() implementation UPSTREAM: arm64: Fix __arch_get_hw_counter() implementation UPSTREAM: x86/vdso/32: Use 32bit syscall fallback UPSTREAM: x86/vdso: Fix flip/flop vdso build bug UPSTREAM: x86/vdso: Give the [ph]vclock_page declarations real types UPSTREAM: x86/vdso: Add clock_gettime64() entry point BACKPORT: x86/vdso: Add clock_getres() entry point BACKPORT: x86/vdso: Switch to generic vDSO implementation UPSTREAM: x86/segments: Introduce the 'CPUNODE' naming to better document the segment limit CPU/node NR trick UPSTREAM: x86/vdso: Initialize the CPU/node NR segment descriptor earlier UPSTREAM: x86/vdso: Introduce helper functions for CPU and node number UPSTREAM: x86/segments/64: Rename the GDT PER_CPU entry to CPU_NUMBER BACKPORT: arm64: vdso: Enable vDSO compat support UPSTREAM: arm64: compat: Get sigreturn trampolines from vDSO UPSTREAM: arm64: elf: VDSO code page discovery UPSTREAM: arm64: compat: VDSO setup for compat layer UPSTREAM: arm64: vdso: Refactor vDSO code BACKPORT: arm64: compat: Add vDSO UPSTREAM: arm64: compat: Generate asm offsets for signals UPSTREAM: arm64: compat: Expose signal related structures UPSTREAM: arm64: compat: Add missing syscall numbers BACKPORT: arm64: vdso: Substitute gettimeofday() with C implementation UPSTREAM: timekeeping: Provide a generic update_vsyscall() implementation UPSTREAM: lib/vdso: Add compat support UPSTREAM: lib/vdso: Provide generic VDSO implementation UPSTREAM: vdso: Define standardized vdso_datapage UPSTREAM: hrtimer: Split out hrtimer defines into separate header UPSTREAM: nds32: Fix vDSO clock_getres() UPSTREAM: arm64: compat: Reduce address limit for 64K pages BACKPORT: arm64: compat: Add KUSER_HELPERS config option UPSTREAM: arm64: compat: Refactor aarch32_alloc_vdso_pages() BACKPORT: arm64: compat: Split kuser32 UPSTREAM: arm64: compat: Alloc separate pages for vectors and sigpage ANDROID: GKI: Update ABI XML representation ANDROID: GKI: Enable GENERIC_IRQ_CHIP ANDROID: GKI: power_supply: Add FG_TYPE power-supply property ANDROID: GKI: mm: export mm_trace_rss_stat for modules to report RSS changes ANDROID: GKI: gki_defconfig: Enable CONFIG_LEDS_TRIGGER_TRANSIENT ANDROID: GKI: gki_defconfig: Enable CONFIG_CPU_FREQ_STAT ANDROID: GKI: arm64: gki_defconfig: Disable HW tracing features ANDROID: GKI: gki_defconfig: Enable CONFIG_I2C_CHARDEV ANDROID: Incremental fs: Use simple compression in log buffer ANDROID: GKI: usb: core: Add support to parse config summary capability descriptors ANDROID: GKI: Update ABI XML representation ANDROID: dm-bow: Fix not to skip trim at framented range ANDROID: Remove VLA from uid_sys_stats.c f2fs: fix missing check for f2fs_unlock_op ANDROID: fix wakeup reason findings UPSTREAM: cfg80211: fix and clean up cfg80211_gen_new_bssid() UPSTREAM: cfg80211: save multi-bssid properties UPSTREAM: cfg80211: make BSSID generation function inline UPSTREAM: cfg80211: parse multi-bssid only if HW supports it UPSTREAM: cfg80211: Move Multiple BSS info to struct cfg80211_bss to be visible UPSTREAM: cfg80211: Properly track transmitting and non-transmitting BSS UPSTREAM: cfg80211: use for_each_element() for multi-bssid parsing UPSTREAM: cfg80211: Parsing of Multiple BSSID information in scanning UPSTREAM: cfg80211/nl80211: Offload OWE processing to user space in AP mode ANDROID: GKI: cfg80211: Sync nl80211 commands/feature with upstream ANDROID: GKI: gki_defconfig: Enable FW_LOADER_USER_HELPER* ANDROID: GKI: arm64: gki_defconfig: Disable CONFIG_ARM64_TAGGED_ADDR_ABI ANDROID: GKI: gki_defconfig: CONFIG_CHR_DEV_SG=y ANDROID: GKI: gki_defconfig: CONFIG_DM_DEFAULT_KEY=m ANDROID: update the ABI xml representation ANDROID: init: GKI: enable hidden configs for GPU Linux 4.19.118 bpf: fix buggy r0 retval refinement for tracing helpers KEYS: Don't write out to userspace while holding key semaphore mtd: phram: fix a double free issue in error path mtd: lpddr: Fix a double free in probe() mtd: spinand: Explicitly use MTD_OPS_RAW to write the bad block marker to OOB locktorture: Print ratio of acquisitions, not failures tty: evh_bytechan: Fix out of bounds accesses iio: si1133: read 24-bit signed integer for measurement fbdev: potential information leak in do_fb_ioctl() net: dsa: bcm_sf2: Fix overflow checks f2fs: fix to wait all node page writeback iommu/amd: Fix the configuration of GCR3 table root pointer libnvdimm: Out of bounds read in __nd_ioctl() power: supply: axp288_fuel_gauge: Broaden vendor check for Intel Compute Sticks. ext2: fix debug reference to ext2_xattr_cache ext2: fix empty body warnings when -Wextra is used iommu/vt-d: Fix mm reference leak drm/vc4: Fix HDMI mode validation f2fs: fix NULL pointer dereference in f2fs_write_begin() NFS: Fix memory leaks in nfs_pageio_stop_mirroring() drm/amdkfd: kfree the wrong pointer x86: ACPI: fix CPU hotplug deadlock KVM: s390: vsie: Fix possible race when shadowing region 3 tables compiler.h: fix error in BUILD_BUG_ON() reporting percpu_counter: fix a data race at vm_committed_as include/linux/swapops.h: correct guards for non_swap_entry() cifs: Allocate encryption header through kmalloc um: ubd: Prevent buffer overrun on command completion ext4: do not commit super on read-only bdev s390/cpum_sf: Fix wrong page count in error message powerpc/maple: Fix declaration made after definition s390/cpuinfo: fix wrong output when CPU0 is offline NFS: direct.c: Fix memory leak of dreq when nfs_get_lock_context fails NFSv4/pnfs: Return valid stateids in nfs_layout_find_inode_by_stateid() rtc: 88pm860x: fix possible race condition soc: imx: gpc: fix power up sequencing clk: tegra: Fix Tegra PMC clock out parents power: supply: bq27xxx_battery: Silence deferred-probe error clk: at91: usb: continue if clk_hw_round_rate() return zero x86/Hyper-V: Report crash data in die() when panic_on_oops is set x86/Hyper-V: Report crash register data when sysctl_record_panic_msg is not set x86/Hyper-V: Trigger crash enlightenment only once during system crash. x86/Hyper-V: Free hv_panic_page when fail to register kmsg dump x86/Hyper-V: Unload vmbus channel in hv panic callback xsk: Add missing check on user supplied headroom size rbd: call rbd_dev_unprobe() after unwatching and flushing notifies rbd: avoid a deadlock on header_rwsem when flushing notifies video: fbdev: sis: Remove unnecessary parentheses and commented code lib/raid6: use vdupq_n_u8 to avoid endianness warnings x86/Hyper-V: Report crash register data or kmsg before running crash kernel of: overlay: kmemleak in dup_and_fixup_symbol_prop() of: unittest: kmemleak in of_unittest_overlay_high_level() of: unittest: kmemleak in of_unittest_platform_populate() of: unittest: kmemleak on changeset destroy ALSA: hda: Don't release card at firmware loading error irqchip/mbigen: Free msi_desc on device teardown netfilter: nf_tables: report EOPNOTSUPP on unsupported flags/object type ARM: dts: imx6: Use gpc for FEC interrupt controller to fix wake on LAN. arm, bpf: Fix bugs with ALU64 {RSH, ARSH} BPF_K shift by 0 watchdog: sp805: fix restart handler ext4: use non-movable memory for superblock readahead scsi: sg: add sg_remove_request in sg_common_write objtool: Fix switch table detection in .text.unlikely arm, bpf: Fix offset overflow for BPF_MEM BPF_DW ANDROID: GKI: Bulk update ABI report. ANDROID: GKI: qos: Register irq notify after adding the qos request ANDROID: GKI: Add dual role mode to usb_dr_modes array UPSTREAM: virtio-gpu api: comment feature flags ANDROID: arch:arm64: Increase kernel command line size ANDROID: GKI: Add special linux_banner_ptr for modules Revert "ANDROID: GKI: Make linux_banner a C pointer" ANDROID: GKI: PM / devfreq: Add new flag to do simple clock scaling ANDROID: GKI: Resolve ABI diff for struct snd_usb_audio ANDROID: GKI: Bulk update ABI ANDROID: GKI: Update the whitelist for qcom SoCs ANDROID: GKI: arm64: gki_defconfig: Set CONFIG_SCSI_UFSHCD=m ANDROID: GKI: scsi: add option to override the command timeout ANDROID: GKI: scsi: Adjust DBD setting in mode sense for caching mode page per LLD ANDROID: add ion_stat tracepoint to common kernel UPSTREAM: gpu/trace: add a gpu total memory usage tracepoint Linux 4.19.117 mm/vmalloc.c: move 'area->pages' after if statement wil6210: remove reset file from debugfs wil6210: make sure Rx ring sizes are correlated wil6210: add general initialization/size checks wil6210: ignore HALP ICR if already handled wil6210: check rx_buff_mgmt before accessing it x86/resctrl: Fix invalid attempt at removing the default resource group x86/resctrl: Preserve CDP enable over CPU hotplug x86/microcode/AMD: Increase microcode PATCH_MAX_SIZE scsi: target: fix hang when multiple threads try to destroy the same iscsi session scsi: target: remove boilerplate code kvm: x86: Host feature SSBD doesn't imply guest feature SPEC_CTRL_SSBD ext4: do not zeroout extents beyond i_disksize drm/amd/powerplay: force the trim of the mclk dpm_levels if OD is enabled usb: dwc3: gadget: Don't clear flags before transfer ended usb: dwc3: gadget: don't enable interrupt when disabling endpoint mac80211_hwsim: Use kstrndup() in place of kasprintf() btrfs: check commit root generation in should_ignore_root tracing: Fix the race between registering 'snapshot' event trigger and triggering 'snapshot' operation keys: Fix proc_keys_next to increase position index ALSA: usb-audio: Check mapping at creating connector controls, too ALSA: usb-audio: Don't create jack controls for PCM terminals ALSA: usb-audio: Don't override ignore_ctl_error value from the map ALSA: usb-audio: Filter error from connector kctl ops, too ASoC: Intel: mrfld: return error codes when an error occurs ASoC: Intel: mrfld: fix incorrect check on p->sink ext4: fix incorrect inodes per group in error message ext4: fix incorrect group count in ext4_fill_super error message pwm: pca9685: Fix PWM/GPIO inter-operation jbd2: improve comments about freeing data buffers whose page mapping is NULL scsi: ufs: Fix ufshcd_hold() caused scheduling while atomic ovl: fix value of i_ino for lower hardlink corner case net: dsa: mt7530: fix tagged frames pass-through in VLAN-unaware mode net: stmmac: dwmac-sunxi: Provide TX and RX fifo sizes net: revert default NAPI poll timeout to 2 jiffies net: qrtr: send msgs from local of same id as broadcast net: ipv6: do not consider routes via gateways for anycast address check net: ipv4: devinet: Fix crash when add/del multicast IP with autojoin hsr: check protocol version in hsr_newlink() amd-xgbe: Use __napi_schedule() in BH context ANDROID: GKI: drivers: of-thermal: Relate thermal zones using same sensor ANDROID: GKI: Bulk ABI update ANDROID: GKI: dma: Add set_dma_mask hook to struct dma_map_ops ANDROID: GKI: ABI update due to recent patches FROMLIST: drm/prime: add support for virtio exported objects FROMLIST: dma-buf: add support for virtio exported objects UPSTREAM: drm/virtio: module_param_named() requires linux/moduleparam.h UPSTREAM: drm/virtio: fix resource id creation race UPSTREAM: drm/virtio: make resource id workaround runtime switchable. BACKPORT: drm/virtio: Drop deprecated load/unload initialization ANDROID: GKI: Add DRM_TTM config to GKI ANDROID: Bulk update the ABI xml representation ANDROID: GKI: spmi: pmic-arb: don't enable SPMI_MSM_PMIC_ARB by default ANDROID: GKI: attribute page lock and waitqueue functions as sched ANDROID: GKI: extcon: Fix Add usage of blocking notifier chain ANDROID: GKI: USB: pd: Extcon fix for C current ANDROID: drm/dsi: Fix byte order of DCS set/get brightness ANDROID: GKI: mm: Export symbols to modularize CONFIG_MSM_DRM ANDROID: GKI: ALSA: compress: Add support to send codec specific data ANDROID: GKI: ALSA: Compress - dont use lock for all ioctls ANDROID: GKI: ASoC: msm: qdsp6v2: add support for AMR_WB_PLUS offload ANDROID: GKI: msm: dolby: MAT and THD audiocodec name modification ANDROID: GKI: asoc: msm: Add support for compressed perf mode ANDROID: GKI: msm: audio: support for gapless_pcm ANDROID: GKI: uapi: msm: dolby: Support for TrueHD and MAT decoders ANDROID: GKI: ASoC: msm: qdsp6v2: Add TrueHD HDMI compress pass-though ANDROID: GKI: ALSA: compress: Add APTX format support in ALSA ANDROID: GKI: msm: qdsp6v2: Add timestamp support for compress capture ANDROID: GKI: SoC: msm: Add support for meta data in compressed TX ANDROID: GKI: ALSA: compress: Add DSD format support for ALSA ANDROID: GKI: ASoC: msm: qdsp6v2: add support for ALAC and APE offload ANDROID: GKI: SoC: msm: Add compressed TX and passthrough support ANDROID: GKI: ASoC: msm: qdsp6v2: Add FLAC in compress offload path ANDROID: GKI: ASoC: msm: add support for different compressed formats ANDROID: GKI: ASoC: msm: Update the encode option and sample rate ANDROID: GKI: Enable CONFIG_SND_VERBOSE_PROCFS in gki_defconfig ANDROID: GKI: Add hidden CONFIG_SND_SOC_COMPRESS to gki_defconfig ANDROID: GKI: ALSA: pcm: add locks for accessing runtime resource ANDROID: GKI: Update ABI for DRM changes ANDROID: GKI: Add drm_dp_send_dpcd_{read,write} accessor functions ANDROID: GKI: drm: Add drm_dp_mst_get_max_sdp_streams_supported accessor function ANDROID: GKI: drm: Add drm_dp_mst_has_fec accessor function ANDROID: GKI: Add 'dsc_info' to struct drm_dp_mst_port ANDROID: GKI: usb: Add support to handle USB SMMU S1 address ANDROID: GKI: usb: Add helper APIs to return xhci phys addresses ANDROID: Add C protos for dma_buf/drm_prime get_uuid ANDROID: GKI: Make linux_banner a C pointer ANDROID: GKI: Add 'refresh_rate', 'id' to struct drm_panel_notifier ANDROID: GKI: Add 'i2c_mutex' to struct drm_dp_aux ANDROID: GKI: Add 'checksum' to struct drm_connector Revert "BACKPORT: drm: Add HDR source metadata property" Revert "BACKPORT: drm: Parse HDR metadata info from EDID" ANDROID: drm: Add DP colorspace property ANDROID: GKI: drm: Initialize display->hdmi when parsing vsdb ANDROID: drivers: gpu: drm: add support to batch commands ANDROID: ABI: update the qcom whitelist ANDROID: GKI: ARM64: smp: add vendor field pending_ipi ANDROID: gki_defconfig: enable msm serial early console ANDROID: serial: msm_geni_serial_console : Add Earlycon support ANDROID: GKI: serial: core: export uart_console_device f2fs: fix quota_sync failure due to f2fs_lock_op f2fs: support read iostat f2fs: Fix the accounting of dcc->undiscard_blks f2fs: fix to handle error path of f2fs_ra_meta_pages() f2fs: report the discard cmd errors properly f2fs: fix long latency due to discard during umount f2fs: add tracepoint for f2fs iostat f2fs: introduce sysfs/data_io_flag to attach REQ_META/FUA ANDROID: GKI: update abi definition due to previous changes in the tree Linux 4.19.116 efi/x86: Fix the deletion of variables in mixed mode mfd: dln2: Fix sanity checking for endpoints etnaviv: perfmon: fix total and idle HI cyleces readout misc: echo: Remove unnecessary parentheses and simplify check for zero powerpc/fsl_booke: Avoid creating duplicate tlb1 entry ftrace/kprobe: Show the maxactive number on kprobe_events drm: Remove PageReserved manipulation from drm_pci_alloc drm/dp_mst: Fix clearing payload state on topology disable Revert "drm/dp_mst: Remove VCPI while disabling topology mgr" crypto: ccree - only try to map auth tag if needed crypto: ccree - dec auth tag size from cryptlen map crypto: ccree - don't mangle the request assoclen crypto: ccree - zero out internal struct before use crypto: ccree - improve error handling crypto: caam - update xts sector size for large input length dm zoned: remove duplicate nr_rnd_zones increase in dmz_init_zone() btrfs: use nofs allocations for running delayed items powerpc: Make setjmp/longjmp signature standard powerpc: Add attributes for setjmp/longjmp scsi: mpt3sas: Fix kernel panic observed on soft HBA unplug powerpc/kprobes: Ignore traps that happened in real mode powerpc/xive: Use XIVE_BAD_IRQ instead of zero to catch non configured IPIs powerpc/hash64/devmap: Use H_PAGE_THP_HUGE when setting up huge devmap PTE entries powerpc/64/tm: Don't let userspace set regs->trap via sigreturn powerpc/powernv/idle: Restore AMR/UAMOR/AMOR after idle xen/blkfront: fix memory allocation flags in blkfront_setup_indirect() ipmi: fix hung processes in __get_guid() libata: Return correct status in sata_pmp_eh_recover_pm() when ATA_DFLAG_DETACH is set hfsplus: fix crash and filesystem corruption when deleting files cpufreq: powernv: Fix use-after-free kmod: make request_module() return an error when autoloading is disabled clk: ingenic/jz4770: Exit with error if CGU init failed Input: i8042 - add Acer Aspire 5738z to nomux list s390/diag: fix display of diagnose call statistics perf tools: Support Python 3.8+ in Makefile ocfs2: no need try to truncate file beyond i_size fs/filesystems.c: downgrade user-reachable WARN_ONCE() to pr_warn_once() ext4: fix a data race at inode->i_blocks NFS: Fix a page leak in nfs_destroy_unlinked_subrequests() powerpc/pseries: Avoid NULL pointer dereference when drmem is unavailable drm/etnaviv: rework perfmon query infrastructure rtc: omap: Use define directive for PIN_CONFIG_ACTIVE_HIGH selftests: vm: drop dependencies on page flags from mlock2 tests arm64: armv8_deprecated: Fix undef_hook mask for thumb setend scsi: zfcp: fix missing erp_lock in port recovery trigger for point-to-point dm verity fec: fix memory leak in verity_fec_dtr dm writecache: add cond_resched to avoid CPU hangs arm64: dts: allwinner: h6: Fix PMU compatible net: qualcomm: rmnet: Allow configuration updates to existing devices mm: Use fixed constant in page_frag_alloc instead of size + 1 tools: gpio: Fix out-of-tree build regression x86/speculation: Remove redundant arch_smt_update() invocation powerpc/pseries: Drop pointless static qualifier in vpa_debugfs_init() erofs: correct the remaining shrink objects crypto: mxs-dcp - fix scatterlist linearization for hash btrfs: fix missing semaphore unlock in btrfs_sync_file btrfs: fix missing file extent item for hole after ranged fsync btrfs: drop block from cache on error in relocation btrfs: set update the uuid generation as soon as possible Btrfs: fix crash during unmount due to race with delayed inode workers mtd: spinand: Do not erase the block before writing a bad block marker mtd: spinand: Stop using spinand->oobbuf for buffering bad block markers CIFS: Fix bug which the return value by asynchronous read is error KVM: VMX: fix crash cleanup when KVM wasn't used KVM: x86: Gracefully handle __vmalloc() failure during VM allocation KVM: VMX: Always VMCLEAR in-use VMCSes during crash with kexec support KVM: x86: Allocate new rmap and large page tracking when moving memslot KVM: s390: vsie: Fix delivery of addressing exceptions KVM: s390: vsie: Fix region 1 ASCE sanity shadow address checks KVM: nVMX: Properly handle userspace interrupt window request x86/entry/32: Add missing ASM_CLAC to general_protection entry signal: Extend exec_id to 64bits ath9k: Handle txpower changes even when TPC is disabled MIPS: OCTEON: irq: Fix potential NULL pointer dereference MIPS/tlbex: Fix LDDIR usage in setup_pw() for Loongson-3 pstore: pstore_ftrace_seq_next should increase position index irqchip/versatile-fpga: Apply clear-mask earlier KEYS: reaching the keys quotas correctly tpm: tpm2_bios_measurements_next should increase position index tpm: tpm1_bios_measurements_next should increase position index tpm: Don't make log failures fatal PCI: endpoint: Fix for concurrent memory allocation in OB address region PCI: Add boot interrupt quirk mechanism for Xeon chipsets PCI/ASPM: Clear the correct bits when enabling L1 substates PCI: pciehp: Fix indefinite wait on sysfs requests nvme: Treat discovery subsystems as unique subsystems nvme-fc: Revert "add module to ops template to allow module references" thermal: devfreq_cooling: inline all stubs for CONFIG_DEVFREQ_THERMAL=n acpi/x86: ignore unspecified bit positions in the ACPI global lock field media: ti-vpe: cal: fix disable_irqs to only the intended target ALSA: hda/realtek - Add quirk for MSI GL63 ALSA: hda/realtek - Remove now-unnecessary XPS 13 headphone noise fixups ALSA: hda/realtek - Set principled PC Beep configuration for ALC256 ALSA: doc: Document PC Beep Hidden Register on Realtek ALC256 ALSA: pcm: oss: Fix regression by buffer overflow fix ALSA: ice1724: Fix invalid access for enumerated ctl items ALSA: hda: Fix potential access overflow in beep helper ALSA: hda: Add driver blacklist ALSA: usb-audio: Add mixer workaround for TRX40 and co usb: gadget: composite: Inform controller driver of self-powered usb: gadget: f_fs: Fix use after free issue as part of queue failure ASoC: topology: use name_prefix for new kcontrol ASoC: dpcm: allow start or stop during pause for backend ASoC: dapm: connect virtual mux with default value ASoC: fix regwmask slub: improve bit diffusion for freelist ptr obfuscation uapi: rename ext2_swab() to swab() and share globally in swab.h IB/mlx5: Replace tunnel mpls capability bits for tunnel_offloads btrfs: track reloc roots based on their commit root bytenr btrfs: remove a BUG_ON() from merge_reloc_roots() btrfs: qgroup: ensure qgroup_rescan_running is only set when the worker is at least queued block, bfq: fix use-after-free in bfq_idle_slice_timer_body locking/lockdep: Avoid recursion in lockdep_count_{for,back}ward_deps() firmware: fix a double abort case with fw_load_sysfs_fallback md: check arrays is suspended in mddev_detach before call quiesce operations irqchip/gic-v4: Provide irq_retrigger to avoid circular locking dependency usb: dwc3: core: add support for disabling SS instances in park mode media: i2c: ov5695: Fix power on and off sequences block: Fix use-after-free issue accessing struct io_cq genirq/irqdomain: Check pointer in irq_domain_alloc_irqs_hierarchy() efi/x86: Ignore the memory attributes table on i386 x86/boot: Use unsigned comparison for addresses gfs2: Don't demote a glock until its revokes are written pstore/platform: fix potential mem leak if pstore_init_fs failed libata: Remove extra scsi_host_put() in ata_scsi_add_hosts() media: i2c: video-i2c: fix build errors due to 'imply hwmon' PCI/switchtec: Fix init_completion race condition with poll_wait() selftests/x86/ptrace_syscall_32: Fix no-vDSO segfault sched: Avoid scale real weight down to zero irqchip/versatile-fpga: Handle chained IRQs properly block: keep bdi->io_pages in sync with max_sectors_kb for stacked devices x86: Don't let pgprot_modify() change the page encryption bit xhci: bail out early if driver can't accress host in resume null_blk: fix spurious IO errors after failed past-wp access null_blk: Handle null_add_dev() failures properly null_blk: Fix the null_add_dev() error path firmware: arm_sdei: fix double-lock on hibernate with shared events media: venus: hfi_parser: Ignore HEVC encoding for V1 cpufreq: imx6q: Fixes unwanted cpu overclocking on i.MX6ULL i2c: st: fix missing struct parameter description qlcnic: Fix bad kzalloc null test cxgb4/ptp: pass the sign of offset delta in FW CMD hinic: fix wrong para of wait_for_completion_timeout hinic: fix a bug of waitting for IO stopped net: vxge: fix wrong __VA_ARGS__ usage bus: sunxi-rsb: Return correct data when mixing 16-bit and 8-bit reads ARM: dts: sun8i-a83t-tbs-a711: HM5065 doesn't like such a high voltage ANDROID: build.config.allmodconfig: Re-enable XFS_FS FROMGIT: of: property: Add device link support for extcon ANDROID: GKI: arm64: gki_defconfig: enable CONFIG_MM_EVENT_STAT ANDROID: GKI: add fields from per-process mm event tracking feature ANDROID: GKI: fix ABI diffs caused by ION heap and pool vmstat additions UPSTREAM: GKI: panic/reboot: allow specifying reboot_mode for panic only ANDROID: GKI: of: property: Add device link support for phys property ANDROID: GKI: usb: phy: Fix ABI diff for usb_otg_state ANDROID: GKI: usb: phy: Fix ABI diff due to usb_phy.drive_dp_pulse ANDROID: GKI: usb: phy: Fix ABI diff for usb_phy_type and usb_phy.reset ANDROID: gki_defconfig: enable CONFIG_GPIO_SYSFS ANDROID: GKI: qcom: Fix compile issue when setting msm_lmh_dcvs as a module ANDROID: GKI: drivers: cpu_cooling: allow platform freq mitigation ANDROID: GKI: ASoC: Add locking in DAPM widget power update ANDROID: GKI: ASoC: jack: Fix buttons enum value ANDROID: GKI: ALSA: jack: Add support to report second microphone ANDROID: GKI: ALSA: jack: Update supported jack switch types ANDROID: GKI: ALSA: jack: update jack types ANDROID: GKI: Export symbols arm_cpuidle_suspend, cpuidle_dev and cpuidle_register_governor ANDROID: GKI: usb: hcd: Add USB atomic notifier callback for HC died error ANDROID: media: increase video max frame number BACKPORT: nvmem: core: add NVMEM_SYSFS Kconfig UPSTREAM: nvmem: add support for cell info UPSTREAM: nvmem: remove the global cell list UPSTREAM: nvmem: use kref UPSTREAM: nvmem: use list_for_each_entry_safe in nvmem_device_remove_all_cells() UPSTREAM: nvmem: provide nvmem_dev_name() ANDROID: GKI: Bulk ABI update ANDROID: GKI: cpuhotplug: adding hotplug enums for vendor code ANDROID: Incremental fs: Fix create_file performance ANDROID: build.config.common: Add BUILDTOOLS_PREBUILT_BIN UPSTREAM: kheaders: include only headers into kheaders_data.tar.xz UPSTREAM: kheaders: remove meaningless -R option of 'ls' ANDROID: GKI: of: platform: initialize of_reserved_mem ANDROID: driver: gpu: drm: add notifier for panel related events ANDROID: include: drm: support unicasting mipi cmds to dsi ctrls ANDROID: include: drm: increase DRM max property count to 64 BACKPORT: drm: Add HDMI colorspace property ANDROID: drm: edid: add support for additional CEA extension blocks BACKPORT: drm: Parse HDR metadata info from EDID BACKPORT: drm: Add HDR source metadata property BACKPORT: drm/dp_mst: Parse FEC capability on MST ports ANDROID: GKI: ABI update for DRM changes ANDROID: ABI: add missing elf variables to representation ANDROID: GKI: power_supply: Add PROP_MOISTURE_DETECTION_ENABLED ANDROID: include: drm: add the definitions for DP Link Compliance tests ANDROID: drivers: gpu: drm: fix bugs encountered while fuzzing FROMLIST: power_supply: Add additional health properties to the header UPSTREAM: power: supply: core: Update sysfs-class-power ABI document UPSTREAM: Merge remote-tracking branch 'aosp/upstream-f2fs-stable-linux-4.19.y' into android-4.19 (v5.7-rc1) ANDROID: drivers: gpu: drm: add support for secure framebuffer ANDROID: include: uapi: drm: add additional QCOM modifiers ANDROID: drm: dsi: add two DSI mode flags for BLLP ANDROID: include: uapi: drm: add additional drm mode flags UPSTREAM: drm: plug memory leak on drm_setup() failure UPSTREAM: drm: factor out drm_close_helper() function ANDROID: GKI: Bulk ABI update BACKPORT: nl80211: Add per peer statistics to compute FCS error rate ANDROID: GKI: sound: usb: Add snd_usb_enable_audio_stream/find_snd_usb_substream ANDROID: GKI: add dma-buf includes ANDROID: GKI: sched: struct fields for Per-Sched-domain over utilization ANDROID: GKI: Add vendor fields to root_domain ANDROID: gki_defconfig: Enable CONFIG_IRQ_TIME_ACCOUNTING ANDROID: fix allmodconfig build to use the right toolchain ANDROID: fix allmodconfig build to use the right toolchain ANDROID: GKI: Update ABI Revert "UPSTREAM: mm, page_alloc: spread allocations across zones before introducing fragmentation" Revert "UPSTREAM: mm: use alloc_flags to record if kswapd can wake" Revert "BACKPORT: mm: move zone watermark accesses behind an accessor" Revert "BACKPORT: mm: reclaim small amounts of memory when an external fragmentation event occurs" Revert "BACKPORT: mm, compaction: be selective about what pageblocks to clear skip hints" ANDROID: GKI: panic: add vendor callback function in panic() UPSTREAM: GKI: thermal: make device_register's type argument const ANDROID: GKI: add base.h include to match MODULE_VERSIONS ANDROID: update the ABI based on the new whitelist ANDROID: GKI: fdt: export symbols required by modules ANDROID: GKI: drivers: of: Add APIs to find DDR device rank, HBB ANDROID: GKI: security: Add mmap export symbols for modules ANDROID: GKI: arch: add stub symbols for boot_reason and cold_boot ANDROID: GKI: USB: Fix ABI diff for struct usb_bus ANDROID: GKI: USB: Resolve ABI diff for usb_gadget and usb_gadget_ops ANDROID: GKI: add hidden V4L2_MEM2MEM_DEV ANDROID: GKI: enable VIDEO_V4L2_SUBDEV_API ANDROID: GKI: export symbols from abi_gki_aarch64_qcom_whitelist ANDROID: Update the whitelist for qcom SoCs ANDROID: Incremental fs: Fix compound page usercopy crash ANDROID: Incremental fs: Clean up incfs_test build process ANDROID: Incremental fs: make remount log buffer change atomic ANDROID: Incremental fs: Optimize get_filled_block ANDROID: Incremental fs: Fix mislabeled __user ptrs ANDROID: Incremental fs: Use 64-bit int for file_size when writing hash blocks Linux 4.19.115 drm/msm: Use the correct dma_sync calls in msm_gem drm_dp_mst_topology: fix broken drm_dp_sideband_parse_remote_dpcd_read() usb: dwc3: don't set gadget->is_otg flag rpmsg: glink: Remove chunk size word align warning arm64: Fix size of __early_cpu_boot_status drm/msm: stop abusing dma_map/unmap for cache clk: qcom: rcg: Return failure for RCG update fbcon: fix null-ptr-deref in fbcon_switch RDMA/cm: Update num_paths in cma_resolve_iboe_route error flow Bluetooth: RFCOMM: fix ODEBUG bug in rfcomm_dev_ioctl RDMA/cma: Teach lockdep about the order of rtnl and lock RDMA/ucma: Put a lock around every call to the rdma_cm layer ceph: canonicalize server path in place ceph: remove the extra slashes in the server path IB/hfi1: Fix memory leaks in sysfs registration and unregistration IB/hfi1: Call kobject_put() when kobject_init_and_add() fails ASoC: jz4740-i2s: Fix divider written at incorrect offset in register hwrng: imx-rngc - fix an error path tools/accounting/getdelays.c: fix netlink attribute length usb: dwc3: gadget: Wrap around when skip TRBs random: always use batched entropy for get_random_u{32,64} mlxsw: spectrum_flower: Do not stop at FLOW_ACTION_VLAN_MANGLE slcan: Don't transmit uninitialized stack data in padding net: stmmac: dwmac1000: fix out-of-bounds mac address reg setting net: phy: micrel: kszphy_resume(): add delay after genphy_resume() before accessing PHY registers net: dsa: bcm_sf2: Ensure correct sub-node is parsed net: dsa: bcm_sf2: Do not register slave MDIO bus with OF ipv6: don't auto-add link-local address to lag ports mm: mempolicy: require at least one nodeid for MPOL_PREFERRED include/linux/notifier.h: SRCU: fix ctags bitops: protect variables in set_mask_bits() macro padata: always acquire cpu_hotplug_lock before pinst->lock net: Fix Tx hash bound checking rxrpc: Fix sendmsg(MSG_WAITALL) handling ALSA: hda/ca0132 - Add Recon3Di quirk to handle integrated sound on EVGA X99 Classified motherboard power: supply: axp288_charger: Add special handling for HP Pavilion x2 10 extcon: axp288: Add wakeup support mei: me: add cedar fork device ids coresight: do not use the BIT() macro in the UAPI header misc: pci_endpoint_test: Avoid using module parameter to determine irqtype misc: pci_endpoint_test: Fix to support > 10 pci-endpoint-test devices misc: rtsx: set correct pcr_ops for rts522A media: rc: IR signal for Panasonic air conditioner too long drm/etnaviv: replace MMU flush marker with flush sequence tools/power turbostat: Fix missing SYS_LPI counter on some Chromebooks tools/power turbostat: Fix gcc build warnings drm/amdgpu: fix typo for vcn1 idle check initramfs: restore default compression behavior drm/bochs: downgrade pci_request_region failure from error to warning drm/amd/display: Add link_rate quirk for Apple 15" MBP 2017 nvme-rdma: Avoid double freeing of async event data sctp: fix possibly using a bad saddr with a given dst sctp: fix refcount bug in sctp_wfree net, ip_tunnel: fix interface lookup with no key ipv4: fix a RCU-list lock in fib_triestat_seq_show ANDROID: GKI: export symbols required by SPECTRA_CAMERA ANDROID: GKI: ARM/ARM64: Introduce arch_read_hardware_id ANDROID: GKI: drivers: base: soc: export symbols for socinfo ANDROID: GKI: Update ABI ANDROID: GKI: ASoC: msm: fix integer overflow for long duration offload playback ANDROID: GKI: Bulk ABI update Revert "ANDROID: GKI: mm: add struct/enum fields for SPECULATIVE_PAGE_FAULTS" ANDROID: GKI: Revert "arm64: kill flush_cache_all()" ANDROID: GKI: Revert "arm64: Remove unused macros from assembler.h" ANDROID: GKI: kernel/dma, mm/cma: Export symbols needed by vendor modules ANDROID: GKI: mm: Export symbols __next_zones_zonelist and zone_watermark_ok_safe ANDROID: GKI: mm/memblock: export memblock_overlaps_memory ANDROID: GKI: net, skbuff: export symbols needed by vendor drivers ANDROID: GKI: Add stub __cpu_isolated_mask symbol ANDROID: GKI: sched: stub sched_isolate symbols ANDROID: GKI: export saved_command_line ANDROID: GKI: Update ABI ANDROID: GKI: ASoC: core: Update ALSA core to issue restart in underrun. ANDROID: GKI: SoC: pcm: Add a restart callback field to struct snd_pcm_ops ANDROID: GKI: SoC: pcm: Add fields to struct snd_pcm_ops and struct snd_soc_component_driver ANDROID: GKI: ASoC: core: Add compat_ioctl callback to struct snd_pcm_ops ANDROID: GKI: ALSA: core: modify, rename and export create_subdir API ANDROID: GKI: usb: Add helper API to issue stop endpoint command ANDROID: GKI: Thermal: thermal_zone_get_cdev_by_name added ANDROID: GKI: add missing exports for CONFIG_ARM_SMMU=m ANDROID: power: wakeup_reason: wake reason enhancements BACKPORT: FROMGIT: kbuild: mkcompile_h: Include $LD version in /proc/version ANDROID: GKI: kernel: Export symbols needed by msm_minidump.ko and minidump_log.ko ubifs: wire up FS_IOC_GET_ENCRYPTION_NONCE f2fs: wire up FS_IOC_GET_ENCRYPTION_NONCE ext4: wire up FS_IOC_GET_ENCRYPTION_NONCE fscrypt: add FS_IOC_GET_ENCRYPTION_NONCE ioctl ANDROID: Bulk update the ABI xml ANDROID: gki_defconfig: add CONFIG_IPV6_SUBTREES ANDROID: GKI: arm64: reserve space in cpu_hwcaps and cpu_hwcap_keys arrays ANDROID: GKI: of: reserved_mem: Fix kmemleak crash on no-map region ANDROID: GKI: sched: add task boost vendor fields to task_struct ANDROID: GKI: mm: add rss counter for unreclaimable pages ANDROID: GKI: irqdomain: add bus token DOMAIN_BUS_WAKEUP ANDROID: GKI: arm64: fault: do_tlb_conf_fault_cb register fault callback ANDROID: GKI: QoS: Enhance framework to support cpu/irq specific QoS requests ANDROID: GKI: Bulk ABI update ANDROID: GKI: PM/devfreq: Do not switch governors from sysfs when device is suspended ANDROID: GKI: PM / devfreq: Fix race condition between suspend/resume and governor_store ANDROID: GKI: PM / devfreq: Introduce a sysfs lock ANDROID: GKI: regmap: irq: Add support to clear ack registers ANDROID: GKI: Remove SCHED_AUTOGROUP ANDROID: ignore compiler tag __must_check for GENKSYMS ANDROID: GKI: Bulk update ABI ANDROID: GKI: Fix ABI diff for struct thermal_cooling_device_ops ANDROID: GKI: ASoC: soc-core: export function to find components ANDROID: GKI: thermal: thermal_sys: Add configurable thermal trip points. ANDROID: fscrypt: fall back to filesystem-layer crypto when needed ANDROID: block: require drivers to declare supported crypto key type(s) ANDROID: block: make blk_crypto_start_using_mode() properly check for support ANDROID: GKI: power: supply: format regression ANDROID: GKI: kobject: increase number of kobject uevent pointers to 64 ANDROID: GKI: drivers: video: backlight: Fix ABI diff for struct backlight_device ANDROID: GKI: usb: xhci: Add support for secondary interrupters ANDROID: GKI: usb: host: xhci: Add support for usb core indexing ANDROID: gki_defconfig: enable USB_XHCI_HCD ANDROID: gki_defconfig: enable CONFIG_BRIDGE ANDROID: GKI: Update ABI report ANDROID: GKI: arm64: smp: Add set_update_ipi_history_callback ANDROID: kbuild: ensure __cfi_check is correctly aligned f2fs: keep inline_data when compression conversion f2fs: fix to disable compression on directory f2fs: add missing CONFIG_F2FS_FS_COMPRESSION f2fs: switch discard_policy.timeout to bool type f2fs: fix to verify tpage before releasing in f2fs_free_dic() f2fs: show compression in statx f2fs: clean up dic->tpages assignment f2fs: compress: support zstd compress algorithm f2fs: compress: add .{init,destroy}_decompress_ctx callback f2fs: compress: fix to call missing destroy_compress_ctx() f2fs: change default compression algorithm f2fs: clean up {cic,dic}.ref handling f2fs: fix to use f2fs_readpage_limit() in f2fs_read_multi_pages() f2fs: xattr.h: Make stub helpers inline f2fs: fix to avoid double unlock f2fs: fix potential .flags overflow on 32bit architecture f2fs: fix NULL pointer dereference in f2fs_verity_work() f2fs: fix to clear PG_error if fsverity failed f2fs: don't call fscrypt_get_encryption_info() explicitly in f2fs_tmpfile() f2fs: don't trigger data flush in foreground operation f2fs: fix NULL pointer dereference in f2fs_write_begin() f2fs: clean up f2fs_may_encrypt() f2fs: fix to avoid potential deadlock f2fs: don't change inode status under page lock f2fs: fix potential deadlock on compressed quota file f2fs: delete DIO read lock f2fs: don't mark compressed inode dirty during f2fs_iget() f2fs: fix to account compressed blocks in f2fs_compressed_blocks() f2fs: xattr.h: Replace zero-length array with flexible-array member f2fs: fix to update f2fs_super_block fields under sb_lock f2fs: Add a new CP flag to help fsck fix resize SPO issues f2fs: Fix mount failure due to SPO after a successful online resize FS f2fs: use kmem_cache pool during inline xattr lookups f2fs: skip migration only when BG_GC is called f2fs: fix to show tracepoint correctly f2fs: avoid __GFP_NOFAIL in f2fs_bio_alloc f2fs: introduce F2FS_IOC_GET_COMPRESS_BLOCKS f2fs: fix to avoid triggering IO in write path f2fs: add prefix for f2fs slab cache name f2fs: introduce DEFAULT_IO_TIMEOUT f2fs: skip GC when section is full f2fs: add migration count iff migration happens f2fs: clean up bggc mount option f2fs: clean up lfs/adaptive mount option f2fs: fix to show norecovery mount option f2fs: clean up parameter of macro XATTR_SIZE() f2fs: clean up codes with {f2fs_,}data_blkaddr() f2fs: show mounted time f2fs: Use scnprintf() for avoiding potential buffer overflow f2fs: allow to clear F2FS_COMPR_FL flag f2fs: fix to check dirty pages during compressed inode conversion f2fs: fix to account compressed inode correctly f2fs: fix wrong check on F2FS_IOC_FSSETXATTR f2fs: fix to avoid use-after-free in f2fs_write_multi_pages() f2fs: fix to avoid using uninitialized variable f2fs: fix inconsistent comments f2fs: remove i_sem lock coverage in f2fs_setxattr() f2fs: cover last_disk_size update with spinlock f2fs: fix to check i_compr_blocks correctly FROMLIST: kmod: make request_module() return an error when autoloading is disabled ANDROID: GKI: Update ABI report ANDROID: GKI: ARM64: dma-mapping: export symbol arch_setup_dma_ops ANDROID: GKI: ARM: dma-mapping: export symbol arch_setup_dma_ops ANDROID: GKI: ASoC: dapm: Avoid static route b/w cpu and codec dai ANDROID: GKI: ASoC: pcm: Add support for hostless playback/capture ANDROID: GKI: ASoC: core - add hostless DAI support ANDROID: GKI: drivers: thermal: Resolve ABI diff for struct thermal_zone_device_ops ANDROID: GKI: drivers: thermal: Add support for getting trip temperature ANDROID: GKI: Add functions of_thermal_handle_trip/of_thermal_handle_trip_temp ANDROID: GKI: drivers: thermal: Add post suspend evaluate flag to thermal zone devicetree UPSTREAM: loop: Only freeze block queue when needed. UPSTREAM: loop: Only change blocksize when needed. ANDROID: Fix wq fp check for CFI builds ANDROID: GKI: update abi definition after CONFIG_DEBUG_LIST was enabled ANDROID: gki_defconfig: enable CONFIG_DEBUG_LIST ANDROID: GKI: Update ABI definition ANDROID: GKI: remove condition causing sk_buff struct ABI differences ANDROID: GKI: Export symbol arch_timer_mem_get_cval ANDROID: GKI: pwm: core: Add option to config PWM duty/period with u64 data length ANDROID: Update ABI whitelist for qcom SoCs ANDROID: Incremental fs: Fix remount ANDROID: Incremental fs: Protect get_fill_block, and add a field ANDROID: Incremental fs: Fix crash polling 0 size read_log ANDROID: Incremental fs: get_filled_blocks: better index_out ANDROID: GKI: of: property: Add device links support for "qcom,wrapper-dev" ANDROID: GKI: update abi definitions due to recent changes ANDROID: GKI: clk: Initialize in stack clk_init_data to 0 in all drivers ANDROID: GKI: drivers: clksource: Add API to return cval ANDROID: GKI: clk: Add support for voltage voting ANDROID: GKI: kernel: Export task and IRQ affinity symbols ANDROID: GKI: regulator: core: Add support for regulator providers with sync state ANDROID: GKI: regulator: Call proxy-consumer functions for each regulator registered ANDROID: GKI: regulator: Add proxy consumer driver ANDROID: GKI: regulator: core: allow long device tree supply regulator property names ANDROID: GKI: Revert "regulator: Enable supply regulator if child rail is enabled." ANDROID: GKI: regulator: Remove redundant set_mode call in drms_uA_update ANDROID: GKI: net: Add the get current NAPI context API ANDROID: GKI: remove DRM_KMS_CMA_HELPER from GKI configuration ANDROID: GKI: edac: Fix ABI diffs in edac_device_ctl_info struct ANDROID: GKI: pwm: Add different PWM output types support UPSTREAM: cfg80211: Authentication offload to user space in AP mode Linux 4.19.114 arm64: dts: ls1046ardb: set RGMII interfaces to RGMII_ID mode arm64: dts: ls1043a-rdb: correct RGMII delay mode to rgmii-id ARM: dts: N900: fix onenand timings ARM: dts: imx6: phycore-som: fix arm and soc minimum voltage ARM: bcm2835-rpi-zero-w: Add missing pinctrl name ARM: dts: oxnas: Fix clear-mask property perf map: Fix off by one in strncpy() size argument arm64: alternative: fix build with clang integrated assembler net: ks8851-ml: Fix IO operations, again gpiolib: acpi: Add quirk to ignore EC wakeups on HP x2 10 CHT + AXP288 model bpf: Explicitly memset some bpf info structures declared on the stack bpf: Explicitly memset the bpf_attr structure platform/x86: pmc_atom: Add Lex 2I385SW to critclk_systems DMI table vt: vt_ioctl: fix use-after-free in vt_in_use() vt: vt_ioctl: fix VT_DISALLOCATE freeing in-use virtual console vt: vt_ioctl: remove unnecessary console allocation checks vt: switch vt_dont_switch to bool vt: ioctl, switch VT_IS_IN_USE and VT_BUSY to inlines vt: selection, introduce vc_is_sel mac80211: fix authentication with iwlwifi/mvm mac80211: Check port authorization in the ieee80211_tx_dequeue() case media: xirlink_cit: add missing descriptor sanity checks media: stv06xx: add missing descriptor sanity checks media: dib0700: fix rc endpoint lookup media: ov519: add missing endpoint sanity checks libfs: fix infoleak in simple_attr_read() ahci: Add Intel Comet Lake H RAID PCI ID staging: wlan-ng: fix use-after-free Read in hfa384x_usbin_callback staging: wlan-ng: fix ODEBUG bug in prism2sta_disconnect_usb staging: rtl8188eu: Add ASUS USB-N10 Nano B1 to device table media: usbtv: fix control-message timeouts media: flexcop-usb: fix endpoint sanity check usb: musb: fix crash with highmen PIO and usbmon USB: serial: io_edgeport: fix slab-out-of-bounds read in edge_interrupt_callback USB: cdc-acm: restore capability check order USB: serial: option: add Wistron Neweb D19Q1 USB: serial: option: add BroadMobi BM806U USB: serial: option: add support for ASKEY WWHC050 mac80211: set IEEE80211_TX_CTRL_PORT_CTRL_PROTO for nl80211 TX mac80211: add option for setting control flags Revert "r8169: check that Realtek PHY driver module is loaded" vti6: Fix memory leak of skb if input policy check fails bpf/btf: Fix BTF verification of enum members in struct/union netfilter: nft_fwd_netdev: validate family and chain type netfilter: flowtable: reload ip{v6}h in nf_flow_tuple_ip{v6} afs: Fix some tracing details xfrm: policy: Fix doulbe free in xfrm_policy_timer xfrm: add the missing verify_sec_ctx_len check in xfrm_add_acquire xfrm: fix uctx len check in verify_sec_ctx_len RDMA/mlx5: Block delay drop to unprivileged users vti[6]: fix packet tx through bpf_redirect() in XinY cases xfrm: handle NETDEV_UNREGISTER for xfrm device genirq: Fix reference leaks on irq affinity notifiers RDMA/core: Ensure security pkey modify is not lost gpiolib: acpi: Add quirk to ignore EC wakeups on HP x2 10 BYT + AXP288 model gpiolib: acpi: Rework honor_wakeup option into an ignore_wake option gpiolib: acpi: Correct comment for HP x2 10 honor_wakeup quirk mac80211: mark station unauthorized before key removal nl80211: fix NL80211_ATTR_CHANNEL_WIDTH attribute type scsi: sd: Fix optimal I/O size for devices that change reported values scripts/dtc: Remove redundant YYLOC global declaration tools: Let O= makes handle a relative path with -C option perf probe: Do not depend on dwfl_module_addrsym() ARM: dts: omap5: Add bus_dma_limit for L3 bus ARM: dts: dra7: Add bus_dma_limit for L3 bus ceph: check POOL_FLAG_FULL/NEARFULL in addition to OSDMAP_FULL/NEARFULL Input: avoid BIT() macro usage in the serio.h UAPI header Input: synaptics - enable RMI on HP Envy 13-ad105ng Input: raydium_i2c_ts - fix error codes in raydium_i2c_boot_trigger() i2c: hix5hd2: add missed clk_disable_unprepare in remove ftrace/x86: Anotate text_mutex split between ftrace_arch_code_modify_post_process() and ftrace_arch_code_modify_prepare() sxgbe: Fix off by one in samsung driver strncpy size arg dpaa_eth: Remove unnecessary boolean expression in dpaa_get_headroom mac80211: Do not send mesh HWMP PREQ if HWMP is disabled scsi: ipr: Fix softlockup when rescanning devices in petitboot s390/qeth: handle error when backing RX buffer fsl/fman: detect FMan erratum A050385 arm64: dts: ls1043a: FMan erratum A050385 dt-bindings: net: FMan erratum A050385 cgroup1: don't call release_agent when it is "" drivers/of/of_mdio.c:fix of_mdiobus_register() cpupower: avoid multiple definition with gcc -fno-common nfs: add minor version to nfs_server_key for fscache cgroup-v1: cgroup_pidlist_next should update position index hsr: set .netnsok flag hsr: add restart routine into hsr_get_node_list() hsr: use rcu_read_lock() in hsr_get_node_{list/status}() vxlan: check return value of gro_cells_init() tcp: repair: fix TCP_QUEUE_SEQ implementation r8169: re-enable MSI on RTL8168c net: phy: mdio-mux-bcm-iproc: check clk_prepare_enable() return value net: dsa: mt7530: Change the LINK bit to reflect the link status net: ip_gre: Accept IFLA_INFO_DATA-less configuration net: ip_gre: Separate ERSPAN newlink / changelink callbacks bnxt_en: Reset rings if ring reservation fails during open() bnxt_en: fix memory leaks in bnxt_dcbnl_ieee_getets() slcan: not call free_netdev before rtnl_unlock in slcan_open NFC: fdp: Fix a signedness bug in fdp_nci_send_patch() net: stmmac: dwmac-rk: fix error path in rk_gmac_probe net_sched: keep alloc_hash updated after hash allocation net_sched: cls_route: remove the right filter from hashtable net: qmi_wwan: add support for ASKEY WWHC050 net/packet: tpacket_rcv: avoid a producer race condition net: mvneta: Fix the case where the last poll did not process all rx net: dsa: Fix duplicate frames flooded by learning net: cbs: Fix software cbs to consider packet sending time mlxsw: spectrum_mr: Fix list iteration in error path macsec: restrict to ethernet devices hsr: fix general protection fault in hsr_addr_is_self() geneve: move debug check after netdev unregister Revert "drm/dp_mst: Skip validating ports during destruction, just ref" mmc: sdhci-tegra: Fix busy detection by enabling MMC_CAP_NEED_RSP_BUSY mmc: sdhci-omap: Fix busy detection by enabling MMC_CAP_NEED_RSP_BUSY mmc: core: Respect MMC_CAP_NEED_RSP_BUSY for eMMC sleep command mmc: core: Respect MMC_CAP_NEED_RSP_BUSY for erase/trim/discard mmc: core: Allow host controllers to require R1B for CMD6 f2fs: fix to avoid potential deadlock f2fs: add missing function name in kernel message f2fs: recycle unused compress_data.chksum feild f2fs: fix to avoid NULL pointer dereference f2fs: fix leaking uninitialized memory in compressed clusters f2fs: fix the panic in do_checkpoint() f2fs: fix to wait all node page writeback mm/swapfile.c: move inode_lock out of claim_swapfile fscrypt: don't evict dirty inodes after removing key Conflicts: Documentation/arm64/silicon-errata.txt Documentation/devicetree/bindings Documentation/devicetree/bindings/net/fsl-fman.txt arch/arm/kernel/setup.c arch/arm/kernel/smp.c arch/arm/mm/dma-mapping.c arch/arm64/Kconfig arch/arm64/Makefile arch/arm64/include/asm/cpucaps.h arch/arm64/include/asm/cputype.h arch/arm64/include/asm/proc-fns.h arch/arm64/include/asm/traps.h arch/arm64/kernel/arm64ksyms.c arch/arm64/kernel/cpu_errata.c arch/arm64/kernel/setup.c arch/arm64/kernel/smp.c arch/arm64/mm/dma-mapping.c arch/arm64/mm/fault.c arch/arm64/mm/proc.S drivers/base/power/wakeup.c drivers/clk/clk.c drivers/clk/qcom/clk-rcg2.c drivers/clocksource/arm_arch_timer.c drivers/devfreq/devfreq.c drivers/devfreq/governor_simpleondemand.c drivers/dma-buf/dma-buf.c drivers/extcon/extcon.c drivers/gpu/Makefile drivers/gpu/drm/drm_connector.c drivers/gpu/drm/drm_dp_mst_topology.c drivers/gpu/drm/drm_edid.c drivers/gpu/drm/drm_file.c drivers/gpu/drm/drm_panel.c drivers/gpu/drm/drm_property.c drivers/iommu/Kconfig drivers/iommu/Makefile drivers/iommu/arm-smmu.c drivers/iommu/dma-iommu.c drivers/iommu/dma-mapping-fast.c drivers/iommu/io-pgtable-arm.c drivers/iommu/io-pgtable-fast.c drivers/iommu/io-pgtable.c drivers/iommu/iommu.c drivers/irqchip/irq-gic-v3.c drivers/media/v4l2-core/v4l2-ioctl.c drivers/mmc/core/Kconfig drivers/mmc/core/block.c drivers/mmc/core/queue.c drivers/mmc/host/cqhci.c drivers/mmc/host/sdhci-msm.c drivers/net/wireless/ath/wil6210/interrupt.c drivers/net/wireless/ath/wil6210/main.c drivers/net/wireless/ath/wil6210/wil6210.h drivers/net/wireless/ath/wil6210/wmi.c drivers/nvmem/core.c drivers/nvmem/nvmem-sysfs.c drivers/of/fdt.c drivers/power/supply/power_supply_sysfs.c drivers/pwm/sysfs.c drivers/regulator/core.c drivers/scsi/sd.c drivers/scsi/ufs/ufshcd.c drivers/tty/serial/Kconfig drivers/tty/serial/Makefile drivers/usb/common/common.c fs/crypto/crypto.c fs/f2fs/checkpoint.c fs/f2fs/f2fs.h include/drm/drm_connector.h include/drm/drm_dp_mst_helper.h include/drm/drm_panel.h include/linux/clk-provider.h include/linux/dma-buf.h include/linux/dma-mapping-fast.h include/linux/dma-mapping.h include/linux/extcon.h include/linux/io-pgtable.h include/linux/iommu.h include/linux/kobject.h include/linux/mm.h include/linux/mm_types.h include/linux/mmc/host.h include/linux/netdevice.h include/linux/power_supply.h include/linux/pwm.h include/linux/regulator/driver.h include/linux/thermal.h include/linux/vm_event_item.h include/net/cfg80211.h include/scsi/scsi_device.h include/sound/pcm.h include/sound/soc.h include/uapi/drm/drm_mode.h include/uapi/linux/coresight-stm.h include/uapi/linux/ip.h include/uapi/linux/nl80211.h include/uapi/linux/videodev2.h include/uapi/sound/compress_offload.h kernel/dma/coherent.c kernel/dma/mapping.c kernel/panic.c kernel/power/qos.c kernel/sched/sched.h mm/Kconfig mm/filemap.c mm/swapfile.c mm/vmalloc.c mm/vmstat.c net/qrtr/qrtr.c net/wireless/nl80211.c net/wireless/scan.c sound/core/compress_offload.c sound/soc/soc-core.c sound/usb/card.c sound/usb/pcm.c sound/usb/pcm.h sound/usb/usbaudio.h Fixed build errors: drivers/base/power/main.c drivers/thermal/thermal_core.c drivers/cpuidle/lpm-levels.c include/soc/qcom/lpm_levels.h Change-Id: Idf25b239f53681bdfa2ef371a91720fadf1a3f01 Signed-off-by: Srinivasarao P <spathi@codeaurora.org>
4104 lines
106 KiB
C
4104 lines
106 KiB
C
/*
|
|
* Resizable virtual memory filesystem for Linux.
|
|
*
|
|
* Copyright (C) 2000 Linus Torvalds.
|
|
* 2000 Transmeta Corp.
|
|
* 2000-2001 Christoph Rohland
|
|
* 2000-2001 SAP AG
|
|
* 2002 Red Hat Inc.
|
|
* Copyright (C) 2002-2011 Hugh Dickins.
|
|
* Copyright (C) 2011 Google Inc.
|
|
* Copyright (C) 2002-2005 VERITAS Software Corporation.
|
|
* Copyright (C) 2004 Andi Kleen, SuSE Labs
|
|
*
|
|
* Extended attribute support for tmpfs:
|
|
* Copyright (c) 2004, Luke Kenneth Casson Leighton <lkcl@lkcl.net>
|
|
* Copyright (c) 2004 Red Hat, Inc., James Morris <jmorris@redhat.com>
|
|
*
|
|
* tiny-shmem:
|
|
* Copyright (c) 2004, 2008 Matt Mackall <mpm@selenic.com>
|
|
*
|
|
* This file is released under the GPL.
|
|
*/
|
|
|
|
#include <linux/fs.h>
|
|
#include <linux/init.h>
|
|
#include <linux/vfs.h>
|
|
#include <linux/mount.h>
|
|
#include <linux/ramfs.h>
|
|
#include <linux/pagemap.h>
|
|
#include <linux/file.h>
|
|
#include <linux/mm.h>
|
|
#include <linux/random.h>
|
|
#include <linux/sched/signal.h>
|
|
#include <linux/export.h>
|
|
#include <linux/swap.h>
|
|
#include <linux/uio.h>
|
|
#include <linux/khugepaged.h>
|
|
#include <linux/hugetlb.h>
|
|
|
|
#include <asm/tlbflush.h> /* for arch/microblaze update_mmu_cache() */
|
|
|
|
static struct vfsmount *shm_mnt;
|
|
|
|
#ifdef CONFIG_SHMEM
|
|
/*
|
|
* This virtual memory filesystem is heavily based on the ramfs. It
|
|
* extends ramfs by the ability to use swap and honor resource limits
|
|
* which makes it a completely usable filesystem.
|
|
*/
|
|
|
|
#include <linux/xattr.h>
|
|
#include <linux/exportfs.h>
|
|
#include <linux/posix_acl.h>
|
|
#include <linux/posix_acl_xattr.h>
|
|
#include <linux/mman.h>
|
|
#include <linux/string.h>
|
|
#include <linux/slab.h>
|
|
#include <linux/backing-dev.h>
|
|
#include <linux/shmem_fs.h>
|
|
#include <linux/writeback.h>
|
|
#include <linux/blkdev.h>
|
|
#include <linux/pagevec.h>
|
|
#include <linux/percpu_counter.h>
|
|
#include <linux/falloc.h>
|
|
#include <linux/splice.h>
|
|
#include <linux/security.h>
|
|
#include <linux/swapops.h>
|
|
#include <linux/mempolicy.h>
|
|
#include <linux/namei.h>
|
|
#include <linux/ctype.h>
|
|
#include <linux/migrate.h>
|
|
#include <linux/highmem.h>
|
|
#include <linux/seq_file.h>
|
|
#include <linux/magic.h>
|
|
#include <linux/syscalls.h>
|
|
#include <linux/fcntl.h>
|
|
#include <uapi/linux/memfd.h>
|
|
#include <linux/userfaultfd_k.h>
|
|
#include <linux/rmap.h>
|
|
#include <linux/uuid.h>
|
|
|
|
#include <linux/uaccess.h>
|
|
#include <asm/pgtable.h>
|
|
|
|
#include "internal.h"
|
|
|
|
#define BLOCKS_PER_PAGE (PAGE_SIZE/512)
|
|
#define VM_ACCT(size) (PAGE_ALIGN(size) >> PAGE_SHIFT)
|
|
|
|
/* Pretend that each entry is of this size in directory's i_size */
|
|
#define BOGO_DIRENT_SIZE 20
|
|
|
|
/* Symlink up to this size is kmalloc'ed instead of using a swappable page */
|
|
#define SHORT_SYMLINK_LEN 128
|
|
|
|
/*
|
|
* shmem_fallocate communicates with shmem_fault or shmem_writepage via
|
|
* inode->i_private (with i_mutex making sure that it has only one user at
|
|
* a time): we would prefer not to enlarge the shmem inode just for that.
|
|
*/
|
|
struct shmem_falloc {
|
|
wait_queue_head_t *waitq; /* faults into hole wait for punch to end */
|
|
pgoff_t start; /* start of range currently being fallocated */
|
|
pgoff_t next; /* the next page offset to be fallocated */
|
|
pgoff_t nr_falloced; /* how many new pages have been fallocated */
|
|
pgoff_t nr_unswapped; /* how often writepage refused to swap out */
|
|
};
|
|
|
|
#ifdef CONFIG_TMPFS
|
|
static unsigned long shmem_default_max_blocks(void)
|
|
{
|
|
return totalram_pages / 2;
|
|
}
|
|
|
|
static unsigned long shmem_default_max_inodes(void)
|
|
{
|
|
return min(totalram_pages - totalhigh_pages, totalram_pages / 2);
|
|
}
|
|
#endif
|
|
|
|
static bool shmem_should_replace_page(struct page *page, gfp_t gfp);
|
|
static int shmem_replace_page(struct page **pagep, gfp_t gfp,
|
|
struct shmem_inode_info *info, pgoff_t index);
|
|
static int shmem_getpage_gfp(struct inode *inode, pgoff_t index,
|
|
struct page **pagep, enum sgp_type sgp,
|
|
gfp_t gfp, struct vm_area_struct *vma,
|
|
struct vm_fault *vmf, vm_fault_t *fault_type);
|
|
|
|
int shmem_getpage(struct inode *inode, pgoff_t index,
|
|
struct page **pagep, enum sgp_type sgp)
|
|
{
|
|
return shmem_getpage_gfp(inode, index, pagep, sgp,
|
|
mapping_gfp_mask(inode->i_mapping), NULL, NULL, NULL);
|
|
}
|
|
|
|
static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
|
|
{
|
|
return sb->s_fs_info;
|
|
}
|
|
|
|
/*
|
|
* shmem_file_setup pre-accounts the whole fixed size of a VM object,
|
|
* for shared memory and for shared anonymous (/dev/zero) mappings
|
|
* (unless MAP_NORESERVE and sysctl_overcommit_memory <= 1),
|
|
* consistent with the pre-accounting of private mappings ...
|
|
*/
|
|
static inline int shmem_acct_size(unsigned long flags, loff_t size)
|
|
{
|
|
return (flags & VM_NORESERVE) ?
|
|
0 : security_vm_enough_memory_mm(current->mm, VM_ACCT(size));
|
|
}
|
|
|
|
static inline void shmem_unacct_size(unsigned long flags, loff_t size)
|
|
{
|
|
if (!(flags & VM_NORESERVE))
|
|
vm_unacct_memory(VM_ACCT(size));
|
|
}
|
|
|
|
static inline int shmem_reacct_size(unsigned long flags,
|
|
loff_t oldsize, loff_t newsize)
|
|
{
|
|
if (!(flags & VM_NORESERVE)) {
|
|
if (VM_ACCT(newsize) > VM_ACCT(oldsize))
|
|
return security_vm_enough_memory_mm(current->mm,
|
|
VM_ACCT(newsize) - VM_ACCT(oldsize));
|
|
else if (VM_ACCT(newsize) < VM_ACCT(oldsize))
|
|
vm_unacct_memory(VM_ACCT(oldsize) - VM_ACCT(newsize));
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* ... whereas tmpfs objects are accounted incrementally as
|
|
* pages are allocated, in order to allow large sparse files.
|
|
* shmem_getpage reports shmem_acct_block failure as -ENOSPC not -ENOMEM,
|
|
* so that a failure on a sparse tmpfs mapping will give SIGBUS not OOM.
|
|
*/
|
|
static inline int shmem_acct_block(unsigned long flags, long pages)
|
|
{
|
|
if (!(flags & VM_NORESERVE))
|
|
return 0;
|
|
|
|
return security_vm_enough_memory_mm(current->mm,
|
|
pages * VM_ACCT(PAGE_SIZE));
|
|
}
|
|
|
|
static inline void shmem_unacct_blocks(unsigned long flags, long pages)
|
|
{
|
|
if (flags & VM_NORESERVE)
|
|
vm_unacct_memory(pages * VM_ACCT(PAGE_SIZE));
|
|
}
|
|
|
|
static inline bool shmem_inode_acct_block(struct inode *inode, long pages)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
|
|
if (shmem_acct_block(info->flags, pages))
|
|
return false;
|
|
|
|
if (sbinfo->max_blocks) {
|
|
if (percpu_counter_compare(&sbinfo->used_blocks,
|
|
sbinfo->max_blocks - pages) > 0)
|
|
goto unacct;
|
|
percpu_counter_add(&sbinfo->used_blocks, pages);
|
|
}
|
|
|
|
return true;
|
|
|
|
unacct:
|
|
shmem_unacct_blocks(info->flags, pages);
|
|
return false;
|
|
}
|
|
|
|
static inline void shmem_inode_unacct_blocks(struct inode *inode, long pages)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
|
|
if (sbinfo->max_blocks)
|
|
percpu_counter_sub(&sbinfo->used_blocks, pages);
|
|
shmem_unacct_blocks(info->flags, pages);
|
|
}
|
|
|
|
static const struct super_operations shmem_ops;
|
|
static const struct address_space_operations shmem_aops;
|
|
static const struct file_operations shmem_file_operations;
|
|
static const struct inode_operations shmem_inode_operations;
|
|
static const struct inode_operations shmem_dir_inode_operations;
|
|
static const struct inode_operations shmem_special_inode_operations;
|
|
static const struct vm_operations_struct shmem_vm_ops;
|
|
static struct file_system_type shmem_fs_type;
|
|
|
|
bool vma_is_shmem(struct vm_area_struct *vma)
|
|
{
|
|
return vma->vm_ops == &shmem_vm_ops;
|
|
}
|
|
|
|
static LIST_HEAD(shmem_swaplist);
|
|
static DEFINE_MUTEX(shmem_swaplist_mutex);
|
|
|
|
static int shmem_reserve_inode(struct super_block *sb)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
if (sbinfo->max_inodes) {
|
|
spin_lock(&sbinfo->stat_lock);
|
|
if (!sbinfo->free_inodes) {
|
|
spin_unlock(&sbinfo->stat_lock);
|
|
return -ENOSPC;
|
|
}
|
|
sbinfo->free_inodes--;
|
|
spin_unlock(&sbinfo->stat_lock);
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
static void shmem_free_inode(struct super_block *sb)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
if (sbinfo->max_inodes) {
|
|
spin_lock(&sbinfo->stat_lock);
|
|
sbinfo->free_inodes++;
|
|
spin_unlock(&sbinfo->stat_lock);
|
|
}
|
|
}
|
|
|
|
/**
|
|
* shmem_recalc_inode - recalculate the block usage of an inode
|
|
* @inode: inode to recalc
|
|
*
|
|
* We have to calculate the free blocks since the mm can drop
|
|
* undirtied hole pages behind our back.
|
|
*
|
|
* But normally info->alloced == inode->i_mapping->nrpages + info->swapped
|
|
* So mm freed is info->alloced - (inode->i_mapping->nrpages + info->swapped)
|
|
*
|
|
* It has to be called with the spinlock held.
|
|
*/
|
|
static void shmem_recalc_inode(struct inode *inode)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
long freed;
|
|
|
|
freed = info->alloced - info->swapped - inode->i_mapping->nrpages;
|
|
if (freed > 0) {
|
|
info->alloced -= freed;
|
|
inode->i_blocks -= freed * BLOCKS_PER_PAGE;
|
|
shmem_inode_unacct_blocks(inode, freed);
|
|
}
|
|
}
|
|
|
|
bool shmem_charge(struct inode *inode, long pages)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
unsigned long flags;
|
|
|
|
if (!shmem_inode_acct_block(inode, pages))
|
|
return false;
|
|
|
|
/* nrpages adjustment first, then shmem_recalc_inode() when balanced */
|
|
inode->i_mapping->nrpages += pages;
|
|
|
|
spin_lock_irqsave(&info->lock, flags);
|
|
info->alloced += pages;
|
|
inode->i_blocks += pages * BLOCKS_PER_PAGE;
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irqrestore(&info->lock, flags);
|
|
|
|
return true;
|
|
}
|
|
|
|
void shmem_uncharge(struct inode *inode, long pages)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
unsigned long flags;
|
|
|
|
/* nrpages adjustment done by __delete_from_page_cache() or caller */
|
|
|
|
spin_lock_irqsave(&info->lock, flags);
|
|
info->alloced -= pages;
|
|
inode->i_blocks -= pages * BLOCKS_PER_PAGE;
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irqrestore(&info->lock, flags);
|
|
|
|
shmem_inode_unacct_blocks(inode, pages);
|
|
}
|
|
|
|
/*
|
|
* Replace item expected in radix tree by a new item, while holding tree lock.
|
|
*/
|
|
static int shmem_radix_tree_replace(struct address_space *mapping,
|
|
pgoff_t index, void *expected, void *replacement)
|
|
{
|
|
struct radix_tree_node *node;
|
|
void __rcu **pslot;
|
|
void *item;
|
|
|
|
VM_BUG_ON(!expected);
|
|
VM_BUG_ON(!replacement);
|
|
item = __radix_tree_lookup(&mapping->i_pages, index, &node, &pslot);
|
|
if (!item)
|
|
return -ENOENT;
|
|
if (item != expected)
|
|
return -ENOENT;
|
|
__radix_tree_replace(&mapping->i_pages, node, pslot,
|
|
replacement, NULL);
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* Sometimes, before we decide whether to proceed or to fail, we must check
|
|
* that an entry was not already brought back from swap by a racing thread.
|
|
*
|
|
* Checking page is not enough: by the time a SwapCache page is locked, it
|
|
* might be reused, and again be SwapCache, using the same swap as before.
|
|
*/
|
|
static bool shmem_confirm_swap(struct address_space *mapping,
|
|
pgoff_t index, swp_entry_t swap)
|
|
{
|
|
void *item;
|
|
|
|
rcu_read_lock();
|
|
item = radix_tree_lookup(&mapping->i_pages, index);
|
|
rcu_read_unlock();
|
|
return item == swp_to_radix_entry(swap);
|
|
}
|
|
|
|
/*
|
|
* Definitions for "huge tmpfs": tmpfs mounted with the huge= option
|
|
*
|
|
* SHMEM_HUGE_NEVER:
|
|
* disables huge pages for the mount;
|
|
* SHMEM_HUGE_ALWAYS:
|
|
* enables huge pages for the mount;
|
|
* SHMEM_HUGE_WITHIN_SIZE:
|
|
* only allocate huge pages if the page will be fully within i_size,
|
|
* also respect fadvise()/madvise() hints;
|
|
* SHMEM_HUGE_ADVISE:
|
|
* only allocate huge pages if requested with fadvise()/madvise();
|
|
*/
|
|
|
|
#define SHMEM_HUGE_NEVER 0
|
|
#define SHMEM_HUGE_ALWAYS 1
|
|
#define SHMEM_HUGE_WITHIN_SIZE 2
|
|
#define SHMEM_HUGE_ADVISE 3
|
|
|
|
/*
|
|
* Special values.
|
|
* Only can be set via /sys/kernel/mm/transparent_hugepage/shmem_enabled:
|
|
*
|
|
* SHMEM_HUGE_DENY:
|
|
* disables huge on shm_mnt and all mounts, for emergency use;
|
|
* SHMEM_HUGE_FORCE:
|
|
* enables huge on shm_mnt and all mounts, w/o needing option, for testing;
|
|
*
|
|
*/
|
|
#define SHMEM_HUGE_DENY (-1)
|
|
#define SHMEM_HUGE_FORCE (-2)
|
|
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
/* ifdef here to avoid bloating shmem.o when not necessary */
|
|
|
|
static int shmem_huge __read_mostly;
|
|
|
|
#if defined(CONFIG_SYSFS) || defined(CONFIG_TMPFS)
|
|
static int shmem_parse_huge(const char *str)
|
|
{
|
|
if (!strcmp(str, "never"))
|
|
return SHMEM_HUGE_NEVER;
|
|
if (!strcmp(str, "always"))
|
|
return SHMEM_HUGE_ALWAYS;
|
|
if (!strcmp(str, "within_size"))
|
|
return SHMEM_HUGE_WITHIN_SIZE;
|
|
if (!strcmp(str, "advise"))
|
|
return SHMEM_HUGE_ADVISE;
|
|
if (!strcmp(str, "deny"))
|
|
return SHMEM_HUGE_DENY;
|
|
if (!strcmp(str, "force"))
|
|
return SHMEM_HUGE_FORCE;
|
|
return -EINVAL;
|
|
}
|
|
|
|
static const char *shmem_format_huge(int huge)
|
|
{
|
|
switch (huge) {
|
|
case SHMEM_HUGE_NEVER:
|
|
return "never";
|
|
case SHMEM_HUGE_ALWAYS:
|
|
return "always";
|
|
case SHMEM_HUGE_WITHIN_SIZE:
|
|
return "within_size";
|
|
case SHMEM_HUGE_ADVISE:
|
|
return "advise";
|
|
case SHMEM_HUGE_DENY:
|
|
return "deny";
|
|
case SHMEM_HUGE_FORCE:
|
|
return "force";
|
|
default:
|
|
VM_BUG_ON(1);
|
|
return "bad_val";
|
|
}
|
|
}
|
|
#endif
|
|
|
|
static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo,
|
|
struct shrink_control *sc, unsigned long nr_to_split)
|
|
{
|
|
LIST_HEAD(list), *pos, *next;
|
|
LIST_HEAD(to_remove);
|
|
struct inode *inode;
|
|
struct shmem_inode_info *info;
|
|
struct page *page;
|
|
unsigned long batch = sc ? sc->nr_to_scan : 128;
|
|
int removed = 0, split = 0;
|
|
|
|
if (list_empty(&sbinfo->shrinklist))
|
|
return SHRINK_STOP;
|
|
|
|
spin_lock(&sbinfo->shrinklist_lock);
|
|
list_for_each_safe(pos, next, &sbinfo->shrinklist) {
|
|
info = list_entry(pos, struct shmem_inode_info, shrinklist);
|
|
|
|
/* pin the inode */
|
|
inode = igrab(&info->vfs_inode);
|
|
|
|
/* inode is about to be evicted */
|
|
if (!inode) {
|
|
list_del_init(&info->shrinklist);
|
|
removed++;
|
|
goto next;
|
|
}
|
|
|
|
/* Check if there's anything to gain */
|
|
if (round_up(inode->i_size, PAGE_SIZE) ==
|
|
round_up(inode->i_size, HPAGE_PMD_SIZE)) {
|
|
list_move(&info->shrinklist, &to_remove);
|
|
removed++;
|
|
goto next;
|
|
}
|
|
|
|
list_move(&info->shrinklist, &list);
|
|
next:
|
|
if (!--batch)
|
|
break;
|
|
}
|
|
spin_unlock(&sbinfo->shrinklist_lock);
|
|
|
|
list_for_each_safe(pos, next, &to_remove) {
|
|
info = list_entry(pos, struct shmem_inode_info, shrinklist);
|
|
inode = &info->vfs_inode;
|
|
list_del_init(&info->shrinklist);
|
|
iput(inode);
|
|
}
|
|
|
|
list_for_each_safe(pos, next, &list) {
|
|
int ret;
|
|
|
|
info = list_entry(pos, struct shmem_inode_info, shrinklist);
|
|
inode = &info->vfs_inode;
|
|
|
|
if (nr_to_split && split >= nr_to_split)
|
|
goto leave;
|
|
|
|
page = find_get_page(inode->i_mapping,
|
|
(inode->i_size & HPAGE_PMD_MASK) >> PAGE_SHIFT);
|
|
if (!page)
|
|
goto drop;
|
|
|
|
/* No huge page at the end of the file: nothing to split */
|
|
if (!PageTransHuge(page)) {
|
|
put_page(page);
|
|
goto drop;
|
|
}
|
|
|
|
/*
|
|
* Leave the inode on the list if we failed to lock
|
|
* the page at this time.
|
|
*
|
|
* Waiting for the lock may lead to deadlock in the
|
|
* reclaim path.
|
|
*/
|
|
if (!trylock_page(page)) {
|
|
put_page(page);
|
|
goto leave;
|
|
}
|
|
|
|
ret = split_huge_page(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
|
|
/* If split failed leave the inode on the list */
|
|
if (ret)
|
|
goto leave;
|
|
|
|
split++;
|
|
drop:
|
|
list_del_init(&info->shrinklist);
|
|
removed++;
|
|
leave:
|
|
iput(inode);
|
|
}
|
|
|
|
spin_lock(&sbinfo->shrinklist_lock);
|
|
list_splice_tail(&list, &sbinfo->shrinklist);
|
|
sbinfo->shrinklist_len -= removed;
|
|
spin_unlock(&sbinfo->shrinklist_lock);
|
|
|
|
return split;
|
|
}
|
|
|
|
static long shmem_unused_huge_scan(struct super_block *sb,
|
|
struct shrink_control *sc)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
|
|
if (!READ_ONCE(sbinfo->shrinklist_len))
|
|
return SHRINK_STOP;
|
|
|
|
return shmem_unused_huge_shrink(sbinfo, sc, 0);
|
|
}
|
|
|
|
static long shmem_unused_huge_count(struct super_block *sb,
|
|
struct shrink_control *sc)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
return READ_ONCE(sbinfo->shrinklist_len);
|
|
}
|
|
#else /* !CONFIG_TRANSPARENT_HUGE_PAGECACHE */
|
|
|
|
#define shmem_huge SHMEM_HUGE_DENY
|
|
|
|
static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo,
|
|
struct shrink_control *sc, unsigned long nr_to_split)
|
|
{
|
|
return 0;
|
|
}
|
|
#endif /* CONFIG_TRANSPARENT_HUGE_PAGECACHE */
|
|
|
|
static inline bool is_huge_enabled(struct shmem_sb_info *sbinfo)
|
|
{
|
|
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE) &&
|
|
(shmem_huge == SHMEM_HUGE_FORCE || sbinfo->huge) &&
|
|
shmem_huge != SHMEM_HUGE_DENY)
|
|
return true;
|
|
return false;
|
|
}
|
|
|
|
/*
|
|
* Like add_to_page_cache_locked, but error if expected item has gone.
|
|
*/
|
|
static int shmem_add_to_page_cache(struct page *page,
|
|
struct address_space *mapping,
|
|
pgoff_t index, void *expected)
|
|
{
|
|
int error, nr = hpage_nr_pages(page);
|
|
|
|
VM_BUG_ON_PAGE(PageTail(page), page);
|
|
VM_BUG_ON_PAGE(index != round_down(index, nr), page);
|
|
VM_BUG_ON_PAGE(!PageLocked(page), page);
|
|
VM_BUG_ON_PAGE(!PageSwapBacked(page), page);
|
|
VM_BUG_ON(expected && PageTransHuge(page));
|
|
|
|
page_ref_add(page, nr);
|
|
page->mapping = mapping;
|
|
page->index = index;
|
|
|
|
xa_lock_irq(&mapping->i_pages);
|
|
if (PageTransHuge(page)) {
|
|
void __rcu **results;
|
|
pgoff_t idx;
|
|
int i;
|
|
|
|
error = 0;
|
|
if (radix_tree_gang_lookup_slot(&mapping->i_pages,
|
|
&results, &idx, index, 1) &&
|
|
idx < index + HPAGE_PMD_NR) {
|
|
error = -EEXIST;
|
|
}
|
|
|
|
if (!error) {
|
|
for (i = 0; i < HPAGE_PMD_NR; i++) {
|
|
error = radix_tree_insert(&mapping->i_pages,
|
|
index + i, page + i);
|
|
VM_BUG_ON(error);
|
|
}
|
|
count_vm_event(THP_FILE_ALLOC);
|
|
}
|
|
} else if (!expected) {
|
|
error = radix_tree_insert(&mapping->i_pages, index, page);
|
|
} else {
|
|
error = shmem_radix_tree_replace(mapping, index, expected,
|
|
page);
|
|
}
|
|
|
|
if (!error) {
|
|
mapping->nrpages += nr;
|
|
if (PageTransHuge(page))
|
|
__inc_node_page_state(page, NR_SHMEM_THPS);
|
|
__mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, nr);
|
|
__mod_node_page_state(page_pgdat(page), NR_SHMEM, nr);
|
|
xa_unlock_irq(&mapping->i_pages);
|
|
} else {
|
|
page->mapping = NULL;
|
|
xa_unlock_irq(&mapping->i_pages);
|
|
page_ref_sub(page, nr);
|
|
}
|
|
return error;
|
|
}
|
|
|
|
/*
|
|
* Like delete_from_page_cache, but substitutes swap for page.
|
|
*/
|
|
static void shmem_delete_from_page_cache(struct page *page, void *radswap)
|
|
{
|
|
struct address_space *mapping = page->mapping;
|
|
int error;
|
|
|
|
VM_BUG_ON_PAGE(PageCompound(page), page);
|
|
|
|
xa_lock_irq(&mapping->i_pages);
|
|
error = shmem_radix_tree_replace(mapping, page->index, page, radswap);
|
|
page->mapping = NULL;
|
|
mapping->nrpages--;
|
|
__dec_node_page_state(page, NR_FILE_PAGES);
|
|
__dec_node_page_state(page, NR_SHMEM);
|
|
xa_unlock_irq(&mapping->i_pages);
|
|
put_page(page);
|
|
BUG_ON(error);
|
|
}
|
|
|
|
/*
|
|
* Remove swap entry from radix tree, free the swap and its page cache.
|
|
*/
|
|
static int shmem_free_swap(struct address_space *mapping,
|
|
pgoff_t index, void *radswap)
|
|
{
|
|
void *old;
|
|
|
|
xa_lock_irq(&mapping->i_pages);
|
|
old = radix_tree_delete_item(&mapping->i_pages, index, radswap);
|
|
xa_unlock_irq(&mapping->i_pages);
|
|
if (old != radswap)
|
|
return -ENOENT;
|
|
free_swap_and_cache(radix_to_swp_entry(radswap));
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* Determine (in bytes) how many of the shmem object's pages mapped by the
|
|
* given offsets are swapped out.
|
|
*
|
|
* This is safe to call without i_mutex or the i_pages lock thanks to RCU,
|
|
* as long as the inode doesn't go away and racy results are not a problem.
|
|
*/
|
|
unsigned long shmem_partial_swap_usage(struct address_space *mapping,
|
|
pgoff_t start, pgoff_t end)
|
|
{
|
|
struct radix_tree_iter iter;
|
|
void __rcu **slot;
|
|
struct page *page;
|
|
unsigned long swapped = 0;
|
|
|
|
rcu_read_lock();
|
|
|
|
radix_tree_for_each_slot(slot, &mapping->i_pages, &iter, start) {
|
|
if (iter.index >= end)
|
|
break;
|
|
|
|
page = radix_tree_deref_slot(slot);
|
|
|
|
if (radix_tree_deref_retry(page)) {
|
|
slot = radix_tree_iter_retry(&iter);
|
|
continue;
|
|
}
|
|
|
|
if (radix_tree_exceptional_entry(page))
|
|
swapped++;
|
|
|
|
if (need_resched()) {
|
|
slot = radix_tree_iter_resume(slot, &iter);
|
|
cond_resched_rcu();
|
|
}
|
|
}
|
|
|
|
rcu_read_unlock();
|
|
|
|
return swapped << PAGE_SHIFT;
|
|
}
|
|
|
|
/*
|
|
* Determine (in bytes) how many of the shmem object's pages mapped by the
|
|
* given vma is swapped out.
|
|
*
|
|
* This is safe to call without i_mutex or the i_pages lock thanks to RCU,
|
|
* as long as the inode doesn't go away and racy results are not a problem.
|
|
*/
|
|
unsigned long shmem_swap_usage(struct vm_area_struct *vma)
|
|
{
|
|
struct inode *inode = file_inode(vma->vm_file);
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct address_space *mapping = inode->i_mapping;
|
|
unsigned long swapped;
|
|
|
|
/* Be careful as we don't hold info->lock */
|
|
swapped = READ_ONCE(info->swapped);
|
|
|
|
/*
|
|
* The easier cases are when the shmem object has nothing in swap, or
|
|
* the vma maps it whole. Then we can simply use the stats that we
|
|
* already track.
|
|
*/
|
|
if (!swapped)
|
|
return 0;
|
|
|
|
if (!vma->vm_pgoff && vma->vm_end - vma->vm_start >= inode->i_size)
|
|
return swapped << PAGE_SHIFT;
|
|
|
|
/* Here comes the more involved part */
|
|
return shmem_partial_swap_usage(mapping,
|
|
linear_page_index(vma, vma->vm_start),
|
|
linear_page_index(vma, vma->vm_end));
|
|
}
|
|
|
|
/*
|
|
* SysV IPC SHM_UNLOCK restore Unevictable pages to their evictable lists.
|
|
*/
|
|
void shmem_unlock_mapping(struct address_space *mapping)
|
|
{
|
|
struct pagevec pvec;
|
|
pgoff_t indices[PAGEVEC_SIZE];
|
|
pgoff_t index = 0;
|
|
|
|
pagevec_init(&pvec);
|
|
/*
|
|
* Minor point, but we might as well stop if someone else SHM_LOCKs it.
|
|
*/
|
|
while (!mapping_unevictable(mapping)) {
|
|
/*
|
|
* Avoid pagevec_lookup(): find_get_pages() returns 0 as if it
|
|
* has finished, if it hits a row of PAGEVEC_SIZE swap entries.
|
|
*/
|
|
pvec.nr = find_get_entries(mapping, index,
|
|
PAGEVEC_SIZE, pvec.pages, indices);
|
|
if (!pvec.nr)
|
|
break;
|
|
index = indices[pvec.nr - 1] + 1;
|
|
pagevec_remove_exceptionals(&pvec);
|
|
check_move_unevictable_pages(pvec.pages, pvec.nr);
|
|
pagevec_release(&pvec);
|
|
cond_resched();
|
|
}
|
|
}
|
|
|
|
/*
|
|
* Remove range of pages and swap entries from radix tree, and free them.
|
|
* If !unfalloc, truncate or punch hole; if unfalloc, undo failed fallocate.
|
|
*/
|
|
static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend,
|
|
bool unfalloc)
|
|
{
|
|
struct address_space *mapping = inode->i_mapping;
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
pgoff_t start = (lstart + PAGE_SIZE - 1) >> PAGE_SHIFT;
|
|
pgoff_t end = (lend + 1) >> PAGE_SHIFT;
|
|
unsigned int partial_start = lstart & (PAGE_SIZE - 1);
|
|
unsigned int partial_end = (lend + 1) & (PAGE_SIZE - 1);
|
|
struct pagevec pvec;
|
|
pgoff_t indices[PAGEVEC_SIZE];
|
|
long nr_swaps_freed = 0;
|
|
pgoff_t index;
|
|
int i;
|
|
|
|
if (lend == -1)
|
|
end = -1; /* unsigned, so actually very big */
|
|
|
|
pagevec_init(&pvec);
|
|
index = start;
|
|
while (index < end) {
|
|
pvec.nr = find_get_entries(mapping, index,
|
|
min(end - index, (pgoff_t)PAGEVEC_SIZE),
|
|
pvec.pages, indices);
|
|
if (!pvec.nr)
|
|
break;
|
|
for (i = 0; i < pagevec_count(&pvec); i++) {
|
|
struct page *page = pvec.pages[i];
|
|
|
|
index = indices[i];
|
|
if (index >= end)
|
|
break;
|
|
|
|
if (radix_tree_exceptional_entry(page)) {
|
|
if (unfalloc)
|
|
continue;
|
|
nr_swaps_freed += !shmem_free_swap(mapping,
|
|
index, page);
|
|
continue;
|
|
}
|
|
|
|
VM_BUG_ON_PAGE(page_to_pgoff(page) != index, page);
|
|
|
|
if (!trylock_page(page))
|
|
continue;
|
|
|
|
if (PageTransTail(page)) {
|
|
/* Middle of THP: zero out the page */
|
|
clear_highpage(page);
|
|
unlock_page(page);
|
|
continue;
|
|
} else if (PageTransHuge(page)) {
|
|
if (index == round_down(end, HPAGE_PMD_NR)) {
|
|
/*
|
|
* Range ends in the middle of THP:
|
|
* zero out the page
|
|
*/
|
|
clear_highpage(page);
|
|
unlock_page(page);
|
|
continue;
|
|
}
|
|
index += HPAGE_PMD_NR - 1;
|
|
i += HPAGE_PMD_NR - 1;
|
|
}
|
|
|
|
if (!unfalloc || !PageUptodate(page)) {
|
|
VM_BUG_ON_PAGE(PageTail(page), page);
|
|
if (page_mapping(page) == mapping) {
|
|
VM_BUG_ON_PAGE(PageWriteback(page), page);
|
|
truncate_inode_page(mapping, page);
|
|
}
|
|
}
|
|
unlock_page(page);
|
|
}
|
|
pagevec_remove_exceptionals(&pvec);
|
|
pagevec_release(&pvec);
|
|
cond_resched();
|
|
index++;
|
|
}
|
|
|
|
if (partial_start) {
|
|
struct page *page = NULL;
|
|
shmem_getpage(inode, start - 1, &page, SGP_READ);
|
|
if (page) {
|
|
unsigned int top = PAGE_SIZE;
|
|
if (start > end) {
|
|
top = partial_end;
|
|
partial_end = 0;
|
|
}
|
|
zero_user_segment(page, partial_start, top);
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
}
|
|
}
|
|
if (partial_end) {
|
|
struct page *page = NULL;
|
|
shmem_getpage(inode, end, &page, SGP_READ);
|
|
if (page) {
|
|
zero_user_segment(page, 0, partial_end);
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
}
|
|
}
|
|
if (start >= end)
|
|
return;
|
|
|
|
index = start;
|
|
while (index < end) {
|
|
cond_resched();
|
|
|
|
pvec.nr = find_get_entries(mapping, index,
|
|
min(end - index, (pgoff_t)PAGEVEC_SIZE),
|
|
pvec.pages, indices);
|
|
if (!pvec.nr) {
|
|
/* If all gone or hole-punch or unfalloc, we're done */
|
|
if (index == start || end != -1)
|
|
break;
|
|
/* But if truncating, restart to make sure all gone */
|
|
index = start;
|
|
continue;
|
|
}
|
|
for (i = 0; i < pagevec_count(&pvec); i++) {
|
|
struct page *page = pvec.pages[i];
|
|
|
|
index = indices[i];
|
|
if (index >= end)
|
|
break;
|
|
|
|
if (radix_tree_exceptional_entry(page)) {
|
|
if (unfalloc)
|
|
continue;
|
|
if (shmem_free_swap(mapping, index, page)) {
|
|
/* Swap was replaced by page: retry */
|
|
index--;
|
|
break;
|
|
}
|
|
nr_swaps_freed++;
|
|
continue;
|
|
}
|
|
|
|
lock_page(page);
|
|
|
|
if (PageTransTail(page)) {
|
|
/* Middle of THP: zero out the page */
|
|
clear_highpage(page);
|
|
unlock_page(page);
|
|
/*
|
|
* Partial thp truncate due 'start' in middle
|
|
* of THP: don't need to look on these pages
|
|
* again on !pvec.nr restart.
|
|
*/
|
|
if (index != round_down(end, HPAGE_PMD_NR))
|
|
start++;
|
|
continue;
|
|
} else if (PageTransHuge(page)) {
|
|
if (index == round_down(end, HPAGE_PMD_NR)) {
|
|
/*
|
|
* Range ends in the middle of THP:
|
|
* zero out the page
|
|
*/
|
|
clear_highpage(page);
|
|
unlock_page(page);
|
|
continue;
|
|
}
|
|
index += HPAGE_PMD_NR - 1;
|
|
i += HPAGE_PMD_NR - 1;
|
|
}
|
|
|
|
if (!unfalloc || !PageUptodate(page)) {
|
|
VM_BUG_ON_PAGE(PageTail(page), page);
|
|
if (page_mapping(page) == mapping) {
|
|
VM_BUG_ON_PAGE(PageWriteback(page), page);
|
|
truncate_inode_page(mapping, page);
|
|
} else {
|
|
/* Page was replaced by swap: retry */
|
|
unlock_page(page);
|
|
index--;
|
|
break;
|
|
}
|
|
}
|
|
unlock_page(page);
|
|
}
|
|
pagevec_remove_exceptionals(&pvec);
|
|
pagevec_release(&pvec);
|
|
index++;
|
|
}
|
|
|
|
spin_lock_irq(&info->lock);
|
|
info->swapped -= nr_swaps_freed;
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
}
|
|
|
|
void shmem_truncate_range(struct inode *inode, loff_t lstart, loff_t lend)
|
|
{
|
|
shmem_undo_range(inode, lstart, lend, false);
|
|
inode->i_ctime = inode->i_mtime = current_time(inode);
|
|
}
|
|
EXPORT_SYMBOL_GPL(shmem_truncate_range);
|
|
|
|
static int shmem_getattr(const struct path *path, struct kstat *stat,
|
|
u32 request_mask, unsigned int query_flags)
|
|
{
|
|
struct inode *inode = path->dentry->d_inode;
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sb_info = SHMEM_SB(inode->i_sb);
|
|
|
|
if (info->alloced - info->swapped != inode->i_mapping->nrpages) {
|
|
spin_lock_irq(&info->lock);
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
}
|
|
generic_fillattr(inode, stat);
|
|
|
|
if (is_huge_enabled(sb_info))
|
|
stat->blksize = HPAGE_PMD_SIZE;
|
|
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_setattr(struct dentry *dentry, struct iattr *attr)
|
|
{
|
|
struct inode *inode = d_inode(dentry);
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
int error;
|
|
|
|
error = setattr_prepare(dentry, attr);
|
|
if (error)
|
|
return error;
|
|
|
|
if (S_ISREG(inode->i_mode) && (attr->ia_valid & ATTR_SIZE)) {
|
|
loff_t oldsize = inode->i_size;
|
|
loff_t newsize = attr->ia_size;
|
|
|
|
/* protected by i_mutex */
|
|
if ((newsize < oldsize && (info->seals & F_SEAL_SHRINK)) ||
|
|
(newsize > oldsize && (info->seals & F_SEAL_GROW)))
|
|
return -EPERM;
|
|
|
|
if (newsize != oldsize) {
|
|
error = shmem_reacct_size(SHMEM_I(inode)->flags,
|
|
oldsize, newsize);
|
|
if (error)
|
|
return error;
|
|
i_size_write(inode, newsize);
|
|
inode->i_ctime = inode->i_mtime = current_time(inode);
|
|
}
|
|
if (newsize <= oldsize) {
|
|
loff_t holebegin = round_up(newsize, PAGE_SIZE);
|
|
if (oldsize > holebegin)
|
|
unmap_mapping_range(inode->i_mapping,
|
|
holebegin, 0, 1);
|
|
if (info->alloced)
|
|
shmem_truncate_range(inode,
|
|
newsize, (loff_t)-1);
|
|
/* unmap again to remove racily COWed private pages */
|
|
if (oldsize > holebegin)
|
|
unmap_mapping_range(inode->i_mapping,
|
|
holebegin, 0, 1);
|
|
|
|
/*
|
|
* Part of the huge page can be beyond i_size: subject
|
|
* to shrink under memory pressure.
|
|
*/
|
|
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) {
|
|
spin_lock(&sbinfo->shrinklist_lock);
|
|
/*
|
|
* _careful to defend against unlocked access to
|
|
* ->shrink_list in shmem_unused_huge_shrink()
|
|
*/
|
|
if (list_empty_careful(&info->shrinklist)) {
|
|
list_add_tail(&info->shrinklist,
|
|
&sbinfo->shrinklist);
|
|
sbinfo->shrinklist_len++;
|
|
}
|
|
spin_unlock(&sbinfo->shrinklist_lock);
|
|
}
|
|
}
|
|
}
|
|
|
|
setattr_copy(inode, attr);
|
|
if (attr->ia_valid & ATTR_MODE)
|
|
error = posix_acl_chmod(inode, inode->i_mode);
|
|
return error;
|
|
}
|
|
|
|
static void shmem_evict_inode(struct inode *inode)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
|
|
if (inode->i_mapping->a_ops == &shmem_aops) {
|
|
shmem_unacct_size(info->flags, inode->i_size);
|
|
inode->i_size = 0;
|
|
shmem_truncate_range(inode, 0, (loff_t)-1);
|
|
if (!list_empty(&info->shrinklist)) {
|
|
spin_lock(&sbinfo->shrinklist_lock);
|
|
if (!list_empty(&info->shrinklist)) {
|
|
list_del_init(&info->shrinklist);
|
|
sbinfo->shrinklist_len--;
|
|
}
|
|
spin_unlock(&sbinfo->shrinklist_lock);
|
|
}
|
|
if (!list_empty(&info->swaplist)) {
|
|
mutex_lock(&shmem_swaplist_mutex);
|
|
list_del_init(&info->swaplist);
|
|
mutex_unlock(&shmem_swaplist_mutex);
|
|
}
|
|
}
|
|
|
|
simple_xattrs_free(&info->xattrs);
|
|
WARN_ON(inode->i_blocks);
|
|
shmem_free_inode(inode->i_sb);
|
|
clear_inode(inode);
|
|
}
|
|
|
|
static unsigned long find_swap_entry(struct radix_tree_root *root, void *item)
|
|
{
|
|
struct radix_tree_iter iter;
|
|
void __rcu **slot;
|
|
unsigned long found = -1;
|
|
unsigned int checked = 0;
|
|
|
|
rcu_read_lock();
|
|
radix_tree_for_each_slot(slot, root, &iter, 0) {
|
|
void *entry = radix_tree_deref_slot(slot);
|
|
|
|
if (radix_tree_deref_retry(entry)) {
|
|
slot = radix_tree_iter_retry(&iter);
|
|
continue;
|
|
}
|
|
if (entry == item) {
|
|
found = iter.index;
|
|
break;
|
|
}
|
|
checked++;
|
|
if ((checked % 4096) != 0)
|
|
continue;
|
|
slot = radix_tree_iter_resume(slot, &iter);
|
|
cond_resched_rcu();
|
|
}
|
|
|
|
rcu_read_unlock();
|
|
return found;
|
|
}
|
|
|
|
/*
|
|
* If swap found in inode, free it and move page from swapcache to filecache.
|
|
*/
|
|
static int shmem_unuse_inode(struct shmem_inode_info *info,
|
|
swp_entry_t swap, struct page **pagep)
|
|
{
|
|
struct address_space *mapping = info->vfs_inode.i_mapping;
|
|
void *radswap;
|
|
pgoff_t index;
|
|
gfp_t gfp;
|
|
int error = 0;
|
|
|
|
radswap = swp_to_radix_entry(swap);
|
|
index = find_swap_entry(&mapping->i_pages, radswap);
|
|
if (index == -1)
|
|
return -EAGAIN; /* tell shmem_unuse we found nothing */
|
|
|
|
/*
|
|
* Move _head_ to start search for next from here.
|
|
* But be careful: shmem_evict_inode checks list_empty without taking
|
|
* mutex, and there's an instant in list_move_tail when info->swaplist
|
|
* would appear empty, if it were the only one on shmem_swaplist.
|
|
*/
|
|
if (shmem_swaplist.next != &info->swaplist)
|
|
list_move_tail(&shmem_swaplist, &info->swaplist);
|
|
|
|
gfp = mapping_gfp_mask(mapping);
|
|
if (shmem_should_replace_page(*pagep, gfp)) {
|
|
mutex_unlock(&shmem_swaplist_mutex);
|
|
error = shmem_replace_page(pagep, gfp, info, index);
|
|
mutex_lock(&shmem_swaplist_mutex);
|
|
/*
|
|
* We needed to drop mutex to make that restrictive page
|
|
* allocation, but the inode might have been freed while we
|
|
* dropped it: although a racing shmem_evict_inode() cannot
|
|
* complete without emptying the radix_tree, our page lock
|
|
* on this swapcache page is not enough to prevent that -
|
|
* free_swap_and_cache() of our swap entry will only
|
|
* trylock_page(), removing swap from radix_tree whatever.
|
|
*
|
|
* We must not proceed to shmem_add_to_page_cache() if the
|
|
* inode has been freed, but of course we cannot rely on
|
|
* inode or mapping or info to check that. However, we can
|
|
* safely check if our swap entry is still in use (and here
|
|
* it can't have got reused for another page): if it's still
|
|
* in use, then the inode cannot have been freed yet, and we
|
|
* can safely proceed (if it's no longer in use, that tells
|
|
* nothing about the inode, but we don't need to unuse swap).
|
|
*/
|
|
if (!page_swapcount(*pagep))
|
|
error = -ENOENT;
|
|
}
|
|
|
|
/*
|
|
* We rely on shmem_swaplist_mutex, not only to protect the swaplist,
|
|
* but also to hold up shmem_evict_inode(): so inode cannot be freed
|
|
* beneath us (pagelock doesn't help until the page is in pagecache).
|
|
*/
|
|
if (!error)
|
|
error = shmem_add_to_page_cache(*pagep, mapping, index,
|
|
radswap);
|
|
if (error != -ENOMEM) {
|
|
/*
|
|
* Truncation and eviction use free_swap_and_cache(), which
|
|
* only does trylock page: if we raced, best clean up here.
|
|
*/
|
|
delete_from_swap_cache(*pagep);
|
|
set_page_dirty(*pagep);
|
|
if (!error) {
|
|
spin_lock_irq(&info->lock);
|
|
info->swapped--;
|
|
spin_unlock_irq(&info->lock);
|
|
swap_free(swap);
|
|
}
|
|
}
|
|
return error;
|
|
}
|
|
|
|
/*
|
|
* Search through swapped inodes to find and replace swap by page.
|
|
*/
|
|
int shmem_unuse(swp_entry_t swap, struct page *page)
|
|
{
|
|
struct list_head *this, *next;
|
|
struct shmem_inode_info *info;
|
|
struct mem_cgroup *memcg;
|
|
int error = 0;
|
|
|
|
/*
|
|
* There's a faint possibility that swap page was replaced before
|
|
* caller locked it: caller will come back later with the right page.
|
|
*/
|
|
if (unlikely(!PageSwapCache(page) || page_private(page) != swap.val))
|
|
goto out;
|
|
|
|
/*
|
|
* Charge page using GFP_KERNEL while we can wait, before taking
|
|
* the shmem_swaplist_mutex which might hold up shmem_writepage().
|
|
* Charged back to the user (not to caller) when swap account is used.
|
|
*/
|
|
error = mem_cgroup_try_charge_delay(page, current->mm, GFP_KERNEL,
|
|
&memcg, false);
|
|
if (error)
|
|
goto out;
|
|
/* No radix_tree_preload: swap entry keeps a place for page in tree */
|
|
error = -EAGAIN;
|
|
|
|
mutex_lock(&shmem_swaplist_mutex);
|
|
list_for_each_safe(this, next, &shmem_swaplist) {
|
|
info = list_entry(this, struct shmem_inode_info, swaplist);
|
|
if (info->swapped)
|
|
error = shmem_unuse_inode(info, swap, &page);
|
|
else
|
|
list_del_init(&info->swaplist);
|
|
cond_resched();
|
|
if (error != -EAGAIN)
|
|
break;
|
|
/* found nothing in this: move on to search the next */
|
|
}
|
|
mutex_unlock(&shmem_swaplist_mutex);
|
|
|
|
if (error) {
|
|
if (error != -ENOMEM)
|
|
error = 0;
|
|
mem_cgroup_cancel_charge(page, memcg, false);
|
|
} else
|
|
mem_cgroup_commit_charge(page, memcg, true, false);
|
|
out:
|
|
unlock_page(page);
|
|
put_page(page);
|
|
return error;
|
|
}
|
|
|
|
/*
|
|
* Move the page from the page cache to the swap cache.
|
|
*/
|
|
static int shmem_writepage(struct page *page, struct writeback_control *wbc)
|
|
{
|
|
struct shmem_inode_info *info;
|
|
struct address_space *mapping;
|
|
struct inode *inode;
|
|
swp_entry_t swap;
|
|
pgoff_t index;
|
|
|
|
VM_BUG_ON_PAGE(PageCompound(page), page);
|
|
BUG_ON(!PageLocked(page));
|
|
mapping = page->mapping;
|
|
index = page->index;
|
|
inode = mapping->host;
|
|
info = SHMEM_I(inode);
|
|
if (info->flags & VM_LOCKED)
|
|
goto redirty;
|
|
if (!total_swap_pages)
|
|
goto redirty;
|
|
|
|
/*
|
|
* Our capabilities prevent regular writeback or sync from ever calling
|
|
* shmem_writepage; but a stacking filesystem might use ->writepage of
|
|
* its underlying filesystem, in which case tmpfs should write out to
|
|
* swap only in response to memory pressure, and not for the writeback
|
|
* threads or sync.
|
|
*/
|
|
if (!wbc->for_reclaim) {
|
|
WARN_ON_ONCE(1); /* Still happens? Tell us about it! */
|
|
goto redirty;
|
|
}
|
|
|
|
/*
|
|
* This is somewhat ridiculous, but without plumbing a SWAP_MAP_FALLOC
|
|
* value into swapfile.c, the only way we can correctly account for a
|
|
* fallocated page arriving here is now to initialize it and write it.
|
|
*
|
|
* That's okay for a page already fallocated earlier, but if we have
|
|
* not yet completed the fallocation, then (a) we want to keep track
|
|
* of this page in case we have to undo it, and (b) it may not be a
|
|
* good idea to continue anyway, once we're pushing into swap. So
|
|
* reactivate the page, and let shmem_fallocate() quit when too many.
|
|
*/
|
|
if (!PageUptodate(page)) {
|
|
if (inode->i_private) {
|
|
struct shmem_falloc *shmem_falloc;
|
|
spin_lock(&inode->i_lock);
|
|
shmem_falloc = inode->i_private;
|
|
if (shmem_falloc &&
|
|
!shmem_falloc->waitq &&
|
|
index >= shmem_falloc->start &&
|
|
index < shmem_falloc->next)
|
|
shmem_falloc->nr_unswapped++;
|
|
else
|
|
shmem_falloc = NULL;
|
|
spin_unlock(&inode->i_lock);
|
|
if (shmem_falloc)
|
|
goto redirty;
|
|
}
|
|
clear_highpage(page);
|
|
flush_dcache_page(page);
|
|
SetPageUptodate(page);
|
|
}
|
|
|
|
swap = get_swap_page(page);
|
|
if (!swap.val)
|
|
goto redirty;
|
|
|
|
/*
|
|
* Add inode to shmem_unuse()'s list of swapped-out inodes,
|
|
* if it's not already there. Do it now before the page is
|
|
* moved to swap cache, when its pagelock no longer protects
|
|
* the inode from eviction. But don't unlock the mutex until
|
|
* we've incremented swapped, because shmem_unuse_inode() will
|
|
* prune a !swapped inode from the swaplist under this mutex.
|
|
*/
|
|
mutex_lock(&shmem_swaplist_mutex);
|
|
if (list_empty(&info->swaplist))
|
|
list_add_tail(&info->swaplist, &shmem_swaplist);
|
|
|
|
if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
|
|
spin_lock_irq(&info->lock);
|
|
shmem_recalc_inode(inode);
|
|
info->swapped++;
|
|
spin_unlock_irq(&info->lock);
|
|
|
|
swap_shmem_alloc(swap);
|
|
shmem_delete_from_page_cache(page, swp_to_radix_entry(swap));
|
|
|
|
mutex_unlock(&shmem_swaplist_mutex);
|
|
BUG_ON(page_mapped(page));
|
|
swap_writepage(page, wbc);
|
|
return 0;
|
|
}
|
|
|
|
mutex_unlock(&shmem_swaplist_mutex);
|
|
put_swap_page(page, swap);
|
|
redirty:
|
|
set_page_dirty(page);
|
|
if (wbc->for_reclaim)
|
|
return AOP_WRITEPAGE_ACTIVATE; /* Return with page locked */
|
|
unlock_page(page);
|
|
return 0;
|
|
}
|
|
|
|
#if defined(CONFIG_NUMA) && defined(CONFIG_TMPFS)
|
|
static void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)
|
|
{
|
|
char buffer[64];
|
|
|
|
if (!mpol || mpol->mode == MPOL_DEFAULT)
|
|
return; /* show nothing */
|
|
|
|
mpol_to_str(buffer, sizeof(buffer), mpol);
|
|
|
|
seq_printf(seq, ",mpol=%s", buffer);
|
|
}
|
|
|
|
static struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo)
|
|
{
|
|
struct mempolicy *mpol = NULL;
|
|
if (sbinfo->mpol) {
|
|
spin_lock(&sbinfo->stat_lock); /* prevent replace/use races */
|
|
mpol = sbinfo->mpol;
|
|
mpol_get(mpol);
|
|
spin_unlock(&sbinfo->stat_lock);
|
|
}
|
|
return mpol;
|
|
}
|
|
#else /* !CONFIG_NUMA || !CONFIG_TMPFS */
|
|
static inline void shmem_show_mpol(struct seq_file *seq, struct mempolicy *mpol)
|
|
{
|
|
}
|
|
static inline struct mempolicy *shmem_get_sbmpol(struct shmem_sb_info *sbinfo)
|
|
{
|
|
return NULL;
|
|
}
|
|
#endif /* CONFIG_NUMA && CONFIG_TMPFS */
|
|
#ifndef CONFIG_NUMA
|
|
#define vm_policy vm_private_data
|
|
#endif
|
|
|
|
static void shmem_pseudo_vma_init(struct vm_area_struct *vma,
|
|
struct shmem_inode_info *info, pgoff_t index)
|
|
{
|
|
/* Create a pseudo vma that just contains the policy */
|
|
vma_init(vma, NULL);
|
|
/* Bias interleave by inode number to distribute better across nodes */
|
|
vma->vm_pgoff = index + info->vfs_inode.i_ino;
|
|
vma->vm_policy = mpol_shared_policy_lookup(&info->policy, index);
|
|
}
|
|
|
|
static void shmem_pseudo_vma_destroy(struct vm_area_struct *vma)
|
|
{
|
|
/* Drop reference taken by mpol_shared_policy_lookup() */
|
|
mpol_cond_put(vma->vm_policy);
|
|
}
|
|
|
|
static struct page *shmem_swapin(swp_entry_t swap, gfp_t gfp,
|
|
struct shmem_inode_info *info, pgoff_t index)
|
|
{
|
|
struct vm_area_struct pvma;
|
|
struct page *page;
|
|
struct vm_fault vmf;
|
|
|
|
shmem_pseudo_vma_init(&pvma, info, index);
|
|
vmf.vma = &pvma;
|
|
vmf.address = 0;
|
|
page = swap_cluster_readahead(swap, gfp, &vmf);
|
|
shmem_pseudo_vma_destroy(&pvma);
|
|
|
|
return page;
|
|
}
|
|
|
|
static struct page *shmem_alloc_hugepage(gfp_t gfp,
|
|
struct shmem_inode_info *info, pgoff_t index)
|
|
{
|
|
struct vm_area_struct pvma;
|
|
struct inode *inode = &info->vfs_inode;
|
|
struct address_space *mapping = inode->i_mapping;
|
|
pgoff_t idx, hindex;
|
|
void __rcu **results;
|
|
struct page *page;
|
|
|
|
if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE))
|
|
return NULL;
|
|
|
|
hindex = round_down(index, HPAGE_PMD_NR);
|
|
rcu_read_lock();
|
|
if (radix_tree_gang_lookup_slot(&mapping->i_pages, &results, &idx,
|
|
hindex, 1) && idx < hindex + HPAGE_PMD_NR) {
|
|
rcu_read_unlock();
|
|
return NULL;
|
|
}
|
|
rcu_read_unlock();
|
|
|
|
shmem_pseudo_vma_init(&pvma, info, hindex);
|
|
page = alloc_pages_vma(gfp | __GFP_COMP | __GFP_NORETRY | __GFP_NOWARN,
|
|
HPAGE_PMD_ORDER, &pvma, 0, numa_node_id(), true);
|
|
shmem_pseudo_vma_destroy(&pvma);
|
|
if (page)
|
|
prep_transhuge_page(page);
|
|
return page;
|
|
}
|
|
|
|
static struct page *shmem_alloc_page(gfp_t gfp,
|
|
struct shmem_inode_info *info, pgoff_t index)
|
|
{
|
|
struct vm_area_struct pvma;
|
|
struct page *page;
|
|
|
|
shmem_pseudo_vma_init(&pvma, info, index);
|
|
page = alloc_page_vma(gfp, &pvma, 0);
|
|
shmem_pseudo_vma_destroy(&pvma);
|
|
|
|
return page;
|
|
}
|
|
|
|
static struct page *shmem_alloc_and_acct_page(gfp_t gfp,
|
|
struct inode *inode,
|
|
pgoff_t index, bool huge)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct page *page;
|
|
int nr;
|
|
int err = -ENOSPC;
|
|
|
|
if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE))
|
|
huge = false;
|
|
nr = huge ? HPAGE_PMD_NR : 1;
|
|
|
|
if (!shmem_inode_acct_block(inode, nr))
|
|
goto failed;
|
|
|
|
if (huge)
|
|
page = shmem_alloc_hugepage(gfp, info, index);
|
|
else
|
|
page = shmem_alloc_page(gfp, info, index);
|
|
if (page) {
|
|
__SetPageLocked(page);
|
|
__SetPageSwapBacked(page);
|
|
return page;
|
|
}
|
|
|
|
err = -ENOMEM;
|
|
shmem_inode_unacct_blocks(inode, nr);
|
|
failed:
|
|
return ERR_PTR(err);
|
|
}
|
|
|
|
/*
|
|
* When a page is moved from swapcache to shmem filecache (either by the
|
|
* usual swapin of shmem_getpage_gfp(), or by the less common swapoff of
|
|
* shmem_unuse_inode()), it may have been read in earlier from swap, in
|
|
* ignorance of the mapping it belongs to. If that mapping has special
|
|
* constraints (like the gma500 GEM driver, which requires RAM below 4GB),
|
|
* we may need to copy to a suitable page before moving to filecache.
|
|
*
|
|
* In a future release, this may well be extended to respect cpuset and
|
|
* NUMA mempolicy, and applied also to anonymous pages in do_swap_page();
|
|
* but for now it is a simple matter of zone.
|
|
*/
|
|
static bool shmem_should_replace_page(struct page *page, gfp_t gfp)
|
|
{
|
|
return page_zonenum(page) > gfp_zone(gfp);
|
|
}
|
|
|
|
static int shmem_replace_page(struct page **pagep, gfp_t gfp,
|
|
struct shmem_inode_info *info, pgoff_t index)
|
|
{
|
|
struct page *oldpage, *newpage;
|
|
struct address_space *swap_mapping;
|
|
swp_entry_t entry;
|
|
pgoff_t swap_index;
|
|
int error;
|
|
|
|
oldpage = *pagep;
|
|
entry.val = page_private(oldpage);
|
|
swap_index = swp_offset(entry);
|
|
swap_mapping = page_mapping(oldpage);
|
|
|
|
/*
|
|
* We have arrived here because our zones are constrained, so don't
|
|
* limit chance of success by further cpuset and node constraints.
|
|
*/
|
|
gfp &= ~GFP_CONSTRAINT_MASK;
|
|
newpage = shmem_alloc_page(gfp, info, index);
|
|
if (!newpage)
|
|
return -ENOMEM;
|
|
|
|
get_page(newpage);
|
|
copy_highpage(newpage, oldpage);
|
|
flush_dcache_page(newpage);
|
|
|
|
__SetPageLocked(newpage);
|
|
__SetPageSwapBacked(newpage);
|
|
SetPageUptodate(newpage);
|
|
set_page_private(newpage, entry.val);
|
|
SetPageSwapCache(newpage);
|
|
|
|
/*
|
|
* Our caller will very soon move newpage out of swapcache, but it's
|
|
* a nice clean interface for us to replace oldpage by newpage there.
|
|
*/
|
|
xa_lock_irq(&swap_mapping->i_pages);
|
|
error = shmem_radix_tree_replace(swap_mapping, swap_index, oldpage,
|
|
newpage);
|
|
if (!error) {
|
|
__inc_node_page_state(newpage, NR_FILE_PAGES);
|
|
__dec_node_page_state(oldpage, NR_FILE_PAGES);
|
|
}
|
|
xa_unlock_irq(&swap_mapping->i_pages);
|
|
|
|
if (unlikely(error)) {
|
|
/*
|
|
* Is this possible? I think not, now that our callers check
|
|
* both PageSwapCache and page_private after getting page lock;
|
|
* but be defensive. Reverse old to newpage for clear and free.
|
|
*/
|
|
oldpage = newpage;
|
|
} else {
|
|
mem_cgroup_migrate(oldpage, newpage);
|
|
lru_cache_add_anon(newpage);
|
|
*pagep = newpage;
|
|
}
|
|
|
|
ClearPageSwapCache(oldpage);
|
|
set_page_private(oldpage, 0);
|
|
|
|
unlock_page(oldpage);
|
|
put_page(oldpage);
|
|
put_page(oldpage);
|
|
return error;
|
|
}
|
|
|
|
/*
|
|
* shmem_getpage_gfp - find page in cache, or get from swap, or allocate
|
|
*
|
|
* If we allocate a new one we do not mark it dirty. That's up to the
|
|
* vm. If we swap it in we mark it dirty since we also free the swap
|
|
* entry since a page cannot live in both the swap and page cache.
|
|
*
|
|
* fault_mm and fault_type are only supplied by shmem_fault:
|
|
* otherwise they are NULL.
|
|
*/
|
|
static int shmem_getpage_gfp(struct inode *inode, pgoff_t index,
|
|
struct page **pagep, enum sgp_type sgp, gfp_t gfp,
|
|
struct vm_area_struct *vma, struct vm_fault *vmf,
|
|
vm_fault_t *fault_type)
|
|
{
|
|
struct address_space *mapping = inode->i_mapping;
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_sb_info *sbinfo;
|
|
struct mm_struct *charge_mm;
|
|
struct mem_cgroup *memcg;
|
|
struct page *page;
|
|
swp_entry_t swap;
|
|
enum sgp_type sgp_huge = sgp;
|
|
pgoff_t hindex = index;
|
|
int error;
|
|
int once = 0;
|
|
int alloced = 0;
|
|
|
|
if (index > (MAX_LFS_FILESIZE >> PAGE_SHIFT))
|
|
return -EFBIG;
|
|
if (sgp == SGP_NOHUGE || sgp == SGP_HUGE)
|
|
sgp = SGP_CACHE;
|
|
repeat:
|
|
swap.val = 0;
|
|
page = find_lock_entry(mapping, index);
|
|
if (radix_tree_exceptional_entry(page)) {
|
|
swap = radix_to_swp_entry(page);
|
|
page = NULL;
|
|
}
|
|
|
|
if (sgp <= SGP_CACHE &&
|
|
((loff_t)index << PAGE_SHIFT) >= i_size_read(inode)) {
|
|
error = -EINVAL;
|
|
goto unlock;
|
|
}
|
|
|
|
if (page && sgp == SGP_WRITE)
|
|
mark_page_accessed(page);
|
|
|
|
/* fallocated page? */
|
|
if (page && !PageUptodate(page)) {
|
|
if (sgp != SGP_READ)
|
|
goto clear;
|
|
unlock_page(page);
|
|
put_page(page);
|
|
page = NULL;
|
|
}
|
|
if (page || (sgp == SGP_READ && !swap.val)) {
|
|
*pagep = page;
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* Fast cache lookup did not find it:
|
|
* bring it back from swap or allocate.
|
|
*/
|
|
sbinfo = SHMEM_SB(inode->i_sb);
|
|
charge_mm = vma ? vma->vm_mm : current->mm;
|
|
|
|
if (swap.val) {
|
|
/* Look it up and read it in.. */
|
|
page = lookup_swap_cache(swap, NULL, 0);
|
|
if (!page) {
|
|
/* Or update major stats only when swapin succeeds?? */
|
|
if (fault_type) {
|
|
*fault_type |= VM_FAULT_MAJOR;
|
|
count_vm_event(PGMAJFAULT);
|
|
count_memcg_event_mm(charge_mm, PGMAJFAULT);
|
|
}
|
|
/* Here we actually start the io */
|
|
page = shmem_swapin(swap, gfp, info, index);
|
|
if (!page) {
|
|
error = -ENOMEM;
|
|
goto failed;
|
|
}
|
|
}
|
|
|
|
/* We have to do this with page locked to prevent races */
|
|
lock_page(page);
|
|
if (!PageSwapCache(page) || page_private(page) != swap.val ||
|
|
!shmem_confirm_swap(mapping, index, swap)) {
|
|
error = -EEXIST; /* try again */
|
|
goto unlock;
|
|
}
|
|
if (!PageUptodate(page)) {
|
|
error = -EIO;
|
|
goto failed;
|
|
}
|
|
wait_on_page_writeback(page);
|
|
|
|
if (shmem_should_replace_page(page, gfp)) {
|
|
error = shmem_replace_page(&page, gfp, info, index);
|
|
if (error)
|
|
goto failed;
|
|
}
|
|
|
|
error = mem_cgroup_try_charge_delay(page, charge_mm, gfp, &memcg,
|
|
false);
|
|
if (!error) {
|
|
error = shmem_add_to_page_cache(page, mapping, index,
|
|
swp_to_radix_entry(swap));
|
|
/*
|
|
* We already confirmed swap under page lock, and make
|
|
* no memory allocation here, so usually no possibility
|
|
* of error; but free_swap_and_cache() only trylocks a
|
|
* page, so it is just possible that the entry has been
|
|
* truncated or holepunched since swap was confirmed.
|
|
* shmem_undo_range() will have done some of the
|
|
* unaccounting, now delete_from_swap_cache() will do
|
|
* the rest.
|
|
* Reset swap.val? No, leave it so "failed" goes back to
|
|
* "repeat": reading a hole and writing should succeed.
|
|
*/
|
|
if (error) {
|
|
mem_cgroup_cancel_charge(page, memcg, false);
|
|
delete_from_swap_cache(page);
|
|
}
|
|
}
|
|
if (error)
|
|
goto failed;
|
|
|
|
mem_cgroup_commit_charge(page, memcg, true, false);
|
|
|
|
spin_lock_irq(&info->lock);
|
|
info->swapped--;
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
|
|
if (sgp == SGP_WRITE)
|
|
mark_page_accessed(page);
|
|
|
|
delete_from_swap_cache(page);
|
|
set_page_dirty(page);
|
|
swap_free(swap);
|
|
|
|
} else {
|
|
if (vma && userfaultfd_missing(vma)) {
|
|
*fault_type = handle_userfault(vmf, VM_UFFD_MISSING);
|
|
return 0;
|
|
}
|
|
|
|
/* shmem_symlink() */
|
|
if (mapping->a_ops != &shmem_aops)
|
|
goto alloc_nohuge;
|
|
if (shmem_huge == SHMEM_HUGE_DENY || sgp_huge == SGP_NOHUGE)
|
|
goto alloc_nohuge;
|
|
if (shmem_huge == SHMEM_HUGE_FORCE)
|
|
goto alloc_huge;
|
|
switch (sbinfo->huge) {
|
|
loff_t i_size;
|
|
pgoff_t off;
|
|
case SHMEM_HUGE_NEVER:
|
|
goto alloc_nohuge;
|
|
case SHMEM_HUGE_WITHIN_SIZE:
|
|
off = round_up(index, HPAGE_PMD_NR);
|
|
i_size = round_up(i_size_read(inode), PAGE_SIZE);
|
|
if (i_size >= HPAGE_PMD_SIZE &&
|
|
i_size >> PAGE_SHIFT >= off)
|
|
goto alloc_huge;
|
|
/* fallthrough */
|
|
case SHMEM_HUGE_ADVISE:
|
|
if (sgp_huge == SGP_HUGE)
|
|
goto alloc_huge;
|
|
/* TODO: implement fadvise() hints */
|
|
goto alloc_nohuge;
|
|
}
|
|
|
|
alloc_huge:
|
|
page = shmem_alloc_and_acct_page(gfp, inode, index, true);
|
|
if (IS_ERR(page)) {
|
|
alloc_nohuge: page = shmem_alloc_and_acct_page(gfp, inode,
|
|
index, false);
|
|
}
|
|
if (IS_ERR(page)) {
|
|
int retry = 5;
|
|
error = PTR_ERR(page);
|
|
page = NULL;
|
|
if (error != -ENOSPC)
|
|
goto failed;
|
|
/*
|
|
* Try to reclaim some spece by splitting a huge page
|
|
* beyond i_size on the filesystem.
|
|
*/
|
|
while (retry--) {
|
|
int ret;
|
|
ret = shmem_unused_huge_shrink(sbinfo, NULL, 1);
|
|
if (ret == SHRINK_STOP)
|
|
break;
|
|
if (ret)
|
|
goto alloc_nohuge;
|
|
}
|
|
goto failed;
|
|
}
|
|
|
|
if (PageTransHuge(page))
|
|
hindex = round_down(index, HPAGE_PMD_NR);
|
|
else
|
|
hindex = index;
|
|
|
|
if (sgp == SGP_WRITE)
|
|
__SetPageReferenced(page);
|
|
|
|
error = mem_cgroup_try_charge_delay(page, charge_mm, gfp, &memcg,
|
|
PageTransHuge(page));
|
|
if (error)
|
|
goto unacct;
|
|
error = radix_tree_maybe_preload_order(gfp & GFP_RECLAIM_MASK,
|
|
compound_order(page));
|
|
if (!error) {
|
|
error = shmem_add_to_page_cache(page, mapping, hindex,
|
|
NULL);
|
|
radix_tree_preload_end();
|
|
}
|
|
if (error) {
|
|
mem_cgroup_cancel_charge(page, memcg,
|
|
PageTransHuge(page));
|
|
goto unacct;
|
|
}
|
|
mem_cgroup_commit_charge(page, memcg, false,
|
|
PageTransHuge(page));
|
|
lru_cache_add_anon(page);
|
|
|
|
spin_lock_irq(&info->lock);
|
|
info->alloced += 1 << compound_order(page);
|
|
inode->i_blocks += BLOCKS_PER_PAGE << compound_order(page);
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
alloced = true;
|
|
|
|
if (PageTransHuge(page) &&
|
|
DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE) <
|
|
hindex + HPAGE_PMD_NR - 1) {
|
|
/*
|
|
* Part of the huge page is beyond i_size: subject
|
|
* to shrink under memory pressure.
|
|
*/
|
|
spin_lock(&sbinfo->shrinklist_lock);
|
|
/*
|
|
* _careful to defend against unlocked access to
|
|
* ->shrink_list in shmem_unused_huge_shrink()
|
|
*/
|
|
if (list_empty_careful(&info->shrinklist)) {
|
|
list_add_tail(&info->shrinklist,
|
|
&sbinfo->shrinklist);
|
|
sbinfo->shrinklist_len++;
|
|
}
|
|
spin_unlock(&sbinfo->shrinklist_lock);
|
|
}
|
|
|
|
/*
|
|
* Let SGP_FALLOC use the SGP_WRITE optimization on a new page.
|
|
*/
|
|
if (sgp == SGP_FALLOC)
|
|
sgp = SGP_WRITE;
|
|
clear:
|
|
/*
|
|
* Let SGP_WRITE caller clear ends if write does not fill page;
|
|
* but SGP_FALLOC on a page fallocated earlier must initialize
|
|
* it now, lest undo on failure cancel our earlier guarantee.
|
|
*/
|
|
if (sgp != SGP_WRITE && !PageUptodate(page)) {
|
|
struct page *head = compound_head(page);
|
|
int i;
|
|
|
|
for (i = 0; i < (1 << compound_order(head)); i++) {
|
|
clear_highpage(head + i);
|
|
flush_dcache_page(head + i);
|
|
}
|
|
SetPageUptodate(head);
|
|
}
|
|
}
|
|
|
|
/* Perhaps the file has been truncated since we checked */
|
|
if (sgp <= SGP_CACHE &&
|
|
((loff_t)index << PAGE_SHIFT) >= i_size_read(inode)) {
|
|
if (alloced) {
|
|
ClearPageDirty(page);
|
|
delete_from_page_cache(page);
|
|
spin_lock_irq(&info->lock);
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
}
|
|
error = -EINVAL;
|
|
goto unlock;
|
|
}
|
|
*pagep = page + index - hindex;
|
|
return 0;
|
|
|
|
/*
|
|
* Error recovery.
|
|
*/
|
|
unacct:
|
|
shmem_inode_unacct_blocks(inode, 1 << compound_order(page));
|
|
|
|
if (PageTransHuge(page)) {
|
|
unlock_page(page);
|
|
put_page(page);
|
|
goto alloc_nohuge;
|
|
}
|
|
failed:
|
|
if (swap.val && !shmem_confirm_swap(mapping, index, swap))
|
|
error = -EEXIST;
|
|
unlock:
|
|
if (page) {
|
|
unlock_page(page);
|
|
put_page(page);
|
|
}
|
|
if (error == -ENOSPC && !once++) {
|
|
spin_lock_irq(&info->lock);
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
goto repeat;
|
|
}
|
|
if (error == -EEXIST) /* from above or from radix_tree_insert */
|
|
goto repeat;
|
|
return error;
|
|
}
|
|
|
|
/*
|
|
* This is like autoremove_wake_function, but it removes the wait queue
|
|
* entry unconditionally - even if something else had already woken the
|
|
* target.
|
|
*/
|
|
static int synchronous_wake_function(wait_queue_entry_t *wait, unsigned mode, int sync, void *key)
|
|
{
|
|
int ret = default_wake_function(wait, mode, sync, key);
|
|
list_del_init(&wait->entry);
|
|
return ret;
|
|
}
|
|
|
|
static vm_fault_t shmem_fault(struct vm_fault *vmf)
|
|
{
|
|
struct vm_area_struct *vma = vmf->vma;
|
|
struct inode *inode = file_inode(vma->vm_file);
|
|
gfp_t gfp = mapping_gfp_mask(inode->i_mapping);
|
|
enum sgp_type sgp;
|
|
int err;
|
|
vm_fault_t ret = VM_FAULT_LOCKED;
|
|
|
|
/*
|
|
* Trinity finds that probing a hole which tmpfs is punching can
|
|
* prevent the hole-punch from ever completing: which in turn
|
|
* locks writers out with its hold on i_mutex. So refrain from
|
|
* faulting pages into the hole while it's being punched. Although
|
|
* shmem_undo_range() does remove the additions, it may be unable to
|
|
* keep up, as each new page needs its own unmap_mapping_range() call,
|
|
* and the i_mmap tree grows ever slower to scan if new vmas are added.
|
|
*
|
|
* It does not matter if we sometimes reach this check just before the
|
|
* hole-punch begins, so that one fault then races with the punch:
|
|
* we just need to make racing faults a rare case.
|
|
*
|
|
* The implementation below would be much simpler if we just used a
|
|
* standard mutex or completion: but we cannot take i_mutex in fault,
|
|
* and bloating every shmem inode for this unlikely case would be sad.
|
|
*/
|
|
if (unlikely(inode->i_private)) {
|
|
struct shmem_falloc *shmem_falloc;
|
|
|
|
spin_lock(&inode->i_lock);
|
|
shmem_falloc = inode->i_private;
|
|
if (shmem_falloc &&
|
|
shmem_falloc->waitq &&
|
|
vmf->pgoff >= shmem_falloc->start &&
|
|
vmf->pgoff < shmem_falloc->next) {
|
|
wait_queue_head_t *shmem_falloc_waitq;
|
|
DEFINE_WAIT_FUNC(shmem_fault_wait, synchronous_wake_function);
|
|
|
|
ret = VM_FAULT_NOPAGE;
|
|
if ((vmf->flags & FAULT_FLAG_ALLOW_RETRY) &&
|
|
!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) {
|
|
/* It's polite to up mmap_sem if we can */
|
|
up_read(&vma->vm_mm->mmap_sem);
|
|
ret = VM_FAULT_RETRY;
|
|
}
|
|
|
|
shmem_falloc_waitq = shmem_falloc->waitq;
|
|
prepare_to_wait(shmem_falloc_waitq, &shmem_fault_wait,
|
|
TASK_UNINTERRUPTIBLE);
|
|
spin_unlock(&inode->i_lock);
|
|
schedule();
|
|
|
|
/*
|
|
* shmem_falloc_waitq points into the shmem_fallocate()
|
|
* stack of the hole-punching task: shmem_falloc_waitq
|
|
* is usually invalid by the time we reach here, but
|
|
* finish_wait() does not dereference it in that case;
|
|
* though i_lock needed lest racing with wake_up_all().
|
|
*/
|
|
spin_lock(&inode->i_lock);
|
|
finish_wait(shmem_falloc_waitq, &shmem_fault_wait);
|
|
spin_unlock(&inode->i_lock);
|
|
return ret;
|
|
}
|
|
spin_unlock(&inode->i_lock);
|
|
}
|
|
|
|
sgp = SGP_CACHE;
|
|
|
|
if ((vmf->vma_flags & VM_NOHUGEPAGE) ||
|
|
test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags))
|
|
sgp = SGP_NOHUGE;
|
|
else if (vmf->vma_flags & VM_HUGEPAGE)
|
|
sgp = SGP_HUGE;
|
|
|
|
err = shmem_getpage_gfp(inode, vmf->pgoff, &vmf->page, sgp,
|
|
gfp, vma, vmf, &ret);
|
|
if (err)
|
|
return vmf_error(err);
|
|
return ret;
|
|
}
|
|
|
|
unsigned long shmem_get_unmapped_area(struct file *file,
|
|
unsigned long uaddr, unsigned long len,
|
|
unsigned long pgoff, unsigned long flags)
|
|
{
|
|
unsigned long (*get_area)(struct file *,
|
|
unsigned long, unsigned long, unsigned long, unsigned long);
|
|
unsigned long addr;
|
|
unsigned long offset;
|
|
unsigned long inflated_len;
|
|
unsigned long inflated_addr;
|
|
unsigned long inflated_offset;
|
|
|
|
if (len > TASK_SIZE)
|
|
return -ENOMEM;
|
|
|
|
get_area = current->mm->get_unmapped_area;
|
|
addr = get_area(file, uaddr, len, pgoff, flags);
|
|
|
|
if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE))
|
|
return addr;
|
|
if (IS_ERR_VALUE(addr))
|
|
return addr;
|
|
if (addr & ~PAGE_MASK)
|
|
return addr;
|
|
if (addr > TASK_SIZE - len)
|
|
return addr;
|
|
|
|
if (shmem_huge == SHMEM_HUGE_DENY)
|
|
return addr;
|
|
if (len < HPAGE_PMD_SIZE)
|
|
return addr;
|
|
if (flags & MAP_FIXED)
|
|
return addr;
|
|
/*
|
|
* Our priority is to support MAP_SHARED mapped hugely;
|
|
* and support MAP_PRIVATE mapped hugely too, until it is COWed.
|
|
* But if caller specified an address hint and we allocated area there
|
|
* successfully, respect that as before.
|
|
*/
|
|
if (uaddr == addr)
|
|
return addr;
|
|
|
|
if (shmem_huge != SHMEM_HUGE_FORCE) {
|
|
struct super_block *sb;
|
|
|
|
if (file) {
|
|
VM_BUG_ON(file->f_op != &shmem_file_operations);
|
|
sb = file_inode(file)->i_sb;
|
|
} else {
|
|
/*
|
|
* Called directly from mm/mmap.c, or drivers/char/mem.c
|
|
* for "/dev/zero", to create a shared anonymous object.
|
|
*/
|
|
if (IS_ERR(shm_mnt))
|
|
return addr;
|
|
sb = shm_mnt->mnt_sb;
|
|
}
|
|
if (SHMEM_SB(sb)->huge == SHMEM_HUGE_NEVER)
|
|
return addr;
|
|
}
|
|
|
|
offset = (pgoff << PAGE_SHIFT) & (HPAGE_PMD_SIZE-1);
|
|
if (offset && offset + len < 2 * HPAGE_PMD_SIZE)
|
|
return addr;
|
|
if ((addr & (HPAGE_PMD_SIZE-1)) == offset)
|
|
return addr;
|
|
|
|
inflated_len = len + HPAGE_PMD_SIZE - PAGE_SIZE;
|
|
if (inflated_len > TASK_SIZE)
|
|
return addr;
|
|
if (inflated_len < len)
|
|
return addr;
|
|
|
|
inflated_addr = get_area(NULL, uaddr, inflated_len, 0, flags);
|
|
if (IS_ERR_VALUE(inflated_addr))
|
|
return addr;
|
|
if (inflated_addr & ~PAGE_MASK)
|
|
return addr;
|
|
|
|
inflated_offset = inflated_addr & (HPAGE_PMD_SIZE-1);
|
|
inflated_addr += offset - inflated_offset;
|
|
if (inflated_offset > offset)
|
|
inflated_addr += HPAGE_PMD_SIZE;
|
|
|
|
if (inflated_addr > TASK_SIZE - len)
|
|
return addr;
|
|
return inflated_addr;
|
|
}
|
|
|
|
#ifdef CONFIG_NUMA
|
|
static int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpol)
|
|
{
|
|
struct inode *inode = file_inode(vma->vm_file);
|
|
return mpol_set_shared_policy(&SHMEM_I(inode)->policy, vma, mpol);
|
|
}
|
|
|
|
static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma,
|
|
unsigned long addr)
|
|
{
|
|
struct inode *inode = file_inode(vma->vm_file);
|
|
pgoff_t index;
|
|
|
|
index = ((addr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
|
|
return mpol_shared_policy_lookup(&SHMEM_I(inode)->policy, index);
|
|
}
|
|
#endif
|
|
|
|
int shmem_lock(struct file *file, int lock, struct user_struct *user)
|
|
{
|
|
struct inode *inode = file_inode(file);
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
int retval = -ENOMEM;
|
|
|
|
/*
|
|
* What serializes the accesses to info->flags?
|
|
* ipc_lock_object() when called from shmctl_do_lock(),
|
|
* no serialization needed when called from shm_destroy().
|
|
*/
|
|
if (lock && !(info->flags & VM_LOCKED)) {
|
|
if (!user_shm_lock(inode->i_size, user))
|
|
goto out_nomem;
|
|
info->flags |= VM_LOCKED;
|
|
mapping_set_unevictable(file->f_mapping);
|
|
}
|
|
if (!lock && (info->flags & VM_LOCKED) && user) {
|
|
user_shm_unlock(inode->i_size, user);
|
|
info->flags &= ~VM_LOCKED;
|
|
mapping_clear_unevictable(file->f_mapping);
|
|
}
|
|
retval = 0;
|
|
|
|
out_nomem:
|
|
return retval;
|
|
}
|
|
|
|
static int shmem_mmap(struct file *file, struct vm_area_struct *vma)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(file_inode(file));
|
|
|
|
if (info->seals & F_SEAL_FUTURE_WRITE) {
|
|
/*
|
|
* New PROT_WRITE and MAP_SHARED mmaps are not allowed when
|
|
* "future write" seal active.
|
|
*/
|
|
if ((vma->vm_flags & VM_SHARED) && (vma->vm_flags & VM_WRITE))
|
|
return -EPERM;
|
|
|
|
/*
|
|
* Since the F_SEAL_FUTURE_WRITE seals allow for a MAP_SHARED
|
|
* read-only mapping, take care to not allow mprotect to revert
|
|
* protections.
|
|
*/
|
|
vma->vm_flags &= ~(VM_MAYWRITE);
|
|
}
|
|
|
|
file_accessed(file);
|
|
vma->vm_ops = &shmem_vm_ops;
|
|
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE) &&
|
|
((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) <
|
|
(vma->vm_end & HPAGE_PMD_MASK)) {
|
|
khugepaged_enter(vma, vma->vm_flags);
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
static struct inode *shmem_get_inode(struct super_block *sb, const struct inode *dir,
|
|
umode_t mode, dev_t dev, unsigned long flags)
|
|
{
|
|
struct inode *inode;
|
|
struct shmem_inode_info *info;
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
|
|
if (shmem_reserve_inode(sb))
|
|
return NULL;
|
|
|
|
inode = new_inode(sb);
|
|
if (inode) {
|
|
inode->i_ino = get_next_ino();
|
|
inode_init_owner(inode, dir, mode);
|
|
inode->i_blocks = 0;
|
|
inode->i_atime = inode->i_mtime = inode->i_ctime = current_time(inode);
|
|
inode->i_generation = prandom_u32();
|
|
info = SHMEM_I(inode);
|
|
memset(info, 0, (char *)inode - (char *)info);
|
|
spin_lock_init(&info->lock);
|
|
info->seals = F_SEAL_SEAL;
|
|
info->flags = flags & VM_NORESERVE;
|
|
INIT_LIST_HEAD(&info->shrinklist);
|
|
INIT_LIST_HEAD(&info->swaplist);
|
|
simple_xattrs_init(&info->xattrs);
|
|
cache_no_acl(inode);
|
|
|
|
switch (mode & S_IFMT) {
|
|
default:
|
|
inode->i_op = &shmem_special_inode_operations;
|
|
init_special_inode(inode, mode, dev);
|
|
break;
|
|
case S_IFREG:
|
|
inode->i_mapping->a_ops = &shmem_aops;
|
|
inode->i_op = &shmem_inode_operations;
|
|
inode->i_fop = &shmem_file_operations;
|
|
mpol_shared_policy_init(&info->policy,
|
|
shmem_get_sbmpol(sbinfo));
|
|
break;
|
|
case S_IFDIR:
|
|
inc_nlink(inode);
|
|
/* Some things misbehave if size == 0 on a directory */
|
|
inode->i_size = 2 * BOGO_DIRENT_SIZE;
|
|
inode->i_op = &shmem_dir_inode_operations;
|
|
inode->i_fop = &simple_dir_operations;
|
|
break;
|
|
case S_IFLNK:
|
|
/*
|
|
* Must not load anything in the rbtree,
|
|
* mpol_free_shared_policy will not be called.
|
|
*/
|
|
mpol_shared_policy_init(&info->policy, NULL);
|
|
break;
|
|
}
|
|
|
|
lockdep_annotate_inode_mutex_key(inode);
|
|
} else
|
|
shmem_free_inode(sb);
|
|
return inode;
|
|
}
|
|
|
|
bool shmem_mapping(struct address_space *mapping)
|
|
{
|
|
return mapping->a_ops == &shmem_aops;
|
|
}
|
|
|
|
static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm,
|
|
pmd_t *dst_pmd,
|
|
struct vm_area_struct *dst_vma,
|
|
unsigned long dst_addr,
|
|
unsigned long src_addr,
|
|
bool zeropage,
|
|
struct page **pagep)
|
|
{
|
|
struct inode *inode = file_inode(dst_vma->vm_file);
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct address_space *mapping = inode->i_mapping;
|
|
gfp_t gfp = mapping_gfp_mask(mapping);
|
|
pgoff_t pgoff = linear_page_index(dst_vma, dst_addr);
|
|
struct mem_cgroup *memcg;
|
|
spinlock_t *ptl;
|
|
void *page_kaddr;
|
|
struct page *page;
|
|
pte_t _dst_pte, *dst_pte;
|
|
int ret;
|
|
pgoff_t offset, max_off;
|
|
|
|
ret = -ENOMEM;
|
|
if (!shmem_inode_acct_block(inode, 1))
|
|
goto out;
|
|
|
|
if (!*pagep) {
|
|
page = shmem_alloc_page(gfp, info, pgoff);
|
|
if (!page)
|
|
goto out_unacct_blocks;
|
|
|
|
if (!zeropage) { /* mcopy_atomic */
|
|
page_kaddr = kmap_atomic(page);
|
|
ret = copy_from_user(page_kaddr,
|
|
(const void __user *)src_addr,
|
|
PAGE_SIZE);
|
|
kunmap_atomic(page_kaddr);
|
|
|
|
/* fallback to copy_from_user outside mmap_sem */
|
|
if (unlikely(ret)) {
|
|
*pagep = page;
|
|
shmem_inode_unacct_blocks(inode, 1);
|
|
/* don't free the page */
|
|
return -ENOENT;
|
|
}
|
|
} else { /* mfill_zeropage_atomic */
|
|
clear_highpage(page);
|
|
}
|
|
} else {
|
|
page = *pagep;
|
|
*pagep = NULL;
|
|
}
|
|
|
|
VM_BUG_ON(PageLocked(page) || PageSwapBacked(page));
|
|
__SetPageLocked(page);
|
|
__SetPageSwapBacked(page);
|
|
__SetPageUptodate(page);
|
|
|
|
ret = -EFAULT;
|
|
offset = linear_page_index(dst_vma, dst_addr);
|
|
max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
|
|
if (unlikely(offset >= max_off))
|
|
goto out_release;
|
|
|
|
ret = mem_cgroup_try_charge_delay(page, dst_mm, gfp, &memcg, false);
|
|
if (ret)
|
|
goto out_release;
|
|
|
|
ret = radix_tree_maybe_preload(gfp & GFP_RECLAIM_MASK);
|
|
if (!ret) {
|
|
ret = shmem_add_to_page_cache(page, mapping, pgoff, NULL);
|
|
radix_tree_preload_end();
|
|
}
|
|
if (ret)
|
|
goto out_release_uncharge;
|
|
|
|
mem_cgroup_commit_charge(page, memcg, false, false);
|
|
|
|
_dst_pte = mk_pte(page, dst_vma->vm_page_prot);
|
|
if (dst_vma->vm_flags & VM_WRITE)
|
|
_dst_pte = pte_mkwrite(pte_mkdirty(_dst_pte));
|
|
else {
|
|
/*
|
|
* We don't set the pte dirty if the vma has no
|
|
* VM_WRITE permission, so mark the page dirty or it
|
|
* could be freed from under us. We could do it
|
|
* unconditionally before unlock_page(), but doing it
|
|
* only if VM_WRITE is not set is faster.
|
|
*/
|
|
set_page_dirty(page);
|
|
}
|
|
|
|
dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl);
|
|
|
|
ret = -EFAULT;
|
|
max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE);
|
|
if (unlikely(offset >= max_off))
|
|
goto out_release_uncharge_unlock;
|
|
|
|
ret = -EEXIST;
|
|
if (!pte_none(*dst_pte))
|
|
goto out_release_uncharge_unlock;
|
|
|
|
lru_cache_add_anon(page);
|
|
|
|
spin_lock_irq(&info->lock);
|
|
info->alloced++;
|
|
inode->i_blocks += BLOCKS_PER_PAGE;
|
|
shmem_recalc_inode(inode);
|
|
spin_unlock_irq(&info->lock);
|
|
|
|
inc_mm_counter(dst_mm, mm_counter_file(page));
|
|
page_add_file_rmap(page, false);
|
|
set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte);
|
|
|
|
/* No need to invalidate - it was non-present before */
|
|
update_mmu_cache(dst_vma, dst_addr, dst_pte);
|
|
pte_unmap_unlock(dst_pte, ptl);
|
|
unlock_page(page);
|
|
ret = 0;
|
|
out:
|
|
return ret;
|
|
out_release_uncharge_unlock:
|
|
pte_unmap_unlock(dst_pte, ptl);
|
|
ClearPageDirty(page);
|
|
delete_from_page_cache(page);
|
|
out_release_uncharge:
|
|
mem_cgroup_cancel_charge(page, memcg, false);
|
|
out_release:
|
|
unlock_page(page);
|
|
put_page(page);
|
|
out_unacct_blocks:
|
|
shmem_inode_unacct_blocks(inode, 1);
|
|
goto out;
|
|
}
|
|
|
|
int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm,
|
|
pmd_t *dst_pmd,
|
|
struct vm_area_struct *dst_vma,
|
|
unsigned long dst_addr,
|
|
unsigned long src_addr,
|
|
struct page **pagep)
|
|
{
|
|
return shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma,
|
|
dst_addr, src_addr, false, pagep);
|
|
}
|
|
|
|
int shmem_mfill_zeropage_pte(struct mm_struct *dst_mm,
|
|
pmd_t *dst_pmd,
|
|
struct vm_area_struct *dst_vma,
|
|
unsigned long dst_addr)
|
|
{
|
|
struct page *page = NULL;
|
|
|
|
return shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma,
|
|
dst_addr, 0, true, &page);
|
|
}
|
|
|
|
#ifdef CONFIG_TMPFS
|
|
static const struct inode_operations shmem_symlink_inode_operations;
|
|
static const struct inode_operations shmem_short_symlink_operations;
|
|
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
static int shmem_initxattrs(struct inode *, const struct xattr *, void *);
|
|
#else
|
|
#define shmem_initxattrs NULL
|
|
#endif
|
|
|
|
static int
|
|
shmem_write_begin(struct file *file, struct address_space *mapping,
|
|
loff_t pos, unsigned len, unsigned flags,
|
|
struct page **pagep, void **fsdata)
|
|
{
|
|
struct inode *inode = mapping->host;
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
pgoff_t index = pos >> PAGE_SHIFT;
|
|
|
|
/* i_mutex is held by caller */
|
|
if (unlikely(info->seals & (F_SEAL_GROW |
|
|
F_SEAL_WRITE | F_SEAL_FUTURE_WRITE))) {
|
|
if (info->seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE))
|
|
return -EPERM;
|
|
if ((info->seals & F_SEAL_GROW) && pos + len > inode->i_size)
|
|
return -EPERM;
|
|
}
|
|
|
|
return shmem_getpage(inode, index, pagep, SGP_WRITE);
|
|
}
|
|
|
|
static int
|
|
shmem_write_end(struct file *file, struct address_space *mapping,
|
|
loff_t pos, unsigned len, unsigned copied,
|
|
struct page *page, void *fsdata)
|
|
{
|
|
struct inode *inode = mapping->host;
|
|
|
|
if (pos + copied > inode->i_size)
|
|
i_size_write(inode, pos + copied);
|
|
|
|
if (!PageUptodate(page)) {
|
|
struct page *head = compound_head(page);
|
|
if (PageTransCompound(page)) {
|
|
int i;
|
|
|
|
for (i = 0; i < HPAGE_PMD_NR; i++) {
|
|
if (head + i == page)
|
|
continue;
|
|
clear_highpage(head + i);
|
|
flush_dcache_page(head + i);
|
|
}
|
|
}
|
|
if (copied < PAGE_SIZE) {
|
|
unsigned from = pos & (PAGE_SIZE - 1);
|
|
zero_user_segments(page, 0, from,
|
|
from + copied, PAGE_SIZE);
|
|
}
|
|
SetPageUptodate(head);
|
|
}
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
|
|
return copied;
|
|
}
|
|
|
|
static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
|
|
{
|
|
struct file *file = iocb->ki_filp;
|
|
struct inode *inode = file_inode(file);
|
|
struct address_space *mapping = inode->i_mapping;
|
|
pgoff_t index;
|
|
unsigned long offset;
|
|
enum sgp_type sgp = SGP_READ;
|
|
int error = 0;
|
|
ssize_t retval = 0;
|
|
loff_t *ppos = &iocb->ki_pos;
|
|
|
|
/*
|
|
* Might this read be for a stacking filesystem? Then when reading
|
|
* holes of a sparse file, we actually need to allocate those pages,
|
|
* and even mark them dirty, so it cannot exceed the max_blocks limit.
|
|
*/
|
|
if (!iter_is_iovec(to))
|
|
sgp = SGP_CACHE;
|
|
|
|
index = *ppos >> PAGE_SHIFT;
|
|
offset = *ppos & ~PAGE_MASK;
|
|
|
|
for (;;) {
|
|
struct page *page = NULL;
|
|
pgoff_t end_index;
|
|
unsigned long nr, ret;
|
|
loff_t i_size = i_size_read(inode);
|
|
|
|
end_index = i_size >> PAGE_SHIFT;
|
|
if (index > end_index)
|
|
break;
|
|
if (index == end_index) {
|
|
nr = i_size & ~PAGE_MASK;
|
|
if (nr <= offset)
|
|
break;
|
|
}
|
|
|
|
error = shmem_getpage(inode, index, &page, sgp);
|
|
if (error) {
|
|
if (error == -EINVAL)
|
|
error = 0;
|
|
break;
|
|
}
|
|
if (page) {
|
|
if (sgp == SGP_CACHE)
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
}
|
|
|
|
/*
|
|
* We must evaluate after, since reads (unlike writes)
|
|
* are called without i_mutex protection against truncate
|
|
*/
|
|
nr = PAGE_SIZE;
|
|
i_size = i_size_read(inode);
|
|
end_index = i_size >> PAGE_SHIFT;
|
|
if (index == end_index) {
|
|
nr = i_size & ~PAGE_MASK;
|
|
if (nr <= offset) {
|
|
if (page)
|
|
put_page(page);
|
|
break;
|
|
}
|
|
}
|
|
nr -= offset;
|
|
|
|
if (page) {
|
|
/*
|
|
* If users can be writing to this page using arbitrary
|
|
* virtual addresses, take care about potential aliasing
|
|
* before reading the page on the kernel side.
|
|
*/
|
|
if (mapping_writably_mapped(mapping))
|
|
flush_dcache_page(page);
|
|
/*
|
|
* Mark the page accessed if we read the beginning.
|
|
*/
|
|
if (!offset)
|
|
mark_page_accessed(page);
|
|
} else {
|
|
page = ZERO_PAGE(0);
|
|
get_page(page);
|
|
}
|
|
|
|
/*
|
|
* Ok, we have the page, and it's up-to-date, so
|
|
* now we can copy it to user space...
|
|
*/
|
|
ret = copy_page_to_iter(page, offset, nr, to);
|
|
retval += ret;
|
|
offset += ret;
|
|
index += offset >> PAGE_SHIFT;
|
|
offset &= ~PAGE_MASK;
|
|
|
|
put_page(page);
|
|
if (!iov_iter_count(to))
|
|
break;
|
|
if (ret < nr) {
|
|
error = -EFAULT;
|
|
break;
|
|
}
|
|
cond_resched();
|
|
}
|
|
|
|
*ppos = ((loff_t) index << PAGE_SHIFT) + offset;
|
|
file_accessed(file);
|
|
return retval ? retval : error;
|
|
}
|
|
|
|
/*
|
|
* llseek SEEK_DATA or SEEK_HOLE through the radix_tree.
|
|
*/
|
|
static pgoff_t shmem_seek_hole_data(struct address_space *mapping,
|
|
pgoff_t index, pgoff_t end, int whence)
|
|
{
|
|
struct page *page;
|
|
struct pagevec pvec;
|
|
pgoff_t indices[PAGEVEC_SIZE];
|
|
bool done = false;
|
|
int i;
|
|
|
|
pagevec_init(&pvec);
|
|
pvec.nr = 1; /* start small: we may be there already */
|
|
while (!done) {
|
|
pvec.nr = find_get_entries(mapping, index,
|
|
pvec.nr, pvec.pages, indices);
|
|
if (!pvec.nr) {
|
|
if (whence == SEEK_DATA)
|
|
index = end;
|
|
break;
|
|
}
|
|
for (i = 0; i < pvec.nr; i++, index++) {
|
|
if (index < indices[i]) {
|
|
if (whence == SEEK_HOLE) {
|
|
done = true;
|
|
break;
|
|
}
|
|
index = indices[i];
|
|
}
|
|
page = pvec.pages[i];
|
|
if (page && !radix_tree_exceptional_entry(page)) {
|
|
if (!PageUptodate(page))
|
|
page = NULL;
|
|
}
|
|
if (index >= end ||
|
|
(page && whence == SEEK_DATA) ||
|
|
(!page && whence == SEEK_HOLE)) {
|
|
done = true;
|
|
break;
|
|
}
|
|
}
|
|
pagevec_remove_exceptionals(&pvec);
|
|
pagevec_release(&pvec);
|
|
pvec.nr = PAGEVEC_SIZE;
|
|
cond_resched();
|
|
}
|
|
return index;
|
|
}
|
|
|
|
static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence)
|
|
{
|
|
struct address_space *mapping = file->f_mapping;
|
|
struct inode *inode = mapping->host;
|
|
pgoff_t start, end;
|
|
loff_t new_offset;
|
|
|
|
if (whence != SEEK_DATA && whence != SEEK_HOLE)
|
|
return generic_file_llseek_size(file, offset, whence,
|
|
MAX_LFS_FILESIZE, i_size_read(inode));
|
|
inode_lock(inode);
|
|
/* We're holding i_mutex so we can access i_size directly */
|
|
|
|
if (offset < 0 || offset >= inode->i_size)
|
|
offset = -ENXIO;
|
|
else {
|
|
start = offset >> PAGE_SHIFT;
|
|
end = (inode->i_size + PAGE_SIZE - 1) >> PAGE_SHIFT;
|
|
new_offset = shmem_seek_hole_data(mapping, start, end, whence);
|
|
new_offset <<= PAGE_SHIFT;
|
|
if (new_offset > offset) {
|
|
if (new_offset < inode->i_size)
|
|
offset = new_offset;
|
|
else if (whence == SEEK_DATA)
|
|
offset = -ENXIO;
|
|
else
|
|
offset = inode->i_size;
|
|
}
|
|
}
|
|
|
|
if (offset >= 0)
|
|
offset = vfs_setpos(file, offset, MAX_LFS_FILESIZE);
|
|
inode_unlock(inode);
|
|
return offset;
|
|
}
|
|
|
|
static long shmem_fallocate(struct file *file, int mode, loff_t offset,
|
|
loff_t len)
|
|
{
|
|
struct inode *inode = file_inode(file);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
struct shmem_falloc shmem_falloc;
|
|
pgoff_t start, index, end;
|
|
int error;
|
|
|
|
if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE))
|
|
return -EOPNOTSUPP;
|
|
|
|
inode_lock(inode);
|
|
|
|
if (mode & FALLOC_FL_PUNCH_HOLE) {
|
|
struct address_space *mapping = file->f_mapping;
|
|
loff_t unmap_start = round_up(offset, PAGE_SIZE);
|
|
loff_t unmap_end = round_down(offset + len, PAGE_SIZE) - 1;
|
|
DECLARE_WAIT_QUEUE_HEAD_ONSTACK(shmem_falloc_waitq);
|
|
|
|
/* protected by i_mutex */
|
|
if (info->seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE)) {
|
|
error = -EPERM;
|
|
goto out;
|
|
}
|
|
|
|
shmem_falloc.waitq = &shmem_falloc_waitq;
|
|
shmem_falloc.start = (u64)unmap_start >> PAGE_SHIFT;
|
|
shmem_falloc.next = (unmap_end + 1) >> PAGE_SHIFT;
|
|
spin_lock(&inode->i_lock);
|
|
inode->i_private = &shmem_falloc;
|
|
spin_unlock(&inode->i_lock);
|
|
|
|
if ((u64)unmap_end > (u64)unmap_start)
|
|
unmap_mapping_range(mapping, unmap_start,
|
|
1 + unmap_end - unmap_start, 0);
|
|
shmem_truncate_range(inode, offset, offset + len - 1);
|
|
/* No need to unmap again: hole-punching leaves COWed pages */
|
|
|
|
spin_lock(&inode->i_lock);
|
|
inode->i_private = NULL;
|
|
wake_up_all(&shmem_falloc_waitq);
|
|
WARN_ON_ONCE(!list_empty(&shmem_falloc_waitq.head));
|
|
spin_unlock(&inode->i_lock);
|
|
error = 0;
|
|
goto out;
|
|
}
|
|
|
|
/* We need to check rlimit even when FALLOC_FL_KEEP_SIZE */
|
|
error = inode_newsize_ok(inode, offset + len);
|
|
if (error)
|
|
goto out;
|
|
|
|
if ((info->seals & F_SEAL_GROW) && offset + len > inode->i_size) {
|
|
error = -EPERM;
|
|
goto out;
|
|
}
|
|
|
|
start = offset >> PAGE_SHIFT;
|
|
end = (offset + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
|
|
/* Try to avoid a swapstorm if len is impossible to satisfy */
|
|
if (sbinfo->max_blocks && end - start > sbinfo->max_blocks) {
|
|
error = -ENOSPC;
|
|
goto out;
|
|
}
|
|
|
|
shmem_falloc.waitq = NULL;
|
|
shmem_falloc.start = start;
|
|
shmem_falloc.next = start;
|
|
shmem_falloc.nr_falloced = 0;
|
|
shmem_falloc.nr_unswapped = 0;
|
|
spin_lock(&inode->i_lock);
|
|
inode->i_private = &shmem_falloc;
|
|
spin_unlock(&inode->i_lock);
|
|
|
|
for (index = start; index < end; index++) {
|
|
struct page *page;
|
|
|
|
/*
|
|
* Good, the fallocate(2) manpage permits EINTR: we may have
|
|
* been interrupted because we are using up too much memory.
|
|
*/
|
|
if (signal_pending(current))
|
|
error = -EINTR;
|
|
else if (shmem_falloc.nr_unswapped > shmem_falloc.nr_falloced)
|
|
error = -ENOMEM;
|
|
else
|
|
error = shmem_getpage(inode, index, &page, SGP_FALLOC);
|
|
if (error) {
|
|
/* Remove the !PageUptodate pages we added */
|
|
if (index > start) {
|
|
shmem_undo_range(inode,
|
|
(loff_t)start << PAGE_SHIFT,
|
|
((loff_t)index << PAGE_SHIFT) - 1, true);
|
|
}
|
|
goto undone;
|
|
}
|
|
|
|
/*
|
|
* Inform shmem_writepage() how far we have reached.
|
|
* No need for lock or barrier: we have the page lock.
|
|
*/
|
|
shmem_falloc.next++;
|
|
if (!PageUptodate(page))
|
|
shmem_falloc.nr_falloced++;
|
|
|
|
/*
|
|
* If !PageUptodate, leave it that way so that freeable pages
|
|
* can be recognized if we need to rollback on error later.
|
|
* But set_page_dirty so that memory pressure will swap rather
|
|
* than free the pages we are allocating (and SGP_CACHE pages
|
|
* might still be clean: we now need to mark those dirty too).
|
|
*/
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
cond_resched();
|
|
}
|
|
|
|
if (!(mode & FALLOC_FL_KEEP_SIZE) && offset + len > inode->i_size)
|
|
i_size_write(inode, offset + len);
|
|
inode->i_ctime = current_time(inode);
|
|
undone:
|
|
spin_lock(&inode->i_lock);
|
|
inode->i_private = NULL;
|
|
spin_unlock(&inode->i_lock);
|
|
out:
|
|
inode_unlock(inode);
|
|
return error;
|
|
}
|
|
|
|
static int shmem_statfs(struct dentry *dentry, struct kstatfs *buf)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(dentry->d_sb);
|
|
|
|
buf->f_type = TMPFS_MAGIC;
|
|
buf->f_bsize = PAGE_SIZE;
|
|
buf->f_namelen = NAME_MAX;
|
|
if (sbinfo->max_blocks) {
|
|
buf->f_blocks = sbinfo->max_blocks;
|
|
buf->f_bavail =
|
|
buf->f_bfree = sbinfo->max_blocks -
|
|
percpu_counter_sum(&sbinfo->used_blocks);
|
|
}
|
|
if (sbinfo->max_inodes) {
|
|
buf->f_files = sbinfo->max_inodes;
|
|
buf->f_ffree = sbinfo->free_inodes;
|
|
}
|
|
/* else leave those fields 0 like simple_statfs */
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* File creation. Allocate an inode, and we're done..
|
|
*/
|
|
static int
|
|
shmem_mknod(struct inode *dir, struct dentry *dentry, umode_t mode, dev_t dev)
|
|
{
|
|
struct inode *inode;
|
|
int error = -ENOSPC;
|
|
|
|
inode = shmem_get_inode(dir->i_sb, dir, mode, dev, VM_NORESERVE);
|
|
if (inode) {
|
|
error = simple_acl_create(dir, inode);
|
|
if (error)
|
|
goto out_iput;
|
|
error = security_inode_init_security(inode, dir,
|
|
&dentry->d_name,
|
|
shmem_initxattrs, NULL);
|
|
if (error && error != -EOPNOTSUPP)
|
|
goto out_iput;
|
|
|
|
error = 0;
|
|
dir->i_size += BOGO_DIRENT_SIZE;
|
|
dir->i_ctime = dir->i_mtime = current_time(dir);
|
|
d_instantiate(dentry, inode);
|
|
dget(dentry); /* Extra count - pin the dentry in core */
|
|
}
|
|
return error;
|
|
out_iput:
|
|
iput(inode);
|
|
return error;
|
|
}
|
|
|
|
static int
|
|
shmem_tmpfile(struct inode *dir, struct dentry *dentry, umode_t mode)
|
|
{
|
|
struct inode *inode;
|
|
int error = -ENOSPC;
|
|
|
|
inode = shmem_get_inode(dir->i_sb, dir, mode, 0, VM_NORESERVE);
|
|
if (inode) {
|
|
error = security_inode_init_security(inode, dir,
|
|
NULL,
|
|
shmem_initxattrs, NULL);
|
|
if (error && error != -EOPNOTSUPP)
|
|
goto out_iput;
|
|
error = simple_acl_create(dir, inode);
|
|
if (error)
|
|
goto out_iput;
|
|
d_tmpfile(dentry, inode);
|
|
}
|
|
return error;
|
|
out_iput:
|
|
iput(inode);
|
|
return error;
|
|
}
|
|
|
|
static int shmem_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
|
|
{
|
|
int error;
|
|
|
|
if ((error = shmem_mknod(dir, dentry, mode | S_IFDIR, 0)))
|
|
return error;
|
|
inc_nlink(dir);
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_create(struct inode *dir, struct dentry *dentry, umode_t mode,
|
|
bool excl)
|
|
{
|
|
return shmem_mknod(dir, dentry, mode | S_IFREG, 0);
|
|
}
|
|
|
|
/*
|
|
* Link a file..
|
|
*/
|
|
static int shmem_link(struct dentry *old_dentry, struct inode *dir, struct dentry *dentry)
|
|
{
|
|
struct inode *inode = d_inode(old_dentry);
|
|
int ret = 0;
|
|
|
|
/*
|
|
* No ordinary (disk based) filesystem counts links as inodes;
|
|
* but each new link needs a new dentry, pinning lowmem, and
|
|
* tmpfs dentries cannot be pruned until they are unlinked.
|
|
* But if an O_TMPFILE file is linked into the tmpfs, the
|
|
* first link must skip that, to get the accounting right.
|
|
*/
|
|
if (inode->i_nlink) {
|
|
ret = shmem_reserve_inode(inode->i_sb);
|
|
if (ret)
|
|
goto out;
|
|
}
|
|
|
|
dir->i_size += BOGO_DIRENT_SIZE;
|
|
inode->i_ctime = dir->i_ctime = dir->i_mtime = current_time(inode);
|
|
inc_nlink(inode);
|
|
ihold(inode); /* New dentry reference */
|
|
dget(dentry); /* Extra pinning count for the created dentry */
|
|
d_instantiate(dentry, inode);
|
|
out:
|
|
return ret;
|
|
}
|
|
|
|
static int shmem_unlink(struct inode *dir, struct dentry *dentry)
|
|
{
|
|
struct inode *inode = d_inode(dentry);
|
|
|
|
if (inode->i_nlink > 1 && !S_ISDIR(inode->i_mode))
|
|
shmem_free_inode(inode->i_sb);
|
|
|
|
dir->i_size -= BOGO_DIRENT_SIZE;
|
|
inode->i_ctime = dir->i_ctime = dir->i_mtime = current_time(inode);
|
|
drop_nlink(inode);
|
|
dput(dentry); /* Undo the count from "create" - this does all the work */
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_rmdir(struct inode *dir, struct dentry *dentry)
|
|
{
|
|
if (!simple_empty(dentry))
|
|
return -ENOTEMPTY;
|
|
|
|
drop_nlink(d_inode(dentry));
|
|
drop_nlink(dir);
|
|
return shmem_unlink(dir, dentry);
|
|
}
|
|
|
|
static int shmem_exchange(struct inode *old_dir, struct dentry *old_dentry, struct inode *new_dir, struct dentry *new_dentry)
|
|
{
|
|
bool old_is_dir = d_is_dir(old_dentry);
|
|
bool new_is_dir = d_is_dir(new_dentry);
|
|
|
|
if (old_dir != new_dir && old_is_dir != new_is_dir) {
|
|
if (old_is_dir) {
|
|
drop_nlink(old_dir);
|
|
inc_nlink(new_dir);
|
|
} else {
|
|
drop_nlink(new_dir);
|
|
inc_nlink(old_dir);
|
|
}
|
|
}
|
|
old_dir->i_ctime = old_dir->i_mtime =
|
|
new_dir->i_ctime = new_dir->i_mtime =
|
|
d_inode(old_dentry)->i_ctime =
|
|
d_inode(new_dentry)->i_ctime = current_time(old_dir);
|
|
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_whiteout(struct inode *old_dir, struct dentry *old_dentry)
|
|
{
|
|
struct dentry *whiteout;
|
|
int error;
|
|
|
|
whiteout = d_alloc(old_dentry->d_parent, &old_dentry->d_name);
|
|
if (!whiteout)
|
|
return -ENOMEM;
|
|
|
|
error = shmem_mknod(old_dir, whiteout,
|
|
S_IFCHR | WHITEOUT_MODE, WHITEOUT_DEV);
|
|
dput(whiteout);
|
|
if (error)
|
|
return error;
|
|
|
|
/*
|
|
* Cheat and hash the whiteout while the old dentry is still in
|
|
* place, instead of playing games with FS_RENAME_DOES_D_MOVE.
|
|
*
|
|
* d_lookup() will consistently find one of them at this point,
|
|
* not sure which one, but that isn't even important.
|
|
*/
|
|
d_rehash(whiteout);
|
|
return 0;
|
|
}
|
|
|
|
/*
|
|
* The VFS layer already does all the dentry stuff for rename,
|
|
* we just have to decrement the usage count for the target if
|
|
* it exists so that the VFS layer correctly free's it when it
|
|
* gets overwritten.
|
|
*/
|
|
static int shmem_rename2(struct inode *old_dir, struct dentry *old_dentry, struct inode *new_dir, struct dentry *new_dentry, unsigned int flags)
|
|
{
|
|
struct inode *inode = d_inode(old_dentry);
|
|
int they_are_dirs = S_ISDIR(inode->i_mode);
|
|
|
|
if (flags & ~(RENAME_NOREPLACE | RENAME_EXCHANGE | RENAME_WHITEOUT))
|
|
return -EINVAL;
|
|
|
|
if (flags & RENAME_EXCHANGE)
|
|
return shmem_exchange(old_dir, old_dentry, new_dir, new_dentry);
|
|
|
|
if (!simple_empty(new_dentry))
|
|
return -ENOTEMPTY;
|
|
|
|
if (flags & RENAME_WHITEOUT) {
|
|
int error;
|
|
|
|
error = shmem_whiteout(old_dir, old_dentry);
|
|
if (error)
|
|
return error;
|
|
}
|
|
|
|
if (d_really_is_positive(new_dentry)) {
|
|
(void) shmem_unlink(new_dir, new_dentry);
|
|
if (they_are_dirs) {
|
|
drop_nlink(d_inode(new_dentry));
|
|
drop_nlink(old_dir);
|
|
}
|
|
} else if (they_are_dirs) {
|
|
drop_nlink(old_dir);
|
|
inc_nlink(new_dir);
|
|
}
|
|
|
|
old_dir->i_size -= BOGO_DIRENT_SIZE;
|
|
new_dir->i_size += BOGO_DIRENT_SIZE;
|
|
old_dir->i_ctime = old_dir->i_mtime =
|
|
new_dir->i_ctime = new_dir->i_mtime =
|
|
inode->i_ctime = current_time(old_dir);
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_symlink(struct inode *dir, struct dentry *dentry, const char *symname)
|
|
{
|
|
int error;
|
|
int len;
|
|
struct inode *inode;
|
|
struct page *page;
|
|
|
|
len = strlen(symname) + 1;
|
|
if (len > PAGE_SIZE)
|
|
return -ENAMETOOLONG;
|
|
|
|
inode = shmem_get_inode(dir->i_sb, dir, S_IFLNK | 0777, 0,
|
|
VM_NORESERVE);
|
|
if (!inode)
|
|
return -ENOSPC;
|
|
|
|
error = security_inode_init_security(inode, dir, &dentry->d_name,
|
|
shmem_initxattrs, NULL);
|
|
if (error) {
|
|
if (error != -EOPNOTSUPP) {
|
|
iput(inode);
|
|
return error;
|
|
}
|
|
error = 0;
|
|
}
|
|
|
|
inode->i_size = len-1;
|
|
if (len <= SHORT_SYMLINK_LEN) {
|
|
inode->i_link = kmemdup(symname, len, GFP_KERNEL);
|
|
if (!inode->i_link) {
|
|
iput(inode);
|
|
return -ENOMEM;
|
|
}
|
|
inode->i_op = &shmem_short_symlink_operations;
|
|
} else {
|
|
inode_nohighmem(inode);
|
|
error = shmem_getpage(inode, 0, &page, SGP_WRITE);
|
|
if (error) {
|
|
iput(inode);
|
|
return error;
|
|
}
|
|
inode->i_mapping->a_ops = &shmem_aops;
|
|
inode->i_op = &shmem_symlink_inode_operations;
|
|
memcpy(page_address(page), symname, len);
|
|
SetPageUptodate(page);
|
|
set_page_dirty(page);
|
|
unlock_page(page);
|
|
put_page(page);
|
|
}
|
|
dir->i_size += BOGO_DIRENT_SIZE;
|
|
dir->i_ctime = dir->i_mtime = current_time(dir);
|
|
d_instantiate(dentry, inode);
|
|
dget(dentry);
|
|
return 0;
|
|
}
|
|
|
|
static void shmem_put_link(void *arg)
|
|
{
|
|
mark_page_accessed(arg);
|
|
put_page(arg);
|
|
}
|
|
|
|
static const char *shmem_get_link(struct dentry *dentry,
|
|
struct inode *inode,
|
|
struct delayed_call *done)
|
|
{
|
|
struct page *page = NULL;
|
|
int error;
|
|
if (!dentry) {
|
|
page = find_get_page(inode->i_mapping, 0);
|
|
if (!page)
|
|
return ERR_PTR(-ECHILD);
|
|
if (!PageUptodate(page)) {
|
|
put_page(page);
|
|
return ERR_PTR(-ECHILD);
|
|
}
|
|
} else {
|
|
error = shmem_getpage(inode, 0, &page, SGP_READ);
|
|
if (error)
|
|
return ERR_PTR(error);
|
|
unlock_page(page);
|
|
}
|
|
set_delayed_call(done, shmem_put_link, page);
|
|
return page_address(page);
|
|
}
|
|
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
/*
|
|
* Superblocks without xattr inode operations may get some security.* xattr
|
|
* support from the LSM "for free". As soon as we have any other xattrs
|
|
* like ACLs, we also need to implement the security.* handlers at
|
|
* filesystem level, though.
|
|
*/
|
|
|
|
/*
|
|
* Callback for security_inode_init_security() for acquiring xattrs.
|
|
*/
|
|
static int shmem_initxattrs(struct inode *inode,
|
|
const struct xattr *xattr_array,
|
|
void *fs_info)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
const struct xattr *xattr;
|
|
struct simple_xattr *new_xattr;
|
|
size_t len;
|
|
|
|
for (xattr = xattr_array; xattr->name != NULL; xattr++) {
|
|
new_xattr = simple_xattr_alloc(xattr->value, xattr->value_len);
|
|
if (!new_xattr)
|
|
return -ENOMEM;
|
|
|
|
len = strlen(xattr->name) + 1;
|
|
new_xattr->name = kmalloc(XATTR_SECURITY_PREFIX_LEN + len,
|
|
GFP_KERNEL);
|
|
if (!new_xattr->name) {
|
|
kfree(new_xattr);
|
|
return -ENOMEM;
|
|
}
|
|
|
|
memcpy(new_xattr->name, XATTR_SECURITY_PREFIX,
|
|
XATTR_SECURITY_PREFIX_LEN);
|
|
memcpy(new_xattr->name + XATTR_SECURITY_PREFIX_LEN,
|
|
xattr->name, len);
|
|
|
|
simple_xattr_list_add(&info->xattrs, new_xattr);
|
|
}
|
|
|
|
return 0;
|
|
}
|
|
|
|
static int shmem_xattr_handler_get(const struct xattr_handler *handler,
|
|
struct dentry *unused, struct inode *inode,
|
|
const char *name, void *buffer, size_t size)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
|
|
name = xattr_full_name(handler, name);
|
|
return simple_xattr_get(&info->xattrs, name, buffer, size);
|
|
}
|
|
|
|
static int shmem_xattr_handler_set(const struct xattr_handler *handler,
|
|
struct dentry *unused, struct inode *inode,
|
|
const char *name, const void *value,
|
|
size_t size, int flags)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(inode);
|
|
|
|
name = xattr_full_name(handler, name);
|
|
return simple_xattr_set(&info->xattrs, name, value, size, flags);
|
|
}
|
|
|
|
static const struct xattr_handler shmem_security_xattr_handler = {
|
|
.prefix = XATTR_SECURITY_PREFIX,
|
|
.get = shmem_xattr_handler_get,
|
|
.set = shmem_xattr_handler_set,
|
|
};
|
|
|
|
static const struct xattr_handler shmem_trusted_xattr_handler = {
|
|
.prefix = XATTR_TRUSTED_PREFIX,
|
|
.get = shmem_xattr_handler_get,
|
|
.set = shmem_xattr_handler_set,
|
|
};
|
|
|
|
static const struct xattr_handler *shmem_xattr_handlers[] = {
|
|
#ifdef CONFIG_TMPFS_POSIX_ACL
|
|
&posix_acl_access_xattr_handler,
|
|
&posix_acl_default_xattr_handler,
|
|
#endif
|
|
&shmem_security_xattr_handler,
|
|
&shmem_trusted_xattr_handler,
|
|
NULL
|
|
};
|
|
|
|
static ssize_t shmem_listxattr(struct dentry *dentry, char *buffer, size_t size)
|
|
{
|
|
struct shmem_inode_info *info = SHMEM_I(d_inode(dentry));
|
|
return simple_xattr_list(d_inode(dentry), &info->xattrs, buffer, size);
|
|
}
|
|
#endif /* CONFIG_TMPFS_XATTR */
|
|
|
|
static const struct inode_operations shmem_short_symlink_operations = {
|
|
.get_link = simple_get_link,
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
.listxattr = shmem_listxattr,
|
|
#endif
|
|
};
|
|
|
|
static const struct inode_operations shmem_symlink_inode_operations = {
|
|
.get_link = shmem_get_link,
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
.listxattr = shmem_listxattr,
|
|
#endif
|
|
};
|
|
|
|
static struct dentry *shmem_get_parent(struct dentry *child)
|
|
{
|
|
return ERR_PTR(-ESTALE);
|
|
}
|
|
|
|
static int shmem_match(struct inode *ino, void *vfh)
|
|
{
|
|
__u32 *fh = vfh;
|
|
__u64 inum = fh[2];
|
|
inum = (inum << 32) | fh[1];
|
|
return ino->i_ino == inum && fh[0] == ino->i_generation;
|
|
}
|
|
|
|
/* Find any alias of inode, but prefer a hashed alias */
|
|
static struct dentry *shmem_find_alias(struct inode *inode)
|
|
{
|
|
struct dentry *alias = d_find_alias(inode);
|
|
|
|
return alias ?: d_find_any_alias(inode);
|
|
}
|
|
|
|
|
|
static struct dentry *shmem_fh_to_dentry(struct super_block *sb,
|
|
struct fid *fid, int fh_len, int fh_type)
|
|
{
|
|
struct inode *inode;
|
|
struct dentry *dentry = NULL;
|
|
u64 inum;
|
|
|
|
if (fh_len < 3)
|
|
return NULL;
|
|
|
|
inum = fid->raw[2];
|
|
inum = (inum << 32) | fid->raw[1];
|
|
|
|
inode = ilookup5(sb, (unsigned long)(inum + fid->raw[0]),
|
|
shmem_match, fid->raw);
|
|
if (inode) {
|
|
dentry = shmem_find_alias(inode);
|
|
iput(inode);
|
|
}
|
|
|
|
return dentry;
|
|
}
|
|
|
|
static int shmem_encode_fh(struct inode *inode, __u32 *fh, int *len,
|
|
struct inode *parent)
|
|
{
|
|
if (*len < 3) {
|
|
*len = 3;
|
|
return FILEID_INVALID;
|
|
}
|
|
|
|
if (inode_unhashed(inode)) {
|
|
/* Unfortunately insert_inode_hash is not idempotent,
|
|
* so as we hash inodes here rather than at creation
|
|
* time, we need a lock to ensure we only try
|
|
* to do it once
|
|
*/
|
|
static DEFINE_SPINLOCK(lock);
|
|
spin_lock(&lock);
|
|
if (inode_unhashed(inode))
|
|
__insert_inode_hash(inode,
|
|
inode->i_ino + inode->i_generation);
|
|
spin_unlock(&lock);
|
|
}
|
|
|
|
fh[0] = inode->i_generation;
|
|
fh[1] = inode->i_ino;
|
|
fh[2] = ((__u64)inode->i_ino) >> 32;
|
|
|
|
*len = 3;
|
|
return 1;
|
|
}
|
|
|
|
static const struct export_operations shmem_export_ops = {
|
|
.get_parent = shmem_get_parent,
|
|
.encode_fh = shmem_encode_fh,
|
|
.fh_to_dentry = shmem_fh_to_dentry,
|
|
};
|
|
|
|
static int shmem_parse_options(char *options, struct shmem_sb_info *sbinfo,
|
|
bool remount)
|
|
{
|
|
char *this_char, *value, *rest;
|
|
struct mempolicy *mpol = NULL;
|
|
uid_t uid;
|
|
gid_t gid;
|
|
|
|
while (options != NULL) {
|
|
this_char = options;
|
|
for (;;) {
|
|
/*
|
|
* NUL-terminate this option: unfortunately,
|
|
* mount options form a comma-separated list,
|
|
* but mpol's nodelist may also contain commas.
|
|
*/
|
|
options = strchr(options, ',');
|
|
if (options == NULL)
|
|
break;
|
|
options++;
|
|
if (!isdigit(*options)) {
|
|
options[-1] = '\0';
|
|
break;
|
|
}
|
|
}
|
|
if (!*this_char)
|
|
continue;
|
|
if ((value = strchr(this_char,'=')) != NULL) {
|
|
*value++ = 0;
|
|
} else {
|
|
pr_err("tmpfs: No value for mount option '%s'\n",
|
|
this_char);
|
|
goto error;
|
|
}
|
|
|
|
if (!strcmp(this_char,"size")) {
|
|
unsigned long long size;
|
|
size = memparse(value,&rest);
|
|
if (*rest == '%') {
|
|
size <<= PAGE_SHIFT;
|
|
size *= totalram_pages;
|
|
do_div(size, 100);
|
|
rest++;
|
|
}
|
|
if (*rest)
|
|
goto bad_val;
|
|
sbinfo->max_blocks =
|
|
DIV_ROUND_UP(size, PAGE_SIZE);
|
|
} else if (!strcmp(this_char,"nr_blocks")) {
|
|
sbinfo->max_blocks = memparse(value, &rest);
|
|
if (*rest)
|
|
goto bad_val;
|
|
} else if (!strcmp(this_char,"nr_inodes")) {
|
|
sbinfo->max_inodes = memparse(value, &rest);
|
|
if (*rest)
|
|
goto bad_val;
|
|
} else if (!strcmp(this_char,"mode")) {
|
|
if (remount)
|
|
continue;
|
|
sbinfo->mode = simple_strtoul(value, &rest, 8) & 07777;
|
|
if (*rest)
|
|
goto bad_val;
|
|
} else if (!strcmp(this_char,"uid")) {
|
|
if (remount)
|
|
continue;
|
|
uid = simple_strtoul(value, &rest, 0);
|
|
if (*rest)
|
|
goto bad_val;
|
|
sbinfo->uid = make_kuid(current_user_ns(), uid);
|
|
if (!uid_valid(sbinfo->uid))
|
|
goto bad_val;
|
|
} else if (!strcmp(this_char,"gid")) {
|
|
if (remount)
|
|
continue;
|
|
gid = simple_strtoul(value, &rest, 0);
|
|
if (*rest)
|
|
goto bad_val;
|
|
sbinfo->gid = make_kgid(current_user_ns(), gid);
|
|
if (!gid_valid(sbinfo->gid))
|
|
goto bad_val;
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
} else if (!strcmp(this_char, "huge")) {
|
|
int huge;
|
|
huge = shmem_parse_huge(value);
|
|
if (huge < 0)
|
|
goto bad_val;
|
|
if (!has_transparent_hugepage() &&
|
|
huge != SHMEM_HUGE_NEVER)
|
|
goto bad_val;
|
|
sbinfo->huge = huge;
|
|
#endif
|
|
#ifdef CONFIG_NUMA
|
|
} else if (!strcmp(this_char,"mpol")) {
|
|
mpol_put(mpol);
|
|
mpol = NULL;
|
|
if (mpol_parse_str(value, &mpol))
|
|
goto bad_val;
|
|
#endif
|
|
} else {
|
|
pr_err("tmpfs: Bad mount option %s\n", this_char);
|
|
goto error;
|
|
}
|
|
}
|
|
sbinfo->mpol = mpol;
|
|
return 0;
|
|
|
|
bad_val:
|
|
pr_err("tmpfs: Bad value '%s' for mount option '%s'\n",
|
|
value, this_char);
|
|
error:
|
|
mpol_put(mpol);
|
|
return 1;
|
|
|
|
}
|
|
|
|
static int shmem_remount_fs(struct super_block *sb, int *flags, char *data)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
struct shmem_sb_info config = *sbinfo;
|
|
unsigned long inodes;
|
|
int error = -EINVAL;
|
|
|
|
config.mpol = NULL;
|
|
if (shmem_parse_options(data, &config, true))
|
|
return error;
|
|
|
|
spin_lock(&sbinfo->stat_lock);
|
|
inodes = sbinfo->max_inodes - sbinfo->free_inodes;
|
|
if (percpu_counter_compare(&sbinfo->used_blocks, config.max_blocks) > 0)
|
|
goto out;
|
|
if (config.max_inodes < inodes)
|
|
goto out;
|
|
/*
|
|
* Those tests disallow limited->unlimited while any are in use;
|
|
* but we must separately disallow unlimited->limited, because
|
|
* in that case we have no record of how much is already in use.
|
|
*/
|
|
if (config.max_blocks && !sbinfo->max_blocks)
|
|
goto out;
|
|
if (config.max_inodes && !sbinfo->max_inodes)
|
|
goto out;
|
|
|
|
error = 0;
|
|
sbinfo->huge = config.huge;
|
|
sbinfo->max_blocks = config.max_blocks;
|
|
sbinfo->max_inodes = config.max_inodes;
|
|
sbinfo->free_inodes = config.max_inodes - inodes;
|
|
|
|
/*
|
|
* Preserve previous mempolicy unless mpol remount option was specified.
|
|
*/
|
|
if (config.mpol) {
|
|
mpol_put(sbinfo->mpol);
|
|
sbinfo->mpol = config.mpol; /* transfers initial ref */
|
|
}
|
|
out:
|
|
spin_unlock(&sbinfo->stat_lock);
|
|
return error;
|
|
}
|
|
|
|
static int shmem_show_options(struct seq_file *seq, struct dentry *root)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(root->d_sb);
|
|
|
|
if (sbinfo->max_blocks != shmem_default_max_blocks())
|
|
seq_printf(seq, ",size=%luk",
|
|
sbinfo->max_blocks << (PAGE_SHIFT - 10));
|
|
if (sbinfo->max_inodes != shmem_default_max_inodes())
|
|
seq_printf(seq, ",nr_inodes=%lu", sbinfo->max_inodes);
|
|
if (sbinfo->mode != (0777 | S_ISVTX))
|
|
seq_printf(seq, ",mode=%03ho", sbinfo->mode);
|
|
if (!uid_eq(sbinfo->uid, GLOBAL_ROOT_UID))
|
|
seq_printf(seq, ",uid=%u",
|
|
from_kuid_munged(&init_user_ns, sbinfo->uid));
|
|
if (!gid_eq(sbinfo->gid, GLOBAL_ROOT_GID))
|
|
seq_printf(seq, ",gid=%u",
|
|
from_kgid_munged(&init_user_ns, sbinfo->gid));
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
/* Rightly or wrongly, show huge mount option unmasked by shmem_huge */
|
|
if (sbinfo->huge)
|
|
seq_printf(seq, ",huge=%s", shmem_format_huge(sbinfo->huge));
|
|
#endif
|
|
shmem_show_mpol(seq, sbinfo->mpol);
|
|
return 0;
|
|
}
|
|
|
|
#endif /* CONFIG_TMPFS */
|
|
|
|
static void shmem_put_super(struct super_block *sb)
|
|
{
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(sb);
|
|
|
|
percpu_counter_destroy(&sbinfo->used_blocks);
|
|
mpol_put(sbinfo->mpol);
|
|
kfree(sbinfo);
|
|
sb->s_fs_info = NULL;
|
|
}
|
|
|
|
int shmem_fill_super(struct super_block *sb, void *data, int silent)
|
|
{
|
|
struct inode *inode;
|
|
struct shmem_sb_info *sbinfo;
|
|
int err = -ENOMEM;
|
|
|
|
/* Round up to L1_CACHE_BYTES to resist false sharing */
|
|
sbinfo = kzalloc(max((int)sizeof(struct shmem_sb_info),
|
|
L1_CACHE_BYTES), GFP_KERNEL);
|
|
if (!sbinfo)
|
|
return -ENOMEM;
|
|
|
|
sbinfo->mode = 0777 | S_ISVTX;
|
|
sbinfo->uid = current_fsuid();
|
|
sbinfo->gid = current_fsgid();
|
|
sb->s_fs_info = sbinfo;
|
|
|
|
#ifdef CONFIG_TMPFS
|
|
/*
|
|
* Per default we only allow half of the physical ram per
|
|
* tmpfs instance, limiting inodes to one per page of lowmem;
|
|
* but the internal instance is left unlimited.
|
|
*/
|
|
if (!(sb->s_flags & SB_KERNMOUNT)) {
|
|
sbinfo->max_blocks = shmem_default_max_blocks();
|
|
sbinfo->max_inodes = shmem_default_max_inodes();
|
|
if (shmem_parse_options(data, sbinfo, false)) {
|
|
err = -EINVAL;
|
|
goto failed;
|
|
}
|
|
} else {
|
|
sb->s_flags |= SB_NOUSER;
|
|
}
|
|
sb->s_export_op = &shmem_export_ops;
|
|
sb->s_flags |= SB_NOSEC;
|
|
#else
|
|
sb->s_flags |= SB_NOUSER;
|
|
#endif
|
|
|
|
spin_lock_init(&sbinfo->stat_lock);
|
|
if (percpu_counter_init(&sbinfo->used_blocks, 0, GFP_KERNEL))
|
|
goto failed;
|
|
sbinfo->free_inodes = sbinfo->max_inodes;
|
|
spin_lock_init(&sbinfo->shrinklist_lock);
|
|
INIT_LIST_HEAD(&sbinfo->shrinklist);
|
|
|
|
sb->s_maxbytes = MAX_LFS_FILESIZE;
|
|
sb->s_blocksize = PAGE_SIZE;
|
|
sb->s_blocksize_bits = PAGE_SHIFT;
|
|
sb->s_magic = TMPFS_MAGIC;
|
|
sb->s_op = &shmem_ops;
|
|
sb->s_time_gran = 1;
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
sb->s_xattr = shmem_xattr_handlers;
|
|
#endif
|
|
#ifdef CONFIG_TMPFS_POSIX_ACL
|
|
sb->s_flags |= SB_POSIXACL;
|
|
#endif
|
|
uuid_gen(&sb->s_uuid);
|
|
|
|
inode = shmem_get_inode(sb, NULL, S_IFDIR | sbinfo->mode, 0, VM_NORESERVE);
|
|
if (!inode)
|
|
goto failed;
|
|
inode->i_uid = sbinfo->uid;
|
|
inode->i_gid = sbinfo->gid;
|
|
sb->s_root = d_make_root(inode);
|
|
if (!sb->s_root)
|
|
goto failed;
|
|
return 0;
|
|
|
|
failed:
|
|
shmem_put_super(sb);
|
|
return err;
|
|
}
|
|
|
|
static struct kmem_cache *shmem_inode_cachep;
|
|
|
|
static struct inode *shmem_alloc_inode(struct super_block *sb)
|
|
{
|
|
struct shmem_inode_info *info;
|
|
info = kmem_cache_alloc(shmem_inode_cachep, GFP_KERNEL);
|
|
if (!info)
|
|
return NULL;
|
|
return &info->vfs_inode;
|
|
}
|
|
|
|
static void shmem_destroy_callback(struct rcu_head *head)
|
|
{
|
|
struct inode *inode = container_of(head, struct inode, i_rcu);
|
|
if (S_ISLNK(inode->i_mode))
|
|
kfree(inode->i_link);
|
|
kmem_cache_free(shmem_inode_cachep, SHMEM_I(inode));
|
|
}
|
|
|
|
static void shmem_destroy_inode(struct inode *inode)
|
|
{
|
|
if (S_ISREG(inode->i_mode))
|
|
mpol_free_shared_policy(&SHMEM_I(inode)->policy);
|
|
call_rcu(&inode->i_rcu, shmem_destroy_callback);
|
|
}
|
|
|
|
static void shmem_init_inode(void *foo)
|
|
{
|
|
struct shmem_inode_info *info = foo;
|
|
inode_init_once(&info->vfs_inode);
|
|
}
|
|
|
|
static void shmem_init_inodecache(void)
|
|
{
|
|
shmem_inode_cachep = kmem_cache_create("shmem_inode_cache",
|
|
sizeof(struct shmem_inode_info),
|
|
0, SLAB_PANIC|SLAB_ACCOUNT, shmem_init_inode);
|
|
}
|
|
|
|
static void shmem_destroy_inodecache(void)
|
|
{
|
|
kmem_cache_destroy(shmem_inode_cachep);
|
|
}
|
|
|
|
static const struct address_space_operations shmem_aops = {
|
|
.writepage = shmem_writepage,
|
|
.set_page_dirty = __set_page_dirty_no_writeback,
|
|
#ifdef CONFIG_TMPFS
|
|
.write_begin = shmem_write_begin,
|
|
.write_end = shmem_write_end,
|
|
#endif
|
|
#ifdef CONFIG_MIGRATION
|
|
.migratepage = migrate_page,
|
|
#endif
|
|
.error_remove_page = generic_error_remove_page,
|
|
};
|
|
|
|
static const struct file_operations shmem_file_operations = {
|
|
.mmap = shmem_mmap,
|
|
.get_unmapped_area = shmem_get_unmapped_area,
|
|
#ifdef CONFIG_TMPFS
|
|
.llseek = shmem_file_llseek,
|
|
.read_iter = shmem_file_read_iter,
|
|
.write_iter = generic_file_write_iter,
|
|
.fsync = noop_fsync,
|
|
.splice_read = generic_file_splice_read,
|
|
.splice_write = iter_file_splice_write,
|
|
.fallocate = shmem_fallocate,
|
|
#endif
|
|
};
|
|
|
|
static const struct inode_operations shmem_inode_operations = {
|
|
.getattr = shmem_getattr,
|
|
.setattr = shmem_setattr,
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
.listxattr = shmem_listxattr,
|
|
.set_acl = simple_set_acl,
|
|
#endif
|
|
};
|
|
|
|
static const struct inode_operations shmem_dir_inode_operations = {
|
|
#ifdef CONFIG_TMPFS
|
|
.create = shmem_create,
|
|
.lookup = simple_lookup,
|
|
.link = shmem_link,
|
|
.unlink = shmem_unlink,
|
|
.symlink = shmem_symlink,
|
|
.mkdir = shmem_mkdir,
|
|
.rmdir = shmem_rmdir,
|
|
.mknod = shmem_mknod,
|
|
.rename = shmem_rename2,
|
|
.tmpfile = shmem_tmpfile,
|
|
#endif
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
.listxattr = shmem_listxattr,
|
|
#endif
|
|
#ifdef CONFIG_TMPFS_POSIX_ACL
|
|
.setattr = shmem_setattr,
|
|
.set_acl = simple_set_acl,
|
|
#endif
|
|
};
|
|
|
|
static const struct inode_operations shmem_special_inode_operations = {
|
|
#ifdef CONFIG_TMPFS_XATTR
|
|
.listxattr = shmem_listxattr,
|
|
#endif
|
|
#ifdef CONFIG_TMPFS_POSIX_ACL
|
|
.setattr = shmem_setattr,
|
|
.set_acl = simple_set_acl,
|
|
#endif
|
|
};
|
|
|
|
static const struct super_operations shmem_ops = {
|
|
.alloc_inode = shmem_alloc_inode,
|
|
.destroy_inode = shmem_destroy_inode,
|
|
#ifdef CONFIG_TMPFS
|
|
.statfs = shmem_statfs,
|
|
.remount_fs = shmem_remount_fs,
|
|
.show_options = shmem_show_options,
|
|
#endif
|
|
.evict_inode = shmem_evict_inode,
|
|
.drop_inode = generic_delete_inode,
|
|
.put_super = shmem_put_super,
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
.nr_cached_objects = shmem_unused_huge_count,
|
|
.free_cached_objects = shmem_unused_huge_scan,
|
|
#endif
|
|
};
|
|
|
|
static const struct vm_operations_struct shmem_vm_ops = {
|
|
.fault = shmem_fault,
|
|
.map_pages = filemap_map_pages,
|
|
#ifdef CONFIG_NUMA
|
|
.set_policy = shmem_set_policy,
|
|
.get_policy = shmem_get_policy,
|
|
#endif
|
|
};
|
|
|
|
static struct dentry *shmem_mount(struct file_system_type *fs_type,
|
|
int flags, const char *dev_name, void *data)
|
|
{
|
|
return mount_nodev(fs_type, flags, data, shmem_fill_super);
|
|
}
|
|
|
|
static struct file_system_type shmem_fs_type = {
|
|
.owner = THIS_MODULE,
|
|
.name = "tmpfs",
|
|
.mount = shmem_mount,
|
|
.kill_sb = kill_litter_super,
|
|
.fs_flags = FS_USERNS_MOUNT,
|
|
};
|
|
|
|
int __init shmem_init(void)
|
|
{
|
|
int error;
|
|
|
|
/* If rootfs called this, don't re-init */
|
|
if (shmem_inode_cachep)
|
|
return 0;
|
|
|
|
shmem_init_inodecache();
|
|
|
|
error = register_filesystem(&shmem_fs_type);
|
|
if (error) {
|
|
pr_err("Could not register tmpfs\n");
|
|
goto out2;
|
|
}
|
|
|
|
shm_mnt = kern_mount(&shmem_fs_type);
|
|
if (IS_ERR(shm_mnt)) {
|
|
error = PTR_ERR(shm_mnt);
|
|
pr_err("Could not kern_mount tmpfs\n");
|
|
goto out1;
|
|
}
|
|
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
if (has_transparent_hugepage() && shmem_huge > SHMEM_HUGE_DENY)
|
|
SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge;
|
|
else
|
|
shmem_huge = 0; /* just in case it was patched */
|
|
#endif
|
|
return 0;
|
|
|
|
out1:
|
|
unregister_filesystem(&shmem_fs_type);
|
|
out2:
|
|
shmem_destroy_inodecache();
|
|
shm_mnt = ERR_PTR(error);
|
|
return error;
|
|
}
|
|
|
|
#if defined(CONFIG_TRANSPARENT_HUGE_PAGECACHE) && defined(CONFIG_SYSFS)
|
|
static ssize_t shmem_enabled_show(struct kobject *kobj,
|
|
struct kobj_attribute *attr, char *buf)
|
|
{
|
|
int values[] = {
|
|
SHMEM_HUGE_ALWAYS,
|
|
SHMEM_HUGE_WITHIN_SIZE,
|
|
SHMEM_HUGE_ADVISE,
|
|
SHMEM_HUGE_NEVER,
|
|
SHMEM_HUGE_DENY,
|
|
SHMEM_HUGE_FORCE,
|
|
};
|
|
int i, count;
|
|
|
|
for (i = 0, count = 0; i < ARRAY_SIZE(values); i++) {
|
|
const char *fmt = shmem_huge == values[i] ? "[%s] " : "%s ";
|
|
|
|
count += sprintf(buf + count, fmt,
|
|
shmem_format_huge(values[i]));
|
|
}
|
|
buf[count - 1] = '\n';
|
|
return count;
|
|
}
|
|
|
|
static ssize_t shmem_enabled_store(struct kobject *kobj,
|
|
struct kobj_attribute *attr, const char *buf, size_t count)
|
|
{
|
|
char tmp[16];
|
|
int huge;
|
|
|
|
if (count + 1 > sizeof(tmp))
|
|
return -EINVAL;
|
|
memcpy(tmp, buf, count);
|
|
tmp[count] = '\0';
|
|
if (count && tmp[count - 1] == '\n')
|
|
tmp[count - 1] = '\0';
|
|
|
|
huge = shmem_parse_huge(tmp);
|
|
if (huge == -EINVAL)
|
|
return -EINVAL;
|
|
if (!has_transparent_hugepage() &&
|
|
huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)
|
|
return -EINVAL;
|
|
|
|
shmem_huge = huge;
|
|
if (shmem_huge > SHMEM_HUGE_DENY)
|
|
SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge;
|
|
return count;
|
|
}
|
|
|
|
struct kobj_attribute shmem_enabled_attr =
|
|
__ATTR(shmem_enabled, 0644, shmem_enabled_show, shmem_enabled_store);
|
|
#endif /* CONFIG_TRANSPARENT_HUGE_PAGECACHE && CONFIG_SYSFS */
|
|
|
|
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
|
|
bool shmem_huge_enabled(struct vm_area_struct *vma)
|
|
{
|
|
struct inode *inode = file_inode(vma->vm_file);
|
|
struct shmem_sb_info *sbinfo = SHMEM_SB(inode->i_sb);
|
|
loff_t i_size;
|
|
pgoff_t off;
|
|
|
|
if (shmem_huge == SHMEM_HUGE_FORCE)
|
|
return true;
|
|
if (shmem_huge == SHMEM_HUGE_DENY)
|
|
return false;
|
|
switch (sbinfo->huge) {
|
|
case SHMEM_HUGE_NEVER:
|
|
return false;
|
|
case SHMEM_HUGE_ALWAYS:
|
|
return true;
|
|
case SHMEM_HUGE_WITHIN_SIZE:
|
|
off = round_up(vma->vm_pgoff, HPAGE_PMD_NR);
|
|
i_size = round_up(i_size_read(inode), PAGE_SIZE);
|
|
if (i_size >= HPAGE_PMD_SIZE &&
|
|
i_size >> PAGE_SHIFT >= off)
|
|
return true;
|
|
/* fall through */
|
|
case SHMEM_HUGE_ADVISE:
|
|
/* TODO: implement fadvise() hints */
|
|
return (vma->vm_flags & VM_HUGEPAGE);
|
|
default:
|
|
VM_BUG_ON(1);
|
|
return false;
|
|
}
|
|
}
|
|
#endif /* CONFIG_TRANSPARENT_HUGE_PAGECACHE */
|
|
|
|
#else /* !CONFIG_SHMEM */
|
|
|
|
/*
|
|
* tiny-shmem: simple shmemfs and tmpfs using ramfs code
|
|
*
|
|
* This is intended for small system where the benefits of the full
|
|
* shmem code (swap-backed and resource-limited) are outweighed by
|
|
* their complexity. On systems without swap this code should be
|
|
* effectively equivalent, but much lighter weight.
|
|
*/
|
|
|
|
static struct file_system_type shmem_fs_type = {
|
|
.name = "tmpfs",
|
|
.mount = ramfs_mount,
|
|
.kill_sb = kill_litter_super,
|
|
.fs_flags = FS_USERNS_MOUNT,
|
|
};
|
|
|
|
int __init shmem_init(void)
|
|
{
|
|
BUG_ON(register_filesystem(&shmem_fs_type) != 0);
|
|
|
|
shm_mnt = kern_mount(&shmem_fs_type);
|
|
BUG_ON(IS_ERR(shm_mnt));
|
|
|
|
return 0;
|
|
}
|
|
|
|
int shmem_unuse(swp_entry_t swap, struct page *page)
|
|
{
|
|
return 0;
|
|
}
|
|
|
|
int shmem_lock(struct file *file, int lock, struct user_struct *user)
|
|
{
|
|
return 0;
|
|
}
|
|
|
|
void shmem_unlock_mapping(struct address_space *mapping)
|
|
{
|
|
}
|
|
|
|
#ifdef CONFIG_MMU
|
|
unsigned long shmem_get_unmapped_area(struct file *file,
|
|
unsigned long addr, unsigned long len,
|
|
unsigned long pgoff, unsigned long flags)
|
|
{
|
|
return current->mm->get_unmapped_area(file, addr, len, pgoff, flags);
|
|
}
|
|
#endif
|
|
|
|
void shmem_truncate_range(struct inode *inode, loff_t lstart, loff_t lend)
|
|
{
|
|
truncate_inode_pages_range(inode->i_mapping, lstart, lend);
|
|
}
|
|
EXPORT_SYMBOL_GPL(shmem_truncate_range);
|
|
|
|
#define shmem_vm_ops generic_file_vm_ops
|
|
#define shmem_file_operations ramfs_file_operations
|
|
#define shmem_get_inode(sb, dir, mode, dev, flags) ramfs_get_inode(sb, dir, mode, dev)
|
|
#define shmem_acct_size(flags, size) 0
|
|
#define shmem_unacct_size(flags, size) do {} while (0)
|
|
|
|
#endif /* CONFIG_SHMEM */
|
|
|
|
/* common code */
|
|
|
|
static struct file *__shmem_file_setup(struct vfsmount *mnt, const char *name, loff_t size,
|
|
unsigned long flags, unsigned int i_flags)
|
|
{
|
|
struct inode *inode;
|
|
struct file *res;
|
|
|
|
if (IS_ERR(mnt))
|
|
return ERR_CAST(mnt);
|
|
|
|
if (size < 0 || size > MAX_LFS_FILESIZE)
|
|
return ERR_PTR(-EINVAL);
|
|
|
|
if (shmem_acct_size(flags, size))
|
|
return ERR_PTR(-ENOMEM);
|
|
|
|
inode = shmem_get_inode(mnt->mnt_sb, NULL, S_IFREG | S_IRWXUGO, 0,
|
|
flags);
|
|
if (unlikely(!inode)) {
|
|
shmem_unacct_size(flags, size);
|
|
return ERR_PTR(-ENOSPC);
|
|
}
|
|
inode->i_flags |= i_flags;
|
|
inode->i_size = size;
|
|
clear_nlink(inode); /* It is unlinked */
|
|
res = ERR_PTR(ramfs_nommu_expand_for_mapping(inode, size));
|
|
if (!IS_ERR(res))
|
|
res = alloc_file_pseudo(inode, mnt, name, O_RDWR,
|
|
&shmem_file_operations);
|
|
if (IS_ERR(res))
|
|
iput(inode);
|
|
return res;
|
|
}
|
|
|
|
/**
|
|
* shmem_kernel_file_setup - get an unlinked file living in tmpfs which must be
|
|
* kernel internal. There will be NO LSM permission checks against the
|
|
* underlying inode. So users of this interface must do LSM checks at a
|
|
* higher layer. The users are the big_key and shm implementations. LSM
|
|
* checks are provided at the key or shm level rather than the inode.
|
|
* @name: name for dentry (to be seen in /proc/<pid>/maps
|
|
* @size: size to be set for the file
|
|
* @flags: VM_NORESERVE suppresses pre-accounting of the entire object size
|
|
*/
|
|
struct file *shmem_kernel_file_setup(const char *name, loff_t size, unsigned long flags)
|
|
{
|
|
return __shmem_file_setup(shm_mnt, name, size, flags, S_PRIVATE);
|
|
}
|
|
|
|
/**
|
|
* shmem_file_setup - get an unlinked file living in tmpfs
|
|
* @name: name for dentry (to be seen in /proc/<pid>/maps
|
|
* @size: size to be set for the file
|
|
* @flags: VM_NORESERVE suppresses pre-accounting of the entire object size
|
|
*/
|
|
struct file *shmem_file_setup(const char *name, loff_t size, unsigned long flags)
|
|
{
|
|
return __shmem_file_setup(shm_mnt, name, size, flags, 0);
|
|
}
|
|
EXPORT_SYMBOL_GPL(shmem_file_setup);
|
|
|
|
/**
|
|
* shmem_file_setup_with_mnt - get an unlinked file living in tmpfs
|
|
* @mnt: the tmpfs mount where the file will be created
|
|
* @name: name for dentry (to be seen in /proc/<pid>/maps
|
|
* @size: size to be set for the file
|
|
* @flags: VM_NORESERVE suppresses pre-accounting of the entire object size
|
|
*/
|
|
struct file *shmem_file_setup_with_mnt(struct vfsmount *mnt, const char *name,
|
|
loff_t size, unsigned long flags)
|
|
{
|
|
return __shmem_file_setup(mnt, name, size, flags, 0);
|
|
}
|
|
EXPORT_SYMBOL_GPL(shmem_file_setup_with_mnt);
|
|
|
|
/**
|
|
* shmem_zero_setup - setup a shared anonymous mapping
|
|
* @vma: the vma to be mmapped is prepared by do_mmap_pgoff
|
|
*/
|
|
int shmem_zero_setup(struct vm_area_struct *vma)
|
|
{
|
|
struct file *file;
|
|
loff_t size = vma->vm_end - vma->vm_start;
|
|
|
|
/*
|
|
* Cloning a new file under mmap_sem leads to a lock ordering conflict
|
|
* between XFS directory reading and selinux: since this file is only
|
|
* accessible to the user through its mapping, use S_PRIVATE flag to
|
|
* bypass file security, in the same way as shmem_kernel_file_setup().
|
|
*/
|
|
file = shmem_kernel_file_setup("dev/zero", size, vma->vm_flags);
|
|
if (IS_ERR(file))
|
|
return PTR_ERR(file);
|
|
|
|
if (vma->vm_file)
|
|
fput(vma->vm_file);
|
|
vma->vm_file = file;
|
|
vma->vm_ops = &shmem_vm_ops;
|
|
|
|
if (IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE) &&
|
|
((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) <
|
|
(vma->vm_end & HPAGE_PMD_MASK)) {
|
|
khugepaged_enter(vma, vma->vm_flags);
|
|
}
|
|
|
|
return 0;
|
|
}
|
|
|
|
/**
|
|
* shmem_read_mapping_page_gfp - read into page cache, using specified page allocation flags.
|
|
* @mapping: the page's address_space
|
|
* @index: the page index
|
|
* @gfp: the page allocator flags to use if allocating
|
|
*
|
|
* This behaves as a tmpfs "read_cache_page_gfp(mapping, index, gfp)",
|
|
* with any new page allocations done using the specified allocation flags.
|
|
* But read_cache_page_gfp() uses the ->readpage() method: which does not
|
|
* suit tmpfs, since it may have pages in swapcache, and needs to find those
|
|
* for itself; although drivers/gpu/drm i915 and ttm rely upon this support.
|
|
*
|
|
* i915_gem_object_get_pages_gtt() mixes __GFP_NORETRY | __GFP_NOWARN in
|
|
* with the mapping_gfp_mask(), to avoid OOMing the machine unnecessarily.
|
|
*/
|
|
struct page *shmem_read_mapping_page_gfp(struct address_space *mapping,
|
|
pgoff_t index, gfp_t gfp)
|
|
{
|
|
#ifdef CONFIG_SHMEM
|
|
struct inode *inode = mapping->host;
|
|
struct page *page;
|
|
int error;
|
|
|
|
BUG_ON(mapping->a_ops != &shmem_aops);
|
|
error = shmem_getpage_gfp(inode, index, &page, SGP_CACHE,
|
|
gfp, NULL, NULL, NULL);
|
|
if (error)
|
|
page = ERR_PTR(error);
|
|
else
|
|
unlock_page(page);
|
|
return page;
|
|
#else
|
|
/*
|
|
* The tiny !SHMEM case uses ramfs without swap
|
|
*/
|
|
return read_cache_page_gfp(mapping, index, gfp);
|
|
#endif
|
|
}
|
|
EXPORT_SYMBOL_GPL(shmem_read_mapping_page_gfp);
|