asgardius/android_kernel_motorola_sm6225
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'android-4.19' into android-4.19-stable

Browse source 
Final merge of android-4.19

Change-Id: Ifa789a66260ed6207e829555ad2050d30e3de555
This commit is contained in:
Todd Kjos 2020-05-18 08:24:14 -07:00
commit 7502b09f02
1024 changed files with 74671 additions and 57934 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
Documentation/ABI/stable/sysfs-bus-nvmem
View file

@ -6,6 +6,8 @@ Description:
This file allows user to read/write the raw NVMEM contents.
Permissions for write to this file depends on the nvmem
provider configuration.
Note: This file is only present if CONFIG_NVMEM_SYSFS
is enabled
ex:
hexdump /sys/bus/nvmem/devices/qfprom0/nvmem

3
Documentation/ABI/testing/sysfs-class-power
View file

@ -144,7 +144,8 @@ Description:
Access: Read
Valid values: "Unknown", "Good", "Overheat", "Dead",
"Over voltage", "Unspecified failure", "Cold",
"Watchdog timer expire", "Safety timer expire"
"Watchdog timer expire", "Safety timer expire",
"Over current", "Warm", "Cool", "Hot"
What: /sys/class/power_supply/<supply_name>/precharge_current
Date: June 2017

15
Documentation/ABI/testing/sysfs-fs-f2fs
View file

@ -323,3 +323,18 @@ What: /sys/fs/f2fs/<disk>/mounted_time_sec
Date: February 2020
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Description: Show the mounted time in secs of this partition.
What: /sys/fs/f2fs/<disk>/data_io_flag
Date: April 2020
Contact: "Jaegeuk Kim" <jaegeuk@kernel.org>
Description: Give a way to attach REQ_META|FUA to data writes
given temperature-based bits. Now the bits indicate:
* REQ_META | REQ_FUA |
* 5 | 4 | 3 | 2 | 1 | 0 |
* Cold | Warm | Hot | Cold | Warm | Hot |
What: /sys/fs/f2fs/<disk>/iostat_period_ms
Date: April 2020
Contact: "Daeho Jeong" <daehojeong@google.com>
Description: Give a way to change iostat_period time. 3secs by default.
The new iostat trace gives stats gap given the period.

4
Documentation/admin-guide/kernel-parameters.txt
View file

@ -4032,7 +4032,9 @@
[[,]s[mp]#### \
[[,]b[ios] | a[cpi] | k[bd] | t[riple] | e[fi] | p[ci]] \
[[,]f[orce]
Where reboot_mode is one of warm (soft) or cold (hard) or gpio,
Where reboot_mode is one of warm (soft) or cold (hard) or gpio
(prefix with 'panic_' to set mode for panic
reboot only),
reboot_type is one of bios, acpi, kbd, triple, efi, or pci,
reboot_force is either force or not specified,
reboot_cpu is s[mp]#### with #### being the processor

27
Documentation/arm64/index.rst Normal file
View file

@ -0,0 +1,27 @@
==================
ARM64 Architecture
==================
.. toctree::
:maxdepth: 1
acpi_object_usage
arm-acpi
booting
cpu-feature-registers
elf_hwcaps
hugetlbpage
legacy_instructions
memory
pointer-authentication
silicon-errata
sve
tagged-address-abi
tagged-pointers
.. only:: subproject and html
Indices
=======
* :ref:`genindex`

1
Documentation/arm64/silicon-errata.txt
View file

@ -59,6 +59,7 @@ stable kernels.
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
| ARM | Cortex-A76 | #1463225 | ARM64_ERRATUM_1463225 |
| ARM | Neoverse-N1 | #1542419 | ARM64_ERRATUM_1542419 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |

75
Documentation/arm64/tagged-pointers.rst Normal file
View file

@ -0,0 +1,75 @@
=========================================
Tagged virtual addresses in AArch64 Linux
=========================================
Author: Will Deacon <will.deacon@arm.com>
Date : 12 June 2013
This document briefly describes the provision of tagged virtual
addresses in the AArch64 translation system and their potential uses
in AArch64 Linux.
The kernel configures the translation tables so that translations made
via TTBR0 (i.e. userspace mappings) have the top byte (bits 63:56) of
the virtual address ignored by the translation hardware. This frees up
this byte for application use.
Passing tagged addresses to the kernel
--------------------------------------
All interpretation of userspace memory addresses by the kernel assumes
an address tag of 0x00, unless the application enables the AArch64
Tagged Address ABI explicitly
(Documentation/arm64/tagged-address-abi.rst).
This includes, but is not limited to, addresses found in:
- pointer arguments to system calls, including pointers in structures
passed to system calls,
- the stack pointer (sp), e.g. when interpreting it to deliver a
signal,
- the frame pointer (x29) and frame records, e.g. when interpreting
them to generate a backtrace or call graph.
Using non-zero address tags in any of these locations when the
userspace application did not enable the AArch64 Tagged Address ABI may
result in an error code being returned, a (fatal) signal being raised,
or other modes of failure.
For these reasons, when the AArch64 Tagged Address ABI is disabled,
passing non-zero address tags to the kernel via system calls is
forbidden, and using a non-zero address tag for sp is strongly
discouraged.
Programs maintaining a frame pointer and frame records that use non-zero
address tags may suffer impaired or inaccurate debug and profiling
visibility.
Preserving tags
---------------
Non-zero tags are not preserved when delivering signals. This means that
signal handlers in applications making use of tags cannot rely on the
tag information for user virtual addresses being maintained for fields
inside siginfo_t. One exception to this rule is for signals raised in
response to watchpoint debug exceptions, where the tag information will
be preserved.
The architecture prevents the use of a tagged PC, so the upper byte will
be set to a sign-extension of bit 55 on exception return.
This behaviour is maintained when the AArch64 Tagged Address ABI is
enabled.
Other considerations
--------------------
Special care should be taken when using tagged pointers, since it is
likely that C compilers will not hazard two virtual addresses differing
only in the upper byte.

6
Documentation/sound/designs/compress-offload.rst
View file

@ -183,6 +183,11 @@ partial drain
EOF is reached and now DSP can start skipping padding delay. Also next write
data would belong to next track
- set_next_track_param
This routine is called to send to DSP codec specific data of subsequent track
in gapless before first write.
Sequence flow for gapless would be:
- Open
- Get caps / codec caps
@ -194,6 +199,7 @@ Sequence flow for gapless would be:
- Indicate next track data by sending set_next_track
- Set metadata of the next track
- then call partial_drain to flush most of buffer in DSP
- set codec specific data of subsequent track
- Fill data of the next track
- DSP switches to second track

1
Documentation/sound/hd-audio/index.rst
View file

@ -8,3 +8,4 @@ HD-Audio
models
controls
dp-mst
realtek-pc-beep

2
Documentation/sound/hd-audio/models.rst
View file

@ -216,8 +216,6 @@ alc298-dell-aio
ALC298 fixups on Dell AIO machines
alc275-dell-xps
ALC275 fixups on Dell XPS models
alc256-dell-xps13
ALC256 fixups on Dell XPS13
lenovo-spk-noise
Workaround for speaker noise on Lenovo machines
lenovo-hotkey

129
Documentation/sound/hd-audio/realtek-pc-beep.rst Normal file
View file

@ -0,0 +1,129 @@
===============================
Realtek PC Beep Hidden Register
===============================
This file documents the "PC Beep Hidden Register", which is present in certain
Realtek HDA codecs and controls a muxer and pair of passthrough mixers that can
route audio between pins but aren't themselves exposed as HDA widgets. As far
as I can tell, these hidden routes are designed to allow flexible PC Beep output
for codecs that don't have mixer widgets in their output paths. Why it's easier
to hide a mixer behind an undocumented vendor register than to just expose it
as a widget, I have no idea.
Register Description
====================
The register is accessed via processing coefficient 0x36 on NID 20h. Bits not
identified below have no discernible effect on my machine, a Dell XPS 13 9350::
MSB LSB
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |h|S|L| | B |R| | Known bits
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|0|0|1|1| 0x7 |0|0x0|1| 0x7 | Reset value
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1Ah input select (B): 2 bits
When zero, expose the PC Beep line (from the internal beep generator, when
enabled with the Set Beep Generation verb on NID 01h, or else from the
external PCBEEP pin) on the 1Ah pin node. When nonzero, expose the headphone
jack (or possibly Line In on some machines) input instead. If PC Beep is
selected, the 1Ah boost control has no effect.
Amplify 1Ah loopback, left (L): 1 bit
Amplify the left channel of 1Ah before mixing it into outputs as specified
by h and S bits. Does not affect the level of 1Ah exposed to other widgets.
Amplify 1Ah loopback, right (R): 1 bit
Amplify the right channel of 1Ah before mixing it into outputs as specified
by h and S bits. Does not affect the level of 1Ah exposed to other widgets.
Loopback 1Ah to 21h [active low] (h): 1 bit
When zero, mix 1Ah (possibly with amplification, depending on L and R bits)
into 21h (headphone jack on my machine). Mixed signal respects the mute
setting on 21h.
Loopback 1Ah to 14h (S): 1 bit
When one, mix 1Ah (possibly with amplification, depending on L and R bits)
into 14h (internal speaker on my machine). Mixed signal **ignores** the mute
setting on 14h and is present whenever 14h is configured as an output.
Path diagrams
=============
1Ah input selection (DIV is the PC Beep divider set on NID 01h)::
<Beep generator> <PCBEEP pin> <Headphone jack>
| | |
+--DIV--+--!DIV--+ {1Ah boost control}
| |
+--(b == 0)--+--(b != 0)--+
|
>1Ah (Beep/Headphone Mic/Line In)<
Loopback of 1Ah to 21h/14h::
<1Ah (Beep/Headphone Mic/Line In)>
|
{amplify if L/R}
|
+-----!h-----+-----S-----+
| |
{21h mute control} |
| |
>21h (Headphone)< >14h (Internal Speaker)<
Background
==========
All Realtek HDA codecs have a vendor-defined widget with node ID 20h which
provides access to a bank of registers that control various codec functions.
Registers are read and written via the standard HDA processing coefficient
verbs (Set/Get Coefficient Index, Set/Get Processing Coefficient). The node is
named "Realtek Vendor Registers" in public datasheets' verb listings and,
apart from that, is entirely undocumented.
This particular register, exposed at coefficient 0x36 and named in commits from
Realtek, is of note: unlike most registers, which seem to control detailed
amplifier parameters not in scope of the HDA specification, it controls audio
routing which could just as easily have been defined using standard HDA mixer
and selector widgets.
Specifically, it selects between two sources for the input pin widget with Node
ID (NID) 1Ah: the widget's signal can come either from an audio jack (on my
laptop, a Dell XPS 13 9350, it's the headphone jack, but comments in Realtek
commits indicate that it might be a Line In on some machines) or from the PC
Beep line (which is itself multiplexed between the codec's internal beep
generator and external PCBEEP pin, depending on if the beep generator is
enabled via verbs on NID 01h). Additionally, it can mix (with optional
amplification) that signal onto the 21h and/or 14h output pins.
The register's reset value is 0x3717, corresponding to PC Beep on 1Ah that is
then amplified and mixed into both the headphones and the speakers. Not only
does this violate the HDA specification, which says that "[a vendor defined
beep input pin] connection may be maintained *only* while the Link reset
(**RST#**) is asserted", it means that we cannot ignore the register if we care
about the input that 1Ah would otherwise expose or if the PCBEEP trace is
poorly shielded and picks up chassis noise (both of which are the case on my
machine).
Unfortunately, there are lots of ways to get this register configuration wrong.
Linux, it seems, has gone through most of them. For one, the register resets
after S3 suspend: judging by existing code, this isn't the case for all vendor
registers, and it's led to some fixes that improve behavior on cold boot but
don't last after suspend. Other fixes have successfully switched the 1Ah input
away from PC Beep but have failed to disable both loopback paths. On my
machine, this means that the headphone input is amplified and looped back to
the headphone output, which uses the exact same pins! As you might expect, this
causes terrible headphone noise, the character of which is controlled by the
1Ah boost control. (If you've seen instructions online to fix XPS 13 headphone
noise by changing "Headphone Mic Boost" in ALSA, now you know why.)
The information here has been obtained through black-box reverse engineering of
the ALC256 codec's behavior and is not guaranteed to be correct. It likely
also applies for the ALC255, ALC257, ALC235, and ALC236, since those codecs
seem to be close relatives of the ALC256. (They all share one initialization
function.) Additionally, other codecs like the ALC225 and ALC285 also have this
register, judging by existing fixups in ``patch_realtek.c``, but specific
data (e.g. node IDs, bit positions, pin mappings) for those codecs may differ
from what I've described here.

21
Documentation/sysctl/vm.txt
View file

@ -64,7 +64,6 @@ Currently, these files are in /proc/sys/vm:
- swappiness
- user_reserve_kbytes
- vfs_cache_pressure
- watermark_boost_factor
- watermark_scale_factor
- zone_reclaim_mode
@ -873,26 +872,6 @@ ten times more freeable objects than there are.
=============================================================
watermark_boost_factor:
This factor controls the level of reclaim when memory is being fragmented.
It defines the percentage of the high watermark of a zone that will be
reclaimed if pages of different mobility are being mixed within pageblocks.
The intent is that compaction has less work to do in the future and to
increase the success rate of future high-order allocations such as SLUB
allocations, THP and hugetlbfs pages.
To make it sensible with respect to the watermark_scale_factor parameter,
the unit is in fractions of 10,000. The default value of 15,000 means
that up to 150% of the high watermark will be reclaimed in the event of
a pageblock being mixed due to fragmentation. The level of reclaim is
determined by the number of fragmentation events that occurred in the
recent past. If this value is smaller than a pageblock then a pageblocks
worth of pages will be reclaimed (e.g. 2MB on 64-bit x86). A boost factor
of 0 will disable the feature.
=============================================================
watermark_scale_factor:
This factor controls the aggressiveness of kswapd. It defines the

2
Makefile
View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
VERSION = 4
PATCHLEVEL = 19
SUBLEVEL = 114
SUBLEVEL = 123
EXTRAVERSION =
NAME = "People's Front"

105869
abi_gki_aarch64.xml
View file

File diff suppressed because it is too large Load diff

5
abi_gki_aarch64_cuttlefish_whitelist
View file

@ -22,7 +22,6 @@
__cpu_online_mask
cpus_read_lock
cpus_read_unlock
debug_smp_processor_id
delayed_work_timer_fn
destroy_workqueue
_dev_err
@ -124,8 +123,6 @@
platform_device_unregister
__platform_driver_register
platform_driver_unregister
preempt_count_add
preempt_count_sub
preempt_schedule
preempt_schedule_notrace
prepare_to_wait_event
@ -437,7 +434,6 @@
copy_page
dev_driver_string
devres_add
__devres_alloc_node
devres_destroy
devres_free
dma_buf_get
@ -610,7 +606,6 @@
kmem_cache_free
memdup_user
put_unused_fd
remove_conflicting_framebuffers
sg_alloc_table_from_pages
sync_file_create
sync_file_get_fence

253
abi_gki_aarch64_qcom_whitelist
View file

@ -16,7 +16,6 @@
__arch_copy_from_user
__arch_copy_to_user
arch_setup_dma_ops
arch_timer_read_ool_enabled
arm64_const_caps_ready
atomic_notifier_call_chain
atomic_notifier_chain_register
@ -48,6 +47,7 @@
cdev_add
cdev_del
cdev_init
__cfi_slowpath
__check_object_size
__class_create
class_destroy
@ -108,22 +108,6 @@
crypto_destroy_tfm
crypto_shash_setkey
_ctype
debugfs_attr_read
debugfs_attr_write
debugfs_create_atomic_t
debugfs_create_bool
debugfs_create_dir
debugfs_create_file
debugfs_create_file_unsafe
debugfs_create_u16
debugfs_create_u32
debugfs_create_u64
debugfs_create_u8
debugfs_create_x32
debugfs_create_x8
debugfs_print_regs32
debugfs_remove
debugfs_remove_recursive
delayed_work_timer_fn
del_timer
del_timer_sync
@ -246,6 +230,7 @@
dma_fence_signal_locked
__dma_flush_area
__dma_inv_area
dmam_alloc_coherent
dma_release_from_dev_coherent
dma_request_slave_channel
do_exit
@ -258,8 +243,6 @@
drm_panel_notifier_unregister
dst_release
dummy_dma_ops
__dynamic_dev_dbg
__dynamic_pr_debug
enable_irq
ether_setup
eth_type_trans
@ -291,23 +274,9 @@
freezing_slow_path
fwnode_property_read_u32_array
gcd
generic_file_llseek
generic_handle_irq
geni_abort_m_cmd
geni_cancel_m_cmd
geni_se_dump_dbg_regs
geni_se_init
geni_se_iommu_map_buf
geni_se_iommu_unmap_buf
geni_se_resources_init
geni_se_rx_dma_prep
geni_se_rx_dma_unprep
geni_se_select_mode
geni_setup_m_cmd
geni_se_tx_dma_prep
geni_se_tx_dma_unprep
geni_write_reg
genlmsg_put
genl_register_family
genl_unregister_family
gen_pool_add_virt
gen_pool_alloc
gen_pool_create
@ -318,9 +287,7 @@
__get_free_pages
get_pid_task
get_random_bytes
get_se_proto
__get_task_comm
get_tx_fifo_depth
get_unused_fd_flags
gpiochip_add_data_with_key
gpiochip_add_pin_range
@ -345,7 +312,6 @@
handle_edge_irq
handle_level_irq
handle_nested_irq
hex2bin
hex_dump_to_buffer
hrtimer_active
hrtimer_cancel
@ -470,12 +436,9 @@
kstrtoint
kstrtoll
kstrtos8
kstrtos8_from_user
kstrtou16
kstrtou8
kstrtou8_from_user
kstrtouint
kstrtouint_from_user
kstrtoull
kthread_cancel_work_sync
kthread_create_on_node
@ -486,7 +449,6 @@
kthread_stop
kthread_worker_fn
ktime_get
ktime_get_coarse_real_ts64
ktime_get_real_ts64
ktime_get_with_offset
kvfree
@ -542,7 +504,6 @@
module_put
__msecs_to_jiffies
msleep
msleep_interruptible
__mutex_init
mutex_lock
mutex_lock_interruptible
@ -572,9 +533,9 @@
nla_put
__nlmsg_put
no_llseek
nonseekable_open
nr_cpu_ids
ns_capable
ns_to_timespec64
ns_to_timespec
nvmem_cell_get
nvmem_cell_put
@ -749,6 +710,7 @@
_raw_spin_lock_bh
_raw_spin_lock_irq
_raw_spin_lock_irqsave
_raw_spin_trylock
_raw_spin_trylock_bh
_raw_spin_unlock
_raw_spin_unlock_bh
@ -790,6 +752,7 @@
__regmap_init
regmap_multi_reg_write
regmap_raw_read
regmap_raw_write
regmap_read
regmap_update_bits_base
regmap_write
@ -798,6 +761,7 @@
regulator_disable
regulator_enable
regulator_get
regulator_get_voltage
regulator_is_enabled
regulator_put
regulator_set_load
@ -817,8 +781,6 @@
rtc_time64_to_tm
rtc_tm_to_time64
rtc_valid_tm
rt_mutex_lock
rt_mutex_unlock
rtnl_is_locked
rtnl_unlock
saved_command_line
@ -827,18 +789,12 @@
schedule
schedule_timeout
scnprintf
se_config_packing
se_geni_clks_off
se_geni_resources_off
se_geni_resources_on
seq_lseek
seq_open
seq_printf
seq_putc
seq_puts
seq_read
seq_release
seq_write
set_normalized_timespec
set_user_nice
sg_alloc_table
@ -848,15 +804,7 @@
sg_next
__sg_page_iter_next
__sg_page_iter_start
simple_attr_open
simple_attr_read
simple_attr_release
simple_attr_write
simple_open
simple_read_from_buffer
simple_write_to_buffer
single_open
single_release
sk_alloc
skb_add_rx_frag
skb_clone
@ -969,6 +917,7 @@
sysfs_remove_group
sysfs_remove_groups
sysfs_streq
system_freezable_wq
system_freezing_cnt
system_highpri_wq
system_long_wq
@ -998,16 +947,22 @@
trace_event_reg
trace_handle_return
trace_print_flags_seq
trace_print_hex_seq
trace_print_symbols_seq
trace_raw_output_prep
trace_seq_printf
try_module_get
tty_flip_buffer_push
typec_register_partner
typec_register_port
typec_set_data_role
typec_set_pwr_role
typec_unregister_partner
uart_add_one_port
uart_register_driver
uart_remove_one_port
uart_unregister_driver
__udelay
uncached_logk
__unregister_chrdev
unregister_chrdev_region
unregister_inet6addr_notifier
@ -1035,7 +990,6 @@
usb_free_all_descriptors
usb_function_register
usb_function_unregister
usb_func_wakeup
usb_gadget_wakeup
usb_get_dr_mode
usb_interface_id
@ -1125,7 +1079,6 @@
iommu_group_remove_device
iommu_group_set_iommudata
iommu_put_dma_cookie
kstrtoull_from_user
of_dma_is_coherent
of_n_addr_cells
of_n_size_cells
@ -1180,9 +1133,6 @@
# required by cam_flash.ko
thermal_zone_get_cdev_by_name
# required by cam_ife_csid.ko
ns_to_timespec64
# required by cam_ife_hw_mgr.ko
__ll_sc_atomic_fetch_add
@ -1199,6 +1149,10 @@
led_trigger_register_simple
led_trigger_unregister_simple
# required by cam_smmu_api.ko
iommu_dma_enable_best_fit_algo
iommu_dma_reserve_iova
# required by cam_utils.ko
gpio_free_array
of_clk_get_from_provider
@ -1206,13 +1160,13 @@
__request_region
# required by citadel-spi.ko
nonseekable_open
spi_bus_lock
spi_bus_unlock
spi_sync_locked
# required by clk-qcom.ko
clk_aggregate_rate
clk_debug_print_hw
__clk_determine_rate
clk_fixed_rate_ops
clk_hw_get_flags
@ -1276,18 +1230,6 @@
send_sig_info
time64_to_tm
# required by dm-default-key.ko
bio_crypt_alloc_ctx
blk_crypto_evict_key
blk_crypto_init_key
dm_get_device
dm_put_device
dm_read_arg_group
dm_register_target
dm_shift_arg
dm_table_get_mode
dm_unregister_target
# required by dwc3-haps.ko
pcim_enable_device
__pci_register_driver
@ -1323,7 +1265,6 @@
pinctrl_pm_select_default_state
pinctrl_pm_select_sleep_state
strcat
system_freezable_wq
usb_add_gadget_udc
usb_del_gadget_udc
usb_ep_set_maxpacket_limit
@ -1368,12 +1309,7 @@
xt_unregister_target
# required by eud.ko
tty_flip_buffer_push
uart_add_one_port
uart_insert_char
uart_register_driver
uart_remove_one_port
uart_unregister_driver
# required by event_timer.ko
cpumask_next_and
@ -1385,7 +1321,9 @@
of_find_i2c_device_by_node
# required by ftm5.ko
ktime_get_coarse_real_ts64
proc_create
seq_write
# required by google-battery.ko
simple_strtoull
@ -1396,12 +1334,6 @@
# required by governor_bw_hwmon.ko
argv_free
argv_split
__tracepoint_bw_hwmon_meas
__tracepoint_bw_hwmon_update
# required by governor_memlat.ko
__tracepoint_memlat_dev_meas
__tracepoint_memlat_dev_update
# required by gpi.ko
dma_async_device_register
@ -1437,11 +1369,13 @@
i2c_put_dma_safe_msg_buf
# required by ion-alloc.ko
dentry_path
dma_buf_export
dma_get_device_base
dma_get_size
__ll_sc_atomic64_sub_return
mm_event_count
__next_zones_zonelist
ptr_to_hashval
sched_setattr
split_page
vm_map_ram
@ -1452,6 +1386,7 @@
add_wait_queue
alloc_etherdev_mqs
eth_mac_addr
kstrtos8_from_user
pci_clear_master
pci_disable_device
pci_enable_device
@ -1471,7 +1406,7 @@
# required by lpm-stats.ko
kobject_get
module_ktype
on_each_cpu
simple_write_to_buffer
# required by machine_dlkm.ko
devm_snd_soc_register_card
@ -1483,6 +1418,7 @@
snd_soc_dai_set_pll
snd_soc_dai_set_sysclk
snd_soc_dai_set_tdm_slot
snd_soc_get_pcm_runtime
snd_soc_of_get_dai_link_codecs
snd_soc_of_parse_audio_routing
snd_soc_of_parse_card_name
@ -1538,7 +1474,6 @@
bpf_trace_run10
_cleanup_srcu_struct
__clk_get_name
debugfs_lookup
devfreq_cooling_unregister
device_show_int
device_store_int
@ -1566,7 +1501,7 @@
__ll_sc_atomic_or
mmap_min_addr
mmput
noop_llseek
mm_trace_rss_stat
of_devfreq_cooling_register
plist_del
rb_last
@ -1577,21 +1512,18 @@
sysfs_create_bin_file
sysfs_remove_bin_file
sysfs_remove_files
trace_print_symbols_seq
unmapped_area_topdown
unregister_shrinker
vm_insert_page
vm_insert_pfn
# required by msm_bus_rpmh.ko
__msm_bus_scale_client_update_request_cb
__msm_bus_scale_register_cb
__msm_bus_scale_register_client_cb
__msm_bus_scale_update_bw_cb
of_clk_get_by_name
raw_notifier_call_chain
raw_notifier_chain_register
raw_notifier_chain_unregister
rt_mutex_lock
rt_mutex_unlock
# required by msm_drm.ko
adjust_managed_page_count
@ -1599,7 +1531,6 @@
bpf_trace_run12
__clk_get_hw
clk_get_parent
debugfs_create_size_t
device_create_with_groups
devm_clk_bulk_get
devm_of_pwm_get
@ -1863,7 +1794,6 @@
shmem_truncate_range
strreplace
timespec64_to_jiffies
tracing_off
unmap_kernel_range
unmap_mapping_range
vm_get_page_prot
@ -1873,15 +1803,25 @@
# required by msm_ext_display.ko
devm_extcon_dev_unregister
# required by msm_gsi.ko
kstrtoint_from_user
# required by msm_icnss.ko
dmam_alloc_coherent
# required by msm_geni_serial.ko
console_stop
console_suspend_enabled
do_SAK
handle_sysrq
oops_in_progress
__tty_insert_flip_char
tty_insert_flip_string_fixed_flag
uart_console_device
uart_console_write
uart_get_baud_rate
uart_parse_options
uart_resume_port
uart_set_options
uart_suspend_port
uart_update_timeout
uart_write_wakeup
# required by msm_ipc_logging.ko
debugfs_file_get
debugfs_file_put
_raw_read_lock_irq
_raw_read_unlock_irq
_raw_write_lock_irqsave
@ -1893,14 +1833,13 @@
__tracepoint_clock_set_rate
# required by msm_minidump.ko
linux_banner
linux_banner_ptr
# required by msm_npu.ko
kmem_cache_create_usercopy
# required by msm_pm.ko
arm_cpuidle_suspend
clock_debug_print_enabled
cpu_do_idle
cpuidle_dev
cpuidle_register_device
@ -1913,10 +1852,8 @@
param_get_bool
param_get_uint
pending_ipi
pm_gpio_debug_mask
pm_qos_request_for_cpu
pm_qos_request_for_cpumask
regulator_debug_print_enabled
s2idle_set_ops
set_update_ipi_history_callback
suspend_set_ops
@ -1934,10 +1871,6 @@
hwrng_register
hwrng_unregister
# required by msm_rtb.ko
arch_timer_read_counter
set_uncached_logk_func
# required by msm_scm.ko
__arm_smccc_smc
@ -1969,14 +1902,10 @@
irq_chip_set_wake_parent
irq_create_fwspec_mapping
irq_domain_free_irqs_top
msm_gpio_dump_builtin_cb
of_irq_domain_map
register_restart_handler
unregister_restart_handler
# required by pinctrl-spmi-gpio.ko
pmic_gpio_dump_builtin_cb
# required by platform_dlkm.ko
__ll_sc___cmpxchg_case_mb_8
of_property_read_variable_u16_array
@ -2115,7 +2044,6 @@
# required by qpnp-smb5-charger.ko
iio_channel_release
regulator_get_voltage
# required by qpnp_pdphy.ko
device_get_named_child_node
@ -2179,9 +2107,7 @@
csum_tcpudp_nofold
__dev_get_by_index
dev_queue_xmit
__dynamic_netdev_dbg
genl_register_family
genl_unregister_family
genlmsg_put
get_current_napi_context
gro_cells_destroy
gro_cells_init
@ -2201,11 +2127,11 @@
skb_append_pagefrags
skb_checksum
synchronize_rcu
trace_print_hex_seq
unregister_netdevice_many
# required by rndis.ko
dev_get_stats
print_hex_dump_bytes
# required by roles.ko
class_find_device
@ -2424,7 +2350,6 @@
# required by snd-soc-wm-adsp.ko
regmap_async_complete
regmap_raw_write
regmap_raw_write_async
snd_compr_stop_error
snd_soc_component_disable_pin
@ -2432,13 +2357,6 @@
# required by spi-geni-qcom.ko
dma_release_channel
geni_read_reg
geni_se_clk_freq_match
geni_se_qupv3_hw_version
get_rx_fifo_depth
get_tx_fifo_width
se_geni_clks_on
se_get_packing_config
__spi_alloc_controller
spi_register_controller
spi_unregister_controller
@ -2450,7 +2368,6 @@
spmi_controller_remove
# required by st21nfc.ko
desc_to_gpio
device_set_wakeup_capable
# required by subsystem-restart.ko
@ -2480,7 +2397,6 @@
typec_set_vconn_role
typec_unregister_altmode
typec_unregister_port
usb_debug_root
# required by tps-regulator.ko
gpiod_export
@ -2488,21 +2404,44 @@
regulator_list_voltage_table
regulator_map_voltage_ascend
# required by ufs_qcom.ko
ufsdbg_pr_buf_to_std
ufshcd_dme_get_attr
ufshcd_dme_set_attr
ufshcd_get_local_unipro_ver
ufshcd_hold
ufshcd_pltfrm_init
ufshcd_pltfrm_resume
ufshcd_pltfrm_runtime_idle
ufshcd_pltfrm_runtime_resume
ufshcd_pltfrm_runtime_suspend
ufshcd_pltfrm_shutdown
ufshcd_pltfrm_suspend
ufshcd_release
ufshcd_remove
# required by ufshcd-core.ko
async_schedule
bio_crypt_should_process
blk_queue_max_segment_size
blk_queue_rq_timeout
blk_queue_update_dma_pad
dev_pm_opp_remove
down_read_trylock
down_write_trylock
keyslot_manager_create
keyslot_manager_destroy
keyslot_manager_private
keyslot_manager_reprogram_all_keys
keyslot_manager_set_max_dun_bytes
__ll_sc_atomic64_fetch_andnot_release
__ll_sc_atomic64_fetch_or_acquire
__scsi_add_device
scsi_add_host_with_dma
scsi_block_requests
scsi_change_queue_depth
scsi_device_get
scsi_device_put
scsi_dma_map
scsi_dma_unmap
__scsi_execute
scsi_host_alloc
scsi_host_put
scsi_print_command
scsi_print_sense_hdr
scsi_remove_device
scsi_remove_host
scsi_report_bus_reset
scsi_scan_host
scsi_set_cmd_timeout_override
scsi_unblock_requests
sdev_prefix_printk
trace_output_call
utf16s_to_utf8s
# required by wlan.ko
bitmap_print_to_pagebuf
@ -2538,7 +2477,6 @@
cfg80211_unlink_bss
cfg80211_update_owe_info_event
cfg80211_vendor_cmd_reply
cld80211_get_genl_family
complete_and_exit
cpufreq_quick_get_max
__cpuhp_remove_state
@ -2550,10 +2488,9 @@
crypto_alloc_skcipher
crypto_shash_final
crypto_shash_update
default_llseek
deregister_cld_cmd_cb
dev_alloc_name
dump_stack
hex2bin
hex_to_bin
ieee80211_channel_to_frequency
ieee80211_frequency_to_channel
@ -2561,6 +2498,7 @@
ieee80211_hdrlen
irq_set_affinity_hint
mac_pton
msleep_interruptible
netif_tx_stop_all_queues
netlink_broadcast
__netlink_kernel_create
@ -2579,17 +2517,16 @@
pm_system_wakeup
proc_create_data
proc_mkdir
_raw_spin_trylock
register_cld_cmd_cb
register_netevent_notifier
register_sysctl_table
regulatory_set_wiphy_regd
rtnl_lock
save_stack_trace_tsk
schedule_timeout_interruptible
seq_hex_dump
seq_vprintf
set_cpus_allowed_ptr
single_open
single_release
skip_spaces
strchrnul
unregister_netevent_notifier
@ -2635,10 +2572,10 @@
# required by usb_f_gsi.ko
dev_get_by_name
kstrtou16_from_user
usb_composite_setup_continue
usb_ep_autoconfig_by_name
usb_func_ep_queue
usb_func_wakeup
usb_gsi_ep_op
# required by usb_f_mtp.ko

217
abi_gki_aarch64_whitelist
View file

@ -1,4 +1,221 @@
[abi_whitelist]
# commonly used symbols
__cfi_slowpath
__const_udelay
_dev_err
devm_ioremap_resource
devm_kmalloc
devm_request_threaded_irq
_dev_warn
dummy_dma_ops
find_next_bit
find_next_zero_bit
kfree
__kmalloc
kmalloc_caches
kmem_cache_alloc_trace
ktime_get
__list_add_valid
__ll_sc_atomic64_andnot
__ll_sc_atomic_add_return
module_layout
__mutex_init
mutex_lock
mutex_unlock
of_find_property
of_property_read_variable_u32_array
platform_get_irq
platform_get_resource
printk
__put_task_struct
___ratelimit
_raw_spin_lock_irqsave
_raw_spin_unlock_irqrestore
snprintf
__stack_chk_fail
__stack_chk_guard
strcmp
__udelay
# required by arm-smmu.ko
alloc_io_pgtable_ops
amba_bustype
bus_set_iommu
devm_free_irq
_dev_notice
driver_find_device
driver_for_each_device
free_io_pgtable_ops
generic_device_group
iommu_alloc_resv_region
iommu_device_link
iommu_device_register
iommu_device_sysfs_add
iommu_device_unlink
iommu_dma_get_resv_regions
iommu_fwspec_add_ids
iommu_fwspec_free
iommu_fwspec_init
iommu_get_dma_cookie
iommu_group_get_for_dev
iommu_group_put
iommu_group_ref_get
iommu_group_remove_device
iommu_present
iommu_put_dma_cookie
__ll_sc_atomic64_fetch_or
of_device_get_match_data
of_dma_is_coherent
of_phandle_iterator_args
of_phandle_iterator_init
of_phandle_iterator_next
param_ops_bool
param_ops_int
pci_bus_type
pci_device_group
pci_for_each_dma_alias
pci_request_acs
platform_bus_type
__platform_driver_register
platform_driver_unregister
put_device
# required by ufshcd-core.ko
__alloc_workqueue_key
async_schedule
bio_crypt_should_process
blk_queue_max_segment_size
blk_queue_update_dma_pad
bpf_trace_run2
bpf_trace_run4
bpf_trace_run5
bpf_trace_run8
cancel_delayed_work
cancel_delayed_work_sync
cancel_work_sync
clk_disable
clk_enable
clk_prepare
clk_set_rate
clk_unprepare
complete
cpu_number
__cpu_online_mask
delayed_work_timer_fn
destroy_workqueue
dev_driver_string
devfreq_add_device
devfreq_remove_device
devfreq_resume_device
devfreq_suspend_device
device_create_file
device_remove_file
devm_clk_get
devm_kfree
devm_regulator_get
dev_pm_opp_add
dev_pm_opp_remove
dmam_alloc_coherent
down_read
down_read_trylock
down_write
event_triggers_call
find_last_bit
finish_wait
flush_work
free_irq
__init_rwsem
init_timer_key
init_wait_entry
__init_waitqueue_head
jiffies
jiffies_to_usecs
keyslot_manager_create
keyslot_manager_destroy
keyslot_manager_private
keyslot_manager_reprogram_all_keys
keyslot_manager_set_max_dun_bytes
kstrtouint
kstrtoull
__ll_sc_atomic64_fetch_andnot_release
__ll_sc_atomic64_fetch_or_acquire
__ll_sc_atomic_sub_return
memcpy
memset
memzero_explicit
__msecs_to_jiffies
msleep
perf_trace_buf_alloc
perf_trace_run_bpf_submit
__pm_runtime_idle
__pm_runtime_resume
preempt_schedule_notrace
prepare_to_wait_event
print_hex_dump
queue_delayed_work_on
queue_work_on
_raw_spin_lock
_raw_spin_unlock
regulator_count_voltages
regulator_disable
regulator_enable
regulator_set_load
regulator_set_voltage
request_threaded_irq
schedule
schedule_timeout
__scsi_add_device
scsi_add_host_with_dma
scsi_block_requests
scsi_change_queue_depth
scsi_device_get
scsi_device_put
scsi_dma_map
scsi_dma_unmap
__scsi_execute
scsi_host_alloc
scsi_host_put
scsi_print_command
scsi_print_sense_hdr
scsi_remove_device
scsi_remove_host
scsi_report_bus_reset
scsi_scan_host
scsi_unblock_requests
sdev_prefix_printk
sg_next
sprintf
strcpy
strlcpy
strlen
strncmp
sysfs_create_groups
sysfs_remove_groups
system_wq
trace_define_field
trace_event_buffer_commit
trace_event_buffer_reserve
trace_event_ignore_this_pid
trace_event_raw_init
trace_event_reg
trace_handle_return
trace_print_hex_seq
trace_print_symbols_seq
trace_raw_output_prep
trace_seq_printf
up_read
up_write
usleep_range
utf16s_to_utf8s
wait_for_completion_timeout
__wake_up
__warn_printk
# required by ufshcd-pltfrm.ko
_dev_info
kstrdup
of_get_property
of_parse_phandle
of_property_read_string_helper
pm_runtime_enable
__pm_runtime_set_status

1
arch/arm/boot/dts/bcm283x.dtsi
View file

@ -476,6 +476,7 @@
"dsi0_ddr2",
"dsi0_ddr";
status = "disabled";
};
thermal: thermal@7e212000 {

5
arch/arm/boot/dts/imx6qdl.dtsi
View file

@ -1013,9 +1013,8 @@
compatible = "fsl,imx6q-fec";
reg = <0x02188000 0x4000>;
interrupt-names = "int0", "pps";
interrupts-extended =
<&intc 0 118 IRQ_TYPE_LEVEL_HIGH>,
<&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
interrupts = <0 118 IRQ_TYPE_LEVEL_HIGH>,
<0 119 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&clks IMX6QDL_CLK_ENET>,
<&clks IMX6QDL_CLK_ENET>,
<&clks IMX6QDL_CLK_ENET_REF>;

1
arch/arm/boot/dts/imx6qp.dtsi
View file

@ -77,7 +77,6 @@
};
&fec {
/delete-property/interrupts-extended;
interrupts = <0 118 IRQ_TYPE_LEVEL_HIGH>,
<0 119 IRQ_TYPE_LEVEL_HIGH>;
};

4
arch/arm/boot/dts/sun8i-a83t-tbs-a711.dts
View file

@ -318,8 +318,8 @@
};
&reg_dldo3 {
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <2800000>;
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
regulator-name = "vdd-csi";
};

7
arch/arm/include/asm/clocksource.h
View file

@ -1,8 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _ASM_CLOCKSOURCE_H
#define _ASM_CLOCKSOURCE_H
struct arch_clocksource_data {
bool vdso_direct; /* Usable for direct VDSO access? */
};
#include <asm/vdso/clocksource.h>
#endif
#endif /* _ASM_CLOCKSOURCE_H */

20
arch/arm/include/asm/cp15.h
View file

@ -50,25 +50,7 @@
#ifdef CONFIG_CPU_CP15
#define __ACCESS_CP15(CRn, Op1, CRm, Op2) \
"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
#define __ACCESS_CP15_64(Op1, CRm) \
"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
#define __read_sysreg(r, w, c, t) ({ \
t __val; \
asm volatile(r " " c : "=r" (__val)); \
__val; \
})
#define read_sysreg(...) __read_sysreg(__VA_ARGS__)
#define __write_sysreg(v, r, w, c, t) asm volatile(w " " c : : "r" ((t)(v)))
#define write_sysreg(v, ...) __write_sysreg(v, __VA_ARGS__)
#define BPIALL __ACCESS_CP15(c7, 0, c5, 6)
#define ICIALLU __ACCESS_CP15(c7, 0, c5, 0)
#define CNTVCT __ACCESS_CP15_64(1, c14)
#include <asm/vdso/cp15.h>
extern unsigned long cr_alignment; /* defined in entry-armv.S */

11
arch/arm/include/asm/processor.h
View file

@ -23,6 +23,7 @@
#include <asm/ptrace.h>
#include <asm/types.h>
#include <asm/unified.h>
#include <asm/vdso/processor.h>
#ifdef __KERNEL__
#define STACK_TOP ((current->personality & ADDR_LIMIT_32BIT) ? \
@ -94,16 +95,6 @@ extern void release_thread(struct task_struct *);
unsigned long get_wchan(struct task_struct *p);
#if __LINUX_ARM_ARCH__ == 6 || defined(CONFIG_ARM_ERRATA_754327)
#define cpu_relax() \
do { \
smp_mb(); \
__asm__ __volatile__("nop; nop; nop; nop; nop; nop; nop; nop; nop; nop;"); \
} while (0)
#else
#define cpu_relax() barrier()
#endif
#define task_pt_regs(p) \
((struct pt_regs *)(THREAD_START_SP + task_stack_page(p)) - 1)

9
arch/arm/include/asm/vdso/clocksource.h Normal file
View file

@ -0,0 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_VDSOCLOCKSOURCE_H
#define __ASM_VDSOCLOCKSOURCE_H
struct arch_clocksource_data {
bool vdso_direct; /* Usable for direct VDSO access? */
};
#endif /* __ASM_VDSOCLOCKSOURCE_H */

38
arch/arm/include/asm/vdso/cp15.h Normal file
View file

@ -0,0 +1,38 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2020 ARM Ltd.
*/
#ifndef __ASM_VDSO_CP15_H
#define __ASM_VDSO_CP15_H
#ifndef __ASSEMBLY__
#ifdef CONFIG_CPU_CP15
#include <linux/stringify.h>
#define __ACCESS_CP15(CRn, Op1, CRm, Op2) \
"mrc", "mcr", __stringify(p15, Op1, %0, CRn, CRm, Op2), u32
#define __ACCESS_CP15_64(Op1, CRm) \
"mrrc", "mcrr", __stringify(p15, Op1, %Q0, %R0, CRm), u64
#define __read_sysreg(r, w, c, t) ({ \
t __val; \
asm volatile(r " " c : "=r" (__val)); \
__val; \
})
#define read_sysreg(...) __read_sysreg(__VA_ARGS__)
#define __write_sysreg(v, r, w, c, t) asm volatile(w " " c : : "r" ((t)(v)))
#define write_sysreg(v, ...) __write_sysreg(v, __VA_ARGS__)
#define BPIALL __ACCESS_CP15(c7, 0, c5, 6)
#define ICIALLU __ACCESS_CP15(c7, 0, c5, 0)
#define CNTVCT __ACCESS_CP15_64(1, c14)
#endif /* CONFIG_CPU_CP15 */
#endif /* __ASSEMBLY__ */
#endif /* __ASM_VDSO_CP15_H */

145
arch/arm/include/asm/vdso/gettimeofday.h Normal file
View file

@ -0,0 +1,145 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <asm/errno.h>
#include <asm/unistd.h>
#include <asm/vdso/cp15.h>
#include <uapi/linux/time.h>
#define VDSO_HAS_CLOCK_GETRES 1
extern struct vdso_data *__get_datapage(void);
static __always_inline int gettimeofday_fallback(
struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_gettimeofday;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (tv), "r" (tz), "r" (nr)
: "memory");
return ret;
}
static __always_inline long clock_gettime_fallback(
clockid_t _clkid,
struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_clock_gettime64;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline long clock_gettime32_fallback(
clockid_t _clkid,
struct old_timespec32 *_ts)
{
register struct old_timespec32 *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_clock_gettime;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline int clock_getres_fallback(
clockid_t _clkid,
struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_clock_getres_time64;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline int clock_getres32_fallback(
clockid_t _clkid,
struct old_timespec32 *_ts)
{
register struct old_timespec32 *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_clock_getres;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static inline bool arm_vdso_hres_capable(void)
{
return IS_ENABLED(CONFIG_ARM_ARCH_TIMER);
}
#define __arch_vdso_hres_capable arm_vdso_hres_capable
static __always_inline u64 __arch_get_hw_counter(int clock_mode)
{
#ifdef CONFIG_ARM_ARCH_TIMER
u64 cycle_now;
/*
* Core checks for mode already, so this raced against a concurrent
* update. Return something. Core will do another round and then
* see the mode change and fallback to the syscall.
*/
if (clock_mode == VDSO_CLOCKMODE_NONE)
return 0;
isb();
cycle_now = read_sysreg(CNTVCT);
return cycle_now;
#else
/* Make GCC happy. This is compiled out anyway */
return 0;
#endif
}
static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
{
return __get_datapage();
}
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */

22
arch/arm/include/asm/vdso/processor.h Normal file
View file

@ -0,0 +1,22 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2020 ARM Ltd.
*/
#ifndef __ASM_VDSO_PROCESSOR_H
#define __ASM_VDSO_PROCESSOR_H
#ifndef __ASSEMBLY__
#if __LINUX_ARM_ARCH__ == 6 || defined(CONFIG_ARM_ERRATA_754327)
#define cpu_relax() \
do { \
smp_mb(); \
__asm__ __volatile__("nop; nop; nop; nop; nop; nop; nop; nop; nop; nop;"); \
} while (0)
#else
#define cpu_relax() barrier()
#endif
#endif /* __ASSEMBLY__ */
#endif /* __ASM_VDSO_PROCESSOR_H */

8
arch/arm/kernel/setup.c
View file

@ -116,6 +116,14 @@ EXPORT_SYMBOL(elf_hwcap2);
char* (*arch_read_hardware_id)(void);
EXPORT_SYMBOL(arch_read_hardware_id);
/* Vendor stub */
unsigned int boot_reason;
EXPORT_SYMBOL_GPL(boot_reason);
/* Vendor stub */
unsigned int cold_boot;
EXPORT_SYMBOL_GPL(cold_boot);
#ifdef MULTI_CPU
struct processor processor __ro_after_init;
#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR)

2
arch/arm/mach-imx/Makefile
View file

@ -89,8 +89,10 @@ AFLAGS_suspend-imx6.o :=-Wa,-march=armv7-a
obj-$(CONFIG_SOC_IMX6) += suspend-imx6.o
obj-$(CONFIG_SOC_IMX53) += suspend-imx53.o
endif
ifeq ($(CONFIG_ARM_CPU_SUSPEND),y)
AFLAGS_resume-imx6.o :=-Wa,-march=armv7-a
obj-$(CONFIG_SOC_IMX6) += resume-imx6.o
endif
obj-$(CONFIG_SOC_IMX6) += pm-imx6.o
obj-$(CONFIG_SOC_IMX1) += mach-imx1.o

4
arch/arm/mm/dma-mapping.c
View file

@ -378,7 +378,7 @@ __dma_alloc_remap(struct page *page, size_t size, gfp_t gfp, pgprot_t prot,
static void __dma_free_remap(void *cpu_addr, size_t size)
{
dma_common_free_remap(cpu_addr, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP);
VM_ARM_DMA_CONSISTENT | VM_USERMAP, false);
}
#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
@ -1648,7 +1648,7 @@ void __arm_iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) == 0) {
dma_common_free_remap(cpu_addr, size,
VM_ARM_DMA_CONSISTENT | VM_USERMAP);
VM_ARM_DMA_CONSISTENT | VM_USERMAP, false);
}
__iommu_remove_mapping(dev, handle, size);

52
arch/arm/net/bpf_jit_32.c
View file

@ -930,7 +930,11 @@ static inline void emit_a32_rsh_i64(const s8 dst[],
rd = arm_bpf_get_reg64(dst, tmp, ctx);
/* Do LSR operation */
if (val < 32) {
if (val == 0) {
/* An immediate value of 0 encodes a shift amount of 32
* for LSR. To shift by 0, don't do anything.
*/
} else if (val < 32) {
emit(ARM_MOV_SI(tmp2[1], rd[1], SRTYPE_LSR, val), ctx);
emit(ARM_ORR_SI(rd[1], tmp2[1], rd[0], SRTYPE_ASL, 32 - val), ctx);
emit(ARM_MOV_SI(rd[0], rd[0], SRTYPE_LSR, val), ctx);
@ -956,7 +960,11 @@ static inline void emit_a32_arsh_i64(const s8 dst[],
rd = arm_bpf_get_reg64(dst, tmp, ctx);
/* Do ARSH operation */
if (val < 32) {
if (val == 0) {
/* An immediate value of 0 encodes a shift amount of 32
* for ASR. To shift by 0, don't do anything.
*/
} else if (val < 32) {
emit(ARM_MOV_SI(tmp2[1], rd[1], SRTYPE_LSR, val), ctx);
emit(ARM_ORR_SI(rd[1], tmp2[1], rd[0], SRTYPE_ASL, 32 - val), ctx);
emit(ARM_MOV_SI(rd[0], rd[0], SRTYPE_ASR, val), ctx);
@ -993,21 +1001,35 @@ static inline void emit_a32_mul_r64(const s8 dst[], const s8 src[],
arm_bpf_put_reg32(dst_hi, rd[0], ctx);
}
static bool is_ldst_imm(s16 off, const u8 size)
{
s16 off_max = 0;
switch (size) {
case BPF_B:
case BPF_W:
off_max = 0xfff;
break;
case BPF_H:
off_max = 0xff;
break;
case BPF_DW:
/* Need to make sure off+4 does not overflow. */
off_max = 0xfff - 4;
break;
}
return -off_max <= off && off <= off_max;
}
/* *(size *)(dst + off) = src */
static inline void emit_str_r(const s8 dst, const s8 src[],
s32 off, struct jit_ctx *ctx, const u8 sz){
s16 off, struct jit_ctx *ctx, const u8 sz){
const s8 *tmp = bpf2a32[TMP_REG_1];
s32 off_max;
s8 rd;
rd = arm_bpf_get_reg32(dst, tmp[1], ctx);
if (sz == BPF_H)
off_max = 0xff;
else
off_max = 0xfff;
if (off < 0 || off > off_max) {
if (!is_ldst_imm(off, sz)) {
emit_a32_mov_i(tmp[0], off, ctx);
emit(ARM_ADD_R(tmp[0], tmp[0], rd), ctx);
rd = tmp[0];
@ -1036,18 +1058,12 @@ static inline void emit_str_r(const s8 dst, const s8 src[],
/* dst = *(size*)(src + off) */
static inline void emit_ldx_r(const s8 dst[], const s8 src,
s32 off, struct jit_ctx *ctx, const u8 sz){
s16 off, struct jit_ctx *ctx, const u8 sz){
const s8 *tmp = bpf2a32[TMP_REG_1];
const s8 *rd = is_stacked(dst_lo) ? tmp : dst;
s8 rm = src;
s32 off_max;
if (sz == BPF_H)
off_max = 0xff;
else
off_max = 0xfff;
if (off < 0 || off > off_max) {
if (!is_ldst_imm(off, sz)) {
emit_a32_mov_i(tmp[0], off, ctx);
emit(ARM_ADD_R(tmp[0], tmp[0], src), ctx);
rm = tmp[0];

104
arch/arm64/Kconfig
View file

@ -100,6 +100,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@ -138,6 +139,8 @@ config ARM64
select HAVE_GENERIC_DMA_COHERENT
select HAVE_HW_BREAKPOINT if PERF_EVENTS
select HAVE_IRQ_TIME_ACCOUNTING
select HAVE_KERNEL_GZIP
select HAVE_KERNEL_LZ4
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP if NUMA
select HAVE_NMI
@ -152,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
@ -503,6 +507,22 @@ config ARM64_ERRATUM_1463225
If unsure, say Y.
config ARM64_ERRATUM_1542419
bool "Neoverse-N1: workaround mis-ordering of instruction fetches"
default y
help
This option adds a workaround for ARM Neoverse-N1 erratum
1542419.
Affected Neoverse-N1 cores could execute a stale instruction when
modified by another CPU. The workaround depends on a firmware
counterpart.
Workaround the issue by hiding the DIC feature from EL0. This
forces user-space to perform cache maintenance.
If unsure, say Y.
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
@ -752,6 +772,18 @@ config HOTPLUG_CPU
Say Y here to experiment with turning CPUs off and on. CPUs
can be controlled through /sys/devices/system/cpu.
# The GPIO number here must be sorted by descending number. In case of
# a multiplatform kernel, we just want the highest value required by the
# selected platforms.
config ARCH_NR_GPIO
int "Number of GPIOs in the system"
default 1280 if ARCH_QCOM
default 256
help
Maximum number of GPIOs in the system.
If unsure, leave the default value.
# Common NUMA Features
config NUMA
bool "Numa Memory Allocation and Scheduler Support"
@ -830,6 +862,37 @@ config ARCH_HAS_CACHE_LINE_SIZE
config CC_HAVE_SHADOW_CALL_STACK
def_bool $(cc-option, -fsanitize=shadow-call-stack -ffixed-x18)
config ARM64_DMA_USE_IOMMU
bool "ARM64 DMA iommu integration"
select ARM_HAS_SG_CHAIN
select NEED_SG_DMA_LENGTH
help
Enable using iommu through the standard dma apis.
dma_alloc_coherent() will allocate scatter-gather memory
which is made virtually contiguous via iommu.
Enable if system contains IOMMU hardware.
if ARM64_DMA_USE_IOMMU
config ARM64_DMA_IOMMU_ALIGNMENT
int "Maximum PAGE_SIZE order of alignment for DMA IOMMU buffers"
range 4 9
default 9
help
DMA mapping framework by default aligns all buffers to the smallest
PAGE_SIZE order which is greater than or equal to the requested buffer
size. This works well for buffers up to a few hundreds kilobytes, but
for larger buffers it just a waste of address space. Drivers which has
relatively small addressing window (like 64Mib) might run out of
virtual space with just a few allocations.
With this parameter you can specify the maximum PAGE_SIZE order for
DMA IOMMU buffers. Larger buffers will be aligned only to this
specified order. The order is expressed as a power of two multiplied
by the PAGE_SIZE.
endif
config SECCOMP
bool "Enable seccomp to safely compute untrusted bytecode"
---help---
@ -981,6 +1044,19 @@ config ARM64_TAGGED_ADDR_ABI
to system calls as pointer arguments. For details, see
Documentation/arm64/tagged-address-abi.rst.
config COMPAT_VDSO
bool "Enable vDSO for 32-bit applications"
depends on !CPU_BIG_ENDIAN && "$(CROSS_COMPILE_COMPAT)" != ""
select GENERIC_COMPAT_VDSO
default y
help
Place in the process address space of 32-bit applications an
ELF shared object providing fast implementations of gettimeofday
and clock_gettime.
You must have a 32-bit build of glibc 2.22 or later for programs
to seamlessly take advantage of this.
menuconfig ARMV8_DEPRECATED
bool "Emulate deprecated/obsolete ARMv8 instructions"
depends on COMPAT
@ -1358,6 +1434,34 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
config KUSER_HELPERS
bool "Enable kuser helpers page for 32 bit applications."
depends on COMPAT
default y
help
Warning: disabling this option may break 32-bit user programs.
Provide kuser helpers to compat tasks. The kernel provides
helper code to userspace in read only form at a fixed location
to allow userspace to be independent of the CPU type fitted to
the system. This permits binaries to be run on ARMv4 through
to ARMv8 without modification.
See Documentation/arm/kernel_user_helpers.txt for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
exploits.
If all of the binaries and libraries which run on your platform
are built specifically for your platform, and make no use of
these helpers, then you can turn this option off to hinder
such exploits. However, in that case, if a binary or library
relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not
need these helpers; otherwise, the safe option is to say Y.
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC

7
arch/arm64/Makefile
View file

@ -49,10 +49,10 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
KBUILD_CFLAGS += $(call cc-disable-warning, psabi)
KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@ -171,6 +171,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
$(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
$(build)=arch/arm64/kernel/vdso32 \
include/generated/vdso32-offsets.h)
endif
define archhelp

3
arch/arm64/boot/dts/allwinner/sun50i-h6.dtsi
View file

@ -70,8 +70,7 @@
};
pmu {
compatible = "arm,cortex-a53-pmu",
"arm,armv8-pmuv3";
compatible = "arm,cortex-a53-pmu";
interrupts = <GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>,

40
arch/arm64/configs/gki_defconfig
View file

@ -2,6 +2,7 @@ CONFIG_AUDIT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_PREEMPT=y
CONFIG_IRQ_TIME_ACCOUNTING=y
CONFIG_TASKSTATS=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
@ -14,12 +15,12 @@ CONFIG_MEMCG=y
CONFIG_MEMCG_SWAP=y
CONFIG_BLK_CGROUP=y
CONFIG_CGROUP_SCHED=y
# CONFIG_FAIR_GROUP_SCHED is not set
CONFIG_CGROUP_FREEZER=y
CONFIG_CPUSETS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_BPF=y
CONFIG_NAMESPACES=y
# CONFIG_UTS_NS is not set
# CONFIG_PID_NS is not set
CONFIG_SCHED_TUNE=y
CONFIG_BLK_DEV_INITRD=y
@ -49,6 +50,7 @@ CONFIG_SCHED_MC=y
CONFIG_NR_CPUS=32
CONFIG_SECCOMP=y
CONFIG_PARAVIRT=y
# CONFIG_ARM64_TAGGED_ADDR_ABI is not set
CONFIG_ARMV8_DEPRECATED=y
CONFIG_SWP_EMULATION=y
CONFIG_CP15_BARRIER_EMULATION=y
@ -64,6 +66,7 @@ CONFIG_ENERGY_MODEL=y
CONFIG_CPU_IDLE=y
CONFIG_ARM_CPUIDLE=y
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_TIMES=y
CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=y
@ -94,6 +97,7 @@ CONFIG_CLEANCACHE=y
CONFIG_CMA=y
CONFIG_CMA_AREAS=16
CONFIG_ZSMALLOC=y
CONFIG_MM_EVENT_STAT=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
@ -106,7 +110,9 @@ CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_NET_IPIP=y
CONFIG_NET_IPGRE_DEMUX=y
CONFIG_NET_IPGRE=y
CONFIG_NET_IPVTI=y
CONFIG_INET_ESP=y
CONFIG_INET_UDP_DIAG=y
@ -118,6 +124,7 @@ CONFIG_INET6_ESP=y
CONFIG_INET6_IPCOMP=y
CONFIG_IPV6_MIP6=y
CONFIG_IPV6_VTI=y
CONFIG_IPV6_GRE=y
CONFIG_IPV6_MULTIPLE_TABLES=y
CONFIG_IPV6_SUBTREES=y
CONFIG_NETFILTER=y
@ -209,6 +216,8 @@ CONFIG_CFG80211=y
CONFIG_MAC80211=y
CONFIG_RFKILL=y
# CONFIG_UEVENT_HELPER is not set
CONFIG_FW_LOADER_USER_HELPER=y
CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y
# CONFIG_FW_CACHE is not set
# CONFIG_ALLOW_DEV_COREDUMP is not set
CONFIG_DMA_CMA=y
@ -221,8 +230,9 @@ CONFIG_UID_SYS_STATS=y
CONFIG_SCSI=y
# CONFIG_SCSI_PROC_FS is not set
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_UFSHCD=y
CONFIG_SCSI_UFSHCD_PLATFORM=y
CONFIG_CHR_DEV_SG=y
CONFIG_SCSI_UFSHCD=m
CONFIG_SCSI_UFSHCD_PLATFORM=m
CONFIG_SCSI_UFS_CRYPTO=y
CONFIG_MD=y
CONFIG_BLK_DEV_DM=y
@ -237,6 +247,7 @@ CONFIG_DM_BOW=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
CONFIG_TUN=y
CONFIG_VETH=y
# CONFIG_ETHERNET is not set
CONFIG_PHYLIB=y
CONFIG_PPP=y
@ -288,31 +299,39 @@ CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_SERIAL_MSM_GENI_EARLY_CONSOLE=y
CONFIG_SERIAL_DEV_BUS=y
CONFIG_HW_RANDOM=y
# CONFIG_HW_RANDOM_CAVIUM is not set
# CONFIG_DEVPORT is not set
# CONFIG_I2C_COMPAT is not set
CONFIG_I2C_CHARDEV=y
# CONFIG_I2C_HELPER_AUTO is not set
CONFIG_SPI=y
CONFIG_SPMI=y
# CONFIG_SPMI_MSM_PMIC_ARB is not set
CONFIG_PINCTRL_AMD=y
CONFIG_GPIO_SYSFS=y
CONFIG_POWER_AVS=y
CONFIG_POWER_RESET_HISI=y
# CONFIG_HWMON is not set
CONFIG_THERMAL=y
CONFIG_THERMAL_STATISTICS=y
CONFIG_THERMAL_WRITABLE_TRIPS=y
CONFIG_THERMAL_GOV_USER_SPACE=y
CONFIG_CPU_THERMAL=y
CONFIG_DEVFREQ_THERMAL=y
CONFIG_THERMAL_EMULATION=y
CONFIG_WATCHDOG=y
CONFIG_WATCHDOG_CORE=y
CONFIG_MFD_ACT8945A=y
CONFIG_MFD_SYSCON=y
CONFIG_REGULATOR=y
CONFIG_REGULATOR_FIXED_VOLTAGE=y
CONFIG_REGULATOR_PROXY_CONSUMER=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
CONFIG_MEDIA_CONTROLLER=y
CONFIG_VIDEO_V4L2_SUBDEV_API=y
# CONFIG_VGA_ARB is not set
CONFIG_DRM=y
# CONFIG_DRM_FBDEV_EMULATION is not set
@ -325,7 +344,6 @@ CONFIG_SND=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_DRIVERS is not set
CONFIG_SND_USB_AUDIO=y
CONFIG_SND_SOC=y
@ -354,14 +372,15 @@ CONFIG_TYPEC=y
CONFIG_MMC=y
# CONFIG_PWRSEQ_EMMC is not set
# CONFIG_PWRSEQ_SIMPLE is not set
CONFIG_MMC_CRYPTO=y
CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_PLTFM=y
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TRANSIENT=y
CONFIG_EDAC=y
CONFIG_RTC_CLASS=y
# CONFIG_RTC_SYSTOHC is not set
CONFIG_RTC_DRV_PL030=y
CONFIG_RTC_DRV_PL031=y
CONFIG_DMADEVICES=y
@ -373,7 +392,10 @@ CONFIG_COMMON_CLK_SCPI=y
# CONFIG_COMMON_CLK_XGENE is not set
CONFIG_HWSPINLOCK=y
CONFIG_MAILBOX=y
CONFIG_ARM_SMMU=y
CONFIG_IOMMU_IO_PGTABLE_FAST=y
CONFIG_ARM_SMMU=m
CONFIG_REMOTEPROC=y
CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y
CONFIG_DEVFREQ_GOV_PERFORMANCE=y
CONFIG_DEVFREQ_GOV_POWERSAVE=y
CONFIG_DEVFREQ_GOV_USERSPACE=y
@ -464,6 +486,8 @@ CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_HARDENED_USERCOPY=y
CONFIG_FORTIFY_SOURCE=y
CONFIG_STATIC_USERMODEHELPER=y
CONFIG_SECURITY_SELINUX=y
CONFIG_INIT_STACK_ALL=y
CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y
@ -487,5 +511,3 @@ CONFIG_PANIC_TIMEOUT=5
CONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_PREEMPT is not set
CONFIG_DEBUG_LIST=y
CONFIG_CORESIGHT=y
CONFIG_CORESIGHT_STM=y

3
arch/arm64/include/asm/cache.h
View file

@ -22,6 +22,7 @@
#define CTR_L1IP_MASK 3
#define CTR_DMINLINE_SHIFT 16
#define CTR_IMINLINE_SHIFT 0
#define CTR_IMINLINE_MASK 0xf
#define CTR_ERG_SHIFT 20
#define CTR_CWG_SHIFT 24
#define CTR_CWG_MASK 15
@ -29,7 +30,7 @@
#define CTR_DIC_SHIFT 29
#define CTR_CACHE_MINLINE_MASK \
(0xf << CTR_DMINLINE_SHIFT | 0xf << CTR_IMINLINE_SHIFT)
(0xf << CTR_DMINLINE_SHIFT | CTR_IMINLINE_MASK << CTR_IMINLINE_SHIFT)
#define CTR_L1IP(ctr) (((ctr) >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK)

9
arch/arm64/include/asm/cacheflush.h
View file

@ -131,6 +131,15 @@ static inline void flush_cache_range(struct vm_area_struct *vma,
extern void __dma_map_area(const void *, size_t, int);
extern void __dma_unmap_area(const void *, size_t, int);
extern void __dma_flush_area(const void *, size_t);
extern void __dma_inv_area(const void *start, size_t size);
extern void __dma_clean_area(const void *start, size_t size);
#define dmac_flush_range(start, end) \
__dma_flush_area(start, (void *)(end) - (void *)(start))
#define dmac_inv_range(start, end) \
__dma_inv_area(start, (void *)(end) - (void *)(start))
#define dmac_clean_range(start, end) \
__dma_clean_area(start, (void *)(end) - (void *)(start))
/*
* Copy user data from/to a page which is mapped into a different

4
arch/arm64/include/asm/clocksource.h
View file

@ -2,8 +2,6 @@
#ifndef _ASM_CLOCKSOURCE_H
#define _ASM_CLOCKSOURCE_H
struct arch_clocksource_data {
bool vdso_direct; /* Usable for direct VDSO access? */
};
#include <asm/vdso/clocksource.h>
#endif

3
arch/arm64/include/asm/cpucaps.h
View file

@ -53,8 +53,9 @@
#define ARM64_HAS_STAGE2_FWB 32
#define ARM64_WORKAROUND_1463225 33
#define ARM64_SSBS 34
#define ARM64_WORKAROUND_1542419 35
/* kabi: reserve 35 - 62 for future cpu capabilities */
/* kabi: reserve 36 - 62 for future cpu capabilities */
#define ARM64_NCAPS 62
#endif /* __ASM_CPUCAPS_H */

2
arch/arm64/include/asm/cputype.h
View file

@ -80,6 +80,7 @@
#define ARM_CPU_PART_CORTEX_A35 0xD04
#define ARM_CPU_PART_CORTEX_A55 0xD05
#define ARM_CPU_PART_CORTEX_A76 0xD0B
#define ARM_CPU_PART_NEOVERSE_N1 0xD0C
#define APM_CPU_PART_POTENZA 0x000
@ -107,6 +108,7 @@
#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_CORTEX_A76 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A76)
#define MIDR_NEOVERSE_N1 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N1)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)

9
arch/arm64/include/asm/device.h
View file

@ -24,9 +24,18 @@ struct dev_archdata {
const struct dma_map_ops *dev_dma_ops;
#endif
bool dma_coherent;
#ifdef CONFIG_ARM64_DMA_USE_IOMMU
struct dma_iommu_mapping *mapping;
#endif
};
struct pdev_archdata {
};
#ifdef CONFIG_ARM64_DMA_USE_IOMMU
#define to_dma_iommu_mapping(dev) ((dev)->archdata.mapping)
#else
#define to_dma_iommu_mapping(dev) NULL
#endif
#endif

19
arch/arm64/include/asm/dma-contiguous.h Normal file
View file

@ -0,0 +1,19 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (c) 2013,2017-2018 The Linux Foundation. All rights reserved.
*/
#ifndef _ASM_DMA_CONTIGUOUS_H
#define _ASM_DMA_CONTIGUOUS_H
#ifdef __KERNEL__
#ifdef CONFIG_DMA_CMA
#include <linux/types.h>
void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size);
#endif
#endif
#endif

37
arch/arm64/include/asm/dma-iommu.h Normal file
View file

@ -0,0 +1,37 @@
/* SPDX-License-Identifier: GPL-2.0-only */
#ifndef ASMARM_DMA_IOMMU_H
#define ASMARM_DMA_IOMMU_H
#ifdef __KERNEL__
#include <linux/err.h>
#include <linux/mm_types.h>
#include <linux/scatterlist.h>
#include <linux/dma-debug.h>
#include <linux/kref.h>
#include <linux/dma-mapping-fast.h>
struct dma_iommu_mapping {
/* iommu specific data */
struct iommu_domain *domain;
bool init;
struct kref kref;
const struct dma_map_ops *ops;
/* Protects bitmap */
spinlock_t lock;
void *bitmap;
size_t bits;
dma_addr_t base;
struct dma_fast_smmu_mapping *fast;
};
#ifdef CONFIG_ARM64_DMA_USE_IOMMU
void arm_iommu_put_dma_cookie(struct iommu_domain *domain);
#else /* !CONFIG_ARM64_DMA_USE_IOMMU */
static inline void arm_iommu_put_dma_cookie(struct iommu_domain *domain) {}
#endif /* CONFIG_ARM64_DMA_USE_IOMMU */
#endif /* __KERNEL__ */
#endif

20
arch/arm64/include/asm/elf.h
View file

@ -209,11 +209,25 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
#ifdef CONFIG_COMPAT_VDSO
#define COMPAT_ARCH_DLINFO \
do { \
/* \
* Note that we use Elf64_Off instead of elf_addr_t because \
* elf_addr_t in compat is defined as Elf32_Addr and casting \
* current->mm->context.vdso to it triggers a cast warning of \
* cast from pointer to integer of different size. \
*/ \
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
(Elf64_Off)current->mm->context.vdso); \
} while (0)
#else
#define COMPAT_ARCH_DLINFO
extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
int uses_interp);
#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
aarch32_setup_vectors_page
aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */

2
arch/arm64/include/asm/lse.h
View file

@ -21,7 +21,7 @@
#else /* __ASSEMBLER__ */
#ifdef CONFIG_LTO_CLANG
#define __LSE_PREAMBLE ".arch armv8-a+lse\n"
#define __LSE_PREAMBLE ".arch_extension lse\n"
#else
__asm__(".arch_extension lse");
#define __LSE_PREAMBLE

9
arch/arm64/include/asm/processor.h
View file

@ -40,6 +40,8 @@
#include <linux/stddef.h>
#include <linux/string.h>
#include <vdso/processor.h>
#include <asm/alternative.h>
#include <asm/cpufeature.h>
#include <asm/hw_breakpoint.h>
@ -53,7 +55,7 @@
* TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area.
*/
#ifdef CONFIG_COMPAT
#ifdef CONFIG_ARM64_64K_PAGES
#if defined(CONFIG_ARM64_64K_PAGES) && defined(CONFIG_KUSER_HELPERS)
/*
* With CONFIG_ARM64_64K_PAGES enabled, the last page is occupied
* by the compat vectors page.
@ -227,11 +229,6 @@ extern void release_thread(struct task_struct *);
unsigned long get_wchan(struct task_struct *p);
static inline void cpu_relax(void)
{
asm volatile("yield" ::: "memory");
}
/* Thread switching */
extern struct task_struct *cpu_switch_to(struct task_struct *prev,
struct task_struct *next);

46
arch/arm64/include/asm/signal32.h
View file

@ -20,7 +20,51 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
struct compat_sigcontext {
/* We always set these two fields to 0 */
compat_ulong_t trap_no;
compat_ulong_t error_code;
compat_ulong_t oldmask;
compat_ulong_t arm_r0;
compat_ulong_t arm_r1;
compat_ulong_t arm_r2;
compat_ulong_t arm_r3;
compat_ulong_t arm_r4;
compat_ulong_t arm_r5;
compat_ulong_t arm_r6;
compat_ulong_t arm_r7;
compat_ulong_t arm_r8;
compat_ulong_t arm_r9;
compat_ulong_t arm_r10;
compat_ulong_t arm_fp;
compat_ulong_t arm_ip;
compat_ulong_t arm_sp;
compat_ulong_t arm_lr;
compat_ulong_t arm_pc;
compat_ulong_t arm_cpsr;
compat_ulong_t fault_address;
};
struct compat_ucontext {
compat_ulong_t uc_flags;
compat_uptr_t uc_link;
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
struct compat_sigframe {
struct compat_ucontext uc;
compat_ulong_t retcode[2];
};
struct compat_rt_sigframe {
struct compat_siginfo info;
struct compat_sigframe sig;
};
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);

4
arch/arm64/include/asm/sysreg.h
View file

@ -60,7 +60,9 @@
#ifndef CONFIG_BROKEN_GAS_INST
#ifdef __ASSEMBLY__
#define __emit_inst(x) .inst (x)
// The space separator is omitted so that __emit_inst(x) can be parsed as
// either an assembler directive or an assembler macro argument.
#define __emit_inst(x) .inst(x)
#else
#define __emit_inst(x) ".inst " __stringify((x)) "\n\t"
#endif

5
arch/arm64/include/asm/unistd.h
View file

@ -33,8 +33,13 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
#define __NR_compat_clock_gettime 263
#define __NR_compat_clock_getres 264
#define __NR_compat_clock_gettime64 403
#define __NR_compat_clock_getres_time64 406
/*
* The following SVCs are ARM private.

3
arch/arm64/include/asm/vdso.h
View file

@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
#ifdef CONFIG_COMPAT_VDSO
#include <generated/vdso32-offsets.h>
#endif
#define VDSO_SYMBOL(base, name) \
({ \

9
arch/arm64/include/asm/vdso/clocksource.h Normal file
View file

@ -0,0 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_VDSOCLOCKSOURCE_H
#define __ASM_VDSOCLOCKSOURCE_H
struct arch_clocksource_data {
bool vdso_direct; /* Usable for direct VDSO access? */
};
#endif

51
arch/arm64/include/asm/vdso/compat_barrier.h Normal file
View file

@ -0,0 +1,51 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __COMPAT_BARRIER_H
#define __COMPAT_BARRIER_H
#ifndef __ASSEMBLY__
/*
* Warning: This code is meant to be used with
* ENABLE_COMPAT_VDSO only.
*/
#ifndef ENABLE_COMPAT_VDSO
#error This header is meant to be used with ENABLE_COMPAT_VDSO only
#endif
#ifdef dmb
#undef dmb
#endif
#if __LINUX_ARM_ARCH__ >= 7
#define dmb(option) __asm__ __volatile__ ("dmb " #option : : : "memory")
#elif __LINUX_ARM_ARCH__ == 6
#define dmb(x) __asm__ __volatile__ ("mcr p15, 0, %0, c7, c10, 5" \
: : "r" (0) : "memory")
#else
#define dmb(x) __asm__ __volatile__ ("" : : : "memory")
#endif
#if __LINUX_ARM_ARCH__ >= 8 && defined(CONFIG_AS_DMB_ISHLD)
#define aarch32_smp_mb() dmb(ish)
#define aarch32_smp_rmb() dmb(ishld)
#define aarch32_smp_wmb() dmb(ishst)
#else
#define aarch32_smp_mb() dmb(ish)
#define aarch32_smp_rmb() aarch32_smp_mb()
#define aarch32_smp_wmb() dmb(ishst)
#endif
#undef smp_mb
#undef smp_rmb
#undef smp_wmb
#define smp_mb() aarch32_smp_mb()
#define smp_rmb() aarch32_smp_rmb()
#define smp_wmb() aarch32_smp_wmb()
#endif /* !__ASSEMBLY__ */
#endif /* __COMPAT_BARRIER_H */

124
arch/arm64/include/asm/vdso/compat_gettimeofday.h Normal file
View file

@ -0,0 +1,124 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <asm/unistd.h>
#include <asm/errno.h>
#include <asm/vdso/compat_barrier.h>
#define __VDSO_USE_SYSCALL ULLONG_MAX
#define VDSO_HAS_CLOCK_GETRES 1
#define BUILD_VDSO32 1
static __always_inline
int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_gettimeofday;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (tv), "r" (tz), "r" (nr)
: "memory");
return ret;
}
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_clock_gettime;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline
int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_clock_getres;
asm volatile(
" swi #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode)
{
u64 res;
/*
* clock_mode == 0 implies that vDSO are enabled otherwise
* fallback on syscall.
*/
if (clock_mode)
return __VDSO_USE_SYSCALL;
/*
* This isb() is required to prevent that the counter value
* is speculated.
*/
isb();
asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
/*
* This isb() is required to prevent that the seq lock is
* speculated.
*/
isb();
return res;
}
static __always_inline const struct vdso_data *__arch_get_vdso_data(void)
{
const struct vdso_data *ret;
/*
* This simply puts &_vdso_data into ret. The reason why we don't use
* `ret = _vdso_data` is that the compiler tends to optimise this in a
* very suboptimal way: instead of keeping &_vdso_data in a register,
* it goes through a relocation almost every time _vdso_data must be
* accessed (even in subfunctions). This is both time and space
* consuming: each relocation uses a word in the code section, and it
* has to be loaded at runtime.
*
* This trick hides the assignment from the compiler. Since it cannot
* track where the pointer comes from, it will only use one relocation
* where __arch_get_vdso_data() is called, and then keep the result in
* a register.
*/
asm volatile("mov %0, %1" : "=r"(ret) : "r"(_vdso_data));
return ret;
}
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */

102
arch/arm64/include/asm/vdso/gettimeofday.h Normal file
View file

@ -0,0 +1,102 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2018 ARM Limited
*/
#ifndef __ASM_VDSO_GETTIMEOFDAY_H
#define __ASM_VDSO_GETTIMEOFDAY_H
#ifndef __ASSEMBLY__
#include <asm/unistd.h>
#define __VDSO_USE_SYSCALL ULLONG_MAX
#define VDSO_HAS_CLOCK_GETRES 1
static __always_inline
int gettimeofday_fallback(struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("x1") = _tz;
register struct __kernel_old_timeval *tv asm("x0") = _tv;
register long ret asm ("x0");
register long nr asm("x8") = __NR_gettimeofday;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (tv), "r" (tz), "r" (nr)
: "memory");
return ret;
}
static __always_inline
long clock_gettime_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_gettime;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline
int clock_getres_fallback(clockid_t _clkid, struct __kernel_timespec *_ts)
{
register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_getres;
asm volatile(
" svc #0\n"
: "=r" (ret)
: "r" (clkid), "r" (ts), "r" (nr)
: "memory");
return ret;
}
static __always_inline u64 __arch_get_hw_counter(s32 clock_mode)
{
u64 res;
/*
* clock_mode == 0 implies that vDSO are enabled otherwise
* fallback on syscall.
*/
if (clock_mode)
return __VDSO_USE_SYSCALL;
/*
* This isb() is required to prevent that the counter value
* is speculated.
*/
isb();
asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
/*
* This isb() is required to prevent that the seq lock is
* speculated.#
*/
isb();
return res;
}
static __always_inline
const struct vdso_data *__arch_get_vdso_data(void)
{
return _vdso_data;
}
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_GETTIMEOFDAY_H */

17
arch/arm64/include/asm/vdso/processor.h Normal file
View file

@ -0,0 +1,17 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2020 ARM Ltd.
*/
#ifndef __ASM_VDSO_PROCESSOR_H
#define __ASM_VDSO_PROCESSOR_H
#ifndef __ASSEMBLY__
static inline void cpu_relax(void)
{
asm volatile("yield" ::: "memory");
}
#endif /* __ASSEMBLY__ */
#endif /* __ASM_VDSO_PROCESSOR_H */

53
arch/arm64/include/asm/vdso/vsyscall.h Normal file
View file

@ -0,0 +1,53 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __ASM_VDSO_VSYSCALL_H
#define __ASM_VDSO_VSYSCALL_H
#ifndef __ASSEMBLY__
#include <linux/timekeeper_internal.h>
#include <vdso/datapage.h>
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
extern struct vdso_data *vdso_data;
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
static __always_inline
struct vdso_data *__arm64_get_k_vdso_data(void)
{
return vdso_data;
}
#define __arch_get_k_vdso_data __arm64_get_k_vdso_data
static __always_inline
int __arm64_get_clock_mode(struct timekeeper *tk)
{
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
return use_syscall;
}
#define __arch_get_clock_mode __arm64_get_clock_mode
static __always_inline
int __arm64_use_vsyscall(struct vdso_data *vdata)
{
return !vdata[CS_HRES_COARSE].clock_mode;
}
#define __arch_use_vsyscall __arm64_use_vsyscall
static __always_inline
void __arm64_update_vsyscall(struct vdso_data *vdata, struct timekeeper *tk)
{
vdata[CS_HRES_COARSE].mask = VDSO_PRECISION_MASK;
vdata[CS_RAW].mask = VDSO_PRECISION_MASK;
}
#define __arch_update_vsyscall __arm64_update_vsyscall
/* The asm-generic header needs to be included after the definitions above */
#include <asm-generic/vdso/vsyscall.h>
#endif /* !__ASSEMBLY__ */
#endif /* __ASM_VDSO_VSYSCALL_H */

48
arch/arm64/include/asm/vdso_datapage.h
View file

@ -1,48 +0,0 @@
/*
* Copyright (C) 2012 ARM Limited
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef __ASM_VDSO_DATAPAGE_H
#define __ASM_VDSO_DATAPAGE_H
#ifdef __KERNEL__
#ifndef __ASSEMBLY__
struct vdso_data {
__u64 cs_cycle_last; /* Timebase at clocksource init */
__u64 raw_time_sec; /* Raw time */
__u64 raw_time_nsec;
__u64 xtime_clock_sec; /* Kernel time */
__u64 xtime_clock_nsec;
__u64 xtime_coarse_sec; /* Coarse time */
__u64 xtime_coarse_nsec;
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
__u32 tb_seq_count; /* Timebase sequence counter */
/* cs_* members must be adjacent and in this order (ldp accesses) */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
__u32 cs_raw_mult; /* Raw clocksource multiplier */
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
__u32 hrtimer_res;
};
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
#endif /* __ASM_VDSO_DATAPAGE_H */

2
arch/arm64/include/uapi/asm/setup.h
View file

@ -22,6 +22,6 @@
#include <linux/types.h>
#define COMMAND_LINE_SIZE 2048
#define COMMAND_LINE_SIZE 4096
#endif

7
arch/arm64/kernel/Makefile
View file

@ -27,8 +27,12 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sys_compat.o
ifneq ($(CONFIG_COMPAT_VDSO), y)
arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
@ -61,6 +65,7 @@ arm64-obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o
obj-y += $(arm64-obj-y) vdso/ probes/
obj-m += $(arm64-obj-m)
obj-$(CONFIG_COMPAT_VDSO) += vdso32/
head-y := head.o
extra-y += $(head-y) vmlinux.lds

13
arch/arm64/kernel/arm64ksyms.c
View file

@ -29,6 +29,7 @@
#include <linux/arm-smccc.h>
#include <linux/kprobes.h>
#include <asm/cacheflush.h>
#include <asm/checksum.h>
EXPORT_SYMBOL(copy_page);
@ -86,3 +87,15 @@ extern long long __ashrti3(long long a, int b);
EXPORT_SYMBOL(__ashrti3);
extern long long __lshrti3(long long a, int b);
EXPORT_SYMBOL(__lshrti3);
/* caching functions */
EXPORT_SYMBOL_GPL(__dma_inv_area);
EXPORT_SYMBOL_GPL(__dma_clean_area);
EXPORT_SYMBOL_GPL(__dma_flush_area);
EXPORT_SYMBOL_GPL(__flush_dcache_area);
EXPORT_SYMBOL_GPL(__bss_stop);
EXPORT_SYMBOL_GPL(__per_cpu_start);
EXPORT_SYMBOL_GPL(__per_cpu_end);
EXPORT_SYMBOL_GPL(_sdata);
EXPORT_SYMBOL_GPL(cpu_do_idle);

2
arch/arm64/kernel/armv8_deprecated.c
View file

@ -604,7 +604,7 @@ static struct undef_hook setend_hooks[] = {
},
{
/* Thumb mode */
.instr_mask = 0x0000fff7,
.instr_mask = 0xfffffff7,
.instr_val = 0x0000b650,
.pstate_mask = (PSR_AA32_T_BIT | PSR_AA32_MODE_MASK),
.pstate_val = (PSR_AA32_T_BIT | PSR_AA32_MODE_USR),

42
arch/arm64/kernel/asm-offsets.c
View file

@ -25,13 +25,14 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@ -84,6 +85,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
#ifdef CONFIG_COMPAT
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.uc_mcontext.arm_r0));
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.uc_mcontext.arm_r0));
BLANK();
#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
@ -108,22 +114,28 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
DEFINE(VDSO_MASK, offsetof(struct vdso_data, mask));
DEFINE(VDSO_MULT, offsetof(struct vdso_data, mult));
DEFINE(VDSO_SHIFT, offsetof(struct vdso_data, shift));
DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));

22
arch/arm64/kernel/cpu_errata.c
View file

@ -643,6 +643,18 @@ needs_tx2_tvm_workaround(const struct arm64_cpu_capabilities *entry,
return false;
}
static bool __maybe_unused
has_neoverse_n1_erratum_1542419(const struct arm64_cpu_capabilities *entry,
int scope)
{
u32 midr = read_cpuid_id();
bool has_dic = read_cpuid_cachetype() & BIT(CTR_DIC_SHIFT);
const struct midr_range range = MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1);
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
return is_midr_in_range(midr, &range) && has_dic;
}
#ifdef CONFIG_HARDEN_EL2_VECTORS
static const struct midr_range arm64_harden_el2_vectors[] = {
@ -834,6 +846,16 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
ERRATA_MIDR_RANGE_LIST(tx2_family_cpus),
.matches = needs_tx2_tvm_workaround,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_1542419
{
/* we depend on the firmware portion for correctness */
.desc = "ARM erratum 1542419 (kernel portion)",
.capability = ARM64_WORKAROUND_1542419,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.matches = has_neoverse_n1_erratum_1542419,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#endif
{
}

1
arch/arm64/kernel/cpuidle.c
View file

@ -42,6 +42,7 @@ int arm_cpuidle_suspend(int index)
return cpu_ops[cpu]->cpu_suspend(index);
}
EXPORT_SYMBOL_GPL(arm_cpuidle_suspend);
#ifdef CONFIG_ACPI

2
arch/arm64/kernel/head.S
View file

@ -674,7 +674,7 @@ ENTRY(__boot_cpu_mode)
* with MMU turned off.
*/
ENTRY(__early_cpu_boot_status)
.long 0
.quad 0
.popsection

66
arch/arm64/kernel/kuser32.S
View file

@ -1,29 +1,14 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Low-level user helpers placed in the vectors page for AArch32.
* AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012 ARM Ltd.
* Copyright (C) 2012-2018 ARM Ltd.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
*
* AArch32 user helpers.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
* existing ones. This mechanism should be used only for things that are
* really small and justified, and not be abused freely.
* The kuser helpers below are mapped at a fixed address by
* aarch32_setup_additional_pages() and are provided for compatibility
* reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
/*
* AArch32 sigreturn code
*
* For ARM syscalls, the syscall number has to be loaded into r7.
* We do not support an OABI userspace.
*
* For Thumb syscalls, we also pass the syscall number via r7. We therefore
* need two 16-bit instructions.
*/
.globl __aarch32_sigret_code_start
__aarch32_sigret_code_start:
/*
* ARM Code
*/
.byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
.byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
/*
* Thumb code
*/
.byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
.byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
/*
* ARM code
*/
.byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
.byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
/*
* Thumb code
*/
.byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
.byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
.globl __aarch32_sigret_code_end
__aarch32_sigret_code_end:

1
arch/arm64/kernel/process.c
View file

@ -75,6 +75,7 @@ void (*pm_power_off)(void);
EXPORT_SYMBOL_GPL(pm_power_off);
void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd);
EXPORT_SYMBOL_GPL(arm_pm_restart);
/*
* This is our default idle handler.

8
arch/arm64/kernel/setup.c
View file

@ -69,6 +69,14 @@ static struct resource *standard_resources;
phys_addr_t __fdt_pointer __initdata;
/* Vendor stub */
unsigned int boot_reason;
EXPORT_SYMBOL_GPL(boot_reason);
/* Vendor stub */
unsigned int cold_boot;
EXPORT_SYMBOL_GPL(cold_boot);
/*
* Standard memory resources
*/

75
arch/arm64/kernel/signal32.c
View file

@ -29,42 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
struct compat_sigcontext {
/* We always set these two fields to 0 */
compat_ulong_t trap_no;
compat_ulong_t error_code;
compat_ulong_t oldmask;
compat_ulong_t arm_r0;
compat_ulong_t arm_r1;
compat_ulong_t arm_r2;
compat_ulong_t arm_r3;
compat_ulong_t arm_r4;
compat_ulong_t arm_r5;
compat_ulong_t arm_r6;
compat_ulong_t arm_r7;
compat_ulong_t arm_r8;
compat_ulong_t arm_r9;
compat_ulong_t arm_r10;
compat_ulong_t arm_fp;
compat_ulong_t arm_ip;
compat_ulong_t arm_sp;
compat_ulong_t arm_lr;
compat_ulong_t arm_pc;
compat_ulong_t arm_cpsr;
compat_ulong_t fault_address;
};
struct compat_ucontext {
compat_ulong_t uc_flags;
compat_uptr_t uc_link;
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@ -92,16 +57,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
struct compat_sigframe {
struct compat_ucontext uc;
compat_ulong_t retcode[2];
};
struct compat_rt_sigframe {
struct compat_siginfo info;
struct compat_sigframe sig;
};
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
@ -398,14 +353,38 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
#ifdef CONFIG_COMPAT_VDSO
void *vdso_base = current->mm->context.vdso;
void *vdso_trampoline;
if (ka->sa.sa_flags & SA_SIGINFO) {
if (thumb) {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_rt_sigreturn_thumb);
} else {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_rt_sigreturn_arm);
}
} else {
if (thumb) {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_sigreturn_thumb);
} else {
vdso_trampoline = VDSO_SYMBOL(vdso_base,
compat_sigreturn_arm);
}
}
retcode = ptr_to_compat(vdso_trampoline) + thumb;
#else
unsigned int idx = thumb << 1;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = AARCH32_VECTORS_BASE +
AARCH32_KERN_SIGRET_CODE_OFFSET +
retcode = (unsigned long)current->mm->context.vdso +
(idx << 2) + thumb;
#endif
}
regs->regs[0] = usig;

46
arch/arm64/kernel/sigreturn32.S Normal file
View file

@ -0,0 +1,46 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* AArch32 sigreturn code.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
* For ARM syscalls, the syscall number has to be loaded into r7.
* We do not support an OABI userspace.
*
* For Thumb syscalls, we also pass the syscall number via r7. We therefore
* need two 16-bit instructions.
*/
#include <asm/unistd.h>
.globl __aarch32_sigret_code_start
__aarch32_sigret_code_start:
/*
* ARM Code
*/
.byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
.byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
/*
* Thumb code
*/
.byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
.byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
/*
* ARM code
*/
.byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
.byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
/*
* Thumb code
*/
.byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
.byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
.globl __aarch32_sigret_code_end
__aarch32_sigret_code_end:

3
arch/arm64/kernel/smp.c
View file

@ -597,6 +597,9 @@ static void __init acpi_parse_and_init_cpus(void)
#else
#define acpi_parse_and_init_cpus(...) do { } while (0)
#endif
/* Dummy vendor field */
DEFINE_PER_CPU(bool, pending_ipi);
EXPORT_SYMBOL_GPL(pending_ipi);
static void (*__smp_update_ipi_history_cb)(int cpu);
/*

11
arch/arm64/kernel/sys_compat.c
View file

@ -19,6 +19,7 @@
*/
#include <linux/compat.h>
#include <linux/cpufeature.h>
#include <linux/personality.h>
#include <linux/sched.h>
#include <linux/sched/signal.h>
@ -28,6 +29,7 @@
#include <asm/cacheflush.h>
#include <asm/system_misc.h>
#include <asm/tlbflush.h>
#include <asm/unistd.h>
static long
@ -41,6 +43,15 @@ __do_compat_cache_op(unsigned long start, unsigned long end)
if (fatal_signal_pending(current))
return 0;
if (cpus_have_const_cap(ARM64_WORKAROUND_1542419)) {
/*
* The workaround requires an inner-shareable tlbi.
* We pick the reserved-ASID to minimise the impact.
*/
__tlbi(aside1is, __TLBI_VADDR(0, 0));
dsb(ish);
}
ret = __flush_cache_user_range(start, start + chunk);
if (ret)
return ret;

9
arch/arm64/kernel/traps.c
View file

@ -482,6 +482,15 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs)
int rt = (esr & ESR_ELx_SYS64_ISS_RT_MASK) >> ESR_ELx_SYS64_ISS_RT_SHIFT;
unsigned long val = arm64_ftr_reg_user_value(&arm64_ftr_reg_ctrel0);
if (cpus_have_const_cap(ARM64_WORKAROUND_1542419)) {
/* Hide DIC so that we can trap the unnecessary maintenance...*/
val &= ~BIT(CTR_DIC_SHIFT);
/* ... and fake IminLine to reduce the number of traps. */
val &= ~CTR_IMINLINE_MASK;
val |= (PAGE_SHIFT - 2) & CTR_IMINLINE_MASK;
}
pt_regs_write_reg(regs, rt, val);
arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);

532
arch/arm64/kernel/vdso.c
View file

@ -1,5 +1,5 @@
/*
* VDSO implementation for AArch64 and vector page setup for AArch32.
* VDSO implementations.
*
* Copyright (C) 2012 ARM Limited
*
@ -31,90 +31,74 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
#include <vdso/datapage.h>
#include <vdso/helpers.h>
#include <vdso/vsyscall.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
#ifdef CONFIG_COMPAT_VDSO
extern char vdso32_start[], vdso32_end[];
#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
#ifdef CONFIG_COMPAT_VDSO
ARM64_VDSO32 = 1,
#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_COMPAT_VDSO
#define VDSO_TYPES (ARM64_VDSO32 + 1)
#else
#define VDSO_TYPES (ARM64_VDSO + 1)
#endif /* CONFIG_COMPAT_VDSO */
struct __vdso_abi {
const char *name;
const char *vdso_code_start;
const char *vdso_code_end;
unsigned long vdso_pages;
/* Data Mapping */
struct vm_special_mapping *dm;
/* Code Mapping */
struct vm_special_mapping *cm;
};
static struct __vdso_abi vdso_lookup[VDSO_TYPES] __ro_after_init = {
{
.name = "vdso",
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
#ifdef CONFIG_COMPAT_VDSO
{
.name = "vdso32",
.vdso_code_start = vdso32_start,
.vdso_code_end = vdso32_end,
},
#endif /* CONFIG_COMPAT_VDSO */
};
/*
* The vDSO data page.
*/
static union {
struct vdso_data data;
struct vdso_data data[CS_BASES];
u8 page[PAGE_SIZE];
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
struct vdso_data *vdso_data = vdso_data_store.data;
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
*/
static struct page *vectors_page[1] __ro_after_init;
static int __init alloc_vectors_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
unsigned long vpage;
vpage = get_zeroed_page(GFP_ATOMIC);
if (!vpage)
return -ENOMEM;
/* kuser helpers */
memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
kuser_sz);
/* sigreturn code */
memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
__aarch32_sigret_code_start, sigret_sz);
flush_icache_range(vpage, vpage + PAGE_SIZE);
vectors_page[0] = virt_to_page(vpage);
return 0;
}
arch_initcall(alloc_vectors_page);
int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
{
struct mm_struct *mm = current->mm;
unsigned long addr = AARCH32_VECTORS_BASE;
static const struct vm_special_mapping spec = {
.name = "[vectors]",
.pages = vectors_page,
};
void *ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
current->mm->context.vdso = (void *)addr;
/* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
&spec);
up_write(&mm->mmap_sem);
return PTR_ERR_OR_ZERO(ret);
}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
static int __vdso_remap(enum arch_vdso_type arch_index,
const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
unsigned long vdso_size = vdso_end - vdso_start;
unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
vdso_lookup[arch_index].vdso_code_start;
if (vdso_size != new_size)
return -EINVAL;
@ -124,7 +108,303 @@ static int vdso_mremap(const struct vm_special_mapping *sm,
return 0;
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __vdso_init(enum arch_vdso_type arch_index)
{
int i;
struct page **vdso_pagelist;
unsigned long pfn;
if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
pr_err("vDSO is not a valid ELF object!\n");
return -EINVAL;
}
vdso_lookup[arch_index].vdso_pages = (
vdso_lookup[arch_index].vdso_code_end -
vdso_lookup[arch_index].vdso_code_start) >>
PAGE_SHIFT;
/* Allocate the vDSO pagelist, plus a page for the data. */
vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
sizeof(struct page *),
GFP_KERNEL);
if (vdso_pagelist == NULL)
return -ENOMEM;
/* Grab the vDSO data page. */
vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
/* Grab the vDSO code pages. */
pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
return 0;
}
static int __setup_additional_pages(enum arch_vdso_type arch_index,
struct mm_struct *mm,
struct linux_binprm *bprm,
int uses_interp)
{
unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
void *ret;
vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
/* Be sure to map the data page */
vdso_mapping_len = vdso_text_len + PAGE_SIZE;
vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
if (IS_ERR_VALUE(vdso_base)) {
ret = ERR_PTR(vdso_base);
goto up_fail;
}
ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
VM_READ|VM_MAYREAD,
vdso_lookup[arch_index].dm);
if (IS_ERR(ret))
goto up_fail;
vdso_base += PAGE_SIZE;
mm->context.vdso = (void *)vdso_base;
ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
VM_READ|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
vdso_lookup[arch_index].cm);
if (IS_ERR(ret))
goto up_fail;
return 0;
up_fail:
mm->context.vdso = NULL;
return PTR_ERR(ret);
}
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
*/
#ifdef CONFIG_COMPAT_VDSO
static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
return __vdso_remap(ARM64_VDSO32, sm, new_vma);
}
#endif /* CONFIG_COMPAT_VDSO */
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
* or (CONFIG_COMPAT_VDSO):
* 0 - kuser helpers
* 1 - vdso data
* 2 - vdso code
*/
#define C_VECTORS 0
#ifdef CONFIG_COMPAT_VDSO
#define C_VVAR 1
#define C_VDSO 2
#define C_PAGES (C_VDSO + 1)
#else
#define C_SIGPAGE 1
#define C_PAGES (C_SIGPAGE + 1)
#endif /* CONFIG_COMPAT_VDSO */
static struct page *aarch32_vdso_pages[C_PAGES] __ro_after_init;
static struct vm_special_mapping aarch32_vdso_spec[C_PAGES] = {
{
.name = "[vectors]", /* ABI */
.pages = &aarch32_vdso_pages[C_VECTORS],
},
#ifdef CONFIG_COMPAT_VDSO
{
.name = "[vvar]",
},
{
.name = "[vdso]",
.mremap = aarch32_vdso_mremap,
},
#else
{
.name = "[sigpage]", /* ABI */
.pages = &aarch32_vdso_pages[C_SIGPAGE],
},
#endif /* CONFIG_COMPAT_VDSO */
};
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
unsigned long vdso_page;
if (!IS_ENABLED(CONFIG_KUSER_HELPERS))
return 0;
vdso_page = get_zeroed_page(GFP_ATOMIC);
if (!vdso_page)
return -ENOMEM;
memcpy((void *)(vdso_page + 0x1000 - kuser_sz), __kuser_helper_start,
kuser_sz);
aarch32_vdso_pages[C_VECTORS] = virt_to_page(vdso_page);
flush_dcache_page(aarch32_vdso_pages[C_VECTORS]);
return 0;
}
#ifdef CONFIG_COMPAT_VDSO
static int __aarch32_alloc_vdso_pages(void)
{
int ret;
vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[C_VVAR];
vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[C_VDSO];
ret = __vdso_init(ARM64_VDSO32);
if (ret)
return ret;
ret = aarch32_alloc_kuser_vdso_page();
if (ret) {
unsigned long c_vvar =
(unsigned long)page_to_virt(aarch32_vdso_pages[C_VVAR]);
unsigned long c_vdso =
(unsigned long)page_to_virt(aarch32_vdso_pages[C_VDSO]);
free_page(c_vvar);
free_page(c_vdso);
}
return ret;
}
#else
static int __aarch32_alloc_vdso_pages(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
unsigned long sigpage;
int ret;
sigpage = get_zeroed_page(GFP_ATOMIC);
if (!sigpage)
return -ENOMEM;
memcpy((void *)sigpage, __aarch32_sigret_code_start, sigret_sz);
aarch32_vdso_pages[C_SIGPAGE] = virt_to_page(sigpage);
flush_dcache_page(aarch32_vdso_pages[C_SIGPAGE]);
ret = aarch32_alloc_kuser_vdso_page();
if (ret)
free_page(sigpage);
return ret;
}
#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return __aarch32_alloc_vdso_pages();
}
arch_initcall(aarch32_alloc_vdso_pages);
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
if (!IS_ENABLED(CONFIG_KUSER_HELPERS))
return 0;
/*
* Avoid VM_MAYWRITE for compatibility with arch/arm/, where it's
* not safe to CoW the page containing the CPU exception vectors.
*/
ret = _install_special_mapping(mm, AARCH32_VECTORS_BASE, PAGE_SIZE,
VM_READ | VM_EXEC |
VM_MAYREAD | VM_MAYEXEC,
&aarch32_vdso_spec[C_VECTORS]);
return PTR_ERR_OR_ZERO(ret);
}
#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
void *ret;
addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
if (IS_ERR_VALUE(addr)) {
ret = ERR_PTR(addr);
goto out;
}
/*
* VM_MAYWRITE is required to allow gdb to Copy-on-Write and
* set breakpoints.
*/
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
VM_READ | VM_EXEC | VM_MAYREAD |
VM_MAYWRITE | VM_MAYEXEC,
&aarch32_vdso_spec[C_SIGPAGE]);
if (IS_ERR(ret))
goto out;
mm->context.vdso = (void *)addr;
out:
return PTR_ERR_OR_ZERO(ret);
}
#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
struct mm_struct *mm = current->mm;
int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
ret = aarch32_kuser_helpers_setup(mm);
if (ret)
goto out;
#ifdef CONFIG_COMPAT_VDSO
ret = __setup_additional_pages(ARM64_VDSO32,
mm,
bprm,
uses_interp);
#else
ret = aarch32_sigreturn_setup(mm);
#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
return ret;
}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
/*
* aarch64_vdso_pages:
* 0 - vvar
* 1 - vdso
*/
#define A_VVAR 0
#define A_VDSO 1
#define A_PAGES (A_VDSO + 1)
static struct vm_special_mapping vdso_spec[A_PAGES] __ro_after_init = {
{
.name = "[vvar]",
},
@ -136,37 +416,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
int i;
struct page **vdso_pagelist;
unsigned long pfn;
vdso_lookup[ARM64_VDSO].dm = &vdso_spec[A_VVAR];
vdso_lookup[ARM64_VDSO].cm = &vdso_spec[A_VDSO];
if (memcmp(vdso_start, "\177ELF", 4)) {
pr_err("vDSO is not a valid ELF object!\n");
return -EINVAL;
}
vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
/* Allocate the vDSO pagelist, plus a page for the data. */
vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
GFP_KERNEL);
if (vdso_pagelist == NULL)
return -ENOMEM;
/* Grab the vDSO data page. */
vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
/* Grab the vDSO code pages. */
pfn = sym_to_pfn(vdso_start);
for (i = 0; i < vdso_pages; i++)
vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
vdso_spec[0].pages = &vdso_pagelist[0];
vdso_spec[1].pages = &vdso_pagelist[1];
return 0;
return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@ -174,84 +427,17 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
void *ret;
vdso_text_len = vdso_pages << PAGE_SHIFT;
/* Be sure to map the data page */
vdso_mapping_len = vdso_text_len + PAGE_SIZE;
int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
if (IS_ERR_VALUE(vdso_base)) {
ret = ERR_PTR(vdso_base);
goto up_fail;
}
ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
VM_READ|VM_MAYREAD,
&vdso_spec[0]);
if (IS_ERR(ret))
goto up_fail;
vdso_base += PAGE_SIZE;
mm->context.vdso = (void *)vdso_base;
ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
VM_READ|VM_EXEC|
VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
&vdso_spec[1]);
if (IS_ERR(ret))
goto up_fail;
ret = __setup_additional_pages(ARM64_VDSO,
mm,
bprm,
uses_interp);
up_write(&mm->mmap_sem);
return 0;
up_fail:
mm->context.vdso = NULL;
up_write(&mm->mmap_sem);
return PTR_ERR(ret);
}
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
++vdso_data->tb_seq_count;
smp_wmb();
vdso_data->use_syscall = use_syscall;
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
tk->tkr_mono.shift;
vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
/* Read without the seqlock held by clock_getres() */
WRITE_ONCE(vdso_data->hrtimer_res, hrtimer_resolution);
if (!use_syscall) {
/* tkr_mono.cycle_last == tkr_raw.cycle_last */
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
vdso_data->raw_time_sec = tk->raw_sec;
vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
vdso_data->xtime_clock_sec = tk->xtime_sec;
vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
vdso_data->cs_mono_mult = tk->tkr_mono.mult;
vdso_data->cs_raw_mult = tk->tkr_raw.mult;
/* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
}
smp_wmb();
++vdso_data->tb_seq_count;
}
void update_vsyscall_tz(void)
{
vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
vdso_data->tz_dsttime = sys_tz.tz_dsttime;
return ret;
}

52
arch/arm64/kernel/vdso/Makefile
View file

@ -6,26 +6,51 @@
# Heavily based on the vDSO Makefiles for other archs.
#
obj-vdso := gettimeofday.o note.o sigreturn.o
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
include $(srctree)/lib/vdso/Makefile
obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
ccflags-y := -shared -fno-common -fno-builtin
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
ldflags-y := -shared -nostdlib -soname=linux-vdso.so.1 --hash-style=sysv \
--build-id -n -T
ccflags-y += $(DISABLE_LTO)
CFLAGS_REMOVE_vgettimeofday.o += $(CC_FLAGS_SCS)
ccflags-y := -fno-common -fno-builtin -fno-stack-protector
ccflags-y += -DDISABLE_BRANCH_PROFILING
VDSO_LDFLAGS := -Bsymbolic
CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
KCOV_INSTRUMENT := n
CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
ifneq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y)
endif
# Clang versions less than 8 do not support -mcmodel=tiny
ifeq ($(CONFIG_CC_IS_CLANG), y)
ifeq ($(shell test $(CONFIG_CLANG_VERSION) -lt 80000; echo $$?),0)
CFLAGS_REMOVE_vgettimeofday.o += -mcmodel=tiny
endif
endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
# Workaround for bare-metal (ELF) toolchains that neglect to pass -shared
# down to collect2, resulting in silent corruption of the vDSO image.
ccflags-y += -Wl,-shared
obj-y += vdso.o
extra-y += vdso.lds
CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
@ -35,7 +60,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
$(call if_changed,vdsold_and_vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@ -52,16 +77,13 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
# Assembly rules for the .S files
$(obj-vdso): %.o: %.S FORCE
$(call if_changed_dep,vdsoas)
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
quiet_cmd_vdsoas = VDSOA $@
cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
quiet_cmd_vdsold_and_vdso_check = LD $@
cmd_vdsold_and_vdso_check = $(cmd_ld); $(cmd_vdso_check)
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/$@

334
arch/arm64/kernel/vdso/gettimeofday.S
View file

@ -1,334 +0,0 @@
/*
* Userspace implementations of gettimeofday() and friends.
*
* Copyright (C) 2012 ARM Limited
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* Author: Will Deacon <will.deacon@arm.com>
*/
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#define NSEC_PER_SEC_LO16 0xca00
#define NSEC_PER_SEC_HI16 0x3b9a
vdso_data .req x6
seqcnt .req w7
w_tmp .req w8
x_tmp .req x8
/*
* Conventions for macro arguments:
* - An argument is write-only if its name starts with "res".
* - All other arguments are read-only, unless otherwise specified.
*/
.macro seqcnt_acquire
9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
tbnz seqcnt, #0, 9999b
dmb ishld
.endm
.macro seqcnt_check fail
dmb ishld
ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
cmp w_tmp, seqcnt
b.ne \fail
.endm
.macro syscall_check fail
ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
cbnz w_tmp, \fail
.endm
.macro get_nsec_per_sec res
mov \res, #NSEC_PER_SEC_LO16
movk \res, #NSEC_PER_SEC_HI16, lsl #16
.endm
/*
* Returns the clock delta, in nanoseconds left-shifted by the clock
* shift.
*/
.macro get_clock_shifted_nsec res, cycle_last, mult
/* Read the virtual counter. */
isb
mrs x_tmp, cntvct_el0
/* Calculate cycle delta and convert to ns. */
sub \res, x_tmp, \cycle_last
/* We can only guarantee 56 bits of precision. */
movn x_tmp, #0xff00, lsl #48
and \res, x_tmp, \res
mul \res, \res, \mult
/*
* Fake address dependency from the value computed from the counter
* register to subsequent data page accesses so that the sequence
* locking also orders the read of the counter.
*/
and x_tmp, \res, xzr
add vdso_data, vdso_data, x_tmp
.endm
/*
* Returns in res_{sec,nsec} the REALTIME timespec, based on the
* "wall time" (xtime) and the clock_mono delta.
*/
.macro get_ts_realtime res_sec, res_nsec, \
clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
add \res_nsec, \clock_nsec, \xtime_nsec
udiv x_tmp, \res_nsec, \nsec_to_sec
add \res_sec, \xtime_sec, x_tmp
msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
.endm
/*
* Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
* used for CLOCK_MONOTONIC_RAW.
*/
.macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
udiv \res_sec, \clock_nsec, \nsec_to_sec
msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
.endm
/* sec and nsec are modified in place. */
.macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
/* Add timespec. */
add \sec, \sec, \ts_sec
add \nsec, \nsec, \ts_nsec
/* Normalise the new timespec. */
cmp \nsec, \nsec_to_sec
b.lt 9999f
sub \nsec, \nsec, \nsec_to_sec
add \sec, \sec, #1
9999:
cmp \nsec, #0
b.ge 9998f
add \nsec, \nsec, \nsec_to_sec
sub \sec, \sec, #1
9998:
.endm
.macro clock_gettime_return, shift=0
.if \shift == 1
lsr x11, x11, x12
.endif
stp x10, x11, [x1, #TSPEC_TV_SEC]
mov x0, xzr
ret
.endm
.macro jump_slot jumptable, index, label
.if (. - \jumptable) != 4 * (\index)
.error "Jump slot index mismatch"
.endif
b \label
.endm
.text
/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
ENTRY(__kernel_gettimeofday)
.cfi_startproc
adr vdso_data, _vdso_data
/* If tv is NULL, skip to the timezone code. */
cbz x0, 2f
/* Compute the time of day. */
1: seqcnt_acquire
syscall_check fail=4f
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=1b
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
/* Convert ns to us. */
mov x13, #1000
lsl x13, x13, x12
udiv x11, x11, x13
stp x10, x11, [x0, #TVAL_TV_SEC]
2:
/* If tz is NULL, return 0. */
cbz x1, 3f
ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
stp w4, w5, [x1, #TZ_MINWEST]
3:
mov x0, xzr
ret
4:
/* Syscall fallback. */
mov x8, #__NR_gettimeofday
svc #0
ret
.cfi_endproc
ENDPROC(__kernel_gettimeofday)
#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
ENTRY(__kernel_clock_gettime)
.cfi_startproc
cmp w0, #JUMPSLOT_MAX
b.hi syscall
adr vdso_data, _vdso_data
adr x_tmp, jumptable
add x_tmp, x_tmp, w0, uxtw #2
br x_tmp
ALIGN
jumptable:
jump_slot jumptable, CLOCK_REALTIME, realtime
jump_slot jumptable, CLOCK_MONOTONIC, monotonic
b syscall
b syscall
jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
.if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
.error "Wrong jumptable size"
.endif
ALIGN
realtime:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
/* All computations are done with left-shifted nsecs. */
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=realtime
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
monotonic:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_mono_mult, w12 = cs_shift */
ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
/* All computations are done with left-shifted nsecs. */
lsl x4, x4, x12
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=monotonic
get_ts_realtime res_sec=x10, res_nsec=x11, \
clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
monotonic_raw:
seqcnt_acquire
syscall_check fail=syscall
ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
/* w11 = cs_raw_mult, w12 = cs_shift */
ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
/* All computations are done with left-shifted nsecs. */
get_nsec_per_sec res=x9
lsl x9, x9, x12
get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
seqcnt_check fail=monotonic_raw
get_ts_clock_raw res_sec=x10, res_nsec=x11, \
clock_nsec=x15, nsec_to_sec=x9
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return, shift=1
ALIGN
realtime_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
seqcnt_check fail=realtime_coarse
clock_gettime_return
ALIGN
monotonic_coarse:
seqcnt_acquire
ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
seqcnt_check fail=monotonic_coarse
/* Computations are done in (non-shifted) nsecs. */
get_nsec_per_sec res=x9
add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
clock_gettime_return
ALIGN
syscall: /* Syscall fallback. */
mov x8, #__NR_clock_gettime
svc #0
ret
.cfi_endproc
ENDPROC(__kernel_clock_gettime)
/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
ENTRY(__kernel_clock_getres)
.cfi_startproc
cmp w0, #CLOCK_REALTIME
ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
b.ne 1f
adr vdso_data, _vdso_data
ldr w2, [vdso_data, #CLOCK_REALTIME_RES]
b 2f
1:
cmp w0, #CLOCK_REALTIME_COARSE
ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
b.ne 4f
ldr x2, 5f
2:
cbz x1, 3f
stp xzr, x2, [x1]
3: /* res == NULL. */
mov w0, wzr
ret
4: /* Syscall fallback. */
mov x8, #__NR_clock_getres
svc #0
ret
5:
.quad CLOCK_COARSE_RES
.cfi_endproc
ENDPROC(__kernel_clock_getres)

25
arch/arm64/kernel/vdso/vgettimeofday.c Normal file
View file

@ -0,0 +1,25 @@
// SPDX-License-Identifier: GPL-2.0
/*
* ARM64 userspace implementations of gettimeofday() and similar.
*
* Copyright (C) 2018 ARM Limited
*
*/
int __kernel_clock_gettime(clockid_t clock,
struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
int __kernel_clock_getres(clockid_t clock_id,
struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}

2
arch/arm64/kernel/vdso32/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
vdso.lds
vdso.so.raw

211
arch/arm64/kernel/vdso32/Makefile Normal file
View file

@ -0,0 +1,211 @@
# SPDX-License-Identifier: GPL-2.0
#
# Makefile for vdso32
#
# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
# the inclusion of generic Makefile.
ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
include $(srctree)/lib/vdso/Makefile
# Same as cc-*option, but using CC_COMPAT instead of CC
ifeq ($(CONFIG_CC_IS_CLANG), y)
COMPAT_GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE_COMPAT)elfedit))
COMPAT_GCC_TOOLCHAIN := $(realpath $(COMPAT_GCC_TOOLCHAIN_DIR)/..)
CC_COMPAT_CLANG_FLAGS := --target=$(notdir $(CROSS_COMPILE_COMPAT:%-=%))
CC_COMPAT_CLANG_FLAGS += --prefix=$(COMPAT_GCC_TOOLCHAIN_DIR)
CC_COMPAT_CLANG_FLAGS += -no-integrated-as -Qunused-arguments
ifneq ($(COMPAT_GCC_TOOLCHAIN),)
CC_COMPAT_CLANG_FLAGS += --gcc-toolchain=$(COMPAT_GCC_TOOLCHAIN)
endif
CC_COMPAT ?= $(CC)
CC_COMPAT += $(CC_COMPAT_CLANG_FLAGS)
else
CC_COMPAT ?= $(CROSS_COMPILE_COMPAT)gcc
endif
cc32-option = $(call try-run,\
$(CC_COMPAT) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
cc32-disable-warning = $(call try-run,\
$(CC_COMPAT) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
cc32-ldoption = $(call try-run,\
$(CC_COMPAT) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
cc32-as-instr = $(call try-run,\
printf "%b\n" "$(1)" | $(CC_COMPAT) $(VDSO_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3))
# We cannot use the global flags to compile the vDSO files, the main reason
# being that the 32-bit compiler may be older than the main (64-bit) compiler
# and therefore may not understand flags set using $(cc-option ...). Besides,
# arch-specific options should be taken from the arm Makefile instead of the
# arm64 one.
# As a result we set our own flags here.
# KBUILD_CPPFLAGS and NOSTDINC_FLAGS from top-level Makefile
VDSO_CPPFLAGS := -D__KERNEL__ -nostdinc -isystem $(shell $(CC_COMPAT) -print-file-name=include)
VDSO_CPPFLAGS += $(LINUXINCLUDE)
# Common C and assembly flags
# From top-level Makefile
VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
ifneq ($(shell $(CC_COMPAT) --version 2>&1 | head -n 1 | grep clang),)
VDSO_CAFLAGS += --target=$(notdir $(CROSS_COMPILE_COMPAT:%-=%))
endif
VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
ifdef CONFIG_DEBUG_INFO
VDSO_CAFLAGS += -g
endif
# From arm Makefile
VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
VDSO_CAFLAGS += -mbig-endian
else
VDSO_CAFLAGS += -mlittle-endian
endif
# From arm vDSO Makefile
VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
# fall back to v7. There is no easy way to check for what architecture the code
# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
-march=armv7-a -D__LINUX_ARM_ARCH__=7)
VDSO_CFLAGS := $(VDSO_CAFLAGS)
VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
# KBUILD_CFLAGS from top-level Makefile
VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
-fno-strict-aliasing -fno-common \
-Werror-implicit-function-declaration \
-Wno-format-security \
-std=gnu89
VDSO_CFLAGS += -O2
# Some useful compiler-dependent flags from top-level Makefile
VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
# The 32-bit compiler does not provide 128-bit integers, which are used in
# some headers that are indirectly included from the vDSO code.
# This hack makes the compiler happy and should trigger a warning/error if
# variables of such type are referenced.
VDSO_CFLAGS += -D__uint128_t='void*'
# Silence some warnings coming from headers that operate on long's
# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
VDSO_CFLAGS += -Wno-int-to-pointer-cast
VDSO_AFLAGS := $(VDSO_CAFLAGS)
VDSO_AFLAGS += -D__ASSEMBLY__
# Check for binutils support for dmb ishld
dmbinstr := $(call cc32-as-instr,dmb ishld,-DCONFIG_AS_DMB_ISHLD=1)
VDSO_CFLAGS += $(dmbinstr)
VDSO_AFLAGS += $(dmbinstr)
VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
# From arm vDSO Makefile
VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
# Borrow vdsomunge.c from the arm vDSO
# We have to use a relative path because scripts/Makefile.host prefixes
# $(hostprogs-y) with $(obj)
munge := ../../../arm/vdso/vdsomunge
hostprogs-y := $(munge)
c-obj-vdso := note.o
c-obj-vdso-gettimeofday := vgettimeofday.o
asm-obj-vdso := sigreturn.o
ifneq ($(c-gettimeofday-y),)
VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
endif
VDSO_CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os
# Build rules
targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
obj-y += vdso.o
extra-y += vdso.lds
CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
# Force dependency (vdso.s includes vdso.so through incbin)
$(obj)/vdso.o: $(obj)/vdso.so
include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
# Strip rule for vdso.so
$(obj)/vdso.so: OBJCOPYFLAGS := -S
$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
$(call if_changed,objcopy)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
$(call if_changed,vdsomunge)
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold_and_vdso_check)
# Compilation rules for the vDSO sources
$(c-obj-vdso): %.o: %.c FORCE
$(call if_changed_dep,vdsocc)
$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
$(call if_changed_dep,vdsocc_gettimeofday)
$(asm-obj-vdso): %.o: %.S FORCE
$(call if_changed_dep,vdsoas)
# Actual build commands
quiet_cmd_vdsold_and_vdso_check = LD32 $@
cmd_vdsold_and_vdso_check = $(cmd_vdsold); $(cmd_vdso_check)
quiet_cmd_vdsold = LD32 $@
cmd_vdsold = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
-Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
quiet_cmd_vdsocc = CC32 $@
cmd_vdsocc = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
quiet_cmd_vdsocc_gettimeofday = CC32 $@
cmd_vdsocc_gettimeofday = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
quiet_cmd_vdsoas = AS32 $@
cmd_vdsoas = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
quiet_cmd_vdsomunge = MUNGE $@
cmd_vdsomunge = $(obj)/$(munge) $< $@
# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
quiet_cmd_vdsosym = VDSOSYM $@
# The AArch64 nm should be able to read an AArch32 binary
cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
vdso.so: $(obj)/vdso.so.dbg
@mkdir -p $(MODLIB)/vdso
$(call cmd,vdso_install)
vdso_install: vdso.so

15
arch/arm64/kernel/vdso32/note.c Normal file
View file

@ -0,0 +1,15 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2012-2018 ARM Limited
*
* This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
* Here we can supply some information useful to userland.
*/
#include <linux/uts.h>
#include <linux/version.h>
#include <linux/elfnote.h>
#include <linux/build-salt.h>
ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
BUILD_SALT;

62
arch/arm64/kernel/vdso32/sigreturn.S Normal file
View file

@ -0,0 +1,62 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* This file provides both A32 and T32 versions, in accordance with the
* arm sigreturn code.
*
* Copyright (C) 2018 ARM Limited
*/
#include <linux/linkage.h>
#include <asm/asm-offsets.h>
#include <asm/unistd.h>
#define ARM_ENTRY(name) \
ENTRY(name)
#define ARM_ENDPROC(name) \
.type name, %function; \
END(name)
.text
.arm
.fnstart
.save {r0-r15}
.pad #COMPAT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_sigreturn_arm)
mov r7, #__NR_compat_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_sigreturn_arm)
.fnstart
.save {r0-r15}
.pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_rt_sigreturn_arm)
mov r7, #__NR_compat_rt_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_rt_sigreturn_arm)
.thumb
.fnstart
.save {r0-r15}
.pad #COMPAT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_sigreturn_thumb)
mov r7, #__NR_compat_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_sigreturn_thumb)
.fnstart
.save {r0-r15}
.pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
nop
ARM_ENTRY(__kernel_rt_sigreturn_thumb)
mov r7, #__NR_compat_rt_sigreturn
svc #0
.fnend
ARM_ENDPROC(__kernel_rt_sigreturn_thumb)

19
arch/arm64/kernel/vdso32/vdso.S Normal file
View file

@ -0,0 +1,19 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Copyright (C) 2012 ARM Limited
*/
#include <linux/init.h>
#include <linux/linkage.h>
#include <linux/const.h>
#include <asm/page.h>
.globl vdso32_start, vdso32_end
.section .rodata
.balign PAGE_SIZE
vdso32_start:
.incbin "arch/arm64/kernel/vdso32/vdso.so"
.balign PAGE_SIZE
vdso32_end:
.previous

82
arch/arm64/kernel/vdso32/vdso.lds.S Normal file
View file

@ -0,0 +1,82 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Adapted from arm64 version.
*
* GNU linker script for the VDSO library.
* Heavily based on the vDSO linker scripts for other archs.
*
* Copyright (C) 2012-2018 ARM Limited
*/
#include <linux/const.h>
#include <asm/page.h>
#include <asm/vdso.h>
OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
OUTPUT_ARCH(arm)
SECTIONS
{
PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
. = VDSO_LBASE + SIZEOF_HEADERS;
.hash : { *(.hash) } :text
.gnu.hash : { *(.gnu.hash) }
.dynsym : { *(.dynsym) }
.dynstr : { *(.dynstr) }
.gnu.version : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.note : { *(.note.*) } :text :note
.dynamic : { *(.dynamic) } :text :dynamic
.rodata : { *(.rodata*) } :text
.text : { *(.text*) } :text =0xe7f001f2
.got : { *(.got) }
.rel.plt : { *(.rel.plt) }
/DISCARD/ : {
*(.note.GNU-stack)
*(.data .data.* .gnu.linkonce.d.* .sdata*)
*(.bss .sbss .dynbss .dynsbss)
}
}
/*
* We must supply the ELF program headers explicitly to get just one
* PT_LOAD segment, and set the flags explicitly to make segments read-only.
*/
PHDRS
{
text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
note PT_NOTE FLAGS(4); /* PF_R */
}
VERSION
{
LINUX_2.6 {
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
__kernel_sigreturn_arm;
__kernel_sigreturn_thumb;
__kernel_rt_sigreturn_arm;
__kernel_rt_sigreturn_thumb;
__vdso_clock_gettime64;
local: *;
};
}
/*
* Make the sigreturn code visible to the kernel.
*/
VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;

45
arch/arm64/kernel/vdso32/vgettimeofday.c Normal file
View file

@ -0,0 +1,45 @@
// SPDX-License-Identifier: GPL-2.0
/*
* ARM64 compat userspace implementations of gettimeofday() and similar.
*
* Copyright (C) 2018 ARM Limited
*
*/
int __vdso_clock_gettime(clockid_t clock,
struct old_timespec32 *ts)
{
return __cvdso_clock_gettime32(clock, ts);
}
int __vdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
int __vdso_clock_getres(clockid_t clock_id,
struct old_timespec32 *res)
{
return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
void __aeabi_unwind_cpp_pr0(void)
{
}
void __aeabi_unwind_cpp_pr1(void)
{
}
void __aeabi_unwind_cpp_pr2(void)
{
}

7
arch/arm64/kvm/guest.c
View file

@ -179,6 +179,13 @@ static int set_core_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
}
memcpy((u32 *)regs + off, valp, KVM_REG_SIZE(reg->id));
if (*vcpu_cpsr(vcpu) & PSR_MODE32_BIT) {
int i;
for (i = 0; i < 16; i++)
*vcpu_reg32(vcpu, i) = (u32)*vcpu_reg32(vcpu, i);
}
out:
return err;
}

4
arch/arm64/mm/cache.S
View file

@ -230,7 +230,7 @@ ENTRY(__inval_dcache_area)
* - start - virtual start address of region
* - size - size in question
*/
__dma_inv_area:
ENTRY(__dma_inv_area)
add x1, x1, x0
dcache_line_size x2, x3
sub x3, x2, #1
@ -269,7 +269,7 @@ ENTRY(__clean_dcache_area_poc)
* - start - virtual start address of region
* - size - size in question
*/
__dma_clean_area:
ENTRY(__dma_clean_area)
dcache_by_line_op cvac, sy, x0, x1, x2, x3
ret
ENDPIPROC(__clean_dcache_area_poc)

409
arch/arm64/mm/dma-mapping.c
View file

@ -26,26 +26,47 @@
#include <linux/genalloc.h>
#include <linux/dma-direct.h>
#include <linux/dma-contiguous.h>
#include <linux/mm.h>
#include <linux/iommu.h>
#include <linux/vmalloc.h>
#include <linux/swiotlb.h>
#include <linux/dma-removed.h>
#include <linux/pci.h>
#include <linux/io.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
#include <asm/dma-iommu.h>
#include <linux/of_address.h>
#include <linux/dma-mapping-fast.h>
static int swiotlb __ro_after_init;
static pgprot_t __get_dma_pgprot(unsigned long attrs, pgprot_t prot,
bool coherent)
{
if (!coherent || (attrs & DMA_ATTR_WRITE_COMBINE))
if (attrs & DMA_ATTR_STRONGLY_ORDERED)
return pgprot_noncached(prot);
else if (!coherent || (attrs & DMA_ATTR_WRITE_COMBINE))
return pgprot_writecombine(prot);
return prot;
}
static bool is_dma_coherent(struct device *dev, unsigned long attrs)
{
if (attrs & DMA_ATTR_FORCE_COHERENT)
return true;
else if (attrs & DMA_ATTR_FORCE_NON_COHERENT)
return false;
else if (is_device_dma_coherent(dev))
return true;
else
return false;
}
static struct gen_pool *atomic_pool __ro_after_init;
#define NO_KERNEL_MAPPING_DUMMY 0x2222
#define DEFAULT_DMA_COHERENT_POOL_SIZE SZ_256K
static size_t atomic_pool_size __initdata = DEFAULT_DMA_COHERENT_POOL_SIZE;
@ -93,13 +114,50 @@ static int __free_from_pool(void *start, size_t size)
return 1;
}
static int __dma_update_pte(pte_t *pte, pgtable_t token, unsigned long addr,
void *data)
{
struct page *page = virt_to_page(addr);
pgprot_t prot = *(pgprot_t *)data;
set_pte(pte, mk_pte(page, prot));
return 0;
}
static int __dma_clear_pte(pte_t *pte, pgtable_t token, unsigned long addr,
void *data)
{
pte_clear(&init_mm, addr, pte);
return 0;
}
static void __dma_remap(struct page *page, size_t size, pgprot_t prot,
bool no_kernel_map)
{
unsigned long start = (unsigned long) page_address(page);
unsigned long end = start + size;
int (*func)(pte_t *pte, pgtable_t token, unsigned long addr,
void *data);
if (no_kernel_map)
func = __dma_clear_pte;
else
func = __dma_update_pte;
apply_to_page_range(&init_mm, start, size, func, &prot);
/* ensure prot is applied before returning */
mb();
flush_tlb_kernel_range(start, end);
}
static void *__dma_alloc(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
unsigned long attrs)
{
struct page *page;
void *ptr, *coherent_ptr;
bool coherent = is_device_dma_coherent(dev);
bool coherent = is_dma_coherent(dev, attrs);
pgprot_t prot = __get_dma_pgprot(attrs, PAGE_KERNEL, false);
size = PAGE_ALIGN(size);
@ -115,6 +173,7 @@ static void *__dma_alloc(struct device *dev, size_t size,
}
ptr = swiotlb_alloc(dev, size, dma_handle, flags, attrs);
if (!ptr)
goto no_mem;
@ -122,19 +181,31 @@ static void *__dma_alloc(struct device *dev, size_t size,
if (coherent)
return ptr;
/* remove any dirty cache lines on the kernel alias */
__dma_flush_area(ptr, size);
/* create a coherent mapping */
page = virt_to_page(ptr);
coherent_ptr = dma_common_contiguous_remap(page, size, VM_USERMAP,
prot, __builtin_return_address(0));
if (!coherent_ptr)
goto no_map;
if (attrs & DMA_ATTR_NO_KERNEL_MAPPING) {
coherent_ptr = (void *)NO_KERNEL_MAPPING_DUMMY;
__dma_remap(virt_to_page(ptr), size, __pgprot(0), true);
} else {
if ((attrs & DMA_ATTR_STRONGLY_ORDERED))
__dma_remap(virt_to_page(ptr), size, __pgprot(0), true);
/* create a coherent mapping */
page = virt_to_page(ptr);
coherent_ptr = dma_common_contiguous_remap(
page, size, VM_USERMAP, prot,
__builtin_return_address(0));
if (!coherent_ptr)
goto no_map;
}
return coherent_ptr;
no_map:
if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) ||
(attrs & DMA_ATTR_STRONGLY_ORDERED))
__dma_remap(phys_to_page(dma_to_phys(dev, *dma_handle)),
size, PAGE_KERNEL, false);
swiotlb_free(dev, size, ptr, *dma_handle, attrs);
no_mem:
return NULL;
@ -148,11 +219,17 @@ static void __dma_free(struct device *dev, size_t size,
size = PAGE_ALIGN(size);
if (!is_device_dma_coherent(dev)) {
if (!is_dma_coherent(dev, attrs)) {
if (__free_from_pool(vaddr, size))
return;
vunmap(vaddr);
if (!(attrs & DMA_ATTR_NO_KERNEL_MAPPING))
vunmap(vaddr);
}
if ((attrs & DMA_ATTR_NO_KERNEL_MAPPING) ||
(attrs & DMA_ATTR_STRONGLY_ORDERED))
__dma_remap(phys_to_page(dma_to_phys(dev, dma_handle)),
size, PAGE_KERNEL, false);
swiotlb_free(dev, size, swiotlb_addr, dma_handle, attrs);
}
@ -164,7 +241,7 @@ static dma_addr_t __swiotlb_map_page(struct device *dev, struct page *page,
dma_addr_t dev_addr;
dev_addr = swiotlb_map_page(dev, page, offset, size, dir, attrs);
if (!is_device_dma_coherent(dev) &&
if (!is_dma_coherent(dev, attrs) &&
(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
__dma_map_area(phys_to_virt(dma_to_phys(dev, dev_addr)), size, dir);
@ -176,7 +253,7 @@ static void __swiotlb_unmap_page(struct device *dev, dma_addr_t dev_addr,
size_t size, enum dma_data_direction dir,
unsigned long attrs)
{
if (!is_device_dma_coherent(dev) &&
if (!is_dma_coherent(dev, attrs) &&
(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
__dma_unmap_area(phys_to_virt(dma_to_phys(dev, dev_addr)), size, dir);
swiotlb_unmap_page(dev, dev_addr, size, dir, attrs);
@ -190,7 +267,7 @@ static int __swiotlb_map_sg_attrs(struct device *dev, struct scatterlist *sgl,
int i, ret;
ret = swiotlb_map_sg_attrs(dev, sgl, nelems, dir, attrs);
if (!is_device_dma_coherent(dev) &&
if (!is_dma_coherent(dev, attrs) &&
(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
for_each_sg(sgl, sg, ret, i)
__dma_map_area(phys_to_virt(dma_to_phys(dev, sg->dma_address)),
@ -207,7 +284,7 @@ static void __swiotlb_unmap_sg_attrs(struct device *dev,
struct scatterlist *sg;
int i;
if (!is_device_dma_coherent(dev) &&
if (!is_dma_coherent(dev, attrs) &&
(attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
for_each_sg(sgl, sg, nelems, i)
__dma_unmap_area(phys_to_virt(dma_to_phys(dev, sg->dma_address)),
@ -284,11 +361,11 @@ static int __swiotlb_mmap(struct device *dev,
void *cpu_addr, dma_addr_t dma_addr, size_t size,
unsigned long attrs)
{
int ret;
int ret = -ENXIO;
unsigned long pfn = dma_to_phys(dev, dma_addr) >> PAGE_SHIFT;
vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot,
is_device_dma_coherent(dev));
is_dma_coherent(dev, attrs));
if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
return ret;
@ -447,7 +524,7 @@ static int __init atomic_pool_init(void)
goto out;
remove_mapping:
dma_common_free_remap(addr, atomic_pool_size, VM_USERMAP);
dma_common_free_remap(addr, atomic_pool_size, VM_USERMAP, false);
destroy_genpool:
gen_pool_destroy(atomic_pool);
atomic_pool = NULL;
@ -468,6 +545,7 @@ static void *__dummy_alloc(struct device *dev, size_t size,
dma_addr_t *dma_handle, gfp_t flags,
unsigned long attrs)
{
WARN(1, "dma alloc failure, device may be missing a call to arch_setup_dma_ops");
return NULL;
}
@ -582,7 +660,7 @@ static void *__iommu_alloc_attrs(struct device *dev, size_t size,
dma_addr_t *handle, gfp_t gfp,
unsigned long attrs)
{
bool coherent = is_device_dma_coherent(dev);
bool coherent = is_dma_coherent(dev, attrs);
int ioprot = dma_info_to_prot(DMA_BIDIRECTIONAL, coherent, attrs);
size_t iosize = size;
void *addr;
@ -596,7 +674,8 @@ static void *__iommu_alloc_attrs(struct device *dev, size_t size,
* Some drivers rely on this, and we probably don't want the
* possibility of stale kernel data being read by devices anyway.
*/
gfp |= __GFP_ZERO;
if (!(attrs & DMA_ATTR_SKIP_ZEROING))
gfp |= __GFP_ZERO;
if (!gfpflags_allow_blocking(gfp)) {
struct page *page;
@ -688,17 +767,16 @@ static void __iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr,
__free_from_pool(cpu_addr, size);
} else if (attrs & DMA_ATTR_FORCE_CONTIGUOUS) {
struct page *page = vmalloc_to_page(cpu_addr);
iommu_dma_unmap_page(dev, handle, iosize, 0, attrs);
dma_release_from_contiguous(dev, page, size >> PAGE_SHIFT);
dma_common_free_remap(cpu_addr, size, VM_USERMAP);
} else if (is_vmalloc_addr(cpu_addr)){
dma_common_free_remap(cpu_addr, size, VM_USERMAP, false);
} else if (is_vmalloc_addr(cpu_addr)) {
struct vm_struct *area = find_vm_area(cpu_addr);
if (WARN_ON(!area || !area->pages))
return;
iommu_dma_free(dev, area->pages, iosize, &handle);
dma_common_free_remap(cpu_addr, size, VM_USERMAP);
dma_common_free_remap(cpu_addr, size, VM_USERMAP, false);
} else {
iommu_dma_unmap_page(dev, handle, iosize, 0, 0);
__free_pages(virt_to_page(cpu_addr), get_order(size));
@ -711,32 +789,31 @@ static int __iommu_mmap_attrs(struct device *dev, struct vm_area_struct *vma,
{
struct vm_struct *area;
int ret;
unsigned long pfn = 0;
vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot,
is_device_dma_coherent(dev));
is_dma_coherent(dev, attrs));
if (dma_mmap_from_dev_coherent(dev, vma, cpu_addr, size, &ret))
return ret;
if (!is_vmalloc_addr(cpu_addr)) {
unsigned long pfn = page_to_pfn(virt_to_page(cpu_addr));
return __swiotlb_mmap_pfn(vma, pfn, size);
}
if (attrs & DMA_ATTR_FORCE_CONTIGUOUS) {
/*
* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped,
* hence in the vmalloc space.
*/
unsigned long pfn = vmalloc_to_pfn(cpu_addr);
return __swiotlb_mmap_pfn(vma, pfn, size);
}
area = find_vm_area(cpu_addr);
if (WARN_ON(!area || !area->pages))
return -ENXIO;
return iommu_dma_mmap(area->pages, size, vma);
if (area && area->pages)
return iommu_dma_mmap(area->pages, size, vma);
else if (!is_vmalloc_addr(cpu_addr))
pfn = page_to_pfn(virt_to_page(cpu_addr));
else if (is_vmalloc_addr(cpu_addr))
/*
* DMA_ATTR_FORCE_CONTIGUOUS and atomic pool allocations are
* always remapped, hence in the vmalloc space.
*/
pfn = vmalloc_to_pfn(cpu_addr);
if (pfn)
return __swiotlb_mmap_pfn(vma, pfn, size);
return -ENXIO;
}
static int __iommu_get_sgtable(struct device *dev, struct sg_table *sgt,
@ -744,27 +821,24 @@ static int __iommu_get_sgtable(struct device *dev, struct sg_table *sgt,
size_t size, unsigned long attrs)
{
unsigned int count = PAGE_ALIGN(size) >> PAGE_SHIFT;
struct page *page = NULL;
struct vm_struct *area = find_vm_area(cpu_addr);
if (!is_vmalloc_addr(cpu_addr)) {
struct page *page = virt_to_page(cpu_addr);
return __swiotlb_get_sgtable_page(sgt, page, size);
}
if (attrs & DMA_ATTR_FORCE_CONTIGUOUS) {
if (area && area->pages)
return sg_alloc_table_from_pages(sgt, area->pages, count, 0,
size, GFP_KERNEL);
else if (!is_vmalloc_addr(cpu_addr))
page = virt_to_page(cpu_addr);
else if (is_vmalloc_addr(cpu_addr))
/*
* DMA_ATTR_FORCE_CONTIGUOUS allocations are always remapped,
* hence in the vmalloc space.
* DMA_ATTR_FORCE_CONTIGUOUS and atomic pool allocations
* are always remapped, hence in the vmalloc space.
*/
struct page *page = vmalloc_to_page(cpu_addr);
page = vmalloc_to_page(cpu_addr);
if (page)
return __swiotlb_get_sgtable_page(sgt, page, size);
}
if (WARN_ON(!area || !area->pages))
return -ENXIO;
return sg_alloc_table_from_pages(sgt, area->pages, count, 0, size,
GFP_KERNEL);
return -ENXIO;
}
static void __iommu_sync_single_for_cpu(struct device *dev,
@ -772,11 +846,12 @@ static void __iommu_sync_single_for_cpu(struct device *dev,
enum dma_data_direction dir)
{
phys_addr_t phys;
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
if (is_device_dma_coherent(dev))
if (!domain || iommu_is_iova_coherent(domain, dev_addr))
return;
phys = iommu_iova_to_phys(iommu_get_domain_for_dev(dev), dev_addr);
phys = iommu_iova_to_phys(domain, dev_addr);
__dma_unmap_area(phys_to_virt(phys), size, dir);
}
@ -785,11 +860,12 @@ static void __iommu_sync_single_for_device(struct device *dev,
enum dma_data_direction dir)
{
phys_addr_t phys;
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
if (is_device_dma_coherent(dev))
if (!domain || iommu_is_iova_coherent(domain, dev_addr))
return;
phys = iommu_iova_to_phys(iommu_get_domain_for_dev(dev), dev_addr);
phys = iommu_iova_to_phys(domain, dev_addr);
__dma_map_area(phys_to_virt(phys), size, dir);
}
@ -798,7 +874,7 @@ static dma_addr_t __iommu_map_page(struct device *dev, struct page *page,
enum dma_data_direction dir,
unsigned long attrs)
{
bool coherent = is_device_dma_coherent(dev);
bool coherent = is_dma_coherent(dev, attrs);
int prot = dma_info_to_prot(dir, coherent, attrs);
dma_addr_t dev_addr = iommu_dma_map_page(dev, page, offset, size, prot);
@ -824,9 +900,11 @@ static void __iommu_sync_sg_for_cpu(struct device *dev,
enum dma_data_direction dir)
{
struct scatterlist *sg;
dma_addr_t iova = sg_dma_address(sgl);
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
int i;
if (is_device_dma_coherent(dev))
if (!domain || iommu_is_iova_coherent(domain, iova))
return;
for_each_sg(sgl, sg, nelems, i)
@ -838,9 +916,11 @@ static void __iommu_sync_sg_for_device(struct device *dev,
enum dma_data_direction dir)
{
struct scatterlist *sg;
dma_addr_t iova = sg_dma_address(sgl);
struct iommu_domain *domain = iommu_get_domain_for_dev(dev);
int i;
if (is_device_dma_coherent(dev))
if (!domain || iommu_is_iova_coherent(domain, iova))
return;
for_each_sg(sgl, sg, nelems, i)
@ -851,13 +931,18 @@ static int __iommu_map_sg_attrs(struct device *dev, struct scatterlist *sgl,
int nelems, enum dma_data_direction dir,
unsigned long attrs)
{
bool coherent = is_device_dma_coherent(dev);
bool coherent = is_dma_coherent(dev, attrs);
int ret;
ret = iommu_dma_map_sg(dev, sgl, nelems,
dma_info_to_prot(dir, coherent, attrs));
if (!ret)
return ret;
if ((attrs & DMA_ATTR_SKIP_CPU_SYNC) == 0)
__iommu_sync_sg_for_device(dev, sgl, nelems, dir);
return iommu_dma_map_sg(dev, sgl, nelems,
dma_info_to_prot(dir, coherent, attrs));
return ret;
}
static void __iommu_unmap_sg_attrs(struct device *dev,
@ -895,50 +980,14 @@ static int __init __iommu_dma_init(void)
}
arch_initcall(__iommu_dma_init);
static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *ops)
{
struct iommu_domain *domain;
if (!ops)
return;
/*
* The IOMMU core code allocates the default DMA domain, which the
* underlying IOMMU driver needs to support via the dma-iommu layer.
*/
domain = iommu_get_domain_for_dev(dev);
if (!domain)
goto out_err;
if (domain->type == IOMMU_DOMAIN_DMA) {
if (iommu_dma_init_domain(domain, dma_base, size, dev))
goto out_err;
dev->dma_ops = &iommu_dma_ops;
}
return;
out_err:
pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
dev_name(dev));
}
void arch_teardown_dma_ops(struct device *dev)
{
dev->dma_ops = NULL;
}
#else
static void __iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu)
{ }
#endif /* CONFIG_IOMMU_DMA */
static void arm_iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size);
void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
const struct iommu_ops *iommu, bool coherent)
{
@ -950,7 +999,7 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
}
dev->archdata.dma_coherent = coherent;
__iommu_setup_dma_ops(dev, dma_base, size, iommu);
arm_iommu_setup_dma_ops(dev, dma_base, size);
#ifdef CONFIG_XEN
if (xen_initial_domain()) {
@ -960,3 +1009,149 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
#endif
}
EXPORT_SYMBOL_GPL(arch_setup_dma_ops);
#ifdef CONFIG_ARM64_DMA_USE_IOMMU
/* guards initialization of default_domain->iova_cookie */
static DEFINE_MUTEX(iommu_dma_init_mutex);
static int
iommu_init_mapping(struct device *dev, struct dma_iommu_mapping *mapping)
{
struct iommu_domain *domain = mapping->domain;
dma_addr_t dma_base = mapping->base;
u64 size = mapping->bits << PAGE_SHIFT;
int ret;
bool own_cookie;
/*
* if own_cookie is false, then we are sharing the iova_cookie with
* another driver, and should not free it on error. Cleanup will be
* done when the iommu_domain is freed.
*/
own_cookie = !domain->iova_cookie;
if (own_cookie) {
ret = iommu_get_dma_cookie(domain);
if (ret) {
dev_err(dev, "iommu_get_dma_cookie failed: %d\n", ret);
return ret;
}
}
ret = iommu_dma_init_domain(domain, dma_base, size, dev);
if (ret) {
dev_err(dev, "iommu_dma_init_domain failed: %d\n", ret);
if (own_cookie)
iommu_put_dma_cookie(domain);
return ret;
}
mapping->ops = &iommu_dma_ops;
return 0;
}
static int arm_iommu_get_dma_cookie(struct device *dev,
struct dma_iommu_mapping *mapping)
{
int s1_bypass = 0, is_fast = 0;
int err = 0;
mutex_lock(&iommu_dma_init_mutex);
iommu_domain_get_attr(mapping->domain, DOMAIN_ATTR_S1_BYPASS,
&s1_bypass);
iommu_domain_get_attr(mapping->domain, DOMAIN_ATTR_FAST, &is_fast);
if (s1_bypass)
mapping->ops = &arm64_swiotlb_dma_ops;
else if (is_fast)
err = fast_smmu_init_mapping(dev, mapping);
else
err = iommu_init_mapping(dev, mapping);
mutex_unlock(&iommu_dma_init_mutex);
return err;
}
/*
* Checks for "qcom,iommu-dma-addr-pool" property.
* If not present, leaves dma_addr and dma_size unmodified.
*/
static void arm_iommu_get_dma_window(struct device *dev, u64 *dma_addr,
u64 *dma_size)
{
struct device_node *np;
int naddr, nsize, len;
const __be32 *ranges;
if (!dev->of_node)
return;
np = of_parse_phandle(dev->of_node, "qcom,iommu-group", 0);
if (!np)
np = dev->of_node;
ranges = of_get_property(np, "qcom,iommu-dma-addr-pool", &len);
if (!ranges)
return;
len /= sizeof(u32);
naddr = of_n_addr_cells(np);
nsize = of_n_size_cells(np);
if (len < naddr + nsize) {
dev_err(dev, "Invalid length for qcom,iommu-dma-addr-pool, expected %d cells\n",
naddr + nsize);
return;
}
if (naddr == 0 || nsize == 0) {
dev_err(dev, "Invalid #address-cells %d or #size-cells %d\n",
naddr, nsize);
return;
}
*dma_addr = of_read_number(ranges, naddr);
*dma_size = of_read_number(ranges + naddr, nsize);
}
static void arm_iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size)
{
struct iommu_domain *domain;
struct iommu_group *group;
struct dma_iommu_mapping mapping = {0};
group = dev->iommu_group;
if (!group)
return;
arm_iommu_get_dma_window(dev, &dma_base, &size);
domain = iommu_get_domain_for_dev(dev);
if (!domain)
return;
/* Allow iommu-debug to call arch_setup_dma_ops to reconfigure itself */
if (domain->type != IOMMU_DOMAIN_DMA &&
!of_device_is_compatible(dev->of_node, "iommu-debug-test")) {
dev_err(dev, "Invalid iommu domain type!\n");
return;
}
mapping.base = dma_base;
mapping.bits = size >> PAGE_SHIFT;
mapping.domain = domain;
if (arm_iommu_get_dma_cookie(dev, &mapping)) {
dev_err(dev, "Failed to get dma cookie\n");
return;
}
set_dma_ops(dev, mapping.ops);
}
#else /*!CONFIG_ARM64_DMA_USE_IOMMU */
static void arm_iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 size)
{
}
#endif

2
arch/arm64/mm/hugetlbpage.c
View file

@ -218,6 +218,8 @@ pte_t *huge_pte_alloc(struct mm_struct *mm,
ptep = (pte_t *)pudp;
} else if (sz == (PAGE_SIZE * CONT_PTES)) {
pmdp = pmd_alloc(mm, pudp, addr);
if (!pmdp)
return NULL;
WARN_ON(addr & (sz - 1));
/*

2
arch/arm64/mm/init.c
View file

@ -53,6 +53,8 @@
#include <asm/tlb.h>
#include <asm/alternative.h>
EXPORT_SYMBOL_GPL(kimage_vaddr);
/*
* We need to be able to catch inadvertent references to memstart_addr
* that occur (potentially in generic code) before arm64_memblock_init()

43
arch/arm64/mm/mmu.c
View file

@ -32,6 +32,8 @@
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/vmalloc.h>
#include <linux/dma-contiguous.h>
#include <linux/cma.h>
#include <asm/barrier.h>
#include <asm/cputype.h>
@ -67,6 +69,40 @@ static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused;
static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused;
struct dma_contig_early_reserve {
phys_addr_t base;
unsigned long size;
};
static struct dma_contig_early_reserve dma_mmu_remap[MAX_CMA_AREAS];
static int dma_mmu_remap_num;
void __init dma_contiguous_early_fixup(phys_addr_t base, unsigned long size)
{
if (dma_mmu_remap_num >= ARRAY_SIZE(dma_mmu_remap)) {
pr_err("ARM64: Not enough slots for DMA fixup reserved regions!\n");
return;
}
dma_mmu_remap[dma_mmu_remap_num].base = base;
dma_mmu_remap[dma_mmu_remap_num].size = size;
dma_mmu_remap_num++;
}
static bool dma_overlap(phys_addr_t start, phys_addr_t end)
{
int i;
for (i = 0; i < dma_mmu_remap_num; i++) {
phys_addr_t dma_base = dma_mmu_remap[i].base;
phys_addr_t dma_end = dma_mmu_remap[i].base +
dma_mmu_remap[i].size;
if ((dma_base < end) && (dma_end > start))
return true;
}
return false;
}
pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
unsigned long size, pgprot_t vma_prot)
{
@ -200,7 +236,8 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end,
/* try section mapping first */
if (((addr | next | phys) & ~SECTION_MASK) == 0 &&
(flags & NO_BLOCK_MAPPINGS) == 0) {
(flags & NO_BLOCK_MAPPINGS) == 0 &&
!dma_overlap(phys, phys + next - addr)) {
pmd_set_huge(pmdp, phys, prot);
/*
@ -299,7 +336,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end,
* For 4K granule only, attempt to put down a 1GB block
*/
if (use_1G_block(addr, next, phys) &&
(flags & NO_BLOCK_MAPPINGS) == 0) {
(flags & NO_BLOCK_MAPPINGS) == 0 &&
!dma_overlap(phys, phys + next - addr)) {
pud_set_huge(pudp, phys, prot);
/*
@ -720,6 +758,7 @@ int kern_addr_valid(unsigned long addr)
return pfn_valid(pte_pfn(pte));
}
EXPORT_SYMBOL_GPL(kern_addr_valid);
#ifdef CONFIG_SPARSEMEM_VMEMMAP
#if !ARM64_SWAPPER_USES_SECTION_MAPS
int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,

12
arch/hexagon/include/asm/io.h
View file

@ -186,16 +186,10 @@ static inline void writel(u32 data, volatile void __iomem *addr)
#define mmiowb()
/*
* Need an mtype somewhere in here, for cache type deals?
* This is probably too long for an inline.
*/
void __iomem *ioremap_nocache(unsigned long phys_addr, unsigned long size);
void __iomem *ioremap(unsigned long phys_addr, unsigned long size);
#define ioremap_nocache ioremap
#define ioremap_uc(X, Y) ioremap((X), (Y))
static inline void __iomem *ioremap(unsigned long phys_addr, unsigned long size)
{
return ioremap_nocache(phys_addr, size);
}
static inline void iounmap(volatile void __iomem *addr)
{

2
arch/hexagon/kernel/hexagon_ksyms.c
View file

@ -33,7 +33,7 @@ EXPORT_SYMBOL(__vmgetie);
EXPORT_SYMBOL(__vmsetie);
EXPORT_SYMBOL(__vmyield);
EXPORT_SYMBOL(empty_zero_page);
EXPORT_SYMBOL(ioremap_nocache);
EXPORT_SYMBOL(ioremap);
EXPORT_SYMBOL(memcpy);
EXPORT_SYMBOL(memset);

2
arch/hexagon/mm/ioremap.c
View file

@ -22,7 +22,7 @@
#include <linux/vmalloc.h>
#include <linux/mm.h>
void __iomem *ioremap_nocache(unsigned long phys_addr, unsigned long size)
void __iomem *ioremap(unsigned long phys_addr, unsigned long size)
{
unsigned long last_addr, addr;
unsigned long offset = phys_addr & ~PAGE_MASK;

2
arch/mips/Kconfig
View file

@ -21,6 +21,7 @@ config MIPS
select GENERIC_CLOCKEVENTS
select GENERIC_CMOS_UPDATE
select GENERIC_CPU_AUTOPROBE
select GENERIC_GETTIMEOFDAY
select GENERIC_IRQ_PROBE
select GENERIC_IRQ_SHOW
select GENERIC_LIB_ASHLDI3
@ -71,6 +72,7 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES

3
arch/mips/cavium-octeon/octeon-irq.c
View file

@ -2199,6 +2199,9 @@ static int octeon_irq_cib_map(struct irq_domain *d,
}
cd = kzalloc(sizeof(*cd), GFP_KERNEL);
if (!cd)
return -ENOMEM;
cd->host_data = host_data;
cd->bit = hw;

15
arch/mips/include/asm/clocksource.h
View file

@ -11,19 +11,6 @@
#ifndef __ASM_CLOCKSOURCE_H
#define __ASM_CLOCKSOURCE_H
#include <linux/types.h>
/* VDSO clocksources. */
#define VDSO_CLOCK_NONE 0 /* No suitable clocksource. */
#define VDSO_CLOCK_R4K 1 /* Use the coprocessor 0 count. */
#define VDSO_CLOCK_GIC 2 /* Use the GIC. */
/**
* struct arch_clocksource_data - Architecture-specific clocksource information.
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
u8 vdso_clock_mode;
};
#include <asm/vdso/clocksource.h>
#endif /* __ASM_CLOCKSOURCE_H */

16
arch/mips/include/asm/processor.h
View file

@ -22,6 +22,7 @@
#include <asm/dsemul.h>
#include <asm/mipsregs.h>
#include <asm/prefetch.h>
#include <asm/vdso/processor.h>
/*
* Return current * instruction pointer ("program counter").
@ -386,21 +387,6 @@ unsigned long get_wchan(struct task_struct *p);
#define KSTK_ESP(tsk) (task_pt_regs(tsk)->regs[29])
#define KSTK_STATUS(tsk) (task_pt_regs(tsk)->cp0_status)
#ifdef CONFIG_CPU_LOONGSON3
/*
* Loongson-3's SFB (Store-Fill-Buffer) may buffer writes indefinitely when a
* tight read loop is executed, because reads take priority over writes & the
* hardware (incorrectly) doesn't ensure that writes will eventually occur.
*
* Since spin loops of any kind should have a cpu_relax() in them, force an SFB
* flush from cpu_relax() such that any pending writes will become visible as
* expected.
*/
#define cpu_relax() smp_mb()
#else
#define cpu_relax() barrier()
#endif
/*
* Return_address is a replacement for __builtin_return_address(count)
* which on certain architectures cannot reasonably be implemented in GCC

78
arch/mips/include/asm/vdso.h
View file

@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
#include <vdso/datapage.h>
#include <asm/barrier.h>
@ -53,84 +54,9 @@ extern struct mips_vdso_image vdso_image_o32;
extern struct mips_vdso_image vdso_image_n32;
#endif
/**
* union mips_vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
* @seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
* @cs_cycle_last: Clock cycle value at last update.
* @cs_mask: Clocksource mask value.
* @tz_minuteswest: Minutes west of Greenwich (from timezone).
* @tz_dsttime: Type of DST correction (from timezone).
*
* This structure contains data needed by functions within the VDSO. It is
* populated by the kernel and mapped read-only into user memory. The time
* fields are mirrors of internal data from the timekeeping infrastructure.
*
* Note: Care should be taken when modifying as the layout must remain the same
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
struct {
u64 xtime_sec;
u64 xtime_nsec;
u64 wall_to_mono_sec;
u64 wall_to_mono_nsec;
u32 seq_count;
u32 cs_shift;
u8 clock_mode;
u32 cs_mult;
u64 cs_cycle_last;
u64 cs_mask;
s32 tz_minuteswest;
s32 tz_dsttime;
};
struct vdso_data data[CS_BASES];
u8 page[PAGE_SIZE];
};
static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
{
u32 seq;
while (true) {
seq = READ_ONCE(data->seq_count);
if (likely(!(seq & 1))) {
/* Paired with smp_wmb() in vdso_data_write_*(). */
smp_rmb();
return seq;
}
cpu_relax();
}
}
static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
u32 start_seq)
{
/* Paired with smp_wmb() in vdso_data_write_*(). */
smp_rmb();
return unlikely(data->seq_count != start_seq);
}
static inline void vdso_data_write_begin(union mips_vdso_data *data)
{
++data->seq_count;
/* Ensure sequence update is written before other data page values. */
smp_wmb();
}
static inline void vdso_data_write_end(union mips_vdso_data *data)
{
/* Ensure data values are written before updating sequence again. */
smp_wmb();
++data->seq_count;
}
#endif /* __ASM_VDSO_H */

20
arch/mips/include/asm/vdso/clocksource.h Normal file
View file

@ -0,0 +1,20 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
#ifndef __ASM_VDSOCLOCKSOURCE_H
#define __ASM_VDSOCLOCKSOURCE_H
#include <linux/types.h>
/* VDSO clocksources. */
#define VDSO_CLOCK_NONE 0 /* No suitable clocksource. */
#define VDSO_CLOCK_R4K 1 /* Use the coprocessor 0 count. */
#define VDSO_CLOCK_GIC 2 /* Use the GIC. */
/**
* struct arch_clocksource_data - Architecture-specific clocksource information.
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
u8 vdso_clock_mode;
};
#endif /* __ASM_VDSOCLOCKSOURCE_H */

Some files were not shown because too many files have changed in this diff Show more