Introduction ============ The Shared Memory Point to Point (SMP2P) protocol facilitates communication of a single 32-bit value between two processors. Each value has a single writer (the local side) and a single reader (the remote side). Values are uniquely identified in the system by the directed edge (local processor ID to remote processor ID) and a string identifier. Version and feature negotiation has been included in the design to allow for phased upgrades of all processors. Software Architecture Description ================================= The data and interrupt coupling between processors is shown in Fig. 1. Each processor is responsible for creating the outgoing SMEM items and each item is writable by the local processor and readable by the remote processor. By using two separate SMEM items that are single-reader and single-writer, SMP2P does not require any remote locking mechanisms. The client API uses the Linux GPIO and interrupt framework to expose a virtual GPIO and a virtual interrupt controller for each entry. ================= | | -----write------>|SMEM item A->B |-----read------ | | | | | ================= | | | | v GPIO API => ------------ ======= Interrupt line ======> ------------ Processor A Processor B Interrupt <= ------------ <====== Interrupt line ======= ------------ API ^ | | | | | | ================= | | | | | ------read-------|SMEM item A<-B |<-----write---- | | ================= Fig 1 Design ====== Each SMEM item contains a header that is used to identify and manage the edge along with an array of actual entries. The overall structure is captured in Fig 2 and the details of the header and entries are covered later in this section. The memory format of all shared structures is little-endian. ----------------------------------------------- | SMEM item A->B | | | | ----------------------------------------- | | |31 24| 16| 8| 0| | | |----------|---------|----------|---------| | | | Identifier Constant(Magic Number) | | | |----------|---------|----------|---------| | | | Feature Flags |Version | | | | |Number | | | |----------|---------|----------|---------| | | | Remote Proc ID |Local Proc ID | | | |----------|---------|----------|---------| | | | Entries Valid | Entries Total | | | |-----------------------------------------| | | | | | | ----------------------------------------- | | | Entry 0 | | | | ---------------------------------- | | | | | Identifier String | | | | | |---------------------------------| | | | | | Data | | | | | |---------------------------------| | | | |---------------------------------------| | | ----------------------------------------- | | | Entry 1 | | | | ---------------------------------- | | | | | Identifier String | | | | | |---------------------------------| | | | | | Data | | | | | |---------------------------------| | | | |---------------------------------------| | | - | | - | | - | | ----------------------------------------- | | | Entry N | | | | ---------------------------------- | | | | | Identifier String | | | | | |---------------------------------| | | | | | Data | | | | | |---------------------------------| | | | |---------------------------------------| | ----------------------------------------------- Fig 2 The header of each SMEM item contains metadata that describes the processors using the edge, the version information, and the entry count. The constant identifier is used as a magic number to enable extraction of the items from a memory dump. The size of each entry depends upon the version, but the number of total entries (and hence the size of each SMEM item) is configurable with a suggested value of 16. The number of valid entries is used to indicate how many of the Entries Total are currently used and are current valid. --------------------------------------------------------------------------- |Field Size Description Valid Values | --------------------------------------------------------------------------- | Identifier 4 Bytes Value used to identify | | Constant structure in memory. Must be set to $SMP | | Useful for debugging. (0x504D5324) | --------------------------------------------------------------------------- | Local 2 Bytes Writing processor ID. Refer Processor ID Table 3| | Processor | | ID | --------------------------------------------------------------------------- | Remote 2 Bytes Reading processor ID. Refer Processor ID Table 3| | Processor | | ID | --------------------------------------------------------------------------- | Version 1 Bytes Refer to Version | | Number Feature Negotiation Must be set to 1. | | section. | --------------------------------------------------------------------------- | Feature 3 Bytes Refer to Version | | flags and Feature Negotiation | | section for details. | | bit 0 SSR_ACK Feature Supported when set to 1 | | bits 1:31 Reserved Must be set to 0. | --------------------------------------------------------------------------- | Entries 2 Bytes Total number of Must be 0 or greater. | | Total entries. | --------------------------------------------------------------------------- | Entries 2 Bytes Number of valid Must be between 0 | | Valid entries. and Entries Total. | --------------------------------------------------------------------------- | Flags 4 Bytes | | bit 0 RESTART_DONE Toggle for every restart | | bit 1 RESTART_ACK Toggle to ACK remote | | RESTART_DONE | | bits 2:31 Reserved Must be set to 0. | --------------------------------------------------------------------------- Table 1 - SMEM Item Header The content of each SMEM entries is described in Table 2 and consists of a string identifier and a 32-bit data value. The string identifier must be unique for each SMEM item. The data value is opaque to SMP2P giving the client complete flexibility as to its usage. ----------------------- --------------------- ----------------------------- | Field | Size | Description | Valid Values | ------------|----------|---------------------|----------------------------- | | | | | | Identifier | 16 Bytes | Null Terminated | NON-NULL for | | String | | ASCII string. | valid entries. | | | | | | ------------|----------|---------------------|----------------------------- | Data | 4 Bytes | Data | Any (client defined) | ------------ ---------- --------------------- ----------------------------- Table 2 - Entry Format The processor IDs in the system are fixed and new processors IDs will be added to the end of the list (Table 3). ------------------------------------------------- | Processor Name | ID value | ------------------------------------------------- | Application processor | 0 | ------------------------------------------------- | Modem processor | 1 | ------------------------------------------------- | Audio processor | 2 | ------------------------------------------------- | Sensor processor | 3 | ------------------------------------------------- | Wireless processor | 4 | ------------------------------------------------- | Modem Fw | 5 | ------------------------------------------------- | Power processor | 6 | ------------------------------------------------- | TrustZone processor | 7 | ------------------------------------------------- | NUM PROCESSORS | 8 | ------------------------------------------------- Table 3 - Processor IDs SMEM Item --------- The responsibility of creating an SMEM item is with the local processor that is initiating outbound traffic. After creating the item, the local and remote processors negotiate the version and feature flags for the item to ensure compatibility. Table 4 lists the SMEM item base identifiers. To get the SMEM item ID for a particular edge, the remote processor ID (Table 3) is added to the base item ID for the local processor (Table 4). For example, the Apps ==> Modem (id 1) SMEM Item ID will be 427 + 1 = 428. --------------------------------------------------- | Description | SMEM ID value | --------------------------------------------------- | Apps SMP2P SMEM Item base | 427 | --------------------------------------------------- | Modem SMP2P SMEM Item base | 435 | --------------------------------------------------- | Audio SMP2P SMEM Item base | 443 | --------------------------------------------------- | Sensors SMP2P SMEM Item base | 481 | --------------------------------------------------- | Wireless SMP2P SMEM Item base | 451 | --------------------------------------------------- | Power SMP2P SMEM Item base | 459 | --------------------------------------------------- | TrustZone SMP2P SMEM Item base | 489 | --------------------------------------------------- Table 4 - SMEM Items Base IDs Version and Feature Negotiation ------------------------------- To enable upgrading without breaking the system and to enable graceful feature fall-back support, SMP2P supports a version number and feature flags. The combination of the version number and feature flags enable: 1) SMP2P software updates to be rolled out to each processor separately. 2) Individual features to be enabled or disabled per connection or edge. The version number represents any change in SMP2P that breaks compatibility between processors. Examples would be a change in the shared data structures or changes to fundamental behavior. Each implementation of SMP2P must be able to support a minimum of the current version and the previous version. The feature flags represent any changes in SMP2P that are optional and backwards compatible. Endpoints will negotiate the supported flag when the SMEM items are created and they cannot be changed after negotiation has been completed. Negotiation Algorithm ---------------------- While creating the SMEM item the following algorithm shall be used. if remote endpoint's SMEM Item exists Read remote version number and flags Local version number must be lower of - remote version number - highest supported local version number Flags value is bitwise AND of - remote feature flags - locally supported flags Create SMEM item and populate negotiated number and flags Interrupt remote processor if version and flags match, negotiation is complete, else wait for remote interrupt below. Else Create SMEM item and populate it with highest supported version and any requested feature flag. Interrupt remote processor. Wait for Interrupt below. Upon receiving the interrupt from remote processor and negotiation is not complete, check the version number and feature flags: if equal, negotiation is complete. if remote number is less than local number, and remote number is supported: Set local version number to remote version number Bitwise AND local flags with remote flags Interrupt remote processor Negotiation is complete if remote number is not supported, then negotiation has failed Set version number to 0xFF and report failure in kernel log. if remote number is more than local number: Wait for remote endpoint to process our interrupt and negotiate down. Creating an SMEM Entry ---------------------- Each new SMEM entry used in data transfer must be created at the end of the entry array in the SMEM item and cannot be deleted until the system is rebooted. The following sequence is be followed: 1) Compare Entries Valid and Entries Total to verify if there is room in the entry array for this request (if not, return error code to client). 2) Populate the Identifier of new entry and do a write memory barrier. 3) Update Entries Valid and Entries Total and do a write memory barrier. 4) Interrupt remote endpoint. Entry Write ----------- An entry write is achieved by the following sequence of operations: 1) Update data field in the entry and do a write memory barrier. 2) Interrupt remote endpoint. Entry Read / Receiving Interrupts --------------------------------- An interrupt will be received from the remote system for one or more of the following events: 1) Initialization 2) Entry change 3) New Entry As long as the SMEM item initialization is complete, then each interrupt should trigger SMP2P to: 1) Compare valid entry data value to cached value and notify client if it has changed. 2) Compare Entries Valid to cached value. If changed, initialize new entries. Security ======== Since the implementation resides in the kernel and does not expose interfaces to userspace, no security issues are anticipated. The usage of separate SMEM items allows for future security enhancements in SMEM. Performance =========== No performance issues are anticipated as the signaling rate is expected to be low and is performed in interrupt context which minimizes latency. Interfaces ================ SMP2P is only supported in the kernel and interfaces with clients through the GPIO and interrupt subsystems. To map an entry to the client, the client must add two nodes to the Device Tree: 1) A node that matches "qcom,smp2pgpio" to create the entry 2) A node that matches the client driver to provide the GPIO pin mapping The details of the device tree entries for the GPIO interface are contained in the file Documentation/devicetree/bindings/gpio/gpio-smp2p.txt. /* SMP2P Test Driver for inbound entry. */ smp2pgpio_smp2p_7_in: qcom,smp2pgpio-smp2p-7-in { compatible = "qcom,smp2pgpio"; qcom,entry-name = "smp2p"; qcom,remote-pid = <7>; qcom,is-inbound; gpio-controller; #gpio-cells = <2>; interrupt-controller; #interrupt-cells = <2>; }; /* SMP2P Test Client for inbound entry. */ qcom,smp2pgpio_test_smp2p_7_in { compatible = "qcom,smp2pgpio_test_smp2p_7_in"; gpios = <&smp2pgpio_smp2p_7_in 0 0>, <&smp2pgpio_smp2p_7_in 1 0>, . . . <&smp2pgpio_smp2p_7_in 31 0>; }; /* SMP2P Test Driver for outbound entries */ smp2pgpio_smp2p_345_out: qcom,smp2pgpio-smp2p-7-out { compatible = "qcom,smp2pgpio"; qcom,entry-name = "smp2p"; qcom,remote-pid = <7>; gpio-controller; #gpio-cells = <2>; interrupt-controller; #interrupt-cells = <2>; }; /* SMP2P Test Client for outbound entry. */ qcom,smp2pgpio_test_smp2p_7_out { compatible = "qcom,smp2pgpio_test_smp2p_7_out"; gpios = <&smp2pgpio_smp2p_7_out 0 0>, <&smp2pgpio_smp2p_7_out 1 0>, . . . <&smp2pgpio_smp2p_7_out 31 0>; The client can use a match entry for "qcom,smp2pgpio_test_smp2p_7_in" to retrieve the Device Tree configuration node. Once that node has been retrieved, the client can call of_get_gpio() to get the virtual GPIO pin and also use gpio_to_irq() to map the GPIO pin to a virtual interrupt. After that point, the standard GPIO and Interrupt APIs can be used to manipulate the SMP2P entries and receive notifications of changes. Examples of typical function calls are shown below: of_get_gpio() gpio_get_value() gpio_set_value() gpio_to_irq() request_irq() free_irq() Please reference the unit test code for example usage. Subsystem Restart ================= SMP2P is unaffected by SubSystem Restart (SSR) on the high-level OS side and is actually used as an underlying communication mechanism for SSR. On the peripheral system that is being restarted, SMP2P will zero out all existing state entries upon reboot as part of the SMP2P initialization process and if the SSR_ACK feature is enabled, then it waits for an acknowledgement as outlined in the following subsections. SSR_ACK Feature - Reboot Use Case (Non-HLOS Only) ------------------------------------------------- If a remote system boots up after an SSR and sees that the remote and local version numbers and feature flags are equal, then it zeros out entry values. If the SSR_ACK feature is enabled, it will wait for an acknowledgement from the other processor that it has seen the zero entry before completing the negotiation sequence. if remote and local version numbers and feature flags are equal Zero out all entry values if SSR_ACK feature is enabled Set local RESTART_DONE flag to inverse of the remote RESTART_ACK Send interrupt to remote system Wait for interrupt and for remote RESTART_ACK to be equal to local RESTART_DONE Continue with normal negotiation sequence Interrupt Use Case ------------------ For every interrupt triggered by a remote change, SMP2P will notify the client of a change in state. In addition, if the SSR_ACK feature is enabled, the SSR handshaking will also be handled. if SSR_ACK feature is enabled if remote RESTART_DONE != local RESTART_ACK Notify client of entry change (will be * -> 0 transition) Toggle local RESTART_ACK Send interrupt to remote system else Notify client of entry change as usual else Notify client of entry change as usual Debug ===== The state values and names for all entries accessible by the Apps are accessible through debugfs nodes for general debug purposes. Debugfs entries for triggering unit-tests are also exported. Internal logging will be performed using the IPC Logging module to enable post-mortem analysis. Testing ======= On-target unit testing will be done to verify internal functionality and the GPIO/IRQ API's. Driver parameters ================= One module parameter will be provided to change the verbosity of internal logging. Config options ============== Configuration of interrupts will be done using Device Tree per the format in Documentation/devicetree/bindings/arm/msm/smp2p.txt. By default, the testing components will be enabled since it does not affect performance and has a minimal impact on kernel size. However, customers can disable the testing component for size optimization. CONFIG_MSM_SMP2P - enables SMP2P CONFIG_MSM_SMP2P_TEST - enables unit test support Dependencies =========== Requires SMEM for creating the SMEM items. User Space utilities ==================== No userspace utilities are planned. Known issues ============ None.