COSM: A Cooperative Scheduling Framework for
Concurrent PIM and CPU Execution on Mobile Devices

In modern DRAM, the latency of control commands (e.g., opening a row in LPDDR5 [JEDECLPDDR52023] requires $tRP+tRCD\approx 30$ DRAM clock cycles) is much longer than the duration of the burst length ($tBL=8$ DRAM clock cycles), where $tRP$, $tRCD$, and $tBL$ refer to row precharge time, the row address strobe to column address strobe delay, and the burst length [salp_6237032, tl_dram_6522354]. To mitigate this overhead, DRAM adopts a hierarchical architecture. Multiple banks in a rank can operate independently, and row operations can overlap with each other across banks that share a common memory bus.

As DRAM density increases with increasing bank counts, the overhead of row opening emerges as a fundamental scaling bottleneck. Although more banks offer higher theoretical parallelism, fixed command bandwidth severely restricts maximum internal bandwidth utilization. For example, in a typical 2-rank LPDDR5 per-channel mobile phone setup, each rank has 32 banks [samsunglpddr5pim10254711] ($\#bank=32$). For a workload with a row hit rate $R_{h}$, the upper bound under saturated external bandwidth (i.e., assuming 100% external bandwidth utilization) is:

The $\#bank$ term in the denominator illustrates that the shared command bus serializes bank accesses, causing the denominator to far outweigh the numerator. This reveals a fundamental problem: even with completely random access that leads to zero hit rate ($R_{h}=0$), the utilization of internal bandwidth cannot exceed $15\%$. Real-world workloads typically achieve far lower utilization, as demonstrated in Fig. 1.

Unlike traditional heterogeneous systems (e.g., CPU-GPU systems) where all compute units share a unified DRAM controller interface, PIM units are spatially distributed across DRAM banks and access memory through dedicated intra-bank pathways. This fundamental architectural distinction requires specialized interfaces for PIM systems. Current implementations predominantly adopt two types of PIM interfaces:

To characterize the impact of memory-side contention, we conduct a sensitivity study on CPU performance in response to increased memory latency. We simulate PIM-induced access delays by injecting additional CPU read latency on a CPU-only system across three real-world applications and a SPEC 2017 benchmark. As shown in Figure 2(a), a 16-cycle latency increase reduces CPU performance by more than 5%. As latency increases, performance significantly degrades: e.g., a 128-cycle latency reduces CPU performance by more than 40% for some workloads. This confirms that CPU workload performance is sensitive to memory access latency.

The results of this study offer two key guidelines for the formulation of memory scheduling strategies in the concurrent execution of CPU-PIM hybrid systems. First, the system must enable fast preemption of PIM operations to reduce CPU request latency. This requires interrupting PIM operations and minimizing the switching overhead between CPU and PIM commands. A single-host design with fine-grained PIM command control is preferred, as a two-host design introduces significant switching latency, and coarse-grained PIM commands hinder timely CPU access. Second, memory scheduling should give precedence to CPU memory accesses and restrict PIM operations to periods when the memory is idle, thereby guaranteeing minimal disruption. Collectively, these guidelines require a closely integrated design of the PIM execution interface alongside the memory scheduler.

We study how the granularity of PIM execution commands affects PIM performance, defining the “command length” as the number of cycles PIM units can execute autonomously per command. Figure 2(b) shows that longer command lengths improve PIM performance in a single-host system with 2 LPDDR5 ranks per channel and 32 banks in total. A command with a length of $\geq$ 128 keeps the command bandwidth below 40%, while a length of $\geq$ 64 is needed for full bank-level parallelism across 32 banks (taking into account the opening overheads of rows). Shorter command lengths cause command bus congestion and external DRAM bandwidth underutilization.

This finding is in tension with the observation in Section 3.1: longer command lengths boost PIM performance, but shorter ones are crucial to minimize CPU memory latency. To eliminate this tension in existing fixed-length command architectures, we propose preemptable PIM execution commands in the COSM framework to balance extended command benefits with CPU access needs.

Although PIM workloads primarily leverage internal DRAM bandwidth between PIM units and their local banks, the host CPU still needs to transfer input data to PIM units and collect results from PIM units. These necessary CPU-mediated transfers create memory contention for co-executing CPU workloads that share the same DRAM devices.

To quantify the impact of this bandwidth contention, we first measure the proportion of CPU-mediated transfers during an attention layer inference of a DeepSeek-R1-1.5B model [deepseekai2025deepseekr1incentivizingreasoningcapability] in the decoding phase on the same system as Section 3.2, as shown in Fig. 2(c). The command length of each PIM execution is set to 128 cycles. When the sequence length dimension $N$ of Key-Value (KV) cache is 64, CPU-mediated transfers account for over 60% of the overall inference time. As $N$ increases during inference, this proportion gradually decreases to around 50% when $N$=4k, demonstrating that CPU-mediated transfers still constitute a substantial fraction of the overall execution time.

Fig. 2(d) further demonstrates the interference caused by CPU-mediated transfers on concurrent CPU workloads. Both CPU-mediated transfers of PIM workload and conventional CPU memory requests in CPU workload share the same memory request queue, which is scheduled using the classic First-Ready, First-Come First-Serve (FR-FCFS) policy [FR-FCFS10.1145/339647.339668, zuravleff1997controller]. We observe that CPU workload performance degrades by more than 80% when concurrent with CPU-mediated transfers. This severe slowdown arises because CPU-mediated transfers exhibit bursty access patterns with high row-buffer locality. Under the FR-FCFS scheduler, such bursts monopolize the memory controller (as described in [memory_performance_attack_268461, fairscheduling4408252, parallelism-aware4556716]) by repeatedly hitting open rows, thereby starving CPU workloads, whose requests often require costly row conflicts.

These results reveal a critical insight: the effect of the observed 80% CPU performance degradation during the 50% data transfer can translate into a 40% reduction in overall CPU performance. This disproportionate impact shows that CPU-mediated transfers must be treated as first-class citizens in memory scheduling in concurrent CPU and PIM access scenarios. Consequently, CPU-mediated transfers must be governed by the same scheduling discipline as PIM units’ execution, i.e., scheduled during idle time windows. However, implementing the CPU-mediated transfers requires simultaneous availability of both external and internal idle time windows, making it difficult to fully utilize memory idle time windows, limiting PIM throughput. To overcome this, COSM introduces bandwidth-decoupled CPU-mediated transfer commands that decouple the usage of external and internal bandwidth, eliminating the need for simultaneous external and internal idle time window availability.

Based on the observations above, we analyze several techniques for concurrent execution of PIM and CPU on shared memory banks. Fig. 3(a) depicts CPU memory request processing under FR-FCFS, showing memory bus and bank occupancy along with per-bank queue lengths. Fig. 3(b)-(d) compare three concurrency techniques under the same CPU workload. Each PIM workload consists of one burst write of CPU-mediated transfer (co-scheduled with normal CPU access under FR-FCFS) followed by multiple execution commands.

The proposed preemptable PIM control interface extends standard DRAM commands with two critical additions: PIM execution command family (PIM_Exec) that allow PIM units to automatically execute computations continuously for an extended period without requiring additional trigger commands, and a PIM pause command (PIM_Pause) that enables preemption by halting PIM execution. Compared to the fixed-length command design discussed in Sections 3.1 and 3.2, the preemptable commands enable immediate reaction to incoming CPU access while avoiding command bus saturation that would otherwise degrade PIM performance. The scope of all the PIM command are at the bank level. We employ a hardware-command co-design approach to achieve preemptable computation. Managed by the memory controller, this interface allows fine-grained execution of PIM workloads while enabling low CPU memory access latency.

PIM Execution Commands. In our setup, the execution commands PIM_Exec for PIM are tailored for LLM operations such as MAC and softmax. Fig. 5 (a) illustrates the execution of a PIM command and the register states of the PEE. If a command with start column 64 is issued at $clk1$, the PEE module of the PIM unit switches from state to by setting the Column State Counter (CSC) with the start address, recording the command type in the Command Register (CR), and setting the PIM Counter (PC) as 0. For every $tCCD$ (column-to-column delay), the PEE autonomously increments the Column State Counter and PIM Counter and issues commands in Command Register using the Column State Counter as the column address ( ). The Command Arbiter can always synchronize the PIM Counter state by inferring the PIM Counter : PC_inf=$(clk-clk1)/tCCD$, where $clk$ is the current clock cycle. This procedure halts after $nPTL$ cycles (i.e., PIM execution command length predefined by the DRAM configuration register) by comparing PIM Counter value with the $nPTL/tCCD$ value if there is no CPU memory access, and PEE returns to its default state ( ). This reduces the usage of the command bus by $nPTL/tCCD$ times compared to fine-grained commands (see Section 3.2). In the setup shown in Fig. 2 (b) (2 ranks ×16 banks), $nPTL$ needs to be at least 64 cycles for more flexibility, with lower values leading to command bus contention.

PIM Pause Command. The PIM_Pause command ensures timely CPU request handling by pausing PIM execution. Fig. 5 (b) shows a CPU memory request interrupting PIM execution when the CPU reads from a bank still processing column 65 at time $clk2=clk1+tCCD+1$. The memory controller’s Command Arbiter sends PIM_Pause to the bank. Upon receiving it, the PEE completes the current column operation in up to $tCCD$ cycles, freezes the Column State Counter (by opening the Switch of PEE in Fig. 4), and releases the bus control (from to ). The Command Arbiter can directly infer the PIM Counter state by calculating the interval between $clk1$ and the end of the current execution ($t1=2tCCD$). After CPU access, the Command Arbiter reissues the origin PIM execution command, resuming from the Column State Counter-stored column address ( ). Knowing that columns 64-65 are completed, the Command Arbiter determines the remaining execution time as $t2=nPTL-2tCCD=62tCCD$ ( ), synchronizing with the PIM Counter without extra signaling and minimizing timing constraints.

Timing Constraints. Table 1 provides the timing constraints for preemptable PIM commands. Similar to DRAM read/write commands, a PIM execution command can only be issued after $tRCD$ after row activation. The PIM_Pause follows at least $tCCD$ cycles after PIM execution command to ensure PIM unit column access completion. Post PIM_Pause, if the execution command is load-only (PIM_Exec(Ld)), a PRE can be issued after $tRTP$ cycles. For execution commands with data stores (PIM_Exec(St)), PRE must wait for data stabilization, requiring $tWR$. This longer latency necessitates careful design to maximize intermediate result reuse and minimize store operations.

Semantic Guarantees of PIM_Pause. The correctness of the PIM pause mechanism relies on strict adherence to DRAM timing constraints and three fundamental properties. First, it guarantees column atomicity by pausing only at column boundaries (after integer multiples of $tCCD$), ensuring deterministic progress tracking for both the PIM unit and the memory controller. Second, it preserves all intermediate data in PIM buffers and architectural states during suspension, preventing context corruption. Third, it correctly restores PIM state by reactivating the original row and reissuing the paused command; the PIM unit then leverages frozen PIM Counter to resume execution precisely at the pause point. Collectively, these guarantees ensure that preempted CPU accesses and DRAM refreshes do not compromise the correctness of PIM execution.

Conventional PIM data transfers are based on standard DRAM read/write commands, which could require a long sequence of row activation, column access, and external data transfer. Moreover, the CPU-mediated transfer must occur continuously, locking the channel for the entire duration of the tranfer.

We introduce a mechanism that decouples the usage of external bandwidth from that of internal bandwidth during CPU-mediated data transfer. As depicted in Fig. 6, the memory interface employs two-phase reads and writes via four commands: PIM_RdBuf and PIM_WrBuf for external transfers between the memory controller and the PIM buffer through the memory bus, and PIM_LdBuf and PIM_StBuf for internal transfers between the buffer and DRAM banks. This division allows independent scheduling of requests, minimizing CPU memory access interference by using bus idle time windows for external transfers and exploiting bank idle time window for internal transfers. It enables the idleness-aware scheduling policy detailed in Section 6.2.

The PIM_LdBuf and PIM_StBuf commands are designed to mirror the memory access behavior of PIM_Exec(Ld) and PIM_Exec(St): they exclusively access the DRAM bank using internal bandwidth, transferring data from/to the bank to/from the buffer. and do not consume external bandwidth. Consequently, they inherit the same execution model: preemptable operation with a command length of $nPTL$. This design choice enables unified scheduling logic in all PIM-initiated bank activities.

Additional PIM Buffer Requirement. To support our mechanism, we add an extra segment to the PIM buffer whose capacity is commensurate with the maximum amount of data transferred by a single PIM_LdBuf or PIM_StBuf command of length $nPTL$. Since internal bandwidth is frequently available during ample idle time windows, the scheduler can always complete these short internal transfers before the next external transfer command arrives. Thus, a small per-bank buffer suffices to avoid stalling, as validated in our experiment (Section 8.5).

Timing Constraints. Table 1 provides the timing constraints for the new commands. PIM_RdBuf and PIM_WrBuf commands only use the external memory channel bandwidth, requiring a $tBL$ delay after standard Read/Write commands on the same channel. These two commands, along with PIM_LdBuf/PIM_StBuf, need the PIM buffer to be ready before execution, adding a $tBL$ delay when issued to the same bank. Additionally, PIM_LdBuf and PIM_StBuf access the DRAM bank like PIM_Exec(Ld) and PIM_Exec(St), following the same timing as DRAM commands like ACT, PRE, Read, and Write.

Memory Ordering and Buffer Consistency. The bandwidth-decoupled transfer mechanism guarantees memory ordering and buffer consistency via strict execution sequencing. Decoupled command pairs (PIM_RdBuf/PIM_WrBuf and PIM_LdBuf/PIM_StBuf), derived from single CPU-mediated data transfers, must execute in program order, enforced by the memory scheduling strategy in Section 6. While other concurrent commands (PIM execution and CPU access) can be interleaved with the decoupled command pairs without ordering constraints since they never access the dedicated PIM buffer regions. These commands can arbitrarily interleave with the data transfer stream, even between decoupled pairs, without violating correctness.

The PIM scheduler dynamically chooses the best PIM command from PRWQ and PEQ to utilize idle time windows. In conventional PIM execution, depicted in Fig. 7(a), the process includes three sequential stages: input data transfer, PIM execution, and result collection, all managed by the CPU software sequentially. This sequential processing prevents simultaneous usage of both internal bandwidth (①) and external bandwidth (②).

To avoid this sequential execution and improve both internal and external bandwidth usage, we propose an overlapped scheduling strategy within the PIM scheduler. Unlike the traditional method where each stage must finish entirely before the next can begin, our strategy divides the PIM workload into loosely-coupled tiles (e.g., submatrices in matrix multiplication). This allows CPU-mediated transfers of one tile to coincide with the PIM execution of another, since there are no data dependencies between the tiles (the CPU handles the reduction). As shown in Fig. 7(b), the PIM scheduler issues compatible commands from different tiles simultaneously (e.g., collecting results for tile $T_{i}$ while executing tile $T_{i+1}$), creating an overlapped execution phase. This approach maximizes the use of both internal and external bandwidth, thereby enhancing the end-to-end performance of PIM workloads (③). To guarantee that data dependencies are satisfied, a PIM_Barrier command is introduced between the overlapped execution phases, ensuring that all operations of one phase are completed before the next phase starts.

During each overlapped execution phase, the PIM scheduler chooses a PIM command from the PRWQ or the PEQ, based on a priority policy. The top priority is to issue a PIM_RdBuf or PIM_WrBuf command from PRWQ, as these commands solely occupy the external memory bus, which is often the bottleneck. Hence, it is crucial to efficiently utilize such external bus idle times. If no bus command is ready, the scheduler gives precedence to PIM_LdBuf and PIM_StBuf to ensure that the data is promptly loaded from or stored into the PIM buffer, preventing the blockage of subsequent bus commands. Note that our scheduler preserves arrival order for PRWQ commands targeting the same bank, correctly handling any ordering requirements between different requests. Specifically, when the reserved buffer segment is full during writing, the next issued command is necessarily PIM_StBuf that clears the buffer. Only when no PRWQ commands can be sent will the scheduler opt for a PIM execution command from the PEQ that matches the current idle period of the bank.

Simulation. The performance of COSM and baselines is evaluated with the Ramulator2 simulator [ramulator2_10321647, ramulator2_github] (based on Ramulator [ramulator1_7063219]). The PIM commands and related timing constraints to the modeled LPDDR5 module. We extend the memory controller model with the additional modules described in Section 4. We use DRAMPower [drampower10.1145/3721848.3721850] to evaluate the power consumption of DRAM. Table 2 summarizes the modeled system configuration. The host CPU configuration is based on a mobile phone with a Qualcomm Snapdragon 888 [qualcomm_snapdragon888] and 32GB of DRAM memory. The PIM unit to DRAM bank bandwidth and energy consumption are modeled on a taped-out PIM chip [samsunglpddr5pim10254711]. We use the O3CPU front-end model of Ramulator2 [ramulator2_10321647, ramulator2_github] for CPU simulation, with the CPU workload input to the front-end as a memory trace. For CPU workloads from SPEC CPU2017 and PolyBench-ACC, we use the zsim simulator [zsim10.1145/2485922.2485963] to generate the memory traces. A memory trace contains the memory accesses and the corresponding latency between the accesses. For mobile phone applications, we use the Xiaomi Mi 11 Pro [xiaomi_mi11pro_specs] smartphone to run the workload and collect memory traces with instrumentation tools, Frida [frida_stalker]. The Xiaomi Mi 11 Pro is equipped with a Qualcomm Snapdragon 888 SoC [qualcomm_snapdragon888] (1 x 2.84 GHz Cortex-X1 + 3 x 2.42 GHz Cortex-A78 + 4 x 1.8 GHz Cortex-A55), 12GB LPDDR5 RAM [JEDECLPDDR52023], and 256GB storage, running Android 15 [android15_developer]. The DRAM configuration follows LPDDR5-6400 standards [JEDECLPDDR52023]. The CPU memory traces are repeatedly replayed to generate background DRAM traffic. To evaluate COSM’s area overhead, we synthesize the hardware modules in the memory controller using the Synopsys Design Compiler with TSMC 90nm technology library at the frequency of 2.4GHz. Since modern smartphone SoCs are typically fabricated in processes below 5nm, we provide a conservative area estimate for a 5nm implementation by scaling logic and memory components separately, based on publicly available technology scaling data [5nm8993577].

CPU Benchmarks. We evaluate COSM using six mobile applications, three PolyBench benchmarks [Polybench-acc6339595], and three SPEC CPU2017 [spec2017] specifically selected for their diverse memory access patterns. We use the following application workloads:

• •

  Tencent Meeting (TM): Video conference app; trace collected during whiteboard sharing.

• •

  Browser: Default mobile web browser; trace collected during page loading.

• •

  X: Social network; trace collected when clicking an article.

• •

  Note: Note-taking app; trace collected during typing.

• •

  YouTube: Video streaming platform; trace collected during video watching.

• •

  Music: System music streaming platform; trace collected during song playback.

PolyBench benchmarks show dense, contiguous memory access, leading to high memory bandwidth utilization and high row hit rates, while SPEC CPU2017 targets broader testing. Table 3 present the row hit rate of the evaluated benchmarks.

PIM Benchmarks. We tested inference on mobile devices using three open-source LLMs, each with around one billion parameters: BLOOM-1B1 [bigscience2022bloom], DeepSeek-R1-1.5B [deepseekai2025deepseekr1incentivizingreasoningcapability], and Qwen2-0.5B [qwen2]. BLOOM-1B1 is a multilingual model, DeepSeek-R1-1.5B specializes in programming, and Qwen2-0.5B is a compact bilingual model. Despite differences, all are suitable for mobile deployment. Benchmarks use 16-bit quantized inputs and 8-bit quantized weights.

Baselines. We compare COSM that use three baselines with different PIM control interfaces and scheduling strategies: All-Bank Command [HBM-PIM-HW9365862], Chopim [Chopim9138972], and AsyncDIMM-Bank, which is the bank-level PIM version of the original AsyncDIMM [AsyncDIMM10946818]. In All-Bank Command, PIM operations are issued using all-bank commands that simultaneously invoke all PIM units. Since CPU memory access is blocked after issuing the all-bank PIM commands, we introduce a time-sliced round-robin strategy: assume that 95% of the time is spent on CPU memory access, while the remaining 5% is allocated to PIM computation: we consider CPU performance degradation to be small enough as long as it is smaller than a 5% threshold. Crucially, to ensure a fair and conservative comparison, we assume idealized zero-overhead switching between PIM and CPU phases in this baseline. This isolates the impact of the memory interface architecture on scheduling potential. In contrast, Chopim and AsyncDIMM-Bank adopt single-bank commands with the PIM execution command length equal to $tBL$. Chopim adopts a CPU-first scheduling strategy, which prioritizes CPU memory accesses over PIM computation by blocking the PIM command queue whenever the CPU memory queue of a bank is not empty. AsyncDIMM-Bank adopts a relatively fair strategy to maximize the row hit rate by switching between the PIM command queue and CPU memory queue upon detecting a PRE command for either CPU or PIM unit. Moreover, it uses a relay memory controller within the rank to reduce command bus pressure. None of the baselines mentions their scheduling strategy of CPU-mediated data transfer. We assume that these memory accesses are scheduled together using the FR-FCFS policy. COSM adopts both the low-interference PIM control interface and the idleness-aware scheduling strategy. We set the PIM execution command length to 128 cycles, a value that balances the command bus contention (if too short) against excessive PIM_Pause commands and their bus bandwidth consumption (if too long), as observed in our experiments.

Fig. 8 shows the performance of COSM’s CPU and PIM benchmarks during simultaneous execution compared to the baselines. CPU performance is normalized to the performance obtained under standalone execution, which indicates the ideal case. The key-value cache is configured to a length of 2k. We also include a baseline called Chopim(128) by increasing the PIM execution command length of Chopim to 128 cycles. For the All-bank baseline, the PIM units are restricted to using only 5% of the time windows, missing the opportunity to take advantage of the idle periods between CPU memory accesses. The CPU-first approach of Chopim results in a small 3.0% slowdown for the CPU but only provides a relatively small 1.9× increase in PIM throughput over All-Bank, due to command-bandwidth contention. AsyncDIMM-Bank provides a 4.2× PIM throughput compared to All-bank, yet it significantly affects CPU performance, reducing it by an average of 89.9%. Although increasing the PIM execution command length to 128 cycles can increase the PIM throughput of Chopim by 3.44×, the CPU performance reduction increased to 13.5%, indicating the trade-off between PIM performance and CPU performance for fixed-length commands. In contrast, the scheduling strategy of COSM maintains CPU performance with only a 2.0% degradation. Additionally, the low-interference PIM control interface and idleness-aware scheduling enable COSM to more effectively use the remaining internal bandwidth for PIM tasks, leading to a 6.0× improvement over the All-Bank baseline and 2.8× over Chopim.

Fig. 9 illustrates the performance of CPU and PIM benchmarks during concurrent execution, comparing traditional fixed-length commands across different lengths (from 16 to 256), against our proposed preemptable PIM execution command. Both PIM and CPU performance is also normalized to performance of standalone execution. The maximum command length of 256 remains below the latency threshold for processing a complete row in our setup. The PIM workload corresponds to a KQV generation layer from the three models that is dominated by PIM execution. For fixed-length commands, there is a clear trade-off between CPU and PIM performance: longer commands enhance PIM workload throughput but negatively impact CPU performance. Notably, when the PIM execution command length surpasses 64 cycles, PIM performance reaches saturation in most cases due to decreased command bus pressure. This phenomenon is aligned to Fig. 2(b). However, CPU performance significantly declines by over 5%, particularly in mobile applications with random access patterns. In contrast, our preemptable interface can achieve maximum PIM performance without with only 3.2% degradation in CPU performance. Compared to a fixed command length of 32 cycles, which maintains CPU interference below 5% for all workloads, our preemptable PIM execution design results in a 2.02× improvement in PIM performance.

To assess how our techniques affect CPU-mediated transfers, we evaluate CPU performance normalized to the standalone execution, under four different configurations using attention layers from the three models that are characterized by substantial CPU-mediated transfer, as depicted in Fig. 10. To assess how our techniques affect CPU-mediated transfers, we evaluate the normalized CPU performance under four configurations using attention layers from three models characterized by substantial CPU-mediated transfers, as depicted in Fig. 10. In the Base configuration, CPU-mediated transfers occur through standard read/write requests, following the traditional three-stage PIM execution. Overlapped execution facilitates the simultaneous scheduling of CPU-mediated transfers and PIM execution commands by utilizing our scheduler design described in Section 6.2. This approach increases CPU performance by an average of 3.7% across mobile workloads compared to base. However, for workloads with high row-hit rates (such as algebraic kernels in PolyBench-ACC), this aggressive overlap can inadvertently cause additional row switches, leading to greater CPU interference. The Overlapped+Decoupled configuration further employs bandwidth-decoupled CPU-mediated PIM data transfer commands (Section 5.2), which reduces contention with CPU accesses and increase CPU performance by 11.5% compared to Base. Finally, the All configuration utilizes IWE to proactively detect memory bus idle time windows that are sufficiently large to accommodate entire burst transfers, thereby maintaining CPU slowdown consistently below 5% across all evaluated workloads.

In Fig. 11(a), we illustrate the performance of PIM workloads when managed by CPU-first scheduling compared to idleness-aware scheduling (Section 6) when running alongside with CPU workloads, with all results normalized to their standalone execution. The PIM workload is derived solely from the execution segment of an attention layer, excluding CPU-mediated data transfer, from the DeepSeek model. Under CPU-first scheduling, the issuance of PIM execution commands halts if a bank queue is not empty. In the idleness-aware configuration, idle time windows are predicted using IWE to enhance performance. This configuration provides an average performance boost of 1.21× for PIM workloads across mobile applications. Fig. 11(b) displays the bandwidth usage under both scheduling strategies. The term “available bandwidth” refers to idle time windows that are long enough for a PIM execution command but are not used due to scheduling limitations, whereas “unavailable bandwidth” pertains to short, fragmented idle periods unsuitable for PIM execution. Compared to CPU-first scheduling, Idleness-Aware scheduling exploits an additional 37.0% of the available bandwidth, leaving less than 1% unused.

Fig. 12 depicts energy consumption per token of COSM and baselines. We only calculate the energy of PIM computation, and exclude the energy of CPU access during concurrent execution. Although COSM’s fine-grained CPU-PIM task interleaving increases row switching overheads, it achieves a reduction in total energy by eliminating long active-idle periods. COSM reduces energy by 1.34×/1.61× compared to the AsyncDIMM-Bank/Chopim baseline.

PIM Execution Command Length (nPTL). Fig. 13(a) analyzes the sensitivity of COSM to the $nPTL$ timing constrain. Generally, smaller command length ($nPTL<32$) saturates the command bus, exacerbating contention between CPU and PIM tasks. However, moderately reducing $nPTL$ can yield benefits by decreasing PIM_Pause injections that potentially stall CPU accesses. For instance, in workload X, reducing $nPTL$ from 128 to 64 improves both CPU and PIM performance.

Rank Count per Channel. Fig. 13(b) analyzes the PIM performance of COSM to the rank count in each channel. Higher rank counts require the memory controller to issue more commands to trigger the execution of all PIM units, making the command bus easily saturated. Therefore, scaling $nPTL$ from 32 to 128 yields a higher speedup for 4 ranks (1.27×) than for 1 rank (1.22×).

KV Cache Size. Fig. 13(c) depicts the CPU performance degradation of COSM to the KV cache size of the DeepSeek attention layer, adopting the same series labels as Fig. 10. As the KV cache grows, CPU-mediated data transfer increasingly dominates the total computation time. This mitigates the performance drop in the Base case, which is highly sensitive to data transfer overhead. In contrast, COSM effectively manages both data transfer and PIM execution, maintaining CPU performance degradation below 5%.

tRP. Fig. 13(d) shows performance under varying $tRP$ (row precharge time) using a DeepSeek attention layer, excluding CPU-mediated data transfer. Under the current timing configuration (listed in Table 2), the CPU-PIM interference remains minimal. Since CPU and PIM requests typically map to different rows, a larger $tRP$ heightens the row-switch penalty between CPU access and PIM execution command. This amplifies interference and degrade both CPU and PIM performance.

Implemented in a Snapdragon 8-class LPDDR5 memory controller (estimated at 0.93 mm², measured from publicly available die photographs), the COSM hardware modules occupy 0.069 mm², resulting in a modest 7.4% area overhead. This includes 0.014 $mm^{2}$ for the PIM scheduler, 0.0085 $mm^{2}$ for the IWE, and 0.0054 $mm^{2}$ for the Command Arbiter.