Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning ^††thanks: The views expressed in this article are those of the authors and do not represent the views of Wells Fargo. This article is for informational purposes only. Nothing contained in this article should be construed as investment advice. Wells Fargo makes no express or implied warranties and expressly disclaims all legal, tax, and accounting implications related to this article.

Quantum computing (QC) provides new computational paradigms that may offer advantages for certain problems [56, 65, 30]. Combined with the success of artificial intelligence and machine learning (AI/ML), this has motivated the development of quantum machine learning (QML), which seeks to design learning models based on quantum computational principles [5, 23, 27, 63, 3, 1, 67]. A central framework in QML is the variational quantum algorithm (VQA) [4, 7, 57, 74], which has been widely used in hybrid quantum-classical models for classification, sequence learning, and reinforcement learning [54, 62, 19, 18, 35, 51, 52, 55, 43, 14, 12, 50, 47, 9, 26, 68, 38, 41, 42, 64].

Sequence learning is an important direction in QML because many real-world data are naturally temporal. Existing quantum recurrent models, such as Quantum Long Short-Term Memory (QLSTM) networks, have shown that parameterized quantum circuits can be used to process sequential information [19, 44, 28, 69, 70, 2, 66, 45, 73, 32, 48, 46, 10]. However, many recurrent quantum models rely on fixed temporal update structures, leaving open how quantum architectures should regulate the interaction between newly generated information and memory accumulated from previous time steps.

Quantum Fast Weight Programmers (QFWPs) provide an alternative approach to quantum sequence modeling by dynamically generating fast-weight parameters during recurrent processing [22]. This mechanism allows the model to encode temporal information into evolving fast-weight states. Nevertheless, directly accumulating fast-weight updates may cause unstable parameter evolution or inefficient memory propagation, especially when the input sequence length or model size changes. Therefore, an explicit mechanism for controlling both new fast-weight updates and historical fast-weight memory is needed.

To address this issue, we propose Self-Modulating Quantum Fast Weight Programmers, or Self-Modulating QFWP. The key idea is to introduce adaptive modulation into the fast-weight evolution, allowing the model to regulate how much newly generated information is injected and how much historical fast-weight memory is retained. This self-modulating mechanism provides a more flexible recurrent structure and aims to improve learning robustness across different quantum model settings.

Contribution Statement. The main contributions of this work are threefold. First, we propose a self-modulating extension of QFWP that adaptively regulates fast-weight dynamics for more robust sequence learning. Second, we conduct comprehensive numerical simulations and ablation studies on multiple time-series tasks, evaluating the effects of different numbers of qubits, input sequence lengths, and modulation designs. Third, we provide theoretical arguments explaining why certain forms of self-modulation more effectively balance new information injection and memory retention, thereby improving temporal information propagation.

Sequence learning is a central topic in both classical and quantum machine learning. In the classical setting, the long short-term memory (LSTM) network [31] has long served as the canonical recurrent architecture, using input, forget, and output gates to mitigate vanishing gradients and capture long-range dependencies. Building on this design, the Quantum LSTM (QLSTM) [19] replaces the classical components inside the LSTM cell with variational quantum circuits (VQCs), and has since been applied across natural language processing [44, 28, 69], generative modeling [25], reinforcement learning [20, 21], time-series forecasting [6, 8], solving partial differential equations [24], and many other domains [72, 29, 71, 11, 75, 13, 34, 33, 58]. Recent QKAN-LSTM work further explores quantum-inspired Kolmogorov–Arnold recurrent gates to achieve better scalability [34, 39]. However, QLSTM inherits the structural drawbacks of recurrent designs: its non-linear gated recurrence couples the hidden state across all time steps, forcing gradient computation to proceed sequentially through backpropagation-through-time (BPTT) and precluding parallelization across time steps. This sequential cost is amplified in the quantum setting, where each step requires repeated VQC evaluations for gradient estimation.

The Fast Weight Programmer (FWP) framework [60, 61] addresses sequence learning through a different mechanism: a slow network generates low-rank parameter updates that reprogram a fast network, yielding a linear recurrence in the fast weights. This formulation was later shown to be equivalent to linearized self-attention [59, 40], and extended through recurrent and self-referential variants [37, 36]. The Quantum FWP (QFWP) [22] adapts this paradigm to the hybrid setting, with a classical slow programmer continually reprogramming a VQC fast programmer. Because temporal dependence is encoded through additive parameter updates rather than a recurrent quantum state, the QFWP avoids deep gradient evaluation such as BPTT through the quantum circuit. Subsequent variants such as the Quantum-Train QFWP [49] and observable-aware extensions [17] further reduce parameters and broaden applicability. However, these models share a purely additive update rule that lacks any input-dependent mechanism to forget or reweight accumulated information, in contrast to QLSTM. To close this gap, the Self-Modulating QFWP introduces input-dependent modulation of both the new update and the previously accumulated fast weights. Because the modulation coefficients depend only on the current input, the recurrence remains linear and parallelizable [53], thereby inheriting the temporal-memory control of LSTM-style temporal-memory control without the sequential gradient bottleneck of non-linear recurrence.

A quantum neural network (QNN), also known as variational quantum circuit (VQC) or parameterized quantum circuit (PQC), consists of a data-encoding circuit $U(\vec{x})$, a trainable variational circuit $W(\Theta)$, and a final measurement stage. Given a classical input $\vec{x}$, the quantum state is prepared as $\ket{\Psi}=W(\Theta)U(\vec{x})\ket{0}^{\otimes n},$ where $n$ is the number of qubits and $\Theta$ denotes the trainable circuit parameters. The model output is obtained by measuring Hermitian observables $\hat{B}_{k}$, yielding expectation values $\langle\hat{B}_{k}\rangle=\bra{\Psi}\hat{B}_{k}\ket{\Psi}.$ Therefore, the QNN defines a quantum function $\vec{f}(\vec{x};\Theta)=(\langle\hat{B}_{1}\rangle,\dots,\langle\hat{B}_{K}\rangle)$.

Let $L$ be the number of trainable variational layers in the fast quantum circuit and $Q$ be the number of qubits. At time step $t$, the fast circuit state is $\Theta_{t}\in\mathbb{R}^{L\times Q}$, where $(\Theta_{t})_{k,q}$ is the circuit angle assigned to layer $k\in\{1,\ldots,L\}$ and qubit $q\in\{1,\ldots,Q\}$.

Denote $\Omega$ as the collection of trainable parameters of the classical slow controller. These parameters are optimized during training and remain fixed during inference. By contrast, $\Theta_{t}$, $\Delta_{t}$, $M_{t}^{\mathrm{new}}$, and $M_{t}^{\mathrm{old}}$ are time-dependent quantities generated from the input sequence.

Let $h_{t}=\phi_{\Omega_{\phi}}(x_{t})\in\mathbb{R}^{H}$ be the hidden state produced by the classical controller $\phi_{\Omega_{\phi}}$ from the input $x_{t}$. The raw update vectors are $\ell_{t}=W_{\ell}\,h_{t}+b_{\ell}\in\mathbb{R}^{L}$, $r_{t}=W_{r}\,h_{t}+b_{r}\in\mathbb{R}^{Q}$ and the raw fast-weight update is the rank-1 matrix $\Delta_{t}=\ell_{t}\cdot r_{t}^{\top}\in\mathbb{R}^{L\times Q}$. Then the original QFWP [22] uses the update rule,

Thus, the fast circuit state is obtained by accumulating all previous raw updates. This makes the original QFWP a quantum recurrent fast-weight model: the classical slow programmer generates a low-rank update from the current input, and the quantum circuit uses the accumulated fast parameters to produce quantum expectation-value features.

We evaluate the performance of the proposed Self-Modulating-QFWP model under the following four configurations: (1) Self-Modulating-QFWP — this is the variant in which both new and old parameters are modulated; (2) Self-Modulating-QFWP (Only Old) — this is the variant in which only old parameters are modulated; (3) Self-Modulating-QFWP (Only New) — this is the variant in which only new parameters are modulated; (4) QFWP (Standard) — this is the original QFWP in which no parameter modulation is implemented.

To compare the performance difference across various model sizes and qubit counts, we consider and train the following settings for hidden size, which is also the number of qubits of VQCs in the fast programmer: 4, 6 ,8, 10, 12, 14. The training and evaluation protocol follows the methodology described in [19, 15, 22, 16]. Specifically, the model is trained to predict the $(N+1)$-th value in a sequence given the preceding $N$ observations. For example, at time step $t$, the input to the model is $[x_{t-4},x_{t-3},x_{t-2},x_{t-1}]$ (with $N=4$), and the target output is $y_{t}$, which should approximate the ground truth $x_{t}$. To investigate the memory performance of the proposed model across various sequence length, we consider $N=\{4,8,16,32,64\}$. For each time-series task, each model configuration is evaluated over $6\times 5=30$ combinations, obtained from six hidden-size/qubit-count settings $H\in\{4,6,8,10,12,14\}$ and five input sequence lengths $N\in\{4,8,16,32,64\}$.

We consider five time-series tasks which are used as benchmarks in previous studies [19, 15, 22, 16]: Damped SHM, Bessel function $J_{2}$, Delayed Quantum Control, NARMA-5, and NARMA-10. All simulations use batch_size=4, number of QNN layers $L=5$, learning_rate=$10^{-3}$ and the Adam optimizer.

In addition to the standard metric such as mean square error (MSE), we consider the following metrics to better capture the learning performance of different variants in our proposed Self-Modulating QFWP models.

• •

  Relative Improvement:

  $$\Delta^{\mathrm{rel}}=\frac{M_{\mathrm{standard}}-M_{\mathrm{variant}}}{M_{\mathrm{standard}}+\varepsilon}.$$

  (4)

  This quantity measures how much the variant improves or degrades the metric $M$ relative to the standard baseline, while using $\varepsilon$ to prevent numerical instability when the denominator is small. When $M_{\mathrm{variant}}<M_{\mathrm{standard}}$, the numerator is positive, so $\Delta^{\mathrm{rel}}>0$. This means that the variant achieves a smaller value of the metric and therefore performs better. Accordingly,

  • –

    $\Delta^{\mathrm{rel}}>0$: variant better

  • –

    $\Delta^{\mathrm{rel}}<0$: variant worse

  • –

    $\Delta^{\mathrm{rel}}\approx 0$: both perform similarly

• •

  Relative Strength: We define Relative Strength to quantify whether old-parameter modulation or new-parameter modulation contributes more strongly to performance improvement. Specifically, Relative Strength is defined as the difference between the relative improvements of the Only-Old-Modulated and Only-New-Modulated variants:

  $$\mathrm{Relative\ Strength}=\Delta_{\mathrm{old}}^{\mathrm{rel}}-\Delta_{\mathrm{new}}^{\mathrm{rel}}.$$

  (5)

  A positive Relative Strength value indicates that old-parameter modulation contributes more strongly than new-parameter modulation, while a negative value indicates the opposite. Values close to zero suggest that the two modulation pathways have comparable effects.

• •

  Synergy: We further define Synergy to measure whether jointly modulating both old and new parameters provides an additional benefit beyond the better single-sided modulation strategy. Formally, Synergy is defined as

  $$\mathrm{Synergy}=\Delta_{\mathrm{both}}^{\mathrm{rel}}-\max\left(\Delta_{\mathrm{old}}^{\mathrm{rel}},\Delta_{\mathrm{new}}^{\mathrm{rel}}\right).$$

  (6)

  A positive Synergy value indicates that jointly modulating both old and new parameters outperforms the better of the two single-sided modulation variants, suggesting a genuine cooperative effect. A value near zero indicates little additional gain beyond the stronger single-sided modulation, while a negative value suggests that the joint modulation strategy does not surpass the better individual component.

For the bessel_j2 task with sequence length 32, as shown in Figure 2, the Self-Modulating QFWP exhibits a clearly faster convergence behavior than the standard QFWP. At epoch 1, the standard model still produces an almost flattened response and fails to capture the oscillatory structure of the target sequence, whereas the Self-Modulating variant already aligns reasonably well with the underlying periodicity, phase, and damping trend. By epoch 15, the prediction of the Self-Modulating QFWP nearly overlaps with the ground truth, while the standard QFWP still shows visible amplitude and local shape mismatches. This advantage remains evident in both the training and testing regions, indicating that self-modulation improves not only fitting speed but also early-stage generalization. Although both models eventually achieve strong prediction quality by epoch 100, the main benefit of the Self-Modulating QFWP in this task lies in its substantially faster and more stable learning of the correct temporal dynamics.

Figure 3 shows representative test MSE convergence curves for the bessel_j2 task under several hidden-size (qubit-count) and sequence-length settings. In the easiest setting (hidden size = 4, sequence length = 4), all four variants eventually converge to nearly zero test error, although the modulated variants exhibit faster and smoother early-stage optimization. As the task becomes more challenging, especially at sequence lengths 16 and 64, the differences between methods become substantially more pronounced. The standard QFWP consistently maintains higher test MSE and exhibits noticeable fluctuations throughout training. The Only New Modulated variant improves over the standard model, but typically settles at a visibly higher error floor. In contrast, both the Self-Modulating QFWP and the Only Old Modulated variant rapidly drive the test MSE to near zero and maintain much more stable convergence trajectories, with Only Old Modulated appearing particularly stable across the harder settings. These results suggest that, for oscillatory sequence modeling in bessel_j2, modulation based on historical or old-state information is likely the dominant contributor to the performance gain, while full self-modulation provides similarly strong and robust optimization behavior.

The final-test-MSE heatmaps (Figure 4) further consolidate the trends suggested by the epoch-wise prediction plots and representative convergence curves. For the bessel_j2 task, the standard QFWP maintains low error only in shorter-sequence settings (e.g., sequence lengths 4 and 8), but its performance degrades substantially as the sequence length increases to 16, 32, and 64. In contrast, the Self-Modulating QFWP remains consistently strong across the entire grid of hidden sizes and sequence lengths, with final test errors staying in a uniformly low range. Even more notably, the Only Old Modulated variant appears to be the strongest overall configuration on this task, with most entries remaining at the $10^{-6}$ to $10^{-5}$ level and showing even greater consistency than the full Self-Modulating QFWP. By comparison, the Only New Modulated variant performs reasonably well for short sequences, but deteriorates markedly for longer ones, exhibiting a failure pattern much closer to the standard QFWP. Taken together, these results strongly suggest that, for the oscillatory and history-dependent dynamics of bessel_j2, old-state modulation is the dominant source of performance improvement, while full self-modulation preserves this advantage and provides robust performance across configurations.

The relative-improvement heatmaps (Figure 5) provide a complementary, ratio-based view of the gains achieved by each variant over the standard QFWP. Using the metric defined earlier (Equation 4), where positive values indicate lower MSE than the standard baseline, both the Self-Modulating and Only Old variants exhibit strong positive improvements across most settings on the bessel_j2 task. In particular, for sequence lengths 16, 32, and 64, their relative-improvement values are almost uniformly close to 1, indicating highly significant gains over the standard QFWP. By contrast, the Only New variant shows much weaker and less consistent improvements, with many entries remaining near zero or even becoming negative, suggesting that new-only modulation is insufficient to reproduce the benefits of old-related modulation. A few isolated large negative values appear in the Self-Modulating heatmap; however, these cases mainly arise because the standard QFWP already attains an extremely small final test MSE in those settings, making the ratio-based metric highly sensitive to tiny absolute differences. In practice, the modulating variants in these cases also converge successfully, and their prediction curves are often visually almost indistinguishable from those of the standard model. Therefore, such isolated negative entries should not be interpreted as failures of the modulating variants, but rather read together with the raw final-test-MSE heatmaps and prediction plots. Overall, these results are consistent with the preceding analyses and further support the conclusion that old-state modulation is the dominant source of performance gain for the bessel_j2 task, while full self-modulation preserves this advantage across most configurations.

For the damped_shm task with sequence length 32 (shown in Figure 6), the Self-Modulating QFWP clearly outperforms the standard QFWP. By around epoch 15, the Self-Modulating model already produces predictions that nearly overlap with the ground truth and continues to track the damped oscillatory trajectory accurately in both the training and testing regions. In contrast, the standard QFWP still exhibits noticeable shape and amplitude mismatches even at epoch 100, indicating that it fails to fully capture the underlying temporal dynamics. Overall, this task again supports the benefit of self-modulation for sequence modeling, with an even more pronounced advantage than in bessel_j2.

The representative test MSE convergence curves for damped_shm (Figure 7) show a pattern consistent with, but even stronger than, that of bessel_j2. The standard QFWP maintains substantially higher and more fluctuating test error in the medium-to-long sequence settings, whereas both the Self-Modulating QFWP and the Only Old Modulated variant rapidly converge to near-zero test MSE and remain stable thereafter. The Only-New variant improves over the standard model, but still lags behind the former two. This indicates that damped_shm is especially sensitive to old-state modulation, which directly improves the stability of temporal memory.

The final-test-MSE heatmaps in Figure 8 confirm the same trend across the full configuration grid. The standard QFWP deteriorates substantially in several medium-to-long sequence settings, while the Self-Modulating QFWP remains low-error and stable across hidden sizes and sequence lengths. The Only-Old variant is again among the most consistent performers, whereas the Only-New variant provides partial improvement but remains less robust. This reinforces the conclusion that the dominant gain in damped_shm comes from old-related modulation.

The relative-improvement heatmaps in Figure 9 further show that both the Self-Modulating and Only-Old variants achieve strong positive gains over the standard QFWP across most settings, with many values approaching 1 in the medium-to-long sequence regime. The Only-New variant is less consistent and still contains several near-zero or negative regions. As in the bessel_j2 case, isolated negative values should be interpreted together with the raw MSE heatmaps, since the ratio-based metric can amplify tiny absolute differences when the baseline error is already extremely small.

For the delayed_quantum_control task with sequence length 64 (shown in Figure 10), the contrast between the two models is nearly binary. The Self-Modulating QFWP reproduces the pulse-like decaying target dynamics almost perfectly from very early epochs and remains highly accurate in both the training and testing regions. In contrast, the standard QFWP fails to learn the correct temporal structure throughout training and still exhibits clearly mismatched low-amplitude oscillatory predictions even at epoch 100. This suggests that, in this task, self-modulation does not merely improve convergence speed, but is crucial for successfully learning the delayed temporal dynamics.

The convergence curves in Figure 11 show that old-related modulation is highly effective for this task. Both the Self-Modulating and Only-Old variants remain near zero and highly stable from early training, whereas the standard QFWP exhibits substantially larger errors and stronger fluctuations in medium-to-long sequence settings. The Only-New variant also improves over the standard model, but remains less stable than the Self-Modulating and Only-Old variants. These results indicate that old-state modulation almost eliminates the original optimization difficulty in delayed_quantum_control.

The final-test-MSE heatmaps for delayed_quantum_control (Figure 12) reveal an even sharper separation than in the previous tasks. The standard QFWP exhibits several clearly failed medium-to-long sequence settings with markedly elevated final test MSE, indicating limited ability to model the delayed temporal structure. In contrast, the Self-Modulating QFWP maintains uniformly low and stable error across almost the entire hidden-size and sequence-length grid. The Only-Old variant performs comparably well and shows similarly consistent low-error behavior, while the Only-New variant, although better than the standard model, still retains a few higher-error configurations in harder settings. Overall, these results again suggest that the dominant gain in delayed_quantum_control comes from old-related modulation, while full self-modulation preserves this advantage across the configuration space.

The relative-improvement heatmaps in Figure 13 are consistent with the raw MSE results. Both the Self-Modulating and Only-Old variants achieve strong positive gains over the standard QFWP across most settings, especially in the longer-sequence regime. The Only-New variant also improves over the standard baseline, but its gains are generally less uniform. Overall, these results again support the conclusion that the dominant gain in delayed_quantum_control comes from old-related modulation, while full self-modulation preserves this advantage across the configuration space.

We next examine the NARMA family—narma_5 (sequence length 32) and narma_10 (sequence length 64)—as progressively harder autoregressive memory tasks. The same overall ordering holds in both: Self-Modulating $\approx$ Only-Old $>$ Only-New $>$ standard QFWP, visible across rollouts (Figure 14, Figure 18), convergence curves (Figure 15, Figure 19), final-MSE heatmaps (Figure 16, Figure 20), and relative-improvement heatmaps (Figure 17, Figure 21).

For narma_5 the standard QFWP captures the coarse trend but smooths out local fluctuations and peak–valley structure; in convergence it eventually reaches low error but with greater instability and occasional spikes in the harder settings, while Self-Modulating and Only-Old stay smooth and low. The same ordering becomes more pronounced for narma_10: the standard model is more clearly underfit in the rollouts, fluctuates more sharply in the convergence curves, and degrades more visibly in the heatmaps, while full Self-Modulating and Only-Old variants converge smoothly and remain near-zero loss. The widening gap from narma_5 to narma_10 indicates that old-related modulation becomes increasingly important as autoregressive memory demand grows.

The convergence curves show that all variants can eventually reach relatively low test error, but the standard QFWP exhibits greater instability and occasional spikes in harder settings, whereas the Self-Modulating and Only-Old variants remain more stable overall (Figure 15). The final-test-MSE and relative-improvement heatmaps further confirm this ranking: Self-Modulating and Only-Old achieve the most consistent gains, Only-New provides intermediate improvement, and the standard QFWP is the least robust, especially at longer sequence lengths (Figure 16 and Figure 17).

The same ordering becomes more pronounced for narma_10 with sequence length 64. In the epoch-wise prediction comparison, the standard QFWP remains more underfit and less responsive to local variations, whereas the Self-Modulating variant tracks the target sequence more faithfully from mid training onward (Figure 18). This stronger gap is also visible in the convergence curves, where the standard model exhibits larger fluctuations and sharper spikes in harder settings, while the Self-Modulating and Only-Old variants converge more smoothly and maintain lower test MSE (Figure 19).

The final-test-MSE and relative-improvement heatmaps again identify Self-Modulating and Only-Old as the most reliable configurations across hidden sizes and sequence lengths, with Only-New remaining intermediate and the standard QFWP showing the most obvious degradation (Figure 20 and Figure 21). Compared with narma_5, the stronger separation in narma_10 suggests that old-related modulation becomes increasingly important as the autoregressive memory demand increases.

The task-wise mean final test MSE (Figure 23) confirms the same ranking at the aggregate level across all five benchmarks: standard QFWP is weakest, Only-New gives partial improvement, and full Self-Modulating and the Only-Old variant are strongest. The advantage is most dramatic on the oscillatory and delayed-control tasks, where full Self-Modulating and Only-Old reduce mean MSE by orders of magnitude; on NARMA absolute errors are smaller but the ordering is unchanged. The gain from self-modulation is therefore broadly task-independent.

The Relative Strength and Synergy heatmaps (Figure 22, Equation 5, Equation 6) interpret these gains mechanistically. Recall that positive Relative Strength means old-state modulation contributes more than new-state modulation, while positive Synergy means the full model exceeds the better single-sided variant. Across nearly all tasks and configurations, Relative Strength is predominantly positive—often strongly so—confirming that the gain is dominated by old-related modulation. By contrast, Synergy values are frequently near zero or negative, especially on bessel_j2 and damped_shm, indicating that the full Self-Modulating model rarely benefits from constructive old/new interaction. Its performance is largely explained by the dominant old-state effect, with new modulation providing only limited or task-dependent additional benefit. NARMA shows a milder, mixed pattern but the same direction. Together with the per-task results, these aggregate views support the conclusion that old-state modulation is the primary driver of improvement, and full self-modulation mainly serves as a robust wrapper preserving this benefit across diverse tasks.

Here we provide an explanation of why the Only-Old model can achieve performance comparable to the full Self-Modulating QFWP.

In this work, we proposed Self-Modulating QFWP, a compact quantum fast-weight framework that adaptively regulates both newly generated updates and accumulated fast-weight memory. Across five time-series benchmarks and extensive hidden-size/sequence-length settings, the proposed model consistently improves convergence stability and predictive accuracy over standard QFWP. Our ablation studies further reveal that old-state modulation is the dominant source of improvement, indicating that effective control of historical quantum fast weights is central to robust sequential learning. These results position self-modulation as a simple but powerful mechanism for building adaptive and scalable quantum sequence models.