The Memorization Plateau: Grokking and Double Descent Emerge in Two-Qubit Quantum Neural Networks

For years, the quantum machine learning conversation has orbited a single anxiety: trainability. Barren plateaus, vanishing gradients, and the cost of every circuit evaluation on noisy hardware dominate conference panels and funding proposals. A quieter question — what happens after a variational circuit has already memorized its training set — has received far less attention.

That is starting to change. In a preprint posted to arXiv on July 9, 2026, researchers at Fraunhofer IAO and the University of Stuttgart report that a minimal two-qubit quantum neural network can exhibit grokking: the delayed transition from memorization to generalization that has fascinated classical deep learning since Power et al. first documented it in 2022. The same circuits also show epoch-wise double descent, where test error worsens at a critical training epoch before recovering into a generalizing state.

The finding does not promise near-term quantum advantage on industrial datasets. It does, however, suggest that the training dynamics of parameterized quantum circuits may be richer — and more structurally similar to classical overparameterized models — than the barren-plateau narrative alone would imply.

A controlled experiment on the SU(4) manifold

Daniel Pranjić, Marco Roth, and Christian Tutschku deliberately chose the smallest architecture that could isolate the phenomenon. Their quantum neural network operates on two qubits with a complete parameterization of the SU(4) gate manifold: 15 trainable weights per layer, embedded through a padded 15-dimensional feature map that keeps the data encoding linear while granting full access to the two-qubit Hilbert space.

The classification task is intentionally demanding. Training points are sampled from two concentric circles in two dimensions — a nonlinear boundary that quantum learners cannot solve with a trivial linear separator. Test points are placed along the margin of a hard-margin support vector machine with a quadratic kernel, creating a diagnostic where even small weight shifts that preserve zero training loss can flip predictions at the boundary.

Optimization runs through AdamW on a mean-squared-error objective with optional weight-norm regularization, for up to 10⁵ epochs. The setup is simulation-only; no claims are made about hardware execution. But the controlled geometry makes the dynamics legible in a way that larger, noisier experiments rarely permit.

Three phases: comprehension, memorization, grokking

In a representative deep-circuit run, the authors trace a characteristic trajectory. Training error collapses quickly — by epoch 184, the model has effectively fit the training set. Test error drops briefly, then stalls: the circuit enters a memorization plateau where empirical loss is flat but internal weights continue to drift.

That plateau is not idle. The team tracks leave-one-out hypothesis stability, a proxy for algorithmic stability, and finds it shifting throughout the memorization phase even while loss is unchanged. Around epoch 3,418, the run exits memorization. Roughly two thousand epochs later, at epoch 5,477, test error collapses again — the grokking transition.

Crucially, this is not a monotonic improvement. Test error undergoes epoch-wise double descent: it improves, degrades at a critical epoch, then improves again into generalization. The pattern mirrors classical grokking curves, but the mechanism unfolds on a compact, periodic quantum gate manifold rather than in flat Euclidean weight space.

To explain the transition, the authors decompose the network output into Fourier coefficients over the two-dimensional input domain. Before grokking, weight magnitudes are diffuse and phase arguments are chaotic. After the transition, the spectrum organizes into a structured, ring-like pattern consistent with an Airy diffraction profile — evidence that the circuit has discovered a simpler, phase-aligned harmonic representation rather than a brittle memorized boundary.

Depth is not optional

Statistical sweeps across 128 independent runs per layer count reveal a sharp architectural threshold. For circuits with fewer than three layers, grokking does not appear within the 10⁵-epoch training horizon. Shallow models either generalize immediately or fail outright.

As depth increases, both the frequency of grokking and the rate of successful generalization rise. At seven or more layers, the failure modes associated with unlucky random initializations disappear entirely: all 128 runs converge to a generalizing solution. Overparameterization via circuit depth, in other words, does not merely add capacity — it smooths the optimization landscape enough that memorization reliably gives way to generalization.

This directly challenges a piece of conventional quantum ML wisdom. Barren plateau theory often recommends narrow parameter initializations to preserve gradient magnitude. In the overparameterized deep-circuit regime studied here, broad initializations paired with strong weight decay do not trap models in plateau forever. They appear to accelerate the structured search that precedes grokking.

The decay after the breakthrough

The paper's most practically relevant finding may be what happens after grokking. In late-stage training, the authors identify a generalization decay: test error climbs again even while training loss remains at zero. The culprit, they argue, is an unconstrained increase in weight-norm — the circuit drifts along the zero-training-loss manifold toward overfitted solutions in Hilbert space, away from the sparse, phase-aligned representations that grokking discovered.

Bridging this behavior to algorithmic stability theory, they show that the decay correlates with rising hypothesis instability. The Lipschitz bound on the quantum model — which depends only on the two data-coupled parameters per layer in this architecture — provides a formal link between weight accumulation and worsening generalization gaps.

The proposed fix is straightforward: explicit weight-norm regularization in the loss function. A weak penalty anchors the circuit in the post-grokking regime and permanently preserves generalization gains. For anyone training variational circuits on simulators today — and eventually on hardware — the implication is clear: stopping at zero training loss is not stopping at a stable solution.

A broader pattern: double descent across quantum models

The Fraunhofer–Stuttgart grokking study arrives alongside complementary work on double descent in quantum machine learning more broadly.

In a separate preprint posted July 23, 2026 — arXiv:2607.21409 — Marie Kempkes and collaborators at Leiden University, Freie Universität Berlin, and industry partners show that gradient-trained data re-uploading parameterized quantum circuits exhibit the classic double-descent profile as parameter count crosses the interpolation threshold. Test loss peaks near p = NK (parameters equal to training samples times output dimension), then falls again in the overparameterized regime. Experiments on MNIST-1D, Fashion MNIST, and synthetic regression tasks with eight-qubit circuits support the theory, which draws on random matrix theory and add-one-in perturbation analysis.

That paper explicitly cites the grokking preprint as evidence of non-monotonic test-error dynamics during training — the two results describe different axes of the same phenomenon. Model-size double descent asks whether deeper circuits generalize better; epoch-wise double descent asks whether longer training eventually finds those solutions after memorization.

Earlier work on quantum kernel ridge regression (arXiv:2604.17202, April 2026) had already established that interpolation peaks appear in kernelized quantum models under random matrix theory. Together, the three lines of research sketch an emerging picture: quantum learners are not exempt from the generalization paradoxes that reshaped classical machine learning after the interpolation threshold was mapped.

What this does — and does not — change

These results are preprints, not peer-reviewed theorems tied to hardware benchmarks. The grokking experiments run on a two-qubit toy architecture with 100 training and 100 test points. The double-descent scaling studies use eight-qubit re-uploading circuits on curated datasets. None of this resolves whether quantum machine learning will outperform classical baselines on problems that matter commercially.

What it does change is the research agenda. Trainability and generalization can no longer be treated as sequential checkpoints — first escape the barren plateau, then worry about test error. The post-convergence dynamics on the zero-loss manifold may determine whether a circuit retains whatever generalization grokking briefly unlocked.

For hardware teams, the near-term lesson is operational: variational training schedules should budget for extended epoch counts in overparameterized regimes, monitor weight-norm and stability metrics after training loss saturates, and treat regularization as structural rather than cosmetic. For theorists, the SU(4) Fourier analysis offers a concrete bridge between quantum circuit geometry and the harmonic-solution frameworks that classical grokking research has been converging toward.

The quantum computing industry spent the last half-decade asking whether its models could learn at all. The next question is whether they can stay learned — and the first rigorous answers are arriving, thousands of epochs after the loss curve went flat.

Sources

• arXiv — Grokking and epoch-wise double descent in quantum neural networks (2026-07-09)
• arXiv — Cautious optimism for deep parameterized quantum circuits (2026-07-23)
• arXiv — Double Descent in Quantum Kernel Ridge Regression (2026-04-19)
• arXiv — Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets (2022-01-06)