What Do Lorentz-Equivariant Jet Taggers Learn?

The geometric algebra over Minkowski spacetime, with metric signature $(+,-,-,-)$, is a $16$-dimensional associative algebra generated by four basis vectors $e_{0},e_{1},e_{2},e_{3}$ satisfying $e_{0}^{2}=+1$, $e_{i}^{2}=-1$ for $i=1,2,3$, and $e_{\mu}e_{\nu}=-e_{\nu}e_{\mu}$ for $\mu\neq\nu$. Elements are called multivectors and decompose into five grades:

• •

  Grade 0 (scalar, 1 component): Lorentz-invariant by construction.

• •

  Grade 1 (vectors, 4 components): transform as 4-vectors under $\mathrm{SO}^{+}(1,3)$.

• •

  Grade 2 (bivectors, 6 components): oriented spacetime planes.

• •

  Grade 3 (trivectors, 4 components): Hodge-dual to G1 vectors via the pseudoscalar $I$.

• •

  Grade 4 (pseudoscalar, 1 component): parity-odd invariant.

We write grade $k$ as G$k$ throughout the paper; for example, G2 denotes bivectors.

The geometric product of two multivectors mixes grades in a way determined by the algebra; for instance, the product of two G2 elements produces G0 and G4 components, while a G2 element times a G1 element produces G1 and G3 components.

L-GATr (Spinner et al., 2024) adapts the Geometric Algebra Transformer framework (Brehmer et al., 2023) from Euclidean to Minkowski spacetime. It represents each jet constituent as a multivector token encoding its 4-momentum, supplemented by beam spurion tokens (a lightlike vector along the beam axis and a time reference) that break the remaining boost symmetry as a fixed reference frame. The network consists of $N_{\mathrm{blocks}}$ GATrBlocks, each applying equilinear transformations (linear maps that commute with the Lorentz group action) and geometric attention over multivector tokens. Each block produces multivector channels $h_{\mathrm{mv}}\in\mathbb{R}^{N\times C_{\mathrm{mv}}\times 16}$ and scalar channels $h_{\mathrm{s}}\in\mathbb{R}^{N\times C_{\mathrm{s}}}$.

The output projection layer is equilinear: it maps grade $k$ inputs to grade $k$ outputs only, with no grade mixing. Since the classification logit is a scalar (G0), the output layer reads only from the G0 component of $h_{\mathrm{mv}}$ at the final layer. This architectural constraint implies that all non-scalar components at the final layer are architecturally inaccessible to the output, regardless of their information content. We use this fact explicitly in interpreting the layer-resolved ablations.

The scalar channels $h_{\mathrm{s}}$ are Lorentz-invariant by construction.

By default L-GATr enforces equivariance to the connected, proper orthochronous Lorentz subgroup $\mathrm{SO}^{+}(1,3)$. Under this subgroup, G0 and G4 carry scalar-type representations, while G1 and G3 carry equivalent vector-type representations via Hodge duality; the implementation therefore uses the mixed groups G0+G4 and G1+G3. An alternative setting that keeps all five grades strictly independent is studied as a side check in Appendix E.3.

L-GATr-slim (Petitjean et al., 2025) is a compact variant that retains only scalars (G0) and 4-vectors (G1) internally, explicitly dropping the outer product and G2, G3, and G4 channels. It matches L-GATr’s task AUC while being $6\times$ faster (Petitjean et al., 2025).

LLoCa-T (Spinner et al., 2025) is designed to enforce Lorentz equivariance through local canonicalization rather than through multivector-valued layers. It maps each jet to a canonical Lorentz frame and applies a standard transformer. Since the classification output is a Lorentz scalar, no inverse frame map is required. By design LLoCa-T is exactly Lorentz-equivariant: the canonicalization is itself an equivariant operation, so the full pipeline inherits exact equivariance. However, the canonicalization step is numerically more sensitive than the architectural approach of L-GATr, which likely accounts for the larger measured errors in our evaluation. LLoCa-T uses no multivector representation; its hidden state is a flat vector, so grade decomposition does not apply directly.

TopTagging (Butter et al., 2018; Kasieczka et al., 2019) is a binary jet tagging benchmark with 1.2M training, 400k validation and 400k test jets at $\sqrt{s}=14\,\mathrm{TeV}$. Signal jets arise from hadronic top quark decays $t\to Wb\to qqb$ (three-pronged substructure); background jets are QCD-initiated single-prong jets. Constituent particles are ordered by transverse momentum $p_{T}$ (descending).

We study five TopTagging models with comparable classification performance (Table 1). L-GATr (Spinner et al., 2024) has 12 GATrBlock layers with $C_{\mathrm{mv}}=16$ and $C_{\mathrm{s}}=32$; L-GATr-slim (Petitjean et al., 2025) has 12 LGATrSlimBlock layers with $C_{v}=32$ and $C_{s}=96$; and LLoCa-T (Spinner et al., 2025) has 12 transformer blocks with Lorentz equivariance enforced by local canonicalization. These three models provide the symmetry-aware comparison set. We compare them to two non-equivariant baselines: Vanilla-T, a 10-block standard transformer using the same general architecture family as L-GATr, and ParT (Qu et al., 2022), a 10-layer kinematics-only ParticleTransformer checkpoint. ParT augments self-attention with learned pairwise physics biases, giving it a soft geometric prior without exact equivariance. All AUC values are 3-seed means.

Equivariance test. For each model, we apply 5 random Lorentz transforms to 200 test jets and measure the mean relative change in the output logit. For L-GATr and L-GATr-slim, the transform is applied to the full embedded input, including beam-spurion/reference tokens, so the tested representation transforms consistently. We additionally perform a boost sweep with $\gamma\in\{1.0,1.25,1.5,1.75,2.0,2.5,3.0,4.0,5.0\}$ to probe behavior from the identity transform through large boosts; the range is capped at $\gamma\approx 5$, motivated by the dataset’s hard rapidity cut of $|\eta_{j}|<2$, which makes boosts well beyond $\gamma\sim 5$ poorly represented in the data. The output metric is $|\ell(\Lambda x)-\ell(x)|/(|\ell(x)|+\varepsilon)$ with $\varepsilon=10^{-8}$. This relative-logit metric is a sensitive invariance diagnostic, not a calibrated performance metric.

Linear probes on $h_{\mathrm{s}}$. We extract scalar channel representations at each layer for 10,000 TopTagging jets, then fit linear probes for 14 physics targets: jet mass, jet multiplicity, N-subjettiness values $\tau_{1},\tau_{2},\tau_{3},\tau_{21},\tau_{32}$ (Thaler & Van Tilburg, 2011), top/QCD classification and particle-level $p_{T}$, $\eta$, $\phi$, $E$, $p_{T}$-quartile and $\Delta R$. The main probe table and probe figures report combined 95% bootstrap intervals obtained by pooling draws across the 3 training seeds ($3\times 200=600$ draws per target). Regression targets use Ridge regression; binary and multi-class targets use logistic regression. For particle-level probes, train/test separation is performed at the jet level.

Grade decomposition. We zero one grade group of $h_{\mathrm{mv}}$ at all layers simultaneously and measure the AUC drop (zero-grade ablation), and separately activate only one group while zeroing all others (keep-only ablation). Layer-resolved ablation zeros one grade group at one layer at a time. Under the default connected-subgroup setting (see Section 2), L-GATr enforces equivariance to the proper orthochronous Lorentz subgroup, so scalar and pseudoscalar components (G0+G4) may mix, as may vector and trivector components (G1+G3); we therefore report three groups: scalar-like (G0+G4), vector-like (G1+G3) and bivector (G2). All ablations are applied to hidden multivector outputs after the corresponding layer module, not to the raw input embedding.

Geometric-algebra-invariant probes on $h_{\mathrm{mv}}$. We construct 832 Lorentz-invariant scalar features per token from $h_{\mathrm{mv}}$ via the geometric-algebra inner product $\langle\tilde{X}Y\rangle_{0}$ across grade pairs, where $\tilde{\cdot}$ denotes reversion. Features include: G0 and G4 direct components (32 features), G1 pairwise inner products (136), G3 pairwise inner products (136), G1 $\times$ G3 cross terms (256) and G2 scalar and pseudoscalar invariants (136+136). Implementation details are in Appendix D.

Table 3 (Appendix C) and Figure 2 report final-layer linear probe scores for all five models as combined-bootstrap medians. Several consistent patterns emerge.

Pseudorapidity $\eta$ as an invariance probe. All three symmetry-aware models suppress particle-level $\eta$ to approximately zero at the final layer (L-GATr: $-0.002$; L-GATr-slim: $0.000$; LLoCa-T: $-0.006$), while Vanilla-T retains $0.151$ and ParT retains $0.287$. The particle-level $\phi$ probe is near zero for all models, largely because inputs use $\Delta\phi$ relative to the jet axis rather than absolute azimuth. The $\eta$ result is a representation-level signature of the imposed symmetry: in the architecturally equivariant models this absence is largely enforced by the scalar channel construction, while LLoCa-T shows a similar empirical pattern through its canonicalization-based representation. The comparison to Vanilla-T and ParT is the non-trivial part: ordinary transformers retain linearly accessible rapidity information.

LLoCa-T makes physics observables most linearly accessible. Despite its larger equivariance error, LLoCa-T achieves the highest final-layer probe scores for jet mass ($0.993$), jet multiplicity ($0.971$), $\tau_{1}$, $\tau_{2}$, $\tau_{3}$ and $\tau_{21}$ ($0.609$) of all five models. This suggests that probe accessibility reflects how linearly the model encodes physics: canonicalization may organize representations in a way that is more linearly decodable in this probe basis, despite its larger measured equivariance error.

N-subjettiness and ratio probes. All models encode individual N-subjettiness values $\tau_{1},\tau_{2},\tau_{3}$ strongly ($R^{2}\approx 0.84$–$0.97$), consistent with these being well-defined jet substructure observables (Thaler & Van Tilburg, 2011). However, the ratio $\tau_{21}=\tau_{2}/\tau_{1}$ and the top-tagging-motivated ratio $\tau_{32}=\tau_{3}/\tau_{2}$ are substantially weaker for all models. This is consistent with Cheng (2019): networks encode the building blocks $\tau_{N}$ more accessibly than their ratios, a pattern that persists across all five architectures studied here.

Full L-GATr makes jet-level observables more linearly accessible. Despite comparable task AUC, L-GATr achieves higher final-layer probe scores than L-GATr-slim for jet mass ($0.983$ vs. $0.953$), jet multiplicity ($0.923$ vs. $0.827$) and $\tau_{3}$ ($0.809$ vs. $0.766$). This advantage is clearest for jet-level substructure targets; several particle-level kinematic probes are higher for L-GATr-slim.

L-GATr-slim self-corrects input leakage. L-GATr-slim takes $\Delta\eta$ as a scalar input feature and its particle-$\eta$ probe starts above zero at early layers before decaying to $\approx 0$ by the final layer, consistent with progressively suppressing linearly accessible non-invariant information in the probed scalar channels.

Figure 3 shows layerwise probe trajectories for $\eta$, $\tau_{32}$, and jet mass. The $\eta$ suppression is visually immediate: equivariant models remain near zero throughout depth, while ParT and Vanilla-T retain substantial non-zero signal.

We fit linear probes on the 832 Lorentz-invariant features from $h_{\mathrm{mv}}$ at each layer of L-GATr (Section 3). Two findings inform the grade discussion below.

Multiple usable pathways in the final layer. At the final layer, probes on scalar-like invariants (G0+G4 components, 32 features) and vector-like invariants (G1+G3 inner products and cross terms, 528 features) both approach full-model classification AUC: scalar-like achieves $\approx 0.986$ and vector-like achieves $\approx 0.982$–$0.986$ under the combined-bootstrap summary, matching the task AUC. This is consistent with L-GATr encoding discriminative information through both the vector-like (G1+G3) pathway and the scalar-like (G0+G4) pathway, as reflected also in the high cross-seed variance of vector-like ablations in Section 6.

$\eta$ is absent from invariant features. The particle $\eta$ probe on all 832 invariant features gives $R^{2}\approx-0.025$ (seed mean, negative for all seeds), confirming that the Lorentz-invariant features carry no frame-dependent rapidity information. The particle $\phi$ probe is near zero across all layers ($R^{2}\approx 0$).

$\tau_{21}$ more accessible from $h_{\mathrm{mv}}$ than $h_{\mathrm{s}}$. The full 832-feature probe achieves $R^{2}\approx 0.648$ for $\tau_{21}$ at the final layer, compared to $0.504$ from $h_{\mathrm{s}}$ alone (Table 3). The ratio appears to be encoded in the inter-grade structure of $h_{\mathrm{mv}}$ but less linearly accessible from the scalar channels. Full per-layer trajectories are in Appendix D.

Table 2 reports grouped grade-ablation results for L-GATr on TopTagging, summarized across three seeds. Figure 4 plots the same zero-grade results and shows how the zero-grade intervention changes with depth.

Bivectors are not load-bearing in this setting. Zeroing all bivector (G2) channels at every layer reduces AUC by only $0.001$, the smallest effect of any grade group and consistent with zero. This finding is seed-robust: no seed shows G2 ablation impact above 0.005. This does not imply that bivectors are never useful in L-GATr; rather, for these TopTagging checkpoints the task information is recoverable through the scalar-like and vector-like pathways under the default connected-subgroup parameterization.

Vector-like channels are dominant but seed-variable. Zeroing vector-like (G1+G3) channels gives $\Delta\mathrm{AUC}=0.239$, the largest zero-ablation effect, but with very high cross-seed variance (seed values: $0.809$, $0.239$, $0.219$). This variability suggests that different training runs may allocate task information differently between the vector-like (G1+G3) pathway and the scalar-like (G0+G4) pathway, as discussed in Section 5.2. More seeds would be needed to characterize this pathway preference definitively.

Scalar-like channels remain competitive. Zeroing scalar-like (G0+G4) channels gives $\Delta\mathrm{AUC}=0.118$. The keep-only view is complementary: keeping only scalar-like channels gives $\Delta\mathrm{AUC}=0.239$, matching the vector-like zero ablation. Taken together with the invariant-probe results, this supports redundancy between scalar-like and vector-like pathways.

Note on keep-only bivector. Keeping only G2 channels gives $\Delta\mathrm{AUC}=0.337$ (Table 2), which may appear in tension with the G2$\approx$0 zero-ablation result. The keep-only intervention does not isolate standalone bivector capacity: the scalar branch remains available to the readout, so the remaining performance can still use scalar information. We therefore interpret the near-zero G2 zero-ablation as the cleaner evidence that bivectors are not load-bearing here.

Figure 4 shows the global and layer-resolved grade ablations for L-GATr on TopTagging. Several patterns are clear.

Vector-like ablation impact is largest at the input stage (mean $\Delta\mathrm{AUC}=0.422$) and decays monotonically with depth. Scalar-like impact is smaller throughout ($\Delta\mathrm{AUC}=0.091$ at the input stage, near zero after the first block). Bivector ablation is negligible at every individual layer ($\Delta\mathrm{AUC}<0.002$ at all depths). All grade-group ablations converge to $\Delta\mathrm{AUC}\approx 0$ at the final layer by architectural necessity: the grade-preserving output projection can only read G0 signal from the final layer, regardless of what other grades encode. Seed 1001 shows substantially larger input-stage vector-like impact ($0.809$) than seeds 1002/1003 (${\approx}0.28$), consistent with the seed-variable global ablation result and the multiple-pathway hypothesis.

To verify that the G2 $\approx$ 0 finding is not an artifact of the trained network suppressing bivectors post-hoc, we also study a model trained with bivectors architecturally disabled (G2 channel zeroed during training). This model achieves baseline AUC $0.9865\pm 0.0002$, statistically indistinguishable from the standard L-GATr ($0.9867\pm 0.0003$). Its vector-like $\Delta\mathrm{AUC}$ ($0.216$) is comparable to seeds 1002/1003 of the standard model. Thus, architecturally removing G2 during training has negligible effect on task performance. Full grade ablations for this model are in Appendix E.1. A complementary check with the alternative subgroup setting (five independent grades rather than three mixed groups) likewise finds G2 negligible and further shows that parity-odd grades (G3, G4) are not load-bearing; full tables are in Appendix E.3.

L-GATr-slim retains only G0 scalars and G1 4-vectors and has no G2, G3, or G4 channels. Its ablation pattern differs sharply from L-GATr: zeroing all scalar channels reduces AUC to $0.500$ ($\Delta\mathrm{AUC}=0.486$), a complete loss of discriminative power, while zeroing the 4-vector channels gives $\Delta\mathrm{AUC}=0.585$. Full component-level ablations are in Appendix E.2. The scalar branch is indispensable for L-GATr-slim because the readout is the global scalar token directly.

In contrast, L-GATr shows a more distributed pattern: vector-like channels are the dominant but seed-variable ablation target, scalar-like channels remain competitive, and the higher-order grades omitted by L-GATr-slim do not appear necessary for TopTagging. Both architectures achieve comparable task AUC, suggesting that grade structure reflects representational strategy rather than task requirement.