How the singularities of gradient flows illuminate the geometry of neural CT reconstruction landscapes
Let $M$ be a compact smooth $n$-manifold and $f:M\to\mathbb{R}$ a Morse function: all critical points are non-degenerate, i.e.\ the Hessian $\mathrm{Hess}_p f$ is invertible at each $p\in\mathrm{Crit}(f)$.
Throughout the geometric sections below, we assume in addition that the gradient flow is Morse–Smale, so the stable and unstable manifolds of critical points meet transversely.
Near every critical point $p$ of index $\lambda$, there exist local coordinates $(x^-,x^+)\in\mathbb{R}^\lambda\times\mathbb{R}^{n-\lambda}$ in which $$f(x^-,x^+)=f(p)-\|x^-\|^2+\|x^+\|^2.$$
The Morse index $\mathrm{ind}(p)$ is the number of negative eigenvalues of $\mathrm{Hess}_p f$: it counts the directions along which $f$ decreases away from $p$.
The negative gradient flow $\dot x = -\nabla f(x)$ descends $f$ strictly: $\tfrac{d}{dt}f(\phi_t(x))=-\|\nabla f(\phi_t(x))\|^2\le0,$ with equality only at critical points. Every non-stationary orbit converges forward to a critical point of $f$.
The Euler characteristic check provides a global constraint: $$\chi(M)=\sum_p(-1)^{\mathrm{ind}(p)}.$$ For $\mathbb{T}^2$: $0=1-2+1$ (one max, two saddles, one min). ✓
Click on the canvas to place critical points. Flow lines are numerically approximated from saddle points along their unstable directions.
Green = minima (λ=0) · Orange = saddles (λ=1) · Red = maxima (λ=2)
Blue arcs = stable manifold segments drawn from saddles to minima
A family of flow lines (blue) from $p$ to $q$, parametrised by how long they linger near the intermediate saddle $r$. As $T\to\infty$ the family converges to the broken trajectory $p\to r\to q$ (orange dashed).
The moduli space of unparametrised flow lines from $p$ to $q$ is $$\widehat{\mathcal{M}}(p,q):=(W^u(p)\cap W^s(q))/\mathbb{R},$$ a smooth manifold of dimension $\mathrm{ind}(p)-\mathrm{ind}(q)-1$.
$\overline{\widehat{\mathcal{M}}(p,q)}$ is compact. Its boundary is exactly the union of broken-trajectory products: $$\partial\,\overline{\widehat{\mathcal{M}}(p,q)} =\bigsqcup_{r:\,\mathrm{ind}(p)>\mathrm{ind}(r)>\mathrm{ind}(q)} \widehat{\mathcal{M}}(p,r)\times\widehat{\mathcal{M}}(r,q).$$
Standard Morse-theoretic compactness combines time-shift normalisation, Arzelà–Ascoli on compact intervals, and the energy identity $$\int_{-\infty}^{\infty}\|\nabla f(\gamma(t))\|^2\,dt=f(\gamma(-\infty))-f(\gamma(+\infty)).$$ The essential point is that bounded energy prevents infinitely many breaks, while the translation symmetry is removed by fixing a transverse slice.
Breaking is not pathological. Near the break parameter, the moduli space is locally homeomorphic to a cone with cone parameter measuring the time spent near $r$. This conical behaviour is a geometric heuristic for the local models discussed in §3.
For any gradient trajectory $\gamma:\mathbb{R}\to M$: $$\int_{-\infty}^{+\infty}\|\nabla f(\gamma(t))\|^2\,dt =f(\gamma(-\infty))-f(\gamma(+\infty))\le E_{\max}.$$ Since each break dissipates energy $\ge\min_{p\neq q}|f(p)-f(q)|>0$, the number of breaks is globally bounded.
The stable manifold closure decomposes as $$\overline{W^s(p)}=\bigsqcup_{\substack{q\in\mathrm{Crit}(f)\\\mathrm{ind}(q)\le\mathrm{ind}(p)}}S_q,\quad S_q:=W^s(q)\cap\overline{W^s(p)}.$$ Each stratum $S_q$ is a smooth submanifold of dimension $n-\mathrm{ind}(q)$.
A natural candidate, suggested by broken-trajectory compactification, is a local model of the form $$U\cap\overline{W^s(p)}\approx (S_q\cap U)\times C(L_x).$$ The website presents this as geometric guidance; the manuscript now treats the precise topological statement more cautiously.
The manuscript investigates whether adjacent strata satisfy Whitney-type conditions. The displayed quadratic estimate should be read as a target local bound to be proved, not as a completed theorem on this page.
Trotman-style openness of transversality is part of the longer-term programme. At present, the rigorous core established in the manuscript is the closure-by-broken-trajectories picture, from which these stronger claims are motivated.
The flow suggests natural projection and energy functions, but full Mather control data require additional verification. They are best viewed here as a structured proposal rather than a finished theorem.
Three strata of $\overline{W^s(p)}$. Dashed: broken trajectory. Gold: a heuristic conical local model near a boundary point.
Navy=$p$ (index $2$) · Green=$a,b$ (index $1$) · Orange=$q$ (index $0$). The drawing visualises the closure of the unstable manifold of $p$ for the negative gradient flow, equivalently the stable picture for the time-reversed flow.
Let $\mathbb{T}^2=(\mathbb{R}/2\pi\mathbb{Z})^2$ with the flat metric $d\theta^2+d\phi^2$, and fix $\varepsilon\in(0,1)$. Consider $$f(\theta,\phi)=\varepsilon\cos\theta+\sin\phi.$$ Its negative gradient flow is the decoupled system $$\dot\theta=\varepsilon\sin\theta,\qquad \dot\phi=-\cos\phi.$$
The critical points are exactly the solutions of $\sin\theta=0$ and $\cos\phi=0$, namely $$p=(0,\pi/2),\quad a=(\pi,\pi/2),\quad b=(0,3\pi/2),\quad q=(\pi,3\pi/2).$$ Since $$\mathrm{Hess}\,f(\theta,\phi)=\begin{pmatrix}-\varepsilon\cos\theta&0\\[2pt]0&-\sin\phi\end{pmatrix},$$ we get $$\mathrm{ind}(p)=2,\qquad \mathrm{ind}(a)=\mathrm{ind}(b)=1,\qquad \mathrm{ind}(q)=0,$$ so $f$ is Morse and $1-2+1=0=\chi(\mathbb{T}^2)$.
Because the system decouples, the coordinate circles $\{\phi=\pi/2\}$, $\{\theta=0\}$, $\{\theta=\pi\}$ and $\{\phi=3\pi/2\}$ are invariant. Along them one obtains exactly four one-step trajectories: $$p\to a\text{ on }\{\phi=\pi/2\},\qquad p\to b\text{ on }\{\theta=0\},$$ $$a\to q\text{ on }\{\theta=\pi\},\qquad b\to q\text{ on }\{\phi=3\pi/2\}.$$ Hence $$\widehat{\mathcal{M}}(p,a)\cong\widehat{\mathcal{M}}(p,b)\cong\widehat{\mathcal{M}}(a,q)\cong\widehat{\mathcal{M}}(b,q)\cong\{*\}.$$
Every trajectory from $p$ to $q$ meets the transverse circle $\Sigma=\{\phi=\pi\}$ exactly once, and it meets it at a unique point $(\theta_*,\pi)$ with $\theta_*\in(0,\pi)$. This gives a bijection $$\widehat{\mathcal{M}}(p,q)\longrightarrow(0,\pi),\qquad [\gamma]\longmapsto \theta_*(\gamma).$$ As $\theta_*\to0$, the trajectory spends longer near $b$ and converges to the broken orbit $p\to b\to q$; as $\theta_*\to\pi$, it converges to $p\to a\to q$. Therefore $$\overline{\widehat{\mathcal{M}}(p,q)}\cong[0,\pi]\cong[0,1],$$ with endpoints corresponding exactly to the two broken trajectories.
Near $q$, the closure of the unstable manifold of $p$ consists of the open two-dimensional region of direct $p\to q$ trajectories together with the two one-dimensional incoming branches through $a$ and $b$. On the compactified moduli side, the link of the endpoint corresponding to $q$ has two points, so the local compactification picture is a cone on two points, i.e. two rays meeting at one vertex. This is the precise source of the figure-X drawn in the animation.
Reconstruction via INRs minimises $\mathcal{L}_\lambda(\theta)=\|\mathcal{R}f_\theta-g\|^2+\lambda\|\theta\|^2$. Gradient descent $\dot\theta=-\nabla_\theta\mathcal{L}_\lambda$ on $\Theta=\mathbb{R}^N$ is formally identical to the Morse gradient flow on $M$.
Compact manifold $M$
Morse function $f:M\to\mathbb{R}$
Flow: $\dot x=-\nabla f(x)$
Stable/unstable manifolds inside $M$
Closure by broken trajectories established; stronger stratified structure remains under investigation
Parameter space $\Theta=\mathbb{R}^N$
Loss $\mathcal{L}_\lambda(\theta)$
Flow: $\dot\theta=-\nabla\mathcal{L}_\lambda$
Basins of attraction
Closures: open problem, see §8
A SIREN-1 network is $f_\theta(x)=W_2\sin(\omega_0(W_1x+b_1))+b_2$. The sign-flip group $G\cong(\mathbb{Z}/2\mathbb{Z})^d$ acts on $\mathrm{Crit}(\mathcal{L}_\lambda)$, creating discrete families of critical points.
Replacing $(W_{1,j\cdot},b_{1,j},W_{2\cdot j})\to(-W_{1,j\cdot},-b_{1,j},-W_{2\cdot j})$ for any neuron $j$ leaves $f_\theta$ and $\|\theta\|^2$ invariant, hence $\mathcal{L}_\lambda$ invariant. Critical points form $G$-orbits of size $\le2^d$.
| Seed | Loss | $\|\nabla\mathcal{L}\|$ | $\lambda_{\min}(H)$ | Type |
|---|---|---|---|---|
| 0 | 0.03131 | 4.6e-4 | −0.02012 | saddle |
| 1 | 0.03184 | 1.2e-5 | +0.00870 | local min |
| 2 | 0.03131 | 2.0e-3 | −0.02668 | saddle |
| 3 | 0.03186 | 1.0e-6 | −0.00379 | saddle |
| 4 | 0.03020 | 2.4e-4 | +0.06812 | local min |
3/5 saddles · 2/5 minima · Paired eigenvalues suggest a $(\mathbb{Z}/2\mathbb{Z})^6$ symmetry pattern · Tikhonov bound $2\lambda=0.10$ is shown for reference
Gradients and Hessians were computed by automatic differentiation; runs were classified numerically using the reported values of $\|\nabla\mathcal{L}\|$ and the smallest Hessian eigenvalue.
$\mathcal{L}_\lambda$ is Morse–Bott with finitely many critical components $C_0,\ldots,C_K$ (each a finite $G$-orbit), $\mathrm{ind}(C_0)=0$. The stable manifold closures $\overline{W^s(C_k)}$ satisfy Whitney regularity and admit conical models, extending Theorems A-D.
Dark green: $C_0$ (seed 4, loss=0.030). Lighter green: $C_1$ (seed 1). Orange: saddles (seeds 0,2,3) with escape directions. Gold arc: $(\mathbb{Z}/2\mathbb{Z})^6$ orbit. Blue dashed: $\partial W^s(C_0)$.
Seed 0: {−0.0201, 0.0213, 0.0213, 0.0761, 0.100, 0.100, …}
Paired values = symmetry orbit directions.
Value 0.100 = 2λ = dead neuron directions (only Tikhonov contributes).
Recent empirical work established that transformer and RL agent latent spaces are not manifolds but stratified spaces, detected via the volume growth transform [Curry et al. 2025, Robinson et al. 2024-25].
For $x$ in a metric space $X$ with measure $\mu$: $$\mathrm{VGT}_x(r):=\mu(B(x,r)),\qquad d_{\mathrm{loc}}(x):=\lim_{r\to0}\frac{\log\,\mathrm{VGT}_x(r)}{\log r}.$$ On a smooth $k$-manifold: $\mathrm{VGT}_x(r)\sim c_k r^k$ (Weyl), so $d_{\mathrm{loc}}=k$ everywhere. On a stratified space: $d_{\mathrm{loc}}$ varies with $x$.
If the local compactification picture from §3 is correct, then one expects the leading small-scale exponent at an interior point of $S_q$ to be $n-\mathrm{ind}(q)$. At a boundary point $y\in S_r\subset\overline{S_q}$ one is naturally led to a mixed asymptotic model $$\mathrm{VGT}_y(r)\approx c_1 r^{n-\mathrm{ind}(r)}+c_2 r^{n-\mathrm{ind}(q)}\quad(r\to0),$$ where the lower-dimensional stratum contributes the dominant exponent. This section should therefore be read as a research direction, not as a finished theorem.
This closes the theoretical gap noted by Curry et al. (2025): "no method currently exists for characterizing precisely the stratified space structure of a noisy point-cloud sample in terms of volume-growth laws." If the stratification arises from a Morse-type gradient flow, the volume growth exponents are exactly the Morse indices.
Given a point cloud from a parameter space or latent space: (1) compute $d_{\mathrm{loc}}(x_i)$; (2) cluster by dimension to identify strata; (3) at boundary points, fit $\mathrm{VGT}_y(r)=c_1r^{d_1}+c_2r^{d_2}$; (4) certify Whitney regularity if $d_1<d_2$ and $c_1,c_2>0$. This translates Theorems A–D into a computable certificate applicable to neural network parameter spaces.
Whether Curry's latent spaces fall into this class depends on the conjecture in §6, which is why that conjecture matters beyond the SIREN setting.
Extend Theorems A–D to Morse–Bott gradient flows where $\mathrm{Crit}(f)$ is a smooth submanifold (not isolated points). The link $L_x$ would encode products of moduli spaces of trajectories between critical submanifolds; its cohomology would recover the Morse–Bott spectral sequence.
Prove the Refined Morse–Bott Conjecture for SIREN-1 with Tikhonov regularisation. First step: prove that the sign-flip group $G=(\mathbb{Z}/2\mathbb{Z})^d$ acts freely on $\mathrm{Crit}(\mathcal{L}_\lambda)$ and that each orbit is a non-degenerate critical set. The challenge: the interaction Hessian $\mathrm{Hess}\,\|\mathcal{R}f_\theta-g\|^2$ is not convex, and understanding its negative eigenvalues requires fine estimates on the Radon operator.
Prove that Algorithm 6.2 is asymptotically consistent: if $M$ points are drawn from a Whitney stratified space, does the algorithm certify Whitney regularity with probability $\to1$ as $M\to\infty$? The main obstacle is controlling the mixed polynomial fit $c_1r^{d_1}+c_2r^{d_2}$ at boundary points with finite samples and noisy volume estimates.
For gradient-like flows on Whitney stratified ambient spaces (not smooth manifolds), characterise the obstruction to Whitney regularity of $\overline{W^s(p)}$ near singularities of the ambient space. Relevant for flows on singular algebraic varieties and for the deep learning models where the parameter space has symmetry-induced strata.
This essay accompanies the manuscript in prep Stratified Geometry of Stable Manifold Closures for Morse–Smale Gradient Flows.
The manuscript incorporates four new directions developed independently, including the direct proof of Trotman conditions, the torus example, the SIREN numerics, and the volume growth connection.
The numerical experiments use only NumPy and SciPy; all code is available in the upcoming preprint supplementary material. The Hessian is computed by finite differences ($\varepsilon=5\times10^{-4}$) at each Adam-converged critical point.
[1] Milnor, J. Morse Theory. Princeton UP, 1963.
[2] Whitney, H. Tangents to an analytic variety. Ann. Math. 81, 1965.
[3] Mather, J. N. Notes on topological stability. Harvard, 1970.
[4] Thom, R. Ensembles et morphismes stratifiés. BAMS 75, 1969.
[5] Trotman, D. J. A. Geometric versions of Whitney regularity. Ann. ENS (4) 12, 1979.
[6] Trotman, D. J. A. Stability of transversality implies Whitney (a)-regularity. Invent. Math. 50, 1979.
[7] Laudenbach, F. On the Thom–Smale complex. Astérisque 205, 1992.
[8] Goresky, M. & MacPherson, R. Stratified Morse Theory. Springer, 1988.
[9] Audin, M. & Damian, M. Morse Theory and Floer Homology. Springer, 2014.
[10] Trotman, D. Stratification theory. Handbook Geom. Top. Sing. I, Springer, 2020.
[11] Nocera, G. & Volpe, M. Whitney stratifications are conically smooth. Selecta Math. 29, 2023.
[12] Sitzmann, V. et al. Implicit neural representations with periodic activation functions. NeurIPS 33, 2020.
[13] Bott, R. Nondegenerate critical manifolds. Ann. Math. 60, 1954.
[14] Robinson, M., Dey, S. & Chiang, T. Token embeddings violate the manifold hypothesis. arXiv:2504.01002, 2025.
[15] Curry, J. et al. Exploring the stratified space structure of an RL game with the volume growth transform. arXiv:2507.22010, 2025.
[16] Hirsch, M., Pugh, C. & Shub, M. Invariant Manifolds. LNM 583, Springer, 1977.
[17] (In prep.) Cissé, K. Stratified Geometry of Stable Manifold Closures for Morse–Smale Gradient Flows…, 2026.