[R] Detecting invariant manifolds in ReLU-based RNNs

In a new #ICLR2026 publication we provide a novel algorithm for semi-analytically constructing the stable and unstable manifolds of fixed points and cycles of ReLU-based RNNs:

https://openreview.net/pdf?id=EAwLAwHvhk

Why is this important?

Because it provides insight into why and how trained RNNs produce their behavior, as important for scientific and medical applications and explainable AI more generally. In scientific ML, RNNs are a common tool for dynamical systems reconstruction (https://www.nature.com/articles/s41583-023-00740-7), where models are trained to approximate the dynamical system underlying observed time series. Trained RNNs are then to be analyzed further as formal surrogates of the systems trained on.

An RNN’s dynamical repertoire depends on the topological and geometrical properties of its state space. Stable and unstable manifolds of fixed and periodic points dissect a dynamical system’s state space into different basins of attraction, their intersections lead to chaotic dynamics with fractal geometry, and – more generally – they provide a type of skeleton for the system’s dynamics, forming structures like separatrix cycles or heteroclinic channels.

https://preview.redd.it/lhwmuqz0ihmg1.png?width=2838&format=png&auto=webp&s=e51c9a6ffa0dd5ea1030fc11b7244eaeb4f7d651