Anup Das, Ph.D.

Language models are surprisingly fragile: uniform corruption of word embeddings hurts low-magnitude embeddings the most, and many of the words that carry grammatical and semantic structure live in that low-norm space. In Robertha, we replace standard attention with a Modern Hopfield mechanism in which semantic patterns act as attractors that pull corrupted embeddings back toward their correct representations. Heavily corrupted embeddings are refined over more iterations than lightly corrupted ones, and Eigenspectrum Regularization shapes the keys into strong, well-separated attractors with wide recovery basins. Across 13 GLUE and SuperGLUE tasks, Robertha outperforms existing robustness methods while keeping competitive clean performance. In Agatha, we target adversarial attacks with a drop-in change to the Transformer encoder: iterative Hopfield attention combined with a gated residual connection that forces adversarial gradients through the energy landscape, where they are contracted layer by layer, without adversarial training. We are now carrying these ideas into sequential decision-making with the Robust Hopfield Decision Transformer. [Robertha, ACL 2026][Agatha, EMNLP 2026][code]

The attractor dynamics behind Robertha and Agatha come from Hopfield networks, a classic model of associative memory in the brain. We are building on this connection to develop language models whose attention and memory retrieval run as sparse, event-driven spiking dynamics instead of dense matrix products, aiming for models that are both robust and energy efficient. Our first steps include spiking neural networks for language tasks such as aspect term extraction. [paper]

Running language models on edge devices calls for hardware that exploits sparsity, low precision, and event-driven computation while minimizing data movement. We design and prototype neuromorphic accelerators on FPGA along these lines: QUDA, a sparse, compressed, quantized dataflow architecture; sparse hashing to reduce the memory latency and bandwidth of spiking neural network accelerators; and QUANTISENC, a fully configurable many-core neuromorphic hardware design. Our goal is to run robust, spike-based language models on low-power edge platforms. [QUDA, FCCM 2026][ICCAD 2025][code]