Skip to content
Paulius Rauba

Research

I am a Founding Research Scientist at Cursive, working on large-scale post-training of long-horizon agentic models. My current work includes asynchronous reinforcement learning (async RL), on-policy self-distillation and memory-augmented models. I hold a PhD in Machine Learning from the University of Cambridge.

My work spans training capable agents, understanding their failures, and making their behavior more reliable. Below are the questions I work on, my technical notes, and the full list of papers and preprints.

Asynchronous RL and long-horizon training

At Cursive, I work on large-scale reinforcement learning and post-training for long-horizon agents. My notes explore the practical questions behind these systems: rollout throughput, policy lag, cheap proxy experiments and communication between agents. This work connects training infrastructure with the behavior of models over extended tasks.

AI safety, alignment and evaluation

How can we detect when a model fails, test whether its behavior changes, and control what it is allowed to do? My work includes context-aware testing, statistical audits of language-model robustness, least-privilege language models and generalization under distribution shift. My work on principal-agent problems examines questions of incentives and alignment in multi-agent systems.

Efficient language models and adaptive computation

My research on deep hierarchical learning asks how a single model can support different computational budgets. Related work explores nested subspace networks, recursive models and masked diffusion. These papers sit alongside my current interests in memory-augmented models and efficient model deployment.

Probabilistic forecasting and AI for science

At Salient, I worked on generative models for global weather forecasting. This research includes probabilistic transformers that jointly model atmospheric dynamics and decision-centric variables, and timestep-conditioned transformers for global forecasts. The papers link model design to the quantities that forecast users need.

Papers & preprints

Google Scholar ↗

Conference papers, workshop papers, preprints and other research. Each entry links to its original record.

  1. 2026 · arXiv preprint · 2608.06241

    Timestep-Conditioned Transformers for Global Weather Forecasting

    Levang S., Bartolic F., Dickinson T., Dwelle C., Rauba P., Cikojevic V.

  2. 2026 · arXiv preprint · 2607.03478

    Demonstrating Generalization Failures via Mixtures of Conditional Policies

    Barzdukas J., Peck J., Schulz J., Rauba P., Basart S., Wells L.

  3. 2026 · arXiv preprint · accepted at BeNTo Workshop, NeurIPS 2026

    Recursive Scaling in Masked Diffusion Models

    Carballo-Castro A., Piskorz J., Rauba P., van der Schaar M., Frossard P.

  4. 2026 · FoGen Workshop @ ICML 2026

    Scaling with Recursion in Masked Discrete Diffusion Models

    Carballo-Castro A., Piskorz J., Rauba P., van der Schaar M., Frossard P.

  5. 2026 · ICLR 2026

    Deep Hierarchical Learning with Nested Subspace Networks for Large Language Models

    Rauba P., van der Schaar M.

    Nested Subspace Networks (NSNs) reparameterize each linear layer of a neural network as a low-rank product W = B A whose columns are nested, so that every rank-r sub-model is a strict prefix of every higher-rank one. A single set of weights then encodes a continuous hierarchy of models that can be traversed at inference time by simply choosing the rank, trading compute for accuracy with no retraining and no model zoo. Applied to large language models, NSNs unlock anytime inference, hardware-aware deployment, and elastic capacity under a fixed memory budget.

  6. 2026 · arXiv:2601.23211

    Multi-Agent Systems Should be Treated as Principal-Agent Problems

    Rauba P., Cepenas S., van der Schaar M.

  7. 2026 · ICML 2026

    No More, No Less: Least-Privilege Language Models

    Rauba P., Seputis D., Vanagas P., van der Schaar M.

  8. 2026 · Salient research model report

    Probabilistic Transformers for Joint Modeling of Global Weather Dynamics and Decision-Centric Variables

    Rauba P., Cikojevic V., Bartolic F., Levang S., Dickinson T., Dwelle C.

  9. 2026 · ICLR 2026 Workshop on AI with Recursive Self-Improvement · Spotlight

    Tiny Autoregressive Recursive Models

    Rauba P., Fanconi C., van der Schaar M.

  10. 2026 · ICLR 2026 Workshop on Principled Design for Trustworthy AI

    Model Organisms for Generalization Resistance Under Distribution Shift

    Barzdukas J., Peck J., Schulz J., Rauba P., Wells L.

  11. 2025 · ICML 2025

    Statistical Hypothesis Testing for Auditing Robustness in Language Models

    Rauba P., Wei Q., van der Schaar M.

    Distribution-Based Perturbation Analysis (DBPA) is a frequentist hypothesis-testing framework for auditing the robustness of large language models. Because LLMs are stochastic, comparing a single response under an original prompt to a single response under a perturbed prompt cannot reliably tell us whether the perturbation actually changed model behaviour. DBPA samples N responses under each prompt, embeds each response into a low-dimensional semantic space, and runs a permutation test on the centroid distance — yielding a p-value with no distributional assumptions. The framework supports arbitrary prompt perturbations and applies Bonferroni correction across multiple audits.

  12. 2025 · AISTATS 2025

    Visualizing Token Importance for Black-Box Language Models

    Rauba P., Wei Q., van der Schaar M.

  13. 2025 · Frontiers in Artificial Intelligence

    Redefining Digital Health Interfaces with Large Language Models

    Imrie F., Rauba P., van der Schaar M.

  14. 2024 · NeurIPS 2024

    Context-Aware Testing: A New Paradigm for Model Testing with Large Language Models

    Rauba P., Seedat N., Luyten M.R., van der Schaar M.

    Context-Aware Testing (CAT) is a new paradigm for ML model testing that uses large language models to read the surrounding context of a task and generate the failure modes that actually matter, rather than exhaustively scanning data slices. The accompanying SMART method generates context-grounded hypotheses about model failures, refutes them with the data using statistical tests, and applies multiple-testing correction so that the set of surfaced failures has controlled false-discovery rate. CAT addresses a fundamental limitation of data-only testing — that exhaustive slice search inflates Type-I error and surfaces failures that are not domain-meaningful.

  15. 2024 · NeurIPS 2024

    Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments

    Rauba P., Seedat N., Kacprzyk K., van der Schaar M.

    Self-Healing Machine Learning (SHML) is a framework for models that autonomously adapt to distribution shift without human intervention. Existing concept-drift methods are reason-agnostic — they pick from a fixed menu of corrective actions without ever asking why the model is failing. SHML instead frames adaptation as an optimization problem over actions and uses an agentic system, ℋ-LLM, in which a language model diagnoses the structure of the shift (covariate, label, concept, or spurious-feature) and prescribes a corrective action that targets the actual cause. The framework provides theoretical guarantees on adaptation quality and empirically outperforms reason-agnostic baselines under real-world distribution shifts.

  16. 2024 · Statistical Foundations of LLMs Workshop @ NeurIPS 2024

    Quantifying Perturbation Impacts for Large Language Models

    Rauba P., Wei Q., van der Schaar M.

  17. 2021 · University of Oxford

    Armed Conflict and State Policy Preferences: A Statistical and Neural Embedding Approach

    Rauba P.

  18. 2021 · WOAH @ ACL 2021

    Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset

    Kirk H.R., Jun Y., Rauba P., et al.