Research programme
Studying how agents can be identified, modelled, and composed, and what those formal choices imply for cognition, living systems, and AI safety.
Areas of research
Control
Control theory
Feedback and regulation are central to control theory, and lead to ideas of goal-directedness, self-preservation and internal models, with connections to Bayesian frameworks via classical dualities.
Questions
When does a system that achieves a goal model its environment?
Conant and Ashby claimed that “every good regulator of a system must be a model of that system”, an idea taken up in control theory as the internal model principle. The extent to which this is true is unclear and formally not well understood.
We can show that for a class of deterministic systems, whenever an agent can perform a regulation task, an observer can interpret it as having possibilistic beliefs about its environment, updated in response to sensory input. The model is then not a mere property of the system, but is imposed on it from outside. The universal property of possibilistic belief updating shows that the agent’s ability to perform the task and the map from its states to beliefs are the same thing.
Papers
- The Universal Property of Possibilistic Belief UpdatingApplied Category Theory Conference (ACT), 2026
- A "good regulator theorem" for embodied agentsArtificial Life Conference (ALIFE), 2025
How do agents decide when they cannot know precisely what is happening around them?
Behaviour and decision making depend on the uncertainty an agent has about its environment, a dynamic system with too many properties to keep track of. How is an agent affected by unknown and ambiguous information, and how can reliable behaviour emerge from uncertain stimuli?
Papers
- Kalman filters as the steady-state solution of gradient descent on variational free energyarXiv:2111.10530, 2021
- A Bayesian perspective on classical control2020 International Joint Conference on Neural Networks (IJCNN), 2020
- On Kalman-Bucy filters, linear quadratic control and active inferencearXiv:2005.06269, 2020
- PID control as a process of active inference with linear generative modelsEntropy 21 (3), 257, 2019
- A probabilistic interpretation of PID controlSimulation of Adaptive Behavior Conference (SAB), 2018
Do the internal models of control theory and the Bayesian models of cognitive science describe the same thing?
The internal model principle claims that, under suitable assumptions, a controller that can regulate against a class of external inputs does so because it contains a model of the system causing them. Similar claims appear in cognitive science, especially in Bayesian treatments of cognitive agents, which often suggest that an agent models its environment to adapt against disturbances and perform goal-directed behaviour. It is unclear, however, whether these Bayesian models bear any formal relation to the internal models of control theory.
For a class of autonomous systems, we show that standard accounts of models as coarse-grainings correspond to a special case of possibilistic Bayesian filtering.
Papers
- A Bayesian Interpretation of the Internal Model PrinciplearXiv:2503.00511, 2025
Cognition
Natural and artificial cognition
Perception-action loops, (minimal) cognition and consciousness support notions of identity and adaptive behaviour in natural and artificial agents, with questions related to what’s really out there as opposed to merely in the eye of the beholder.
Questions
How do natural and artificial agents acquire and use causal information?
Causal cognition is the ability to acquire and exploit causal information about oneself and the world. But how does this information come about? And how can it be learnt from the subjective point of view of an agent?
Papers
- Ecological causal cognition needs disentanglement: Reply to comments on "Disentangled representations for causal cognition"Physics of Life Reviews 58, 7-9, 2026
- Disentangled Representations for Causal CognitionPhysics of Life Reviews, 2024
What is the difference between an observer and an agent?
The frameworks used to study life and cognition, from information theory, control theory and dynamical systems to (stochastic) thermodynamics and (applied) category theory, all give observers a fundamental role: entropy in information theory and thermodynamics, observability in dynamical systems, the very definition of a target state in control theory. Can observers be defined in a way that unifies these notions? Can observers not be agents? Is agency in the eye of the beholder, or an intrinsic property of a system?
Autopoietic proposals in theoretical biology and Bayesian frameworks in cognitive science suggest further questions. Can intrinsic goals explain living and cognitive organisms without observers? What does it mean for an agent to be autonomous? Can non-agents become agents, and can non-agentic parts be composed into one?
Unpopular opinion: the concept of agency captures mostly just a series of choices made by an observer (cf. intentional stance). Less unpopular opinion: an interesting definition of agency could arise from the use of the word “mostly” in the line above.
Papers
- A Braitenberg vehicle’s habitatArtificial Life Conference (ALIFE), 2026
Can an artificial system keep an identity through time?
Despite their progress in reasoning and language, current AI systems exist only within isolated episodes of computation. What they lack is temporal continuity, the persistence of internal dynamics that sustain an unbroken stream of computation, analogous to the stream of consciousness.
A roadmap towards it is an architecture of persistent recursive inference, in which the output of each cognitive cycle becomes the input to the next, with continual learning, switching between inward and outward cognition, and sleep-like phases that separate learning from inference. Systems built this way would be lifelong agents, whose individuality and identity arise from the continuity of their own computational existence.
Papers
- The Stream of Computation: Temporal Continuity as a Missing Ingredient for Artificial ConsciousnessJournal of Consciousness Studies, 2026
Artificial Life
(Hybrid) Agents
Research on living, artificial and hybrid agents to answer questions about autonomy, individuality and regulation. The focus is on transitions between non-living and living, and on transitions from multiple systems interacting to a cohesive identity.
Questions
When does a physical system count as an agent, and where do its goals come from?
Agency is central to biology, cognitive science and AI, yet no widely accepted formal definition of agency or purpose exists. Agency is usually tied to the idea of a system able to take an action, without saying what an action is: is a ball taking an action when it falls to the ground, or when someone bounces it on a court?
Goals are often assumed simply to exist, without saying whether they are intrinsic properties of a system or attributed to it by an observer, or whether the same system can have different goals depending on who looks at it.
Papers
- Mathematical approaches to the study of agentsPhilosophical Transactions of the Royal Society B, 2025
Links
Lectures for the CHAIN Winter School on Minimal Cognition and Agency, Hokkaido University, February 2023.
Is agency a precursor of life, and are there sharp lines between the living and the non-living?
Formal definitions of agency, life and cognition come down to what separates agentic from non-agentic systems, living from non-living organisms, and cognitive from non-cognitive architectures. Do sharp distinctions between these classes exist, and are they meaningful? If there are none, why are some systems perceived to be qualitatively different? And what mathematical theories can bring out such apparent diversity?
Agency can be seen as a relative of life, or as one of its precursors. Definitions of life usually rest on two pillars, Darwinian evolution and self-maintenance in chemical systems. Evolution has been abstracted and modelled mathematically, while self-sustaining systems have resisted a comparable formalisation, and recasting them as agents, goal-directed systems acting in an environment, gives them a more abstract setting.
Papers
- Mathematical approaches to the study of agentsPhilosophical Transactions of the Royal Society B, 2025
Links
When does interaction turn into integration, and how are boundaries negotiated?
Agents change over time, their boundaries change. Biological agents grow, replicate, replace parts of their bodies. Artificial agents recruit new tools, memories, break a few parts but still carry out a task. Frameworks attempting to define agents often assume that they have a fixed boundary/interface. The notion of “agent” is much less clear with anything that involves some kind of extended cognition possibility, like tool use or sensory augmentation, or anything that involves multiple agents and the idea that they can organise in macro-agents.
We need some formal notion of identity beyond just some kind of consistency across space and time, to include some level of substrate independence and criteria about changing boundaries. Not every change to the interface leads to a new agent.
Papers
- Hybrid Life: Integrating Biological, Artificial, and Cognitive SystemsWIREs Cognitive Science, 2023
- The Emperor is Naked: Replies to the commentaries on the target article.Behavioral and Brain Sciences, 2022
- The Emperor’s New Markov BlanketsBehavioral and Brain Sciences, 2021
Links
AI safety
Agent foundations
Identifying AI agents in and among modern AI systems, together with their boundaries, goals and models, in order to estimate their capabilities and improve interpretability. The aim is to make safety questions precise enough to analyse and test.
Questions
What does an AI agent model, and how can we check that its behaviour fits its intended purpose?
A formal link developed between the physical behaviour of a system and an intentional-stance description in terms of beliefs and goals. Interpretations of when dynamical systems can be understood as goal-directed agents are a central part of this proposal, explaining how this can support tools for checking whether AI behaviour is consistent with its intended purpose. The project (2025–2027) is supported by ARIA’s Trust Everything, Everywhere opportunity space.
Papers
- The Universal Property of Possibilistic Belief UpdatingApplied Category Theory Conference (ACT), 2026
- Bayesian updates from coalgebraic determinisationarXiv:2607.00034, 2026
- AI in a vat: Fundamental limits of efficient world modelling for agent sandboxing and interpretabilityReinforcement Learning Conference (RLC), 2025
Links
Which behaviours survive when a system is abstracted?
State abstraction is key to scaling reinforcement learning to complex but structured systems, where many behavioural structures have been studied, from value functions and invariants to bisimulation relations and behavioural metrics. A general principle for which of them are provably preserved under abstraction is still lacking. Compositional behavioural semantics specify such structures from local, one-step descriptions of a system’s dynamics, show how they can be safely transferred between abstract and concrete systems, and build quantitative metrics from logical semantics with soundness guarantees.
Papers
- Compositional Behavioral Semantics for State Abstraction in Reinforcement LearningInternational Conference on Machine Learning (ICML), 2026
What is a world model?
World models are central to model-based reinforcement learning, and can be told apart by, at least, the channel they model: the environment, the agent, or their joint process. Can we identify core features of different kinds of models? When can they be factored? And when can they not, and why?
Papers
- Beliefs in feedback: Factored representations of interacting agents in transformersarXiv:2611.02047, 2026
- World models of environment, agent and joint agent-environment systemsarXiv:2608.20401, 2026
- From monoliths to modules: Decomposing transducers for efficient world modellingarXiv:2512.02193, 2025