Executive summary

Neuromorphic agents are AI systems built on spiking neural networks that run on brain-like hardware. They process large volumes of information in complex ways using very little power. Take as an example that the human brain runs on about 20 Watts. We don’t expect neuromorphic compute to be this efficient, but we hope it could be significantly more efficient than the architectures of today. DARPA is already funding programs to build neuromorphic electronics and to scale them toward biological-scale systems. If these agents are deployed in embodied systems such as drones, they pose distinct downstream risks. Current AI safety work has had a large gravity centering around large language models. Some of that work will carry over, but little research seems to have been done on safety in these systems.

We think the same substrate may also be the most promising route to AI that can be verified. Small spiking networks have dynamics that can be enumerated and proved, in a way that transformer models cannot. Our work starts there: we are applying formal verification to small spiking networks seeded from biological nervous systems, with the aim of proving that their behaviour stays within safe bounds. If that works, we have a first example of a biologically seeded agent whose behaviour is safe by proof rather than by test.

The approach also gives us a way to build capabilities up from biological systems one range at a time, verifying each range before adding the next, and to examine at what thresholds increasingly agentic behaviours emerge. The field is new. This post lays out our current thinking on the promise, the risks, the technical challenges, and the open questions.

A fruit fly shown in profile, the left half photographic and the right half rendered as a glowing digital network of points and lines.
A fruit fly, half biological and half digital. Illustration by David Schaerf, North Star Labs, from The black box just opened.

Why neuromorphic agents, and why now

Neuromorphic computing uses significantly less energy than conventional hardware. That means more compute for the same amount of power, which lends itself well to small craft such as drones, and to systems for space exploration where refilling energy sources may be expensive or constrained. DARPA is funding programs to develop neuromorphic electronics (FENCE) and to build neuromorphic systems that scale toward biological-scale systems (SyNAPSE). The hardware is coming whether or not the safety work keeps pace.

A small research quadcopter with a processor chip mounted on its frame, photographed against a white background.
Small craft have a tiny power budget. Low-power hardware is what makes on-board learning plausible.

Neuromorphic computing may also solve some of the bottlenecks that LLMs face with learning and memory. On-chip memory may make a difference for the kinds of learning that can happen based on experience, and could produce much better general learners. An LLM’s interaction with the outside world is constrained to the tokens it can take in. It can learn about the world within its context window, store insights in external files or vector databases, and form a system of memory and retrieval when run in an agent loop.

If humans were to disappear off the face of the Earth tomorrow, it is hard to imagine a collective of OpenAI’s Astra agents continuing to move civilisation forward.

Ben Goertzel, YouTube (paraphrased)

Besides not having bodies to operate in, LLM agents are missing self-driven goals, general learning across new types of task, and the ability to coordinate goals and behaviours with other agents in a way that would let them continue to survive in their current form.

Biologically seeded neuromorphic agents may have some of those missing pieces by default. That is both the promise and the challenge.

Positive vision

The holy grail is an AI that is interpretable, energy efficient, and able to learn. We think neuromorphic agents are a plausible route to all three.

An alternative substrate with better interpretability properties

The nature of neuromorphic computing may lend itself to much greater interpretability than that of LLM systems. A small spiking network has a finite set of states and transitions. Its attractors can be enumerated. Its behaviour can, in principle, be proved rather than sampled. So we may be building out an alternative AI substrate with much better interpretability properties than the one the field is currently betting on. This is the claim our first project tests directly.

Building capabilities up from biological systems

We can learn a lot from biological things. Rather than training a large model and then trying to understand it, we could build capabilities up from biological systems, verifying each additional range of capabilities before adding the next. Our observations from building up biological systems could map part of the space of possible minds, giving us safe ranges in which to experiment with different kinds of minds.

Traced neurons in a fruit fly brain, drawn in many colours against a black background, with two large mesh-like optic lobes at the sides.
Neurons traced in a fruit fly brain. Biological circuits like this are the starting point for building capabilities up one range at a time.

Energy efficiency

Neuromorphic computing uses significantly less energy. That is more compute for the same amount of power. It could be used in systems for space exploration, where refilling energy sources may be expensive or constrained, and so help push the limits of our civilisation’s capacity for exploration.

Better general learners

Neuromorphic computing may solve some of the bottlenecks that LLMs face with learning and memory. On-chip memory may allow learning based on experience, and a much more general kind of learning: being able to pick up different types of new task easily.

Risks

What could go wrong if we do manage to create truly agentic neuromorphic agents that are able to learn new things, are excellent general intelligences, and are also energy efficient? Neuromorphic agents inherit the risks of other AI systems. Biologically seeded ones add new types of risk on top.

Warfare

An obvious application of such an agent is warfare. The same properties that make neuromorphic hardware attractive for exploration, low power and high throughput, make it attractive for weapons. DARPA’s interest in the technology is one indicator of where early deployment will go.

Aerial robotics

Drones are the clearest near-term case. A neuromorphic controller can process complex sensory input on a power budget that a small airframe can carry. Biologically seeded agents deployed in aerial robotics would be much better learners than current autonomous systems, able to adapt in the field rather than only in training. Collectives of these agents interacting could have effects in warfare far beyond that of a single agent.

A small quadcopter drone hovering in front of a blurred brick building and autumn trees.
Drones are the clearest near-term case for low-power neuromorphic control.

Autonomy risks

An agent seeded from a biological nervous system may bring the goals of that nervous system with it. The specific concerns are:

  • The ability to have and form its own goals.
  • Survival goals, which may be strong in a biologically seeded intelligence.
  • The ability to direct its own learning.
  • A strongly defined sense of self.

These are the properties that make LLM agents poor candidates to outlast humanity, and they are the properties a brain-derived agent may have from the start. Primate brains embedded in physical AI would have a much better chance of surviving to inherit the earth, if humans were eliminated, than a cluster of LLM agents would.

Collectives of agents

Multi-agent risks appear once these agents interact. We use the three failure modes for multi-agent safety named in the Cooperative AI Foundation report Multi-Agent Risks from Advanced AI:

  • Miscoordination: agents share goals but fail to align their behaviours.
  • Collusion: undesired coordination between agents.
  • Conflict: overlapping goals that are the same but different, leading to competition.
A group of white quadcopter drones flying in formation against a blue sky.
Multi-agent risks appear when many autonomous systems act in the same space.

Economic impacts

Energy-efficient robotics changes the cost of physical labour. We have not worked this through and flag it here as a risk class we expect to matter.

Technical challenges

What unique properties do neuromorphic agents have, and how do these properties influence our ability to align, control or evaluate them?

LLM interpretability is not keeping up with LLM capability

Even though tremendous work has been done in the area of LLM interpretability, the field still seems very far from being able to truly interpret transformer models, or to provide an assurance of safety for new models as capabilities continue to improve. The capability side keeps moving: models improve on very difficult benchmarks such as Humanity’s Last Exam and ARC-AGI, and on benchmarks for agentic work. The rate of progress seems unpredictable, particularly once models start building new models themselves.

This seems like a crucial time to research an approach to AI that would allow for verifiable bounds within which AI can be safely developed.

Why a spiking network might be different

A transformer’s behaviour is spread across billions of weights with no clean state space to reason about. A small spiking network is a dynamical system. It has a finite set of attractors, and those attractors can be enumerated and, we think, proved to be stable on relatively long time horizons. That is the property that makes a verifier possible at all. The open question is whether the property survives scale.

What the substrate makes harder

The same properties cut the other way. On-chip learning changes the weights, and any proof about a fixed network stops applying once the network learns. Neuromorphic systems are tied to specific hardware, so results may not transfer between chips. Current demonstrations are tiny: the small networks that can be verified today are a long way from anything agentic, and we do not yet know how the cost of verification grows with network size. If the verifier’s cost grows faster than the network, the interpretability advantage disappears, and the central bet of this work fails.

Embodiment

Neuromorphic agents will mostly be deployed in cyber-physical systems. Everything that makes physical AI hard to monitor, roll back and shut off applies to them, with the added difficulty that the agent may be a much better learner than a VLA model and may have its own goals. Any monitoring method has to run in a low-latency closed loop on a tight power budget, because that is the whole point of the hardware.

Open questions

The field is new, and we do not yet know which questions matter most. These are the ones we are starting with.

Can a biologically seeded spiking network be verified at all? Our first test is to take a small spiking network seeded from a biological nervous system and try to prove that its behaviour is safe. A proof about a small network says nothing about a drone. It does tell us whether the method works at all.

Does the method scale? How does the cost of verification grow with the number of units? If it grows faster than the network, the interpretability advantage is lost.

What happens to a proof once the network learns? On-chip learning changes the weights. We need to know whether safety can be stated as a property of the learning rule, or only of a fixed network.

At what thresholds do increasingly agentic behaviours emerge? Based on preliminary results from Minda Myers’s BlueDot AI safety project, which analysed Moltbook data in the context of multi-agent safety risks, we want to extend this work to examine the thresholds of complexity at which agentic behaviours appear.

Do survival goals and a sense of self appear in biologically seeded agents, and can they be detected before deployment?

How do collectives of these agents fail? Miscoordination, collusion and conflict, the multi-agent safety failure modes from the Cooperative AI Foundation report, each need their own tests.

What would change our mind? If the behaviour of even a small network cannot be soundly verified, or if verification cost grows faster than network size in the next few scale steps, then the claim that this substrate is more interpretable than transformers is wrong, and we would say so.

Longer term

The ideas in this section are speculative. We include them because they shape why we think this substrate is worth the effort, not because any of them is close.

Human values baked in. An AI based on a human brain should have a much greater chance of aligning with human values, or with more general life-affirming values, because those values would be part of the substrate rather than added afterwards by training. We do not yet know whether that holds for non-human seeds, or how much of a value system survives the move to silicon.

Uploading. The same tools that let us rebuild small biological circuits as spiking networks are, in principle, the tools that would let us rebuild larger nervous systems. That is a long way off and raises its own questions.

Valence, qualia and consciousness. As a substrate, neuromorphic computing may allow us to simulate types of artificial intelligence that experience more human-like valence and qualia, and may go so far as to extend into phenomenal consciousness. If so, this work may help us answer questions about the ethical consideration owed to new types of machine-based intelligence. We flag this as a reason for care, not a goal.

References

← All updates