The Frankenstein Problem: What Happens When AI Outgrows Its Creators?

Written by: Andrew Valenti, PhD

More than two centuries before artificial intelligence, alignment research, autonomous agents, or recursive self-improvement entered our vocabulary, Mary Shelley posed a question that feels remarkably contemporary: What happens when we create an intelligence that we can no longer fully control?

Frankenstein is usually remembered as a Gothic horror story, but I increasingly find it more interesting as a story about science, intelligence, and the responsibilities of creation. Victor Frankenstein does something extraordinary. Through science, he creates an autonomous being capable of learning, reasoning, adapting, and ultimately acting independently of its creator. He then discovers that creating something and controlling what you have created are very different problems. That distinction sits near the center of today's debate about advanced artificial intelligence.

The comparison has become particularly timely. Some of the people leading the development of frontier AI are now publicly discussing whether capabilities may advance faster than our ability to understand and control them. Anthropic CEO Dario Amodei has argued for deliberately pacing frontier development, while OpenAI CEO Sam Altman has expressed support for the idea of pacing the frontier. OpenAI has also discussed shared standards under which development might be slowed or stopped if adequate safeguards are not available. Others in the technology community disagree with coordinated slowing and argue that rapid innovation and responsible development can proceed together.

I find the existence of this debate at least as interesting as the positions themselves. It is no longer simply outsiders warning technologists about what they are creating. Increasingly, people involved in building the most capable systems are asking how quickly development should proceed and what evidence of safety should be required before taking the next step. In a remarkably modern form, we have arrived at Victor Frankenstein's problem.

The Creature That Learns

One of the most important aspects of Shelley's story is often lost in popular versions of Frankenstein. The creature is not initially the monster we expect. He learns. He observes people from a distance, acquires language by listening to the De Lacey family, learns to read, and encounters Paradise Lost, Plutarch and Goethe. Through observation and experience, he constructs an increasingly sophisticated model of human society and of himself.

In modern terminology, we might describe this as a remarkable process of self-directed learning. More importantly, the creature's capabilities increase without Victor's involvement. Language enables reading. Reading enables abstract reasoning. Abstract reasoning produces a deeper understanding of human behavior. That understanding enables planning, persuasion and eventually manipulation. Each new capability becomes a platform for acquiring additional capabilities.

The creature is not recursively self-improving in the technical sense. He cannot rewrite his own cognitive architecture or construct a more capable version of himself. But Shelley describes something conceptually related: an artificial creation whose capacity to learn allows it to become substantially more capable than its creator anticipated.

That distinction becomes especially important when we consider recursive self-improvement, or RSI, in artificial intelligence.

From Learning to Recursive Self-Improvement

Imagine an AI system capable of making substantial contributions to AI research itself. It can write code, design experiments, analyze results, identify weaknesses in model architectures, and help design the next generation of systems. If the resulting system is more capable, it can perform those same tasks more effectively, contributing to another generation that is better still. The process becomes recursive because improved intelligence contributes to the development of further improved intelligence.

We are not at the point of fully autonomous recursive self-improvement today, and it is important not to imply otherwise. AI systems, however, are already being used to accelerate portions of software development and AI research. OpenAI has explicitly discussed preparing for recursive self-improvement while distinguishing that possibility from what current systems can autonomously accomplish. Researchers disagree about whether rapid RSI will actually occur, how quickly it could develop, and what technical or physical constraints might limit it.

Nevertheless, even the possibility changes the nature of the engineering problem. With most technologies, human beings remain outside the improvement loop. We design a better automobile, aircraft, computer, or drug. The automobile does not design the next automobile. A sufficiently capable AI might participate directly in designing the next AI.

That is a fundamentally different kind of technology, and it brings us directly back to Victor Frankenstein.

Capability Without Control

Victor's great scientific question is essentially whether he can create life. He becomes consumed by that problem, and he succeeds. What he does not adequately consider are the questions that follow. What will his creation become? What will it learn? How will he understand what it is doing? How will he constrain it? What happens if its objectives diverge from his own?

Victor solves the capability problem while largely ignoring the control problem.

Much of technological progress follows a similar pattern. We ask whether something can be built, how quickly it can be built, and whether someone else will build it first. Questions about control, incentives, unintended consequences, and responsibility often arrive later. With ordinary technologies, that sequence may be manageable. With increasingly autonomous intelligence, it may not be.

This is also where Frankenstein offers an unexpectedly sophisticated analogy for the AI alignment problem. Victor and his creature do not begin with completely incompatible goals. The creature wants companionship, acceptance, and meaning. Victor wants his creation to disappear. Their conflict develops as each acts in ways that frustrate the objectives of the other.

Eventually the creature proposes a bargain. Victor will create a companion for him, and in return the two creatures will withdraw from human society. Victor initially agrees but then destroys the unfinished second creature because he fears what might happen if he creates another autonomous being. From Victor's perspective, the concern is understandable. From the creature's perspective, Victor has broken their agreement and destroyed his only prospect for companionship. The result is escalation.

Neither participant needs to begin with evil intentions for the system as a whole to become catastrophic. That resembles an important aspect of AI alignment. Alignment is not simply about teaching a machine the difference between good and evil. It is about ensuring that the objectives and behavior of increasingly capable systems remain compatible with human intentions, including under circumstances their designers did not anticipate.

A highly capable AI would not need to hate humanity to become dangerous. It might simply pursue an objective that we specified imperfectly.

Why Control Has to Come Early

Victor also discovers something engineers understand very well: control mechanisms are easiest to design before deployment. Once the creature exists, Victor has remarkably few options. He cannot deactivate it, reliably constrain it, predict where it will go, or fully understand what it has learned. His creation has crossed a threshold from artifact to autonomous actor.

This may be one of Shelley's most important lessons for AI. We should not assume that control problems can be solved after systems become sufficiently powerful. Today we talk about alignment, interpretability, monitoring, red teaming, sandboxing, authorization boundaries, independent evaluation, and human oversight. These are sometimes treated as constraints placed around the more exciting work of making AI increasingly capable. I think that gets the engineering problem backward. They are part of the engineering. A system that is extraordinarily capable but cannot be reliably understood, constrained, or stopped is not a completed engineering achievement. It is an unfinished one.

There is an intriguing parallel here with today's debate about slowing frontier AI development because Victor eventually decides that there is something he should not create. He begins constructing a second creature and then stops. His reasoning is essentially about uncontrolled consequences. The second creature might refuse to honor the agreement. The two creatures might reproduce. Victor might be creating the beginning of something humanity would no longer be able to control.

In a sense, Victor eventually discovers the precautionary principle. But he discovers it too late. The first creature already exists. There is no governance structure, no containment mechanism, no independent evaluation, and no agreement about acceptable risk. There is not even another scientist involved in the decision. Victor must make an enormously consequential decision alone.

This suggests that the question facing AI researchers today is more subtle than simply asking whether we should slow down AI. A more useful question is: At what point should capability wait for control?

The Problem of the Race

There is, however, one enormous difference between Victor's laboratory and ours. Victor works alone. Modern AI development is taking place simultaneously across corporations, universities, open-source communities, and governments. Companies compete for markets, researchers compete for breakthroughs, and nations compete for strategic advantage. Once a technology appears achievable, restraint becomes difficult because every participant knows that someone else may continue.

This is one of the strongest arguments raised against unilateral slowing. If one laboratory slows down, what happens if another does not? If one country imposes strict safeguards, what happens if another sees that restraint as an opportunity? These are legitimate questions, and they make the modern problem considerably more complicated than Shelley's fictional one.

But competition does not make the underlying safety problem disappear. It transforms an engineering problem into a coordination problem. The challenge is therefore not merely to create safe AI. It is to create incentives, standards, and institutions under which safety does not become a competitive disadvantage.

That may prove harder than the technical problem itself.

Where the Frankenstein Analogy Ends

We should also be careful not to push the analogy too far. Shelley's creature is conscious, emotional, and embodied. He experiences loneliness, humiliation, anger, and a desire for companionship. We have no established basis for assuming that today's AI systems experience anything comparable.

Anthropomorphizing AI can itself become a source of confusion. An AI system does not need to hate us, resent us, fear us, or even be conscious to become dangerous. In fact, that may make the AI problem stranger than Frankenstein. Victor confronts another being with recognizable human motivations. We may eventually confront highly capable systems whose internal representations and objectives bear little resemblance to human psychology at all.

The danger would not necessarily come from creating a monster. It could come from creating an extraordinarily competent optimizer whose goals are not quite the goals we thought we gave it.

What Mary Shelley Got Right

I do not think Frankenstein tells us that we should stop developing artificial intelligence. That interpretation is too simple. Science itself is not the villain of Shelley's novel. The tragedy comes from separating scientific achievement from responsibility for its consequences.

Victor asks whether he can create something. He spends far less time asking what it will become, what it might learn to do that he did not explicitly teach it, what happens if its objectives diverge from his, how he will recognize that divergence, and what mechanisms will allow him to intervene. Today we can add another question Shelley could not have anticipated: What happens when the creation begins participating in its own improvement?

Those questions could have been written for an AI safety conference in 2026. Mary Shelley wrote them, in another form, in 1818.

Perhaps that is why Frankenstein remains so unsettling. The novel is not fundamentally a warning about monsters. It is a warning about creators. The tragedy begins not when Victor Frankenstein loses control of his creation, but much earlier, when he becomes so consumed with the possibility of creating it that he fails to think deeply enough about what responsibility for an autonomous creation actually means.

Today we are building AI systems that can reason, use tools, write software, conduct increasingly sophisticated research tasks, operate as agents, and contribute to work that will help produce their successors. We have not created fully autonomous recursive self-improvement, and that distinction matters. But for perhaps the first time in human history, we are developing a technology that may eventually participate meaningfully in its own technological evolution.

The current debate over whether and when to pace frontier AI should therefore not be reduced to a choice between technological progress and fear of technology. The more interesting question is whether our ability to understand, align, and control increasingly capable systems can keep pace with our ability to create them.

Mary Shelley could not have anticipated artificial intelligence. But she understood the human beings who would create it.